| Visually Induced Synesthesia |
 |
| Visually induced synesthetes may perceive particular colours when they see letters/numbers, or elicit specific locations when thinking about numbers or time adapted from: http://www.mondolithic.com |
Visually induced synesthesia is the most common form of synesthesia in which a certain type of visual stimuli is perceived with an additional sensory modality. Although there are many subtypes of visual synesthesia, the most studied classical form is grapheme-colour synesthesia (including number-colour synesthesia) characterized by the involuntary and autonomic association of letters and colours. Each synesthete experiences a distinct coupling that persists over lifetime (e.g. the letter “a” may elicit the colour blue in one individual whereas others may perceive it as red). [34] The novel difference in neural activation between grapheme-colour synesthetes and control is the hyper-connectivity between the left inferior parietal and primary visual sensory area in the parietal cortex. [33] Even to this day, the mechanism for synesthesia is still under investigation. It is also interesting to address the questions regarding the extent to which attention is required to evoke synesthetic perception and the effects on which the synesthetic sensations have in memory encoding and retrieval. In addition, the ongoing research in synesthesia that investigates the phenomenon of how the vivid synesthetic visual experiences alter neurocognitive processing and enhance visual memory can potentially assist students with synesthesia in learning by way of further utilizing the unusual paring abilities.
|
Table of Contents
|
1. Structural brain topology on visually induced synesthesia
1.1 Grapheme-colour synesthesia
| Figure 1. The five major forms of synesthesia. |
 |
| The size of the circle is proportional to the prevalence of each form of synesthesia. Image adapted from Novich et al, 2011. [16] |
Grapheme-colour synesthesia is a common form of synesthesia that is estimated to affect greater than 1% of the total population, and there are more female incidences reported than those of male. [8] Each grapheme-colour synesthete perceives unique and consistent pairing events of letters (or numbers) and colours. [33] It is interesting to note that among the five main groups of synesthesia shown in Figure 1, grapheme-colour form is the one that is most likely to co-exist with another type of synesthesia. Various technologies have been used to study the hypothetical models that delineate the differences in structural and functional connectivity between synesthetes and non-synesthetes.
The complete picture of synesthetic hyperconnectivity remains to be elucidated. Structural topological mapping is constructed and supported by heterogeneous imaging techniques. Some fMRI studies reported that the colour percept of graphemes is related to the activation of the V4 colour centre in occipital lobe. [15], [16], [23] Other research provided evidence for the functional hyperactivation and structural connectivity between fusiform gyrus (FG) and intraparietal cortex (IPC) in grapheme-colour synesthesia by using fMRI-BOLD (functional magnetic resonance imaging-blood oxygen level dependent) and voxel-based morphometry (VBM) respectively. [5], [13], [23]. Those studies hypothesized that IPC is highly correlated to sensory integration and cross-modal association [32], [5], [13] The hyperconnectivity between FG and IPC can also be validated by high anisotropic diffusion value, which indicates an increased grey and white matter volume in FG and IPC shown on diffusion tensor imaging (DTI). [5], [13] Most recent studies on the brain topology of grapheme-colour synesthetes confirmed the involvement of parietal cortex in synesthesia, while the role of V4 activation remains controversial. [12] Sample sizes and varied methodologies may contribute to the diverse findings. Thirion et al, 2007 suggests that in order to achieve high sensitivity and sufficient reliability, at least 20 subjects are required to generate fMRI results. [30]
| Figure 2. Main regions in the brain involved in grapheme-colour synesthesia. |
 |
| Green represents the dorsolateral prefrontal cortex; blue represents the inferior parietal lobe; red represents the primary visual area in the occipito-temporal lobe. The current model suggests that the neural basis of grapheme-colour synesthesia involves the hyper-connectivity between the left inferior parietal and primary visual sensory area. The activation of the dorsolateral prefrontal cortex may correspond to increased neurocognitive processing by synesthetic visual experiences. Image adapted from Rothen et al, 2012. [21] |
Sinke et al are the first to identify and analyse the regions in brain that are involved in the activation and connectivity differences in 18 grapheme-colour synesthetes and 18 non-synesthetes by using event-related fMRI. The subjects were also matched carefully in terms of age, gender, general intelligence, and vividness of visual imagination (VVI) particularly. This technological analysis (event-related fMRI) with high temporal resolution has the capacity in examining the interaction among regions in brain, and thus in identifying the areas associated with synesthetic percepts. These experiments were designed to maintain subject’s attention on the coloured and non-coloured letters and pseudoletters. [33] The identification of functional active areas characterized by grapheme-colour synesthesia was achieved by 2D echo planar imaging that measured BOLD signal. This fMRI study showed that Broadmann Area 7 (BA 7) in the left inferior parietal cortex is functionally connected with the right hemispheric primary visual region in grapheme-colour synesthetes. [33] The finding presented left IPC as the synesthetic modality binding site, whereas no superactivation in V4 was observed in synesthetes compared to control individuals. Sinke et al’s study suggested that complex neuronal network is involved in the induction of synesthetic perception; it is not restrained to a specific area, such as the brain colour center (V4) or the fusiform cortex. [33] Some previous research has shown that VVI is stronger in synesthetes than in non-synesthetes, and the differential VVI strength in synesthetes leads to V4 activation. [15], [24] Since people with synesthesia are more likely than non-synesthetes to experience higher level of VVI, Sinke et al keep VVI as a controlled variable. Another speculation is that V4 may not be the critical region for synesthetic perception. This model is supported by Hupe et al’s study in which they hypothesized that the areas of interlinkage between colour perception and achromatic graphemes are distributed across the visual cortex. Hanggi et al also evident that global hyperactivity is probably the driving force for synesthetic perceptions. [12] The widespread communication inside synesthete’s brain is not restricted within a specific region. [17]
 |
| Figure 3. The upper row illustrates the differences in activity of the left IPL (BA 7 region) between grapheme-colour synesthetes and control. The lower row shows the areas of the brain activated by letter stimuli: BA 18 in the middle occipital gyrus, BA 40 in the right IPL, and BA 7 in the parietal lobe (this is listed in the order of cluster significance). The X, Y, Z is the MNI coordinates and the colour bar on the side represents the F-value of the ANOVA analysis. Image adapted from Sink et al, 2012. [24] |
1.2 Time and/or Number synesthesia
| Figure 4. TNS perception |
 |
| This figure illustrates TNS's temporal/number-space perception in 3D. Image adapted from Kadoch et al, 2012. [7] |
Number-space synesthesia is the phenomenon that numbers are associated with specific spatial locations. This form of synesthesia is not exclusive to numerical symbols, but also highly associated with duration of time, magnitude of size and length. The subtype that relates to time periods is Time and/or Number Synesthesia. [4], [20] A theory of magnitude proposed by Walsh in 2003 describes that the parietal cortex is the venue where the interactions among time, number, magnitude, space and motion take place. [20], [21], [33] TNS is a unique form of synesthesia that rarely co-exists with other synesthesias. The current hypothesis explains that TNS is the result of insufficient neuronal specialization in the region of parietal cortex where time, number, and spatial information are processed. [20], [31] Also, fMRI studies revealed that control individuals developed less overlapping clusters of neurons coding for time, number, and space in IPS than TNS subjects as shown in Figure 5. [7], [21], [25], [29] According to the neuronal reuse theory, the brain reuses a part of pre-specified space processing structures that arise earlier in development for depiction of time and number. Therefore, TNS may be a consequence of improper neuronal recycling. [20]
 |
| Figure 5. The neuronal reuse theory. Blue, red, and green represent the neurons that encode time, space, and number in the parietal cortex. The overlap is greater in newborns than adults as a result of neural specialization. In TNS individuals, there may be less neural reuse or specialization, which leads to autonomic hyperbinding ability. Image adapted from Kadoch et al, 2012. [7] |
2. Influence of synesthesia on visual attention
 |
| Figure 6. Panel A: Example trials for coloured, grey scaled, and mixed stimuli. Panel B: Example trials for stimuli in different sizes (4, 8, 12). Image adapted from Nijboer et al, 2011. [15] |
In synesthesia research, the degree of visual attention required to elicit synesthetic perceptions has been an ongoing debate. The capacity of synesthetic percepts to direct synesthete’s attention in visual tasks is controversial. This contentious topic is approached by different studies discussed below.
2.1 Eye movement and grapheme-colour synesthesia
The influence of grapheme-colour synesthesia on visual attention can be examined by oculomotor target selection task. Synesthetic and non-synesthetic subjects performed visual search tasks with coloured, grey, or mixed stimuli in Nijboer et al’s study in 2011. Participants identified a target among a set of distracters via eye movement measured by infrared video-based eye tracker. [15] Nijboer et al found that the reaction time and accuracy are similar between both synesthetes and control; no “pop-out effect” was observed for synesthetes when viewing achromatic stimuli in an eye movement task. Synesthetic subjects were only able to react faster when the target was in a fixed position where attention can be fully processed. Some research in the last decade, however, showed synesthetes perceived a colour induced by the target that differed from the distracters; synesthetes’ eyes were directed toward the target significantly faster than control due to the “pop-out effect”. [9], [10], [27] The contrasting results are possibly due to different experimental paradigm in which Nijboer et al allowed a single eye movement for the oculomotor task, while others permitted both eyes to execute target search together. [15] Consequently, Sinke et al showed that synesthetes require attention and recognition for the synesthetic colour perception to be triggered.
2.2 Visuo-spatial attention and TNS
 |
| Figure 7. Comparison of the reaction times to targets between synesthetes and control following the presence of valid or invalid cues (arrows, words, and months). The asterisk denotes that the data is statistically significant. Image adapted from Teuscher et al, 2010. [29] |
The current hypothetical model for TNS postulates that time-space synesthetes may autonomically direct their visual attention to the associated spatial location induced by the temporal stimuli (for instance, thinking about June may guide a subject with TNS to attend to his/her right hand side). [2] Teuscher et al, 2010 studied the effect of TNS on visuo-spatial attention by investigating the result of target detection tasks; reaction time and event related potentials (ERPs) were recorded. Synesthetes and control responded to the directional stimuli (arrows, verbal cues, or months) by pressing a button when appropriate probe appeared on either the side a stimulus corresponds to (valid cue) or the opposite side (invalid cue) [29]. The result (figure 7) illuminated that the response time for synesthetes is less when following valid temporal cues, and substantially more when after invalid temporal cues when compared with control. [29] This suggests that time related stimuli are capable of directing visuo-spatial attention in individuals with TNS.
3. Visual synesthesia and memory enhancement
The unusual coupling of the additional sensation in the presence of an inducer may enhance synesthetes’ memory by richer encoding of information. Currently, this is a popular research field in neuroscience that works on the connection between synesthetic perceptions with modifications of brain cognition that further leads to enhanced memory consolidation. Many individual and group case studies have been investigated in order to examine the behavioural differences between synesthetes and non-synesthetes.
3.1 Case studies – synesthesia and memory
In this section, prominent case studies regarding synesthetes of exceptional memory will be discussed. The earliest popular case was recorded in 1968 that an editor inadvertently realized Solomon Shereshevskii, who worked for a newspaper company, never took notes on any work related details, such as news stories, names, numbers, or addresses. [28] Shereshevskii was capable of remembering a huge list of graphemes at once or even after a long period of time (years). Moreover, he was a multi-synesthete who had grapheme-colour, auditory-visual, and auditory-taste synesthesia. Shereshevskii described that he took the advantage of the concurrent senses to recall associative information. [28] His superior memory was probably achieved by the multiple forms of synesthesia and autism. [28]
 |
| Figure 8. Mental time-space calendar of TNS individuals. image adapted from Rothen et al, 2012. [21] |
Another case study presented a time-space synesthete, AJ, who was able to organize her time-related memories exceptionally. The mental synesthetic calendar (shown in Figure 8) inside her brain allowed AJ to retrieve historical times and autobiographical memories to her interest perfectly. [28]
In 2002, a student, C, with grapheme-colour and auditory-colour synesthesia of the University of Waterloo exhibited extraordinary memory at her psychology class when she participated in her professor’s demonstration. [28] By paring the colours with digits of the list shown in class, C was capable of remembering the list even two months after the lecture. When C was instructed to memorize an array of digits in incongruent colours within 3 minutes, she was not able to recall any of the digits once the screen shut down. This experiment demonstrated that C’s synesthetic percepts played an important role in enhancing her digital memory. [22]
The above typical case studies provide evidence for the relationship between synesthetic perceptions and memory enhancement. The changes in information encoding and processing as well as the underlying mechanism remain to be elucidated.
3.2 Memory-colour congruency, isolation and false memory effects
It is common for grapheme-colour synesthestes to exhibit memory advantages. The question regarding how synesthetic perceptions have influences on memory encoding and retrieval can be addressed from multiple aspects. Radvansky et al, 2011 examined the impacts of incongruent colour on synestheste’s memory, isolation, and false memory effects exhibited by synesthetes. [19] The von Restorff isolation effect describes the phenomenon that there is a bias in better remembering outstanding items; false memory refers to memory of information or events which have never been presented or occurred to the individual. In consistent with C’s case study, grapheme-colour synesthetes are better at recalling digits or words in congruent colours or in grey scale than control, but synesthetes are worse at committing graphemes in incongruent colours to memory than non-synesthetes. Furthermore, synesthetic experiences emphasize the letter or digit specific processing and surface characteristics of the graphemes which in term lead to boosted memory. The performances of tasks on deeper level and complex meaning-based processing, such as verbal comprehension, are not improved by synesthetic experiences. [19] Therefore, based on shallow memory processing, only the information related to synesthetic percepts is encoded with memory enhancement. In addition, the experiment proved the absence of both von Restorff isolation and Deese-Roediger-McDermott (DRM) false memory effect in synesthetes. [19] Since the isolation and DRM false memory effect only engaged with semantic meaning associated memory, the result further supports the theory that synesthesia influences memory by accentuating peripheral features of stmuli.
3.3 Benefits of synesthesia in learning
Since learning and memory are highly interconnected, the research on the extent to which synesthesia have on memory is critical in assisting students with synesthesia to build up self-confidence and to utilize their extraordinary perceptual ability in efficient learning. Many factors can potentially enhance memory and learning at different level of neurocognitive processing. Imagery, information transferring process, structuring and organizing incoming information to incorporate associate it into previously stored associative memory are variables that may contribute to learning enhancement in a non-exclusive fashion. [28] It is hypothesized that synesthetic perceptions may link to some of the above factors that relate to the superior memory in synesthetes. [28]
3.3.1 Synesthesia and memory advantage
Grapheme-colour synesthetes appear to perform better in learning and retaining novel information than non-synesthetes. [6] Similar to imagery, synesthetic concurrent reinforces the information of the inducing stimuli in consistent with the dual theory of memory. [28] Synesthetes are more capable of sufficiently processing and encoding shallow surface characteristics of the inducer (grapheme or temporal cues). Unlike non-synesthetes who learn better by semantic encoding of information, predominant encoding of apparent visual features of stimuli is more effective and efficient for synesthetes. [7] Moreover, structural alterations in synesthete's brain may also account for their memory performance. [28] Barnett et al’s EEG study suggested that the parvocellular pathway is more responsive than the magnocellular pathway in synesthete’s brain. [2] The parvocellular pathway corresponds to colour and visual recognition processing resides within the left posterior fusiform gyrus (PFG), while the magnocellular pathway relates to motion and scene recognition processing localizes in the left anterior fusiform gyrus (AFG). [2] This model is supported by Banissy et al’s finding of increased grey matter volume in PFG and decreased cortical volume in AFG. [1] Although the mechanism underlying the structural modification inside synesthete’s brain is still unclear, the differential responsiveness of the two visual processing pathways may partially explain the influence of visually induced synesthesia on memory enhancement.
 |
| Figure 9. Red indicates the increased grey matter in the left posterior fusiform gyrus. Green indicates the decreased grey matter volume in the left anterior fusiform gyrus. Blue indicates the decreased grey matter volume in the left V5 region. Image adapted from Banissy et al, 2012. [1] |
4. See Also
Auditory Visual Synesthesia
Lexical-Gustatory Synesthesia
Why Synesthesia Occurs
Synesthesia Technology
5. Synesthesia Links
American Synesthesia Association
Belgin Synesthesia Association
Synesthesia Research and tests
Hey, for your section 3.2 you might want to link to my neurowiki on False Memories :)
Hey Emily, yes I saw your page few days ago and I was planning on linking it to your page!!
CC - this is really cool! Nice interface. Emily! Thanks for the suggestion to CC! Love the amazing work and collegiality. One small suggestion - the first figure - why not put an actual caption for that nice colour figure?
Yes Dr. Ju, I am going to put captions on for all figures :) thanks for the suggestion
This page is so easy on the eyes.. haha!
As far as your writing goes, I have no issues except maybe one thing:
"Until today," in your first paragraph - I think you can omit this completely. It sounds repetitive with "still under investigation." Alternatively, you can say "Even to this day, …. is under investigation."
Excellent organization!
ohh okay, thank u rufina!!
The organization on this page is impeccable! You have great figures, very informative.
I have two suggestions. Would it be possible to make some of your figures larger? It's a little difficult to see some of them. Also, I think adding any remaining captions would make your page perfect!
Hi, thank you Ishita~
ya, i'll try to make some of them bigger and finish up the rest of the work :))
Carrie - that's alot of work you put - I was just checking the log on this - amazing!
Thank you Dr. Ju, I'm still editing some parts though :)
This page is very informative and visually appealing! I would recommend changing the size of your figure 5 and maybe add a case study video.
Hello! Thank you for posting and linking to my page (Cognitive Benefits of Video Games). The attention topics are definitely visually based, though the way I covered it was attention in general. Vision and attention are probably the two main improvements video gamers have. I will be adding a link to your page.
I think you have some of the most visually stimulating page, with pictures that are relevant and varied. Nothing to suggest other than italicizing and bolding the references for the Nature formatting. Good job~
I really enjoyed reading this and learning more about this fascinating phenomena. I just suggest putting Fig. 7 after the text, but other than that it looks great!