Colourful language: U of T psychologists discover enhanced language learning in synesthetes

“When I see equations, I see the letters in colors. I don’t know why,” wrote Nobel Prize-winning physicist Richard Feynman. “I see vague pictures of Bessel functions with light-tan j’s, slightly violet-bluish n’s, and dark brown x’s flying around.”

Feynman was describing his grapheme-colour (GC) synesthesia – a condition in which individuals sense colours associated with letters and numbers.

Synesthesia is a family of conditions where individuals perceive stimulation through more than one sense. GC synesthesia is just one form. In others, musical notes evoke colours; words have associated tastes; sequences of numbers are sensed as points in space; numbers suggest people, like an elderly man or a baby girl.

Many synesthetes have expressed how their condition has enhanced their lives. When designing a set for a ballet or opera, the American artist David Hockney uses the colours he “sees” in the musical score. Grammy award-winner Pharrell Williams has said that without his music-to-colour synesthesia, “I’d be lost.”

Now, an experiment led by University of Toronto psychologists has shown for the first time that grapheme-colour synesthesia provides a clear advantage in statistical learning – an ability to discern patterns – which is a critical aspect of learning language. The result provides insight into how we learn, and how children and adults may learn differently.

According to psychology graduate student Tess Forest, “Our result shows that when people experience the same patterns with more than one sense – for example, aurally and visually – they are better able to learn the patterns.”

Forest is lead author of the paper describing the research, published online recently in the journal Cognition. She is a member of Assistant Professor Amy Finn’s Learning and Neural Development Lab in the department of psychology in the Faculty of Arts & Science. Finn is senior author on the paper and co-authors include researchers from the department of psychology at the University of California, Berkeley.

The team obtained its result with an experiment in which subjects listened to an artificial “language” comprising nonsense words – for example, “muh-keh” and  “beh-od.”

They were then asked to listen to a second set of words that included the artificial words, plus new artificial words they had not previously heard. The new artificial words contained combinations of syllables not included in the original artificial language; in other words, the new artificial words represented a “foreign” language to the participants. Participants were then asked to guess whether a word was part of the original artificial language or not.

The result? GC synesthetes scored higher at distinguishing between the two “languages” than other participants in the experiments.

“You can think about it this way: The GC synesthetes have twice the number of senses providing the same information,” says Forest. Not only are they sensing the patterns of syllables by listening, they are also sensing those patterns using synesthesia-evoked colours.

“This result is important for thinking about how we learn,” says Forest, “because real-world learning outside the lab uses multiple senses. Our result sheds light on the learning mechanisms in the brain that might be supporting statistical learning.”

One focus of Finn’s overall research is understanding how children and adults learn differently.

“This result has clear implications for understanding how children and adults might learn differently,” says Finn. “This study shows that you can learn more when you have redundant information, that is, information perceived through more than one sense.

“This could be truer in infants if they are synesthetes – though I don’t think the evidence is very clear on that question yet. But, it’s something that could be more true in kids because their developing attentional systems make them less able to focus on only one thing at a time. When this information is redundant, this broader focus, could be quite useful for learning.”