Seeing Music, Hearing Colour: Accessing Synaesthesia

Have you ever heard a piece of music and imagined its rhythms and melody as a pattern of swirling colours, each note igniting a different hue in your mind? Or have you smelled the sweet, floral aroma of blooming jasmine on a warm summer evening and felt transported to a distant garden, a place you’ve never visited but seems etched in memory, as though the fragrance itself wove threads of a forgotten dream? Have you ever seen a painting so dynamic that it seemed to hum with musical melodies that felt as though they emerged directly from the brushstrokes? Or perhaps you’ve tasted a dish, and its complex flavours brought an unexpected sense of warmth or chill, as though your taste buds could feel the temperature of emotions?

These experiences are not just intended poetic descriptions; they allude to a phenomenon called synaesthesia, a term derived from the Greek words “syn” (together) and “aisthesis” (sensation). Synaesthesia describes a condition where one sensory experience evokes another, blurring the boundaries between perception and imagination. The earliest recorded accounts of synaesthesia date back to the 19th century, when researchers and philosophers began documenting cases of individuals who could “see” sounds or “taste” words. Since then, it has fascinated scientists and artists alike, inspiring questions about the unique ways in which the human brain connects and integrates sensory information. This fascinating interplay of the senses leads us to question how and why the human brain creates such vivid cross-connections, and how this phenomenon could be harnessed to enhance and supercharge creativity. To answer these questions, we must journey into the intricate neurological pathways that give rise to synaesthesia.

A Neurological Perspective

Synaesthesia is a condition where the brain’s sensory pathways intersect in unusual and fascinating ways. Neurologically, it’s believed that synaesthesia occurs when cross-activation between different brain regions causes one sense to trigger another. For example, when the auditory cortex, responsible for processing sound, activates the visual cortex, a synaesthete may “see” colours in response to music.

This phenomenon is thought to result from increased connectivity between sensory regions of the brain. During normal brain development, many neural connections are pruned—a process that eliminates unnecessary pathways to create efficiency. However, in individuals with synaesthesia, some of these pathways remain intact, leading to unique sensory crossovers. This neural architecture is supported by studies showing increased white matter density in synaesthetes, particularly in regions associated with sensory perception and integration (Michael J Banissy, Vincent Walsh, Jamie Ward).

Recent research also highlights the role of genetics and early sensory experiences in synaesthesia. Studies, such as Asher et al., 2009, suggest a heritable component, with the condition often running in families. These findings are further supported by evidence from Barnett et al., 2008, which indicates that certain genetic markers may contribute to the development of synaesthetic traits. Early exposure to certain sensory pairings, such as associating specific colours with numbers in childhood, may also reinforce these neural links, solidifying them into adulthood.

From a functional perspective, synaesthesia involves the hyperconnectivity hypothesis: neurons in sensory areas have atypical communication, either through direct connections or via intermediary hubs like the thalamus, a central relay station in the brain. This could explain why synaesthetic experiences often feel automatic and consistent. One notable example comes from a popular children’s alphabet game sold in the early 1970s, which featured red-coloured “A” tiles, potentially imprinting this connection in the minds of young synaesthetes. This seemingly innocuous toy highlights the ways in which early sensory experiences can leave lasting impressions, subtly bridging the neurological underpinnings of synaesthesia with the consistent sensory pairings observed in individuals. These early influences might explain why some synaesthetic experiences appear so vivid and automatic, seamlessly connecting past sensory encounters to present-day perceptions.

Living with Interwoven Senses

Throughout history, synaesthesia has shaped the lives and works of remarkable individuals, weaving itself into their creative processes and achievements in profound ways. Take, for instance, the Russian abstract painter Wassily Kandinsky, who lived between 1866 and 1944. Kandinsky’s world was one where sounds conjured vibrant colours and shapes. His condition allowed him to “hear” colours and “see” music. His groundbreaking works, such as “Composition VIII“, were vivid translations of these sensory crossovers. For Kandinsky, synaesthesia was more than an experience; it was a tool he used to express universal emotions through abstract forms, creating a visual language that transcended traditional artistic boundaries.

Similarly, Pharrell Williams, the American Grammy-winning musician and producer, has often spoken about how synaesthesia shapes his creative process. For Williams, musical notes and chords evoke specific colours, forming a dynamic interplay between sound and sight that guides his compositions. In interviews, he has described how this sensory fusion enables him to craft music that feels alive, blending auditory and visual elements into seamless harmony.

Then there is Daniel Tammet, a British savant born in 1979, whose extraordinary abilities in mathematics and language are deeply tied to his rare form of synaesthesia. Tammet perceives numbers as complex shapes, colours, and textures. The number “5,” for instance, feels smooth and yellow to him, while “9” appears tall and dark blue. This vivid sensory relationship has allowed him to achieve incredible feats, such as reciting over 22,000 digits of pi from memory. For Tammet, synaesthesia is not just an enhancement of perception but also a window into understanding the structure of the world.

These narratives not only provide a glimpse into the lives of synaesthetic individuals but also illuminate the extraordinary ways this condition can enrich creativity and perception. By exploring their stories, we uncover how synaesthesia intertwines with art, music, and mathematics, offering new perspectives on how sensory experiences shape human expression.

Real-World Applications

Synaesthesia has long been a source of inspiration for innovation, reshaping the way we integrate sensory experiences across disciplines. In 1940, Walt Disney created a groundbreaking masterpiece with Fantasia. The film paired vivid animated sequences with orchestral compositions, an unprecedented attempt to visually interpret music. Each scene transformed sound into a dynamic, visual experience, evoking the kind of multisensory perception typical of synaesthesia.

Between 1999 and 2003, inspired partially by Fantasia and my own experiences of musical synaesthesia, I developed JamTunz, a media-synchronising software engine designed to create interactive, syneasthesia-inspired music videos. Driven by a fascination with how senses interplay, JamTunz allowed users to experience music as more than sound, mapping its structures to dynamic animations and tactile outputs. Ultimately, this intellectual property was acquired by Adobe Corporation, marking a milestone in the project’s journey and my own creative career. While JamTunz’s potential was never fully realised, its development remains a personal testament to the enduring allure of synaesthesia and its ability to inspire creativity and new forms of sensory connection.

The influence of synaesthesia extends beyond the arts into marketing and emerging technologies. Companies like Pantone and Spotify have embraced chromesthesia to create “colour playlists” that connect auditory and visual experiences. Similarly, virtual reality platforms like Tilt Brush allow artists to “paint” in immersive 3D spaces, where sound and sight merge into a unified canvas. These innovations expand the boundaries of creativity, offering entirely new ways to experience the world.

Synaesthesia and the Creative Generalist

For creative generalists, synaesthesia is more than a neurological curiosity—it is a lens through which interdisciplinary connections can flourish. This unique phenomenon invites us to merge skills across diverse fields, such as visual art, music, language, and technology, to craft experiences that transcend traditional boundaries. Through the facilitation and enhancement of synaesthetic experiences, creators can build bridges between sensory modalities, inspiring new ways to perceive and interact with the world.

Consider a digital designer who listens to a symphony and imagines its emotional journey as a series of vibrant colour palettes. The designer might then translate the crescendos and silences into aesthetically parallelled bold reds or calming blues, developing a user experience that evokes not only sight but the essence of sound. By intertwining auditory and visual cues, such a designer could create a sensory experience that feels immersive and unforgettable.

Imagine, too, a musician who composes not just for the ear but also for the eye. By designing visual projections that pulse and shift in harmony with live performances, this artist creates an experience far beyond a typical light show. The visuals are tightly linked to the artistic intention, with each movement and form acutely synchronised to the music’s structure and emotional tone. Each chord might ripple across a screen as waves of colour or manifest as abstract forms, turning a concert into an immersive multimedia journey where sight and sound are inseparably intertwined, amplifying the depth and impact of the performance.

For a multidisciplinary artist, synaesthesia could become a playground for invention. Picture an installation where visitors walk through a space and “feel” sound as gentle vibrations underfoot or “see” flavours as projected bursts of light. By challenging the ways we traditionally engage with art, such experiences invite participants to rethink how their senses interact with the environment, transforming the act of perception into something wondrous and new.

Synaesthesia ultimately teaches us that perception is not confined to a single sense. By embracing this interplay, creative generalists can unlock untapped realms of imagination and invention. As digital tools continue to evolve, offering greater integration of sensory inputs, the potential for crafting multisensory experiences grows exponentially. From augmented reality to AI-driven media, these tools empower creators to inspire awe and connection in ways that were once impossible, enriching both the arts and the human experience.