Synesthesia - there's an app for that

What if sight came prepackaged with sound, or we experienced numbers as colors as well as two-dimensional symbols?

Speaking at the IdeaFestival in 2010, prodigious savant Daniel Tammet described how this unusual ability, synesthesia, had shaped his life, from the day of the week - "Wednesday's are always blue, as you should know" - to how numbers appeared to him in rolling landscapes, the geography of which he was able to exploit to recite Pi out to 22,000 decimal places. Since synesthesia has a physical basis, his unique creative ability was not so unusual after all, he argued. Each of us might benefit.

I'm not sure he thought that benefit would come so soon.

Conceptually, synesthesia can be related to a couple of trends. One is the sheer volume of data being collected by Big Science and creatively mapped in innovative ways by the Allosphere, for example. Other projects like Zooniverse make participatory science possible. Given a pair of eyes capable of recognizing patterns, something that we humans are particularly good at, or a few spare clock cycles in your personal computer, that extra data can be meaningfully experienced and parsed. Synesthetes experience natural data that our mono-sensory selves don't.

Secondly, trans-media efforts have also become more and more common. As someone who loves the imaginative and functional art of studio furniture, and who has developed some skill in furniture making, projects like a 'Net-connected end table that prints memories open whole new fields for the talented craftsperson. Its creator, John Kestner, has gone on to partner in an interesting and consumer-oriented project, the highly modular and intelligent Twine. These "supermechanical" pieces give, as he suggests, "a soul" to electronics and bring the Web of things that much closer.

Ariel Waldman recently explained to me that a "synesthesia mask" had been created by hackers who had 24 hours to create something unique at the San Francisco Science Hack Day. It's a little creepy, but in its mind-expanding originality, that project should go nicely with the particle wind chime developed last year. Call it super-biological.

So given a surfeit of data and a willingness to experiment across media, perhaps its no surprise that the world described by Daniel Tammet might be closer than we think. It is. "Inside the Mind of a Synaesthete," a post at Neuro Tribes, takes all of this a step further, not only describing this unique condition in some detail, but introducing the synesthete, musician and media artist Perry Hall, who has developed an iPhone app that can be downloaded and used to sonify and record what the phone is seeing. A video recorded with the software is posted above. I've downloaded and been playing around with it, and as Steve Silberman explains in the blog post, the effect really doesn't feel forced. In fact, it feels like another layer of reality is opened up to senses of sight and sound tuned by history to be wary, and not necessarily expansive. If you have an iPhone, give "Sonified" a try.

Wayne