When Numbers have Colours

Filed Under Science

I just finished my latest Coursera on-line course on Visual perception and the Brain. As part of the course assignments, I had to write a final essay on a topic related to visual perception. I have long been fascinated by synaesthesia, so I researched it and wrote a little about it. What follows is an adapted version, suitable for a blog.

Synesthesia2Daniel Tammet has synaesthesia. To Daniel, numbers on a page have a unique three-dimensional shape and unique colour. That is, the digit 1 is a hazy glow of white light. The number 4 is a blue boomerang. The 5 is a yellow cross-hatched square. The number 6 is a tiny black hole. Daniel “sees” these shapes and colours both when looking at the number on a printed page as well as when he is simply imagining the number. Number-colour synaesthetes, like Daniel, have a unique multi-modal experience of the world. A stimulus to their visual sense triggers an associated experience in another of their senses (eg. colour, auditory, haptic or olfactory). In Daniel’s brain, as in the brains of approximately 0.5-2% of the world’s population, there is an unusual amount of cross-talk between sensory systems that are usually somewhat isolated from one another. This gives rise to one sensory modality triggering another (eg, a graphical representation of a number causing a sensation of colour). There are several different sub-types of synaesthesia where one modality of perception triggers another modality. The most common is the number–colour synaesthesia (like Daniel) in which numbers have shape and colour. The literature also documents word-colour, sound-colour, texture-colour, sound-taste and several other forms of inter-modality triggering. In a recent TED talk, Daniel used painting to create a pictorial representation of the colours, emotions, textures he experiences when the thinks of the first 20 digits of PI. Very cool.

Why Number-Colour? Why Not Number-Smell?

Could any human sense cross-talk with any other? Could a number have a particular smell, for instance? It seems not. Adjacency in the brain is among the stronger theories of why synaesthetes have certain sensory modalities paired with other modalities. Cross-connections are more likely in adjacent areas than they would be in part of the brain that were distant from one-another. In a 2001 paper by the neuro-psychologists Ramachandran and Hubbard, the authors propose that the number-colour type of synaesthesia may result from hyperconnectivity between nerve systems in the fusiform gyrus and/or in the angular gyrus areas of the brain. These locations in the brain are known as integration points for different visual-perceptual and emotional systems. Dr. Jamie Ward of University of Sussex points out that our neural systems for taste are close to those of systems which support spoken language in Broca’s area of the Brain. Thus, there are synaesthetes who pair the sound of words with a particular taste.

Hereditary Basis of Synaesthesia

There is some evidence that synaesthesia is hereditary (passed along the X chromosome). Thus, one theory postulates that synaesthesia could arise due to a genetic mutation that inhibits the pruning of neural connections between perceptual areas of the brain during fetal development. Reinforcing this observation, Ramachandran and Hubbard refer to experiments where researchers found much larger neural feedback from inferior temporal areas to the V4 visual area of the brain in prenatal monkeys than they did in adult monkeys. They speculate that should connection pruning during fetal development fail to eliminate these links, then connections between the number-grapheme area and the V4 area would persist into adulthood supporting the ability to the experience of colour when viewing numbers or letters. While genetic factors appear to be part of the picture, genetics by itself cannot account for the whole phenomenon. The neural cross-connections merely permit a number to evoke a colour. Learning is an essential reinforcing element because people are not born with an understanding of number and letter symbols. Ramachandran and Hubbard point out the fact that different synaesthetes have different colours evoked by the same numbers. For example, to one synaesthete 3 is red while to another it is blue.

The “Inverse Problem” in Visual Cognition

It is a bit hard to believe, but our eyes do not perceive the world as it is.The Inverse Problem The image that impinges on the back of our retinas is in fact an amalgam, a blend, of a number of distortions that occur as light from (say) the sun bounces off an object and the photons are picked up by your eye. It works like this … light from the sun traverses the turbulent atmosphere, is partially absorbed by the object in front of us and partly reflected by it, is reflected by near-by objects that blend with the main reflection, traverses yet more atmosphere, gets inverted and bent by your eye’s cornea and lens and then finally ends up on the rods and cones that make up your retina. These are only sensitive to certain wavelengths of light and their response changes with time. By the time the image is being picked up by your retinal cells and sent to your brain, it is a mere approximation of what you are looking at.

Figuring out what is ‘really’ out there in the physical world is called The inverse problem in visual cognition. The crux of the inverse problem stems from the fact that the same image on the retina can be generated by different physical objects with different shape, colour, motion or orientation. The classic example illustrating this is that if the sun was a little brighter and the paper a little darker, our retina would register the same image as when the sun was duller and the paper brighter. Just using our visual system, we can’t know which of these two scenarios are in fact ‘reality’. Grey_square_optical_illusion-300x232The reverse is also true. We may see the same patch of colour as lighter or darker depending on the surrounding area in the image. For example, the patches A and B in the image to the right are in fact the same shade of gray — we just see them differently due to the context.

This fundamental limitation causes ambiguity in that we cannot be sure what physical source is causing the visual stimuli we see. Is that a tiger hiding in the bushes or is it just the play of shadow and light on the leaves? Our retinal images are the same in both cases. Answering such questions quickly and accurately can be the difference between living to procreate and being somebody’s lunch. Dr. Dale Purves, in the Coursera course I’m taking called Visual Perception and the Brain, makes the point that images are only representations of the physical world. They only exist for us in our perceptual and cognitive systems. Our visual perception of the physical world is distorted due to variances in luminance, reflectance, transmittance and the physiological limitations of our perception systems. As such, as we try to make sense of what our eyes see, our brain must make a number of assumptions and extrapolations based on heuristics and memory. As synaesthetes have a special kind of integrated milti-sensory perception, could they have an evolutionary advantage over non-synaesthetes in their ability to make sense of the physical world? For example, Dr. Purves has shown that the same patch of colour in the physical world is experience differently by our perceptual system depending on the context that the patch appears in.

Could There be an Evolutionary Advantage to Synaesthesia?

Could someone with synaesthesia, using senses triggered by that patch of colour, have memories and experiences which could afford him/her an evolutionary advantage? Dr. Jamie Ward, in a recent video interview outlines a number of theories as to why their might be an advantage to being a synaesthete. For example, he relates a study where synaesthetes have been shown to have an ability to remember certain shapes or colours better than non-synaesthetes. This ability to use other sensory systems to help remember more visual information and remember it more accurately could be an adaptation that could be selected for by natural selection. Having more and better data in which to associate images with phenomenon in the physical world could help address the inverse problem in visual perception and thus could result in better survival and reproduction. That being said, Dr. Ward asserts that more work is needed to better understand the phenomenon because there are different forms of synaesthesia with different genetic and phenotypic characteristics.

The phenomenon of synaesthesia has interested me for years. I can’t help but feel a bit impoverished knowing that there are people who have a more integrated and rich experience of the world than I do.

P.S. Did you know that Leonard Bernstein, Richard Feynman and Vincent Van Gogh were all synaesthetes? So were a lot of creative forces in history (here is a list of more famous people with synaesthesia).

Being Unanonymous

Filed Under Science

On the Internet, nobody knows you are a dog

Nobody knows you are a dogThere are probably many folks out there that are familiar with the classic cartoon on the left. It was drawn in 1993, at a time when there was a universal expectation that your activity on the Internet was anonymous and untraceable. Even a dog could mascaraed as a person.

Today, the opposite is true. Everything you do on the Internet and with your cell phone is tracked, correlated, analysed, identified and recorded. You and your activity become commodities that are bought and sold between firms or used by governments to ‘protect you’. You cannot so much as walk around the city with your smart-phone’s WiFi or Cell service turned on without your activity being tracked and used for commercial (at best) or nefarious (at worst) uses.

In this brave new world, commerce has changed. All that wonderful stuff out there in the cloud that we access all day long cost money to make and it has to be paid for. We consumers have tacitly agreed to pay for it, but with a different kind of currency – our identities. One of my favourite Internet security gurus, Bruce Schneirer , says it succinctly:

Nothing is free, so if you won’t pay cash, you’ll have to pay with personal information.

Okay, so we can no longer trust that our Internet activity is anonymous. But I have also found that it is true that no one trusts us even when we choose to declare our identity.

I had a personal taste of this one night 2-weeks ago. Working on my home computer in my basement, I hit the enter key to submit my answers to a final exam in Introductory Human Physiology, an on-line Coursea course. Rather than the results being submitted, something curious happened. I was taken to a web page that asked me to type a particular sentence into a box on the screen. As I typed, a program ran in the background analysing my typing style.

Identity Validation

Identity Validation: Convincing someone that I am me

When that was done, a message appeared that said that my typing passed the test. Then, I was then taken to a different page that asked me to sit-up straight and look right into my webcam.  After I clicked an OK button, a picture was taken of me. A few seconds of analysis followed. Again, a message appeared saying that my image passed. Finally, I was taken to a page that said that my exam results have been linked to my identity and submitted for grading.

In truth, this wasn’t a total surprise to me. You see, when I enrolled in this Coursera course, I chose their ‘Signature Track’ – an optional service where my participation in the course is linked to my identity. When I opted into my course’s Signature Track, Coursera put me through a process where my typing style and facial features were captured by their identity verification system. I also had to snap a picture of photo-ID. Upon submitting each exam (there were three in the 12-week course), my typing style and facial image were captured again and compared to the reference prior to them accepting my answers.

Signature Track Certificate

The Signature Track is the systems where Coursera takes special pains to ensure the person who sits for the exams is the person who signed up for the course in the first place.  Coursera markets the Signature Track as a premium feature with these extra security features.  I paid the $50 fee largely to see for myself how they do this identity verification. It was certainly interesting, but I found a number of ways to circumvent it. For example,  the certificate itself is  self-printed and easily spoofable if you have some proficiency with Photoshop.

Anyway, earlier this week I got an email confirming that I passed the course. A link took me to a special customized web page featuring a JPG facsimile of a classic certificate with the words Verified Certificate  above an official looking seal. My fancy certificate is on the left; click it to go to the special Coursera page where you will see the official confirmation of my taking the course.


The Course Itself: Introductory Human Physiology

As the for the course itself, it was harder and more intense than any I have taken thus far. Drs. Carbrey and Jakoi of Duke University really know their stuff and packaged up a fascinating romp through the body’s major systems. They did their best to simplify things, but there was an assumption that you were comfortable with high-school chemistry and biology and could dive into fine details about ATP metabolism, PH regulation and organic chemical reactions and without much explanation. We learned about organ systems, homoeostasis, endocrine pathways, cellular transport, normal organ operations and pathological states of various systems. You had to keep up with the lectures and readings – everything had a deadline.

Lecture Notes Screen Shot  Questions

The Course materials were the usual on-line learning variety including lecture notes (PDFs), videos with in-video quizzes, power point slides, discussion forums, post-lecture problem sets, FAQs and practice exams for review. It was a respectable (if by now somewhat ordinary) set of media that I have come to expect with Coursera courses.

Discussion Forum Course Description FAQ

Exam Takers = 5% of Participants

I was among 27,870 active participants from all over the world.  Of these nearly 28,000 learners, only about 1,300 (5%) wrote the final exam. Of these less than half passed. In fact only 560 learners (2% of the total population of participants) both wrote the final exam and also got a grade above the passing level of 75/100.

Class Map

A 2% passing rate would be a terrible result in a traditional class-room. But in a MOOC, that might be normal; who knows?

In all, it was a great course and a good experience, but far more hours and intensity than I anticipated. I’ll be more careful next time to carefully read the course descriptions of my next Coursera course.

Broken Dishes

Filed Under Science

Book Thinking Fast and SlowSome books are just so good, that I read them again and again. One such book is Daniel Kahneman’s Thinking, Fast and Slow. I have both hard-copy and e-book versions which I find myself going back to a few times a month.

Kahneman is a remarkable fellow — a psychologist by training, he won the Nobel prize in Economics in 2002 for integrating insights from psychological research into economic science, primarily human judgment and decision-making under uncertainty. His book is a treasure-trove of empirical examples of how we think; nicely organized in an easily understandable theoretical framework. I highly recommend it for those who thirst to understand why we are the way we are.

Last week, while working on a complex grant application here at Baycrest , I had an opportunity to apply one of the insights from Dr. Kahneman’s book. We were working on a section of the grant application where we were to outline the economic benefits of creating a centre dedicated to the commercialization of innovations in aging and brain health. We had come up with a list of 2 strong benefits and 6 additional benefits — a total of 8 points. The 6 additional benefits were good and added somewhat to the overall business case but they lacked the clarity and resonance of the first two benefits. The small team gathered around my office meeting table were working on the text that supported our list of 8 benefits when I recalled an experiment described in Khaneman’s book and decided to change our approach. Here is the experiment that came to mind (the text in green below is copied directly from the book):

Christorpher Hsee, of the University of Chicago, asked people to price sets of dinnerware offered in a clearance sale in a local store, where dinnerware regularly runs between $30 and $60. There were three groups in his experiment. The display below was shown to one group; Hsee labels that joint evaluation, because it allows a comparison of the two sets. The other two groups were shown only one of the two sets; this is single evaluation. Joint evaluation is a within-subject experiment, and single evaluation is between-subjects.



Set A: 40 Pieces

Set B: 24 Pieces

Dinner Plates 8, all in good condition 8, all in good condition
Soup/Salad bowls 8, all in good condition 8, all in good condition
Dessert plates 8, all in good condition 8, all in good condition
Cups 8, 2 of them broken  
Saucers 8, 7 of them broken  


Assuming that the dishes in the two sets are of equal quality, which is worth more? This question is easy. You can see that Set A contain all the dishes of Set B, and seven additional intact dishes, and it must be valued more. Indeed, the participants in Hsee’s joint evaluation experiment were willing to pay a little more for Set A than for Set B: $32 versus $30.

The results reversed in single evaluation, where Set B was priced much higher than Set A: $33 versus $23. We know why this happened. Sets (including dinnerware sets!) are represented by norms and prototypes. You can sense immediately that the average value of the dishes is much lower for Set A than for Set B, because no one wants to pay for broken dishes. If the average dominates the evaluation, it is not surprising that Set B is valued more. Hsee called the resulting pattern less is more. By removing 16 items from Set A (7 of them intact), its value is improved.

This is a fascinating and illuminating piece of work that provides an important lesson about parsimony.

Reflecting on this experiment, I reasoned that the grant reviewers would be engaging in single evaluation of the applications they receive. This is largely because most of the 40 or so grant applications they will receive would come from different sectors of the economy (healthcare, energy, finance, manufacturing, etc.).  It would be difficult for them to make a direct comparison between the economic benefits listed in one application to the benefits listed in the others. Benefits to the economy are not as easy to compare as are dishes. Moreover, the evaluation team will likely not each review the 40 or so applications. It is more likely that a small number of applications will be distributed to each reviewers. Thus, the reviewers will be making a somewhat independent judgement of the economic benefit list as it appears in each individual application.

My epiphany was to eliminate the 6 additional benefits in our list and instead to focus entirely on the first two. In this way, the ‘average’ value of the economic benefit argument we were making would not be diluted by lesser (albeit positive) points.  The team around my office table agreed to this approach and we spent our time fleshing out the two strong points. The result was a highly focused discussion of just the two strong points we had come up with.

The grant application went in on Friday and I think we stand a good chance to be selected for the short-list. Should we get selected, it will be amusing that we used research on how the brain works to win a grant competition for a center focusing on research about the brain.


The Astronaut

Filed Under Science

I’m cleaning out the basement and throwing out decades of accumulated stuff. On the one hand, it’s very cathartic to make space an get rid of dust-collecting do-dads of dubious value. But on the other hand, the act of selecting which artifacts to keep and which to throw out brings on a kind of melancholy. Original copies of my Master’s Thesis, letters from my old girlfriend, a vintage slide-rule from my collage days, magazine articles mentioning my name … the flotsam and jetsam of half a life lived. What’s it good for? Why keep it?

Anyway, in a dusty box I came across a little gem. There, shoved into an envelop with kitchy birthday greeting cards was nearly 30 year-old, yellowed newspaper clipping. It was an ad from the careers section of the paper — Canada needed astronauts. Along with the ad, were two letters paper-clipped to the back.

Newspaper Ad

Ad Looking for Canadians with the Right Stuff

It’s the summer of 1983. A young, enthusiastic, but hopelessly under-qualified Ron Riesenbach sees an ad in the newspaper seeking candidates for Canada’s inaugural astronaut program. While only an undergraduate still in University with no military experience, flying skills or biological payload expertise, he decides to go for it. After weeks of filling out forms (“Please list the number of hours you have logged flying the following aircraft ….”, Uhhh, zero.) and getting required medical tests, an application is submitted with a carefully crafted cover letter which celebrates the candidates (pathetically meager) qualifications.

Letter of Application to Astronaut Program

Letter of Application to Astronaut Program

An embarrassingly short time later, a letter arrives from the National Research Council. In gentle (but firm) bureaucrateese, the Chief of Personnel celebrates the great response the ad has engendered, but regrets to inform Mr. Riesenbach that his application will not be moved forward for further consideration.

We Regret To Inform You

We Regret To Inform You ...

Sigh … a dream squashed. But the innocent enthusiasm and bravery of the attempt makes me proud.

This document I’ll keep.

Baroque Cycle – 2nd Rotation

Filed Under Science, Tech

I just finished reading the remarkable “Baroque Cycle” by Neil Stephenson.

The Baroque Cycle by Neil Stephenson

It’s a 3-volume, 8 book, 3,000 page historical fiction covering a pantheon of characters, in the late 17th and early 18th century. It’s about international trade, wars, espionage, heists, slave-trade, cryptography, enlightenment, alchemy, the Newton-Leibniz calculus controversy, piracy, Cromwell, vagabonds and lords, monetary policy, the age or reason, English and French monarchies, baroque architecture and city planning, King Salomon’s Gold, South-East Asian warlords, duels, unrequited love, the Plague, horology, Tzar Nicholas, buried treasure, Catholic-Protestant-Puritan battle for ascendancy, sea voyages, hand-to-hand combat with period arms, the Inquisition, the fire of London, The Royal Society, and 20 other joyously interconnected ideas.

What is more it is imbued with colorful heroes and villains, many based on unimaginably detailed research of real people living at the time. It carries on at a good pace and many sections leave you smiling and shaking your head at the vocabulary and virtuosity of this remarkable author.

The Baroque cycle is an epic masterpiece by a genius.

It is so good, that this is the second time I’ve read it.


Filed Under Science

I don’t know why, but lately, I’ve been studying ants.

It started with a reference in a Richard Dawkins book to the work of E.O. Wilson — the famous mymecologist (a $10 word for ant biologist). Wilson is credited with popularizing the controversial area of sociobiology – an evolutionary explanation of social behaviors such as altruism and aggression.

I have recently read a number of books on ants including The Leafcutter Ants by E.O. Wilson and Bert Holldobler and also Before The Swarm by Nicholas Griffin.

Evolution has imbued many ant species with mind-numbing social organization and economic sophistication. Some species form super-colonies with millions of individuals. These species have evolved the specialization of agriculture – they have been farming fungus millions of years. The Ants and their fungus have formed a mutualistic dependence so that particular species of fungus are only found in Ant colonies — no where else in nature. The ants form massive marching columns to go out to cut fresh leaves and stems. They don’t eat the leaves (they can’t digest it). Instead, they bring back the pieces, chew them and secrete special liquids onto the bolus, and then feed the fungus with it.  The Ants grow and eat this one specific species of fungus and the fungus uses the Ants to bring it organic matter and to weed out parasites and competing species of fungus.

Something like 99.99% of all the ants in a leaf-cutter colony are sisters. There is usually only one queen who is an egg-laying machine. She produces virtually all female workers who, according to size and age, taken on specific tasks within the colony. In some species, they switch tasks when they reach a certain size or age.

The colonies can be huge (acres of underground tunnels, chambers and air ventilation shafts) and are built up over decades.

Atta cephalotes carrying a leaf with nestmates hitching a ride

The queen will produce male and ‘royal’ offspring (female ants capable of reproduction and flight)  only a few times — right before mating season. When these princesses are ready to leave the next for their mating flight, they grab a chunk of fungus and lodge it in a special cavity under their head.

After mating, they dig a burrow, remove the fungus and start to farm it. She dosen’t eat any of it until the crop is well underway and she has produced enough daughters to tend it properly.

After studying this, I can’t help but feel humbled.