Inputs and Outputs

I am ‘tech support’ for my parents and in-laws. Whenever I travel to Winnipeg to visit them, I spend hours installing drivers, purging spam, defragmenting disks, installing printers and memory, etc. etc. 

During my last visit, my mother-in-law was telling me something about her ‘computer’. It took me a moment but then I realized she was actually talking about her CRT monitor. To her,  the monitor was the computer. That is because that is the part that she looks at. The glowing screen is where she receives feedback on her keyboard and mouse inputs and see the resulting outputs. She doesn’t interact with the grey box full of boards, wires and spinning disks — it is irrelevant to what she is trying to accomplish. To her (as to many people) the input/output transducers are the computer.

My mother-in-law’s cognitive model of computing brought to mind a time in the 90’s where I worked with a group of user-interface geniuses at the University of Toronto.  I was a research manager funding and promoting the work of professors, students and staff working on multi-disciplinary projects within the departments of computer science, engineering, sociology and psychology.  These folks were creating the intellectual framework for many of the user interfaces that we use today.

Hiroshi Ishii working the Active Desk at the Ontario Telepresence Project

I worked closely with a team that was on the forefront of what was then called Computer Supported Collaborative Work.  The Ontario Telepresence Project  built and experimented with numerous computing, audio and video systems to enable collaboration at a distance. It was there, years before the Internet became part of our society’s fabric, that I learned about distance collaboration through videoconferencing and computing systems.

Another one of my favourite groups were the Neural Networks guys.  This eclectic bunch were exploring how to create dynamic, adaptive computing systems which simulated how our brains learn.  Their research accomplishments were many, but among the most evocative for me was student Sidney Fels’ Glove Talk II.

Sidney Fels' Glove Talk II

Listen to the Hand (click for 30-second movie)

In Sidney’s own words:

GloveTalkII is a system that translates hand gestures to speech through an adaptive interface. Hand gestures are mapped continuously to ten control parameters of a parallel formant speech synthesizer. The mapping allows the hand to act as an artificial vocal tract that produces speech in real time. This gives an unlimited vocabulary in addition to direct control of fundamental frequency and volume. Currently, the best version of Glove-TalkII uses several input devices (including a Cyberglove, a ContactGlove, a three space tracker, and a foot pedal), a parallel formant speech synthesizer, and three neural networks. One subject has trained to speak intelligibly with Glove-TalkII. He speaks slowly but with far more natural sounding pitch variations than a text-to-speech syntesizer. 

It is incredible to think that this ground-breaking work was done 15-years ago and only now are gesture-based inputs finding their way into the marketplace.  Apple’s iPod with it’s gesture control is only the most famous example of many new products in this field.

I believe that increased processor speeds, miniturization and mobilitiy are important factors in bringing the benefits of information technology to our society.  But we will also need innovative, adaptive, intuitive user-interfaces which fit the way we work and interact.