The Grill: MIT Media Labs' David Merrill on tangible computing

Computerworld - David Merrill, a graduate student at the MIT Media Lab, wants to make your computer work for you -- literally. Imagine arranging images, composing electronic music and completing math and language tasks on your computer using a more three-dimensional approach.

Merrill believes that the little miracle blocks he calls Siftables will help us interact with digital media in a more natural, tangible way. As their co-creator, he is passionate about this science of "embodied media."

What is embodied media, and in what ways does it overcome the obstacles of traditional computing? The computational power of everyday computers has grown enormously in recent years. As a result, the most important bottleneck is now the effectiveness of the human-computer interaction rather than the speed of the processor. We've had the keyboard and mouse for more than 40 years now; while these are still useful interfaces, I don't believe they are the best we can do for all of our computing needs.

Embodied media offers a new point in the interaction design space between tangible and graphical user interfaces. It combines elements of both paradigms -- physically embodied manipulatives that can be grasped and moved by hand, and screens that can show visual information. The graphical display is a key feature compared to other "tangibles," since it allows the interactive roles and content assignments to manipulatives to be visually legible to the user and dynamically assigned at runtime.

Can you talk about your past experience with computer science? How did you become involved with this type of work? My first programming experience was using Logo to draw geometric patterns when I was in fourth grade. Then, in high school, I wrote some math applications and games for my TI-82 graphing calculator after a friend showed me how to use variables and loops. But it wasn't until I was an undergraduate at Stanford that I really fell in love with computer science.

At first, the elegance and flexibility of software satisfied my budding inner geek, but after I took an electronic-instrument-building class taught by Bill Verplank and Max Matthews at the computer music center, I realized that designing totally new hardware devices opened a world of possibilities for interfaces. Building physical systems for human-computer interaction became my obsession. I built several musical interfaces at Stanford, and in my research at the MIT Media Lab I developed a number of new physical interfaces.

When did you begin working on Siftables, and what are they made of? Siftables began a few years ago as a brainstorm with Jeevan Kalanithi; we imagined how people might interact with digital information by using their hands to manipulate a sea of tiny physical, active, computational objects. We were influenced by ideas from tangible interfaces, pervasive computing and sensor networks, but only later would Siftables be contextualized against the backdrop of these ideas as a hybrid platform that blended these themes with the flexibility of pixels that defines graphical user interfaces. The beginning was pure inspiration, an uninhibited "what if" speculation about a system that would permit compelling new physical interactions.