User Interfaces: See me, hear me...

Computerworld - Computers of the future may perform tasks based on what you say to them, how you gesture or even where you look. But that doesn't mean your keyboard and mouse are going away anytime soon.

"There's never a need to throw away something that works," says Joseph Olive, a director at the multimedia communications research laboratory at Lucent Technologies Inc.'s Bell Labs.

Researchers working on new generations of interfaces are taking a pragmatic approach to how we'll interact with machines, adding support for text, speech, gaze, gesture and more, depending on circumstances and what seems to make sense.

Ideally, tomorrow's computers will be better at anticipating what users want, without needing typed commands. This will involve "context-aware" interfaces, according to Ted Selker, head of the context-aware computing research group at MIT's Media Lab. That might mean a Web application will be able to sense both your mouse and eye movements to determine whether you've visited a site before and what items most interest you—and then dynamically generate a page based on those interests.

By 2006, Selker says, "the computer will know more about why you're doing what you're doing and what it can do to help you."

Say What?

Speech-recognition software is already making inroads in telephone-based customer service applications. Today's systems are often limited to narrow uses, such as saying the name of a company to get a stock quote. Work on broader-based systems is under way, though.

"What we're vigorously researching . . . is to let you speak much more freely," Olive says. Murray Hill, N.J.-based Lucent recently tested a prototype of an automated phone operator at a financial institution. Callers could say things like "I lost my checkbook" instead of wading through menu options.

The test involved routing callers to one of about 40 departments. About 8% of the calls had to be switched to a human receptionist because requests weren't specific enough. Among calls handled by the computer, the accuracy rate was 96%, Olive said.

Speech-recognition experts believe the technology will be increasingly used in mobile applications. After all, says Mike Phillips, chief technology officer at SpeechWorks International Inc. in Boston, it's tough to design an easy- to-use knob interface for a car MP3 player that's got a few thousand songs stored in it.

Other potential uses include more sophisticated hands-free dialing as well as climate control systems.

Speech recognition should also make it easier to use a small device to access data stored on a larger one. "Basically, anything that's on your desktop will be available to you by voice," Olive predicts.