Skip the navigation

Good Morning, Dave . . .

The Defense Department is working on a self-aware computer.

By Kathleen Melymuka
November 11, 2002 12:00 PM ET

Computerworld - Any sci-fi buff knows that when computers become self-aware, they ultimately destroy their creators. From 2001: A Space Odyssey to Terminator, the message is clear: The only good self-aware machine is an unplugged one.
We may soon find out whether that's true. The Defense Advanced Research Projects Agency (DARPA) is accepting research proposals to create the first system that actually knows what it's doing.
The "cognitive system" DARPA envisions would reason in a variety of ways, learn from experience and adapt to surprises. It would be aware of its behavior and explain itself. It would be able to anticipate different scenarios and predict and plan for novel futures.
"It's all moving toward this grand vision of not putting people in harm's way," says Raymond Kurzweil, an artificial intelligence guru and CEO of Kurzweil Technologies Inc. in Wellesley Hills, Mass. "If you want autonomous weapons, it's helpful for them to be intelligent."
Cognitive systems will require a revolutionary break from current computer evolution, which has been adding complexity and brittleness as it adds power.
"We want to think fundamental, not incremental improvements: How can we make a quantum leap ahead?" says Ronald J. Brachman, director of DARPA's Information Processing Technology Office in Arlington, Va. Brachman will manage the agency's cognitive system initiative.
The goal is to create systems that take better care of themselves, and some manufacturers have already made small advances, Brachman points out. Software that tests itself automatically is a step in the right direction. So is software that walls itself off to avoid taking down the larger system in case it crashes.
Add advances in speech recognition and machine learning, and there may be enough "bits and pieces" to achieve the critical mass necessary for a real breakthrough, Brachman says.
"You get enough really smart people working on a really hard problem, and you get outcomes you didn't really expect," he adds. "We're hoping for a little serendipity."
They'll need it. The problems to be addressed are nearly as imposing as the dream. For example:
• How can a cognitive system learn from experience and use what it has learned to cope with new situations?
• How can it prioritize "standing orders," given complex and conflicting goals?
• How can it recognize important low-frequency events among the huge amounts of data in its "experience?"
• How can it use context to decipher complex actions, events and language?
Despite the challenges, Brachman is undaunted. "DARPA is about looking out of the box, the big reach," he says. "If we succeed, we can change the world in very dramatic ways."
Kurzweil agrees. "DARPA research tends

Our Commenting Policies