Brain behind IBM's Watson not unlike a human's

Computerworld - IBM's Watson supercomputer, which shellacked Jeopardy's top human champions during airings of the game show this week, is powered by 90 servers and a network-attached storage (NAS) cluster with 21.6TB of data.

In the end, though, its brain only uses the equivalent of 80% of the processing power of a human brain.

Tony Pearson, master inventor and senior consultant at IBM, explained that Watson only uses about 1TB of data to process real-time answers to Jeopardy questions after its back end storage is configured as RAID, and then the data is further culled to be loaded into the clustered server system's memory.

Pearson cited the estimate of technology futurist and author Ray Kurzweil that the human brain can hold about 1.25TB of data, and performs at roughly 100 teraflops. In comparison, Watson is an 80-teraflop system with 1TB of memory.

"So it's 80% human," Pearson said. "Yes, we could have handled a lot more information. We could have put more memory in each server, but once we got the answers to three seconds, we didn't need to go further."

Pearson explained that reaching the three-second answer threshold was just a matter of simple mathematics.

The original algorithm ran as a single threaded process on a single core processor took two hours to scan memory and produce an answer. So the IBM technologists just divided two hours by 2,880 CPUs, which produced the ability to answer questions in three-seconds.

If IBM's Watson were just some other human Jeopardy contestant, viewers probably would have tuned out in the midst of such a landslide victory. Instead, interest in the man vs. machine battle gave the show its highest ratings in nearly six years.

Competition between humans and computers have long captured the public's imagination. Remember the 1996 chess match between IBM's Deep Blue computer and the reigning world champion Garry Kasparov?

In this case, though, Watson has more in common with humans than Deep Blue. Like us, he only uses a small percentage of his massive memory capacity to answer questions.

Behind Watson's simple scribble-faced avatar that he used as a Jeopardy contestant are 90 IBM Power 750 Express servers powered by 8-core processors -- four in each machine for a total of 32 processors per machine. The servers are virtualized using a Kernel-based Virtual Machine (KVM) implementation, creating a server cluster with a total processing capacity of 80 teraflops. A teraflop is one trillion operations per second.

On top of the processing power, each server has 160GB of DRAM to provide the full machine with almost 15TB of memory.

On the backend of the computer is IBM's SONAS General Parallel File System (GPFS). SONAS, or Scale-Out NAS, is a Linux-based clustered file system that IBM released almost exactly one year ago.