Brain behind IBM's Watson not unlike a human's
Like humans, Watson only uses a fraction of its memory to generate answers to Jeopardy questions
Computerworld - IBM's Watson supercomputer, which shellacked Jeopardy's top human champions during airings of the game show this week, is powered by 90 servers and a network-attached storage (NAS) cluster with 21.6TB of data.
In the end, though, its brain only uses the equivalent of 80% of the processing power of a human brain.
Tony Pearson, master inventor and senior consultant at IBM, explained that Watson only uses about 1TB of data to process real-time answers to Jeopardy questions after its back end storage is configured as RAID, and then the data is further culled to be loaded into the clustered server system's memory.
Pearson cited the estimate of technology futurist and author Ray Kurzweil that the human brain can hold about 1.25TB of data, and performs at roughly 100 teraflops. In comparison, Watson is an 80-teraflop system with 1TB of memory.
"So it's 80% human," Pearson said. "Yes, we could have handled a lot more information. We could have put more memory in each server, but once we got the answers to three seconds, we didn't need to go further."
Pearson explained that reaching the three-second answer threshold was just a matter of simple mathematics.
The original algorithm ran as a single threaded process on a single core processor took two hours to scan memory and produce an answer. So the IBM technologists just divided two hours by 2,880 CPUs, which produced the ability to answer questions in three-seconds.
If IBM's Watson were just some other human Jeopardy contestant, viewers probably would have tuned out in the midst of such a landslide victory. Instead, interest in the man vs. machine battle gave the show its highest ratings in nearly six years.
Competition between humans and computers have long captured the public's imagination. Remember the 1996 chess match between IBM's Deep Blue computer and the reigning world champion Garry Kasparov?
In this case, though, Watson has more in common with humans than Deep Blue. Like us, he only uses a small percentage of his massive memory capacity to answer questions.
Behind Watson's simple scribble-faced avatar that he used as a Jeopardy contestant are 90 IBM Power 750 Express servers powered by 8-core processors -- four in each machine for a total of 32 processors per machine. The servers are virtualized using a Kernel-based Virtual Machine (KVM) implementation, creating a server cluster with a total processing capacity of 80 teraflops. A teraflop is one trillion operations per second.
On top of the processing power, each server has 160GB of DRAM to provide the full machine with almost 15TB of memory.
On the backend of the computer is IBM's SONAS General Parallel File System (GPFS). SONAS, or Scale-Out NAS, is a Linux-based clustered file system that IBM released almost exactly one year ago.
- In exascale, Japan stands apart with firm delivery plan
- Here comes a supercomputing app store
- An HPC champion helps Trek Bicycle shift gears
- D-Wave pitches quantum co-acceleration to supercomputing set
- Why the U.S. may lose the race to exascale
- Top500 shows growing inequality in supercomputing power
- Supercomputing's big problem: What's after silicon?
- Cray brings Hadoop to supercomputing
- Intel rushes to exascale with redesigned Knights Landing chip
- China still has the fastest supercomputer in the world
- Case Study: Murphy USA Gains Application Visibility Without Agents Murphy USA has more than 700 stores that share a 10Mbps VSAT link. So when something goes wrong with their applications, it's the...
- Path Selection Infographic Path Selection Infographic
- Hyperconvergence Infographic A wide range of observers agree that data centers are now entering an era of "hyperconvergence" that will raise network traffic levels faster...
- Preparing Your Infrastructure for the Hyperconvergence Era From cloud computing and virtualization to mobility and unified communications, an array of innovative technologies is transforming today's data centers.
- LIVE EVENT: 5/7, The End of Data Protection As We Know It. Introducing a Next Generation Data Protection Architecture. Traditional backup is going away, but where does this leave end-users?
- On-demand webinar: "Mobility Mayhem: Balancing BYOD with Enterprise Security" Check out this on-demand webinar to hear Sophos senior security expert John Shier deep dive into how BYOD impacts your enterprise security strategy... All High Performance Computing White Papers | Webcasts