Brain behind IBM's Watson not unlike a human's
Like humans, Watson only uses a fraction of its memory to generate answers to Jeopardy questions
Computerworld - IBM's Watson supercomputer, which shellacked Jeopardy's top human champions during airings of the game show this week, is powered by 90 servers and a network-attached storage (NAS) cluster with 21.6TB of data.
In the end, though, its brain only uses the equivalent of 80% of the processing power of a human brain.
Tony Pearson, master inventor and senior consultant at IBM, explained that Watson only uses about 1TB of data to process real-time answers to Jeopardy questions after its back end storage is configured as RAID, and then the data is further culled to be loaded into the clustered server system's memory.
Pearson cited the estimate of technology futurist and author Ray Kurzweil that the human brain can hold about 1.25TB of data, and performs at roughly 100 teraflops. In comparison, Watson is an 80-teraflop system with 1TB of memory.
"So it's 80% human," Pearson said. "Yes, we could have handled a lot more information. We could have put more memory in each server, but once we got the answers to three seconds, we didn't need to go further."
Pearson explained that reaching the three-second answer threshold was just a matter of simple mathematics.
The original algorithm ran as a single threaded process on a single core processor took two hours to scan memory and produce an answer. So the IBM technologists just divided two hours by 2,880 CPUs, which produced the ability to answer questions in three-seconds.
If IBM's Watson were just some other human Jeopardy contestant, viewers probably would have tuned out in the midst of such a landslide victory. Instead, interest in the man vs. machine battle gave the show its highest ratings in nearly six years.
Competition between humans and computers have long captured the public's imagination. Remember the 1996 chess match between IBM's Deep Blue computer and the reigning world champion Garry Kasparov?
In this case, though, Watson has more in common with humans than Deep Blue. Like us, he only uses a small percentage of his massive memory capacity to answer questions.
Behind Watson's simple scribble-faced avatar that he used as a Jeopardy contestant are 90 IBM Power 750 Express servers powered by 8-core processors -- four in each machine for a total of 32 processors per machine. The servers are virtualized using a Kernel-based Virtual Machine (KVM) implementation, creating a server cluster with a total processing capacity of 80 teraflops. A teraflop is one trillion operations per second.
On top of the processing power, each server has 160GB of DRAM to provide the full machine with almost 15TB of memory.
On the backend of the computer is IBM's SONAS General Parallel File System (GPFS). SONAS, or Scale-Out NAS, is a Linux-based clustered file system that IBM released almost exactly one year ago.
- Money talks, and that's all quantum maker D-Wave has to say
- IBM project aims to forecast and control Beijing's air pollution
- China has the fastest supercomputer, but the U.S. still rules
- ISC: Cray makes Lustre palatable for storage administrators
- SC500: China wins a slowing supercomputer race
- Fujitsu 56 Gbps circuit doubles communication speeds between CPUs
- HP enters supercomputing market with water-cooled Apollo system
- In exascale, Japan stands apart with firm delivery plan
- Here comes a supercomputing app store
- An HPC champion helps Trek Bicycle shift gears
- Enable secure remote access to 3D data without sacrificing visual perfomance Design and manufacturing companies must adapt quickly to the demands of an increasingly global and competitive economy. To speed time to market for...
- The Truth About Virtual Computing for CAD If you're a user of graphics-intensive software such as 3D modeling, simulation and analysis, and visualization, you might be skeptical about moving to...
- Simplifying Product Design In A Complex World Product design engineering has moved far beyond the confines of ever-more powerful workstations. Companies can't afford to restrict projects to using only local...
- A Reference Architecture for the Internet of Things The aim of this is to provide Architects and Developers of IoT projects with an effective starting point that covers the major requirements...
- What should I look for in a Next Generation Firewall? SANS Provides Guidance With so many vendors claiming to have a Next Generation Firewall (NGFW), it can be difficult to tell what makes each one different....
- Why Are Customers Really Deploying an NGFW? It seems every IT Security expert is talking about the NGFW, but what are people really doing? This webcast covers 5 real-world customer... All Hardware White Papers | Webcasts
Our new weekly Consumerization of IT newsletter covers a wide range of trends including BYOD, smartphones, tablets, MDM, cloud, social and what it all means for IT. Subscribe now and stay up to date!