Brain behind IBM's Watson not unlike a human's
Like humans, Watson only uses a fraction of its memory to generate answers to Jeopardy questions
Computerworld - IBM's Watson supercomputer, which shellacked Jeopardy's top human champions during airings of the game show this week, is powered by 90 servers and a network-attached storage (NAS) cluster with 21.6TB of data.
In the end, though, its brain only uses the equivalent of 80% of the processing power of a human brain.
Tony Pearson, master inventor and senior consultant at IBM, explained that Watson only uses about 1TB of data to process real-time answers to Jeopardy questions after its back end storage is configured as RAID, and then the data is further culled to be loaded into the clustered server system's memory.
Pearson cited the estimate of technology futurist and author Ray Kurzweil that the human brain can hold about 1.25TB of data, and performs at roughly 100 teraflops. In comparison, Watson is an 80-teraflop system with 1TB of memory.
"So it's 80% human," Pearson said. "Yes, we could have handled a lot more information. We could have put more memory in each server, but once we got the answers to three seconds, we didn't need to go further."
Pearson explained that reaching the three-second answer threshold was just a matter of simple mathematics.
The original algorithm ran as a single threaded process on a single core processor took two hours to scan memory and produce an answer. So the IBM technologists just divided two hours by 2,880 CPUs, which produced the ability to answer questions in three-seconds.
If IBM's Watson were just some other human Jeopardy contestant, viewers probably would have tuned out in the midst of such a landslide victory. Instead, interest in the man vs. machine battle gave the show its highest ratings in nearly six years.
Competition between humans and computers have long captured the public's imagination. Remember the 1996 chess match between IBM's Deep Blue computer and the reigning world champion Garry Kasparov?
In this case, though, Watson has more in common with humans than Deep Blue. Like us, he only uses a small percentage of his massive memory capacity to answer questions.
Behind Watson's simple scribble-faced avatar that he used as a Jeopardy contestant are 90 IBM Power 750 Express servers powered by 8-core processors -- four in each machine for a total of 32 processors per machine. The servers are virtualized using a Kernel-based Virtual Machine (KVM) implementation, creating a server cluster with a total processing capacity of 80 teraflops. A teraflop is one trillion operations per second.
On top of the processing power, each server has 160GB of DRAM to provide the full machine with almost 15TB of memory.
On the backend of the computer is IBM's SONAS General Parallel File System (GPFS). SONAS, or Scale-Out NAS, is a Linux-based clustered file system that IBM released almost exactly one year ago.
- Money talks, and that's all quantum maker D-Wave has to say
- IBM project aims to forecast and control Beijing's air pollution
- China has the fastest supercomputer, but the U.S. still rules
- ISC: Cray makes Lustre palatable for storage administrators
- SC500: China wins a slowing supercomputer race
- Fujitsu 56 Gbps circuit doubles communication speeds between CPUs
- HP enters supercomputing market with water-cooled Apollo system
- In exascale, Japan stands apart with firm delivery plan
- Here comes a supercomputing app store
- An HPC champion helps Trek Bicycle shift gears
- Move Mission-Critical Apps to the Cloud with AWS and F5 Read this paper to learn about adoption inhibitors of the cloud, potential solutions, and how advanced Application Delivery Controller (ADC) technologies are critical...
- Pivotal Melds Big Data and Platform-as-a-service The value of Information has increased, so has the business's thirst for more information. Access to data and collaboration are at the heart...
- Operationalizing the Buzz: Big Data 2013 The 2013 EMA/9sight Big Data research surveyed 259 business and technology stakeholders around the world.
- The Pivotal Big Data Suite- Reducing the Risks of Big Data The explosion of big data and the rapid evolution of big data tools and technologies is challenging IT to meet the demands of...
- What Does it Take to Deliver a Superior Customer Experience? The Two Top-Rated Online Retailers, B&H Photo and Crutchfield Electronics, Share Their Secrets Discuss practical CX tools and service methods such as contact center agents and the use of realtime speech analytics to help contact center...
- Keep Servers Up and Running and Attackers in the Dark An SSL/TLS handshake requires at least 10 times more processing power on a server than on the client. SSL renegotiation attacks can readily... All High Performance Computing White Papers | Webcasts