Gap emerges between high-performance computing hardware, software

Computerworld - WASHINGTON -- High-performance computing (HPC) is emerging as a critical business need for companies that use simulation and virtualization to test and design products, because such companies can potentially cut product development time and improve speed to market.
"Our innovation process is insatiable and perpetual," said Thomas Lange, director of corporate research and development, modeling, and simulation at Cincinnati-based The Proctor & Gamble Co., and "our need for speed is huge."
But there's a growing gap in hardware and software capabilities in high-performance systems. While hardware vendors can build systems with hundreds or thousands of processors, many HPC applications produced by independent software vendors typically use just 12 or 16 processors in parallel, according to users and a newly released study by Framingham, Mass.-based market research company IDC.
"Hardware is getting there," said Lange, but "software is way behind."
That was one of the conclusions in the IDC study on HPC applications. The study, sponsored by the Defense Advanced Research Projects Agency and the Council on Competitiveness, was released yesterday in conjunction with the council's High Performance Users Conference here. Lange and other HPC users, vendors and researchers attended the one-day event.
The study found that most HPC vendors focus on the technical system market, which relies on PCs, workstations and smaller servers, because that's where the demand and revenue is, said Earl Joseph, an analyst at IDC. The number of users looking to scale their systems across hundreds or thousands of processors isn't large enough to justify the cost of rewriting and testing applications on large systems, he said.
The council believes HPC is critical for U.S. companies to be competitive. For instance, in Proctor & Gamble's case, even something as simple as removing a bottle cap can require millions of calculations to recreate and test in a computer-generated environment. But because of the limits of software, physical testing is often still needed, said Lange.
"Full virtualization is impossible," he said.
Conference attendees generally agreed that there's great business value in being able to conduct product development in a virtual environment, and some industries -- including life sciences, transportation, and oil and gas -- develop code that can scale across many processors. But broader adoption of HPC is needed, some argued at the conference.
"Innovation will be the single most important factor in determining America's competitiveness," council President Deborah Wince-Smith said at the conference. HPC "is a fundamental platform for business advantage," she said.
Donald Paul, vice president and chief technology officer at Chevron Corp. in San Ramon, Calif., said the main role for government should be research, and