The National Oceanic and Atmospheric Administration (NOAA) Thursday switched on two new supercomputers that are expected to improve weather forecasting.
The "go live" switch over to the new systems was made today without any fanfare, just a box of donuts on hand to mark 18 months of preparation and testing. The new IBM systems are now responsible for producing forecast data that's relied on in the U.S. and around the world.
The agency had planned to go live next Tuesday, but strong storms now in the Pacific and Atlantic Oceans prompted the decision to move the date up, said Ben Kyger, director of central operations at the National Centers for Environmental Prediction.
NOAA's National Weather Service (NWS) runs the two identical, or clone systems, one in Reston, Va., and in Orlando, Fla. They can switch over in about six minutes.
The supercomputers are each 213 teraflop systems, running a Linux operating system on Intel processors. The U.S. is paying about $20 million a year to operate the leased systems.
"These are the systems that are the origin of all the weather forecast you see," said Kyger. The computing system is called the Weather and Climate Operational Supercomputing System or WCOSS.
The two previously used four-year-old IBM supercomputers, also identical, run IBM's AIX operating system and Power 6 chips, and are each capable of 74 teraflops. The upgrade will continue to improve weather forecasting over time.
It's been a complicated process to get to this point. The NWS has had to ensure that the software running on the new system is producing scientifically correct results. It had been running the old and new systems in parallel for months, and comparing the output.
This comparative testing involved examining output data to determine whether it is numerically reproducible out to five decimal places. There is also a statistical analysis of weather predictions on the new system against the actual weather conditions.
The process wasn't just an examination of numerical data. NWS scientists also studied the weather products and examined them for subtle differences. "There is a lot of human, highly experienced, subjective evaluation," said Kyger.
There are computational differences involved in switching to new chips and a new operating system. They are subtle, and appear in decimal places six through 12.
As you go further out in a forecast, the differences compound.
The changes may appear in the fifth day of an extended, five-day forecast as a difference of one degree.
The NWS has a new hurricane model, Hurricane Weather Research and Forecasting (HWRF), which is 15% more accurate in day five of a forecast both for forecast track and intensity. That model is now operational and running on the new systems. That's important, because U.S. is expecting a busy hurricane season.
"That is a huge improvement, and we can't make those kinds of improvements unless we have a bigger computer," said Kyger, of the 15% gains.
More compute power allows for higher resolution, which enables scientists to look at forecasting impacts over much smaller areas.
Right now, the NWS global forecasting system runs at a resolution of 27 kilometers, with 64 vertical levels, which shows the atmospheric conditions at various heights.
Last fall, when Hurricane Sandy struck, there was a belief that the European Centre for Medium-Range Weather Forecasts (ECMWF) had a better storm track model further out.
Criticism over the U.S. forecasting ability has followed post Sandy.
The European model "did a better job at some points in the storm track that the U.S. models did," said Barry Lee Myers, the CEO of AccWeather Inc., at a House hearing in May on "Restoring U.S. Leadership in Weather Forecasting."
During Sandy, the European Center had computers capable of producing resolution at 16 kilometers, with 130 vertical levels, according to NWS officials.
In nine months, the NWS expects to be at 13 kilometers resolution.
The increased resolution is "going to improve the lead time for all significant weather events," said Kyger.
In June, ECMWF announced that it had signed a contract to purchase two Cray XC30 systems. The speed of these systems is not being disclosed but a Cray spokesman said "it will be a petascale system."
Kyger referred to the Europeans as "teammates" and said they use each other's models. "I would call it friendly competition," he said.
Patrick Thibodeau covers SaaS and enterprise applications, outsourcing, government IT policies, data centers and IT workforce issues for Computerworld. Follow Patrick on Twitter at @DCgov, or subscribe to Patrick's RSS feed . His email address is pthibodeau@computerworld.com.
Read more about hardware in Computerworld's Hardware Topic Center.