Los Alamos shuts down supercomputers as fire advances

It's a record year for wildfires and something IT equipment doesn't like: smoke

The Los Alamos National Lab has shut down two of its largest supercomputers, as wildfires continue to burn near this sprawling New Mexico facility.

There were no wildfires on laboratory property Wednesday, but smoke is prevalent throughout the area.

The National Weather Service has called air quality "potentially unhealthy," and Los Alamos residents are being urged to minimize exposure to it. The lab will be closed through Thursday.

A Los Alamos spokeswoman said the laboratory conducted an "orderly shutdown" of two of its largest supercomputers, IBM's Roadrunner, the first the break the petaflop barrier in 2008, and now the 10th ranked most powerful supercomputer in the world, and Cielo, a Cray system that is ranked No. 6 on the Top500 list.

The supercomputer shutdowns were conducted "early on," but an exact day or reason for the action wasn't clear. The laboratory was first closed on Monday in response to wildfires that have now burned more than 100 square miles.

But smoke is a potential threat to data centers and IT equipment. It can trigger a shutdown of air handling systems and IT equipment. Particulates from smoke can damage components. And this has been a record year for smoke-producing wildfires.

According to the National Climate Data Center, there were 6,625 fires which burned approximately 1.1 million acres in May -- the most acres burned during the month of May on record. The amount of acreage burned so far this year is 3.45 million acres, "the largest in the 12-year period of record," the climate center said.

The problem with smoke is that it "is so fine that it can get into things like your hard drive, it can get between your processors and your motherboards, it can get in between your ram and motherboards," said Jason Burnett, the director of infrastructure at NeoSpire, a managed hosting provider in Dallas.

Texas has experienced some big wildfires, and it also had a lot of heat, which is why NeoSpire uses internally air-conditioned air for cooling and not external air, in what's called a closed-loop system. "For our specific purposes I wouldn't be affected [by wildfires] at all because I don't take any air from the outside in," Burnett said.

He said his air cooling approach is a common one in the southwest because of hot temperatures.

In more northern and cooler climates, data centers are increasingly using outside air, or air-side economizers. These designs typically include sufficient filtration to address external issues, said Tad Davies, executive vice president of the Bick Group, a company whose work includes data center design.

In the case of wild fire activity, the data center would resort to a backup closed loop system. If smoke has infiltrated the building, the smoke detectors will shut down the air handling units.

Once the cooling systems are shut down "you are kind of done," said Vince Renaud, managing principal and vice president of the Uptime Institute. Meaning that once cooling systems turn off, the IT equipment will hit their temperature thresholds rather quickly. At that point, systems have to be shut down, he said.

Jeff Pederson, vice president of data recovery operations at Kroll Ontrack, whose work includes recovering data from damaged storage systems, said the threat of smoke is more of an issue for small businesses that aren't running traditional data centers with monitoring equipment.

Pederson said he sees problems in businesses and in homes with cigarette smokers, where the smoke affects performance and increases the likelihood of a physical crash.

But Burnett does expect to see more wildfires in Texas. He said he grew up in the country, on a cattle ranch out in south Texas, and sees the population getting closer to rural areas and places which don't have same firefighting equipment available in developed areas.

"I think [wildfires] are going to be a lot more prevalent as the urban creep keeps happening," Burnett said.

Patrick Thibodeau covers SaaS and enterprise applications, outsourcing, government IT policies, data centers and IT workforce issues for Computerworld. Follow Patrick on Twitter at  @DCgov or subscribe to Patrick's RSS feed . His e-mail address is pthibodeau@computerworld.com.

Copyright © 2011 IDG Communications, Inc.

7 inconvenient truths about the hybrid work trend
Shop Tech Products at Amazon