Hurricane Sandy leaves wounded servers in its wake

Computerworld - Data recovery experts have been busy in the wake of Hurricane Sandy, which left a slew of data centers underwater, damaging equipment and posing a significant threat to business-critical data.

Apparently disregarding weather forecasters' widespread warnings and underestimating the power of the storm that hit the East Coast late last month, many businesses didn't begin moving computer and IT communications equipment out of harm's way until it was too late, say officials at companies that specialize in data recovery.

Many data centers were casualties of the massive storm, and the damage threatened to shut down major New York-based businesses and interrupt Internet service across the country, according to experts.

For instance, the storm forced two so-called carrier hotels -- monolithic buildings that serve as major U.S. network hubs -- in lower Manhattan to operate on generator power for a significant period of time.

The two buildings -- a 2.9 million-square-foot structure at 111 8th Ave. and a 1.8 million-square-foot facility at 60 Hudson St. -- are said to be critical to the nation's infrastructure because they allow data sharing between users of different online networks.

"There is a high probability that your Internet traffic, every time you go on a website, passes through 111 8th Ave. at some point," said Michael Levy, an analyst at Datacenters Tier1 Research, a division of 451 Research.

"Everybody just underestimated the strength of the hurricane," said Todd Johnson, vice president of operations at Kroll Ontrack, which provides data recovery services.

Weeks after the storm, service providers like Kroll were still working to recover data from enterprise servers overwhelmed by storm water surges or by spikes in power in the New York metropolitan area.

Johnson said some Kroll Ontrack customers found servers sitting in water that was 10 to 13 feet deep.

The storm-damaged equipment ranges from desktop computers to servers, including stand-alone RAID systems running office systems at midsize to large businesses located in coastal areas, Johnson said.

Another data recovery firm, Drive Savers, was also still working weeks after the storm to restore waterlogged drives for its customers, said spokeswoman Michelle Taylor.

Experts say it's possible that storm-related damage in data centers could lead to significant server problems down the road. Data center systems usually operate in controlled environments with steady temperatures and humidity levels, but Sandy caused flooding that may have damaged the systems that control heating and cooling equipment.

One data center reported temperatures rising above 100 degrees Fahrenheit as staffers scrambled to repair a generator.

By breaking the environmental cocoons that protect IT equipment, the storm may have wounded some servers and set them up for component failures weeks or months from now, said Scott Kinka, CTO of cloud services provider Evolve IP.

If equipment operates at higher-than-recommended temperatures, it could face a higher risk of component failure, and data center managers might see an uptick in component problems, he added.

However, it could be months before such problems arise, and by that time, it would be very difficult to definitively trace a failure back to its root cause, according to Kinka.

"The hard part about this one is you are just not going to know," he said.

This version of this story was originally published in Computerworld's print edition. It was adapted from an article that appeared earlier on Computerworld.com.

Read more about Data Center in Computerworld's Data Center Topic Center.