The newest equipment concentrates more power into a smaller footprint on the raised floor, but the infrastructure needed to support every square foot of high-density compute space -- including cooling systems, power distribution equipment, UPSs and generators -- is getting proportionally larger.
Data center managers are taking notice. In a 2009 IDC survey of 1,000 IT sites, 21% of the respondents ranked power and cooling as the No. 1 data center challenge. Nearly half (43%) reported increased operational costs, and one-third said that they had experienced server downtime as a direct result of power and cooling issues.
Christian Belady is the lead infrastructure architect in Microsoft Corp.'s Global Foundation Services group, which designed and operates the company's newest data center in Quincy, Wash. He says the cost per square foot of a raised floor is too high. In the Quincy data center, he says, infrastructure costs accounted for 82% of the total project.
"We're beyond the point where more density is better," Belady says. "The minute you double compute density, you double the footprint in the back room."
As compute density per square foot increases, overall electromechanical costs tend to stay about the same, Gross says. But because power density also increases, the ratio of electromechanical floor space needed to support a square foot of high-density compute floor space also goes up.
IBM's Schmidt says the cost per watt, not the cost per square foot, remains the biggest construction expense for new data centers.
"Do you hit a power wall down the road where you can't keep going up this steep slope? The total cost of ownership is the bottom line here," he says. Those costs have for the first time pushed some large data center construction projects past the $1 billion mark. "The C-suites that hear these numbers get scared to death because the cost is exorbitant," Schmidt says.
Ever-higher energy densities aren't sustainable from an energy use or cost perspective, says Rakesh Kumar, an analyst at Gartner Inc. Fortunately, most enterprises still have a ways to go before they see average per-rack loads in the same range as ILM's. About 40% of Gartner's enterprise customers are pushing beyond the range of 8 to 10 kW per rack, and some are as high as 12 to 15 kW per rack. But those numbers are creeping up.
In response, some enterprise data centers, and managed services providers like Terremark Inc., are monitoring power use and factoring it into what they charge for data center space. "We're moving toward a power model for larger customers," says Ben Stewart, senior vice president of engineering at Terremark. "You tell us how much power, and we'll tell you how much space we'll give you."
Buying Kilowatts
But is it realistic to expect customers to know not just how much equipment they need hosted, but how much power will be needed for each rack of equipment?
"For some customers, it is very realistic," Stewart says. In fact, Terremark is moving in this direction in response to customer demand. "Many of them come to us with a maximum-kilowatt order and let us lay the space out for them," he says. If a customer doesn't know what its energy needs per cabinet will be, Terremark sells power per "whip," the power cable feed to each cabinet.
IBM's Schmidt thinks further power-density increases are possible, but the methods by which data centers cool those racks will need to change.