Power Pinch

For John Rowell, chief technology officer at OpSource Inc., keeping a lid on data center power costs is a make-or-break proposition. The Santa Clara, Calif.-based company hosts software-as-a-service offerings. As OpSource expanded its operations to meet customer demand between 2005 and 2006, its electricity costs spun out of control. We had a 2.75 multiple in power costs over a nine-month period, he says, but OpSources business model doesnt allow it to pass on those costs. I had to eat it, Rowell says. Now hes more mindful of energy use.

Data center energy demands, once a line-item footnote, are becoming a bigger concern as power and cooling loads continue to rise, according to Computerworlds latest quarterly Vital Signs survey. Of 194 IT professionals surveyed in February and March, 82% said they consider energy efficiency a factor when selecting IT equipment, and 20% of those at large companies said its a big consideration.

Servers are central to the problem, representing 60% to 80% of power used in data centers, according to John Koomey, staff scientist at Lawrence Berkeley National Laboratory. A study he recently conducted showed that server electricity use in U.S. data centers doubled from 2000 to 2005.

Data centers in the U.S. are now consuming as much energy per square foot as the industrial sector, says Paul Perez, vice president of storage, network and infrastructure at Hewlett-Packard Co. That trend caught the attention of Congress, which last year directed the U.S. Environmental Protection Agency to study ways to promote the use of energy-efficient servers in data centers. The EPAs work with Lawrence Berkeley is expected to lead to an Energy Star rating for servers.

Another study, by Christian Belady, distinguished technologist at HP, demonstrates that the per-server life-cycle cost of data center infrastructure already exceeds the per-server acquisition cost. Electricity costs will surpass initial hardware costs next year and that doesnt include the expense of cooling, which typically doubles the total power requirement. Rising operating costs also lead to higher capital expenses, because infrastructure from cooling systems to power distribution and power supply systems must scale to meet demand.

And power density is expected to continue its upward spiral. Industry projections show per-rack power densities hitting 45 kilowatts by 2014 (current designs top out at around 30 kilowatts), and research firm IDC predicts that power costs will grow at four times the rate of spending on new servers through 2010.

OpSource reacted to its power problem by renegotiating its contracts with the service providers that house its servers, basing the agreements on power requirements first and space and cooling second. Thats smart because 75% to 80% of infrastructure costs are now related to watts, not area, says Amory Lovins, chairman and chief scientist at the Rocky Mountain Institute Inc., a nonprofit energy-efficiency consulting firm in Snowmass, Colo.

Rowell now factors in energy costs when he buys new equipment. We deploy servers based on watts per CPU. If we spend 10% more upfront for a potential energy life savings of 30% to 40%, thats very interesting to us, he says.

On the equipment side, the low-hanging fruit [is] the power supply, says Koomey. The inefficient power supplies used in many volume servers can waste more than one-third of electricity before it reaches the IT equipment. Thats because efficiency drops with the IT load. Server utilization rates of around 15% and widespread use of redundant power supplies keep efficiency low.

High-efficiency designs cost 15% to 20% more, but they extend efficiency well beyond 80%, even at low utilization levels, according to Al Rozman, vice president of engineering at ColdWatt Inc., a power supply vendor in Austin. Major server vendors all claim to be shipping or planning to ship high-efficiency power supplies in their volume server lines, and they expect to push efficiency above 90%. Belady says he expects power conversion efficiencies to improve by 30% during the next two to three years.

It makes good business sense to improve the energy efficiency of data centers, even if doing so means paying more for equipment upfront, says Rowell. Because we have to run our infrastructure as a profit center, we are very focused on [getting] the most efficiency we can out of the infrastructure we have in place, he says. Traditional IT cost centers should be doing the same. Its irresponsible not to.

Identifying the Problem

Unfortunately, many data center operators still dont see the problem coming. Forty-one percent of Vital Signs survey respondents said they still dont know how much energy their data centers use because they dont pay for it.

Philip Borneman, assistant director of IT for the city government of Charlotte, N.C., says he didnt know what his energy costs were until he moved to a new data center. Suddenly, power was metered separately and billed to the IT budget. That was the rude awakening, he says. Now the city has a strong incentive to keep costs under control.

Most people are caught off-guard, says Sabet Elias, chief technology officer at financial services firm Lehman

Brothers Holdings Inc. in New York. Due to the lack of transparency [in data center energy costs], most people only become aware of the problem when theyre out of power, he says, noting that there are limits on how much electricity the local utility can run to a given facility. As power demands increase, more and more data centers are hitting that wall.

Joe Hedgecock, senior vice president and head of platform and data centers at Lehman Brothers, says power consumption is becoming one of his top concerns. Were more constrained by power and cooling these days than by space, he says.

The Wall Street firm has 13,000 servers in six data centers worldwide and is migrating many of them onto server blades. That saves space but creates hot spots that require supplemental, targeted cooling systems located directly above the racks. The design pipes liquid refrigerant to a heat exchanger, which blows cold air into the racks. Targeted cooling is more energy-efficient than room air conditioning because the chilled air must travel a much shorter distance to cool the load.

Energy efficiency is a big factor in Lehman Brothers data center designs. The data centers were building have a high focus on power and cooling, says Elias. His strategy includes the use of blades, virtualization, grid computing and multicore processors to reduce power, cooling and space demands.

Those technologies offer a one-time savings as servers are consolidated, but the underlying cause of the problem

compute density thats rising faster than energy efficiency gains continues unabated. Gartner Inc. analyst Michael Bell predicts that by 2008, half of all data centers will lack the power and cooling resources to meet the demand of higher-density computers.

Server manufacturers are beginning to respond with better power management and more efficient hardware designs. This is the year when youll start seeing action from the platform vendors, says Brent Kerby, Opteron product manager with the server team at chip maker Advanced Micro Devices Inc.

Energy-efficient data centers offer another benefit. In a coal-fired plant, it takes a pound of coal to generate a kilowatt-hour of electricity, says Lovins. A data center that saves 1 watt eliminates the burning of two tons of coal over the course of 20 years. As concern about global warming brings increased regulation, energy-efficient data centers will be ahead of the game. But those savings also directly affect the bottom line. If you can design a watt out of your data center now, its worth upwards of 20 bucks in your pocket today in avoided costs, Lovins says.

While the consequences of inattention to power and cooling issues can be disastrous, the problem is addressable, says Belady. Its only a crisis if you let it happen to you, he says. All of these things can be solved.