Water Returns to the Data Center

As server heat loads grow, vendors are reintroducing water cooling options — and stoking old fears.

In some data centers, you can still find water pipes under raised floors that were once used to cool mainframes. And the idea of turning that water on again to cool off hot electronics is something that chills longtime IT professionals like Chad Gerbick, manager of operations at Kent State University in Ohio.

"I remember the water-cooled mainframes, and it was just such a hassle with that stuff because you always had to worry about water under the floor and pipes and the quality of the water," says Gerbick. "[Water cooling] is another point of failure that I wouldn't want to deal with."

Gerbick's reaction to the idea of bringing water close to very expensive electronics is widely shared among IT managers. But as server performance and density rise, water cooling may gradually move back into the mainstream of data center technologies.

Water cooling "is very definitely a viable technology and a necessary technology," says Bob Sullivan, a consultant at The Uptime Institute in Santa Fe, N.M. "Heat loads are going up so drastically that there is going to be no other way to cool these high heat densities other than water cooling."

Sullivan is hardly alone in this belief. Ken Baker, a data center infrastructure technologist at Hewlett-Packard Co., says that water cooling is "inevitable."

"The unrelenting power increase ... is forcing [users] to look at new ways to cool the hardware," says Baker. "Real-estate constraints will drive this."

Recognizing Limitations

The upper limit of air cooling alone in a single cabinet is about 10 kilowatts, says Baker. But power demands will continue to rise. For instance, Baker estimates that a 1U (1.75 in. high) server that draws 350 watts today will increase to more than 600 watts over the next few years. At today's levels, 10kw would equal about 30 servers.

Large rack systems with 98 blades in seven chassis can consume as much as 24kw of power. Data center managers typically fill just half the rack space to keep temperatures down, but that means spreading out the servers, which uses more floor space.

"The water scares the heck out of folks," says Thomas Roberts, director of data center operations at Novi, Mich.-based Trinity Health, which operates hospitals and outpatient facilities. His data center, which was built two and a half years ago, has 100% more cooling than it needs. "I view [water cooling systems] as being needed in an environment where you have no scalability," he says.

That's what vendors believe as well. They think early adopters of water cooling systems will be those with facilities in urban areas, such as Globix Corp.'s data center in London's business district.

In March, Globix, which provides networking and infrastructure services to businesses, installed a cooling system for its high-density racks. The InfraStruXure High Density system from American Power Conversion Corp. (APC) in West Kingston, R.I., is a self-contained data center with air and water cooling systems for Globix servers that consume 15kw in a rack containing some 80 blades.

"We found we did not want to be an [Internet service provider] that kept throwing space at the problem," says Philip Cheek, U.K. managing director at New York-based Globix.

The APC system uses water to help cool the system and can handle up to 20kw, says Dave Brooks, facilities manager at the Globix London center. He expects to reach that level someday. The water connections are welded, and the system poses "no more risk than a standard unit," he says.

Vendors are beginning to turn out products that use water, but approaches vary widely. Last month, IBM released a water-cooled door capable of handling 15kw of heat that can be fitted on its enterprise server rack.

APC also has a unit that can handle about 18kw of heat and is fully enclosed—almost like a refrigerator. The APC unit can be operated outside a data center.

Not all liquid cooling systems use water. Liebert Corp. in Columbus, Ohio, is using a refrigerant in some of its cooling products.

New Designs

One sign that the idea of water cooling is regaining credence is that newer data centers are being designed to potentially handle water. Some are moving wiring to the ceiling and walls and are limiting or eliminating raised floors.

While water cooling units are beginning to appear on trade show floors, Ron Miglini, president of the Dallas/Fort Worth chapter of AFCOM, says the issue of water cooling rarely comes up at meetings of his group, an association for data center professionals. But Miglini, who is also president of Sealco Inc., a company in Richardson, Texas, that cleans data centers and improves air flows, says newer data centers are being built with utility trenches that can isolate water supplies from electrical wiring.

Data center cooling is a top issue for data managers, who are largely addressing cooling needs by carefully laying out data centers. But as server densities increase and equipment is added, users will have to either expand the size of their data centers for new generations of servers that have improved price/performance but consume more power, or they'll have to look at cooling system that use fluids.

Gordon Haff, an analyst at Illuminata Inc. in Nashua, N.H., says IBM's recent entry into the water cooling systems market will add a lot of credibility to the idea of using fluids to pull heat out of the data center. He believes, however, that adoption of water-based systems will remain small and in the category of specialty devices.

Charles King, an analyst at Pund-IT Research in Hayward, Calif., says companies "tend not to want to update their data centers every few years." He views water-cooled systems, such as IBM's, as stopgap measures.

IBM's new rack-cooling device is allowing data center managers to "buy some time and get a few more generations out of their facilities before they think about having to rebuild or retrofit," says King.

Top-Down View of APC’s Water-based Cooling System
Top-Down View of  APC's Water-based Cooling System
As air passes through servers, it absorbs heat.

1. Servers exhaust air at high temperatures.

2. Fans pull air across coils, cooling it in the process.

3. Fans distribute the cooled air to the front of the servers.

Source: American Power Conversion Corp.

Related:

Copyright © 2005 IDG Communications, Inc.

  
Shop Tech Products at Amazon