Data center density hits the wall
Why the era of packing more servers into the same space may have to end
Computerworld - Industrial Light & Magic has been replacing its servers with the hottest new IBM BladeCenters -- literally, the hottest.
For every new rack ILM brings in, it cuts overall power use in the data center by a whopping 140 kW -- a staggering 84% drop in overall energy use.
But power density in the new racks is much higher: Each consumes 28 kW of electricity, versus 24 kW for the previous generation. Every watt of power consumed is transformed into heat that must be removed from each rack -- and from the data center.
The new racks are equipped with 84 server blades, each with two quad-core processors and 32GB of RAM. They are powerful enough to displace seven racks of older BladeCenter servers that the special effects company purchased about three years ago for its image-processing farm.
To cool each 42U rack, ILM's air conditioning system must remove more heat than would be produced by nine household ovens running at the highest temperature setting. This is the power density of the new infrastructure that ILM is slowly building out across its raised floor.
These days, most new data centers have been designed to support an average density of 100 to 200 watts per square foot, and the typical cabinet is about 4 kW, says Peter Gross, vice president and general manager of HP Critical Facilities Services. A data center designed for 200 W per square foot can support an average rack density of about 5 kW. With carefully engineered airflow optimizations, a room air conditioning system can support some racks at up to 25 kW, he says.
At 28 kW per rack, ILM is at the upper limit of what can be cooled with today's computer room air conditioning systems, says Roger Schmidt, IBM fellow and chief engineer for data center efficiency. "You're hitting the extreme at 30 kW. It would be a struggle to go a whole lot further," he says.
The sustainability question
The question is, what happens next? "In the future are watts going up so high that clients can't put that box anywhere in their data centers and cope with the power and cooling? We're wrestling with that now," Schmidt says. The future of high-density computing beyond 30 kW will have to rely on water-based cooling, he says. But data center economics may make it cheaper for many organizations to spread out servers rather than concentrate them in racks with ever-higher energy densities, other experts say.
Kevin Clark, director of information technologies at ILM, likes the gains in processing power and energy efficiency he has achieved with the new BladeCenters, which have followed industry trends to deliver more bang for the buck. According to IDC, the average server price since 2004 has dropped 18%, while the cost per core has dropped by 70%, to $715. But Clark wonders whether doubling compute density again, as he has in the past, is sustainable. "If you double the density on our current infrastructure, from a cooling perspective, it's going to be difficult to manage," he says.
- Pennsylvania, a 'Fortune 20' state, consolidates IT
- EPA urges efficiency, many data centers still far from it
- HP creates a new way to sell data centers
- The background on Apple's '103-degree data center'
- Network firewalls aren't dead yet
- Are CIOs losing power?
- As Unix fades away from data centers, it's unclear what's next
- Goldman Sachs, with 10,000 tech workers, embraces open computing
- IT managers are increasingly replacing servers with SaaS
- Microsoft sees huge potential in fuel cells
- Maintain Less. Create More. Spend less on maintenance and spend more time creating with Red Hat Enterprise Linux. Read on to learn how Red Hat can help...
- Mobile First: Securing Information Sprawl Learn how the partnership between Box and MobileIron can help you execute a "mobile first" strategy that manages and secures both mobile apps...
- AIIM Trendscape: The New Mobile Reality This AIIM Trendscape report shares data, expert opinions, and a unique perspective on the impact of cloud and mobility in the enterprise, surfacing...
- Empowering Your Mobile Workers A modern mobile IT strategy is no longer an option, it is an absolute necessity. Here's how some of the nation's most progressive...
- Live Webcast How to serve up a Grand Slam with a scalable IT Infrastructure for cloud, big data and advanced analytics Register today to attend this webcast, and see examples of how The U.S. Tennis Association, Wimbledon and U.S. Golf Association are using the...
- Live Webcast Security Vulnerabilities Associated With Having Local Administrator Privileges Viewfinity will demonstrate how removing admin rights and granularly managing privileges at the application level reduces the attack surface.
- Live Webcast IBM FlashSystem V840: Leveraging Software-Defined Flash to Drive Your Business With end-to-end, tightly integrated functionality and super-fast flash technology, products like IBM FlashSystem V840 Enterprise Performance Solution empower businesses to leverage the efficiency...
- On Demand: Mastering the Art of Mobile Content Management Mobile device usage in the enterprise has skyrocketed, and it continues to escalate. IT must answer to users who demand access to their...
- What should I look for in a Next Generation Firewall? SANS Provides Guidance With so many vendors claiming to have a Next Generation Firewall (NGFW), it can be difficult to tell what makes each one different....