Supercomputer center turns to 'data center in a box' for extra capacity

When the Stanford Linear Accelerator Center (SLAC) wanted to increase its computing capacity last year, it considered a satellite data center or an extension to the existing building, but in the end the center chose a faster and more novel approach: It ordered a data center in a box.

The center, which does high-energy physics research for the U.S. Department of Energy, was one of the first customers for Sun Microsystems Inc.'s Project Blackbox, which takes standard shipping containers measuring 20 by 8 by 8 feet (6.1 by 2.4 by 2.4 meters) and turns them into mini data centers that can be delivered and operational in a few weeks, according to Sun.

SLAC's Blackbox was delivered in July last year and was up and running by September. Aside from a few challenges -- like figuring out how to service the unit when it's raining -- the center is pleased with its choice and is in the process of installing a second unit.

It is one of four customers that Sun identified Tuesday to illustrate momentum for Project Blackbox. It also renamed the product the Sun Modular Datacenter, or Sun MD. The boxes start at $559,000 without the "payload," or the computers inside, and Sun will ship them around the world for an extra fee. Delivery to Amsterdam by air, for example, costs about $15,000.

SLAC turned to Project Blackbox because it needed to expand its compute capacity fairly quickly. "The data center here was at its capacity, especially in terms of the electrical service to the building and the amount of heat we could take back out. And the experiments needed their next year's worth of computing," said Chuck Boeheim, SLAC's assistant director of computing services.

Modifying its existing data center to accommodate another major electrical feed and "chiller plant" would have taken one to two years and cost several million dollars. "We also looked at doing a satellite data center in a smaller building, but with the approvals and lead time, that was a couple of years out as well. Blackbox was something we could do very quickly," Boeheim said.

The raw shipping containers are customized by a subcontractor, and Sun typically installs the payload before delivery. SLAC's Blackbox arrived on a flatbed truck last July, fitted with 252 Sun Fire X2200 rack-mounted servers, the same type it uses in its data center. Sun had also wired the servers to a Cisco Catalyst 6509 switch that SLAC provided before delivery.

Boeheim described the process in a white paper on his Web site, along with photos and time-lapse videos that show the box being hoisted into place by a crane.

Customers can put other vendors' hardware in the unit, but not all equipment will fit. SLAC bought some Dell PowerEdge 1950 servers that it wanted to put in its second Blackbox, but they were too long for the server racks. Instead, it will move some existing Sun servers from its data center to the box and put the Dell Inc. machines inside the building.

The computing center already had a 4-megawatt substation serving its area of the site. "So it was a fairly simple matter to put a 220-kilowatt power pane on the outside of that and bring up a fairly standard chiller unit and put that on a concrete pad next to the Blackbox. Both of those things could just be forklifted in and connected up," Boeheim said.

SLAC wanted to do a rigorous installation that would last several years, so it spent a few weeks getting the unit hooked up. It had to reconnect about half the power cords, which had shaken loose during the truck ride. Aside from that, the installation went smoothly, and the machines were turned on for batch work Sept. 25, about two months after delivery. The 60-ton chiller and the outside power unit were also delivered in that time.

Servicing the unit has presented some challenges. "We're trying to figure out how to do it on a day like today when it's pouring with rain," Boeheim said recently. "The wind doesn't have to be blowing very hard for the rain to blow in the end, and you pretty much have to open both ends of the box to service it."

It's quite cramped in the box, he said, and both ends need to be open so that a person can maneuver when a server rack is slid out into the narrow central aisle.

"Our stance right now is to wait for a clear day to open it up," he said. Two people are needed to service the box in case one gets trapped inside or injured, he wrote in the white paper on his Web site. SLAC put a phone in the box in case someone gets locked in.

The first Blackbox gave the center a roughly 25% boost in computing capacity, and the servers have been running as smoothly as their counterparts inside the data center, Boeheim said. "Once we got them turned on, they just ran. We went a full 30 days not even opening the thing." He put rack servers in the box rather than storage gear, because storage gear tends to need servicing more often, he said.

Sun's Blackbox isn't for everyone. Brookhaven National Laboratory in New York, considered a Blackbox but decided on a new wing for its data center instead, Boeheim said. SLAC's experience may have been easier than that of an average business because it is a federal government site located on county land. A business in a city may need more planning permissions and permits.

A few other companies offer comparable products, such as Ice Cube from Rackable Systems Inc. and Infrastructure Express from American Power Conversion Corp.

Sun said it is happy with the momentum for Blackbox, although it wouldn't say how many customers it has. The other three it named are Hansen Transmissions, a Belgian industrial manufacturer that is using one at a new plant in India; Mobile TeleSystems OJSC, Russia's largest mobile operator; and the Radboud University Nijmegen Medical Centre in the Netherlands.

Most of the customers had no room to add capacity in their data centers, said Darlene Yaplee, vice president for integrated platforms at Sun's systems marketing group. "Half the customers are using it while they build extra data center capacity, and then it will remain for disaster recovery," she said.

The next size up in standard shipping containers is 40 feet long. "We could do 40 feet technically, but most people have been happy with 20 feet," Yaplee said. Boeheim said he'd probably choose a wider box rather than a longer one, which would accommodate his Dell 1950 servers.

Copyright © 2008 IDG Communications, Inc.

7 inconvenient truths about the hybrid work trend
Shop Tech Products at Amazon