Lessons from Carnegie Mellon's in-house cloud

Network World - Three years ago, Carnegie Mellon University opened the Data Center Observatory –- an answer to the ever-rising operational costs in IT. Administrative expenses were spiraling out of control because individual research groups within the university were running their own IT infrastructure, characterized by short periods of heavy use followed by many hours sitting idle and wasting energy.

The solution was to build an administered utility that provides computational and storage resources to the university community. Besides improving administrative efficiency, the DCO helped control power and cooling costs while letting researchers focus on what they do best rather than worry about maintaining their own mini data centers.

"We didn't have the name cloud computing [at the time], but as it turns out, that's exactly what I was pitching to the university," says Greg Ganger, a professor of electric and computer engineering and director of Carnegie Mellon's Parallel Data Lab, a storage systems research center.

So far, the DCO houses 325 computers connected to 12 network switches, 38 power distributors and 12 remote console servers. More than 1,000 cables and 530TB of storage are in use, while environmental conditions are monitored by 13 sensor nodes. Most equipment is donated by vendors or bought with grants.

Two thousand square feet in size, the DCO is being built in zones, with two out of four zones online at this time.

The DCO gets the "observatory" part of its name because it was designed not only to provide real data center resources but also to serve as a testbed for systems researchers looking to "understand the sources of operational costs and to evaluate novel solutions," according to Carnegie Mellon. A windowed wall with a view of an LCD display showing electrical usage and other statistics gives people walking by a sense of what's happening inside the Data Center Observatory.

Building the DCO was not without its challenges, however. Besides "playing Tetris with the room" to figure out how best to place equipment, Ganger found that convincing researchers to share was not always easy.

"We learned how hard it is to get people in the same space," says Ganger, who described the project at a recent event hosted by Schneider Electric and in an interview with Network World. "Each group had its own operating system that they had to have, and their own set of libraries and unique setups. Early on it was clear we had to use virtual machines."

Rather than use the expensive VMware virtualization tools, Ganger opted for the open-source Xen and KVM platforms. About a third of DCO machines have been virtualized, making it easier to increase and decrease resources provisioned to each research group. Overall, virtualization has been very useful but raised some interesting concerns, he says.