A tale of two U.S. government data center projects
One project, for NOAA, came in on time; the other, for the Air Force, didn't
Computerworld - ORLANDO -- This is a story of two federal government data center projects: One, undertaken by the National Oceanic and Atmospheric Administration (NOAA), met its schedule and budget. The other, a U.S. Air Force initiative, went over budget and was late.
How did this happen? What was different about the projects and the approaches that led to different outcomes?
The NOAA project
The NOAA project was a $27.6 million renovation of an approximately five-year-old office and laboratory building in an office park in Fairmount, W.Va. The building was newly leased by the government. The goal of the renovation was to upgrade the capabilities of the computer systems that support NOAA research and to consolidate several research data centers. The agency picked West Virginia because it was looking for locations in a 120-mile radius away from an existing data center.
The entire facility is 54,000 square feet, with 16,000 square feet of raised floor space.
The data center, which opens next month, will house a 29,000-core Intel Xeon supercomputer built by SGI that's capable of reaching speeds of up to 383 teraflops.
The facility wasn't originally intended to house a data center, but NOAA estimates that building a new data center from scratch would have cost approximately double the amount budgeted for the project.
The Air Force project
The Air Force Research Laboratory's Defense Supercomputing Center needed to upgrade mechanical and electrical systems in a 40-year-old data center to accommodate a new supercomputer. Among other things, this nearly 83,000-square-foot building needed an increase in its electric load capacity from 3.3 megawatts to 8MW. It also needed new water-cooled chillers capable of handling a Cray supercomputer with 30, 45kW racks. It has 26,600 square feet of raised floor space.
The project was expected to be completed in June 2010 at a cost of $5.1 million. It wasn't finished until this month, 14 months late, and it was about $1 million over budget.
What went wrong?
The Air Force project's problems, outlined by the project managers during a presentation last week at the Afcom data center conference here, were wide-ranging -- so much so that one person in the audience asked if the project was cursed.
For instance, even though the site was surveyed twice in an effort to discover all buried utility conduits, a TV cable went undetected. And during construction, a previously unknown spring was discovered, causing delays. The list goes on: Three breakers kept failing because of moisture that poor sealing couldn't keep out; the mechanical room needed more work than originally expected to support new equipment; and a worker fell off a ladder and broke both wrists.
Some of those problems could have been addressed quickly, but in many cases they weren't. That's because the Air Force used a design-bid-build contract for the project, meaning it had separate contractors for design and construction, instead of a design-build contract, where one contractor is responsible for both.
In a previous $5 million facility upgrade, the Air Force had used a design-build contract through the Army Corps of Engineers. While that project was a success, the Army Corps of Engineers charged an 8% premium to manage it. This time around, the Air Force wanted to cut costs. It saw the design-bid-build route as a less-expensive option.
One throat to choke
But what the Air Force lost by using a design-bid-build contract was the proverbial "one throat to choke." As problems arose, new requests for information were issued, and multiple teams had to negotiate fixes -- adding months of delays. The Air Force had to work with contracting officers, financial managers, engineers, project managers and others, all of whom were being pulled by other priorities.
"It can be a very tedious process," said Brian Schafer, the infrastructure management chief for the Air Force's supercomputing center. "You can introduce delays that you really didn't anticipate."
With a design-build, there's a single point of contact, someone who knows the project and "can do that change quickly," said Schafer. "I think that was our main issue with this project."
The NOAA project: What went right?
While the Air Force was dealing with an aging facility and some old mechanical systems that presented some unique challenges, NOAA had a newer property to work with in an industrial park designated for high-tech tenants.
Unlike the Air Force, NOAA opted for a design-build contract, with the contractor acting as designer. NOAA had a conceptual design in advance, which it then used in its solicitation.
Darren Smith, the project manager who works in the office of NOAA's CIO, said the U.S. General Services Administration, which manages federal properties, favored the design-build option because it had a very tight budget. "It meant that we got a fixed, firm price for the whole thing," he said.
One drawback, said Smith, is that NOAA couldn't be very specific about the type of equipment used and where it is located. But the NOAA project was completed in a year, on schedule and within its budget.
Patrick Thibodeau covers cloud computing and enterprise applications, outsourcing, government IT policies, data centers and IT workforce issues for Computerworld. Follow Patrick on Twitter at @DCgov, or subscribe to Patrick's RSS feed . His email address is email@example.com.
- Network firewalls aren't dead yet
- Are CIOs losing power?
- As Unix fades away from data centers, it's unclear what's next
- Goldman Sachs, with 10,000 tech workers, embraces open computing
- IT managers are increasingly replacing servers with SaaS
- Microsoft sees huge potential in fuel cells
- Snap, crackle, pop: Electrical troubles shed light on NSA's data center
- Tornadoes and data centers are OK in Oklahoma
- It's twilight for small in-house data centers
- GM CIO calls HP hiring probe a 'fishing expedition'
Read more about Data Center in Computerworld's Data Center Topic Center.
- 15 Non-Certified IT Skills Growing in Demand
- How 19 Tech Titans Target Healthcare
- Twitter Suffering From Growing Pains (and Facebook Comparisons)
- Agile Comes to Data Integration
- Slideshow: 7 security mistakes people make with their mobile device
- iOS vs. Android: Which is more secure?
- 11 sure signs you've been hacked
- HP HAVEn: See the big picture in Big Data HP HAVEn is the industry's first comprehensive, scalable, open, and secure platform for Big Data. Enterprises are drowning in a sea of data...
- Piecing Together the Business Intelligence Puzzle Business intelligence (BI) technology collects and analyzes company data, delivering relevant information to corporate decision-makers in an effort to produce favorable outcomes.
- Harness IT -- An Introduction to Business Intelligence Solutions Learn the key selection criteria required to provide your organization with the capability to address structured data, unstructured data and mobile demands so...
- Business Intelligence Shows its Smarts Today's Business Intelligence (BI) tools provide a new way to think about data with self-service capabilities and user-friendly analytics that can be used...
- Cloud Knowledge Vault Learn how your organization can benefit from the scalability, flexibility, and performance that the cloud offers through the short videos and other resources...
- Testimonial: Cystic Fibrosis Trust Peter Hawkins, the Head of IT for Cystic Fibrosis Trust, discusses the role CommVault's Simpana software platform plays in improving the company's information... All Data Center White Papers | Webcasts