Should you maintain three data centers for disaster recovery?

Computerworld - Pushed in part by U.S. business regulations concerning data preservation, financial and other high-end organizations are moving to a three data center architecture for disaster recovery, says Wikibon.org community member and data center consultant Josh Krischer.

In this architecture, two nearby data centers are linked synchronously with a third, located farther away, linked asynchronously. However, he warned, some data is always lost in a disaster, even when the remote copy is done via a synchronous link. Keeping data losses to a minimum is critical for some applications, but a more important issue is assuring data consistency and integrity at the recovery site. Inconsistent data at the recovery site usually requires time-consuming recovery processes, which may take days.

Speaking at Wikibon.org's weekly Peer Incite teleconference, which is open to all interested parties, Wikibon.org co-founder David Floyer related his experience consulting with one such company that was considering implementing very high-speed continuous asynchronous data transfer from its U.S. to its European data centers to guard against a potential major loss. "The company had two data centers, 15 miles apart, synchronously connected so transactional data is written to both simultaneously," he says. "If one goes down, it can recover from the other, theoretically with very little loss of data."

The proximity of the two centers, determined in part by the distance over which a synchronous link can be maintained, also avoided one of the common errors in disaster recovery planning, putting the recovery site too far from the main data center. "Putting them far apart may make you feel safer," says Floyer, "but it actually makes recovery harder and more expensive and may therefore decreases the plan's effectiveness."

However, he says, this company was concerned about the possibility of a regionwide disaster that might bring down both data centers. The organization was sending a 2TB incremental backup to its European data center twice daily, but in a regional disaster that could result in the loss of up to 20 hours of transactions. It wanted to invest in an advanced network-based system to create an asynchronous link between the U.S. and European data centers to reduce the maximum potential loss to a few minutes. The implementation and operational costs for this upgrade was estimated at about $25 million over three years.

The business leads

This might seem to be an extravagant solution to the problem, and Floyer emphasizes that this isn't the answer for everyone. "I worked with a retailer, for instance, who decided that local backup site was sufficient for their DR needs. If a regional disaster took out both data centers and distribution centers, they expected their business would not survive in any case."