Flirting with disaster

Computerworld - What are the odds that your organization can recover to its stated recovery point objective and recovery time objective?

Disaster recovery, by its nature, is a gamble -- you can never cover every contingency that might occur. Recent accounts of Hurricane Katrina recovery efforts refer to the vital need for "aggressive improvisation" to return to business operations.

Despite this unpredictability, many organizations needlessly increase their exposure. One way that they do this is through self-delusion. By this, I mean that their recovery expectations bear no semblance to their actual ability to recover. This is typically the result of either an outdated DR infrastructure or just a lack of serious attention to DR processes.

Another area of exposure is created by not understanding application interdependencies. Today's mission-critical applications consist of components that often reside on a multitude of servers, as well as numerous cross-application data dependencies that must be considered for successful recovery. These dependencies frequently span the boundaries between the mainframe and open-systems worlds, often an organizational chasm. Without a solid understanding of how data flows among these applications, it is very possible to have a situation in which each server is individually recoverable, yet, due to lack of appropriate synchronization, the application can't be recovered properly. This scenario inevitably adds many hours or days to the recovery time.

Problems like this can be uncovered through a solid DR testing process, another area that often does not receive appropriate attention. Successful replication and server recovery are important components of DR testing, but unless you are also testing and validating application recovery, your process is incomplete. We have seen situations where a critical ERP application, for example, took days to recover after adopting a more comprehensive, application-focused testing approach. Major underlying flaws in the data protection process were uncovered by these tests.

Finding problems like this are a good thing. It is the reason we test, and why testing should be viewed positively even when failures occur. We may not be able to plan for every DR contingency, but it is certainly true that each problem we can anticipate and address through the planning process is one less that we will need to solve through "aggressive improvisation" should the need ever arise.

Jim Damoulakis is chief technology officer at GlassHouse Technologies Inc., a leading provider of independent storage services. He can be reached at jimd@glasshouse.com.