IT staffers at Tulane University, the largest private employer in New Orleans, were working last week to get the school's financial systems operating again after the floodwaters spurred by Hurricane Katrina surrounded its data center near the Superdome.
John Lawson, Tulane's CIO, found himself as displaced as the university's students and faculty. Lawson is staying in a Houston hotel near a temporary administrative operations center that Tulane has set up. From there, he's managing the effort to restore critical IT services and replace some of the processing capabilities previously handled by the systems in New Orleans.
But while Lawson and other IT managers are dealing with personal hardships and working to reassemble their IT operations, they're also assessing what went right and what went wrong with their disaster recovery plans. And they're starting to consider some big IT issues that will have to be decided in the months ahead, such as whether to keep their data centers in New Orleans once it's inhabitable again.
Located on the 43rd floor of a downtown office building, the data center of Adams and Reese LLP, a New Orleans law firm, was safely above the floodwaters, said CIO David Erwin.
But whether Adams and Reese's data center will remain in New Orleans "is to be debated," Erwin said. "If the [Army] Corps of Engineers doesn't come up with a better levee system, probably not."
Tulane's data center is on the 14th floor of a high-rise building and is also intact and dry, though without power, Lawson said. But he doesn't know when the data center will be operational again.
"That's really dependent on the city," he said, indicating that the location of the data center is one of the long-term issues he will have to evaluate.
Since the storm, Tulane has hired SunGard Data Systems Inc.'s outsourcing service to load data from the school's student information systems and payroll application from backup tapes onto a mainframe at a SunGard facility in Voorhees, N.J. Lawson hoped to have those systems up and running again by Friday.
Tulane's IT staff began backing up data before the hurricane struck on Aug. 29. But when workers came to move the tapes to an off-site location as Katrina began its assault on the Gulf Coast, the building where the data center is located was closed and locked down. Staffers had to return to the dark and hot building in the days following the hurricane to get the tapes. As a result of that experience, Lawson said he plans to look into doing online backups.
The hurricane's aftermath also revealed some weaknesses in disaster recovery planning at Adams and Reese. For instance, the law firm had stored backup tapes at an off-site location, but that site was also in New Orleans and couldn't be accessed by the firm's IT staffers. "That's a lesson learned," Erwin said.
However, Erwin had started online backups in advance of the storm, and he said his data is safe on servers in satellite offices. In addition, Adams and Reese used MessageOne Inc.'s messaging fail-over system to provide continued e-mail access to its employees.
 |  |
Pumps began removing water from New Orleans last week. Image Credit: The Associated Press | |
Telecommunications and Internet access issues have risen to the top of the priority list for many IT managers in the Gulf Coast region.
Malcolm Martin, support services administrator and comptroller at Montgomery, Barnett, Brown, Read, Hammond & Mintz LLP, said the New Orleans law firm's office was decimated by Katrina, forcing him to rent space in an empty office building in the state capital of Baton Rouge.
One change Martin said he will make is to set up a toll-free telephone number outside of the South to give employees a central way to communicate with one another.
"That would have saved us two days' time" that was spent contacting employees, Martin said over a newly installed satellite phone connection.
Martin contracted with a disaster recovery company just three months ago - a move that he's now especially glad he made. As part of the deal, Charlotte, N.C.-based Agility Recovery Solutions Inc. installed the satellite phones plus the servers, workstations and software needed to replicate the law firm's front- and back-office systems.
In order to meet the demand it's seeing for temporary facilities, Louisiana Technology Park LLC, a Baton Rouge-based IT business incubator, is increasing the amount of work spaces in its facility.
Doug Lee, the technology park's executive director, said many of the IT managers he has spoken with never believed the area would experience the devastation it did.
"They left dogs tied up on the porches and their servers not terribly high and dry," Lee said. "I even had calls from companies that said, 'All I got is a backup tape and the shirt on my back -- what can we do?' "