IT staff acts as wildfire advances on Pepperdine's data center

Computerworld - The first warning that Timothy Chester, the CIO of Pepperdine University, had of the wildfire that would threaten the Malibu, Calif., campus came when the power went out in his home. It was 5 a.m. Sunday.

Within a matter of hours, brush fires came within 100 feet of the data center -- and there was a point, said Chester, where we had serious concern that the data center itself was going to be jeopardized.

Chester lives on the 830-acre, Pacific Coast campus and heard the backup power systems in his house switching on. Looking out from his hilltop home, it was apparent that the power was out throughout the campus. Chester quickly left for the data center, and as he drove to it, he could see light from the fire on the other side of a ridge.

Chester wasnt the only one on campus who moved into action. Other administrators were responding as well, and by 5:30 a.m., the campus administration had called a meeting of the university's Emergency Operations Committee.

The Malibu wildfire was out of control Sunday morning and remained that way throughout the day; the Los Angeles County Fire Departments Web site Sunday evening described fire at 0% containment, with 1,200 acres already consumed. But in a telephone interview Sunday night, Chester said the immediate threat to the campus had passed.

A wild weekend for the IT crew

On Sunday morning, however, Chester was by no means certain of the fires outcome. IT staff had been paged, and a half-dozen staffers were working to ensure that the data was safe.

Wildfires are an ongoing threat in the area, and the university is prepared for that contingency as well as other threats. It routinely sends its backup tapes to Iron Mountain Inc. for protection. In addition, the latest tape backup copies were moved to a fireproof safe. The ERP applications were shut down, and the hard drives were removed and also safely stored. All that work was completed in 35 minutes, said Chester. It was still before 8 a.m.

The whole purpose of planning is to make sure you've always got options, said Chester, so that when you find yourself in a situation, you are familiar what those options are as opposed to having to think them through with very little response time.

While the IT staff scrambled, the fire advanced toward the data center building and nearby university administration building. We had about 10 minutes' notice that the fire was coming down the hill this way, said Chester.