IT staff acts as wildfire advances on Pepperdine's data center

Solid planning (and firefighters) spare Malibu campus from rapidly moving blaze

The first warning that Timothy Chester, the CIO of Pepperdine University, had of the wildfire that would threaten the Malibu, Calif., campus came when the power went out in his home. It was 5 a.m. Sunday.

Within a matter of hours, brush fires came within 100 feet of the data center -- and there was a point, said Chester, where we had serious concern that the data center itself was going to be jeopardized.

Chester lives on the 830-acre, Pacific Coast campus and heard the backup power systems in his house switching on. Looking out from his hilltop home, it was apparent that the power was out throughout the campus. Chester quickly left for the data center, and as he drove to it, he could see light from the fire on the other side of a ridge.

Chester wasnt the only one on campus who moved into action. Other administrators were responding as well, and by 5:30 a.m., the campus administration had called a meeting of the university's Emergency Operations Committee.

The Malibu wildfire was out of control Sunday morning and remained that way throughout the day; the Los Angeles County Fire Departments Web site Sunday evening described fire at 0% containment, with 1,200 acres already consumed. But in a telephone interview Sunday night, Chester said the immediate threat to the campus had passed.

A wild weekend for the IT crew

On Sunday morning, however, Chester was by no means certain of the fires outcome. IT staff had been paged, and a half-dozen staffers were working to ensure that the data was safe.

Wildfires are an ongoing threat in the area, and the university is prepared for that contingency as well as other threats. It routinely sends its backup tapes to Iron Mountain Inc. for protection. In addition, the latest tape backup copies were moved to a fireproof safe. The ERP applications were shut down, and the hard drives were removed and also safely stored. All that work was completed in 35 minutes, said Chester. It was still before 8 a.m.

The whole purpose of planning is to make sure you've always got options, said Chester, so that when you find yourself in a situation, you are familiar what those options are as opposed to having to think them through with very little response time.

While the IT staff scrambled, the fire advanced toward the data center building and nearby university administration building. We had about 10 minutes' notice that the fire was coming down the hill this way, said Chester.

Firefighters from Los Angeles County and other jurisdictions acted immediately. There were about 25 firefighters in the way of the advancing fire whose entire goal was to protect the buildings, said Chester. They were able to contain those fires and keep them from spreading further, he said.

Pepperdines University Data Center never went offline, ensuring the campus of network services, including voice communications.

Our staff remained in the data center operations area as long as the building fire-/smoke-detection system was not triggered by outside smoke or fire -- and that never happened, said Chester.

Inherited wealth

Among the means used to alert the campus, include a system put in this year by 3n (National Notification Network LLC), a Glendale, Calif.-based system that can use voice and text messaging, including instant messaging, pages and other means, to alert people to an emergency. The system has also been installed at Virginia Tech in Blacksburg, Va., since shooting rampage there earlier this year.

Pepperdine canceled classes Monday, but students who were relocated to other parts of the campus were allowed to return to their dorms by mid-day Sunday according to a notice on the campus Web site.

Chester says he has only been CIO for seven months at Pepperdine, so I get to take no credit for any of this. Instead, he said he has inherited a very good organization [that] has over 100 years of experience managing and responding to incidents such as we saw today.

Our faculty, students, and staff were well served today in this regard by a very experienced team, said Chester.

Copyright © 2007 IDG Communications, Inc.

Shop Tech Products at Amazon