Northeast, Canada power failure exposes infrastructure frailty

Computerworld - Federal and state emergency officials are scrambling to determine the source of a major power outage that rippled through the Northeast from New York to Canada and as far west as Detroit.
While it's too early to tell exactly what caused the cascading failure, which hit about 4 p.m. EDT yesterday, national security experts said it was a prime example of how fragile the nation's critical infrastructures are to both self-inflicted disruptions and deliberate sabotage.
There were no quick determinations of the source of the failure. New York City Mayor Michael Bloomberg told reporters late this afternoon that there was no evidence that the failures were related to terrorism.
There were reports, however, of a fire at a Consolidated Edison power facility in New York -- although Bloomberg dismissed those in a news conference about 6 p.m. Others speculated that the failure could be related to the ongoing outbreak of the Blaster worm, which has disrupted hundreds of thousands of computers worldwide since Aug. 11. The CERT Coordination Center this afternoon said it has seen no evidence that Blaster was involved in the blackout.
"The big question is what caused it," said Roger Cressey, former chief of staff of the President's Critical Infrastructure Protection Board. "If the fire in the Con Edison facility led to this, then it's a wonderfully graphic example of how fragile and interdependent our critical infrastructure is," he said.
Paula Scalingi, the former director of critical-infrastructure protection at the U.S. Department of Energy, said the failure proves the nation hasn't prepared for these types of events beyond where it was a few years ago, and said it will likely be a while before the exact cause is known.
"What customarily happens is that they do an engineering study and that may take them a while," said Scalingi. "What we do know, however, is the way that the grid is set up and configured means that certain failures will have cascading impacts. It's nothing new."
And while research in more adaptive systems is ongoing, Scalingi said the effort has suffered from lack of focus and urgency.
Joe Weiss, an analyst at Kema Consulting in Fairfax, Va., and the former technical manager of the Enterprise Infrastructure Security Program at the Electric Power Research Institute in Palo Alto, Calif., said the power grid is frail from lack of capacity and because it is highly interconnected, making such widespread cascading failures possible.
"The fail-over systems are there, but the grid is comprised of a lot of systems that will automatically trip [off] during a failure or disruption," said Weiss. "We've assumed we have enough cushion in the system so that you can lose a number of power plants and substations and still have enough transmission capability to handle the load. But the truth is you may or may not have that cushion."
Weiss also acknowledged that much of the research and development work for more resilient IT systems for the electric power grid -- called for last year by the National Research Council (NRC) -- hasn't evolved to the extent officials would like.
In that report, "The Role of Science and Technology in Countering Terrorism," the NRC warned that a regional transmission grid failure could occur if damage or destruction to important parts of the grid were followed by a cascading failure of interconnected components.
The report recommended that the U.S. Department of Energy work with the private sector to develop "intelligent and adaptive" electric-power grid systems.

"Such an intelligent grid would provide the system with the ability to fail gracefully, minimizing damage to components and enabling more rapid recovery of power," the report stated. "A key element would be adaptive islanding, a concept employing fast-acting sensors and controls to isolate parts of the power system. Operations models and intelligence would be needed to differentiate between failure of a single component and the kind of concurrent or closely coupled serial failures, at several key nodes, that could indicate the onset of a concerted attack," according to the report.
In an interview last year shortly after the NRC released its report, Ed Badolato, the former deputy assistant secretary for energy emergencies at the Energy Department, said the government's intelligent-grid initiatives should be sped up to allow for a better understanding of when the nation is being attacked and allow operators to prevent catastrophic system failure.
Private-sector cybersecurity experts have for years also warned of the vulnerabilities posed by the energy industry's deliberate efforts to connect Supervisory Control and Data Acquisition (SCADA) systems -- the real-time computers used to manage grid capacity and flow -- to corporate LANs as a way of improving statistical tracking and sales of excess grid capacity.
In a white paper prepared by Alexandria, Va.-based Riptech Inc. in January 2001, the company detailed how the power industry's demand for remote access has encouraged many utility firms to establish connections to SCADA systems. "The security strategy for utility corporate network infrastructures rarely accounts for the fact that access to these systems might allow unauthorized access and control of SCADA systems," the white paper concluded.