Faulty McAfee update burns IT execs

Computerworld - McAfee Inc. moved swiftly to make amends to corporate and individual customers whose PCs were crippled late last month by a faulty antivirus update that it distributed.

Less than a week after the security vendor had pushed the flawed update to users, it offered affected business customers a free one-year subscription to its automated security assessment service, and reimbursement to consumers for any "reasonable expenses" related to the incident.

The faulty update, released on April 21, had corporate IT administrators scrambling when the new signatures quarantined a critical Windows system file, causing some computers running Windows XP Service Pack 3 to crash and reboot repeatedly.

McAfee said later that a small fraction of its corporate customers -- less than 0.5% -- were affected by the glitch. But those that were faced a time-consuming repair process. Virtually all of the affected PCs were unable to connect to a network, so corporate support personnel had to manually fix each machine impaired by the faulty update.

An Intel Corp. spokesman said an unknown number of the chip maker's systems were knocked offline by the bad update. He said the resulting problems had a "significant" impact on the company.

"There were quite a few clients, laptops and PCs [affected]," the spokesman said. "We were able to get it stopped fairly early on, but clearly not soon enough."

About 40% of machines used by the government of Washington's Snohomish County were affected by the problem, according to John Storbeck, the county's engineering services supervisor. In an e-mail, he called the incident "a nightmare."

In Iowa, a disaster response exercise was disrupted when the update caused 9-1-1 computer systems to crash, said Deb Hale, a security administrator at Long Lines, an Internet service provider in Sioux City. "Thanks to McAfee, we were forced to test our response to a disaster while in the midst of a real 'disaster,' " she wrote in a blog post on the SANS Institute's Internet Storm Center site.

"This is the worst glitch that I've ever had to deal with," said Ken Whittaker, a desktop support technician at a Michigan university where some 10,000 desktops were affected by the defect. He asked that the school not be identified.

It's not unheard of for antivirus vendors to mistakenly impair software with their updates. Criminals have become so good at switching up their code that companies like McAfee must now churn out millions of signatures in a cat-and-mouse game to identify malware that is in circulation. That leads to errors.

Still, the fact that McAfee allowed a major Windows component to be misidentified demonstrates "a failure in their quality control process," said Amrit Williams, chief technology officer at systems management software vendor BigFix Inc.

"You're not talking about some obscure file from a random third party; you're talking about a critical Windows file," said Williams, a former director of engineering at McAfee. "The fact that it wasn't found is extremely troubling."

McMillan is a reporter for the IDG News Service.

Read more about Security in Computerworld's Security Topic Center.