Microsoft Blames WGA Meltdown on Human Error

Computerworld - Microsoft Corp. last week blamed human error on the part of its IT staff for a server problem that caused the companys Windows Genuine Advantage (WGA) validation service to incorrectly tag legitimate users of Windows XP and Windows Vista as software pirates.

The software vendor also promised that internal changes are being made to avoid a repeat of the glitch, which affected users for nearly 20 hours on Aug. 24 and 25. Users whose copies of Windows erroneously failed WGAs antipiracy tests were prevented from downloading most software from Microsofts Web site. And those with Vista were unable to use some of the operating systems features.

Alex Kochis, Microsofts senior WGA product manager, wrote in a blog posting that the troubles began after preproduction code was installed on live servers.

Those systems had yet to be upgraded with another code change designed to enable stronger encryption and decryption of product keys, Kochis added. As a result, the production servers declined activation and validation requests that should have passed, he wrote.

A quick code rollback fixed the problem on the product-activation servers within 30 minutes, according to Kochis. But it didnt reset the validation servers, which handle legitimacy checks on downloads and other transactions.

We now realize that we didnt have the right monitoring in place to be sure the fixes had the intended effect, Kochis wrote. He also said that Microsoft is taking steps such as increasing the speed of escalations and adding checkpoints before changes can be made to production servers.

Earlier last week, Microsoft said that fewer than 12,000 systems were affected worldwide. But users lit up the companys support forums with more than 450 messages about the snafu.

A system thats not totally reliable really should not be so punitive, said Gartner Inc. analyst Michael Silver.

Michael Cherry, an analyst at Directions on Microsoft in Kirkland, Wash., said he was surprised that it was even possible to accidentally load the wrong code onto live servers. It just begs the question of, what other things have they not done? Cherry said.