Microsoft blames WGA meltdown on human error
It wasn't a server outage or blackout after all, says company
Computerworld - Microsoft Corp. said late yesterday that last weekend's failure of the antipiracy process it requires of Windows XP and Vista was due to "human error" and shouldn't be called an "outage" since the servers didn't go off-line. The company also promised that changes have been made to avoid a repeat.
In an earlier statement, Microsoft had downplayed the scope of the problem, saying that fewer than 12,000 systems worldwide had been affected.
In a post to the Windows Genuine Advantage (WGA) blog, program manager Alex Kochis, normally the public voice for the team, explained the malfunction of the company's validation servers in the greatest detail so far.
"Nothing more than human error started it all," said Kochis. "Preproduction code was sent to production servers. The production servers had not yet been upgraded with a recent change to enable stronger encryption/decryption of product keys during the activation and validation processes. The result of this is that the production servers declined activation and validation requests that should have passed."
Microsoft's anticounterfeit measures come in two flavors: activation and validation. The former requires users to enter a valid 25-character product key to prove they've paid for a license; the latter is the term used for all subsequent proof-of-purchase demands and engages, for instance, before users are allowed to download most software from the company's Web site.
The problem affected both the activation and validation servers, but while a quick rollback solved the activation servers' problems within 30 minutes, according to Kochis, it failed to reset the validation servers. "We now realize that we didn't have the right monitoring in place to be sure the fixes had the intended effect," he said.
From the timeline he offered up in earlier postings, the failure started on Friday, Aug. 24 about 6:30 p.m. EDT. It's unclear how long Microsoft was unaware of the problem, although it was presumably measured in hours rather than minutes. "Through a combination of posts to our forum and customer support, the issue was discovered by [Friday] evening," Kochis said. By 2:15 p.m. Saturday, Aug. 25, the servers were again validating Windows correctly. The total time of the malfunction: 19 hours, 45 minutes.
Kochis also took pains to set the record straight about how the problem had been characterized. "It's important to clarify that this event was not an outage," he said. "This event was not the same as an outage because in this case the trusted source of validations itself responded incorrectly."
Contrary to what most users believed when they lit up Microsoft's support forums Friday night and Saturday, Kochis said PCs running XP or Vista automatically default to "genuine" -- the term Microsoft applies to machines running legitimate copies of its operating systems -- if Microsoft's validation servers are off-line. "In other words, we designed WGA to give the benefit of the doubt to our customers. If our servers are down, your system will pass validation every time."
- Path Selection Infographic Path Selection Infographic
- Hyperconvergence Infographic A wide range of observers agree that data centers are now entering an era of "hyperconvergence" that will raise network traffic levels faster...
- Preparing Your Infrastructure for the Hyperconvergence Era From cloud computing and virtualization to mobility and unified communications, an array of innovative technologies is transforming today's data centers.
- How WAN Optimization Helps Enterprises Reduce Costs If you wanted to break down innovation into a tidy equation, it might go something like this: Technology + Connectivity = Productivity. Productivity...
- Cloud Knowledge Vault Learn how your organization can benefit from the scalability, flexibility, and performance that the cloud offers through the short videos and other resources...
- LIVE EVENT: 5/7, The End of Data Protection As We Know It. Introducing a Next Generation Data Protection Architecture. Traditional backup is going away, but where does this leave end-users? All Windows White Papers | Webcasts