What started as a normal weekend morning for Internet users turned into a paranoid panic when the Google search engine started reporting that every site - and we mean every site - was contaminated with malware.
To quote my friend Bob, who woke me up this morning at 7 AM with the news, "What the hell?"
It took me a few minutes to determine that while all Google searches were indeed reporting that every site "may harm your computer," this was not really the case. Google was reporting one of my Web sites was potentially filled with malware... and I had taken it down for maintenance the day before. Without any connection to the Internet I knew that server wasn't infected with anything.
After kicking around the Web for an hour or so, I was absolutely sure that it was Google, not the Web, that was in trouble. The logical place where this problem would have erupted was in a Google internal program that marks sites as bad when Google's bots and/or staffers determine that a site is trying to install malicious programs.
At first, Google stated that the problem came from its use of the StopBadware list of bad sites. StopBadware, which is partnership of academic institutions, technology industry leaders, and volunteers that generates its own list of bad sites, denied this. In an official StopBadware blog posting, the group stated that. "Google generates its own list of badware URLs, and no data that we generate is supposed to affect the warnings in Google's search listings."
Google has yet to correct this in its own public statements. Google engineers, however, tell me that StopBadware is correct and that Google generates its own list of suspicious Web sites.
So what did happen? According to a statement by Google's Marissa Mayer, VP of Search Products & User Experience, "Very simply, human error. Google flags search results with the message 'This site may harm your computer' if the site is known to install malicious software in the background or otherwise surreptitiously. We do this to protect our users against visiting sites that could harm their computers."
Specifically, "We periodically receive updates to that list and received one such update to release on the site this morning. Unfortunately (and here's the human error), the URL of '/' was mistakenly checked in as a value to the file and '/' expands to all URLs."
In other words, a single point of failure lead to Google search being rendered useless this morning. Mayer went on to write, "Fortunately, our on-call site reliability team found the problem quickly and reverted the file. Since we push these updates in a staggered and rolling fashion, the errors began appearing between 6:27 a.m. and 6:40 a.m. and began disappearing between 7:10 and 7:25 a.m., so the duration of the problem for any particular user was approximately 40 minutes."
With this correction, Google search should now be working normally for everyone. If you run into a search result that states that "This site may harm your computer," you can be reasonably sure that it is really a malicious site and you should avoid it.