A daily digest of IT news, curated from blogs, forums and news sites around the web each morning. We highlight the key commentary and demystify the real story.
Google Apps and Gmail outage outrage: Problem NOT resolved
Update: Google's Gmail fail is still causing users anguish, as many as 35,000 consumer Gmail and Google Apps users have been affected since Sunday. Google has had to resort to restoring from tape, which must hurt a bit. The company blames an update to its software for losing the replicated cloud stores.
Google says the problem causing an outage in Gmail and Google Apps has finally been resolved is still in-play for some users. It blamed an errant software update for nuking all online copies of a few users' data, causing it to go back to the backup tapes. In IT Blogwatch, bloggers try to visualize 0.02% of a user base.
Your humble blogwatcher curated these bloggy bits for your entertainment. Not to mention Celebrity facial Photoshops are the stuff of nightmares...
(GOOG)
Sha-ron Gau-din re-ports:
Tens of thousands of Google's Gmail users woke up Sunday morning to missing e-mails.
...
The company's estimate of the number of users affected by the outage has dropped from Sunday's figure of 0.08% to 0.02%. That brings the number of affected users down to about 35,000. ... Google engineers have restored service to about a third of those affected and hope to have everyone else up and running [by now].
...
Google first acknowledged the problem on its Apps Status Dashboard Sunday at 3:09 p.m. EST. ... This kind of outage could very well affect Google and its efforts to work its way into the enterprise market.
Ja-red New-man em-bell-ish-es thus-wise:
[It's] a reminder of how Web storage isn't completely immune to failure. If you rely on Gmail ... consider backing up your account.
...
The lost e-mails are reminiscent of a Hotmail server problem that ... deleted the e-mails of more than 17,000 users in December. Microsoft was able to restore access a few days later.
[Nitpicker's corner: nobody puts a hyphen in "email" any longer. Also, "email" is a mass noun, not a count noun, just like "mail" -- would you write "The mail carrier burned my mails?"]
Google's Ben Treynor is contrite:
Were very sorry. The good news is that email was never lost and weve restored access for many. ... things should be back to normal for everyone soon.
...
We have multiple copies of your data, in multiple data centers ... [but] in some rare instances software bugs can affect several copies of the data ... [so] we also back it up to tape.
...
Well post a detailed incident report outlining what happened to the Apps Status Dashboard, as well as the corrective actions were taking to help prevent it from occurring again. ... Thanks for bearing with us as we fix this, and sorry again for the scare.
But Tim Acheson isn't satisfied by that reply:
It's not true that "no email was lost"! This is a falsehood. Email sent while accounts were down bounced back! ... I would thank Google for not trying to make it sound like we have not ... lost emails.
...
Google makes no guarantees. I have switched to Hotmail where my data is safe. ... We're not really Google's customers. Google is in the business of selling advertising.
The Google Apps Status Dashboard later confirmed everything was A-OK:
5:57 PM [EST] The problem with Google Mail should be resolved. We apologize for the inconvenience and thank you for your patience.
...
System reliability is a top priority at Google, and we are making continuous improvements to make our systems better.
However, the Google Apps Status Dashboard says it's still broken:
10:58 PM [EST] Google Mail service has already been restored for some users, and we expect a resolution for all users in the near future. Please note this time frame is an estimate and may change.
Meanwhile, Zoli Erdos wishes people would learn from a "fiasco" like this:
Lets stop and think here a minute. Is your lifetimes worth of information sitting in email? Do you have redundant backups? ... There are a few steps anyone can take easily to have redundant copies. ...
Keep an offline copy in an email client. ...
Auto-copy to another Google account. ...
Auto-copy to a non-Google account. ...
All these methods give you more then a backup: an actual live copy of your email account that you can switch over to temporarily.
Richi Jenningsis an independent analyst/consultant, specializing in blogging, email, and security. A cross-functional IT geek since 1985, you can follow him as @richi on Twitter, pretend to be richij's friend on Facebook, or just use good old email: itbw@richij.com.
