Skype blames buggy Windows software, swamped servers for outage
Promises to look into auto-updating its Windows client software to prevent another blackout
Computerworld - Skype today blamed last week's outage on a combination of overloaded instant messaging servers, buggy software, and the failure of its "supernode" infrastructure.
In a lengthy blog entry Wednesday, Lars Rabbe, Skype's chief information officer, provided more details on the outage that kept the instant message, Internet telephone and video chat service offline for much of Dec. 22 and parts of Dec. 23.
Previously, Skype had tapped its supernodes, the term for systems running Skype that also act as directories, for the outage.
Today, Rabbe said a bug in an older version of the Windows Skype client was at the root of the service's failure, although the flaw did not trigger the blackout.
The bug in version 5.0.0152 caused those Windows clients to crash when they received a delayed response from "a cluster of support servers responsible for offline instant messaging" that had been overloaded, Rabbe said. About 50% of all Skype users were running the buggy 5.0.0152 version of the Windows client last week.
Rabbe did not explain how or why those servers -- which triggered the Windows client crashes, and thus, the outage -- became unresponsive last Wednesday. Today, Skype said it was still investigating the cause of the server overload.
When the Windows clients began crashing -- at the peak, about four out of every 10 copies of version 5.0.0152 failed -- they also took down as many as 30% of Skype's supernodes, which were also running the problem-plagued edition.
The downfall of those supernodes eventually took all the rest offline as well, as users swamped the remaining supernodes with requests after experiencing a crash. The supernodes were designed to automatically shut down when loads reached certain limits, a measure Rabbe said was designed to preserve performance on their Windows hosts, which are not dedicated to Skype, but simply PCs run by users outside a firewall.
"This further increased the load on remaining supernodes and caused a positive feedback loop, which led to the near complete failures that occurred a few hours after the triggering event," Rabbe reported.
Other factors that contributed to the outage, said Rabbe, was the time of day when the incident began. "The initial crashes happened just before our usual daily peak hour (10 a.m. Pacific) ... which resulted in traffic to the supernodes that was about 100 times what would normally be expected at that time of day," he said.
Skype was offline for approximately 24 hours, from 8 a.m. Pacific on Dec. 22 to 8 a.m. the next day, Rabbe said, a claim that was somewhat at odds with earlier reports from the company, which said on Dec. 23 that two-thirds of its users were still unable to connect as of 3 a.m. that day.
To get its peer-to-peer network back on its feet, Skype added several thousand instances of its software to the network. Those copies were dedicated supernodes, and nicknamed "mega-supernodes" by the company.
By Friday, Dec. 24, Skype was pulling most of the mega-supernodes out of service as the network was restored and the usual supernodes stabilized under their loads.
Rabbe said Skype would revisit its policies on when it automatically pushes updates to clients, and investigate ways to recover faster from a failure. The former seemed to be the way Skype was leaning to prevent another outage in the future.
"We believe these measures will reduce the possibility of this type of failure occurring again," Rabbe said of client software automatic updating.
Last week, Skype CEO Tony Bates apologized for the outage, and promised paying customers that the company would e-mail vouchers for 30 minutes of free calling or extend their subscriptions by one week.
Gregg Keizer covers Microsoft, security issues, Apple, Web browsers and general technology breaking news for Computerworld. Follow Gregg on Twitter at @gkeizer or subscribe to Gregg's RSS feed . His e-mail address is firstname.lastname@example.org.
Read more about Mobile Apps in Computerworld's Mobile Apps Topic Center.
- Best iPhone, iPad Business Apps for 2014
- 14 Tech Conventions You Should Attend in 2014
- 10 Desktop Apps to Power Your Windows PC
- How to Add New Job Skills Without Going Back to School
- Slideshow: 7 security mistakes people make with their mobile device
- iOS vs. Android: Which is more secure?
- 11 sure signs you've been hacked
- Performance Management: The Mobile App Development Playbook This comprehensive 16 page Forrester Research, Inc. report, authored by Jeffrey Hammond, Forrester VP and Principal Analyst, details a number of valuable, commonly...
- New Problems Require Innovative Solutions The mobile market is expected to be worth $25 billion by 2015
- Getting Agnostic about Mobile Devices The idea of being able to interact with customers, prospects, and stay attuned to competitive pressures is not new, but the velocity at...
Red Hat Enterprise Linux - The Original Cloud Operating System
Linux adoption is growing against a number of measures, such as the
number of supercomputers that run Linux and the size of the contributing...
- Webinar: Building a Big Data solution that's production-ready Big data solutions are no longer just a nice-to-have.
- Meg Whitman presents Unlocking IT with Big Data During this Web Event you will hear Meg Whitman, President and CEO, HP discuss HAVEn - the #1 Big Data platform, as well... All Mobile Apps White Papers | Webcasts
As emerging technologies evolve they often find an initial niche in highly specialized scenarios, or in specific industry verticals, before expanding to wider areas of applicability. Within these initial niches, the early adopters can be anything from digital enthusiasts to fashionistas, or they can be folks simply using the technology because it serves a specific need extremely well. (free registration required) more