IDG News Service - To curb disruptions and scale up its service while keeping costs down, Twitter has drastically changed its core infrastructure and has adopted some open-source tools.
Twitter processes about 6,000 messages per second -- that's more than 500 million a day and about 3.5 billion a week. The company set a record earlier this year, with 143,000 messages per second during the airing of a movie in Japan, said Chris Aniszczyk, Twitter's head of open-source computing, at a recent conference in Europe.
When Twitter debuted in 2006, it used a monolithic Ruby on Rails application. That worked fine until 2008, when the microblogging service started to suffer a lot of "fail whales" -- the company's term for service disruptions.
The company decided to break the one application it had been using for everything into different services, Aniszczyk said. It also started using open-source tools such as Apache Mesos, a cluster manager; Netty, for creating high-performing protocol servers; and Scalding, for writing big data jobs.
And in another move, Twitter switched its core infrastructure to a Java virtual machine. The end result, said Aniszczyk, is a performance improvement and fewer disruptions.
- The Benefits of IBM: The Savings of Open Source Download Now
- Path Selection Infographic Path Selection Infographic
- Hyperconvergence Infographic A wide range of observers agree that data centers are now entering an era of "hyperconvergence" that will raise network traffic levels faster...
- Preparing Your Infrastructure for the Hyperconvergence Era From cloud computing and virtualization to mobility and unified communications, an array of innovative technologies is transforming today's data centers.
- Cloud Knowledge Vault Learn how your organization can benefit from the scalability, flexibility, and performance that the cloud offers through the short videos and other resources...
- LIVE EVENT: 5/7, The End of Data Protection As We Know It. Introducing a Next Generation Data Protection Architecture. Traditional backup is going away, but where does this leave end-users? All Open Source White Papers | Webcasts