Facebook engineer: Going large requires thinking small
IDG News Service - When managing a constantly expanding system with many moving parts, it is crucial to break the system into large numbers of small pieces and manage them with lots of small, dedicated teams, advised Bobby Johnson, director of engineering for Facebook, at the Usenix Annual Technical Conference in Boston.
The topic of his presentation was "Lessons of Scale at Facebook." Johnson, who began working at the social networking service four years ago, when it had 7 million users, has seen exponential growth at the company. Today, Facebook has about 400 million users and Johnson oversees about 400 engineers.
One major lesson he has learned over the years: Keep the projects to add new features small, and manage them with small teams that have direct control over the features.
At a high level, Facebook has a simple three-layer architecture, consisting of Web servers that assemble pages for users, the cache layer, which keeps much of the data that is frequently used, and a database layer, which serves mainly as "persistent storage," Johnson said.
Each layer has been scaled horizontally, meaning the layers are run across thousands of servers.
It is largely an open-source stack, though one that has been heavily modified: The Web server layer runs modified copies of Linux. The cache layer relies on the Memcache, and MySQL powers the databases. The page components (each page is assembled from dozens of smaller components) are written in PHP, though the code has been pre-compiled.
These days the site, which people expect to be up constantly, can get up to 100 million messages per second to the cache layer. The system handles about 1 billion chat messages and 100 million search messages per day. Because the service has a global audience, it remains busy 24 hours a day. And it keeps growing.
"Every week, we have our biggest day ever," he said.
For building in new features and enhancing the old ones, the approach that Facebook has taken has been one of deploying "very small teams who move quickly," Johnson said. "We make small changes frequently," he said, noting that when something goes wrong, they can isolate the problem quickly.
The data set is unusual in that it is highly connected. For instance, Facebook runs the world's largest photo-sharing site, even though it doesn't offer many of the features of other sites, such as Flickr. But the one advantage it does have is that people are tagged in photos.
And this is true overall for Facebook, he added. The true value it adds is connectivity, the ability for users to connect with other users.
- Best iPhone, iPad Business Apps for 2014
- 14 Tech Conventions You Should Attend in 2014
- 10 Desktop Apps to Power Your Windows PC
- How to Add New Job Skills Without Going Back to School
- Slideshow: 7 security mistakes people make with their mobile device
- iOS vs. Android: Which is more secure?
- 11 sure signs you've been hacked
- System and Data Protection, Recovery and Availability This white paper describes how ARCserve works and the benefits it can provide IT environments of all sizes.
- Simplifying Data Protection, Reducing Risk of Data Loss and System Downtime This white paper outlines what IT organizations should look for in a data protection solution, including simplicity and ease of deployment, comprehensive protection,...
- Boosting Storage Performance for I/O Intensive Enterprise Applications Implementing enterprise applications can be a complex undertaking for IT organizations. Successful deployments depend on high-performance storage systems that are easy to manage...
- Defense Throughout the Vulnerability Life Cycle This whitepaper provides insight into how to leverage threat and log management technologies to protect your IT assets throughout their vulnerability life cycle.
- Live Webcast Best Practices for the Hyperconverged Enterprise Network To the Age of Constant Connectivity and Information overload
- Live Webcast Unmasking the Differences between Consumer and Enterprise File Sync & Share The consumerization of IT combined with the rapid pace of the modern mobile workplace is forcing enterprise IT teams to evaluate file sync...
- Live Webcast Government Agency Webifies Outdated COBOL Applications Let this CTO tell you how his agency converted 1980s-era green screens into an e-filing portal for the 100,000 cases handled each year...
- Four Myths of High-Productivity App Dev Debunked Debunk the main myths surrounding high-productivity application development and how both platforms have overcome them.
- The New Way to Work Knowledge Vault This Knowledge Vault focuses on how, in today's increasingly virtual world, it's more important than ever to engage deeply with employees, suppliers, partners,... All Applications White Papers | Webcasts