Facebook temporarily loses more than 10% of photos in hard drive failure

Computerworld - Popular social networking site Facebook.com admitted in a blog post today that over the weekend, a hard drive failure led to the temporary loss of 10% to 15% of the photographs stored by its users.

According to the company, several drives failed at once during a routine upgrade Friday night.

"You may have noticed in the past day that some photos aren't appearing or are displaying a 'question mark' graphic when you go to view them. We're trying to fully understand what happened, since simultaneous hardware failures like this are rare," Evan Priestley, an engineer at Facebook Inc., stated in his blog.

Facebook said its users' photos are safe because it stores multiple copies of the data for disaster recovery and business continuity purposes, and it is working to make the photos affected by the system failure available again as soon as possible.

"We've already repaired about one-third of affected photos and expect to complete repairs on another third tonight," Priestley wrote. "However, even though your photos are safe, we can't serve photos off the affected storage volumes until they're repaired."

The repair process involves copying large amounts of data to new drives, Priestley said. "This is why some photos aren't showing up right now."

Facebook is restoring photos as it repairs the hard drives, so some of those drives will likely be working again today. The social network site should be back to normal by early next week, it said.

"New photo uploads will continue to work properly during the repairs, because we write them to different storage volumes. Thanks for bearing with us while we return things to normal," Priestley said.

Read more about disaster recovery in Computerworld's Disaster Recovery Knowledge Center.