Internet Archive to unveil massive Wayback Machine data center
The Wayback Machine stores 85 billion Web pages dating back to '96
Computerworld - The Internet Archive organization plans next week to announce the opening of a new data center to house two petabytes of information for its Wayback Machine, the digital time capsule that stores archived versions of Web pages dating back to 1996.
The Wayback Machine houses 85 billion Web pages archived for more than a dozen years, which amounts to three petabytes of data, or about 150 times the content of the Library of Congress. Only five years ago, the Wayback Machine contained about 30 billion Web pages. It is expected to continue to grow by 100TB of data per month now that it's live.
The Internet Archive's massive database is mirrored to the Bibliotheca Alexandrina, the new Library of Alexandria in Egypt, for disaster recovery purposes.
According to an event invitation from Sun Microsystems Inc., the Internet Archive is moving from a traditional data center filled with standard Linux servers to one that runs Solaris 10 with ZFS on Sun Fire x4500s servers inside a Sun Modular Datacenter. The modular system is an all-in-one data center housed in a metal shipping container for mobility.
Because of the modular design, Sun said the data center was deployed in a tenth of the time it would take to build a typical bricks-and-mortar data center. The Wayback Machine Sun Modular Datacenter can service 500 inquiries a second, Sun said. A spokesperson for the Internet Archive said the user interface on the Wayback Machine will not change.
The Internet Archive is a nonprofit organization located in the Presidio in San Francisco, with data centers in Redwood City and Mountain View, Calif. The archive not only keeps snapshots of Web pages, but also software, movies, books, and audio clips.
Users can surf the Wayback Machine by typing in the Web address of a Web site or Web page and then choose from a series of dates that reflect the stored images. The site does not currently support keyword search.
Read more about Data Storage in Computerworld's Data Storage Topic Center.
- Best iPhone, iPad Business Apps for 2014
- 14 Tech Conventions You Should Attend in 2014
- 10 Desktop Apps to Power Your Windows PC
- How to Add New Job Skills Without Going Back to School
- Slideshow: 7 security mistakes people make with their mobile device
- iOS vs. Android: Which is more secure?
- 11 sure signs you've been hacked
- OpenStack Hype vs. Reality: CIO Quick Pulse Open-source architecture can enable IT departments to build infrastructure-as-a-service (IaaS) clouds running on standard hardware.
- OpenStack and Red Hat: IDC White paper Most OpenStack deployments are by public cloud providers that are early adopters of technology and use OpenStack in a do-it-yourself deployment and support...
- Red Hat Enterprise Linux OpenStack Platform Datasheet Seamlessly transition to the cloud. Red Hat Enterprise Linux OpenStack Platform delivers an integrated foundation to create, deploy, and scale a secure and...
- Pay-as-you-Grow Data Protection: IBM Tivoli's Full-featured Data Protection Suite for Small to Medium Businesses IBM Tivoli Storage Manager Suite for Unified Recovery gives small and medium businesses the opportunity to start out with only the individual solutions...
- Make or Break: New Auto Products Must Go To Market On Time This Webcast quantifies the value of time to market for the auto industry and highlights how Primavera Enterprise Portfolio Management can help organizations.
- IBM Flash Webcast: Optimizing your Datacenter for Efficient Storage & ROI Register for this webcast to learn the benefits of flash storage from IBM Customer, Leonardo Irastorza of Royal Caribbean Cruise Ltd and Storage... All Data Storage White Papers | Webcasts