A particle collider leads data grid developers to unprecedented dimensions.
Computerworld - In 2007, scientists will begin smashing protons and ions together in a massive, multinational experiment to understand what the universe looked like tiny fractions of a second after the Big Bang. The particle accelerator used in this test will release a vast flood of data on a scale unlike anything seen before, and for that scientists will need a computing grid of equally great capability.
The Large Hadron Collider (LHC), which is being built near Geneva, will be a circular structure 17 miles in circumference. It will produce data in the neighborhood of 1.5GB/sec., or as many as 10 petabytes of data annually, 1,000 times bigger than the Library of Congress' print collection. The data flows will likely begin in earnest in 2008.
As part of this effort, which is costing about 5 billion euros ($6.3 billion U.S.), scientists are building a grid using 100,000 CPUs, mostly PCs and workstations, available at university and research labs in the U.S., Europe, Japan, Taiwan and other locations. Scientists need to harness raw computing power to meet computational demands and to give researchers a single view of this disbursed data.
This latter goalcreating a centralized view of data that may be located in Europe, the U.S. or somewhere elseis the key research problem.
Centralizing the data virtually, or creating what is called a data grid, means extending the capability of existing databases, such as Oracle 10g and MySQL, to scale to these extraordinary data volumes. And it requires new tools for coordinating data requests across the grid in order to synchronize multiple, disparate databases.
Tony Doyle, project leader of Grid Particle Physics (GridPP) project
Researchers believe that improving the ability of a grid to handle petabyte-scale data, split up among multiple sites, will benefit not only the scientific community but also mainstream commercial enterprises. They expect that corporationsespecially those involved in fields such as life scienceswill one day need a similar ability to harness computing resources globally as their data requirements grow.
"If this works, it will spawn companies that will just set up clusters to provide grid computing to other people," says Steve Lloyd, who chairs the GridPP Collaboration Board, based at the Rutherford Appleton Laboratory in Oxfordshire, England. GridPP is working with the international team to develop the grid the LHC will use.
CERN, the European laboratory for particle physics, is leading the LHC and its grid effort. From CERN's facility near Geneva, the data produced by the particle accelerator will be distributed to nine other major computing centers, including the Brookhaven National Laboratory and the Fermi National Accelerator Laboratory in the U.S., says Fabio Hernandez, grid technical leader at one of the major project sites in France.
- Reduce IT Burdens: Regulation, Legal Hold & Discovery Download this white paper to learn how Delphix can be an integral part of ensuring your archiving strategy actually enables your business.
- Budd Van Lines Moves Data Closer to Home Shipping and logistics company Budd Van Lines uses Infinio to improve performance on their VDI environment. The company employs a virtualized datacenter based...
- Storage Performance with Cost Control As IT groups expand their server virtualization initiatives, central storage performance can become the bottleneck and create poor end user experience.
- Server-side Caching for the VMware Admin vExpert David Davis weights in on how best-in-class server-side caching solutions can drastically improve storage performance and reduce latency without the addition of...
- The Key to Happiness: Throw out Your Data Warehouse In this webinar, Kerry Reitnauer, Director, Solution Architect at FairPoint Communications will discuss the challenges the data warehouse brought, how they migrated to...
- The Next Generation of Affordable Storage As data needs continue to grow and capacity expansion becomes a necessity, companies of all sizes have to address the delicate balance between... All Data Storage White Papers | Webcasts