Planet-Scale grid

A particle collider leads data grid developers to unprecedented dimensions.

By Patrick Thibodeau

October 10, 2005 12:00 PM ET

Computerworld - In 2007, scientists will begin smashing protons and ions together in a massive, multinational experiment to understand what the universe looked like tiny fractions of a second after the Big Bang. The particle accelerator used in this test will release a vast flood of data on a scale unlike anything seen before, and for that scientists will need a computing grid of equally great capability.

The Large Hadron Collider (LHC), which is being built near Geneva, will be a circular structure 17 miles in circumference. It will produce data in the neighborhood of 1.5GB/sec., or as many as 10 petabytes of data annually, 1,000 times bigger than the Library of Congress' print collection. The data flows will likely begin in earnest in 2008.

As part of this effort, which is costing about 5 billion euros ($6.3 billion U.S.), scientists are building a grid using 100,000 CPUs, mostly PCs and workstations, available at university and research labs in the U.S., Europe, Japan, Taiwan and other locations. Scientists need to harness raw computing power to meet computational demands and to give researchers a single view of this disbursed data.

This latter goal—creating a centralized view of data that may be located in Europe, the U.S. or somewhere else—is the key research problem.

Centralizing the data virtually, or creating what is called a data grid, means extending the capability of existing databases, such as Oracle 10g and MySQL, to scale to these extraordinary data volumes. And it requires new tools for coordinating data requests across the grid in order to synchronize multiple, disparate databases.


Tony Doyle, project leader of Grid Particle Physics (GridPP) project

"It's all about pushing the envelope in terms of scale of robustness," says Tony Doyle, project leader of Grid Particle Physics (GridPP) project, a U.K.-based scientific grid initiative that's also part of the international effort to develop the grid middleware tools.

Researchers believe that improving the ability of a grid to handle petabyte-scale data, split up among multiple sites, will benefit not only the scientific community but also mainstream commercial enterprises. They expect that corporations—especially those involved in fields such as life sciences—will one day need a similar ability to harness computing resources globally as their data requirements grow.

"If this works, it will spawn companies that will just set up clusters to provide grid computing to other people," says Steve Lloyd, who chairs the GridPP Collaboration Board, based at the Rutherford Appleton Laboratory in Oxfordshire, England. GridPP is working with the international team to develop the grid the LHC will use.

Recommend Digg Twitter ShareThis

Email Print Send Feedback Reprints

Additional Resources

POLL RESULTS

Leading IT Pros Weigh In on Top IT Issues

Accelerate your knowledge of the IT world you inhabit by viewing the results of a series of polls taken by your IT peers. These polls of 100+ IT professionals each are available for full viewing. They cover key topics such as virtualization, processor performance, green IT, cloud computing and many others. Be a part of the buzz.

WHITE PAPER

For Best Results, Think Beyond the Box

Technology is complex. Keeping it running productively shouldn't be. To that end, you want to minimize the number of solutions needed in-house to simplify operations, maintenance, and support. Kodak offers a best-practices model. One company provides support for both scanner and software, for fast problem resolution without vendor finger-pointing. Download now!

WHITE PAPER

Accelerating the Path to Demand Intelligence with a Demand Signal Repository

Utilizing demand intelligence improves the precision of pricing, product assortments, channel/store placement, and promotion, which are all essential for sustainable revenue management performance. Learn more, download this free whitepaper today.

White Papers & Webcasts

Speeding business innovation with HP Data Center Transformation solutions
Data center transformation enables your IT organization to focus more on business priorities and innovation by decreasing spending on maintenance and management by...

Four Principles for Reducing Storage TCO
(Source: Hitachi Data Systems) Difficult economic times require new strategies for reducing costs. Where storage technology and economics meet, there are...

HP Data Center Transformation Solutions
CIOs today are challenged to respond to economic and business pressures, to change from being cost centers to becoming strategic business enablers. There...

Boost your CAE productivity, and break-away from the pack
(Source: Sun) Join Clemson University as they present their groundbreaking engineering simulations research at their Computational Center for Mobility Systems. Dr. James Leylek,...

Using Symark PowerBroker™ to Enrich Your Organization's RBAC Model
The essential notion of Role-Based Access Control (RBAC) for IT security administration is establishing permissions based on the functional roles within the enterprise,...

Deduplication and Other Strategies for Protecting Your Assets with the Veritas NetBackup Platform
(Source: Symantec) Many companies find their backup and storage resources strained by data growth and increased regulatory requirements for data retention. In today's...

Using VMware Site Recovery Manager to Simplify DR
(Source: NetApp) Nothing is scarier than the prospect of having to recover an entire site after a disaster. VMware® Site Recovery Manager (SRM)...

Controlling Email and File Server Growth and Costs with Intelligent Archiving
(Source: Symantec) According to IDC 54% of the storage capacity added by organizations in 2008 will be dedicated to the storage of file-based...

NetApp and VMware Virtual Infrastructure 3 Storage Best Practices
(Source: NetApp) NetApp has been providing advanced storage features to VMware ESX solutions since the product began shipping in 2001. During that time,...

Maximize Storage Assets with Thin Provisioning, Tiered Storage, and Cluster File Systems
(Source: Symantec) Thin Provisioning is an opportunity to immediately optimize your storage systems and make more capacity available to your applications. In order...

All White Papers | All Webcasts