Data Finds a Place on the Grid

Computerworld - The data grid has been playing second fiddle to the compute grid when it comes to media attention. But companies and public institutions searching for better ways to share and manage large amounts of data are beginning to take notice.

A compute grid allows users to take the computing resources in a distributed and heterogeneous environment, manage those disparate resources as one and focus them on problem solving.

A data grid acts in a similar way. It has a middleware layer and metadata framework to give users a centralized view of distributed data without physically centralizing the data.

That means the data can be located on Windows, Unix or Linux systems running multiple formats. It can be structured or unstructured and can consist of different media types. A data grid and a compute grid can operate together—the principles are the same.

But there are limits to what a grid can do. A grid, for instance, doesn't offer a means for discovering and categorizing unstructured data. What the data grid provides is a standards-based framework for interconnecting that information once those tasks are addressed.

Data-grid technology is in the early-adopter phase, drawing the interest of research institutions with large and scattered data repositories, such as Pfizer Global Research & Development in Groton, Conn., and the University of Arkansas Center for Advanced Spatial Studies in Fayetteville, as well as research consortiums such as the European Union's DataGrid project, led by particle physics research center CERN.

Data grids will find broader applications as standards mature and technology problems, such as managing security in a grid's distributed environment, are solved, say analysts and users.

"I think the whole promise of grid is pretty exciting," says Paul Lewis, director of research information architecture at Pfizer. But more work is needed, he adds.

Seeking Support

Products that support data in a grid environment are emerging. For example, Pfizer uses Avaki Inc.'s data-grid software. The Center for Advanced Spatial Studies takes advantage of the grid capabilities in Oracle 10g, Oracle Corp.'s flagship database.

But the very concept of grids involves interconnectedness among disparate applications and data sources. Until vendors include standards-based grid capabilities, interfaces and processes in their products, data-grid adoptions are going to be limited.

"Vendors have got to step up and say, 'We're going to make our products grid-enabled,' " says Lewis. "If more vendors grid-enable products, it makes our job easier, because then we can plug in more computers when we need more capacity."

Emerging data-grid products, such as Avaki's, are being used within companies. But some of the leading thinkers behind the data-grid effort imagine developing systems that connect large numbers of enterprises, entire supply chains and customer bases.

"The equivalent of the Internet Protocol for remote access to data is still a work in progress," says Ian Foster, senior scientist and head of the Distributed Systems Lab at Argonne National Laboratory in Illinois and co-director of the grid standards effort at the Globus Alliance.