Future Watch: Intelligent storage

Computerworld - Imagine a storage device that uses its own horsepower to manage data, requires no manual settings for security and doesn't care if the client server speaks in blocks or files. That's the promise of object-based storage. Object-based storage technologies shield the application or operating system from the low-level details of managing file storage. In one method, intelligence is added to the storage device in order to offload low-level storage management tasks traditionally handled by the operating system, such as mapping files to actual storage blocks on the disk drive and managing file attributes and other associated metadata.

Although widespread use of object-based storage is still some years away, the technology could result in storage systems that are more scalable, reliable, secure and manageable.

The T10 Technical Committee, which is part of the Washington-based InterNational Committee for Information Technology Standards and the Mountain View, Calif.-based Storage Networking Industry Association (SNIA), is working on a specification for object-based storage, called Object-Based Storage Devices (OSD). OSD turns files, directories and storage-related elements into objects that storage management software accesses using an extended SCSI-3 command set.

"But SCSI is just one component of what we're doing," says Michael Mesnier, a storage architect at Intel Corp. and co-chairman of the SNIA OSD Technical Work Group. "We're also looking at a more general-purpose definition of object-based storage which is irrespective of the transport, which means you can run it over SCSI, you could run it over Fibre Channel ... over TCP/IP or whatever. To me, that's a much stronger impact."

By putting some of the intelligence for accessing objects into the storage array instead of the application server, networks could be infinitely scalable because servers would no longer have to eat up bandwidth searching for and accessing files or blocks of data.

"Just like you could plug a different hard drive into your PC, you could add another server to a storage system in the same way," says Scott A. Brandt, an assistant professor at the Storage Systems Research Center at the University of California, Santa Cruz (UCSC).

UCSC's Jack Baskin School of Engineering is designing a high-performance storage network, based on commodity hardware, that can store up to 2 petabytes of data based on the proposed OSD model.

"While you're still dealing with blocks, they're hidden from the file system," Brandt says. "As you add more storage, you're adding more smarts. What might have been prohibitive details added to a large system are now details handled by the storage device itself."

Moving the object metadata and attributes out of the file system also eliminates the file server as a scalability bottleneck, Brandt says.