Lustre among the first object-based storage systems

Computerworld - One soon-to-be-released incarnation of object-based storage is Lustre, an open-source file system that's under development by Cluster File Systems Inc. in Mountain View, Calif.
Lustre, which is short for Linux Cluster, is a file system that can serve storage systems with tens of thousands of nodes and move hundreds of gigabytes per second. The 1.0 release of Lustre is expected this quarter and will target clusters of up to 1,000 nodes with 100TB of storage.
Lustre runs on commodity hardware using object-based disks for storage and metadata servers for storing file system metadata. According to Cluster File Systems, the design provides for replicated fail-over metadata servers to maintain a transactional record of high-level file and file-system changes. Distributed object-based storage disks, instead of individual servers, are responsible for actual file system I/O and for interfacing with storage devices.
"The idea is that the disk array is used in a different way. We build object storage controllers using Linux systems. We use these to front large disk arrays," says Peter Braam, president and CEO of Cluster File Systems.
Peter Gerr, an analyst at Enterprise Storage Group in Milford, Mass., says object-based storage systems save on I/O and reduce bottlenecks because the application server doesn't have to go through the hierarchy of a file system to find the applications it's looking for.
"The distinction is you are really assigning a unique fingerprint [identifier] to an object -- a bit of data or file," Gerr says. "The address is tied to the content itself, not the content address."
In a cluster system like Lustre, the actual physical location of data is less important than the user knowing that the content is protected because it's replicated at least once. And it's secure because users can only access a specific object using its unique address, rather than a block of data or an entire file system, Gerr says.
Currently, the Pacific Northwest National Laboratory in Richmond, Wash., and the Lawrence Livermore National Laboratory in Livermore, Calif., are coupling supercomputers using Lustre. The machines are using it to more efficiently retrieve huge volumes of information by eliminating I/O bottlenecks, and they are expected to be 10 times cheaper than existing supercomputer clusters.