The 100-Year Archive Dilemma

As more organizations store more data longer, the IT industry seeks a better way.

By Lucas Mearian

July 25, 2005 12:00 PM ET

Computerworld - A record is a record, whether it's a sheet of paper, an e-mail, an electronic document or a digital image.
"It's the content that drives retention, not the media it's written on," says Adam Jansen, a digital archivist for the state of Washington. And recent federal regulations are requiring more companies to save more content for longer periods of time.
While content may be king in theory, in practice, the media on which it's stored and the software that stores it present problems. As digital tapes and optical discs pile higher and higher in the cavernous rooms of off-site archive providers, businesses are finding them increasingly expensive to maintain.
The software that created the data has limited backward compatibility, so newer versions of a program may not be able to read data stored under older versions.
Moreover, the media on which the data is stored degrade relatively quickly. "Ten years is pushing it as far as media permanence goes," says Jansen.
Varied Approaches
Today, the only safe path to long-term archiving is repeated data migration from one medium and application to another throughout the data's life span, experts say.
But the storage industry is working on the problems from various angles.
One solution to the backward-compatibility problem is to convert data to common plain-text formats, such as ASCII or Unicode, which support all characters across all platforms, languages and programs. Using plain-text formats to store data enables virtually any software to read the files, but it can cause the loss of data structure and rich features such as graphics.
Another approach is to use PDF files to store long-term data. There can be backward-compatibility problems with PDFs, but the file format's developer, Adobe Systems Inc., has created an archival version of its software, called PDF/A, that addresses them.


Adam Jansen, digital archivist for the state of Washington Image Credit: Craig Sweat

To date, the most promising standard data-storage technologies are emerging in new XML-based formats, according to analysts and studies. XML is a file format and self-describing markup language that is independent of hardware and operating systems.
On the media side, the Storage Networking Industry Association (SNIA) is working toward solving what it calls the "100-year archive dilemma" through a standards effort for media. The goal is to store data in a format that will always be readable by a generic reader.
"Degrading media is not at all the issue. Rather, the real issue is long-term readers and compatibility -- the logical problem which we intend to address," says Michael Peterson, president

Recommend Digg Twitter ShareThis

Email Print Send Feedback Reprints

Additional Resources

POLL RESULTS

Leading IT Pros Weigh In on Top IT Issues

Accelerate your knowledge of the IT world you inhabit by viewing the results of a series of polls taken by your IT peers. These polls of 100+ IT professionals each are available for full viewing. They cover key topics such as virtualization, processor performance, green IT, cloud computing and many others. Be a part of the buzz.

WHITE PAPER

For Best Results, Think Beyond the Box

Technology is complex. Keeping it running productively shouldn't be. To that end, you want to minimize the number of solutions needed in-house to simplify operations, maintenance, and support. Kodak offers a best-practices model. One company provides support for both scanner and software, for fast problem resolution without vendor finger-pointing. Download now!

WHITE PAPER

Accelerating the Path to Demand Intelligence with a Demand Signal Repository

Utilizing demand intelligence improves the precision of pricing, product assortments, channel/store placement, and promotion, which are all essential for sustainable revenue management performance. Learn more, download this free whitepaper today.

White Papers & Webcasts

Speeding business innovation with HP Data Center Transformation solutions
Data center transformation enables your IT organization to focus more on business priorities and innovation by decreasing spending on maintenance and management by...

Four Principles for Reducing Storage TCO
(Source: Hitachi Data Systems) Difficult economic times require new strategies for reducing costs. Where storage technology and economics meet, there are...

HP Data Center Transformation Solutions
CIOs today are challenged to respond to economic and business pressures, to change from being cost centers to becoming strategic business enablers. There...

Boost your CAE productivity, and break-away from the pack
(Source: Sun) Join Clemson University as they present their groundbreaking engineering simulations research at their Computational Center for Mobility Systems. Dr. James Leylek,...

Using Symark PowerBroker™ to Enrich Your Organization's RBAC Model
The essential notion of Role-Based Access Control (RBAC) for IT security administration is establishing permissions based on the functional roles within the enterprise,...

Deduplication and Other Strategies for Protecting Your Assets with the Veritas NetBackup Platform
(Source: Symantec) Many companies find their backup and storage resources strained by data growth and increased regulatory requirements for data retention. In today's...

Using VMware Site Recovery Manager to Simplify DR
(Source: NetApp) Nothing is scarier than the prospect of having to recover an entire site after a disaster. VMware® Site Recovery Manager (SRM)...

Controlling Email and File Server Growth and Costs with Intelligent Archiving
(Source: Symantec) According to IDC 54% of the storage capacity added by organizations in 2008 will be dedicated to the storage of file-based...

NetApp and VMware Virtual Infrastructure 3 Storage Best Practices
(Source: NetApp) NetApp has been providing advanced storage features to VMware ESX solutions since the product began shipping in 2001. During that time,...

Maximize Storage Assets with Thin Provisioning, Tiered Storage, and Cluster File Systems
(Source: Symantec) Thin Provisioning is an opportunity to immediately optimize your storage systems and make more capacity available to your applications. In order...

All White Papers | All Webcasts