Computerworld -
"Migrate every five years." That might sound like a guideline for dysfunctional birds, but it's actually a software setting that the National Film Board of Canada uses in its digital archiving system.
"The data has to be kept for infinity, so there has to be a migration process," says Julie Dutrisac, head of research and development for the film board in Montreal, which preserves 13,000 Canadian films. "When you get into the digital domain, you are stuck migrating."
Migration, of course, means moving the material to new storage hardware, because the old hardware can't be expected to last much more than five years, or because of expected obsolescence.
To continue reading, register here to become an Insider
It's FREE to join
Computerworld - "Migrate every five years." That might sound like a guideline for dysfunctional birds, but it's actually a software setting that the National Film Board of Canada uses in its digital archiving system.
"The data has to be kept for infinity, so there has to be a migration process," says Julie Dutrisac, head of research and development for the film board in Montreal, which preserves 13,000 Canadian films. "When you get into the digital domain, you are stuck migrating."
Migration, of course, means moving the material to new storage hardware, because the old hardware can't be expected to last much more than five years, or because of expected obsolescence.
"Most storage products have a five-year warranty, and most users are in the practice of replacing their systems every five years, with infrastructures becoming completely transformed, like a snake shedding its skin, in a maximum of 10 years," says John Monroe, an analyst at research firm Gartner. "But it has to be done without changing any data bit. People are terrified about it. It keeps IT managers up at night."
But better sleep may be on the way in as little as five years because researchers are working to develop storage technology that should be reliable for decades rather than years, slashing the need for migration.
Racetrack Memory
Scientists at IBM, for example, are cooking up a technology called "racetrack memory," which consists of microscopic segments of U-shaped nanowires of ferromagnetic material suspended vertically in CMOS chips. Each nanowire carries about 100 bits, encoded as nonvolatile spots of magnetism.
"I think it will be the storage Utopia," says Stuart Parkin, an IBM fellow at the Almaden Research Center in San Jose. "There are no trade-offs."
He foresees response times of 1 nanosecond, rather than the tens of nanoseconds of today's system DRAM. The cost should be on par with that of disk drives, but with a millionfold performance advantage. Meanwhile, racetrack units should be much more compact than hard drives, since they don't need a motor or spindle, Parkin notes.
"We move the data without moving any atoms," he adds. "There is no mechanical motion. Instead, we are rotating magnetic moments, i.e., the direction of the magnetic field."
The expected longevity of the data would depend on the final design of a specific racetrack system and could extend for decades. "Most magnetic devices are designed for 10 years, and ours should last at least that long," Parkin says. But he predicts that few people would want to keep any system longer than a decade, due to the pace of technological progress.
Parkin expects to see the technology on the market in five to seven years.
And Don't Forget...
Active Archiving
Experts agree that data preservation doesn't happen by itself -- and this will remain the case, even with any future long-term nonvolatile, migrationless storage technology.
"It takes a lot of care and maintenance to keep digital information around," notes Alistair Veitch, director of HP's Storage and Information Management Platforms Lab. "To do it right, you need multiple copies, probably in multiple formats, and you need to audit them periodically."
Multiple formats are used for logical preservation, as a hedge against format or software obsolescence. Many organizations archive data in multiple formats, including the original format and an image format, in hopes that one will remain interpretable, says Veitch.
"Images are the lowest common denominator, and I think a straight GIF or JPG will be interpretable decades from now," Veitch says, adding that some organizations also use PDF.
Beyond that, the archive itself needs to be a fully documented, open system, especially the file system and the metadata database, so that someone could write a program to read the information in a future environment, adds Molly Rector, chairman of the board of the Active Archive Allliance and vice president at archive system vendor Spectra Logic.
"That is the big shift -- the solutions were all proprietary until a couple of years ago," Rector says, adding that newer open archiving systems often aren't as feature-rich but may be significantly less expensive. As for auditing, even if the storage hardware lasts for decades, there must be ongoing data-integrity validation. "You can't just let it sit there for 50 years and trust your archives," says Rector.
Overall, "there is no one piece of hardware that will save us," Veitch adds, "but I think that we will get solid-state storage that will last for decades, and I hope to see it in five years."
—Lamont Wood
You are previewing premium content. 
Free Insider Guide