Solid state drives refuse to delete data

Quirk of flash memory software creates enormous security risk

The first time I was briefed on developments that would lead to solid-state hard drives for laptops I thought it was such a great idea I couldn't wait to get one. Improve speed, extend battery life and eliminate all that complaining when I close the lid and sling the laptop around before the disk stops spinning? Oh yeah.

Unfortunately former colleague Galen Gruman was in the same meeting, and managed to shoot the idea into my "maybe someday" file before I got back to my desk. (Galen is wildly enthusiastic about technology himself, but has annoyingly accurate reasons for it when his enthusiasms conflict with mine.)

In this case the reasons were that the mean time between failure on flash memory was far shorter than for disk, making a solid-state drive (SSD) a good risk for data loss.

They also wore out faster than hard drives. Even the best-quality flash memory had write endurance (number of times you can write data onto a particular sector, erase it and write other data onto the same sector) a fraction of what even a low-quality hard drive could manage.

He could have made the case that the stuff I filled my hard drive up with would have been better off lost (though he might have had to stand in line).

As it turns out, even that idea wouldn't have flown.

Though SSDs are far faster, more reliable and cheaper than they would have been then, it turns out it's incredibly difficult to actually erase data from a flash-based SSD.

The firmware called the Flash Translation Layer, which makes the SSD look to the OS like a hard drive spreads reads and writes around sectors of the SSD not only to make retrieval efficient, but to maximize the life of the drive by keeping reads and writes to any one sector to a minimum.

Because the process is so different from storing on disk, the operating system and SSD don't cooperate well when the user tries to erase the disk.

Of 12 SSDs tested, only four were completely erased using the standard "Erase Unit" command and one kept almost all its data, according to a report from University of California-San Diego computer-science Ph.D. candidates Michael Wei and Laura Grupp.

Overwriting the drives left an average of only 1 percent of the original data – but only after overwriting the whole disk 20 times, which is way, way slower than you'd expect.

Some of the disks took 58 hours for one full overwrite pass.

What about just deleting one file, though. Easy, right?

"All single-file overwrite sanitization protocols failed: between 4 percent and 75 percent of the files' contents remained on the SATA SSDs. USB drives performed no better: between 0.57 percent and 84.9 percent of the data remained," according to Wei and Grupp's paper Reliably Erasing Data from Flash-Based Solid State Drives (PDF).

On a disk drive files are "erased" by having other data placed on the same segments of disk.

With the Flash Translation Layer controlling all the reads and writes and trying its best to keep from reading or writing data to the same spot, even trying to erase one file was almost impossible.

The file system in the OS thought the files were erased; the spot the FTL told it the data was stored had other data in it.

The actual physical sector of the flash drive was untouched most of the time.

If you never went looking for the file you'd probably never know it was still there. If you used recovery software, or if someone else did, using forensic software to extract data you thought you'd deleted, you'd be out of luck.

Some SSDs have routines that do erase the full drive, but there's no way to tell which ones unless you try them and run recovery or forensics on the result.

The authors suggest SSD developers add software extensions that let the OS and Flash Translation Layer talk more clearly so you can tell whether the SSD is really overwriting a file or just leaving it be and hoping no one notices.

In the meantime, SSDs may be faster and more efficient than disk, but they're also an enormously greater security risk and will be until the Flash software is fixed.

Kevin Fogarty writes about enterprise IT for ITworld. Follow him on Twitter @KevinFogarty.

This story, "Solid state drives refuse to delete data " was originally published by ITworld.

Copyright © 2011 IDG Communications, Inc.

Shop Tech Products at Amazon