AOL installs 50TB of SSD; boosts DB performance by 4X

Computerworld - GRAPEVINE, Texas -- When it came to managing most of AOL's 6 petabytes of data, a Fibre Channel SAN sufficed. But for its most critical relational database, AOL found that the SAN was too constrained and caused its IT shop to fail to live up to its service level agreements with business units more than 50% of the time.

After investigating what may have been causing I/O bottlenecks, AOL found the problem was back-end storage. To fix the problem, AOL decided to build a 50TB storage-area network (SAN) from solid-state technology.

The upgrade worked, delivering four times more throughput to the SQL database than the Fibre Channel SAN was capable of, while still giving storage admins the flexibility to migrate data between storage systems because the NAND flash memory sat behind an existing virtualization appliance, which aggregates all of the back-end storage and serves it up as if it were from a single pool.

AOL engineers found that the throughput problems didn't appear to be based on hardware. The company has five large high-end arrays with 15,000- or 10,000-rpm drives for primary storage and two lower-end arrays with serial ATA (SATA) drives for nearline backup. But the storage arrays could only feed data to internal drives as fast as the SAS backbone allowed, which was about 6Gbit/sec.

AOL's storage infrastructure supports about 4,000 servers that feed information to both online users and to the company's own back-end applications.

Dan Pollack, senior operations architect at AOL, said he considered adding solid-state drives to the company's existing storage arrays, but decided against it because they would be held back by the array's controller bandwidth.

"So you find you can take a high-performance SSD device and half the performance is lost at the head of the array," Pollack told an audience at the Storage Networking World conference here this week.

Pollack also considered using solid-state drives in servers, but didn't because he would not have been able to cluster them into his SAN, they don't offer the required capacity, such drives could not be nondisruptively swapped out and data couldn't be migrated between systems.

Pollack settled on a solid-state array from Mountain View, Calif.-based Violin Memory. It could plug directly into his Fibre Channel network, and it could sit behind his storage virtualization appliance from YottaYotta, which was purchased last year by storage giant EMC.

Pollack said the rollout, including planning and testing, took only eight weeks. The system has been live since July 1. Since then, the SSD array has had zero downtime and it has allowed his shop to live up to its SLAs with business units without exception. I/O response times are typically less than 1 millisecond and there was no impact in his IT team's ability to manage the back-end storage.