EBay attacks server virtualization with 100TB of SSD storage

Computerworld - When online auction site eBay had problems meeting the I/O storage demands of business units starved for more virtual machine (VM) deployments, its quality assurance division had an idea: swap out hard disk drive-based arrays with solid-state drive (SSD) storage.

After replacing 100TB of storage in a year, eBay saw a 50% reduction in standard storage rack space, a 78% drop in power consumption and a five-fold boost in I/O performance. That speed boost now allows eBay to bring a new VM online in five minutes, compared to 45 minutes previously.

EBay's Quality Assurance Division had been using both network-attached storage (NAS) and a storage area network (SAN) prior to deploying modular SSD arrays from Nimbus Data Systems. EBay began by deploying a single 2U (3.5-in high) Nimbus S-class SSD system and HALO storage operating system a year ago.

Today, it has 12 SSD arrays with capacity that rivals the amount of traditional hard drive storage used, according to said Michael Craft, eBay's manager of QA Systems Administration.

"One rack [of SSD storage] is equal to eight or nine racks of something else," Craft said.

EBay's QA division has more than 4,000 VMware ESX virtual servers residing on about 200 machines. Half of those physical servers are now attached to SSD primary storage. The online retailer clones from templates on its storage to the VMs.

Because of its success with SSD, eBay expects to boost its use of the technology.

"This is a pure play in our virtualization stack, but we're looking to expand that as it fits other people's needs," Craft said. "We were having challenges...keeping our space in check by not taking up half the data center with [hard drive] spindles. With the SSD, literally, I've been able to eliminate racks of gear."

Last year, eBay's QA division was looking to move to block-based iSCSI protocol while still supporting the Network File System protocol as well as Fibre Channel. What Craft found was that the SSDs delivered near line-rate 10 Gigabit Ethernet iSCSI performance to the VMware hosts, or about 170,000 I/Os per second (IOPS).

Though SSD is typically an order of magnitude more expensive than hard disk drive storage, Craft said the Nimbus arrays were "on par" with the cost of his previous storage, which a Nimbus spokesman said was from NetApp and HP 3PAR. (Craft declined to identify the vendors).

According to Nimbus CEO Thomas Isakovich, eBay was using Nimbus to replace the NetApp environment, which used 15,000rpm hard drives.

Isakovich said his company differs from many other SSD storage vendors in that it's not looking to be a supplement to traditional storage. His company is positioning itself to replace hard drive storage.

"We're not approaching this market to accelerate hot data or to be an appliance for high-end workloads. We're not a drive, or a card. Our hardware is 100% flash based, and it is complimented by a full OS," he said.

Nimbus competes against vendors such as Violin Memory and Texas Memory Systems, which builds storage arrays made entirely of SSD. Nimbus prices its product on a per-terabyte basis - it charges $10,000 per usable terabyte, according to Isakovich. Its 2U-high arrays, or "flash modules," come in 2.5TB, 5TB or 10TB capacities. Up to 25 modules can be clustered with a 6Gbit/sec SAS backbone.

Craft said eBay liked the Nimbus technology because it was simple to deploy and required less management.

"We're not a storage team. We're Windows administrators who got into virtualization, and of course then you have to get into storage. But the longer we're into storage, the more we want to get away from it," Craft said. "With this, we could just deploy it, flip it on and go."

Lucas Mearian covers storage, disaster recovery and business continuity, financial services infrastructure and health care IT for Computerworld. Follow Lucas on Twitter at @lucasmearian or subscribe to Lucas's RSS feed . His e-mail address is lmearian@computerworld.com.

Read more about Virtualization in Computerworld's Virtualization Topic Center.