A latency-killing, cloud storage network is coming to a city near you

ClearSky Data says its innovations have made it possible for companies to store data needed in real time in the cloud.

cloud storage

There have been a lot of innovations in enterprise storage in recent years and months -- much of it focused on driving storage to the public cloud to save costs.

But what storage vendors haven’t been able to truly crack is the latency factor associated with moving the data back and forth from the cloud to the enterprise. That, and unpredictable internet transport performance, have limited companies’ use of the public cloud as a primary storage location. As for storing data needed by a company in real-time in the cloud, it is out of the question.

This is where Boston-based ClearSky Data is hoping to make its mark. The startup, which just launched a few months ago out of stealth mode, has developed a cloud-based storage network system that it promises has solved this issue.

The company refers to network as on-demand storage or storage provided as a fully-managed service, but personally I think its traction would accelerate if it branded itself as storage’s latency killer.

A look at what is happening in enterprise storage puts that thought in better context.

A sign of the times

After years of little change, the industry is now making a shift. Sales of server-based storage rose 10 percent to reach $2.1 billion in the second quarter, according to the latest global IDC report on enterprise storage systems. 

But while traditional external systems like SANs (storage area networks) are still the biggest part of the enterprise storage business, sales in this segment were down 3.9 percent.

Guess where companies were increasing their spending? In new technologies, such as cloud-based storage, integrated systems, software-defined storage, and flash-optimized storage system.

Drilling into this trend a little further, you can see that many of these innovations are all about moving the stored data around in response to the enterprise’s needs, managing data’s life cycle and smart provisioning of networking and other resources as needed.

Recently, to cite one example, EMC introduced automatic tiering of data from its VMAX and VNX systems to public clouds. 

ClearSky Data’s Tiered Caching Breakthrough

ClearSky’s contribution to these developments is a process called Smart Tiered Caching (STC). STC continually categorizes data, rotating it among the appropriate cache layers of hot, warm and cold. The hot cache is where the most essential data for the company resides. It also handles the security and can integrate with VMware

It sits as close to the company’s applications as possible.

The warm cache resides in ClearSky’s own point of presence, of which there are currently three. This PoP must be located within 120 miles of the user, so at the moment ClearSky’s reach is limited.

The cold cache sits at the edge of the public cloud.

In the company's data center, ClearSky embeds a flash-based storage appliance that moves the data among the various cache layers. A low-latency gigabit metro Ethernet connects this storage appliance to Clearsky’s PoP.

The resulting service, according to ClearSky, delivers hundreds of thousands of input/output operations per second (IOPs) and less than two milliseconds of latency. As of the beginning of November, the company did not have any customers that had gone live, so first-person validation -- outside of the prepared statements by the early adopters -- will have to wait. 

Coming to a city near you

Earlier this month ClearSky secured $27 million in a Series B investment round. It plans to use the proceeds to open PoPs in other major cities in the U.S. -- currently there are PoPs in Boston, Philadelphia and Las Vegas -- and then throughout North America.

All of the major cities are targets, ClearSky co-founder Ellen Rubin told me. It's not a question of "when" and "where", but "in what order" will ClearSky open a local PoP, she said.

That funding round, which was led by Polaris Partners, with a strategic investment from Akamai Technologies brings ClearSky's total funding haul to $39 million. That's not bad for a start up, but then Rubin has some serious creds to her name, including an MBA from Harvard Business School and an undergraduate degree from Harvard College.

Perhaps more to the point, though, Rubin also was involved in two tech game changers. One was when she was at Netezza, which was later acquired by IBM. Her second go-round at disruption came when she was founder of CloudSwitch, which was eventually snapped up by Verizon.

Netezza helped established the warehouse appliance product category and CloudSwitch developed a hybrid-cloud management approach to migration, networking and security -- long before the hybrid cloud was viewed as mainstream.

Now she's trying to do the same thing for enterprise storage -- disrupt it, that is, in this case with the premise that storage can also be provided as a fast and cost-efficient on-demand offering.

A tough story to sell?

It sounds like a great story, except for one thing: It can be a tough sell, especially to ClearSky’s Fortune 1000 target client base.

"If you say to somebody, 'we have a global storage network', they don’t know what that means," Rubin said.

"So we tell them, 'we are offering you a managed-service, on-demand approach to store your primary data in the cloud and it gives you the same capacity and security as an on-premise storage array without the headaches and costs.'"

And that's when the "it's too good to be true" antennas come out.

Technology is, after all, a market-soaked world and more than a few companies have gotten burned by promises made by new companies. Also, there is the fact that ClearSky is a start up. If it goes away, how much work will be required to migrate the data somewhere else? Exactly.

But those are business problems that every start up must overcome. It gets worse when the company is introducing a new concept.

Caching versus tiering

ClearSky CTO and co-founder Laz Vekiarides -- who has his own impressive CV, including his last position of executive director of Software Engineering for Dell’s EqualLogic Storage Engineering group -- makes this point in a recent blog post.

It seems that some people are confusing caching and tiering, even in technical conversations. This is frustrating for Vekiarides as a lot of people have been disappointed with auto-tiering technology.

He writes:

In data storage, a tier is a permanent, protected home for a piece of data. It is really expensive and complex to have multiple copies of the same data on different tiers in a system, so when the system decides certain data needs to be in a faster tier for performance reasons, it needs to move it. To do that, it needs to move some other data to a slower tier to make room. A system does this while servicing all of the data requests from users and their servers. It’s a lot of work and it takes up a lot of bandwidth in the system. In fact, this churn can double or triple the workload that needs to be handled. To prevent this from being a problem, data moves across tiers very deliberately, or sometimes not at all.

ClearSky, by contrast, has built its network on advanced caching technology and optimization, he continued, which "results in more efficient utilization of expensive resources, which positively lowers the cost of operating data storage with our service."

Maybe, Vekiarides should focus on ClearSky’s 2-millisecond latency lapse -- a benchmark cited in its service-level agreement -- when pitching the concept. That, apparently, resonates with a lot of companies. In a separate blog post, Rubin recently noted that many companies would love to use the cloud for storage management, but they find the high latency unacceptable.

“Based on our discussions with industry analysts, we’ve heard that as many as 50 percent of cloud customers have brought workloads back on-prem due to latency and performance issues,” she wrote.

The title of the post? “Latency: The hidden killer of cloud adoption.”

This article is published as part of the IDG Contributor Network. Want to Join?

To express your thoughts on Computerworld content, visit Computerworld's Facebook page, LinkedIn page and Twitter stream.
Windows 10 annoyances and solutions
Shop Tech Products at Amazon
Notice to our Readers
We're now using social media to take your comments and feedback. Learn more about this here.