Opinion: Demystifying de-duplication
De-duplication can be applied anywhere there is a significant amount of data commonality
Computerworld - Of the assortment of technologies swarming around the storage and data-protection space these days, one that can be counted on to garner both lots of interest and lots of questions among users is de-duplication. The interest is understandable, since the potential value proposition, in terms of reduction of required storage capacity, is at least conceptually on a par with the ROI of server virtualization. The win-win proposition of providing better services (e.g., disk-based recovery) while reducing costs is undeniably attractive.
However, while the benefits are obvious, the road to get there isn't necessarily as clear. How does one make a decision to adopt a particular technology when that technology manifests itself in so many different forms? De-duplication, like compression before it, can be incorporated into a number of different products types. While by no means a complete list, the major options for our purposes include backup software, NAS storage devices, and virtual tape libraries.
Even within these few categories, there are dramatic differences in how de-duplication is implemented, with each offering having its own benefits. The scorecard of feature trade-offs includes the following:
- Source vs. target de-duplication
- Inline vs. postprocessing
- Global vs. local span
- Single- vs. multiple-head processing
- Indexing methodology
- Level of granularity
As with any set of products, these trade-offs reflect optimization for specific design or market targets: high performance, low cost, enterprise, SMB, etc. For more detail on the range of de-duplication options and their implications, you may want to check out my colleague Curtis Preston's Backup Central blog.
Until recently, one aspect of de-duplication that was generally unquestioned was its focus: secondary data, particularly backup. However, there are growing signs that this too is changing. In theory, de-duplication can be applied anywhere there is a significant amount of data commonality — which is why backup is such a good fit.
However, if we look around for more examples of high data commonality, one area that comes to mind is virtualized server environments. Consider the number of nearly identical virtual C: drives in a VMware server cluster, for example. Recently, NetApp has been leading the way among storage vendors in suggesting de-duplication for primary storage in these environments. In fact, it has been steadily expanding its support of de-duplication, initially offering the technology on its secondary NearStore platforms, then on its primary FAS line, and as of last week on its V-Series NAS gateways, where it can de-duplicate the likes of EMC, HDS, HP and other storage.
Of course, for many, this is uncharted territory and the performance and management impact needs to be better understood. But given the higher costs of primary storage vs. secondary, the potential to achieve a 20:1 savings in storage, even for just a portion of the environment, is quite tempting.
Read more about Networked Storage in Computerworld's Networked Storage Topic Center.
- Best iPhone, iPad Business Apps for 2014
- 14 Tech Conventions You Should Attend in 2014
- 10 Desktop Apps to Power Your Windows PC
- How to Add New Job Skills Without Going Back to School
- Slideshow: 7 security mistakes people make with their mobile device
- iOS vs. Android: Which is more secure?
- 11 sure signs you've been hacked
- Gartner 2013 Magic Quadrant for Enterprise Backup/Recovery Software See why CommVault was positioned as the #1 leader in Gartner's 2013 Magic Quadrant for Enterprise Backup/Recovery software for the 3rd year in...
- Forrester Report: CommVault is a Leader in Enterprise Backup and Recovery In this report, Forrester takes a deep dive into the evaluation criteria, how CommVault is positioned and the features and functionality that make...
- Forrester Wave for Enterprise Backup and Recovery Read this report to see how CommVault continues to outpace its competitors and why Forrester positioned CommVault Simpana as the top backup and...
- Architecting the Network of the Future Networks need to change, as does the way IT thinks about and manages them. In addition to reliability, IT must now add higher...
- Four Myths of High-Productivity App Dev Debunked Debunk the main myths surrounding high-productivity application development and how both platforms have overcome them.
On-Demand Webcast: 7 Reasons to Choose VoIP
Thinking about a new phone system for your business?
Be sure to watch this informative webcast. Steve Strauss, small business columnist for USA...
All Networked Storage White Papers |