The scale of data growth today is so massive it can be numbing. According to Gartner*, in the last minute there were 204 million emails sent, 61,000 hours of music listened to on Pandora, 20 million photo views and 3 million uploads to Flickr, 100,000 tweets, 6 million views and 277,000 Facebook logins, and 2 million plus Google searches.
Every day, more iPhones are sold than babies born.
Data is continuing to grow at a phenomenal pace. According to IDC’s Predictions 2013: Competing on the 3rd Platform report** the “digital universe”, which is the total of all digital data created and replicated will reach 4 zettabytes in 2013, almost 50 percent more than 2012 volumes.
These figures are staggering. Candidly, almost incomprehensible!
While these numbers reflect the growth of data on a global scale, the story is the same if you look at enterprise organizations. The average enterprise sees data growth of 40-50 percent each year.
So, what’s driving this incredible growth in data or digital content?
Companies are opening their electronic doors to more interactions with customers, partners and supply chain vendors, which is accelerating data growth on a minute-by-minute basis. Universal access to mobile devices and the ability to access services via the cloud means that more applications are available to many more users than ever before. By default, this creates increased volumes of data. With the consumerization of IT – or BYOD – IT now has to find a way to manage a growing amount of data and find a way to separate the personal data that lives on these devices from the company’s proprietary data. Finally, regulations require companies in many industries to hold onto data for many years. In fact, I believe more highly regulated industries see higher than average data growth rates.
The growth of data also provides an opportunity for organizations to analyze the information being gathered and, if done right, turn it into a competitive advantage. Think about a telecommunications company who by law needs to keep records of every cell phone and text sent over their network. They can analyze traffic patterns to tell where they might need additional network capacity such as cell phone towers. And, they can leverage the insights from the data to see where and when they might need more capacity on their IT infrastructure. Ideally, organizations can mine their data to improve customer service levels and revenue opportunities.
Solving the data challenge with technology and policies
There have been many technological advances that have helped the enterprise deal with increasing data volumes. For one, data deduplication has been a game-changer. This technology has been able to eliminate dozens and dozens of redundant copies while reducing the sheer amount of data that must be managed.
Deduplication went from being a niche technology to a very large business in no time. Its relevance can be demonstrated by the fact that today, no storage vendor offers a product that does not include deduplication.
Versioning and indexing technologies are also getting smarter, helping organizations manage versions across users and, ultimately, reducing the number of instances of a file. Similar to deduplication, this technology has become a requirement in all modern operating systems.
Having thoughtful policies and strategies in place are also important to managing data growth. Organizations need to categorize their information and determine what data should be kept and for how long.
Ensuring a scalable infrastructure to meet demand
Many of the trends that accelerate data growth simultaneously put enormous strain on the underlying IT infrastructure. With more people online at all times of the day – and night – IT has to manage more workloads and process more data than ever. As a result, IT is managing a larger number of devices than ever before. Data centers are doubling or tripling in size as IT struggles to keep up.
Having an IT infrastructure that can scale effectively to meet the demands of more users and more data is essential. Many in the enterprise are turning to the cloud to help them gain flexibility, efficiency and speed.
Cloud computing can certainly help IT scale its infrastructure. It’s also important that IT pay close attention to the performance of the infrastructure as a whole and put in place solutions that enable them to quickly identify and remediate issues to ensure the availability of the data and applications.
It doesn’t look like data growth is going to slow down any time soon so it will take a combination of the right technologies and processes to address it. How IT steps up to meet the challenge will make the difference between agile, growing companies and those that get left behind.
*Gartner Symposium/ITxpo Presentation, Ten Critical Trends and Technologies Impacting IT During the Next Five Years, D. Cappuccio, October 2012.
** IDC Predictions 2013: Competing on the 3rd Platform, Doc #238044, November 2012.