It’s no big news that big data initiatives have huge potential business value. What’s concerning is that so many projects fail. In fact, a mere 8% of executives describe their big data projects as “very successful” according to a recent Capgemini report.
Before you embark on your big data initiative, it’s important to understand why some big data initiatives fail so you can avoid the pitfalls.
Don’t Let Your Data Lake Become a Data Swamp
In many cases, big data initiatives fail to deliver business value because the data lake resembles a swamp. That’s how Barbara Latulippe, Chief Data Governance Officer at EMC, described their data lake before they mastered the data in it.
Why is big data so often murky? It’s murky because early big data hype drove organizations to just capture and dump as much data as they could onto Hadoop. They didn’t invest in the data management capabilities they would need to do something valuable with all that data—they felt they could tackle that problem “tomorrow.”
Except “tomorrow” is today! And, a lack of understanding is costing companies a lot of money. EMC calculated its swampy data was costing them $40 million per year in lost opportunities. What’s behind those costs?
- Lack of insight into customer habits and preferences
- Low marketing campaign effectiveness
- Poor conversion rates
- Lack of sales integration to drive upsell and cross-sell
- A disjointed customer experience
- Longer time-to-resolution for customer problems
- Weak customer loyalty
The reason data lakes aren’t delivering business value is because they are full of redundancies, conflicts, and errors. The complete set of core data about any given customer (or partner, supplier, product, asset, and so on) hasn’t been consolidated into a single master profile. Until you have mastered trusted, business-critical data in your data lake, you can’t uncover the business insights or gain the business value you expect.
Let’s take a look back at the history of how master data management (MDM) evolved and how it helps turn big data into big value.
MDM’s Value Journey
Fueling the Data Warehouse With Great Data for Reporting
MDM originated from a need to improve the accuracy of reporting by resolving duplicates and managing the history and lineage of data from multiple applications.
Once mastered, the data would be fed into the data warehouse. This was valuable for both compliance and operational reporting.
For example, MDM ensured that senior executives at complex organizations received an accurate list of the top 500 customers across lines of business, channels, and regions. Business analysts spent less time manually reconciling data in reports. Take General Electric, for example, which could be represented inconsistently across source applications: GE, General Electric, and GE Corporation.
Fueling Business Applications With Great Data In Real Time
People soon realizedthey could use master data in business applications, providing trusted data in real time.Why not fuel the CRM or support application with that mastered customer data about General Electric? Now the sales and service teams have the most accurate and up-to-date customer data in the applications they use every day.
But the use of MDM hasn’t stopped there.
Connecting the Dots for Big Data Analytics
The newest use of MDM is to provide much needed context for big data. MDM can deliver the same benefits in the data lake that it has delivered for years in more traditional data stores: eliminating inconsistency, resolving duplicates, and creating a single version of the truth. It also manages the relationships between master data.
There are two ways to master data in a big data lake:
1. Feeding mastered data into the lake from the MDM hub
2. Mastering data in the data lake itself
In the first approach, companies use an MDM hub to master the data. The MDM hub improves the quality of core data that is fed into the data lake.
EMC uses this traditional approach. They created a customer hub that serves as the single trusted source for all customer data, which includes the relationships between customer accounts and contacts. Trusted data flows from the hub to the data lake as well as every application and customer touch point, and fuels the Total Customer Experience initiative that’s at the heart of EMC’s operations.
But there is another option for companies that have an extraordinarily high number of records. They can master the data within the data lake itself. This frees up data scientists to spend more time exploring and analyzing and less time trying to fix data issues, such as duplicate customer records. It also helps data scientists understand the relationships between the data; to see all members of a household, for example.
Following Data to New Insights
The data lake is all about exploring data and finding new opportunities—and answering questions, such as:
- Why did this customer buy?
- What specific interactions and events led to conversion and revenue?
- What prospects have the same characteristics as our best customers?
“Why did this customer buy?” was precisely the question that the CMO of a leading insurance company needed to answer, which drove the company’s master data initiative for its data lake. The company wanted a 360-degree view of customer data as well as prospect data—totaling 20 million unique persons and households. All told, the company managed data from 40+ source systems, totaling 1.2 billion records and 30 TB of data.
By mastering its data in the data lake, the insurer was able to add context from relationship, social, and other third-party data, which improved the quality of customer segmentation. This additional context removed the swampy, murky conditions, making it easy for the company to get actionable insights from their data.
You Can’t Get Business Value Without Strategically Managing Your Data
If you are embarking on a big data initiative and you want to deliver business value, you can’t afford to just dump data onto Hadoop. Learn from others’ mistakes. Take a data-first approach. Invest in the data management capabilities you’ll need to get actionable insights. Only MDM can deliver the trusted data you need for your data lake to deliver business value.
Learn more during this webinar, Turn Big Data into Big Value with MDM on Tuesday, June 7, 2016, at 8:00 am PT.