Finding the business value in big data is a big problem
Gathering data isn't the issue; what to do with all that data is the challenge, say IT executives
Computerworld - PHOENIX -- For all the promise that big data holds, the fundamental challenge with collecting massive volumes of data from different sources is finding new business uses for it, according to several IT managers at Computerworld's BI & Analytics Perspectives event here this week.
Technology vendors and industry analysts tout the enormous business benefits that enterprises can gain from mashing up traditional structured data with unstructured data from the cloud, mobile devices, social media channels and other sources. But business executives have little idea of how to take advantage of big data or how to articulate their requirements to IT, according to several executives at the show.
Business leaders often "don't know what they don't know," said one frustrated IT manager, and therefore they are incapable of explaining to IT shops what to do with all this data that's being accumulated.
Over the past couple of years, private investors and venture capital firms have poured hundreds of millions of dollars into startups developing new technologies for collecting, storing, organizing and analyzing petabyte-scale volumes of structured and unstructured data.
The tools have made it easier than ever for companies to pull in data from Web logs, clickstreams, social media, video and audio files, machine sensors and microblogging sites such as Twitter.
The real challenge isn't using the technology -- it's finding the business value hidden in all of the data that can be collected, said Reid Nuttall, CIO of OGE Energy, an Oklahoma City-based energy company.
OGE owns nine power plants and delivers power to more than 758,000 customers in a 30,000-square-mile area. The company recently installed smart meters across its customer base that provide readings in two-hour increments, compared to the once-a-month readings OGE previously received.
Nuttall sad he's optimistic that the large volume of data generated by the smart meters can help OGE analyze and influence customer behavior and reduce peak demand over the next few years. He is looking for people within his organization to start extracting this kind of business value from the data.
"We have lots of data, and we are figuring out what to do with it," he said.
Nuttall set up an information "factory" and a business analyst competency center inside the organization to help spur creative uses of the data at OGE's disposal. OGE is investing in business intelligence tools and new data visualization and presentation capabilities to help analysts think about, and use, the data in new and different ways.
"Big data is forcing IT and business intelligence [teams] together" to find ways of exploring new data together, he said.
Payroll processor ADP is taking the same approach. The company has set up an innovation lab to manage the way it stores, processes and analyzes extremely large data sets.
The idea is to create an environment where subject matter experts from different industries and backgrounds can work together to tackle big data analytics, Roberto Masiero, vice president of ADP's innovation labs, said in a keynote address.
In a sense, what is happening now is reminiscent of what went on when enterprises first started using online analytical processing (OLAP) tools, said William Herridge, managing director of emerging solutions at the Tribune Company.
"When we made the transition to OLAP, it was hard to get business users to get over their [existing] mindset," of using tabular data, he said. "They didn't have any idea of the value of OLAP till you started showing them." IT organizations face the same challenge with big data, he added.
"We see the value in this, but getting users to understand that value" is a huge challenge, especially when dealing with unstructured data, Herridge said. "Until business users can see some benefits, they are not going to sign on to big data projects."
The hardest part of using big data is trying to get business people to sit down and define what they want out of the huge amount of unstructured and semistructured data that is available to enterprises these days, said Vivek Ratna, a partner with Digital Learning Solutions in Irving, Texas.
"The fault is ours, because IT has not articulated as well as we should have what value business can derive" from big data, Ratna said. Many IT organizations haven't begun to collect or use unstructured data because they're unsure of the business value -- not because they don't know how to use the technology, he said.
"Unless we can define what value can be derived [from big data] or the business leaders can tell us what value they want to get out of it, we are just playing in the dark," he said.
The sentiments are consistent with those expressed by respondents in a recent survey by market research firm TheInfoPro. The survey of 255 IT professionals showed that a majority of companies had no big data plans because they didn't have a specific business case for using big data.
Jaikumar Vijayan covers data security and privacy issues, financial services security and e-voting for Computerworld. Follow Jaikumar on Twitter at @jaivijayan, or subscribe to Jaikumar's RSS feed . His email address is email@example.com.
BI and analytics
- Big data key to bringing hyperlocal weather forecasts to Georgia farmers
- Brewer taps Bud Lab at University of Illinois
- Splunk woos Hadoop users
- RSA brings big data analytics to security threat management
- Moving beyond Hadoop for big data needs
- Q&A: What's needed to get a big data job?
- SAS extends analytics support for unstructured data
- Time has come for chief analytics officers
- Big data brings big academic opportunities
- Finding the business value in big data is a big problem
Read more about Big Data in Computerworld's Big Data Topic Center.
- Top 3 Myths about Big Data Security : Debunking common misconceptions about big data security Big data represents massive business possibilities and competitive advantage for organizations that are able to harness and use that information. But how are...
- Magic Quadrant for Data Masking Technology IBM is a leader in Gartner Inc's Magic Quadrant for Data Masking Technology. Read the full report to learn about IBM.
- Best Practices for Securing Hadoop Historically, Apache Hadoop has provided limited security capabilities. To protect sensitive data being stored and analyzed in Hadoop, security architects should use a...
- Top Tips for Securing Big Data Environments: Why Big Data Doesn't Have to Mean Big Security Challenges Organizations must come to terms with the security challenges they introduce. As big data environments ingest more data, organizations will face significant risks...
- Live Webcast Charting Your Analytical Future - "Making predictive analytics part of your business processes" Webinar This session will show how predictive analytics can be used throughout the organization by anyone looking for answers and how organizations can make...
- Charting Your Analytical Future - "Making predictive analytics part of your business processes" Webinar This session will show how predictive analytics can be used throughout the organization by anyone looking for answers and how organizations can make...
- Improved Data-centric Application Development and Hadoop Operations with BMC and Hortonworks Join this webinar to hear from BMC and Hortonworks how their combined solutions help customers unlock the value of Big Data by implementing... All Big Data White Papers | Webcasts
Our new bimonthly Internet of Things newsletter helps you keep pace with the rapidly evolving technologies, trends and developments related to the IoT. Subscribe now and stay up to date!