Overcoming Business Skepticism
While hiring a consultant can help with some of the technical details, that's only part of the challenge, says John Elder, a principal at Elder Research, a consultancy that worked with Jones and his team. "Over 16 years, we have solved over 90% of the technical problems we've been asked to help with, but only 65% of the solutions have gone on to be implemented."
The problem, generally, is that the people that the model is intended to help don't use it. "We technical people have to do a better job making the business case for the model and showing the payoff," Elder says.
Persuading decision-makers to use the results can be as difficult as getting them to go along with the project in the first place, because the predictions may be the exact opposite of what their business intuition tells them, says Anne Robinson, president-elect of the Institute for Operations Research and the Management Sciences (Informs), a professional society for business analytics. "As you get more involved with analytics, it becomes counterintuitive. But it's those deviations from what you're doing that bring the rewards, because when the results are intuitive, you find that most people are already doing them."
Several years ago, Cisco Systems created "propensity to buy" models that were designed to help calculate the probability that customers would buy this quarter, next quarter or never. The models cover every product in every sales territory. The salespeople felt they already knew what some of the people identified by the model were going to buy, so Cisco excluded those sales when calculating the return on its effort. "The first year we did it, we generated $1 billion in sales uplift," says Theresa Kushner, Cisco's senior director of customer and influencer intelligence. "We had an experience to line up against what they thought they believed."
Ultimately, predictive analytics is forcing a showdown between data-driven and intuition-based decision-making, says Eric Siegel, president of Prediction Impact, an analytics training firm and conference organizer. "That's the big ideological battle. It's a religious debate."
Data: Getting to Good Enough
On the technology side, both building the model and preparing the data can be stumbling blocks. Predictive analytics is an art as well as a science, and it takes time and effort to build that first model and get the data right, says Abbott. "But once you build the first one, the next one is much less expensive to model" -- assuming you're using the same data. Analysts building an entirely different model with new data might find the second project just as time-consuming as the first. Nonetheless, he says, "the more experience one gains, the faster the process becomes."
Data preparation issues can quickly derail a project, says Siegel. "Software vendors skip that point," he says, noting that "all of the data in a demo has already been put into the correct format. They don't get into it because it's the biggest obstacle on the technical side of project execution -- and it can't be automated. It's a programming job."
When Perez started the Orlando Magic's predictive analytics initiative in 2010, he miscalculated the time it would take to prepare the data. "All of us were thinking that it would be easier than it was," he says. Pulling data from Ticketmaster, concession vendors and other business partners into a data warehouse took much longer than anticipated. "We went almost the entire season without a fully functional data warehouse. The biggest thing we learned was that this really requires patience," he says.
"Everyone is embarrassed about the quality of their data," says Elder, but waiting until all of the data is cleaned up is also a mistake. Usually, he says, the data that really matters is in pretty good shape.
Iterate First, Scale Later
At Intuit, every project starts small and goes through cycles of improvement. "That's our process: iterative and driven by small scale before going big," says George Roumeliotis, data science team leader. The financial services company started using predictive analytics to optimize its marketing and upsell efforts, and now focuses on optimizing customers' experiences with its products.
Intuit developed predictive task algorithms to anticipate how users will categorize financial transactions in products such as Mint and QuickBooks. Based on the results of those algorithms, Intuit applications make suggestions as users enter new transactions. They also anticipate questions users might have and proactively provide content and advice that could help them.
"Start with a clearly articulated business outcome, formulate a hypothesis about how the process will contribute to that outcome, and then create an experiment," says Roumeliotis. Through A/B testing, analysts can gain the confidence of business leaders by creating parallel business processes and demonstrating a measurable improvement in outcomes.
Just be sure to start by choosing an existing business process that can be optimized with minimal risk to the business, he advises. Customer support, retention and user experience are great places to get started.
While predictive analytics projects can require a substantial investment up front, studies indicate that they can deliver positive returns on investment, as Cisco's experience shows. Ultimately, even small-scale projects can have an enormous impact on the bottom line. "Predictive analytics is about projecting forward and transforming the company," says Peri.
The risks are high, but so are the rewards, says Robinson. "Take it to the end," she says. "Be successful. And act on what you learn."
This version of this story was originally published in Computerworld's print edition. It was adapted from an article that appeared earlier on Computerworld.com.
Read more about applications in Computerworld's Applications Topic Center.