12 predictive analytics screw-ups

Make these mistakes and you won't need an algorithm to predict the outcome

1 2 3 4 5 6 Page 3
Page 3 of 6

6. Don't just proceed, but rush the process because you know your data is perfect.

Between 60% and 80% of the time spent on a new predictive analytics project is consumed by preparing the data, according to Elder Research. Analysts have to pull data from various sources, combine tables, roll things up and aggregate, and that process can take as much as a year to get everything right. Some organizations are absolutely confident that their data is pristine, but Abbott says he's never seen an organization with perfect data. Unexpected issues always crop up.

Source: Flickr/Sebastian Mary.

Consider the case of the pharmaceutical business that hired Elder Research for a project, but balked at the time allocated for data work and insisted on speeding up the schedule. The consultancy relented, and the project moved forward with a shortened schedule and smaller budget. But soon after the project started, the firm discovered a problem: The ship dates for some orders preceded the dates when the orders had been called in. "Those weren't problems we couldn't overcome, but they took time to fix," Deal says -- time that was no longer in the budget.

Once he pointed out the issue, the executive realized there was a problem and had to go back to the management team to explain why the project was going to take longer. "It became a credibility issue for him at that point," Deal says. Lesson learned: No matter how good you think your data is, expect problems: It's better to set expectations conservatively and then exceed them.

7. Start big, with a high-profile project that will rock their world.

   Mick Jones
Mick Jones, co-founder of the British punk rock band The Clash, in 2011. Source: REUTERS/Mario Anzuoni.

A large pharmaceutical company had grandiose plans that it thought were too big to fail. As it began to build an internal predictive analytics service, the team decided to do something that would "revolutionize the health care industry," Deal recalls them proclaiming in an initial meeting.

But the project's goals were just too big and required too large of an investment to pull off -- especially for a new team. "If you don't see results quickly you don't have anything to encourage you to maintain that level of investment," he says.

Eventually the project collapsed under the weight of its own ambitions. So don't swing for the fences, especially your first time at bat. "Set small, realistic goals, succeed with those and begin to build from there," Deal advises.

8. Ignore the subject matter experts when building your model.

It's a common misconception that to create a great predictive model you simply insert your data into a black box and turn the crank -- and accurate predictive models just pop out. But data mining experts who take the data, go away and come back with a model usually end up with flawed results.

   Giant black box
Source: Flickr/Abode of Chaos.

That's what happened at a computer repair business that worked with Abbott Analytics. The business wanted to predict which parts a technician should bring for each service call based on the text description of the problem from the customer call record.

"It's hard to pull out key concepts from text in a way that's useful for predictive modeling because language is so ambiguous," Abbott says. The business needed a 90% accuracy rate in predicting a parts requirement, and the first models attempted to make predictions based on certain keywords that appeared in the text. "We created a variable for each keyword and populated it with a "1" or "0" indicating the existence of that keyword in the particular problem ticket," which included the text of the customer call.

"We failed miserably," Abbott says.

1 2 3 4 5 6 Page 3
Page 3 of 6
It’s time to break the ChatGPT habit
Shop Tech Products at Amazon