Doubtful BI

By Mark Hall

Computerworld |

Google's "I'm Feeling Lucky" single-answer option is an apt metaphor for the appeal of business intelligence systems. When you can get the one and only right answer to a query, you do feel extraordinarily lucky because, in truth, there is seldom one correct response to any BI question.

Good BI simply points you in the right direction. Picture something like this: 52% of consumers who chose gray slacks also chose a white shirt, unless a blue sweater was a viable choice, then only 31% chose the white shirt. If you were a marketer with that information and you wanted to sell lots of shirts, you'd try to make darn sure blue sweaters were nowhere in sight for the consumer. But even then, you'd sell a white shirt only a little better than half of the time.

What happens when the shirt marketer learns he can push that 52% up to 67% when the shirt comes in blue? A lot of blue shirts will get produced. But what if he also learns that if the consumer is exposed to blue sweaters when the shirt available is blue, shirt purchases plummet to 14%? Well, now he's in a pickle, especially if he has no control over sweater availability.

If he's the gambling type, he'll probably go with the blue shirts and hope that sweater availability is limited. But if he's more conservative, he'll want more information before making the decision. Where's the overlap in shirt and sweater distribution? Does the color of the sweater matter? Does the time of day? Region of the country? The conservative BI user will keep peppering the system with queries until the best course of action reveals itself.

That approach seems logical because the more you know, the more decisive you will be. Right?

I don't necessarily agree. Rather, I take the side of the great German thinker Johann Wolfgang von Goethe (1749-1832), who observed, "Doubt grows with knowledge."

However, most end users expect just the opposite. They expect straightforward answers. Or, better still, they expect the single, true answer that will make them feel lucky just like Google does. When they don't get lucky, end users are likely to get cranky, confused and skeptical about the benefits of BI.

This wasn't a big problem when BI was used only by experts who could understand and apply the statistical rigor of software from, say, SAS Institute. But as our story "BI for the Masses" points out, BI tools are now being put into the hands of more and more workers. And, well, let's just say more than a few of us slept through our statistics classes in school.

More to the point, the masses don't understand the value of doubt. Yet, it's through the creation of doubt that BI truly performs its service to business. It forces us to question our conclusions, go back to the drawing board and develop new assumptions and conditions. It makes us think. It doesn't deliver the truth.

BI lets you query a broad range of related data from various points of view to get closer to an approximation of reality for a given set of conditions. That's a long way from delivering the truth. The masses of end users need to understand that before they start using BI. If they don't, they're more likely to make a lot of bad decisions that will prove costly to your company.

That makes BI training mandatory for end users. But not just training on how to use a tool. You need to train your workers about why to use BI. You want your BI tools to assist in making the best decision, not the right one.

We have too much information available to us to drill down to reach a single answer for a BI problem. And the more information we feed into our query, the less lucky we are likely to be.

In their book The Minding Organization (Wiley, 1999), Moshe Rubinstein and Iris Firstenberg relate that a European gentleman in the 17th century "was exposed to less information in his entire lifetime than there is today in a single edition of a daily newspaper."

That's probably what helped those gentlemen be so cocksure about their world views, which led to such pointless things as the Thirty Years War and powdered wigs. They didn't know enough to doubt what they thought they knew.

This, to me, is the genius of BI tools: not that they give you assurances about what you know, but that they inspire even more questions and more doubt.

Mark Hall is a Computerworld editor at large. Contact him at mark_hall@computerworld.com.

The Future of BI

Stories in this report:

Business Intelligence

Bing’s AI chatbot came to work for me. I had to fire it.