BI's Power Couple

The marriage of BI and text analytics promises to give deeper meaning to BI data.

The marriage of business intelligence and text analytics is starting to have a profound impact on companies in several industries, including health care, insurance and finance, which are just waking up to the benefits of tying structured BI data to unstructured text.

Text analytics tools use linguistics, rules-based natural-language processing, specialized algorithms and other methods to impose order on unstructured text scattered throughout the enterprise. More IT executives are using text analytics software to mine disparate document- management applications, e-mail and phone systems, or even blogs and Web sites.

The goal is to breathe new life into static BI reports. By extracting facts, concepts and data relationships buried in text, text analytics software transforms this unstructured information into modeled data that can then be tied to BI databases. Hence, text analytics promises to enhance the context and meaning of BI data, which is often presented as canned reports scraped from data warehouses or major applications, such as ERP and customer relationship management (CRM) databases.

Though powerful, the combination of text analytics and BI isnt yet typical. Most people associate business intelligence with online analytical processing [OLAP], which focuses on structured data, as far as the process and user interface are concerned, says Boris Evelson, an analyst at Forrester Research Inc. in Cambridge, Mass.

However, to become more effective, OLAP experiences need to bring unstructured data into the analysis in a seamless way that is transparent to the user, he says.

Indeed, despite spending bundles to build sophisticated BI databases, many corporate IT officials find that a lot of vital data stays locked up as text throughout the enterprise, notes David OConnell, an analyst at Nucleus Research Inc. in Wellesley, Mass.

Within this data is important competitive, marketing, sales campaign and CRM trend data. However, you can only find and track these trends by automating analysis and combining it with BI, says OConnell. By bolting text analytics onto traditional BI applications a process that is not terribly expensive, since little data cleansing is necessary the value of BI efforts is extended. Eventually, companies get new ROI on existing BI investments.

Order, Please

BlueCross BlueShield of Tennessee Inc. (BCBS) provides a good example of the benefits of extending BI through text analytics. BCBS has successfully linked the two technologies to hone analysis of the costs of insuring high-risk and low-risk members in four disease categories.

By combining related structured and unstructured data, we were able to deliver new business insight, enable new forms of analysis and present actionable information to users in the form of enhanced BI, says Frank Brooks, chief data architect and senior manager of data resources and management at the Chattanooga-based insurance provider.

Fueling the BCBS system is Cognos 8 BI Version 8.2 from Ottawa, Ontario-based Cognos Inc. and two text analytics tools: Text Miner from Cary, N.C.-based SAS Institute Inc. and IBMs Omni­Find Analytics Edition. Both text analytics tools are playing big roles in a BCBS proof-of-concept application. [That application] has demonstrated the power of transforming the meaning hidden in unstructured data with the meaning in existing structured data, says Brooks.

SASs Text Miner manipulates data contained in several file types PDF, ASCII, HTML and Microsoft Word and renders text as numerical representation using Singular Value Decomposition technology. These numerical models are packaged to reside in BI clients, including Microsoft Excel and SASs many BI offerings.

IBMs text analytics offerings mostly center on the Unstructured Information Management Architecture, a product of the companys research division. UIMA uses core algorithms to perform the language proc­essing needed to transform unstructured text into components that can be integrated with middleware and systems like WebSphere Portal Server and Lotus Workplace that often host enterprise BI applications.

Along with insurance giants such as BCBS, financial services firms are also ripe for combined BI-text analytics applications. For instance, text analytics is applicable to areas such as risk management, according to a recent report by Forrester. The report gives an example in which an antifraud professional at a major financial institution used the two technologies to generate watch lists and compile legal discovery documentation that would have been impossible to gather through manual association of data sets.

Financial planning systems provider Kettley Publishing Co. in Newport Beach, Calif., has combined BI and text analytics capabilities to allow its customer base of financial planners to access the most relevant content. BI and analytics turns noise into a form that can actually support and defend decisions, says Jim Connolly, Kettleys director of development.

Kettley developed its text analytics capabilities in-house using Microsoft Windows Workflow Foundation programming model to whip text into shape an exercise that would serve as a forerunner to stepped-up enterprise search capabilities. The implementation went smoothly and took less than one person in one months time to implement, says Connolly.

As software vendors scramble to add text analysis functionality to their BI portfolios, systems integrators will be the first to capitalize on corporate interest in the combined systems, says Forresters Evelson.

This is still very much an integration game, he says. So in addition to investment in software, one needs to budget at least $3 to $5 on systems integration for every $1 spent on software.

McAdams is a freelance writer in Vienna, Va. Contact her at

Copyright © 2007 IDG Communications, Inc.

7 inconvenient truths about the hybrid work trend
Shop Tech Products at Amazon