BI & Search: A Marriage of Convenience

Search technology can make business intelligence -- both text and data -- more accessible.

Often, a company’s most valuable information is squirreled away in various departmental silos, and it takes an IT or statistics guru to coax it out. But what if workers could get that information — whether it’s in the form of data tables or text — using an interface that’s as easy to use as a Google search box?

That’s the thinking behind the marriage of business intelligence (BI) software and enterprise search technology. It’s an especially powerful concept for a decentralized organization such as the National Education Association (NEA), a teachers union with 3.2 million members and 14,000 state and local affiliates. “We wanted software that allowed people, particularly at the state-level organization, to create and edit their own reports,” says Bill Thompson, the NEA’s director of financial and membership services.

The organization is setting up Fast Radar, a BI/search product from Fast Search and Transfer SA, to give users access to financial, legal, research and membership information that is contained in four separate data silos. A small pilot is being conducted with a few affiliates, and broad deployment is scheduled to start in May. Initially, users will gain point-and-click access to information in the four databases. Later, they will be able to conduct text searches of the BI and text data.

Anticipation is running high. “Whenever you bring on a piece of software and people see its potential, they don’t want to wait,” says Thompson. “There is a growing number of people who say they want it now.”

While other aspects of IT have become more user-friendly, the creation of BI reports has remained an arcane specialty. In the past year, however, the technology that can change this finally became available. In April, Google Inc. released OneBox for Enterprise, which enabled Google’s search appliances to search structured data contained in databases and data warehouses in addition to text.

“The introduction of the Google OneBox API was a watershed event,” says Gartner Inc. analyst Whit Andrews. “We have seen the BI vendors approach search with real eagerness, and it is a rare BI vendor today who can’t tell you about an integration [of its BI software] to an enterprise search product.”

Cognos Inc., Information Builders Inc., Oracle Corp. and SAS Institute Inc. are among the BI vendors that have created links between their BI products and OneBox. In December, IBM and Yahoo Inc. released a free enterprise search application called IBM OmniFind Yahoo Edition, and Cognos announced a link between its Cognos 8 Go Search engine and OmniFind. Last month, Information Builders announced WebFocus Magnify, a search navigation tool that indexes structured data and provides BI reports within the search results.

Furthermore, Business Objects SA has a tool set coming out in the first half of 2007 that includes the ability to do text searches of its BI data. This month, Hyperion Solutions Corp. will unveil Hyperion System 9 Smart Search, which links to Google’s OneBox.

John Rome, Arizona State University’s director of data administration and data warehousing, is already using Google’s search appliance and Hyperion BI software. He also plans to deploy Smart Search once it’s available.

“When you put in a request, it brings you back live results or a list of reports or queries you can run that will get you the information you need,” he says. “If I want to find out about student retention rates or graduation rates, I can just put that in the search engine and get back the results, instead of building a query or duplicating what someone else has done.”

Joining BI and search technology provides two big benefits. One is the ability to combine the information contained in structured and unstructured data sources. That gives users a broader understanding of an issue. IDC analyst Sue Feldman says that creating a single point of access to all corporate information is one of the top three information management problems that companies are trying to solve.

“This problem has only become greater in the last couple years as enterprises realize their decision-making systems have often been ignoring the content side,” says Feldman. “Some of the most important parts of their business are in the unstructured or semistructured data — customer e-mails, text fields in CRM systems — which can help them know about impending problems and avoid product recalls.”

Carl Turza

Carl TurzaCarl Turza, CIO at Sigma-Aldrich Corp., a $1.7 billion pharmaceutical and chemical company, is using Endeca Technologies Inc.’s search software to give customers Web-based access to structured and unstructured data about the St. Louis-based company’s products.

Turza explains that some customers may need to know the certificate of origin and certificate of authenticity for pharmaceutical products, while others may also need information about drug or chemical safety issues. “If they need to initiate a separate search for both, I am wasting their time,” he says. “Instead, I can expose that data as a little icon or a little note they can click on if they want to open it, but I don’t have to increase their cognitive load.”

With this Endeca technology in place on Sigma-Aldrich’s Web site, Turza plans to deploy it internally by pointing the search engine at the data warehouse and other enterprise data stores. “We will see a proliferation of that technology throughout the calendar year, internally and externally,” he says.

The second advantage of the BI/search combo is that it makes BI more accessible. With the ability to access BI data through a simple text search, instead of writing a database query, staffers can get the data they need without waiting for BI specialists to craft a custom report.

“We are seeing a significant interest from clients in the notion of BI for the masses,” says Gartner’s Andrews. “They want to be able to get access to BI through a simpler process so that more people are able to solve the problems that BI addresses.”

Thompson plans to achieve both goals with the NEA’s rollout. Instead of requiring users to search each of the organization’s four data silos separately, he says, “we wanted a product that would allow us to pull from any one of those databases, merge the data and give us a look we didn’t have in the past.” With Fast Radar in place, he has already built some queries that go across three of the four categories. “If we get someone who has a legal case, we want to know if the person is a member and what we know about that member,” Thompson explains. “Or if we help affiliates with legal bills, we want to [connect the legal information] to the financial database.”

The NEA’s project is still in the pilot stage, but it’s already proving useful, he says. An executive at a state affiliate can look at a map of his state, click on a county, drill down into the local associations and analyze the membership types of an affiliate. Executives can also click to view a bar graph of the membership information and see where the local groups stand in meeting their financial goals.

“The drill-down capability is a tremendous plus,” says Thompson. “It is not only the availability of the data but the presentation that is helpful.”

Cautionary Notes

Turning a Google-like search engine loose on corporate information sounds easy. It isn't.

•  What looks like magic in vendor demos actually requires painstaking, behind-the-scenes work to map data queries to the data sets.•  Organizations need search algorithms that help employees find important reports within the company not necessarily algorithms based on popularity rankings.•  Different departments (finance or sales, for example) will have different search requirements.•  Access controls are required so that employees cant gain unauthorized access to sensitive information.•  Companies may already have a hodgepodge of search engines embedded in various applications and portals.•  Buyers will have to choose among various technology options, ranging from Googles search appliance to sophisticated text-mining tools that analyze unstructured data.•  There are numerous vendors with advanced search technologies born in the academic and government sectors that are in search of problems to solve in the business sector.

Source: Jim Murphy, AMR Research Inc.

Robb is a Computerworld contributing writer in Los Angeles.

Copyright © 2007 IDG Communications, Inc.

7 inconvenient truths about the hybrid work trend
Shop Tech Products at Amazon