ClearForest: Scaling Dow's Paper Mountain

Computerworld -

ClearForest Corp.

CATEGORY: Data management

LOCATION: New York

TECHNOLOGY: ClearTags

HOW IT WORKS: ClearForest's technology assimilates text data of any size and structure, extracts key terms, assigns the terms to meaningful categories (a taxonomy) and establishes their interrelationships. By combining semantic, statistical and structural analysis, ClearTags automatically classifies documents and discovers pertinent entities, facts, events and relationships buried deep within the text.

CUSTOMER Sampling: Credit Suisse Group, General Motors Co., Dow Chemical Co., Eastman Kodak Co.

TIP: "Get ugly early," says ClearForest CEO Barak Pridor. In other words, figure out what you need to get done and what your requirements are from your content, then put a solution in place that will address the problem.

WHAT'S IN STORE: "What ClearForest does is part of a solution," says Laura Ramos, an analyst at Giga Information Group Inc. in Santa Clara, Calif.

"The problem that people are struggling with is how to get their information organized," she explains. "In the future, this technology plays an important role across a lot of different types of applications, whether focused specifically on content management, knowledge management or even business intelligence and analytics and decision-making. So the ability to add structure to unstructured information is clearly a missing piece to that."

User Profile

After merging with Union Carbide Corp. in February 2001, The Dow Chemical Co. had to figure out how to index and categorize the several hundred thousand technical reports it inherited.

The reports covered an 80-year period and included information from companies acquired by Union Carbide. Most of the collections were in paper form. In addition, documents and chemical substance registries were scattered across many disparate collections, and document management and indexing practices were inconsistent.

"We needed to find a way to integrate those reports into the Dow system, preferably without hiring 100 human indexers to sit and read through them," says Anne Rogers, leader of proprietary information services at Midland, Mich.-based Dow.

She says Dow was able to use ClearForest's technology to do automatic indexing and categorizing by document type and then set up rules for understanding the chemistry.

Rogers says Dow worked with ClearForest, which tuned and developed its ClearTags software to index all of the company's content. Before doing that, however, Dow had to digitize the reports so the ClearForest tools could read them electronically.

Rogers says ClearForest was able to take all the content and sort it by document type—reports that needed to be saved for a long time vs. reports to be saved for a short time.

The second step was to teach the autotagger about chemistry and use some existing chemical indexes.

"[We] set it up so it could read through these reports and say, 'OK, here's what the report's about,' " Rogers says. "We actually did this in conjunction with human indexers, so it was a machine-aided indexing process rather than just purely automatic." ClearTags was able to speed up the human interaction component, she says.

Today, the entire Union Carbide collection is fully integrated with Dow's electronic collection system that provides search capabilities for litigation support, divestitures, and research and development projects, which are mission-critical to Dow.