When good data goes bad

Computerworld - Data quality is critical to the success of any enterprise application. Systems from business intelligence to customer relationship management are destined to fail without high-quality data -- the "garbage in, garbage out" theory.

While many IT professionals expect improved data quality to be a side effect of implementing new systems, ensuring high data quality is an important first step in the process. An even more difficult challenge is maintaining high data quality on an ongoing basis.

For instance, contact data, one of the most critical elements of a CRM system, typically erodes at a rate of 33% per year. Without proper attention, the data will inevitably become incorrect, unusable and ultimately untrustworthy. In its groundbreaking 2001 report, Gartner Inc. cited poor data quality as the single greatest inhibitor to successful CRM implementation.

In response, IT professionals are increasingly instituting processes and procedures to ease the task of maintaining high-quality data. Some of the most effective ideas include the following:

Build in Change Management Rules
A centralized enterprise system allows all users to contribute to the database. However, not all enterprise data should be blindly updated, and not all users are careful about their changes. As a result, IT professionals should establish submission and review processes that let them filter which user changes are saved to the centralized repository.

Increasingly, "data stewards" -- users tasked with maintaining data quality -- are establishing change management rules by which they can discern good changes from bad ones. User changes to data are routed to the data steward in the form of a "ticket"; the data steward can then evaluate the ticket to determine whether to accept the modification. For instance, if a low-level CRM user changes the corporate name of the company's top customer, this should serve as a red flag to the data steward to double-check this edit prior to accepting the change.

Without a submittal and review process, the only way to prevent end users from directly updating the central database is to lock certain fields. Two significant problems can result. First, end users become frustrated when they aren't able to make changes that they need and that they know should be made to contact information. Second, those managing the central database miss out on getting update information from those end users in the best position to know that information has changed.

Establish Workflow Processes
If change management rules are instituted, workflow processes should also be thought through so as not to bog down the data steward with unimportant change requests. For instance, some changes are less prone to error and therefore less suspect -- such as changes to e-mail address information. A data steward might establish a rule enabling such changes to be saved immediately to the centralized repository. Other changes, such as those to company names and titles, are critical and might be rejected pending verification of the change. The workflow process should empower the data steward to take appropriate action and move on to the next change ticket.

Categorize and Prioritize Data
The frequency and quantity of changes to data that occur within an enterprise system are staggering. Even with rules and processes in place to manage modifications, a data steward could still be easily overwhelmed with change tickets.

To prevent this, companies must have a system to categorize and prioritize which changes warrant the oversight of the data steward and which don't. Not all changes should be submitted to a data steward for processing. Depending on the importance of the data and the type of change, it's perfectly acceptable to allow certain changes to be completed and to perform the data quality review later and en masse.

For instance, in a CRM system, changes made to contacts at a company's top customers have much greater effect on the organization than changes to contacts of noncustomer vendors. Also, changes made by certain trusted users may not need to be reviewed by a data steward.

Companies should take time to determine which types of data are critical to the organization and therefore warrant heightened management. Change management rules and workflow processes should be applied specifically to these highly managed data. Changes to data not deemed to be strategic to the organization can be saved directly to the central database without review. By categorizing data and applying different change management and workflow rules to different data types, companies can more easily prioritize the limited resources of their data stewards -- and help ensure that CRM, business intelligence and other systems contain accurate information.

Barry Solomon is the executive vice president of Oak Brook, Ill.-based Interface Software Inc., which sells CRM software and services. He can be reached at bsolomon@interfacesoftware.com.