Getting the most out of big data

Without proper information governance, enterprises are likely to lose out on valuable insights

self service big data
Credit: Thinkstock

The inevitability of the rise of big data was determined nearly 30 years ago when Sam Walton’s Retail Link, which collected sophisticated consumer information using product bar codes, made Walmart a pioneer in data-driven management and delivered significant business and competitive advantages. Today, Facebook, Google and Amazon rely on data collection, aggregation and analysis as strategic to their businesses. Because of this, they easily justify huge investments in the technologies and processes required to gain a greater and timelier understanding of the buying patterns and sentiments of customers and users. It allows these companies to optimize promotional campaigns, improve customer experiences, enhance product mixes, and discover other significant competitive advantages.

With so many benefits to gain, most companies would have launched their own “big data” initiatives long ago except that until recently, few companies could commit the necessary resources to create a data-driven organization. Today, however, affordable big data solutions are allowing organizations of all sizes and across all industries, including the public sector, to benefit from big data and analytics. This means the race is on to take full advantage of the tangible value of data and gain a competitive edge. But is your company really ready for an information deluge? The painful reality is that it probably isn’t.

Most enterprises are saddled with complex, disparate and distributed operations. Across the organization, various initiatives are launched, technologies are deployed, and information is collected. Rarely is there a centralized, consistent approach to data collection, which means the company ends up with multiple disconnected information stores, and too often this information is not shared in any meaningful way. One part of the company may have no idea what information initiatives are active in another part, leading to duplication of effort and inconsistent policies and processes when it comes to regulatory compliance and records retention. And the more information the company collects, the more damaging the situation can be.

Consider some of the challenges a company faces when it acquires another company that relies on different CRM and ERP systems. Multiple, overlapping contact databases can lead to repeated and unnecessary communications, resulting in widespread customer frustration. A lack of insight into what information is stored in the acquired company’s CRM and ERP systems can lead to these systems being overlooked when the acquiring company receives a discovery request. Similarly, as government regulations evolve, they may be applied carefully to the acquiring company’s data but not to the data of the acquired company, creating the potential for violations and fines. Finally, with no program in place to allow for the disposal of unnecessary information — “data debris” — in a way that ensures regulatory and legal compliance, both organizations may end up keeping everything. An approach that can result in wasting hundreds of thousands of dollars on unnecessary infrastructure and maintenance costs while making it more difficult for business users to find the high-value information they need.

The optimal way to resolve this and many other more complex situations is to take a holistic approach to information management that creates alignment between business, legal, regulatory and IT needs and processes. This requires a top-down approach to developing an information governance (IG) program.

Information governance

The objectives of an IG program are simple, if not always easy to implement:

  • Bring all information stakeholders — business users, Legal, records information management (RIM), privacy and security, and IT — together to create a single, centralized program.
  • Develop the processes and deploy the technologies necessary to improve data quality and identify the value of existing and new information to each stakeholder. This must include the ability to identify and retain information for legal and compliance purposes and to dispose of information to comply with new privacy regulations.
  • Eliminate data silos or reduce the barriers between them so all information can be centrally managed.
  • Create a common glossary for referring to information assets across the enterprise.
  • Move information to a less costly tier of storage as the need for ready access diminishes through its life cycle.
  • Dispose of data debris in accordance with applicable regulations in all relevant jurisdictions where the organization operates.

In many organizations, the need for such a significant coordination effort and the possibility of an investment in new technologies can be paralyzing and lead to inaction. But the risk of inaction is too great, and history has shown that successful IG programs require direct support from a top executive, which is why many organizations are now looking to the chief data officer (CDO) — often hired as a champion of big data initiatives — to also be the driver of the IG effort.

Including governance within the purview of the CDO makes sense. After all, what value can the CDO hope to deliver from an analytics effort if the quality of data being analyzed is poor? With a seat at the executive table, the CDO is in the best position to understand the need for IG and weave it into the fabric of the corporate culture. Successful IG is not just about the technologies and processes that are needed. Involving the people responsible for creating information is essential for making IG transformative in nature across the organization.

There is increasing agreement with this point of view. For example, according to a Gartner News post, Gartner defines the CDO as “a senior executive who bears responsibility for the firm’s enterprise wide data and information strategy, governance, control, policy development, and effective exploitation. The CDO’s role will combine accountability and responsibility for information protection and privacy, information governance, data quality and data life cycle management, along with the exploitation of data assets to create business value.”

Does this mean the CDO must have a background in information governance? Not necessarily. While such a background is useful, it is only essential that the CDO understand the value of IG to their primary analytics focus. The quality of data inputs to the analytics engine determines the quality of the output and the decisions made on those results, i.e. quality in, quality out (QIQO). As CDOs recognize the value effective IG delivers, they will lead efforts to put the appropriate team and technology in place (or work with the existing team) to ensure success.

In the world that wasn’t so driven by data, executives often relied on their intelligence and insightfulness to overcome the limitations of initiatives that relied on lower quality or inconsistent quality of information. Not anymore. Organizations simply cannot thrive in the era of big data with a fractured and poorly managed data infrastructure. Both the CDO and the organization benefit if the CDO becomes a champion of, if not an expert in, IG. And the time to take the initiative is now.

Derek Gascon is executive director of the Compliance, Governance & Oversight Council and program lead for ILG Thought Leadership at IBM.

Computerworld's IT Salary Survey 2017 results
Shop Tech Products at Amazon