The Story So Far: Business Intelligence

Computerworld - William H. Inmon is commonly known as the father of data warehousing. While speaking with Frank Hayes, Inmon recalled the development of the data warehouse idea, starting with his 1983 Computerworld article, "What Price Relational?"

"Years ago, the rage was relational technology, DB2 and Oracle. And in healthy skepticism, I wrote a couple of articles saying that while relational technology certainly had merits, it simply wasn't fulfilling everything that was being ascribed to it.

"I got hate mail—people said I should never be allowed to speak in public, that I was setting our industry back 25 years. But I learned from that to ask the question, 'If relational technology isn't the answer, then what is?' And based on the premise that you need to have integrated, historical, easy-to-access information, that was the genesis of data warehousing.

"From that, I started to build data warehouses. My day job was working at American Management Systems, so I was able to try some of my theories out. Some of the early data warehouses we built were very novel and very innovative, and I know I learned a lot.

"The first issue that hit me like a two-by-four was the fact that integrating data from legacy sources is a very nontrivial thing. In the beginning, we thought, 'Well, you've got this source of data over here; you just write a program and you bring the data forward into this data warehouse.' I'll never forget saying, 'Gee, what's so hard about this?' Today, there's a whole industry called ETL [extract, transform and load] that does that.

"The second thing we learned was that the volumes of data that aggregate inside the data warehouse surpassed anything that's ever been seen in the world of transaction processing.

"A third thing is something that the world is still struggling with: How do you cost-justify a data warehouse?

"I wrote the book [Building the Data Warehouse] in 1989. Then, all of a sudden, people I'd never heard of started calling me and asking questions. I started to work with Claudia Imhoff, Sue Osterfelt, Chuck Kelley, some of the early pioneers, doing seminars and conferences and consulting and building data warehouses. It began to take on a life of its own.

"I was surprised about the industrial usage of it. The first data warehouses I did were at PacTel Cellular, Aetna Casualty and Blue Cross/Blue Shield of Michigan, so the early data warehouses were in the telephone and insurance environments. I never thought that manufacturing, transportation, retailing, government—I never thought data warehouses would be as applicable to those environments as they are.

"It also surprised me that [the] data warehouse forms a foundation for all kinds of analytical processing. We've actually had analytics around for a while: the churn analysis the telephone company does, the elasticity analysis that retailers do. But we're just now starting to see the vendors make things available for widespread usage. Sagent, ProClarity, SAP, PeopleSoft, Cognos and Business Objects all have their own flavors of analytical applications, and I think that's one of the futures for data warehousing.

"Another major issue is that the size of the warehouses is drastically changing things. It's one thing to build a warehouse of 10GB or 20GB; it's another thing to build a data warehouse of terabytes' worth of data. How do you do data management for supersize warehouses? An index of 100TB of data may take three or four months to build. How long is it going to take to load the data? A week? A month? Six months? The next major trend is learning how to cope with volumes of data the likes of which haven't even been imagined by most people.

"After Sept. 11, I began to adapt the material I had written for data warehousing into something we call the Government Information Factory. I'm extremely excited about it. There is a lot of really novel and useful information in there, and we're just now starting to talk with and work with government agencies. So that's the direction I'm off on."

And now, on with the story. ...

1983: William H. Inmon begins work on data warehousing concepts. 1964: Michael S. Scott Morton first describes the concept of decision-support systems. 1961: Charles Bachman at General Electric develops the first database management system, IDS. 1964: Michael S. Scott Morton first describes the concept of decision-support systems. 1969: Ted Codd invents the relational database. 1970: Express, a multidimensional analytic processing tool for time-sharing systems, becomes available. 1978: Work begins on the Management Information and Decision Support system, an early executive information system, at Lockheed-Georgia Co. 1983: William H. Inmon begins work on data warehousing concepts. 1985: Procter & Gamble uses the first business-intelligence system to analyze data from checkout-counter scanners. 1993: Ted Codd coins the term OLAP (online analytical processing). 1985: Procter & Gamble uses the first business-intelligence system to analyze data from checkout-counter scanners. 1989: Gartner analyst Howard Dresner coins the term business intelligence. 1991: Inmon publishes a practical how-to guide, Building the Data Warehouse. 1993: Ted Codd coins the term OLAP (online analytical processing).

Mining for Gems

Stories in this report:

• Editor's Note: Mining for Gems
  
• The Story So Far: Business Intelligence
  
• The Forescat is Clear
  
• Unexpected Insights from Data Mining
  
• Opinion: Trust, But Verify Your Data Gatekeepers
  
• Real-time Data: Too Much of a Good Thing?
  
• The Almanac: Data Management
  
• Securing Business Intelligence Data
  
• QuickStudy: Data Models
  
• How Your Career Can Thrive as a Data Architect
  
• The Next Chapter: The Future of Business Intelligence
  
• Management Dashboards Becoming Mainstream
  
• Open-source Database Buying Suggestions
  
• What Web Services Can Do for Business Intelligence
  

Read more about business intelligence in Computerworld's Business Intelligence Knowledge Center.