The Story So Far: Business Intelligence
A first-person account from the first person to develop the data warehouse concept.
By Frank Hayes
April 14, 2003 12:00 PM ET
Computerworld - William H. Inmon is commonly known as the father of data warehousing. While speaking with Frank Hayes, Inmon recalled the development of the data warehouse idea, starting with his 1983 Computerworld article, "What Price Relational?"
"Years ago, the rage was relational technology, DB2 and Oracle. And in healthy skepticism, I wrote a couple of articles saying that while relational technology certainly had merits, it simply wasn't fulfilling everything that was being ascribed to it.
"I got hate mail—people said I should never be allowed to speak in public, that I was setting our industry back 25 years. But I learned from that to ask the question, 'If relational technology isn't the answer, then what is?' And based on the premise that you need to have integrated, historical, easy-to-access information, that was the genesis of data warehousing.
"From that, I started to build data warehouses. My day job was working at American Management Systems, so I was able to try some of my theories out. Some of the early data warehouses we built were very novel and very innovative, and I know I learned a lot.
"The first issue that hit me like a two-by-four was the fact that integrating data from legacy sources is a very nontrivial thing. In the beginning, we thought, 'Well, you've got this source of data over here; you just write a program and you bring the data forward into this data warehouse.' I'll never forget saying, 'Gee, what's so hard about this?' Today, there's a whole industry called ETL [extract, transform and load] that does that.
"The second thing we learned was that the volumes of data that aggregate inside the data warehouse surpassed anything that's ever been seen in the world of transaction processing.
"A third thing is something that the world is still struggling with: How do you cost-justify a data warehouse?
"I wrote the book [Building the Data Warehouse] in 1989. Then, all of a sudden, people I'd never heard of started calling me and asking questions. I started to work with Claudia Imhoff, Sue Osterfelt, Chuck Kelley, some of the early pioneers, doing seminars and conferences and consulting and building data warehouses. It began to take on a life of its own.
"I was surprised about the industrial usage of it. The first data warehouses I did were at PacTel Cellular, Aetna Casualty and Blue Cross/Blue Shield of Michigan, so the early data warehouses were in the telephone and insurance environments. I never thought that manufacturing, transportation, retailing, government—I never thought data warehouses would be as applicable to those environments as they are.
"It also surprised me that [the] data warehouse forms a foundation for all kinds of analytical processing. We've actually had analytics around for a while: the churn analysis the telephone company does, the elasticity analysis that retailers do. But we're just now starting to see the vendors make things available for widespread usage. Sagent, ProClarity, SAP, PeopleSoft, Cognos and Business Objects all have their own flavors of analytical applications, and I think that's one of the futures for data warehousing.
"Another major issue is that the size of the warehouses is drastically changing things. It's one thing to build a warehouse of 10GB or 20GB; it's another thing to build a data warehouse of terabytes' worth of data. How do you do data management for supersize warehouses? An index of 100TB of data may take three or four months to build. How long is it going to take to load the data? A week? A month? Six months? The next major trend is learning how to cope with volumes of data the likes of which haven't even been imagined by most people.
"After Sept. 11, I began to adapt the material I had written for data warehousing into something we call the Government Information Factory. I'm extremely excited about it. There is a lot of really novel and useful information in there, and we're just now starting to talk with and work with government agencies. So that's the direction I'm off on."
And now, on with the story. ...
Mining for Gems
Stories in this report:
Read more about BI and Analytics in Computerworld's BI and Analytics Topic Center.
Free Insider Guide