Why Data Management Needs a New Approach

Computerworld - Last month, I said there's a database management crisis; the relational model (practical or theoretically pure) won't solve it, and alternative, more pragmatic ways of thinking about database management need to be emphasized.
This month, I'll illustrate the point with several examples of situations in which the inability to access known information has cost large numbers of human lives.
Homeland security 1: antiterrorism. Middle Eastern men, some of a suspicious nature, were discovered seeking flight lessons. Alert FBI agents suspected that they might be planning to take over civilian aircraft. But this data was never combined with other FBI information, or with CIA knowledge of al-Qaeda interest in airplane hijackings. There just wasn't an application that could relate keyword and concept searches across various FBI, CIA and public data banks, let alone factor in connections among various individuals and organizations.
Four years later, this application need still hasn't been met.
Health care records. The potential benefits from solving the health care record challenge are almost incalculable. Tens of thousands of lives could be saved annually, and David Brailer, national coordinator of health information technology, has estimated cost savings in the hundreds of billions of dollars.
The technical challenges are immense as well. Almost every data type is relevant -- character, numeric, date, text, image, time series, genomic, maybe even geospatial. New sources of data are invented every year. The most important data of all -- physicians' and nurses' observations and conclusions -- is subjective, incomplete, inconsistent, commonly illegible. And it's usually missing entirely. (Just how many years of your medical records exist anymore?) Even the rules for evaluating and summarizing patient data change as a result of advances in medicine.
Nontechnical problems are also forbidding, involving cost, privacy, organizational politics and the like. This is especially true in countries that, like the U.S., have private-sector health care, but these issues are no picnic in single-payer countries, either.
Homeland security 2: intelligence analysis. In the run-up to the Iraq invasion, the U.S. loudly trumpeted various pieces of "intelligence" related to weapons of mass destruction that actually turned out to be false, specifically in the areas of mobile bioweapons labs, yellowcake uranium ore and aluminum tubing. Intelligence analysts knew each claim was highly unreliable, yet officials presented each one as a near-certain fact. Whatever one's theories about the motives for these errors or the likely policy outcome had they not been made, one thing is clear -- something in the intelligence community needs a great deal of improvement.
One thing that's needed