June 21, 2004 (Computerworld) --
Searching for the elusive competitive advantage increasingly means parsing, correlating and analyzing mountains of data into refined molehills of business intelligence, which raises the risk of exposing private information.
Better tools and more sophisticated analysis are bringing more granular analysis of operational and transactional data, according to Keith Gile, a business intelligence analyst at Forrester Research Inc. in Cambridge, Mass. Granularity is good for identifying trends to segment markets or to spot suspicious behavior, but it can also identify an individual.
"The tendency in 2004 is to drill deeper into aggregate data to get additional information, bringing us closer to data that implies a privacy breach. All the BI vendors are adding increasing analytical functions, and it is clear that the more granular the data, the more valuable it is," Gile said.
Most business intelligence, until now, focused on aggregated data, Gile said, which poses little risk of uncovering personally identifiable information subject to the protections of privacy regulations such as HIPAA or the Gramm-Leach-Bliley Act.
But the trend in business intelligence is to integrate data from transactional systems containing customer or patient information, such as ERP or CRM, with business intelligence applications, Gile said.
"Greater granularity is valuable. However, it increases the risk of exposing too much information," he said, resulting in situations where crucial, identifiable information "must be stripped off" business intelligence analysis performed by a user community beyond the scrutiny of privacy officers. "IT must be prepared to deal with BI and privacy and security because there is an issue with privacy in terms of data being released through BI technologies," Gile said.
Deleting personally identifiable information from a business intelligence system isn't an option in health care, said Jonathan Rothman, director of data management at Livingston, N.J.-based Emergency Medical Associates.
EMA developed a business intelligence system that's the patient information repository for more than 1,500 physicians, nurses, pharmacists, billing clerks and other health care professionals. It includes both operational and transactional patient data that's integrated into a data warehouse that further collects data from other sources. The result is a unique blend of protected and nonprotected health care information.
"All of our patient information is private," said Debbie Clark, EMA's director of client services. That means the entire business intelligence application and all the information in the data warehouse was designed with policies, procedures and safeguards for HIPAA compliance.
"Anytime a user can see patient information, we have to know who they are," Clark said. EMA has implemented password authentication, role-based authorization and auditing from its business intelligence vendor, Business Objects SA in San Jose.
"Anytime we report information out of the data warehouse, there is no reason for me to provide any PHI [personal health information] unless it's aggregated and made nonpublic," Rothman said. The business intelligence analysis, he said, such as "how many heart ailments seen by a physician or how many stomach ailments on a given day," stays in the business intelligence application, which is accessible in standard browser sessions.
"We bring together on average 2,200 emergency room visits per day, 365 days per year, 24 hours per day. There's tremendous security around our data warehouse," which includes virtual private networks to connect the 26 hospitals that use the data warehouse.
EMA is "just starting to use wireless access," Clark said. "The hospitals are very security-aware, and they're installing secure, HIPAA-compliant wireless networks now."
U.S. government is big BI user
The largest potential user of business intelligence that reveals personally identifiable information is the U.S. government. According to a General Accounting Office survey released May 27, there are 52 federal systems analyzing 131 operational data warehouses. An additional 68 data warehouses are being created, the GAO report said, and government officials plan to include personally identifying information in at least 122 of the 199 current and planned data warehouses.
The federal business intelligence systems will use private-sector data, such as credit card transactions, phone records or airline flight data, in 54 data warehouses, the GAO said. Seventy-seven data warehouses contain information from multiple government sources with the Defense Department making the greatest use of business intelligence, with 47 data warehouse projects reported.
All the federal data gathering and analysis has raised the hackles of the privacy community. Congress should pass laws protecting the privacy of citizens from government information analysis, according to the recommendation sent to Defense Secretary Donald Rumsfeld in May by a DOD advisory committee.
The DOD panel said federal agencies should generally restrict their data analysis to aggregated and anonymous data. If personally identifiable information is needed, it should require a court order, the panel recommended.
The Pentagon's Defense Advanced Research Projects Administration has funded business intelligence projects to combat terrorism. One project is looking into software agents that can automatically burrow into databases seeking suspicious patterns in travel reservations or questionable data in e-mails.
Privacy also is a top priority as the Department of Homeland Security ramps up its business intelligence systems. Peter Sand, the DHS director of privacy technology, said in a speech on May 21 to the Electronic Privacy Information Center's Freedom 2.0 conference that privacy is designed into new data analysis systems in the DHS.
The DHS is subject to the provisions of the Freedom of Information Act, the Privacy Act of 1974 and the E-Government Act of 2002, he said, and the DHS privacy office works as an independent watchdog inside the DHS.
Forrester's Gile said census data also may be vulnerable for making personally identifiable associations, followed by local election records.
"The Democrats and Republicans have spent a lot of money capturing highly atomic data on individual voter patterns and tendencies," he said, and are busy correlating voting data with other large consumer databases such as BuyRite, Nielsen and Information Resource Inc.
And the lawyers and courts are waiting to rule on all privacy breaches and interpret regulations. "Somebody will ultimately be taken to court on just such privacy issues," Gile said.
One positive development stemming from the collapse of Wall Street may be a boost in interest in computer science and IT careers among students who were previously interested in financial services jobs.
Computerworld Executive Briefing: The Compliance Era
Get this briefing free (a $195 value), for a limited time, courtesy of VeriSign. The new Computerworld report, The Compliance Era, explains why regulatory compliance has zoomed to the top of the IT agenda and shows how real-world IT executives are dealing with the storage, security and privacy challenges. Get this briefing free (a $195 value), for a limited time, courtesy of VeriSign. Download this executive briefing
Success Story: U.S. Intelligence Agency
Download this white paper, free for a limited time, compliments of Novell! (Source: Novell) A large intelligence agency in the U.S. Federal Government was struggling with managing the vast volume of security information that crossed its networks, as well as managing this data to ensure security at all times. Using Sentinel" from Novell?, the agency now has centralized security knowledge management and can better comply with regulatory standards. Download this white paper
From Laggard to Leader: Transforming the Data Center
From Laggard to Leader: Transforming the Data Center Register for this complimentary webcast today! Go to the webcast
White Papers
Read up on the latest ideas and technologies from companies that sell hardware, software and services.
Eliminate SPAM, Gain Productivity Learn all about the dangers and the costs of spam in all its forms – from stock-touting to spreadsheet. Also, understand the drawbacks of traditional hardware- and software-based defenses – and the unique benefits of MessageLabs multi-layered, managed Anti-Spam solution; as illustrated by a real-world case study where MessageLabs stopped spam cold.
Download this white paper now! See more Whitepapers
As the volume of information inside enterprises explodes, most executives recognize the importance of a Google-like search solution for business content. To this end, Google has developed Universal Search for Business, powered by the Google Search Appliance, which searches all enterprise content through one secure box. Attend this webinar to learn how your business can benefit from universal search capabilities.
See your link here
Subscribe to
Computerworld 
or
Other Business Intelligence Stories
Get this briefing free (a $195 value), for a limited time, courtesy of VeriSign. 
Download this white paper, free for a limited time, compliments of Novell!
From Laggard to Leader: Transforming the Data Center
Read up on the latest ideas and technologies from companies that sell hardware, software and services.
Learn all about the dangers and the costs of spam in all its forms – from stock-touting to spreadsheet. Also, understand the drawbacks of traditional hardware- and software-based defenses – and the unique benefits of MessageLabs multi-layered, managed Anti-Spam solution; as illustrated by a real-world case study where MessageLabs stopped spam cold.
Download this white paper now! 
