Computerworld - Emerging business analytics needs are exposing limitations in traditional database management technologies and fueling the growth of highly specialized analytics platforms, according to a report by two leading industry analysts.
The new products are designed to analyze large and diverse data sets more efficiently than traditional database management products allow. And they integrate new database and storage management technologies to enable more efficient data management and retrieval capabilities, the report said.
The report, prepared for Beye Network by Merv Adrian, principal at IT Market Strategy, and Colin White, president of DataBase Associates Inc. and founder of BI Research, was sponsored by eight vendors of the new technologies. It incorporates feedback from 168 enterprises that are not currently using the new technologies.
"Over the last several years, the notion that a specialized engine for analytics is a good idea has achieved significant commercial acceptance," Adrian said. "People are buying alternatives to their incumbent database management system" to handle new, increasingly complex, real-time business analysis needs, he said.
Many of the emerging products are from relatively new and small vendors such as Netezza, Vertica, Aster Data, Greenplum and ParAccel. The overall market for products from such vendors is still relatively small, hovering at around $500 million. Even so, the growing corporate interest in specialized analytic database management systems (ADBMS) has forced big vendors to take notice.
In September, IBM announced plans to acquire Netezza, one of the pioneers in this area, for $1.7 billion. A month earlier, high-end data warehouse appliance vendor Teradata acquired Kickfire, a startup maker of specialized analytics technologies. And in July, EMC agreed to buy Greenplum, a developer of high-end data warehousing software.
A lot of the activity is being driven by growing corporate interest in analyzing structured data in enterprise data warehouses as well as unstructured data outside those warehouses, Adrian said. Increasingly, companies are looking to perform complex, real-time analytics on unstructured text data, Weblogs, scientific feeds, photographs, videos and sound files.
The Beye survey showed that nearly 45% of the respondents are currently feeding their analytics platforms with XML data, 24% with unstructured file data and more than 23% who are analyzing weblog data. Traditional database management systems, analytic products and storage mechanisms are simply not designed for these new forms of data and are ill-equipped to work with them, he said.
"The technology was designed to process transaction stacks. That design point is different from the design point that says what I want to do is to analyze a lot of data and do it in real-time," he said.
The trend is upending traditional data analytics approaches in which analysis is typically performed outside the database on rich client software on desktops, Adrian said.
More and more often, a database management system is being required to also power the analytics. "It has become evident that for many applications you want to move the analysis to the data," he said. That requirement is pushing many vendors, including established ones such as IBM, Microsoft, Oracle, SAS and others, to support in-database analytics and in-database mining capabilities.
Many of the new specialized systems use "modern storage paradigms" to allow more efficient data retrieval by columns, the Beye report noted. They also encode data for better compression and sometimes use new technologies to do some of the basic analysis work at the storage layer in order to free up the processor for the heavier work.
They also "connect processors with one another and with data storage across faster networks to scale processing and storage in sync," the report noted. "Classic RDBMS products were not built with these innovations in mind and are not always easy to update to leverage these new opportunities."
Jaikumar Vijayan covers data security and privacy issues, financial services security and e-voting for Computerworld. Follow Jaikumar on Twitter at 
@jaivijayan or subscribe to Jaikumar's RSS feed 
. His e-mail address is jvijayan@computerworld.com.
Read more about Business Intelligence/Analytics in Computerworld's Business Intelligence/Analytics Topic Center.
You are previewing premium content. 
Free Insider Guide