Updated Sybase IQ supports Hadoop, MapReduce, 'Big Data'

IDG News Service - Sybase is hoping its IQ analytic database can make its mark in the burgeoning "Big Data" market with an array of new features, including native integration with the open-source MapReduce and Hadoop programming frameworks for large-scale data processing.

The 15.4 release of IQ, scheduled for no later than Dec. 1, will come just several months after version 15.3, which added massively parallel-processing features to the platform. But work on 15.4 actually began about a year-and-a-half ago, because Sybase set up a few different teams to work on separate upcoming editions, said Dan Lahl, senior director of product marketing. "This is not a three-month hack job."

"Big Data" is an industry buzz-phrase that refers generally to the huge amounts of information created by websites, sensors and other sources outside of traditional enterprise applications. With Tuesday's announcement, Sybase joins the wave of database, data warehousing and analytics vendors that have been moving quickly to develop premium features and services around Hadoop and MapReduce.

New features in the 15.4 release include a native MapReduce programming interface that uses standard SQL; a Hadoop integration that provides various ways to tie together data from Sybase and Hadoop; a Java interface and additional extensions for existing C++ interfaces for running in-database algorithms; support for PMML (Predictive Model Markup Language) via a partnership with Zementis; and a data mining and statistics library from Fuzzy Logix for use in conjunction with MapReduce.

Sybase is also now offering an Express edition of IQ, which can be used indefinitely, but for development purposes only and with a 5GB database size limit.

One might think that Sybase IQ lies in an increasingly uncertain position, despite the rash of recent improvements, given SAP's unrelenting hype of late around HANA, its homegrown in-memory computing engine that so far has been targeted primarily at analytic workloads.

But the two databases can play complementary roles, said Joydeep Das, director of analytics product management.

HANA's blazing performance makes it a good fit for the real-time analysis of operational data from SAP applications, he said. As for IQ, "if you have very large enterprise data warehouse with growing numbers of users and data, Sybase IQ is a very good fit because of the scalability and speed."

Companies could also analyze data in HANA and use Sybase as "near-line" storage for that information once they're finished, he added.

Overall, a customer's database environment could be compared to a wedding cake, he added. In this scenario, Hadoop would sit at the widest bottom layer, crunching large data pools to find the "needles in the haystack," or actual information of value. Those needles would then be fed into IQ for further analysis, and potentially to an even faster system like HANA, he said.