Sybase is latest RDB maker to embrace MapReduce

Computerworld - Who says there is a war raging between traditional SQL database vendors and the upstarts touting the so-called 'NoSQL' technology?

A slew of relational database vendors, ranging from startups touting massively-parallel data warehousing appliances to grizzled veterans of the online transaction processing (OLTP) wars, are moving to support the MapReduce data processing technology invented by Google Inc.

Sybase Inc. is the latest vendor to jump on the NoSQL bandwagon -- CTO Irfan Khan disclosed that the veteran database vendor is now working on adding MapReduce functionality inside its popular Sybase IQ analytic database.

The new functionality should help Sybase IQ crunch more data faster, says Khan. Sybase IQ is the most established of the column-based databases that have become trendy for BI and analytics applications.

"By incorporating MapReduce in a user-defined function (UDF) approach, we'll be able to break down the complexity of a problem [and] push processing closer to where the data resides while keeping the friendly SQL-like exterior," Khan said.

Startups Aster Data Inc. and Greenplum Inc. kicked off the trend last year by announcing plans to add the MapReduce technology to their respective data warehousing apps in order to accelerate the crunching of large analytical data sets.

Proponents claim that MapReduce, along with its Java-based open-source sibling, Hadoop, are faster than relational databases that hold huge amounts of data -- think hundreds of terabytes or even petabytes -- spread out over large-scale server grids like Google's search server installations.

MapReduce and Hadoop usage began among Web-centric companies like Yahoo, Facebook, and LinkedIn, all with data processing needs similar to Google's.

Meanwhile, SQL proponents argue that the advantages of MapReduce andHadoop are limited only to certain types of data-crunching, such as text indexing or data mining, and that the SQL language and tools are faster and better-suited for wide-ranging needs of most enterprises.

Judging by the companies presenting at the first HadoopWorld conference, held in October, though, interest in the new technology is widespread. The list included speakers from Visa, JP Morgan Chase, Booz Allen, the New York Times and China Mobile.

Relational database pioneer Michael Stonebraker co-authored a paper earlier this year contending that that SQL technology still beats MapReduce in most cases. But that conclusion didn't stop Vertica Systems Inc., the startup where he serves as CTO, , from adding Hadoop functionality to its new Vertica 3.5 database.

Meanwhile, data warehouse appliance supplier Netezza and longtime data warehousing firm Teradata Inc. are working on adding MapReduce functionality.

Also, a team of researchers led by Yale researcher Daniel J. Abadi have developed HadoopDB, which melds PostgreSQL, Hadoop and Facebook's version of Hadoop, called Hive.