Stream processing tools play by the rules

The secret to using stream processing tools effectively lies in knowing what you want to monitor for and creating the proper rules. The tools for creating queries fall into two competing camps: rules engines built on if/then statement logic and variants of SQL, which have been extended to support time-based queries.

Mike Stonebreaker, chief technology officer at StreamBase Systems Inc., prefers the latter. "SQL is great paradigm. People understand it and what you do is adapt SQL to real-time streaming data," he says. Other vendors that use SQL include Aleri Labs and Celequest Corp.

"The problems with rule engines is, 1) there's no standard; there's no rule equivalent of SQL," he says. But while SQL is a standard, a variant of SQL with extensions to support stream-based queries doesn't exist. Stonebreaker also criticizes rules engine syntax, which he says is overly complex. "Rule languages have problems of inscrutable notation, and it's completely unordered," he says.

If anyone was going to support SQL, one would think it would be Progress Software Corp., but the database vendor favors the rules engine that came with its acquisition of Apama Inc. "The important part here is to have a robust enough language to express temporal logic. Time as the first order issue as opposed to filtering," says Mark Palmer, vice president of event stream processing at Progress' real time division.

Copyright © 2006 IDG Communications, Inc.

7 inconvenient truths about the hybrid work trend
Shop Tech Products at Amazon