NYSE accelerates BI with switch from Oracle, SAS to data warehousing appliances
Computerworld - NYSE Euronext Inc., the operator of the New York Stock Exchange, Euronext and several other stock exchanges, has adopted data warehousing appliances from Greenplum Inc. and Netezza Corp. to accelerate its analysis of internal business and performance data.
NYSE Euronext has been using the Greenplum 4500 appliance for the past nine months to store and analyze data to determine where there are slowdowns in the process of taking, executing and acknowledging trades for stocks, bonds and other securities, according to chief data officer Steve Hirsch.
The time for an order to arrive at the NYSE's network, be executed, and then have an acknowledgment, or "ack," sent back to the trading party, is already fast: 1 to 2 milliseconds, Hirsch said. But using Greenplum to analyze bottlenecks in this system will help the NYSE "get closer to zero," he said in an interview Thursday.
The Greenplum software includes both data warehousing and business intelligence analytical capabilities. This allows the appliance, which includes a Thumper server from Sun Microsystems Inc. and fiber-attached storage totaling 100TB in capacity, to analyze the data itself, rather than sending large amounts of data to a separate server for analysis.
"This makes us hundreds to thousands of times faster. It's incredible," Hirsch said. He cited one task that formerly took 12 hours to run and now takes "just minutes," and another report that formerly took 50 to 60 hours to generate and now "runs in a single hour."
Accelerating trade times is an important but new emphasis for the tradition-bound, 216-year-old NYSE. It only began providing electronic trading two years ago while maintaining traditional human auctioneering.
This was partly spurred by the NYSE's merger with electronic-trading pioneer, Archipelago Holdings, in 2005 and Euronext in 2006.
The NYSE has been using Netezza's appliances slightly longer than Greenplum's appliance in order to track and audit trades and orders, Hirsch said.
Hirsch said he has evaluated and continues to evaluate "most of the products out there," including those from DATAllegro Inc. (now part of Microsoft), ParAccel and Aster Data Systems Inc., as well as a Teradata appliance that includes SAS analytics on board.
Both Greenplum and Netezza are far faster than its prior infrastructure, which primarily consisted of Oracle data warehouses paired with BI servers running SAS, Hirsch said. Having separate servers for data warehousing and BI "doesn't work when you are processing terabytes of data per day ... it doesn't scale to our current levels."
Hirsch said that despite the NYSE doing "pretty sophisticated types of analytics," it hasn't run into problems where the Greenplum or Netezza appliances could not perform desired analytical runs.
"We look at Greenplum as the leader in terms of software-based solutions, and Netezza as the leader in hardware-based solutions," he said.
Hirsch declined to say how much he paid for the Greenplum and Netezza appliances. Greenplum has said it charges about $20,000 per TB of user data, while Netezza is estimated to charge about $29,000 per TB for its 10100 data appliance.
The NYSE is about to go live with a latency monitoring system that will plug into Greenplum and use GPS signals to synchronize measured times within the network, for better precision and accuracy, as well as scale up to billions of rows of data per day while performing "near real-time analysis."
Read more about Business Intelligence/Analytics in Computerworld's Business Intelligence/Analytics Topic Center.
- Deep Security +VMware vSphere with Operations Management Most midsize organizations are highly virtualized on VMware, and while this has produced significant savings, it also has created new challenges when it...
- Single-Vendor Security Ecosystems Offer Concrete Benefits Over Point Solutions IT security decision-makers from companies with 100 to 5,000 employees evaluates the current endpoint security solution market based on Forrester's own market data,...
- Unlocking the Promise of Demand Sensing and Shaping through Big Data Analytics Many organizations have limited insight into big data. These limitations have significant opportunity costs and can have a negative effect on identifying and...
- The Brave New World of Customer-Centric Manufacturing The Unique Opportunity for Manufacturers to Better Understand their Consumers
- Fundamentals of the Unified Communications BE 6000 The Cisco Business Edition 6000 delivers the superior performance, system redundancy, and broad application integration you need. Well suited for businesses with 100...
- Cisco Collaboration for Midsize Business Learn what Cisco Collaboration Solutions can do for your midsize business. In this 2-minute video, see how collaboration tools can make your business... All Business Intelligence/Analytics White Papers | Webcasts
Our new bimonthly Internet of Things newsletter helps you keep pace with the rapidly evolving technologies, trends and developments related to the IoT. Subscribe now and stay up to date!