NYSE accelerates BI with switch from Oracle, SAS to data warehousing appliances
Computerworld - NYSE Euronext Inc., the operator of the New York Stock Exchange, Euronext and several other stock exchanges, has adopted data warehousing appliances from Greenplum Inc. and Netezza Corp. to accelerate its analysis of internal business and performance data.
NYSE Euronext has been using the Greenplum 4500 appliance for the past nine months to store and analyze data to determine where there are slowdowns in the process of taking, executing and acknowledging trades for stocks, bonds and other securities, according to chief data officer Steve Hirsch.
The time for an order to arrive at the NYSE's network, be executed, and then have an acknowledgment, or "ack," sent back to the trading party, is already fast: 1 to 2 milliseconds, Hirsch said. But using Greenplum to analyze bottlenecks in this system will help the NYSE "get closer to zero," he said in an interview Thursday.
The Greenplum software includes both data warehousing and business intelligence analytical capabilities. This allows the appliance, which includes a Thumper server from Sun Microsystems Inc. and fiber-attached storage totaling 100TB in capacity, to analyze the data itself, rather than sending large amounts of data to a separate server for analysis.
"This makes us hundreds to thousands of times faster. It's incredible," Hirsch said. He cited one task that formerly took 12 hours to run and now takes "just minutes," and another report that formerly took 50 to 60 hours to generate and now "runs in a single hour."
Accelerating trade times is an important but new emphasis for the tradition-bound, 216-year-old NYSE. It only began providing electronic trading two years ago while maintaining traditional human auctioneering.
This was partly spurred by the NYSE's merger with electronic-trading pioneer, Archipelago Holdings, in 2005 and Euronext in 2006.
The NYSE has been using Netezza's appliances slightly longer than Greenplum's appliance in order to track and audit trades and orders, Hirsch said.
Hirsch said he has evaluated and continues to evaluate "most of the products out there," including those from DATAllegro Inc. (now part of Microsoft), ParAccel and Aster Data Systems Inc., as well as a Teradata appliance that includes SAS analytics on board.
Both Greenplum and Netezza are far faster than its prior infrastructure, which primarily consisted of Oracle data warehouses paired with BI servers running SAS, Hirsch said. Having separate servers for data warehousing and BI "doesn't work when you are processing terabytes of data per day ... it doesn't scale to our current levels."
Hirsch said that despite the NYSE doing "pretty sophisticated types of analytics," it hasn't run into problems where the Greenplum or Netezza appliances could not perform desired analytical runs.
"We look at Greenplum as the leader in terms of software-based solutions, and Netezza as the leader in hardware-based solutions," he said.
Hirsch declined to say how much he paid for the Greenplum and Netezza appliances. Greenplum has said it charges about $20,000 per TB of user data, while Netezza is estimated to charge about $29,000 per TB for its 10100 data appliance.
The NYSE is about to go live with a latency monitoring system that will plug into Greenplum and use GPS signals to synchronize measured times within the network, for better precision and accuracy, as well as scale up to billions of rows of data per day while performing "near real-time analysis."
Read more about Business Intelligence/Analytics in Computerworld's Business Intelligence/Analytics Topic Center.
- Best iPhone, iPad Business Apps for 2014
- 14 Tech Conventions You Should Attend in 2014
- 10 Desktop Apps to Power Your Windows PC
- How to Add New Job Skills Without Going Back to School
- Slideshow: 7 security mistakes people make with their mobile device
- iOS vs. Android: Which is more secure?
- 11 sure signs you've been hacked
- The value of smarter oil and gas fields With global energy requirements continuing to rise, the exploration, development and production of new oil and gas resources are shifting to increasingly challenging...
- Smarter Environmental Analytics Solutions: Offshore Oil and Gas Installations Example This IBM Redbooks® Solution Guide describes a solution for implementing smarter environmental monitoring and analytics for oil and gas industries. The solution implements...
- Piecing Together the Business Intelligence Puzzle Business intelligence (BI) technology collects and analyzes company data, delivering relevant information to corporate decision-makers in an effort to produce favorable outcomes.
- Harness IT -- An Introduction to Business Intelligence Solutions Learn the key selection criteria required to provide your organization with the capability to address structured data, unstructured data and mobile demands so...
- Live Webcast Increasing the Value of Your Reports and Dashboards Learn how incorporating other analytical capabilities such as predictive modeling and visualization can increase the value of your reports and dashboards by providing...
- The Software-Defined Data Center: Is your ADC ready? Data center transformation is accelerating beyond virtualization to next-generation cloud architectures and software-defined data centers, bringing new challenges for application performance, scalability and...
- Application Acceleration: Optimize the End-User Experience Watch this on-demand webcast and learn how you can optimize your web content, accelerate performance across any device and browser combination, and offload... All Business Intelligence/Analytics White Papers | Webcasts