Hadoop challenger works to add developers
LexisNexis says benchmark testing found its HPCC System is much faster on less hardware than Hadoop
Computerworld - LexisNexis has worked for more than a decade to develop a large scale system for Big Data manipulation, and it believes that it has produced something that's better and more mature than the better known Hadoop technology.
The company just needs developers to agree.
LexisNexis developed the parallel processing data platform to handle the demands of its own data intensive research business. It wants it extend use of the technology, dubbed HPCC Systems, to broader markets, but is clearly aware that open source Hadoop has already established itself as a strong presence.
The company says there are now about 1,000 HPCC Systems developers worldwide, most of who have been trained since the platform was opened sourced in June,
By contrast, a Hadoop developer conference last summer drew a crowd of some 1,700.
To help demonstrate its capabilities, a Terasort benchmark was run to compare HPCC against a similar benchmark and workload by SGI on a Hadoop cluster, announced in October.
LexisNexis says its benchmark was 25% faster, and ran on far less hardware: A 4-node cluster versus a 20-node cluster on the SGI system. The LexisNexis test was done on a Dell PowerEdge, two socket servers, with six core Intel Xeon processors.
Flavio Villanustre, vice president of infrastructure and products at LexisNexis Risk Solutions, credited the test results, in part, on the number of lines in code needed for the sorting versus Hadoop.
LexisNexis developed its own language, ECL, for this system
It took three lines of ECL code to do the sorting, compared to 100 plus lines in Java, which is what is used in Hadoop, said Villanustre.
Asked to respond to the HPCC benchmark, an Bill Mannel, vice president of product marketing at SGI said in a statement that "there are many variations of distributed processing which can run Terasort. HPCC Systems is running Terasort on ECL code, which is different than SGI running on a MapReduce-based Hadoop. SGI remains committed to pushing the bar on performance and beating and improving our own record." MapReduce is a software framework.
Villanustre believes HPCC could do well in the marketplace against Hadoop, but he doesn't take anything for granted. He said that he wants to avoid ending up like Betamax, which lost the video format wars to VHS, or IBM's OS/2 operating system, which was crushed by Microsoft Windows.
"We want to ensure adoption and that's why we are pushing so much," said Villanustre.
The company has also made its HPCC system available in the cloud via Amazon Web Services.
BI and analytics
- Big data key to bringing hyperlocal weather forecasts to Georgia farmers
- Brewer taps Bud Lab at University of Illinois
- Splunk woos Hadoop users
- RSA brings big data analytics to security threat management
- Moving beyond Hadoop for big data needs
- Q&A: What's needed to get a big data job?
- SAS extends analytics support for unstructured data
- Time has come for chief analytics officers
- Big data brings big academic opportunities
- Finding the business value in big data is a big problem
- Seattle Children's Accelerates Citrix Login Times by 500% with Cross-Tier Insight Seattle Children's is a leading research hospital with a large and growing Citrix XenDesktop deployment. With ExtraHop, the IT team at Seattle Children's...
- McKesson Makes Application Hosting for Hospitals Faster, More Efficient With ExtraHop, McKesson identified the root cause of slow Citrix XenApp application launches and adopted a more intelligent, proactive IT operations model that...
- A Non-Geek's Big Data Playbook A visual playbook for the non-geek yet technically savvy business professional who is still trying to understand how big data impacts the enterprise...
- Reducing Expenses Starts with the Programming Skills of Operations Groups Reducing expenses can start with the programming skills of operations groups. Read this article to learn how to improve the bottom line, and...
- Live Webcast Charting Your Analytical Future - "Making predictive analytics part of your business processes" Webinar This session will show how predictive analytics can be used throughout the organization by anyone looking for answers and how organizations can make...
- Charting Your Analytical Future - "Making predictive analytics part of your business processes" Webinar This session will show how predictive analytics can be used throughout the organization by anyone looking for answers and how organizations can make...
- Capturing Data in Motion: Delivering Real-Time Insight from Data Streams This webcast will help organizations of all types and sizes learn about a technology and business strategy for tapping into the wealth of... All Business Intelligence/Analytics White Papers | Webcasts
Our new bimonthly Internet of Things newsletter helps you keep pace with the rapidly evolving technologies, trends and developments related to the IoT. Subscribe now and stay up to date!