IDG News Service - LexisNexis is planning to release its internally developed supercomputing platform as open source, providing developers with an alternative to the Hadoop framework for large-scale data processing, the company said Wednesday.
LexisNexis has been developing the technology, dubbed HPCC Systems, for the past 10 years, according to the company, which provides a variety of information services to legal firms, libraries, corporations and government entities.
"We've been doing this quietly for years for our customers with great success. We are now excited to present it to the community to spur greater adoption," said James Peck, CEO of LexisNexis' Risk Solutions division, in a statement. "We look forward to leveraging the innovation of the open source community to further the development of the platform for the benefit of our customers and the community."
HPCC Systems runs on clusters of commodity hardware and is made up of a number of components, centered around the company's Enterprise Control Language, a "declarative, data-centric programming language optimized for large-scale data management and query processing," LexisNexis said.
A component called Thor handles data ETL (extraction, transformation and loading) chores, while a third system named Roxie delivers "highly scalable, high-performance online query processing and data warehouse capabilities," LexisNexis said.
The system is able to analyze petabyte-sized volumes of data "significantly faster and more accurately than current technology systems," scaling up to thousands of nodes, the company said.
LexisNexis will offer both a community edition and commercial enterprise edition of HPCC Systems, which will be overseen by company CTO Armando Escalante.
At first, HPCC Systems will be offered as a virtual machine for testing by the community, with full binaries and the source code to be issued a number of weeks later.
The community edition will be released under the GNU Affero GPL v3 license. New code contributed by LexisNexis and community members will go to the open-source edition first, according to a detailed FAQ document on the company's site.
However, LexisNexis stressed that HPCC Systems won't involve the release of any of its "data sources, data products, the unique data linking technology, or any of the linking applications that are built into its products."
The community edition will also have a number of limitations compared to the enterprise edition, such as a restriction of one Thor process per node, according to a comparison chart. It will also get only "basic testing against different Linux distributions," while the enterprise edition will undergo a much more rigorous certification.
Pricing for the enterprise edition, which is offered with a number of support tiers, was not available.
Enterprise Edition subscription customers have access to a number of add-on modules as well, including a tool that converts the Pig Latin language used in Hadoop to ECL.
- 15 Non-Certified IT Skills Growing in Demand
- How 19 Tech Titans Target Healthcare
- Twitter Suffering From Growing Pains (and Facebook Comparisons)
- Agile Comes to Data Integration
- Slideshow: 7 security mistakes people make with their mobile device
- iOS vs. Android: Which is more secure?
- 11 sure signs you've been hacked
- The Critical Role of Support in Your Enterprise Mobility Management Strategy Most business leaders underestimate the importance of tech support when they choose an EMM solution. Here's what to put on your checklist.
- Separating Work and Personal at the Platform Level: How BlackBerry Balance Works BlackBerry® Balance™ separates work from personal on the same mobile device, right at a platform level. Find out how it can work for...
- Protection for Every Enterprise: How BlackBerry Security Works Get an IT-level review of BlackBerry® Security, addressing data leakage protection, certified encryption, containerization and much more.
- Future Focus: What's Coming in Enterprise Mobility Management (EMM) Find out why Enterprise Mobility Management (EMM) solutions that are truly future-ready must be designed to enable Machine-to-Machine (M2M) capabilities and much more.
- Live Webcast Best Practices for the Hyperconverged Enterprise Network To the Age of Constant Connectivity and Information overload
- Live Webcast Unmasking the Differences between Consumer and Enterprise File Sync & Share The consumerization of IT combined with the rapid pace of the modern mobile workplace is forcing enterprise IT teams to evaluate file sync...
- Live Webcast Government Agency Webifies Outdated COBOL Applications Let this CTO tell you how his agency converted 1980s-era green screens into an e-filing portal for the 100,000 cases handled each year...
- The New Way to Work Knowledge Vault This Knowledge Vault focuses on how, in today's increasingly virtual world, it's more important than ever to engage deeply with employees, suppliers, partners,...
- Getting Ready for BlackBerry Enterprise Service 10.2 Find out how BlackBerry® Enterprise Service 10 helps organizations address the full spectrum of EMM challenges, while balancing the needs of both the... All Applications White Papers | Webcasts