Hadoop ready for corporate IT, execs say
Despite some concerns, Hadoop has a growing place in the enterprise, say IT execs from JP Morgan Chase, eBay
Computerworld - NEW YORK -- Despite some lingering technology issues, Hadoop is ready for enterprise use, IT executives said Tuesday at the Hadoop World conference here.
Larry Feinsmith, managing director at JP Morgan Chase, told a keynote audience that the financial services firm has been using the open source storage and data analysis framework for close to three years now and is currently leveraging the technology for fraud detection, IT risk management, self service and other applications.
Chase still relies heavily on core relational database technologies for transaction processing, but uses Hadoop-based products for a growing number of tasks, Feinsmith said. Five out of seven Chase business units use Hadoop in some way, he added.
Hadoop's ability to store vast volumes of unstructured data has allowed Chase to collect and store weblogs, transaction data and social media data, Feinsmith said.
The company is aggregating the data into a common platform, and runs a range of customer-focused data mining and data analytics applications to utilize it, he said.
With over 150 petabytes of online storage, 30,000 databases and 3.5 billion logins to Chase user accounts, data is the lifeblood of the company, Feinsmith said.
For the moment at least, relational database technologies appear to be more suited for running transaction applications, he said.
The big debate among technologists at the bank right now is whether incumbent relational database technologies will evolve to meet the bank's emerging big data needs, or Hadoop-based technology can become adept at transaction processing, Feinsmith said.
Hugh Williams, vice president of experience, search and platforms at eBay, said that the auction site is revamping its core search engine technology using Hadoop and Hbase, a technology that enables real-time analysis of data in Hadoop environments.
The new eBay search engine, code-named Cassini, will replace the Voyager technology that's been used since the early 2000s. The update is needed in part due to surging volumes of data that needs to be managed, Williams said.
Williams said that eBay currently has more than 97 million active buyers and sellers and over 200 million items across 50,000 categories for sale. The auction site handles close to 2 billion page views, 250 million search queries and tens of billions of database calls each day, he said.
The company has 9 petabytes of data stored on Hadoop and Teradata clusters, and the amount of data is growing quickly, he said
Hadoop and Hbase allow EBay to build a far more sophisticated search engine than Voyager. Cassini will deliver more accurate and more context-based results to user search queries, he said.
With more than 100 engineers assigned to Project Cassini full time, the development effort is one of the largest ever at EBay.
BI and analytics
- Brewer taps Bud Lab at University of Illinois
- Splunk woos Hadoop users
- RSA brings big data analytics to security threat management
- Moving beyond Hadoop for big data needs
- Q&A: What's needed to get a big data job?
- SAS extends analytics support for unstructured data
- Time has come for chief analytics officers
- Big data brings big academic opportunities
- Finding the business value in big data is a big problem
- IT-centric enterprise BI models unsustainable, says Forrester
- Best iPhone, iPad Business Apps for 2014
- 14 Tech Conventions You Should Attend in 2014
- 10 Desktop Apps to Power Your Windows PC
- How to Add New Job Skills Without Going Back to School
- Slideshow: 7 security mistakes people make with their mobile device
- iOS vs. Android: Which is more secure?
- 11 sure signs you've been hacked
- Four Myths of High-Productivity App Dev Debunked Debunk the main myths surrounding high-productivity application development and how both platforms have overcome them.
- IDC: Dual Perspectives on ITaaS This global survey by IDG Research Services of more than 350 IT and BU directors at enterprises of 1,000 employees or more reveals...
- ESG Whitepaper: Integrated Computing Platforms: Infrastructure Builds Tomorrow's Data Center Thought Leadership Report: 'Integrated Computing Platforms: Infrastructure Builds for Tomorrow's Data Center.' Highlights the challenges customers face when planning to deploy a Cloud...
- Analyst Report-Mixed All Flash Arrays Delivers Safer Higher Performance What is the impact of an all-flash array with enterprise features and reliability on the mainstream data center? In the mainstream environment, storage...
- Live Webcast Increasing the Value of Your Reports and Dashboards Learn how incorporating other analytical capabilities such as predictive modeling and visualization can increase the value of your reports and dashboards by providing...
On-Demand Webcast: 7 Reasons to Choose VoIP
Thinking about a new phone system for your business?
Be sure to watch this informative webcast. Steve Strauss, small business columnist for USA...
- Top 8 Communications Tools for Small Businesses Powerful technology is available to help your small business improve its communications with customers, employees and suppliers. View this free On-Demand Webcast produced... All Business Intelligence/Analytics White Papers | Webcasts
By Rob F. Walker, Ph.D.
In the previous installment, we looked at and discussed strategies for business simulation and the infrastructure needed to make such initiatives successful. Now, we¿re ready to discuss some practical examples of business simulation. Imagine a mail order company selling products together with the necessary financing. more