Hadoop skills are in high demand
Enterprise adoption of technology has outstripped number of skilled Hadoop pros
Computerworld - NEW YORK -- The growing enterprise interest in Hadoop and related technologies is driving demand for professionals with big data skills.
Analysts and IT managers at the Hadoop World conference here this week repeatedly pointed to skills availability as one of the key challenges companies face in adopting Hadoop and said that those with the right skills could command healthy premiums.
One indication of just how limited that skills supply is: IT executives from JP Morgan Chase and EBay who delivered keynote addresses at the conference used the opportunity to recruit from the audience.
Hugh Williams, vice president of experience, search and platforms at EBay, told audience members that the auction site is recruiting Hadoop professionals and he invited those interested in exploring opportunities to speak with him.
Larry Feinsmith, managing director at JP Morgan Chase, who followed Williams, only half-jokingly told the audience that Chase was also hiring and would be willing to pay 10% more than EBay.
"Hadoop is the new data warehouse. It is the new source of data" within the enterprise, said James Kobielus, an analyst with Forrester Research. "There is a premium on people who know enough about the guts of Hadoop" to help companies take advantage of it, he said.
Hadoop allows companies to store and manage far larger volumes of structured and unstructured data than can be managed affordably by today's relational database management systems.
A growing number of companies have begun tapping the technology to store and analyze petabytes of data such as weblogs, click stream data and social media content to gain better insights about their customers and their business.
The increasing enterprise adoption is driving demand for people with advanced analytics skills, Kobielus said. That includes people with backgrounds in areas such as multivariate statistical analysis, data mining, predictive modeling, natural language processing, content analysis, text analysis and social network analysis, he said.
"Big data in the broader sense -- and Hadoop in particular -- is driving demand for people who have experience doing advanced analytics using newer approaches such as MapReduce and R for predictive and statistical modeling," he said. These are the data analysts or data scientists who will work with structured and unstructured data in Hadoop environments to deliver new insights and intelligence to the business, he said.
Interest in Hadoop is also creating demand for Hadoop platform management professionals, Kobielus said. Their job will be to implement Hadoop clusters, secure, manage and optimize them and to ensure that the cluster remains available for enterprise use. "These are the people who build out and optimize the platform" on which Hadoop applications run, he said.


- Excel 2010 Cheat Sheet
- Register for this Computerworld Insider Cheat Sheet and gain access to hundreds of premium content articles, guides, product reviews and more.
- Thinking Outside The Data Warehouse
- This high level, business problem focused eBook uses 5 customer scenarios to show how people and organizations are tackling real issues using IBM...
- Using BD for Smarter Decision Making
- This paper looks at new developments in business analytics and discusses the benefits analyzing big data bring to the business.
- Measuring the Business Value of CI in the Data Center
- One of the key strategies that IT teams are pursuing to reduce capital costs while boosting asset utilization and employee productivity is the...
- Switching Schedulers - Not As Complicated As You Think
- Changing or consolidating job schedulers may seem daunting. However, the benefits of switching to enterprise workload automation outweigh the risks. Read how BMC...
- Capture-Enabled Business Process Management
- Organizations today must deal with a vast amount of incoming information from many different sources. Efficient, automated business processes are critical to managing... All BI and Analytics White Papers
- InfoSphere Warehouse Packs Demo
- These flash modules make warehousing more tangible and relevant to business users through detailed explanations of the InfoSphere Warehouse Packs.
- Delivery Management -- Extending Lifecycle Management
- Date: Wednesday, June 20, 2012, 1:00 PM EDT
Siloed organizations continue doing the wrong things and doing things wrong, leading to increased costs,... - Leverage automation today to reduce IT complexity
- Date: Tuesday, June 5, 2012, 2:00 PM EDT
Whether your B2B complexity is caused by multiple technologies due to M&A, business or application specific... - BMC Control-M - Single Point of Control Demo
- With BMC Control-M, you schedule and manage everything - down to the very last platform and application - from one simple interface. It's...
- BMC Control-M - Single Point of Control Demo
- With BMC Control-M, you schedule and manage everything - down to the very last platform and application - from one simple interface. It's... All BI and Analytics Webcasts