Skip the navigation

Big data, big jobs?

By Tam Harbert
September 20, 2012 06:00 AM ET

"Everybody's asking, how do you identify these people? What skills do you look for? What is their degree?" echoes Greta Roberts, CEO of Talent Analytics Corp., which makes software designed to help employers correlate employees' skills and innate characteristics to business performance.

The skills most often mentioned in connection with big data jobs, say Roberts, Phillips and others, include math, statistics, data analysis, business analytics and even natural language processing. And although not consistent, some titles, such as data scientist and data architect, are becoming more common.

Must have: Intense curiosity

As companies search for talent, they are looking more towards application developers and software engineers than to IT operations, says Josh Wills, senior director of data science at Cloudera, which sells and supports a commercial version of the open-source Hadoop framework for managing big data.

That's not to say IT operations are not needed in big data. After all, they build the infrastructure and enable the big data systems. "This is where the Hadoop guys come in," says D.J. Patil, data scientist in residence at Greylock Partners, a venture capital firm.

"Without these guys, you can't do anything. They are building incredible infrastructure, but they are not necessarily doing the analysis." IT staff can quickly and easily learn Hadoop through traditional classes or by teaching themselves, he notes. Burgeoning training programs at the major Hadoop vendors testify to the fact that many IT folks are doing so.

That said, most of the jobs emerging in big data require knowledge of programming and the ability to develop applications, as well as knowing how to meet business needs.

The most important qualifications for these positions are not academic degrees, certifications, job experience or titles. Rather, they seem to be the soft skills: a curious mind, the ability to communicate with non-technical people, a persistent -- even stubborn -- character and a strong creative bent.

(Article continues on next page)

Big data skills and titles

Without conventional titles, or even standard qualifications, it's hard to know what makes someone suitable for a big data job. This listing, based on interviews of big data experts and recruiters, attempts to match up some of the most common titles with the skills required.

  • Data scientists: The top dogs in big data. This role is probably closest to what the McKinsey report calls "deep analytical talent." Some companies are creating high-level management positions for data scientists. Many of these people come out of math or traditional statistics. Some have backgrounds or degrees in artificial intelligence, natural language processing or data management.
  • Data architects: Programmers who are good at working with messy data, disparate types of data, undefined data and lots of ambiguity. They may be people with traditional programming or business intelligence backgrounds, and are often familiar with statistics programs. They need the creativity and persistence to be able to harness the data in new ways to create new insights.
  • Data visualizers: Technologists who translate analytics into information a business can use. They harness the data and put it in context, in layman's language, exploring what the data means and how it will impact the company. They need to be able to understand and communicate with all parts of the business, including C-level executives.
  • Data change agents: People who drive changes in internal operations and processes based on data analytics. They may come from a Six Sigma background, but also have the communications skills to translate jargon into terms others can understand.
  • Data engineer/operators: The designers, builders and managers of the big data infrastructure. They develop the architecture that helps analyze and supply data in the way the business needs, and make sure systems are performing smoothly.

Be sure not to miss...



Our Commenting Policies