Data and AI trends 2018: reality replaces hype through deep learning, data governance, conversational AI


AI became a ubiquitous buzzword in 2017, but the promised rise of the machines has thus far failed to materialise.

Despite the buzz, real-world use cases have remained somewhat limited, but this looks set to change in 2018.

Reality replaces hype

Companies are already ploughing cash into AI, but much of this investment is yet to reap rewards. According to research firm Forrester, 55 percent of firms have not yet achieved any tangible business outcomes from AI, and 43 percent of them say it’s too soon to say whether their investment has been a success.

"There's been a lot of pilots and a lot of proof of concepts, but the reality is not only what AI is and what AI can do, but also what is required to build these new intelligent systems is starting to be realised,” says Michele Goetz, one of the authors of a Forrester report titled The Honeymoon For AI Is Over.

Read next: How to use data scientists and machine learning in the enterprise

She adds that organisations are starting to understand the need to invest in the technology with a clear business objective in mind. The business owners, data scientists and technologists all need to be involved in the process.

“It's going to require that you have experts not just thinking about how to create the plumbing of a technology, but thinking about it in terms of what the business gets out of this.

“If you can't define an objective that’s not just small incremental values, your opportunity cost for artificial intelligence is overwhelming and won't realise the return on investment.”

Emerging industries for AI

Goetz’s sentiment is echoed by Jane Zavalishina, the CEO of Yandex Data Factory. She believes that the growing understanding of advanced data science will yield results in traditional industries as well as digital-first companies.

“They've realised that it's not just about an exciting technological investment," she says. "Now they understand that AI is actually a practical business tool. So if in 2017 it was about understanding what AI can bring to the table, in 2018 it will be about real practical adoption.

"What that means is we'll change the picture of what AI is, from thinking of AI as those exotic cases with robotic assistants and winning over humans in chess and Go, we'll see that 'invisible AI'."

This 'invisible AI' is the application of machine learning to optimise processes in the background for traditional businesses, such as the oil and gas industry.

By analysing historical data, these organisations can create a more accurate predictive model of how their equipment functions and generate real-time recommendations on maintenance and where to make changes that drive efficiencies.

The growth in IoT sensors and analytics at the edge is also making industrial AI more accessible, and now that the benefits are gaining publicity, other industries renowned for their reluctance to embrace new technology are starting to understand the value to their businesses.

Read next: What is edge computing?

Research, administrative, consulting services and any other sector that makes suggestions or reveals findings based on customer information also face significant disruption.

"We're going to see a lot of disruption because machines can look across vast amounts of different types of information and pool that together much faster and come up with insights a lot faster and more insights than a human can do," says Goetz.

Focus on the data

Data science made significant strides in 2017, but its potential can be inhibited by poor data quality. This often leads data scientists to spend an excessive quantity of time on preparing the data and filtering out anything of poor quality.

"I think one of the practical areas that is going to have to be addressed in 2018 and beyond is that a formal part of big data efforts and analytics efforts is going to be a data quality focus that has to be factored in,” says Ravi Rao, SVP of pre-sales at analytics and software provider Infogix.

"From that standpoint, I think data governance is also going to become a major formal consideration as part of any analytics effort and not an afterthought or an assumption."

Read next: GDPR tips: How to ensure compliance with GDPR

Rao believes data governance will become more automated in 2018, to ensure that as new data automatically updates, the controls that need to be put in in place around it in the new systems or the new datasets follow.

"It's going to become a necessary technical challenge going forward," he explains. "That's where I think artificial intelligence or machine learning starts playing a key part, not just in the advanced analytics, but in all the things leading up to the advanced analytics, such as data quality and data governance."

Deep learning availability

Data scientists remain in short supply and high demand, but deep learning should become more accessible due to a lower cost of entry.

The three major cloud providers of Google, Amazon and Microsoft all now offer an hourly rental service for their graphical processors, which customers can use to train their neural networks.

Read next:  What is deep learning?

"Once the training is done, it doesn't actually require very much compute at all to make the predictions," says Smith.

"Being able to rent these very powerful graphics processing units for a few hours and then, once the training is done, being able to use that rather than having to actually physically go out and buy them really helps accelerate that."

Rise of the chatbots

Insightly CEO Anthony Smith believes that 2018 will be the year that AI-driven chatbots become commonplace in businesses hoping to streamline the customer experience and reduce the need for call centre agents.

"One of the things that I think is going to be very hot for CRM and marketing in 2018 is going to be conversational interfaces, whether they be voice-based conversational interfaces or text-based conversational interfaces," says Smith.

Read next: Why did Salesforce Ventures invest in’s enterprise chatbot company?

"They've been very hard to build in the past couple of years. There's been some work done by Facebook and Google in the consumer space to make the tooling to build these chatbots easier, and we're now starting to see some of that tooling mature, and we can utilise some of that in the B2B space as well."

The voice can also provide a compelling alternative to online forms, through chat-based interfaces that refine or limit the questions asked based on the user's previous answers.

"It's really a lot quicker and for a lot of people, it's a lot less friction than having to fill out the entire form every time you want something from a website," says Smith. "We feel that's going to be very hot in 2018."

Changing roles

Rao believes that the growing importance of data analytics means that the entire organisation now has a supporting role to play.

"To put a data quality and data governance programme in place and to implement it right, there has to be an organisation to support that," he says. "It cannot be the role of the analytics team. It cannot be the role of the traditional IT team, because their core functions are not this.

"I think that's where the organisation of the chief data officer comes in place, to recognise and make sure that any analytics initiative includes a formal data quality and data governance approach. In that sense, I think the CDO is going to become an increasingly important figure and department in any organisation,"

The rise of the Chief Data Officer showed no signs of abating, but other data roles look likely to also gain prominence.

"The chief analytics officer I think is also going to play a strong role in further extending the sophistication of capabilities for analytics," says Goetz.

Read next: How to get a job as a data scientist

The Forrester analyst is less convinced that the Chief AI Officer will become commonplace.

"I don't think so," she says. "I think the officer that is really looking broadly at what technology does for a business and the strategic aspect of that is really the chief digital officer, and oftentimes the chief digital officer and the chief data officer work together, or sometimes the chief data officer is reporting to the digital officer."

Zavalishina believes new roles are set to emerge that are yet to be understood.

"It helps if you understand how data science works of course, but you also need to be a very business-minded person, at the intersection of management and data science. While data scientists are very much in demand and it's hard to find a good one and it's very expensive, this new type of people, they don't even exist yet.

"We'll see much more need for these kind of people, and we'll see companies start to learn how to do that."

Copyright © 2017 IDG Communications, Inc.

Shop Tech Products at Amazon