Splunk ups the machine-learning ante

Last week a competitor decided to poke the bear. This week Splunk comes out swinging.

security code big data cyberespionage byte
Gerd Altmann (CC0)

I wrote just the other day of Elastic’s acquisition of machine-learning company Prelert and how Elastic believed the deal would see it take market share from Splunk, the publicly listed granddaddy in the space (assuming, of course, you can have a granddaddy in a space that is only a decade or so old).

Anyway, Splunk heard the claim, and didn’t like it. The company had a machine learning announcement Tuesday and is trying to articulate just how much it offers beyond what Prelert (and other vendors) can offer.

The overarching announcement is that of the new version of Splunk’s various products -- Splunk Enterprise, Splunk IT Service Intelligence, Splunk Enterprise Security and Splunk User Behavior Analytics. All of these individual products have been “souped up” with the latest Splunk take on machine learning in order to leverage machine data within a particular use case.

Despite these being individual product offerings, they leverage a common machine-learning platform that Splunk asserts will allow non-technical users to benefit from machine learning. Diving into the nitty gritty, the latest integrations now support both univariate and multi-variate anomaly detection. To translate, multi-variate algorithms can analyze a set of metrics or KPIs that are expected to behave in a similar fashion in order to alert organizations when one or more of them departs from their peers. For example (since this is a little bit rocket science and hence examples make it easy for us mere mortals to understand), in the case of data center operations, Splunk would alert users when a combination of CPU usage, disk usage and network usage metrics start behaving abnormally based on their historical behavior.

The underlying trend here is one of leveraging data and learnings and applying them to basic business problems. The management mantra of continuous improvement, so prevalent in past decades, can be automated and replaced by systems that continuously improve themselves within human intervention:

“Digital transformation has changed the way that organizations work. The big secret is that all of the change is underpinned by machine data. Machine learning enables organizations to get deeper insights from their machine data and ultimately increases the opportunity our customers can gain from digital transformation,” said Doug Merritt, president and CEO of Splunk. “The enterprise machine data fabric is the foundation for managing and deriving insights from that data at scale -- and only Splunk provides the end-to-end analytics platform and ecosystem to support it.”

This sounds good, but how does it actually apply to real-world use cases?

Machine learning changes the conversation around big data -- no longer is it simply data for its own sake, but rather it is the ability to automate the learning, improvement and action-taking parts of what data can lead to. This is a natural extension for Splunk who have always been about slurping in as much data as possible. Potential use cases the company sees include:

  • Focused investigation: Identify and resolve IT and security incidents by automatically detecting anomalies and patterns in data.
  • Intelligent alerting: Reduce alert fatigue by identifying normal patterns for specific sets of circumstances.
  • Predictive actions: Anticipate and react to circumstances such as proactive maintenance that might otherwise disrupt operations or revenue.
  • Business optimization: Forecast demand, manage inventory and react to changing conditions through analysis of historical data and models.

So these different product offerings take the broad platform of Splunk’s machine learning and apply them to different business problems -- security, service management etc. Splunk’s machine learning focuses on three main areas: 

  • Clustering: Taking a bunch of data and putting them into groups.
  • Classification: Coming with a prediction.
  • Regression: Using historical values and coming up with future predictions.

Splunk is thinking about both broadly applicable fabric, and highly specific vertical use case here. For the former, the Splunk platform offers more than 20 machine-learning commands out of the box. The Machine Learning Toolkit extends the Splunk platform to take advantage of open source Python libraries with over 300 different algorithms. These can be applied directly to the data for detection, alerting or analysis for specific use cases, whether for IT or security. Additionally, the ML Toolkit provides a guided workbench for data scientists to build their own models.

All of this sounds great, but it is where the machine learning rubber hits the road that really counts. So how is Splunk’s platform being utilized out in the field? Some examples:

  • Telus uses machine learning to monitor noise rise from more than 20,000 cell towers to increase service and device availability and mean time to repair (MTTR).
  • Zillow uses custom outlier detection to find server pools that cause massive deviations in error 500s due to code and configuration changes.
  • Kinney Group used the Splunk ML Toolkit for Schmidt Peterson Motorsports in the Indy 500. In conjunction with support from Splunk, Kinney Group monitored track conditions and car performance at the Indy500 and during qualifying. Real-time operational data analysis was conducted on all three SPM race cars during the event.


I’m a sucker for companies that think both broad and deep. By offering a broadly-applicable machine-learning platform, Splunk offers organizations the ability to resolve all the random business problems that Splunk will never design a specific offering for.

But at the same time, the various deep vertical products it offers make a ton of sense and allow the easiest on ramp for organizations with those particular problem sets to begin using machine learning. Splunk has done a good job of growing into its publicly listed skin and remaining focused on product execution -- this new platform looks to be a good example of that.

Copyright © 2016 IDG Communications, Inc.

7 inconvenient truths about the hybrid work trend
Shop Tech Products at Amazon