Splunk Marries ‘Machine Data’ with ‘Machine Learning’ for New-Gen Insights for IT Ops, Business, Security

Splunk is infusing its range of machine data analysis offerings with machine learning. The result: more intelligence and drive faster insights for IT, business and security.

Tags: alerts, analytics, data prep, Hadoop, machine data, machine learning, real-time, Splunk, UBA,

Splunk is marrying machine learning with machine data across all major product lines to deliver more intelligence and drive faster insights for IT, business and security. 


The initiatives, revealed during this week’s Splunk .conf2016, reflect what Splunk execs see as a growing and insatiable appetite by companies to tap business value from exploding data streams.


IDN hears from a top Splunk exec on the move, and takes a tour of Splunk’s current and coming machine learning updates.


Splunk’s latest efforts to form a new equation -- machine data + machine learning -- equals the opening of “a new era” in analytics, analysis and business insights according to Splunk chief executive Doug Merritt.


“Splunk is enabling companies to use predictive analytics to help optimize IT, security and business operations. Machine learning is being integrated as a core capability of the Splunk portfolio with packaged or custom algorithms to operationalize machine data in a variety of valuable use cases,” he added in a statement.


Under the covers, Splunk products are using machine learning techniques to identify anomalies and patterns that can speed investigations and intelligence discovery, according to the company’s website. Notably, the capabilities come as ‘pre-built’ options, designed to meet specific use cases. Splunk has also harnessed machine learning to identify ‘noise’ and outliers that can distort results – a common problem when dealing with massive number of events in the data.


How Splunk’s Adoption of Machine Learning Will Modernize IT Operations

Splunk Cloud and Splunk Enterprise 6.5 are leveraging machine learning to deliver benefits across the data lifecycle – from capture-to-analysis-to-insight. Notable ones include a new user experience for data analysis and preparation:

  • Simplification of data preparation.
  • A guided workbench to create custom machine learning models for IT, security and business use cases.
  • Advanced analytics available using a rich new set of user commands.
  • Expand availability of data analysis to more users thanks to a new, easier-to-use intuitive interface (including table data views friendly to both specialist and occasional users).
  • Tighter integration of machine data with Hadoop ‘big data’ sources. Organizations can now roll historical data to Hadoop and utilize hybrid search to analyze all of their data in Splunk.

The latest edition of Splunk ITSI - IT Service Intelligence (version 2.4) also applies ‘pre-built’ machine learning capabilities to event data. This focus will help improve productivity across IT and the business in various ways:

  • Present real-time service insights in a more consumable way by prioritizing incidents through event analytics.
  • A more unified ‘single view’ of operations by bringing multiple data together to present an intuitive interface.
  • Improve service operations thanks to the ability to baseline normal operational patterns and later to dynamically adapt thresholds.

Coming this fall, Splunk ES - Enterprise Security (version 4.5) will provide a common interface to automate data retrieval and sharing, as well as a coordinated incident response based on applying machine learning to data from multi-vendor environments.


Also this fall, Splunk UBA - User Behavior Analytics will work with Splunk ES to improve detection, investigation and remediation times. Impacts of security metrics will be easier to understand and threat detection will be improved--thanks to packaged machine learning-based anomaly detection.


Splunk’s Merritt put the company’s commitment to machine learning into context in a recent blog post:

Everyone from big data pioneers to industrial enterprises now have to adapt to this digital transformation. As we look around us, it’s clear what’s at the core of this digital evolution: a constant stream of data being emitted from the always connected, always on, world around us. At Splunk we believe that this machine data, the data generated from your servers, containers, firewalls, networks, sensors and countless other sources, is at the center of your business operations. Digital transformation demands that you capture and harness this constant flow of data and make it easily accessible and valuable across all of your departments, users and use cases.

Merritt went on to share how Splunk’s move into machine learning will change many high-profile use cases for data analysis.

Focused Investigation: Identify and resolve IT and security incidents by automatically detecting anomalies and patterns in data.


Intelligent Alerting: Cut down on false alarms and reduce ‘alert fatigue’ by identifying normal patterns for specific sets of circumstances.


Predictive Actions: Better anticipate and react to circumstances such as proactive maintenance that might otherwise disrupt operations or revenue.


Business Optimization: Forecast demand, manage inventory and react to changing conditions through analysis of historical data and models.

Interested users can download Splunk Enterprise or try Splunk Cloud. They can also sign up for a free online sandbox of Splunk ITSI.


Beyond efforts to infuse operational intelligence with machine learning, Splunk is also interested in human smarts.  The company announced investments in a ‘developer ecosystem’ to spur further innovations for the Splunk platform.


Notably, Splunk revealed targeted investments in two promising partner companies: Acalvio focuses on advanced threat defense and combatting cyberthreats that help companies detect, engage and respond to advanced attacks in a precise and timely manner.  Insight Engines empowers users to natural language search to more quickly uncover and analyze data.