Splunk Expands Ease of Use, Value of Machine Data with Latest Parade of Updates

Splunk is shipping key technology updates to extend the value and ease-of-access of machine data for both IT and business users. Splunk also is offering cloud-based versions to simplify access and allow pay-per-hour use.    On top of everything, new mobile tools are on tap.  IDN speaks with Splunk execs.

Tags: Amazon, analytics, big data, cloud, clustering, dashboards, Hadoop, Hunk, logs, machine data, MongoDB, NoSQL, MINT, real-time, Splunk,

Leena Joshi
senior director,
solutions marketing

"With these updates. . . Splunk will greatly broaden the number of users that can easily do advanced analytics."

Splunk is shipping key technology updates to extend the value and ease-of-access of machine data to both IT and business users. Splunk also is offering cloud-based options via Amazon Web Services to simplify access, and even allow pay-per-hour usage plans.


“With these updates, we spent a lot of time improving many areas for how to prepare and work with machine data. As a result, Splunk will greatly broaden the number of users that can easily do advanced analytics,” Leena Joshi, Splunk’s senior director of solutions marketing, told IDN. “Splunk Enterprise 6.2 aims to put analytics in the hands of more non-technical users by delivering simplified analysis and powerful pattern detection that can uncover relationships in vast amounts of data,” she added.


To simplify analytics for non-technical users, Splunk Enterprise 6.2 adds these innovations:

  • A new ‘Instant Pivot’ feature. This gives users a pivot interface to easily and automatically discover relationships in their data and even build reports without using Splunk Search Processing Language. “Now, a user can instantly pivot and get an Excel-like interface to easily rearrange fields and filter and other tasks for new insights – all without knowing how to use Splunk’s search language,” Joshi said.
  • Pre-built dashboard components (called panels). These let users easily assemble and integrate multiple charts and views for real-time data display. Even more valuable, Splunk experts (who often end up having to do the hands-on work for business users looking for Splunk analytics reports) can create sets of custom pre-built panels that will allow self-service among end users, Joshi explained. The new dashboard panels work with both conventional and mobile devices, she added.
  • A new intuitive wizard for machine data prep. This modern UI makes it easier and faster for users to onboard any machine data. A clever interface guides users through previewing, onboarding and preparing their machine data for downstream analysis.

Splunk Enterprise 6.2 also has some goodies for savvier Splunk users. Expanded scalability thanks to “enhanced clustering” supports up to double the number of concurrent users and searches, as well as better redundancy. Further, a new distributed monitoring console lets IT managers monitor multiple Splunk Enterprise large-scale deployments from one place.


Splunk Enterprise 6.2 even adds features to help the core Splunk platform itself better understand new data sources. As one example, Splunk’s newly-added Advanced Field Extractor simplifies the ability to add context and meaning to machine data by identifying, naming and tagging fields and data points. Splunk can even auto-detect meaningful patterns, making pattern detection more discernable, Joshi elaborated.


Splunk’s Hunk 6.2 Takes Hadoop Projects To New Levels of Simplicity and Efficiency

Big Data in Motion Summit
Manage Expanding Data Volumes for Analytics & Operations
Jan 29, 2015
Online Conference

Splunk is also shipping a revved-up version of its Hunk for Hadoop and NoSQL data stores. Splunk’s Hunk 6.2 extends the power of exploratory analytics with a range of features.


Like Splunk Enterprise 6.2, Hunk 6.2 delivers ‘Instant Pivot,’ pattern detection and prebuilt dashboards / panels. In addition, it delivers specially-tuned upgrades for deriving analytics from Hadoop and NoSQL stores, whether on-premises or cloud -- without the need to build fixed schemas or move data, Clint Sharp, director of product management, told IDN


Sharp described the advances on tap for Splunk’s cloud-based Hunk features, which are designed to work with Amazon architectures.


“Historically it’s been difficult to analyze data in [Amazon] S3 buckets, mainly because you’d have to make a copy of that data and move to a local resource. This update is brilliant because Hunk works with Splunk’s app for [Amazon’s] Elastic Load Balancing service, which writes its logs files to S3 buckets,” Sharp said. The result: Users can use the traditional Splunk interface to access that Amazon data. “This means users don’t need to know anything about the architecture or the location. S3 is just another cluster to Splunk now,” he added.


To make cloud-based analysis of machine data even easier and affordable, Sharp said Splunk is offering one-click purchase of the Amazon EMR (Elastic MapReduce) web service that uses Hadoop, an open-source framework, to process vast amounts of data. The Console 1-Click Purchase feature instantly leverages automatically-configured Hunk instances provisioned by AWS, priced hourly, for data in Amazon EMR.


The ability to access Splunk from the Amazon EMR console provides an easy method to leverage Hunk analytics, added Alys Woodward, research director for IDC’s Advanced and Predictive Analytics unit.


Hunk also provides an efficient and simple alternative to Apache Hive or SQL on Hadoop. As described in Splunk’s Hunk product tour

Hunk is built upon Splunk ‘schema-on-the-fly’ technology, which means that you don't need to know anything about your data in advance. Search results return with an automatic structure based on fields, keywords, patterns over time, top values and more. Even event breaking and timestamp extraction are done at search time. Hunk automates access to text files and sequence files along with popular Apache Hive data formats including Record Columnar Files (RCFile), Optimized Row Columnar (ORC) files and Parquet columnar files. All without having to move data out of your Hadoop cluster or NoSQL store in order to analyze or visualize it.

When you run a query in Hunk it streams back interim results immediately while the MapReduce job continues to run in the background. This delivers a faster, more interactive experience because you can pause and refine queries without having to wait for full MapReduce jobs to finish.

Other Hunk 6.2 improvements include:

  • Hunk Sandbox, which lets users rapidly learn Hunk interactive search and analytics in a single download -- without having to set up a Hadoop cluster.
  • Data Explorer, a feature that lets users select the most relevant data sets for analysis in Hunk with a visual wizard to browse and prepare raw data in Hadoop.
  • Hunk Apps to let users search, analyze and visualize data in NoSQL and other data stores through prepackaged connections. The apps include: Hunk App for MongoDB and Sqrrl App for Hunk (Apache Accumulo) and for AWS Elastic Load Balancing.

Adding it all together, Splunk’s senior vice president of products Guido Schroeder, described the total benefits this way “By giving people the chance to spin up instant analytics the moment they put data in Hadoop, Splunk on AWS will drastically decrease time to value for organizations.” Terry Wise, director of Amazon Web Services worldwide partner ecosystem agreed, noting in a statement, “Using Hunk, AWS customers can better utilize massive data sets stored in Amazon EMR to get clear and concise business insights that ultimately contribute to the most accurate IT and business decisions.”


Splunk MINT Set To Deliver Operational Analytics to Mobile Apps, Performance

Beyond these Splunk updates, new mobile analytics tools are also on tap from the company. A cloud-based Splunk MINT Express service is now live, which offers devs a way to configure and capture a rich set of operational analytics about quality, usage and performance from their mobile apps. Splunk MINT Express provides key capabilities across the mobile lifecycle to track performance, use and success of apps, including:

  • Quick instrument mobile apps with a single line of code.
  • Gain real-time insight into the performance, quality and usage of mobile apps.
  • Find and fix network and transaction performance issues.

To further extend insights mobile app analytics, Splunk is also in beta with Splunk MINT Enterprise, a platform to integrate this rich set of real-time mobile app data with data in Splunk Enterprise or Splunk Cloud. The combination will give customers end-to-end visibility and real-time, omni-channel application analytics, Bill Emmett, Splunk’s director of solutions marketing, told IDN.


“These are our first foray into machine data for mobile apps, and we’ve got approaches to drive value from mobile apps for both enterprise and consumer,” Emmett said. Both mobile-centric initiatives are also the first to arise from Splunk’s acquisition of Bugsense.


“Mobile apps data can provide all types of valuable insights into the success of an app or how users use them. But, combining mobile analytics with machine data from web and traditional apps can tell all of IT and business [managers] about business-critical interactions across channels,” Emmett added.


One Splunk partner, Cvent, is keen on the value from Splunk MINT Express. "Splunk MINT Express will help us identify unusual app performance issues before they impact our mobile users and gain new insights into user engagement, so we can continuously improve the customer experience,” said Al Funk, Cvent’s director of mobile product development, in a statement.


Free downloads / access are available here to Splunk Enterprise Online Sandbox, Splunk Cloud and Hunk