Cloud Integration 2011: SnapLogic Brings Easy Design, Hot Restarts

Start-up firm SnapLogic is shaping the cloud integration sector in 2011 with two core capabilities. The company’s Snaps let users assemble integrations for data and apps as easily as putting together Lego blocks. Also, with RESTful services it connects any-to-any data sources and offers hot restart for failed integrations.

Tags: cloud, integration, REST, SanpLogic, SaaS,

smartlinkingStart-up firm SnapLogic is shaping the cloud integration sector in 2011 with two core capabilities. The company’s Snaps let users assemble integrations for data and apps as easily as putting together Lego blocks. Also, with RESTful services it connects any-to-any data sources and offers hot restart for failed integrations.


Integration Developer News speaks with members of the SnapLogic team to learn more about their approach to cloud integration, and the market opportunities the company sees in 2011.


“With the SaaS explosion, and a growing demand to connect data back and forth from the cloud to legacy systems, there will be too many apps for anyone to build individual connectors for each,” Clark Newby, SnapLogic senior vice president, told IDN.


SnapLogic has found in 2011 a popular cloud integration use case arises from the adoption of multiple SaaS solutions within a company, sometimes known as SaaS sprawl. “Many businesses we work with are adopting multiple SaaS solutions, but they also intend to keep a lot of their data in on-premise systems. We let users define and manage their data integration to and from the cloud by a data pipeline,” Newby said.  


To bring control out of chaos, SnapLogic’s approach is based on what the company calls “intelligent Snaps,”  which are a library of application-specific connectors that also transforms the data and abstract many aspects of connectivity, so they can be pre-built and easily configured using a simple drag and drop UI.


CloudCon Integration“We think it’s a more scalable approach than connectors,” Greg Benson, director of research at SnapLogic, told IDN. Assembling Snaps, users can design pipelines, define transforms, build joins or even more complicated integration tasks without programming. Once designed, the pipelines are then put into runtime by running on a discreet SnapServer, which understands and executes the abstracted instructions, he added. The SnapLogic suite of technology addresses  the full design-test-deploy-manage lifecycle of cloud-to-cloud and cloud-to-onsite integration using these core components.


SnapLogic’s architecture takes a REST-based approach to connect any-to-any, with each data source having its own “Snap.”


Inside SnapLogic’s Cloud Integration Architecture
The SnapLogic architecture spans the full lifecycle approach to cloud integration, including design, testing, deployment, as well as on-going operations and management. The following overview details the key aspects of the SnapLogic composite framework and modular offerings.


SnapLogic Snaps
These are the smart connectors that hold integration logic and information needed to achieve the integration. Snaps are distinct for each application or resource, and are built to understand the context, data and other discrete characteristics of the application it is connecting with.


In specific, Snaps are containerized data connectors that provide a quick way to join applications and data sources resulting in dramatic reductions in system costs and implementation times. As a result, by mixing and matching with Snaps, customers have addressed approximately 2,000 different integration scenarios, Elias Terman, Snaplogic’s director of product marketing, told IDN.


“SnapLogic now offers 87 Snaps in the store, with new ones added each week,” Terman said, adding that if customers need a Snap not already in the SnapStore developers can create their own Private Snaps using a SnapLogic Eclipse plug-in and Maven scripts to get most of the project completed. “A project can be done quickly because all Snaps follow the same pattern, use the same API, and leverage functionality of the SnapLogic integration platform,” Terman said. As an example, SnapLogic’s Snap for Twitter was built in just a few days by a mid-level Java programmer, he added. 


A [cloud integration] project can be done quickly because all Snaps follow the same pattern, use the same API and leverage functionality of the SnapLogic integration platform.

Elias Terman
Director, Product Marketing

Some Snaps also come with components (such as wizards) that can capture underlying information about the data (metadata, SQL fields, etc.) to help align fields, maps and even processes. With Snaps’ modular approach to step-by-step integration and pipelines, users can build and run all different sizes of pipelines using drag-and-drop assembly via the SnapLogic Designer.


SnapLogic Designer
This GUI tool is a drag-and-drop browser-based application used to create and configure end-to-end data integrations, or what Snaplogic execs call workflow pipelines. Pipelines are discreet end-to-end integration tasks broken down into a step-by-step sequence of sub-tasks, Benson told IDN.


To build a pipeline, users use their GUI tools to choose the snap components they need (from the SnapLogic store) to mirror their intended dataflow and integration needs. This could be SafesForce to another SaaS or on-premise application or database.


“With the SnapLogic Designer, we provide a representative screen example of what customers are trying to do, and that can be a lot of things including tie three sources of data together and then do some transformations if needed and even do a complex join so we can continue the integration to a new on-premise or cloud application,” Benson said. “We focus on the steps users can do which are best at moving and synchronizing data.” 


SnapLogic Designer also provides a wizard-like applet that can help designers align their data by proposing field mappings between sources and targets. For example, for the company offers a wizard that can extract exactly what information any account may have and lets you build deeper and more valuable connectivity across to other data records.


The tool also comes with a debugger to let designers review input and output of each component. Once designed and pre-tested, the pipeline is published to the SnapLogic Server for runtime operations.

SnapLogic Server
The Snaplogic Server is the heart of the runtime capability for data integration. It serves as the execution engine for all the design-time Snap pipelines.


Notably, it uses RESTful architecture to deliver many mission-critical features, including hot restart, in case of integrations that fail. “We think the RESTful nature of the tech make them more reliable—everything has a URI and an address and is built to run on the public internet,” Benson said. “This means Snaps know if they lose a connection, and can retry or restart that connection. Because there is a lot of intelligence in a Snap, using REST bring users even more benefits.” 


The SnapLogic Server platform also supports server clustering and workflow optimization for enterprise integrations, predictive field linking, and performs transformations, joins and many other core integration tasks. It can be run on-premise or in the cloud, and supports Linux (CentOS and Ubuntu), Windows and Mac OSX.


The SnapLogic Server also manages security, authentication, logging and runtime activities. Users and IT can get alerts and manage their integrations using the SnapCenter Console.


SnapLogic Console
This console provides an easy-to-navigate GUI dashboard with functions for managing integrations and pipelines. Customizable for a full range of IT and business stakeholders, the console: 

  • Displays for an entire pipeline ecosystem for one or more SnapLogic Servers
    Offers click-on details about each element in the ecosystem
    Shows immediate updates of any pipeline, so that each user has insight into critical business processes
    Lets users pre-configure alerts for email notifications about notable exceptions, failures during operations
    Offers customizable logging to drill down into history by pipeline attribute (name, server name, time started and ended, errors, etc.) >

SnapLogic does not charge per connection, so users can deploy the SQL Server Snap, for example, to connect as many discrete SQL Server databases as needed without added per-connection charges.