JavaOne: Unlocking the ESB Secrets

During JavaOne, devs will get a much better look at how an ESB (Enterprise Service Bus) might boost their careers. Dave Chappell, one of Java's foremost experts on JMS and integration, in his latest book spells out why ESBs will transform how Java devs think about APIs, data sharing and workflow.

Tags: ESB, Java, Integration, Chappell, Standards, Bus, Java Devs,

During JavaOne, devs will get a much better look at how an ESB (Enterprise Service Bus) might boost their careers. Dave Chappell, one of Java's foremost experts on JMS and integration, in his latest book spells out why ESBs will transform how Java devs think about APIs, data sharing and workflow.

"One-hundred-percent portability is getting more difficult to achieve, as Java is fragmenting already," Chappell, CTO at Sonic Software, told Integration Developer News. "And while there is still a core set of [Java standards] that depend on the JCP, engineers can become more marketable if they can understand how Java can work in a cross-platform, service-oriented integration environment."

The solution, Chappell says, is the Enterprise Service Bus.

Chappell's book Enterprise Service Bus will be released next week by O'Reilly. IDN conducts an in-depth interveiw with Chappell on just what he thinks is important about ESBs, and why developers/architects should take notice -- and get ready to exploit them and the technologies they use and connect to (including XML, XPath, SOAP, JMS and even C#).

"If a developer focuses on making sure his silo is the best in the enterprise, and he's not also focused on making sure that his silo is built to integrate, then," Chappell told Integration Developer News,"he's headed for problems because new applications that are being built today should be built to integrate using standards-based interfaces."

Chappell says Java devs who want to avoid a stall in their career should set their sights on becoming "integration architects." This new high-visibility, high-reward job will shake Java devs out of their silos, helping them blend their current Java talents with new web services skills that can link Java and non-Java assets (app logic, data and business rules).

In that vein, Chappell also sees that ESBs may actually provide a way for the Java community to get back to core basics, by offloading current and future complex functionality from the baseline J2EE spec.

"The future of the J2EE stack today can't continue to grow and advance as well as it once did," Chappell says, "if everything that gets added to it has to be part of that stack, it takes too much time. The stack is too heavy and rigid, , which makes the certification process only attainable to a handful of vendors. This doesn't necessarily provide a fertile environment for fostering innovation.

Inside the "Developer-as-Integration Architect"
The word "integration" can raise the hackles -- and anxieties -- of many of today's Java developers. But Chappell says that ESBs will provide Java devs an easier way to perform complex integration tasks without making them learn every nuance about every back-end system (Java or non-Java) they need to connect with.

The ESB, as Chappell envisions it, is a powerful "enterprise architecture for a highly distributed integration network," encompassing a wide range of integration capabilities including routing, data transformation, loosely-coupled data exchange, and the separation of application logic from business processing rules.

Further, ESB will likely not even be built up from 100% Java code, Chappell said. While ESBs will leverage some new Java standards, such as JSR 208 [for Java Business Integration], they'll be built on core cross-platform standards such as XML, WSDL, XSLT, and various security standards. But, Chappell adds, no matter how much core Java code makes up the ESB, the focus of the ESB will be Java-outward, rather than deeper and deeper knowledge of Java/J2EE APIs, Chappell said. And, he added, it's a shift in thinking outside the Java silo will benefit Java devs.

ESBs Update Pure Java-Driven Integration
Chappell says that the standards and technologies that will comprise an ESB have matured enough that it's safe for Java devs to begin thinking about leaving 100% Java-driven integration behind.

Traditional Java messaging and middleware approaches, he says, focus on "one point-to-one-point" or "one-to-many" data transfers. These approaches do not easily upgrade to a services-oriented architecture "many-to-many" approach for sharing data, as well as higher-level application and business logic as services.

Chappell cites five (5) reasons why CIOs will consider using ESBs, and shift away from traditional integration approaches:

  • Increased ability to integrate inter-departmentally, as well as with business partners using standard interfaces and standard protocols.
  • Leverage existing IT staff as architects: The amount of information available on Java standards such as JAXP, JCA, and JMS, and XML-related standards such as SOAP, WSDL, XSLT, XPATH and XQuery is expanding at an ever-increasing rate. This means that the average IT professional can readily attain the expertise he or she needs to become the in-house integration architect.
  • Reduced vendor lock-in by using a standards-based approach to integrating at the protocol, data and business intelligence layers. ESBs will reduce the appeal of proprietary middleware or integration stacks.
  • Reduced need for wide appserver deployments: Because integration functionality and business process routing is inherent in the ESB fabric,, IT shops can avoid the need to purchase and install J2EE app servers at all target and source integration points throughout their organization, simply for the purpose of providing integration functionality.
  • More predictable project milestones/deliverables: The ESB standards-based integration approach means that devs can reuse common design patterns and successful templates/models from project to project, without expensive consultants and vendor lock-in.

    The result: By 2006 (and probably sooner for many firms), Chappell predicts, IT execs will increasingly turn to ESB over traditional Java messaging and queuing approaches. "Today, while web services can provide some interoperability between applications, what customers really want is a cheaper, less complex way to doing integration," Chappell told IDN. "Java developers who keep that requirement in their minds will be the ones who do best over the coming years."

    The integration architect will "tie applications and services together in a service-oriented architecture using itinerary-based routing, content-based routing and orchestration."

    Inside the ESB -- Just How Does It Work?
    To envision how an ESB works Chappell suggests devs envision how a PC hardware bus works -- a central backplane with a simple, common interface and multiple slots (or plug-in points).

    To interact with a device plugged into the bus, another bus plug-in needs only to conform to the standard interfaces of the bus itself (along with the "rules of the road" that the bus requires for two-way communication). The plug-in device on the bus does not need to know the intricate details of another downstream device to connect/share with that device. The interfaces to the bus ensure compliance with all other bus plug-ins.

    Likewise, in an ESB, the "bus" has the internal transmission infrastructure to get data to the applications that need it, in the target data formats that they need to see it in. The key is that the interface to the ESB is a simple plug-in, defined by standard interfaces to web services and Java-based standards. This means that for a developer to "integrate" Source A with Target B, she or he only needs to know the ESB interfaces, and the addressing and schema of that target. All the data transformations that might be needed by the Target B application to interact with data from Source A are done by the bus itself.

    The ESB "Services Container" -- The Key to Next-Gen Integration
    So, how does the developer unlock the power of ESB-driven integration? Chappell says the ESB "services container" will be key -- the interface through which the dev-turned-integration architect will integrate.
    The ESB services container is a managed environment (using JMX MBean container, with event-driven XML based interfaces described using WSDL) that is capable of housing integration services -- both Java and non-Java. The container receives JMX management input, and has special output endpoints for error handling and auditing. Rather than tying source/target information to a message, the ESB services container allows the "integration architect" to administratively define destinations and other configurable aspects (data exchange, workflow, etc.) in a declarative fashion using configuration and process modeling tools.
    Chappell has very little interest in some of the "bundled app/business processing logic" approach being followed by some J2EE app server vendors. "I think it's [a] step… toward creating a big, unmanageable mess. Down the road, if you have all your routing logic built into your Java doc and your source code that's spread out all across your enterprise, how do you deal with all that? Furthermore, the resulting process definitions are compiled into class files that need to be redeployed every time a change needs to be made."
    In contrast, an ESB allows configurations to be changed without redeploying all of the affected services. The ESB approach separates application logic from business processing logic/workflow rules, and applies the process flow logic as needed to source/target assets that are plugged into the ESB as event-driven services. In specific, Chappell explains, an ESB services container uses a Java servlet-like in/out processing metaphor. Underneath the covers, the ESB invocation framework evaluates the message itineraries that are associated with the message to determine where to route a message next. The in/out endpoints are mapped to JMS topics and queues that carry SOAP envelopes. The topic or queue can be directed at an MDB or a JCA adapter, a specialized integration service such as XML caching, XSLT transformation, or could be mapped administratively to call out to an external web service. This mapping is done using SOAP over HTTP, or even WS-Reliability/WS-ReliableMessaging as those protocols reach maturity and become prevalent.
    Chappell concedes that just how the "integration architect" would access the ESB service API has yet to be decided. But the current direction could allow APIs to be implemented using such familiar (and open) toolkits such as Apache SOAP or Axis, and could conform to popular Java standards, such as JMS, JAX-RPC and SAAJ.