Hands-On: Is JBoss Ready for Your Enterprise?

It's been only a month since JBoss, the Open Source J2EE app server, received its full J2EE certification from Sun. Now, in the first detailed, hands-on review of JBoss offerings -- including technologies, service and documentation -- Integration Developer News turns to Open Source Java expert Bernard Golden to answer the question, "Is JBoss a good fit for your enterprise?"

Tags: JBoss, Open Source, OSMM, Support, Maturity, Assessment, Documentation,

IDN Executive Overview
Among topics covered in this article are:

  • A 6-step assessment of JBoss' maturity -- or enterprise-readiness. Areas investigated include: (a) Software, (b) Support, (c) Documentation, (d) Training, (e) Product integration, and (f) Professional Services
  • An overview of the Open Source Maturity Model (OSMM), a product-independent methodology that architects/developers can use to assess the suitability of a wide range of Open Source offerings; and
  • JBoss total "score" for enterprise-readiness, along with suggestions for weighting values based on your enterprise's risk tolerance and in-house available skills.

  • ======================================

    by Bernard Golden
    It's been about a month since JBoss, the Open Source J2EE application server, received its full certification from Sun.

    Now, in the first detailed, hands-on review of JBoss offerings -- technologies, service and documentation, Integration Developer News turns to Open Source Java expert Bernard Golden to answer the question, "Is JBoss a good fit for your enterprise?"

    While adherence to the J2EE certification standard offers certainty about JBoss conformance and functionality, it offers no such assurance about other aspects that may be critical for mainstream IT use. Such topics likely to be important for IT managers include.

    J2EE certification reveals nothing about thesuitability and quality of important elements technical support, documentation, training and even to some extent, code quality . So, clearly, IT executives need another kind of help to assess whether JBoss -- or other maturing Open Source technologies -- is ready for heavy-duty enterprise use.
    I. Intro -- An Eval Methodology for JBoss
    The Open Source Maturity Model (OSMM) provides technical and procurement executives a methodology to assess all the important technical and non-technical elements of an Open Source product. The OSMM is designed to be a lightweight process that can evaluate an Open Source product's maturity in two weeks or less. OSMM appears in Bernard Golden's Succeeding with Open Source (Addison-Wesley, 2004).

    OSMM answers the question "Is this Open Source product mature enough (and full featured enough) for my enterprise?"

    Products, especially Open Source offerings, typically develop through identifiable stages as they move from initial release to well-established production use. These stages map the product's "maturity" and in turn, can also be used to help organizations assess the appropriateness of an Open Source option.

    The OSMM is a three-phase process for selecting, assessing and implementing Open Source products. (Please see the accompanying figure to illustrate the process.)

    The challenge that faces organizations considering Open Source software is mainly that Open Source can differ significantly from commercial software. Processes used with commercial offerings can be inapplicable to Open Source.

    In commercial software, selection often focuses on defining the right relationship with the chosen vendor. Aspects such as contract negotiations, price discounts, service level agreements (SLAs), maintenance and support commitments enter the picture, and can even take center stage.

    Too often, this is not so with Open Source. Because most products are obtained at no cost, the battle of contracts and discounts is bypassed. But in averting that battle, a prospective user can find himself on a lengthy quest for the range of technical support he has come to expect from a commercial vendor.

    The challenge for prospective Open Source users is locating and assessing the individual elements that make up a complete product -- one that meets all the needs of the organization. Using key concepts of software maturity OSMM assesses the following elements:
    • Software
    • Support
    • Documentation
    • Training
    • Product integration
    • Professional Services
    The output of an OSMM assessment is a numeric score between 0 and 100 that may be compared against recommended levels for different purposes, which vary according to whether an organization is an early adopter or a pragmatic user of IT. The following table lists recommended minimum OSMM scores.

    Purpose of UseEarly AdopterPragmatist
    Table 1. Recommended Minimum OSMM Scores

    Phase 1: Assessing Product Elements
    In Phase 1 of the OSMM, an organization evaluates each product element with a four-step process: define requirements, locate resources, assess element maturity, and assign element score. Based upon the organization's particular requirements, the available resources are assessed for their maturity, and a score between 1 and 10 assigned. The output of Phase 1 is a set of scores for each of the key product elements.

    Phase 2: Apply Product Element Weightings
    Of course, not every product element is of equal importance. Software is fundamental; support is critical; documentation, though necessary, is less important than the previous two elements.

    In Phase 2 of the OSMM, weightings are applied to the individual element scores to reflect their overall importance for the product maturity. Default weightings are provided, but each organization is free to adjust the default weightings to reflect its particular needs. Here are the default weightings:
    Open Source ComponentScore
    Professional Services1
    Table 2: Default OSMM Element Weightings

    Phase 3: Calculate Overall OSMM Score
    This overall "maturity score" can be compared to the recommended minimum scores to determine if the product is suitable for an organization's needs. The final score can also be evaluated to determine if there are problems with the product that the organization needs to mitigate.

    The recommended minimum scores are, of course, just that: recommendations. The organization does not have to follow them rigidly; the recommended scores serve as guidelines to help it determine if an Open Source product will serve its needs.

    II. Assessment: Applying OSMM to JBoss
    So, what's JBoss' score? Let's apply scoring for each of the six (6) OSMM elements to JBoss.

    The OSMM assesses the software component of a product in four dimensions: (i) functionality, (ii) longevity (length of time the product has been in release), (iii) quality and (iv) development team.
  • Functionality: As previously noted, JBoss has recently passed Sun Microsystems Inc.'s Compatibility Test Suite for J2EE, indicating that it provides a full set of J2EE functionality. While it is important for any organization to understand its particular functionality requirements for a J2EE server, passing this test provides peace of mind that no basic functionality is missing from JBoss. JBoss's one shortcoming in product functionality compared to its commercial brethren relates to their tightly coupled IDEs. These increase programmer productivity for engineers working on the products, and therefore JBoss suffers in comparison by not offering similar functionality.

  • Longevity: The JBoss product has been available for over three years -- not as long as some commercial products, but a respectable length of time. Since JBoss has had so many downloads (over 5 million), it might be said that "it's older than it looks" due to the intense workout it has received from its very large user community.

  • Quality: JBoss has an impressive number of tests (over 1700) as well as having passed the already-mentioned Compatibility Test Suite (comprised of over 23,000 tests). Furthermore, bugs that are reported against the product are rapidly addressed, and fixes are delivered as patches. JBoss averages nine patches per month, a sign of good attention to quality.

  • Development Team: JBoss has a very large development team -- over 91 members. The number of active engineers is about a dozen, many of whom work for the JBoss company itself. This is a good-sized team, which indicates a strong and active community.

    SOFTWARE SCORE: 8 points (out of 10 points possible)

    [***JBoss loses 2 points due to the lack of a tightly coupled IDE and the fact that it has relatively shorter longevity compared to commercial products.]

    There are three options for Open Source technical support: (i) community, (ii) commercial (paid) and (iii) internal (where an organization provides its own support mechanism for its users; e.g., an internal help desk). Support is one of JBoss's real strengths.

  • Community: JBoss has an active community that posts and answers many questions on the support forums. In fact, many answers are provided by JBoss employees, which is a real benefit to product users.

  • Commercial: JBoss also offers paid support in two varieties: development and production. Development support is aimed at software engineers writing JBoss-based applications, while production support is offered to operations staffs responsible for keeping JBoss-based applications up and running.

  • Internal: The caliber of its online support and technical documentation and training options may make internal support a viable option for some companies, depending on their staff's technical skills.

  • SUPPORT SCORE: 8 points(out of 10 points possible)***

    [***There are two drawbacks to JBoss's support: Some questions in the user forums go unanswered (raisng doubts about the forums' reliability), and while JBoss has recently offered onsite support, it is early to judge its effectiveness.]

    Documentation for Open Source software offerings can come in many forms:
    1. Informal Web postings written by users, and even
    2. Reference and tutorials published by commercial publishing houses.
    As one might expect, this can mean that documentation quality can vary from offering to offering. This quality question-mark is often a reflection of whether the Open Source offering has a commercial interest supporting it (either the software creators or an adjunct publisher).

    From the start, JBoss gave documentation a high priority, which makes it unusual among Open Source creators. Early on, JBoss managers assigned a dedicated project contributor to write documentation, offering the material for sale at about $30 -- even while the software download remained free.

    JBoss documentation is widely praised by users as unusually good. In addition, two independent publishers now offer their own versions of JBoss documentation: One version (written by a JBoss project committer ) is called "too advanced" for some J2EE programmers, while the other is considered a stale rehash of the project documentation. (Despite these detractors, there are two additional JBoss books in the pipeline, which should be much better.)

    DOCUMENTATION SCORE: 6 points (out of 10 possible points)***

    [***Overall, the JBoss documentation is a mixed bag: excellent project documentation, good Web postings (although diligence is required to find them amid the other, less useful postings), and not-very-good commercial products (with improvement in the near future)].

    Training is one of the areas in which Open Source shines, in that it offers many options, including tutorials developed by users and made available to the entire product community.

    There are five broad types of training available for Open Source products, which range from self-guided to formal classes. The list includes:
    1. Informal Web-based how-to examples posted by product users,
    2. Developer-created online tutorials,
    3. Commercially published tutorials,
    4. Classroom training developed and delivered by the Open Source development team, and
    5. Classroom training developed and delivered by commercial entities).
    JBoss offers all five.

    In addition to the many JBoss user who regularly post to mailists and community bulletin boards, JBoss also offers developer-created tutorials envisioned and built by the community itself. JBoss also offers its own internally developed product training, which is highly praised by attendees. Several commercial training firms also offer JBoss training.

    TRAINING SCORE: 8 points(out of a 10 possible points)***.

    [***The only real shortcoming in JBoss training options is the lack of commercially published tutorials, which, as noted in the documentation section above, should be addressed in the near future.]

    One of the key questions when considering an Open Source product is: "Are all the necessary integrations available or easily constructed?"

    Many Open Source projects have foundered when, late in the deployment cycle, a critical integration with another product is not available.

    The state of an Open Source offering's integrations with other products -- especially commercial products is very instructive in assessing its maturity. Simply put, the better known and more widely used a product, the more likely other products will have integrated with it. Consequently, if a product under consideration has all necessary product integrations available, that can be considered a strong sign of product maturity.

    In general, J2EE application servers provide many integration options, as pulling and pushing data to already existing applications is crucial for new J2EE-based applications. J2EE conformance requires the presence of two widely used integration mechanisms: Web services and JMS.

    Beyond the "vanilla" J2EE integration mechanisms, JBoss offers integration options of its own. A number of organizations have made their integration efforts available to the community. Furthermore, JBoss is bundled with many applications and is therefore certified "out of the box" for integration with those apps. Finally, many applications have been certified by their providers as JBoss-compliant.

    Of all the product elements that are assessed as part of the OSMM, integrations are the most organization-specific. Every company's software stack is unique, and therefore the integration requirements for a new product will vary for each installation.

    Overall, JBoss excels at making sure its offering is fully compliant with J2EE, meaning virtually all commercial J2EE apps will work with JBoss. JBoss also supports practically all traditional "vanilla" J2EE integration mechanisms supported by commercial J2EE providers.

    INTEGRATIONS Score: 10 points(of a possible 10 points).

    Availability of professional services is one of the trickiest areas to assess for Open Source offerings. The caliber of these offerings, and their usefulness, can vary depending of how they are provided -- through e-mail, phone or hands-on/onsite services and support.

    So, assessing professional services is a key step in an overall OSMM assessment for a product. A moment's reflection will demonstrate why:

    Professional services organizations begin to actively support a product only when they feel there is significant demand, or, in other words, when there is a large user base that is likely to require help with the product.

    In the Open Source world, a key proxy for maturity of an offering is the size of the Open Source community that uses it (and contributes to it). Therefore, to gauge the maturity of an Open Source offering, it is instructive to assess the services available for that Open Source product, even if there is no intention of using them.

    Overall, the range of services from Open Source versus commercial vendors can be spotty. As an example, large commercial J2EE vendors such as BEA and IBM have the power to develop relationships with many types of service providers, consulting firms or system integrators. One can look in vain for such a large list of "services partners" for an Open Source project.

    Nonetheless, the key question to ask about regarding services for an Open Source project is not "How many?" It's simply: "Does the Open Source offering have a services component at all?" If you find an Open Source product with services available, you can be assured that the product is well established and quite mature.

    JBoss illustrates this very well, offering its own range of e-mail, phone and select on-site services. The company has a number of "approved" professional services partners. And most notably, JBoss is developing key partnerships with high-visibility enterprise firms, such as Novell, Hewlett-Packard and Computer Associates, which in the near future will likely form the basis of an even deeper professional services offering.

    PROFESSIONAL SERVICES Score: 6 points(out of 10 possible points)***

    [***As JBoss deepens its relations with enterprise hardware and software firms (such as HP, CA and Novell, among others), we would expect JBoss points in this category to grow.]

    The Overall OSMM Assessment Process
    As described in the section outlining the OSMM, the output of Phase 1 of the OSMM is a set of score for each product element. Weightings are applied in Phase 2, with an overall product maturity score the output of Phase 3. Table 2 illustrates JBoss's element scores, weightings and final maturity calculation.

    ElementPotential ScoreActual ScoreWeighted FactorElement Weighted Score
    Tactical Support108216
    Professional Services10616
    The overall OSMM evaluation awards JBoss 78 total OSMM points. Table 3. JBoss OSMM Scoring by Element

    With 78 OSMM points, JBoss qualifies it to be used broadly in commercial enterprise settings.

    The JBoss breadth of maturity across all lines would make JBoss appropriate for the whole gamut of deployments: experimentation, pilot and production. Further, the maturity of JBoss's internal systems and external partner network makes it appropriate for the whole range of users -- from "cutting edge" early adopters all the way to extremely pragmatic IT organizations.

    Naturally, to see where your particular company may feel comfortable with the OSMM assessment of JBoss, you must give more weight to your company's particular requirements. (Do you need very in-depth documentation because of the immaturity of your core J2EE base? Are your J2EE integration needs very specialized or intricate?, etc.)

    Final Thought: Documenting Your Needs
    Documenting your internal requirements is truly the vital first step in conducting a successful OSMM assessment. Further, the OSMM process will not negate the need to actively use and test the product in a quasi-production environment before moving to actual use; however, having worked through the OSMM for a product ensures that it is a good candidate for pilot assessment.

    Finally, of course, the assessment process may identify specific areas that must be addressed before final implementation.

    For example, the product's training options may be somewhat weak; therefore, an organization may need to devote some time and attention to developing or obtaining additional training materials. In this way, the OSMM moves beyond mere diagnosis and into prescription, helping ensure a healthy outcome with an Open Source product.

    About the Author
    Bernard Golden is CEO of Navica, a California-based IT consulting firm specializing in Open Source strategy, implementation, and training services for the enterprise. Golden is also author of "Succeeding with Open Source" (Addison-Wesley, August 2004), which offers enterprise IT execs valuable and practical advice of using Open Source to gain strategic advantage.