DataDirect: Houston, We Have XQuery 1.0

After 8 years of at-times-intense vendor discussions, the W3C has finally adopted XQuery 1.0 (and seven other XML standards). The standards looks to help enterprise IT unify diverse databases, documents, emails and web pages into a single, searchable datastore. But how will XQuery 1.0 change things, and how should IT pro's treat the news? IDN speaks with W3C XQuery Working Group member DataDirect Technologies' XML Architect Carlo Innocenti.

Tags: XQuery, XML, Database, Standards, Vendors, Implementations, Query,

After more than 8 years of discussions between web, XML and database vendors, the W3C has published XQuery 1.0 and 7 other XML standards to help enterprise IT integrate their databases, documents and even emails and web pages into unified targets for metasearches.

XQuery 1.0 reflects supports more than 40 implementations of XQuery, and a test suite of more than 14,000 test cases, and more than 1,000 developer/architect comments. W3C Member organizations have also announced implementations of XQuery or plans for implementations.

Upon release of the W3C standards, no less an expert than Don Chamberlin of IBM Almaden Research Center, the co-inventor of SQL, said in a statement: "I expect XQuery to play a central role in unifying information from many different sources." Chamberlin also was one of the co-editors of XQuery 1.0.

IDN spoke with an XQuery expert at longtime XML vendor DataDirect Technologies to learn more about how far XQuery 1.0 pushes the needle, and for suggestions on how enterprise developers and architects should react to the new XML Family standards.

Integration Developer News
Interview with
Carlo Innocenti, Architect, XML Technologies,
DataDirect Technologies

IDN: Many experts have celebrated XQuery 1.0 as the start of a push to put XML on a par with SQL for data. Is this practical, or whimsical, in today's IT world? What do you see as the keys to XQuery 1.0 (and related W3C standards)?

Innocenti: XQuery 1.0 is surely a milestone in the relatively short history of XML; XQuery as an XML query language is the best mix we have seen so far: it's flexible and concise, easy to learn and use but powerful at the same time, easy to optimize and to abstract to work against data sources that are not only XML.

XQuery will help bridge the dichotomy we have often seen in the XML world between "data" and "document" people: XQuery looks familiar to people thinking about XML as data because they can easily relate to it through their SQL knowledge and experience; but at the same time XQuery is also appealing to "document" people because it allows them to query unstructured data and to transform such data into different representations suitable for presentation, reporting or other.

While the number of people storing native XML data in their "traditional" databases will keep growing in time, I don't expect to see a huge increase any time soon. Even if database companies tend to limit the value of XQuery to its ability in dealing with data that is stored as XML in their own systems, XQuery is much more powerful than that: DataDirect XQuery allows you to query both traditional, non-XML data, and native XML data stored in relational databases (or elsewhere) through XQuery; and it does that leveraging the performance and scalability benefits of executing queries against a database.

For that reason, I wouldn't try to limit the success/future of XQuery to how much XML data is natively stored in a database; XQuery is more than that; it helps you querying relational or XML data, XML or non-XML documents, legacy and flat documents, and more. The keys to its future success will be the availability of implementations that leverage its flexibility, its inherent possibility to be abstracted from the underlying physical data representation, and the possibility to be easily optimized.

IDN: Does Data Direct have recommendations for what steps interested customers should take in exploring their next XQuery and XML data options?

Innocenti: Developers and architects curious about how XQuery can help in their solutions should ask themselves these questions:
  • Do I need to deal with XML in any part of my solution? Is some of the data sources or documents I'm dealing with XML? Or do I need to generate XML as the result of some data manipulation?
  • Do I need to deal with a set of heterogeneous data sources? Do I need to integrate documents with data stored in traditional tables in relational databases? Or maybe I need to deal with both traditional "rectangular" and XML data in my database?
  • Do I need to transform a variety of documents (some of them XML, some of them legacy data) into some form suitable for presentation (like XHTML, or PDF)?

  • If any of the answers to those questions is "yes", then XQuery is something that should be investigated very seriously. is a great starting point.

    IDN: What took so long? It seems like we've been hearing about XQuery for at least 5 years. Does this XQuery 1.0 standard (as it is written now) finally set the stage for the promises we heard about the power of XML data from enterprise vendors, or do we still need further refinements?

    Innocenti: XQuery 1.0 is the result of the work and compromises of many companies and individuals, most of them thinking about XQuery and XML from different angles.

    Relational database companies have been thinking about XQuery as a way to let their users more easily query XML types stored in their traditional databases; other companies have been more involved in the XML specific features of XQuery; others, like DataDirect, have been always focused on making XQuery a easy to use, powerful, flexible and easy to optimize language.

    You can understand how such large group of different interests and people has made the process of making XQuery a standard quite slow. It's also true that XQuery has been in "final call", open to suggestions and comments, for three years; which is quite a long time; but the amount feedback received from the community has been huge; processing that feedback is a time consuming task.

    Even if XQuery 1.0 is a big step in the right direction of providing the right tools to people who need to query XML and a variety of other data sources, I believe XQuery 1.0 is a starting point, not the final one: while XQuery 1.0 was being declared a Recommended W3C standard, most of the same companies and individuals involved in the initial XQuery Working Group were already thinking about the next steps.

    What happens now?
    IDN: What are the next steps from vendors, toolmakers and others for ramping up XQuery adoption?

    Innocenti: Vendors and toolmakers have been supporting and building products based on XQuery for quite a few years now. As I mentioned, XQuery has been in "final call" for a few years; during those years we have seen several XQuery implementations by vendors like DataDirect, Microsoft, Oracle, IBM and BEA becoming quickly available, much earlier than XQuery became a formal standard.

    Even if there isn't a formal "compliancy test" for XQuery 1.0, W3C has released an XQuery Test Suite, which is probably the closest approximation to a compliancy test that W3C could have created. It is impressive to see how may XQuery implementations already exist out there, and the high percentage of passed tests in average (

    In the next few months, we should see the XQuery for Java API (XQJ, JSR 225) becoming a standard too; that will make it easier for vendors to provide a unified Java API for XQuery in a way very similar to what JDBC provides for SQL.

    Impact from "other" XML Family standards
    IDN: How significant are the related XML Transformation and X Path upgrades to have more companies use native XML to store their data, or work better with distributed data?

    Innocenti: XSLT 2.0 is the "natural" evolution of XSLT 1.0; and it makes XSLT more powerful and simpler to use in a variety of typical Use Cases. Both XSLT 2.0 and XQuery 1.0 rely on XPath 2.0, which is the expression language shared by both languages. I would expect that the introduction of XSLT 2.0 will make existing XSLT users happy; but I would be surprised to see a surge in the number of XSLT users simply because XSLT 2.0 is now available.

    On the other hand, XQuery 1.0 will intrigue a number of architects and developers mainly because they will be able to learn and develop more quickly. And, thanks to some highly sophisticated implementation like DataDirect XQuery, they will be able to use the same language to query XML, relational and legacy data.

    Being able to rely on a single language that allows you to query any data source that exposes XML data, or data sources that can be "virtually" exposed as XML, being able to do that using implementations that are highly performant and scalable, and being able to deploy such implementations in virtually any architecture, all that makes XQuery extremely appealing to most IT organizations.