SOA in 2011: Report Sees Surge in Complex Data Driving SOA, Integration Projects 
Enterprises are looking for more agile, cost-effective and real-time integration across multiple data types, says a study by Informatica Corp. and Aberdeen Group. Key drivers of these SOA and integration projects include: a jump in complex and unstructured data; Big Data and BI projects and higher perceived business value of data from partners and other third parties.
Enterprises are looking for more agile, cost-effective and real-time integration across multiple data types, says a study by Informatica Corp. and Aberdeen Group. Key drivers of these SOA and integration projects include: a jump in complex and unstructured data; Big Data and BI projects and higher perceived business value of data from partners and other third parties.
“Complex data, including XML unstructured data, and semi-structured data is increasingly used and growing in importance to organizations,” said David White, Senior Analyst, Aberdeen Group. “Aberdeen research found that organizations are making significant investments to harness more complex data from internal and external sources.”
Learn more about the Informatica/Aberdeen report Future
Integration Needs: Embracing Complex Data
Notably, the report also found with this push to support, integrate and leverage complex data types, SOA architects are playing much bigger roles in such data projects. Karen Hsu, Informatica’s director of B2B product marketing, shared some perspective on this trend.
"SOA architects are wielding a lot more overall influence complex data [integration] projects."
Karen Hsu
Director, B2B Product Marketing
Informatica
“While the need to support XML and complex data types is a driving force for many data integration projects, we also found SOA architects are wielding a lot more overall influence for these data projects,” Hsu told IDN. “One major reason is that data integration does not only require support for various data formats, but IT needs to make sure data is transported securely, and complies with standards and policies, when necessary.”
The Informatica/Aberdeen report’s findings underscore this convergence between SOA architects and data integration professionals.
Among the results:
Effective Complex Data Integration Drives ROI – Firms that effectively integrate complex data can achieve several key benefits, including: Use up to 50 larger data sets for BI and analytics; Integrate external unstructured data into business processes more effectively; Deliver timely and critical information via automated data refresh; Slash data errors by nearly 50% (compared to manual updates and refreshes); and Cut integration software/consulting expenses by nearly 50%.
Rapidly Increasing Data Volumes – In 2010, organizations experienced a staggering average data volume growth of 40%. These huge increases underscore the reality of Big Data concerns, Hsu said.
XML Adoption Surges – Today, XML is the most common semi-structured data source organizations integrate. Two-thirds (66%) reported they are integrating XML from internal sources. Even more eye-popping, almost three-fourths (74%) are integrating XML from external sources.
Complex, Unstructured Data Expected To Soar – In the next 12 months enterprises plan to introduce many complex unstructured data sources – office productivity documents, email, web content and social media data. These new complex types will surpass conventional data types.
External Data Sources Proliferating – On average, organizations are integrating 14 external data sources, up from 11 a year ago. Such sources include third-party partners and online data services, Hsu said.
Up Close: How Businesses Capture More
Value, Business Insight from Data Integration
Organizations are seeing huge business value and insights from combining their own data with external data sources, including complex, unstructured and semi-structured data, such as industry-specific standard formats HL7 and ACCORD, Hsu said.
“Different data sources often come in using different formats or are in unstructured form. So, putting this data all together requires a lot of integration work,” she told IDN. “In turn, this is driving an increased interest in data integration between silos and especially with third parties.”
This business insight from such complex data integration provides businesses a way to identify long-terms trends, but also supports day-to-day operations, she added.
Hsu offered this example: “Banks want to understand when a customer pays on time because they are evaluating that customer for other business, such as a new loan,” she said. This level of insight, not easily gathered from one piece of data, provides a bank near real-time information that helps banks make business decisions based on a better understanding of their customers, she added.
Complex data integration also provides valuable visibility to compliance and regulatory officers. “We also found complex data integration helps meet regulatory rules more consistently and generate reports more quickly,” Hsu told IDN. “It used to be OK to take a month or even a few months for such [compliance] reports, but now that window is definitely shrinking, and will get smaller with new rules such as Dodd-Frank [banking regulations].”
To cope with a growth in data volumes and complexity, Aberdeen recommends companies should:
- Automate the data transformation of complex data (including unstructured data, semi-structured data and XML).
- Focus on data quality to ensure data consistency and accuracy of data exchanges from the point of entry.
- Consolidate data transformation and data exchange capabilities in a single platform (to support real-time integration, Big Data and data warehousing). </LI>
The Informatica/Aberdeen report is based on data collected in June, 2011 from 112 organizations actively engaged in data integration.
