"Asset Harvester" Speeds .NET, Java Reuse

See how a start-up ISV is changing the face of reuse with a spiffy technology for quickly harvesting your legacy Java and .NET assets into reusable components. OSTNet's API-based reuse framework eliminates long-lead times and cost-overruns. Story also includes a look at OSTnet's "2-Minute Quick Start to Reuse" program.

Tags: Asset, Reuse, AMS, Asset Management, Technologies, Coutu, Developers,

A growing number of enterprise architects realize that asset management and software reuse techniques could improve the workings of their IT shops.

But many remain uneasy about vendors' complicated solutions with long lead times, proprietary technologies and high pricetags. OSTnet, a Canadian ISV, has taken a more enterprise-friendly approach for Java, .NET and even legacy assets.

AMS, OSTnet has invested some 40 man-years of engineering time into crafting what their engineers say is a more flexible and open asset management/reuse framework, which is designed from the ground up to easily plug-in to an enterprise' existing in-house technologies and skillsets.

"Our approach basically says, 'It's not the corporate enterprise or the developer teams that have to adapt to [asset management] technology. It's the reuse initiative that has to adapt to the way companies, and their architects and developer teams, work," OSTnet CEO Gino Coutu told Integration Developer News.

In fact, AMS sports a rich set of APIs -- for hooking up to IDEs at the front-end, and a variety of data and asset repositories on the back end. Because of this dual API approach, Coutu said AMS can easily plug into leading development tools/IDEs including Eclipse, IBM's WebSphere Studio and Microsoft's Visual Studio.NET, as well as leading repository offerings including ClearCase, Visual SourceSafe and MKS Source Integrity, among others.

This plug-in architecture approach of AMS stems, Coutu said, from OSTnet's corporate and engineering focus on evolving asset management/reuse technologies to be more accommodative of today's existing enterprise infrastructure, and the IT staff that work there.

3 Keys to Better Reuse

Coutu says that AMS's customer research revealed three (3) keys to adapting current asset management and reuse technologies that will "lead to wider use" of such tools by IT. According to Coutu, asset management and reuse tools should be:

Key 1 -- Adapted to better map the way people already design, build and deploy software;

Key 2 -- Adjusted to be more flexible to work with the code repositories, IDEs and interfaces already in use by architects and developers; and

Key 3 -- Provide better searching and leveraging technologies to help IT staff ramp up their reuse projects, quickly uncovering existing assets that could be easily reused

A Customer-Driven Reuse Approach
At the core of AMS' approach is a triad of three technologies:

  • An open and modular API-based approach to integrating asset management features with in-use dev tools and data repositories;

  • A rich metadata tagging discipline that makes it easier to locate, refactor and reuse assets/artifacts, as well as allow these assets/artifacts to maintain their complex relationships or dependencies; and

  • A set of repository-independent searching and harvesting technologies which automate the time-consuming chore of searching for and refining "candidate artifacts" for possible reuse.

OSTnet has also adopted a very broad definition of "asset," -- ranging well beyond code snippets or components. The assortment includes:

  1. Requirements;
  2. Models;
  3. Patterns;
  4. Web Services;
  5. XML Schema;
  6. Components;
  7. Source Code;
  8. Documents; and even
  9. Knowledge items and
  10. Best Practices.
"If you broaden the scope of what assets can be managed, and also lower the cost and the pain of implementing reuse, your [dev] team can gain big benefits with smaller investments. It's simple math," Coutu said.

This technology-agnostic approach to assets allows OSTnet to work with Java, Microsoft and legacy assets. For Microsoft developers, for instance, Coutu said that OSTnet's work with Microsoft (via the VSIP [Visual Studio Industry Partner] program "provides us with the open APIs and environment needed to integrate seamlessly into market leading tools, and provide traceability as artifacts move between each tool throughout the software lifecycle."

AMS Flexibility -- A Key to Effective Reuse

Coutu said he feels "flexibility" on all fronts -- integration, asset type, ability to work with multiple repositories and IDEs -- is the key to making asset management and software reuse more attractive to IT professionals.

"We now know we can knock on any company's door and ask a customer, "What technology do you use?" and be able to work within their environment," Coutu said. "Because we don't expect them to mention one or two [technologies]; we expect all sorts of technologies in play. That's why AMS has been designed to be open and flexible from the start."

OSTnet's CTO and cofounder Lior Amar put it this way: "The complex, monolithic approach used by many of today's asset management/reuse vendors is just missing the mark for many customers," Amar told IDN. "These solutions are just too heavy, too complex and too proprietary for all but a few huge companies or government agencies to take it on."

"The concept of asset management and software reuse has been around for a long time, since 1967. So, one key question we sought to answer was: 'Why doesn't it [software reuse] happen more often in more companies?' "Amar noted. "The real reason, we think, is that since 1967, we as an industry still haven't been able to do it properly," he said.

What Devs Want From Reuse Tools

"Most developers don't want to recode everything," Coutu said. "It's absolute nonsense when somebody says developers want to reinvent everything. But the trouble is, there's just a lack of simple, effective solutions." Going to school on the hard ramp-up to software reuse, OSTnet engineers set out to automate the search/identify process of locating candidate assets that might be able to be reused.

Customers told him so, Amar added. "In preparing to build AMS, we heard from customers over and over that the traditional proprietary approach [to asset management and reuse] just isn't the way to go."

Why? Well, Amar said, "That style of reuse that says to the customer: 'Stop what you're doing today! Now, let's reimplement a brand-new process and change everything you're doing. It'll be worth it'.

"That [approach] just doesn't work for many companies, mostly because that approach requires large changes, large amounts of time and money, and often it takes 100% buy-in from all departments."

After customer research, OSTnet engineers decided they needed to build a more fleixble, modular approach that would, as OSTnet CEO Coutu describes it, "make a set of asset management and reuse technologies that just work better with how IT pros do their jobs day-to-day."

Early on, Coutu said, "we decided AMS had to be an 'open' platform, with APIs, and lots of integratable features. The whole point behind all this openness is to let the enterprise professionals, whether they be architects, developers or testers, work within the tools and the IDEs they know best.

"So, at the end of the day," Coutu added, "what we are trying to do [with AMS] is to provide a federated [asset] registry, where you can access all assets from a central location, even if they are located in local [existing] repositories throughout your enterprise."

Inside AMS's Open APIs

The key aspect of AMS's architecture is its rich set of open APIs, which enable AMS to work with a variety of installed tools, IDEs and repositories at the customer site.

Because of the front-end APIs, AMS can easily plug into leading development tools/IDEs including Eclipse, IBM's WebSphere Studio and Microsoft's Visual Studio.NET. Similarly, thanks to AMS' back-end APIs, it works with artifacts/assets already in software repositories such as ClearCase, Visual SourceSafe and MKS Source Integrity, among others.

Further, according to Coutu, AMS's use of rich metadata constructs and tagging for handling and identifying artifacts lets architects/developers "retain the relationships between pre-existing assets when you put them into the repository. This lets users 'tie assets together,' so to speak, so you can have traceability down to the artifact level."

AMS also provides a role-based GUI console, which enables all members of an application lifecycle -- requirements/business analysts, architects, developers and even Quality Assurance team members -- to have single-click access to their company's artifacts, including code, test scripts and even web services, regardless of where they might reside.

AMS's Management/Reuse Checklist

AMS's open APIs, rich GUI options and use of metadata combine to provide a rich array single-click access to assets and artifacts. This gives architects, devs and IT managers many key benefits, Coutu said, including capabilities to:

  • Attach descriptive metadata to assets to better expose them to other devs in the organization and maintain "ties" with related assets/web services;

  • Manage the publication of new assets via customizable workflows, which ensures that all "published assets" have been thoroughly reviewed, tested and certified before being made widely available;

  • Track the use of available assets, including where, by whom and on which project(s);

  • Limit consumption of assets by project, or group of trusted projects;

  • Receive "live" notifications when assets are created, modified or deleted;

  • Gather, analyze and report metrics back to the administration on created assets; and

  • Ensure that asset reuse meets compliance with internal metrics or SLAs.

AMS Automates Asset "Harvesting"

"Populating your reuse repository should be a lot easier than it often is today," Amar said. "Nobody wants to sit in front of a million lines of code that they've never looked at before, and then try to extract something reusable."

The best way to start, Amar added, is by getting acquainted with your software, and the folks that built it. "Because you have the author of the asset, in a short 2-minute conversation you can get most of what you need for reuse," Amar said, including

  1. the context for which the asset was built initially, where it was first used, and
  2. how it was updated, if it was.
OSTnet engineers set out to automate the search/identify process of locating candidate assets that might be reusable. The result is an AMS module called the Asset Harvester. Before commercializing the technology, OSTnet turned the Asset Harvester against its own code base, with some surprising results.

AMS's Asset Harvester can review source controlled libraries, read millions of lines of code, Amar added, "and then suggest to me anything that is reusable in there."

In fact, when OSTnet ran the AMS Harvester against its own code base, Amar said "we found the same exceptions 4 times. That made us realize, 'Hey, we don't standardize our exceptions.' Everybody renames exception classes [in Java], and tends to create their own. But we discovered for ourselves, exceptions are reusable with the right search and the right tools."

So, how does the Asset Harvester work? In specific, it employs several incremental steps:

Step 1: AMS actually reviews the source code to see if it's compilable.

Step 2: To prepare the asset for placement in a repository, AMS extracts as many as 30 different software metrics (depending on the language used) to ascertain the traits and reusability of the source code. These metrics form a part of AMS's internally developed AI filters.

Step 3: AMS uses a variety of mathematical algorithms to discern which parts of the source code are reusable, and alert AMS users.

"We do more than just store assets with AMS," Amar said. "We take into account the context of the asset. For instance, you might be parsing procedural code, object-oriented code or even a mixed version. By taking all that into context," Amar added, "we can have the Asset Harvester suggest to the end user, 'Hey, you have something reusable here. Instead of just leaving it behind, why don't you pre-populate your repository with this?'"

Amar added: "This is all done automatically, which is powerful, because now, instead of starting with a blank repository; you can start with 1,000; 2,000; 3,000 assets inside of it."

Other AMS Modular Technologies

AMS modules are "meant to be hosted, even cannibalized by other development tools or IDEs," Amar said, and so include a range of host connectors. Once the AMS-to-host connection is established, "AMS can pretty much do whatever a user needs in that environment," to access or publish assets, he added. These functions include:

  • Highlighting code, right clicking and publishing the asset;

  • Selecting a file from a dev tool/IDE by right-clicking on it and publishing it;

  • Selecting a full a project and publishing it;

  • Consuming an asset to the code page or directly into a class.
Other AMS modules include:

  • Asset Explorer --Allows users to navigate or search through your environment and locate assets that are reusable. The Asset Explorer allows you to navigate assets belonging to ongoing or past projects and set them using their own taxonomy. The taxonomy that is built to store assets is absolutely different for every company.

  • Adaptive Search -- A high-powered neural-based search technology that enables IT professionals to quickly and easily locate relevant software assets with increased speed and accuracy, and have a compiled list of all related or "tied-in" asset classes, including related components, test scripts and models.

  • Management Tools/Utilities - AMS's configurable metrics, which can be integrated and localized with departmental IT, provide real-time feedback on the status of ongoing development efforts.

  • Administration Security Tools -- Manage permissions to groups, project/team members, management roles and even specific users. The tools work beyond internal firewalls, to extend permissions management to remote/Internet users. (Admins are provided tools to also extend Windows NT security and grant permissions to NT users/groups.)
AMS's 2-Minute Start to Reuse

With AMS, OSTnet engineers have taken a hard look at how today's IT shops work (from the designers to the developers and testers and so on). Based on this research, they've invented an open, modular approach to asset management and reuse, that they hope will overcome many of the objections to such approaches.

"AMS is a set of technologies that look at your existing assets, and then help you try to jump-start your reuse initiative by locating reusable items," Amar told IDN. "These assets can then be easily shared, and information can be provided on when and where you did the sharing" he said.

Because AMS keeps track of every time and place that an asset is being used, once it has been found and needs to be modified, a manager can easily extract from AMS a report that notes the asset in question has been reused by the following developers, in the following projects, and then identify the last project it was used on. As well, this report can also identify the project manager associated with these projects, so you can contact these individuals first.

Coutu puts it this way: "It's our job to centralize your asset information, no matter how decentralized your environment." The approach marries technologies (open platform, integration capabilities and smart search) with what might be called Best Practices, toward the goal of determining just which of an enterprise's thousands of software assets can be reused.

"A lot of vendors seem to think that software reuse is all about building assets and putting them into a repository so they can be used again, to cut cost and [deployment] times. We've found that customers really are looking for more," Coutu said, "including advice on better ways to leverage what they've already done, with the hardware and software they've already bought. That's what we've done with our AMS open framework approach, we think, and it should lead to a quicker and less painful return-on-investment.

This commitment to a quicker ROI turnaround time, in fact, seems to be an underlying focus of much of OSTnet's engineering and product design work. "Customers just can't take the long view like they did in the '90s with these kinds of projects," Coutu said.

Reflecting on the "realties" surrounding the presure to reuse costly-to-rebuild software assets, Coutu added, "Nobody these days is looking to spend a million [dollars] today to save three million in 5 years. They need results next quarter, if possible,"