The Future of Application Integration

By Tom Yager, InfoWorld

Of the myriad challenges facing IT staffs, few are as complicated as enterprise application integration (EAI). According to InfoWorld's 2002 Application Integration Survey of IT leaders, EAI is still a tough and expensive nut to crack despite advances in technology: 38% of InfoWorld's readers see room for improvement in their EAI efforts.

The marketplace of EAI middleware, tools and services is mature, yet no one vendor or IT consultancy stands out in respondents' minds as the clear front-runner in the field. With no market leader, an IT staff must solve EAI problems by cobbling together software and services from multiple vendors. That's been the way of things since electronic data interchange started its evolution toward EAI, and it has kept EAI from delivering on its promise of turning an enterprise full of disjointed applications into a well-orchestrated, unified solution.

But change is coming.

In January, InfoWorld surveyed 500 IT leaders about their experiences with and plans for EAI as well as how they expect Web services to affect their integration efforts.

The survey indicates that IT executives don't expect any miracles from Sun Microsystems Inc.'s Java 2 Enterprise Edition (J2EE) or Microsoft's .Net, the frameworks touted by their creators as the ultimate solution to integration woes; 56% of respondents don't consider .Net important to their overall EAI strategy; and 45% call J2EE unimportant. Instead, companies are looking past vendor-specific technologies and betting on standards-based Web services. Sun and Microsoft will play important roles, but no vendor can own a standard. Small players and open-source projects have room to make names for themselves in Web services.

This trend is taking hold despite the fact that vital enterprise requirements, including security and data integrity, aren't yet codified in Web services standards. Business leaders seem willing to take it on faith that these and other issues will be resolved, but it will take time for all of the pieces to come together. Until then, IT leaders will have to keep doing EAI the way they've been doing it all along, while simultaneously paving the way for Web services.

The trouble with the old way

Traditional EAI, done with middleware, application adapters and tons of custom code, is a huge business. According to the survey, integration costs consume an average of 24% of yearly IT budgets -- an impressive figure representing millions of dollars for midsize to large companies. But that number is probably less than IT departments are actually spending on EAI because not all integration costs land in the same column on a company's ledger. The acquisition of enterprise software, covering everything from enterprise resource planning and customer relationship management products to large-scale databases, usually incurs consulting fees for installation and integration. If a company tries to save money by handling its own integration, application vendors can still sell training, support and EAI-enabling software such as middleware connectors.

Trimming integration expenses is rough going. Business needs evolve, vendors revise their applications, better tools emerge and IT departments have to adjust to changes in staff and budgets -- it's a continuous effort. Some vendors offer suites of applications, promising that buying everything from a single source will eliminate integration hassles. Others partner with selected middleware providers, consulting firms and vendors of complementary applications. Again, the pitch is that the integration problems have already been worked out.

This looks good on paper, but in reality, the savings realized by suites and partnerships are offset by the loss of choice. IT managers bristle at the notion of abdicating to a vendor the selection of crucial enterprise components. And although the initial cost savings offered by a single-source or partnered solution may appear substantial, suppliers can still build in recurring fees that make the deal less sweet in the long run.

Banking on partnerships can also be risky. If any of the relationships between partnered vendors and firms sour later, companies that bought the package deal are stuck: They must either replace the outcast software or service or risk paying higher fees to continue doing business with the estranged partner.

Integration can be as complicated as it is expensive. Java has become the language of choice for EAI, but having a common language doesn't ease technical challenges. Each application and middleware vendor has its own method for packaging data and exposing network communication interfaces to programmers. IT architects aim for one middleware layer that talks to all of the applications in the enterprise, but at least one application often wants to go its own way. Sometimes the only solution is to tie multiple middleware layers together with custom code. When the architects must resort to this, the result can be the IT equivalent of a house of cards: If any part of the integration infrastructure fails or changes, the whole solution could collapse.

The impetus for change

Regardless of the stock you place in Web services, you can learn a lot about a vendor by studying its response to this emerging technology. In one camp, you have the true believers such as IBM and Microsoft. They have aggressive strategies to refit their software products with Web services interfaces, and they're at the forefront of Web services tools development.

That's understandable for Microsoft, which is one of the very few big software companies that doesn't rely on consulting for revenue. If Microsoft can't get companies to switch to its Web services-enabled enterprise applications, at least it can offer to wire companies' existing software together using products such as BizTalk Server and Visual Studio .Net.

IBM, on the other hand, relies heavily on sales of EAI consulting. It seems incongruous that a vendor so successful at selling services would endorse technology that enables companies to do EAI on their own. But IBM has remade itself many times; it wasn't always a consulting heavyweight. If Web services cuts into IBM's consulting revenue, IBM trusts that it can make up the difference selling such commodities as systems and development tools to an expanded EAI market.

Besides, vendors will always be needed for consulting help on very large or time-sensitive integration projects. The Application Integration Survey results seem to support IBM's approach: 55% of those polled state that Web services will spur them to take on more integration projects.

Despite a late start, Java and J2EE eventually will have pervasive support for Web services, no question. But this won't happen without some resistance. Java is solidly established as a traditional EAI glue technology, and some Java vendors, including trendsetter Sun, are playing down the value of Web services in the overall EAI picture. Entire companies have been built around supplying Java-based EAI software and services. Their business could be dramatically affected if simple, standards-based Web services catch on as a mainstream EAI technology. Even so, Sun can't risk allowing Java licensees such as IBM, Oracle Corp. and BEA Systems Inc. to define the shape of J2EE Web services.

Some middleware vendors are holding out, insisting that Web services can never replace current EAI technologies. These vendors are about to get a wake-up call from the market: 74% of respondents expect Web services to reduce their need for EAI software and services. And 17% expect Web services to eliminate or drastically cut their EAI expenditures. More than half of the InfoWorld readers polled believe that Web services will make integration cheaper, easier and faster. These are impressive numbers for a technology that's barely out of the gate.

Green light

Given the nascent state of Web services, it's reasonable to question the wisdom of derailing or redirecting existing EAI projects in favor of a shift to Web services. A fair number of respondents aren't balking: 39% said they're implementing Web services right now. When asked if they're worried that Web services' missing enterprise features, such as security and guaranteed delivery, would hamper development, 58% expressed confidence that these issues would be resolved.

This optimism reflects the excitement surrounding Web services, but the enthusiasm should be tempered with a dose of reality grounded in sound business sense. By now, most companies have elaborate infrastructures in place that tie their applications together. If an EAI substructure is a rickety mess that falls to pieces at the slightest perturbation, replacing it makes good business sense. But in most shops, existing integration technology is working.

Web services will eventually bring radical change to the EAI landscape, but the revolution hasn't happened yet. The old approach to EAI, with all its limitations and costs, is still valid, and companies seem to have integration down to a routine. Readers report that their integration projects take an average of about eight months to deploy. As big IT projects go, that's not long.

Rather than pull the plug on integration projects already in the pipeline, IT managers should start steering their companies toward Web services. IT leaders should query application vendors and consultants on their Web services strategies and use their answers to determine their place in integration plans. New projects should include preparation for Web services, starting with the identification of the services and data each application should expose. XML should already be used for interapplication data exchange. Defining XML vocabularies and business processes now, both for internal use and for connections to partners, will make the later transition to Web services much easier.

If Web services will eventually wipe out traditional EAI software and tools, it won't happen quickly. For at least the next couple of years, traditional integration technologies will evolve in parallel with Web services.

In the short term, IT managers will be able to blend the old and new approaches to achieve cost benefits and to reduce project complexity. Looking farther ahead, advanced tools will emerge to help IT departments transform cumbersome EAI infrastructures into simpler, more streamlined collections of Web services.

Yager is East Coast technical director at InfoWorld in San Mateo, Calif.

New Tools, New Choices

Stories in this report:

This story, "The Future of Application Integration" was originally published by InfoWorld.

Software Development

It’s time to break the ChatGPT habit