Unix Insider –
Foreword
Middleware vendors are now jockeying for a piece of the new enterprise integration architecture market, commonly called EAI. This month, we discuss what EAI is, what it means for the business, and how, as technologists, we can leverage this type of technology. Integration isn't easy -- figuring out what to integrate, what technologies need to communicate, and what they're sharing is a time-consuming effort. And blindly loading EAI products won't help at all; EAI requires in-depth knowledge of the environment and the business model in the organization. Only then will the IT professional be able to provide a seamless infrastructure that is able to link disparate technologies, applications, and package solutions together in a single enterprise technology architecture.
--Kara Kapczynski
The information gathered and stored in each of these systems is isolated and not shared with other systems. As time has passed, the need to share information between systems has increased. Industry analysts report that companies spent almost $40 billion on integration problems in 1997, and project that $60 billion will be spent in 2001. Individually, companies are using up to 40 percent of their annual information technology budgets on building and maintaining application integration solutions.
Exchanging information between systems in realtime or near realtime in order to make better, faster decisions is a necessity, not a luxury. Furthermore, corporations are finding out that they need to exchange and/or share information with business partners in a realtime or near-realtime manner. Companies know that they must now move toward managing and automating business processes, such as order processing, rather than individual business functions, such as order entry, shipping, inventory, or billing. This requires companies to seamlessly integrate the disparate applications that implement these business processes. Enterprise application integration (EAI) software is a new kind of software that aids this transformation.
What is EAI?
There are many definitions of EAI depending on whom you ask. For the purposes of this article, we'll say that enterprise application integration is the process of tying together multiple applications to support the flow of information across multiple business units and IT systems.
EAI is all about interoperability and information synchronization across multiple applications -- mainframe, packaged or purchased systems, and custom application systems. EAI enables sharing of information, not just within an enterprise or organization but within a business environment that includes a company, its suppliers, and its customers.
EAI products offer a framework that provides a significant portion of the integration solution right out of the box. An EAI product must provide the following services:
- Messaging
- Connectivity
- Security
Messaging
A messaging backbone is the foundation of an EAI framework. This backbone transports messages between resources, reconciling network and protocol differences. It allows applications to share information with the outside world by sending and receiving messages. It adds quality-of-service options to message delivery, such as security and queuing. The messaging services should ideally support both synchronous and asynchronous messaging. Asynchronous messaging is particularly useful when applications must integrate in broadly distributed environments where the availability of applications and network segments is unpredictable.
Connectivity
Having established a common messaging backbone for interapplication communication, the next step in deploying an EAI framework is to plug the applications into the messaging backbone and allow them to send and receive messages. Because each application has its own interface, communication protocols, and data formats, the connectivity setup must include services for orchestrating the flow of data between integrated resources. It has to be able to resolve data coding differences, translate to and from the common protocol used by the messaging service, and transform data content to and from a standard format.
Security
Typically, each application provides security through the user ID and password entered by the user. Integration technology must provide similar services for the entire business process, including user authentication, resource access control, and information encryption. This technology must also integrate with existing security implementations.
Traditional solutions
Traditional application integration has been done using point-to-point programming with or without a messaging infrastructure and database-to-database integration. Point-to-point programming integrates applications in a pair-wise fashion through low-level communications programming. With this approach, the process and messaging infrastructures are highly fragmented. Messaging infrastructure facilitates communication between applications and shields developers from low-level system and networking details. Database-to-database integration is simple when you have detailed knowledge of both databases. (The June 1998 IT Architect column, "Application integration without band-aids," offers detailed information about these two traditional solutions.)
Standalone stovepipe applications increasingly must be connected via automated processes. In the beginning, each application that must talk to another application will build its own conduit (see Figure 1). Application A will build a conduit to get information Bb from application B. Application C will also build its own conduit to get information Bb from application B.
The resulting architecture ends up with point-to-point solutions that get complicated when the number of applications and/or systems involved increases. As you can see, if information Bb in application B changed, all the applications that built a conduit to access information Bb would have to change. Modifying programs takes time and demands scarce and costly programmer resources. This type of solution requires writing code on the source system as well as the target system. This makes sharing information with business partners and coordinating changes over time difficult, at best. Maintenance is a big issue and is very expensive and time-consuming. Most types of point-to-point application integration use message-oriented middleware (MOM) to facilitate the communication between systems.
Figure 1. Point-to-point integration
Another type of application integration that has been traditionally used is database-to-database integration. Database-to-database integration means that we simply share information between two applications at the database level. You can use the replication features built into many databases to move information to the target database as long as they both use the same logical schema. Or you can use database replication and transformation software to move information between different schemata on different types of databases.
Simplicity is the main advantage of using database-to-database integration. The downside is that the logic of manipulating the data resides in the database. When dealing with packaged applications such as SAP R/3 or PeopleSoft, you'll have to deal with the application programming interface (API) provided by the vendor. This requires additional expertise and makes the integration work more expensive.
Message broker: A better alternative?
The most flexible type of application integration utilizes the message broker. Built on existing message-oriented middleware, it enables you to integrate systems at either the application level or the database level.
A typical scenario involves "events." Once an event is defined in the source system, those systems interested in that event can "subscribe" to it. A message broker takes care of delivering event messages to subscribers. For example, an order-entry system publishes an order event, and a shipping system, a billing system, and an inventory system subscribe to the event. When a customer orders a book, an order event is published, and the shipping, billing, and inventory subscriber systems receive the event message and initiate whatever processes are required to update their respective systems.
This type of middleware can split, reformat, and combine messages based on predefined rules, forwarding the reformatted messages to appropriate target systems. It utilizes rule engines, transformation engines, and intelligent routing mechanisms. When using message broker software, you typically don't need to change the source or target system.
Most message brokers offer "adapters" or "connectors" to link to common packaged applications such as SAP, PeopleSoft, Baan, Vantive, and Clarify, as well as most relational databases and legacy systems like CICS. Most of them also offer software development kits (SDKs) to build custom adapters, should you need to develop them.
As a developer, when you use MOM for traditional program-to-program communication, you need to take care to transform messages from their incoming format to a different format so that the participating application systems can communicate in a form that each understands. If workflow is involved in the application, you must program that part yourself. Message broker software provides message transformation services and a workflow control engine to reduce the developer's programming load.
Figure 2. Message broker integration
Evaluating EAI packages
Here are a number of things to consider when choosing your EAI package:
- Administration tools
- Development tools
- Scalability and redundancy
- Application integration tools
- Extensibility
- Time-to-implement
Administration tools
An EAI package should provide good administration tools to monitor the messaging backbone and components and to deploy the final solutions.
Development tools
These tools are important to the developers who put the pieces together. The more comprehensive and integrated the tools are with the other parts of the package, the less time it will take to use the package.
Scalability and redundancy
The fundamental infrastructure should be designed to scale up in order to support current message volume and future growth. The package should also provide redundancy to support fault-tolerant configurations in order to be used as part of the mission-critical application solution.
Application integration tools
What are the systems you need to integrate currently and in the future? Pick an EAI solution that supplies an adapter to the packaged applications you intend to integrate and one that makes it easy to build your own adapters for custom-built application systems.
Extensibility
A good integration solution must be customizable and extensible. A company should be able to add to and change business processes without affecting the underlying applications, and IT should be able to change applications without affecting business processes.
The application systems of an organization may support common environment and standards, but every organization will have some applications that are just a little bit different. In addition, common environments and standards are constantly evolving. Integration technology must be adaptable to those changes. For example, although integration technology may support the key packages and languages, it should be extensible to support, through custom development, all the other packages and languages that an installation may use to implement its business processes.
Time-to-implementation
One critical factor in application integration is the time it takes to complete the work. The faster application integration can be accomplished, the faster a business can begin to enjoy its benefits. One of the key selling points of EAI products is that they are able to produce results faster than custom solutions. As a result, they must deliver on this point, even in the first project, in order to maintain credibility with business and IT management.
Implementing EAI solutions
As you start implementing a message broker-based EAI solution, there are a number of things to keep in mind:
- How mature is the message broker software package? The less mature the package, the more prototyping and testing it will require.
- Does the package have the proper infrastructure to support large-scale, mission-critical applications? If not, you'll have to provide some of the infrastructure services through custom programming.
- If you need to integrate into enterprise resource planning (ERP) software packages, call-center packages, or sales-force-automation packages, do you have the expertise required?
- Do you need to integrate with custom-built applications?
- Can you establish an enterprisewide view of data?