Testing SOA: Peeling the Onion

Computerworld - The good news about service-oriented architectures (SOA) is the flexibility they provide for ad hoc integration within and across enterprises. It's also the bad news when it comes to testing, which relies on predictability and repeatability.

The key is to first isolate the architectural layers so that functionality can be tested -- and issues exposed -- at their source as early as possible, then to gradually integrate the components until end-to-end business processes can be verified.

Equally important is to engage subject matter experts throughout the development life cycle to ensure compliance not only with technical specifications but also with business requirements. Waiting for business process acceptance testing to be performed when all the layers are complete is to invite disaster and delay: a service that returns the wrong result in the right way is perhaps even more dangerous than one which fails to execute altogether because it has a higher risk of being undetected, and uncovering the root source of a problem takes longer the more layers are involved.

The challenge for testing an SOA is that the test design should be approached from the top down, and test execution from the bottom up. This paradoxical approach assures that as each layer is implemented the overall enterprise objective -- supporting critical business processes -- is maintained.

It's All About the Data

As exciting as SOA is, never forget that it's just another way to move data from one place to another, and if the data is wrong, then nothing else matters.

This is why subject matter experts are crucial to the test process for SOA. Their role is to define the data contents of the transactions that support the business processes. For each business process, they must identify the supporting data that is required, and for each service request and reply, they must provide the expected values. This can be done in a simple spreadsheet format or in a test repository database long before the service architecture is actually implemented, and can be incrementally expanded as the services take shape.


For example, if an insurance company receives an electronic claim whose adjudication, payment calculation and remittance is handled through services behind the scenes, then it's essential that the final outcome be correct: the claim is approved or refused appropriately and the amount paid, if any, is accurate. Only an insurance claims specialist can or should define the test cases that will verify the functionality of each application's role in the overall process.

The importance of starting with data analysis is that it maintains focus on what business functions must be exposed as services in order to meet high-level business process requirements. Further, it provides the data contents for the messages that will be used to exercise the infrastructure as it's implemented. Without this data focus, the implementation of an SOA runs the risk of delivering a technically acceptable solution that fails to deliver business value.

The Medium is the Message

Once the business experts have identified the data values, these must be formatted into messages that express service requests and replies. For testing, messages must be created for each request and reply between each provider and consumer application that can be used to test each layer of the architecture.

Message templates can be defined by industry standards or internal specifications. Some industries have already promulgated voluntary standards such as Association for Cooperative Operations Research and Development (ACORD) for example, or regulatory standards such as Health Insurance Portability and Accountability Act (HIPAA) for health care-related transactions.

With a language as verbose as XML, the test data should be maintained separately from the templates and merged only for execution. Default values should be provided for any data elements not critical to the verification of a particular service in order to keep the volumes manageable. This shields subject matter experts from the details of formatting, minimizes the opportunity for formatting errors and permits verification of the templates against standards.


Once the templates and data values are merged into useful messages they can be used to verify every layer of the architecture. Isolating a consumer or provider application can be done by substituting a message stored in a data file for the other side of the transaction. For example, a consumer request can be sent to a file or a reply received from a file instead of the provider.