Testing in an Organic World

Computerworld - Don't let the innocuous name Web services fool you. It is actually the first step into a new dimension. Just as the Internet opened computers up to a global, organic means of connecting, Web services are really about connecting applications in the same way. By enabling what will be, in effect, a standard software interface, the applications of today will become the components of tomorrow. In the past, enterprise applications integrated functions within individual companies and managed relationships with their customers, but in the future, Web services will provide worldwide application integration and manage relationships across the planet.

The potential is so profound, it gives me chills, but it also scares me silly.

The very nature and structure of software could change. Today, we build monolithic masses of proprietary functionality with customized interfaces. Tomorrow, we will be able to buy granular components of open functionality using a standard interface. Computing on demand will expand to functionality on demand.

But what frightens me are the testing implications. The most basic premise of testing is that conditions are known and controlled so that the outcome can be evaluated. For example, you can't test the efficacy of a weight-reduction program unless you know how much the person weighs at the beginning and end and what he is eating in between. When applied to software, this means you need a test environment where the variables are known and under your control: the platform, software, data and processing are orchestrated to verify the functionality you are interested in.

But if your test process has to take into account interactions with other applications in other enterprises, and their functionality can be defined on the fly and might involve others -- or not -- along the way, then you've just crossed into the Twilight Zone. How can you possibly test this?

Well, the old, conventional way or a new, organic way.

	Linda Hayes is the CTO of WorkSoft Inc., a Dallas-based developer of next-generation test automation solutions. She is the founder of three software companies and holds degrees in accounting, tax and law. A frequent industry speaker and award-winning author on software quality, she pioneered automated testing tools. You can contact her at Linda@worksoft.com or by visiting www.worksoft.com.

The Conventional Way

The old way would be to write a simulator that generates fake incoming messages and responses to your outbound ones. You even could parse the metadata and automatically generate the messages, which would give you the ability to instantly create large volumes of test cases. Although this sounds cool and makes perfect sense for component level testing, the problem for testing interenterprise Web services is that it's like living in The Matrix: It isn't real.

The moment you circumscribe reality by simulating it, your test loses validity. You are essentially creating a virtual reality where you can predict the flow, type and content of incoming messages or responses from other systems, when the real underlying premise of this brave new world is that it's organic and anything can change at any time. How can you gain confidence that your business processes will work with others when you have only tested them against themselves?

You could try to inject reality by actually coordinating test sessions across companies -- think Y2k or the stock-market decimal conversion -- but these efforts are massive, costly and infrequent. Also, the very need for orchestration defies the true nature of an organic environment. The conundrum is that the very predictability you need for testing is being taken out of the equation.

The Organic Way

A new way would be to take advantage of a golden opportunity to build testability into the essential fabric of this new world order and open a new era of quality and reliability. Imagine: Software could grow up and take responsibility for its own behavior.

Software quality has suffered for years because testing has been approached as an afterthought and has been manually laborious because testability wasn't built in and testing wasn't planned for at the outset. It's been hard enough just to get a marginally acceptable test environment for your own applications let alone access to and control of other applications to test interfaces. Testing practices within many companies are fragmented, inconsistent and fitful, and adding another dimension of complexity may be the proverbial straw on the camel’s back.

But wait. What if we saw this coming and decided to actually design testing in from the beginning? Conceptually, it's not that hard. A company could enable Web test services by providing a parallel universe of services that used something as simple as using the word test to name the service. So, test product pricing would be the test version of the product pricing Web service, but it would be executed either by a test server or by a segregated area of production that handles only test transactions.

The current UDDI registry strategy presently contemplates at least part of this scenario through a private registry for test systems that can interact with the outside world but not vice versa. But why limit external accessibility? Why not open up the test environment to the world in both directions? Why not plan for a test community that mirrors the real one?

This would allow testing to become organic. Testers at each point could experiment with new versions, services and providers or with suppliers just as organically as they plan to in production. Companies could also do slick tricks like mirror the traffic to their production services to the test services registry, allowing real-world traffic to be handled in production but also tested by new or revised services before they go live.

Whatever the detailed implementation, the key is to treat testing as an integral aspect of the future of Web services, so that we not only enable new dimensions of functionality but also new levels of quality and reliability. Wouldn’t that be powerful?

Previous columns by Linda Hayes:

• Give Me a Test Hook — or Else
  Is it ignorance, paranoia or pure laziness that keeps developers from implanting components in their code that are needed for successful automated testing? Join the discussion about this column.
• Smarter Tools, Dumber Developers?
  Can anyone explain how tools can get so much smarter and developers get so much dumber? Or am I missing something? Join the online discussion about this column.
• Deliver Us From Genius
  Sometimes your development projects don't need bleeding-edge innovation and invention. Offshore outsourcing companies are threatening U.S. developers simply by delivering stable, quality goods on time and within budget. Join the online discussion about this column.