Testing in an Organic World

Implementing KM: Practitioners Share Best Practices Enhancing Online Sales and Support: New Research Reveals What Works - and What Doesn't Sold on SOA Sign up to receive Security Resource Alerts

May 19, 2003 (Computerworld) -- Don't let the innocuous name Web services fool you. It is actually the first step into a new dimension. Just as the Internet opened computers up to a global, organic means of connecting, Web services are really about connecting applications in the same way. By enabling what will be, in effect, a standard software interface, the applications of today will become the components of tomorrow. In the past, enterprise applications integrated functions within individual companies and managed relationships with their customers, but in the future, Web services will provide worldwide application integration and manage relationships across the planet.

The potential is so profound, it gives me chills, but it also scares me silly.

The very nature and structure of software could change. Today, we build monolithic masses of proprietary functionality with customized interfaces. Tomorrow, we will be able to buy granular components of open functionality using a standard interface. Computing on demand will expand to functionality on demand.

But what frightens me are the testing implications. The most basic premise of testing is that conditions are known and controlled so that the outcome can be evaluated. For example, you can't test the efficacy of a weight-reduction program unless you know how much the person weighs at the beginning and end and what he is eating in between. When applied to software, this means you need a test environment where the variables are known and under your control: the platform, software, data and processing are orchestrated to verify the functionality you are interested in.

But if your test process has to take into account interactions with other applications in other enterprises, and their functionality can be defined on the fly and might involve others -- or not -- along the way, then you've just crossed into the Twilight Zone. How can you possibly test this?

Well, the old, conventional way or a new, organic way.

	Linda Hayes is the CTO of WorkSoft Inc., a Dallas-based developer of next-generation test automation solutions. She is the founder of three software companies and holds degrees in accounting, tax and law. A frequent industry speaker and award-winning author on software quality, she pioneered automated testing tools. You can contact her at Linda@worksoft.com or by visiting www.worksoft.com.

The Conventional Way

The old way would be to write a simulator that generates fake incoming messages and responses to your outbound ones. You even could parse the metadata and automatically generate the messages, which would give you the ability to instantly create large volumes of test cases. Although this sounds cool and makes perfect sense for component level testing, the problem for testing interenterprise Web services is that it's like living in The Matrix: It isn't real.

The moment you circumscribe reality by simulating it, your test loses validity. You are essentially creating a virtual reality where you can predict the flow, type and content of incoming messages or responses from other systems, when the real underlying premise of this brave new world is that it's organic and anything can change at any time. How can you gain confidence that your business processes will work with others when you have only tested them against themselves?

You could try to inject reality by actually coordinating test sessions across companies -- think Y2k or the stock-market decimal conversion -- but these efforts are massive, costly and infrequent. Also, the very need for orchestration defies the true nature of an organic environment. The conundrum is that the very predictability you need for testing is being taken out of the equation.

The Organic Way

A new way would be to take advantage of a golden opportunity to build testability into the essential fabric of this new world order and open a new era of quality and reliability. Imagine: Software could grow up and take responsibility for its own behavior.

Software quality has suffered for years because testing has been approached as an afterthought and has been manually laborious because testability wasn't built in and testing wasn't planned for at the outset. It's been hard enough just to get a marginally acceptable test environment for your own applications let alone access to and control of other applications to test interfaces. Testing practices within many companies are fragmented, inconsistent and fitful, and adding another dimension of complexity may be the proverbial straw on the camel’s back.

But wait. What if we saw this coming and decided to actually design testing in from the beginning? Conceptually, it's not that hard. A company could enable Web test services by providing a parallel universe of services that used something as simple as using the word test to name the service. So, test product pricing would be the test version of the product pricing Web service, but it would be executed either by a test server or by a segregated area of production that handles only test transactions.

The current UDDI registry strategy presently contemplates at least part of this scenario through a private registry for test systems that can interact with the outside world but not vice versa. But why limit external accessibility? Why not open up the test environment to the world in both directions? Why not plan for a test community that mirrors the real one?

This would allow testing to become organic. Testers at each point could experiment with new versions, services and providers or with suppliers just as organically as they plan to in production. Companies could also do slick tricks like mirror the traffic to their production services to the test services registry, allowing real-world traffic to be handled in production but also tested by new or revised services before they go live.

Whatever the detailed implementation, the key is to treat testing as an integral aspect of the future of Web services, so that we not only enable new dimensions of functionality but also new levels of quality and reliability. Wouldn’t that be powerful?

Previous columns by Linda Hayes:

• Give Me a Test Hook — or Else
  Is it ignorance, paranoia or pure laziness that keeps developers from implanting components in their code that are needed for successful automated testing? Join the discussion about this column.
• Smarter Tools, Dumber Developers?
  Can anyone explain how tools can get so much smarter and developers get so much dumber? Or am I missing something? Join the online discussion about this column.
• Deliver Us From Genius
  Sometimes your development projects don't need bleeding-edge innovation and invention. Offshore outsourcing companies are threatening U.S. developers simply by delivering stable, quality goods on time and within budget. Join the online discussion about this column.

Print this Story

Send Us Feedback

E-mail this Story

Digg this Story

Slashdot this Story

Toward Chaos: Is the Decay of Code Inevitable? You're Not Crazy Testing in an Organic World

Give Me a Test Hook ¿ or Else Smarter Tools, Dumber Developers? Deliver Us From Genius

"IBM's old AS400 technology is fading fast, if product names are any indication...." Read more... "Systems support pilot fish starts getting reports that a special Y2k tool has stopped working — in early 2008...." Read more... Read more Development posts or See all Blogs

Lithuania: Attacks focused on hosting company Google bows to pressure, adds 'Privacy' link to home page Microsoft promises four patches next week More top stories...

Google gives away home-cooked Web application security scanner Expect iPhone, Fourth of July scams, security firm says Microsoft trumpets security additions in upcoming IE8

This old laptop: Revitalizing an aging notebook on the cheap All it takes is a couple hours and about $125 to breathe new life into an old laptop. Here's how. Bill Gates moves on Is Microsoft's Golden Age over? What are Gates' most memorable quotes? Find out in Computerworld's complete coverage of the end of the Bill Gates era at Microsoft. Five things you should never tell your boss There are some things your CIO definitely doesn't want to hear. Also don't miss the flipside, Five things you should always tell your boss. Hands-On: Firefox 3 fixes what's broke and keeps what's right With its latest version, Mozilla's browser continues to raise the bar for what Web browsers should be. Windows Vista A to Z Reviews, analyses, how-tos, visual tours, hot issues and predictions about Microsoft's new OS. More Continuing Coverage After 9/11: Homeland SecurityAsk a Premier 100 IT LeaderBlackBerry Legal BattleData Security BreachesElectronic VotingFirefoxH-1B VisasHP's Boardroom ScandalHandheld DevicesHurricane Katrina AftermathMicrosoft Legal IssuesMicrosoft's VistaOpen SourceRFID TechnologySCO's Linux FightSarbanes-Oxley ActSpamVoice Over IPWi-FiWindows Meta File VulnerabilityWindows XP Service Pack 3 IT Careers 2010 Four years from now, the IT field will be a vastly different place. Will you be ready? More Special Reports Premier 100 IT Leaders 2007 Best in Class2006 Computerworld Horizon Awards2006 Premier 100 IT Leaders2006 Salary Survey2007 Best Places to Work in IT2007 IT Forecast2007 Premier 100 IT Leaders40 Years of ComputerworldASPs, Take TwoBest Places to Work in IT 2003Best Places to Work in IT 2004Best Places to Work in IT 2005Best Places to Work in IT 2006Business Intelligence Home RunsBusiness Intelligence: Smarter BIBusiness Intelligence: The Future of BICRM Goes VerticalCRM: Sober CRMCareer Planning Guide 2005Careers: IT Profession 2010Computerworld Horizon Awards 2005Data Management: Mining for GemsData Management: Taming Data ChaosDevelopment: Hard-Workin' Web SitesDevelopment: New Tools New ChoicesDevelopment: The New World of Application DevelopmentDevelopment: The Web Services TsunamiDevelopment: Web Services HurdlesDisaster Recovery: Preparing for the WorstE-commerce Grows UpE-mail/Groupware: Big DecisionsFaces of Mobile ITForecast 2006Global MobileHardware: Multiple Cores, Multiple ChallengesHardware: On-Demand Un-HypedHardware: The Shape of Things to ComeHorizon Awards: 10 Cool Cutting Edge TechnologiesIT Management: Guide to Managing VendorsIT Management: Navigating Global ITIT Management: Recovery AheadIT Management: The Future of ITIT Management: The Resourceful Project ManagerInnovative Technology Awards 2004Management: Reinventing ITMicrosoft in TransitionMobile/Wireless Leaders and LaggardsMobile/Wireless: The Untethered WorkerMobile/Wireless: Tiny Gadgets, Huge CostsNetwork Management: Taking ControlNetworking: Turbulence Ahead!Networking: VoIP Goes MainstreamOperating Systems: In the Slow LaneOperating Systems: Linux Goes GlobalOperating Systems: Should You Nix Unix?Outsourcing: Offshore Buyer's GuideOutsourcing: Outsourcing DangersPremier 100 IT Leaders 2004 Best in ClassPremier 100 IT Leaders 2005 Best in ClassROI: Do the Math!Salary Survey 2003Salary Survey 2004Salary Survey 2005Security Action PlanSecurity: Compliance HeadachesSecurity: Proactive SecuritySecurity: Risk and RewardSecurity: Tips From Security ProsSouped-up SecurityStorage: Battling ComplexityStorage: Cheap & Secure Data StoresStorage: Stretching Your Storage DollarsStorage: The Lean Storage MachineStorage: The New Rules of StorageStorage: The Ultimate Backup GuideSupply Chain: Missing LinksSupply Chain: RFID Reality CheckThe Business of SecurityWeb 2.0 SecurityWireless: Wireless At Work All Zones Application Performance Zone Business Continuity Zone Data Center Management Zone Enterprise-Class Security Zone The File Data Management Zone Grid Computing on Windows Zone Security Management Zone ITIL Best Practices Zone The SAS Zone Storage Virtualization Zone Business Intelligence and Analytics Zone See your link here

Sold on SOA (Source: Computerworld) It's the hot technology for most large companies, but business, technical and cultural issues must be addressed for a successful SOA implementation. Get the whole story, from the big picture to the how-to-do-it details, in this Executive Bulletin. Download this Executive Bulletin (a $49.95 value) for Free, compliments of Fujitsu. Download this executive briefing  Driving Business Success Through Workgroup Choice and Flexibility Download this white paper, free, compliments of Novell! (Source: Novell) The structure of your workgroup environment plays a vital role in enabling your knowledge workers to be productive and collaborate securely. And IT choice and flexibility can mean the difference between reactive spending and proactive investment. Boost your competitive advantage with a workgroup infrastructure that lets you deliver the tools and services that are right for you. Download this white paper to learn how Novell offers a variety of solutions that give you the flexibility to address critical business initiatives and workforce productivity. Download this white paper  Safeguarding Critical Data Safeguarding Critical Data View this on demand webcast, free, compliments of IBM! Go to the webcast  White Papers Read up on the latest ideas and technologies from companies that sell hardware, software and services. Deploying Virtualized NetWare on Linux Whitepaper Toward More Flexible, Next-Generation Collaboration Solutions Driving Business Success Through Workgroup Choice and Flexibility View more whitepapers

• Toward Chaos: Is the Decay of Code Inevitable?
• You're Not Crazy
• Testing in an Organic World
• Give Me a Test Hook ¿ or Else
• Smarter Tools, Dumber Developers?
• Deliver Us From Genius

• Lithuania: Attacks focused on hosting company
• Google bows to pressure, adds 'Privacy' link to home page
• Microsoft promises four patches next week
• More top stories 

Intercept Spam & Viruses With MessageLabs MessageLabs is offering a complimentary 30 day trial of its managed Anti-virus and Anti-spam security solutions. MessageLabs guarantees complete protection against all know and unknown email threats. By providing 24 hour support, your business can increase productivity and decrease risk. Register for a complimentary trial and receive a free datasheet. Download this white paper now!

Detect, identify, and locate RF interference in 802.11 WLANs. AnalyzeAir software provides IT network professionals with the vision they need into the hidden world of RF, providing them with the ability to see the spectrum in a visible and intelligible format. AnalyzeAir software lets you see, monitor, analyze, and manage all the RF sources and wireless devices that influence your Wi-Fi network's performance and security, even if those devices are unauthorized or transient. AnalyzeAir Trial Software v3.1 highlights the features found in AnalyzeAir Software using a set of saved spectrum files. Replay the data and experience the visibility that AnalyzeAir Wi-Fi Spectrum Analyzer provides. Note: The trial software is limited to a player version only. It does not communicate with an AnalyzeAir PC card so it does not collect actual spectrum data. Register for this trial now.