Give Me a Test Hook — or Else

Keeping Pace with Changing Business Needs - Requirements Change Management Implementing KM: Practitioners Share Best Practices Enhancing Online Sales and Support: New Research Reveals What Works - and What Doesn't Sold on SOA Sign up to receive Security Resource Alerts

April 02, 2003 (Computerworld) -- In case you don't know it, without automation, software testing is hopeless. The latest development tools and component libraries allow developers to develop so much functionality so fast that it is literally impossible, actually inconceivable, that you can test the finished applications by hand. Most test budgets are measured in fractions of development, so it's not like you can spend more time and people testing than you do developing.

Of course it's easy for developers to automate unit testing; after all, they have control of the source. They can use debuggers, instrument their code, insert breakpoints, whatever. But if you have ever tried to automate black box automated testing, you quickly discover that test tools can't drive applications whose components aren't strictly vanilla. Custom controls, third party components, complex objects within containers and just about any user-defined or modified object classes give test tools fits. They can't get the object names, let alone the methods and properties needed to interact with them. (Getting good names is a whole other story—see Smarter Tools, Dumber Developers.)

	Linda Hayes is the CTO of WorkSoft Inc., developer of next-generation test automation solutions. She is the founder of three software companies and holds degrees in accounting, tax and law. A frequent industry speaker and award-winning author on software quality, she pioneered automated testing tools. Care to comment? Go here!

In my experience, more than half of all test automation time is spent—wasted—trying to deal with these complications, and in too many cases automation fails completely. But what is really inexcusable is that it doesn't have to be that way. Most test tools provide source code implants or DLL files that, when compiled into the code, give the tools access to the object names, methods and properties that are needed to do test automation. Adding this capability takes minutes—usually only a single line of code—but it can make the difference between automated and manual testing.

So what's the problem? In my opinion, it's either ignorance, paranoia or pure laziness.

Ignorance because a lot of companies think that if they compile in a test hook, then do the testing, then remove it for shipment that they have not really tested the production code. This is nonsense. True, the production code does not have the same hook as the tested version, but so long as the source is otherwise identical between the two compiles the only difference is the test hook.

The key is that these hooks don't do anything unless called. They are usually just a DLL that lets the tool inside the application's process space so it can see the objects. They only provide information, they don't alter or create it. So, compiling the code without the hook should have zero effect on the application functionality.

But if you just can't shake this superstition, then leave the hook in when you ship. This is where paranoia comes in. Haven't you just created a security problem? After all, now someone could use that hook to spy on your software. To that I say, so what?

Let's face it. The newer runtime-based languages (Java, .Net) are basically interpreted anyway. You can reverse the original source code out to the letter. So don't kid yourself that someone might be able to somehow peek into your software just because of a hook. Good grief, there are OS security holes right now that let strangers across the globe take complete control of your whole computer and everything it is attached to.

If you are still really freaked out about it, then identify those objects that are high risk and make their methods and properties private. Even the hooks can't get into those. But for goodness' sake don't do it to any that are needed for interaction with user interface objects; since they are exposed to the user there can't be much to hide in the first place, and in the second place you'll cripple automation.

Which leads me to the last problem: laziness. I go nonlinear when I hear developers complain about the "effort" it takes to address automated testability. Last time I checked it takes only a few minutes to either add or remove lines of code, and since compilation is usually automated anyway it may take zero time after the initial setup. If you can't be bothered to invest a few precious minutes to save your company weeks or months of work, or to enable automation that could make a difference in orders of magnitude in quality or time to market, maybe you should just retire since you don't really want to work in the first place.

But in the final analysis, the reason doesn't really matter. The real question is not whether you should provide a test hook for automation, it is why would there even be a controversy in the first place. Why would management even contemplate, let alone tolerate, applications requiring manual testing that increases costs while reducing quality? Why aren't developers required to deliver test hooks as a matter of course? If you think about it, it's completely crazy to even argue about it. The benefits are so undeniable and the risks are so debatable that there should not even be a question, let alone a war.

Of course I could be wrong, but in 20 years of test automation I have never had or even heard of a test hook backfiring. Has anyone else?

Print this Story

Send Us Feedback

E-mail this Story

Digg this Story

Slashdot this Story

Toward Chaos: Is the Decay of Code Inevitable? You're Not Crazy Testing in an Organic World

Give Me a Test Hook ¿ or Else Smarter Tools, Dumber Developers? Deliver Us From Genius

"Effective SOA deployments require a true standards-based development process, argues one vendor...." Read more... "IBM's old AS400 technology is fading fast, if product names are any indication...." Read more... Read more Development posts or See all Blogs

DNS hole prompts synchronized patching effort by IT vendors Microsoft plugs nine holes in Windows, DNS, SQL Symantec warns of new Word attack More top stories...

Microsoft sets XP SP3 automatic download for Thursday Don't give Google a free pass on data collection, privacy advocates say after YouTube ruling XP SP3 to reach most users 'shortly,' says Microsoft

This old laptop: Revitalizing an aging notebook on the cheap All it takes is a couple hours and about $125 to breathe new life into an old laptop. Here's how. Bill Gates moves on Is Microsoft's Golden Age over? What are Gates' most memorable quotes? Find out in Computerworld's complete coverage of the end of the Bill Gates era at Microsoft. Five things you should never tell your boss There are some things your CIO definitely doesn't want to hear. Also don't miss the flipside, Five things you should always tell your boss. Hands-On: Firefox 3 fixes what's broke and keeps what's right With its latest version, Mozilla's browser continues to raise the bar for what Web browsers should be. Windows Vista A to Z Reviews, analyses, how-tos, visual tours, hot issues and predictions about Microsoft's new OS. More Continuing Coverage After 9/11: Homeland SecurityAsk a Premier 100 IT LeaderBlackBerry Legal BattleData Security BreachesElectronic VotingFirefoxH-1B VisasHP's Boardroom ScandalHandheld DevicesHurricane Katrina AftermathMicrosoft Legal IssuesMicrosoft's VistaOpen SourceRFID TechnologySCO's Linux FightSarbanes-Oxley ActSpamVoice Over IPWi-FiWindows Meta File VulnerabilityWindows XP Service Pack 3 IT Careers 2010 Four years from now, the IT field will be a vastly different place. Will you be ready? More Special Reports Premier 100 IT Leaders 2007 Best in Class2006 Computerworld Horizon Awards2006 Premier 100 IT Leaders2006 Salary Survey2007 Best Places to Work in IT2007 IT Forecast2007 Premier 100 IT Leaders40 Years of ComputerworldASPs, Take TwoBest Places to Work in IT 2003Best Places to Work in IT 2004Best Places to Work in IT 2005Best Places to Work in IT 2006Business Intelligence Home RunsBusiness Intelligence: Smarter BIBusiness Intelligence: The Future of BICRM Goes VerticalCRM: Sober CRMCareer Planning Guide 2005Careers: IT Profession 2010Computerworld Horizon Awards 2005Data Management: Mining for GemsData Management: Taming Data ChaosDevelopment: Hard-Workin' Web SitesDevelopment: New Tools New ChoicesDevelopment: The New World of Application DevelopmentDevelopment: The Web Services TsunamiDevelopment: Web Services HurdlesDisaster Recovery: Preparing for the WorstE-commerce Grows UpE-mail/Groupware: Big DecisionsFaces of Mobile ITForecast 2006Global MobileHardware: Multiple Cores, Multiple ChallengesHardware: On-Demand Un-HypedHardware: The Shape of Things to ComeHorizon Awards: 10 Cool Cutting Edge TechnologiesIT Management: Guide to Managing VendorsIT Management: Navigating Global ITIT Management: Recovery AheadIT Management: The Future of ITIT Management: The Resourceful Project ManagerInnovative Technology Awards 2004Management: Reinventing ITMicrosoft in TransitionMobile/Wireless Leaders and LaggardsMobile/Wireless: The Untethered WorkerMobile/Wireless: Tiny Gadgets, Huge CostsNetwork Management: Taking ControlNetworking: Turbulence Ahead!Networking: VoIP Goes MainstreamOperating Systems: In the Slow LaneOperating Systems: Linux Goes GlobalOperating Systems: Should You Nix Unix?Outsourcing: Offshore Buyer's GuideOutsourcing: Outsourcing DangersPremier 100 IT Leaders 2004 Best in ClassPremier 100 IT Leaders 2005 Best in ClassROI: Do the Math!Salary Survey 2003Salary Survey 2004Salary Survey 2005Security Action PlanSecurity: Compliance HeadachesSecurity: Proactive SecuritySecurity: Risk and RewardSecurity: Tips From Security ProsSouped-up SecurityStorage: Battling ComplexityStorage: Cheap & Secure Data StoresStorage: Stretching Your Storage DollarsStorage: The Lean Storage MachineStorage: The New Rules of StorageStorage: The Ultimate Backup GuideSupply Chain: Missing LinksSupply Chain: RFID Reality CheckThe Business of SecurityWeb 2.0 SecurityWireless: Wireless At Work All Zones Application Performance Zone Business Continuity Zone Data Center Management Zone Enterprise-Class Security Zone The File Data Management Zone Grid Computing on Windows Zone Security Management Zone ITIL Best Practices Zone The SAS Zone Storage Virtualization Zone Business Intelligence and Analytics Zone See your link here

Sold on SOA (Source: Computerworld) It's the hot technology for most large companies, but business, technical and cultural issues must be addressed for a successful SOA implementation. Get the whole story, from the big picture to the how-to-do-it details, in this Executive Bulletin. Download this Executive Bulletin (a $49.95 value) for Free, compliments of Fujitsu. Download this executive briefing  Driving Business Success Through Workgroup Choice and Flexibility Download this white paper, free, compliments of Novell! (Source: Novell) The structure of your workgroup environment plays a vital role in enabling your knowledge workers to be productive and collaborate securely. And IT choice and flexibility can mean the difference between reactive spending and proactive investment. Boost your competitive advantage with a workgroup infrastructure that lets you deliver the tools and services that are right for you. Download this white paper to learn how Novell offers a variety of solutions that give you the flexibility to address critical business initiatives and workforce productivity. Download this white paper  Safeguarding Critical Data Safeguarding Critical Data View this on demand webcast, free, compliments of IBM! Go to the webcast  White Papers Read up on the latest ideas and technologies from companies that sell hardware, software and services. Virtualization Analysis for VMware A Guide to Understanding Messaging Archiving Archiving Compliance with Sunbelt Exchange Archiver View more whitepapers

• Toward Chaos: Is the Decay of Code Inevitable?
• You're Not Crazy
• Testing in an Organic World
• Give Me a Test Hook ¿ or Else
• Smarter Tools, Dumber Developers?
• Deliver Us From Genius

• DNS hole prompts synchronized patching effort by IT vendors
• Microsoft plugs nine holes in Windows, DNS, SQL
• Symantec warns of new Word attack
• More top stories 

Intercept Spam & Viruses With MessageLabs MessageLabs is offering a complimentary 30 day trial of its managed Anti-virus and Anti-spam security solutions. MessageLabs guarantees complete protection against all know and unknown email threats. By providing 24 hour support, your business can increase productivity and decrease risk. Register for a complimentary trial and receive a free datasheet. Download this white paper now!

Detect, identify, and locate RF interference in 802.11 WLANs. AnalyzeAir software provides IT network professionals with the vision they need into the hidden world of RF, providing them with the ability to see the spectrum in a visible and intelligible format. AnalyzeAir software lets you see, monitor, analyze, and manage all the RF sources and wireless devices that influence your Wi-Fi network's performance and security, even if those devices are unauthorized or transient. AnalyzeAir Trial Software v3.1 highlights the features found in AnalyzeAir Software using a set of saved spectrum files. Replay the data and experience the visibility that AnalyzeAir Wi-Fi Spectrum Analyzer provides. Note: The trial software is limited to a player version only. It does not communicate with an AnalyzeAir PC card so it does not collect actual spectrum data. Register for this trial now.