Survival of the fittest

Gary Anthes   Today’s Top Stories   or  Other Business Intelligence Stories  

Take Control! Managing Your Application Library The Advantages of Migrating to Windows Vista® and Office 2007 Simultaneously Eliminate Excedrin Headaches: Simple, Time-Effective, Powerful PC Image Management The Trend from Unix to Linux in SAP Data Centers 3PAR Thin Copy Desktop: A VDI-Optimized Storage Solution Improving Correspondence Management for Financial Services Firms with Adobe LifeCycle ES Learn-Fast Guide: Software as a Service is Growing Up SaaS Solutions for Remote Systems Management The Business Value of Tape Storage Sign up to receive Software Resource Alerts

September 6, 2004 (Computerworld) -- Survival analysis could help you predict that one of your best customers is about to jump ship for a competitor. Or it could help you decide whether that costly promotion is really going to be worth it. Or it could help you tailor that next catalog mailing and double your return.

The aptly named analytic technique, also called survival data mining, has been used by doctors for decades to predict the life expectancy of heart-transplant patients and by biologists to assess the probability that a cell invaded by a virus will die within 24 hours. Engineers have long used it to estimate the mean time to failure of a disk drive or a robotic welder. More recently, sociologists and psychologist have started using it to predict when certain types of people will divorce or seek help for depression.

But until recently, attempts to apply survival analysis to business problems have been mostly university projects, says Edward Malthouse, a marketing communications professor at Northwestern University. "Now it's really taking off in the database marketing world—for credit cards, hotels, airlines, catalogs and so on," he says.

Survival analysis refers to a family of "time to event" prediction techniques mathematically geared to problems with the following characteristics:

• They deal with discrete events that will occur to some but not all members of a given population. Certain patients will die, some disk drives will fail, a certain number of prescriptions for Valium will be written, and some of your best customers will desert you.

• They involve time-dependent outcomes. Is that key customer going to cut up his charge card tomorrow (better call him today), next quarter (send him mail) or not in the next five years (leave him be)?

• The outcomes of interest, or "dependent variables," aren't continuous—like income, height or IQ—but are dichotomous. Either the patient will die within six months or he won't. Your customer will leave this year or he won't.

• Outcomes often can be anticipated by trigger events, such as customer complaints.

Randy Collica, a senior analyst at Hewlett-Packard Co., says use of survival mining to understand and predict customer behavior has sprung up in the past couple of years. He says it's really just an extension of older practices in which a company would take its most recent customer data, or data from a time slice such as a quarter or year, and try to predict attrition based on that.

But survival mining puts time itself into the analysis as a variable. "It is a superset technique," Collica says. "Including time as an element is adding that much more information."

Choosing a Technique

Software for survival analysis most often uses some kind of regression technique, notably logistic regression or a kind of regression called Cox proportional hazards modeling. In addition, some vendors sell products that do time-to-event predicting using neural networks or genetic algorithms.

Continued...
1 | 2 | NEXT  

Print this Story

Send Us Feedback

E-mail this Story

Digg this Story

Slashdot this Story

Survival of the fittest

Sidebar: The Perils of Prediction

"The social network search engine goes offline...." Read more... "Salary surveys musings often cause the mind to stray. IT jobs seem in a growth cycle, despite all the new..." Read more... Read more Business Intelligence posts or See all Blogs

At 10, Google reiterates commitment to CIOs Microsoft explains Seinfeld-Windows TV ad: just a 'teaser' Continuing coverage: Google's Chrome browser More top stories...

iPhone 3G owner sues Apple, AT&T over dropped calls, app crashes Mozilla: Firefox is faster than Chrome Upcoming Microsoft patch lineup could be 'massive,' says researcher

Microsoft warns of IE8 lock-in with XP SP3 Users of Windows XP SP3 who try out IE8 Beta 2 won't be able to uninstall either one under certain circumstances. Opinion: Why Google has lost its mojo -- and why you should care Google has gone from innovative upstart to fat-and-happy industry leader in what seems like record time. Preston Gralla explains. Review: Internet Explorer 8 Beta 2 offers some nifty new features Microsoft's latest beta of IE8 includes better tab management, new services such as Web Slices and Accelerators, and the new 'porn mode.' IT Schools to Watch 2008 These leading-edge graduate schools are moving at the pace of the IT workplace, delivering coursework that's relevant to today's IT professionals. Windows Vista A to Z Reviews, analyses, how-tos, visual tours, hot issues and predictions about Microsoft's new OS. More Continuing Coverage After 9/11: Homeland Security Apple's iPhone Ask a Premier 100 IT Leader Bill Gates steps down BlackBerry Legal Battle Data Security Breaches Electronic Voting Firefox H-1B Visas HP's Boardroom Scandal Handheld Devices Hurricane Katrina Aftermath Microsoft Legal Issues Microsoft's Vista Open Source RFID Technology SCO's Linux Fight Sarbanes-Oxley Act Spam Voice Over IP Wi-Fi Windows Meta File Vulnerability Windows XP Service Pack 3 IT Careers 2010 Four years from now, the IT field will be a vastly different place. Will you be ready? More Special Reports Premier 100 IT Leaders 2007 Best in Class 2006 Computerworld Horizon Awards 2006 Premier 100 IT Leaders 2006 Salary Survey 2007 Best Places to Work in IT 2007 IT Forecast 2007 Premier 100 IT Leaders 40 Years of Computerworld ASPs, Take Two Best Places to Work in IT 2003 Best Places to Work in IT 2004 Best Places to Work in IT 2005 Best Places to Work in IT 2006 Business Intelligence Home Runs Business Intelligence: Smarter BI Business Intelligence: The Future of BI CRM Goes Vertical CRM: Sober CRM Career Planning Guide 2005 Careers: IT Profession 2010 Computerworld Horizon Awards 2005 Data Management: Mining for Gems Data Management: Taming Data Chaos Development: Hard-Workin' Web Sites Development: New Tools New Choices Development: The New World of Application Development Development: The Web Services Tsunami Development: Web Services Hurdles Disaster Recovery: Preparing for the Worst E-commerce Grows Up E-mail/Groupware: Big Decisions Faces of Mobile IT Forecast 2006 Global Mobile Hardware: Multiple Cores, Multiple Challenges Hardware: On-Demand Un-Hyped Hardware: The Shape of Things to Come Horizon Awards: 10 Cool Cutting Edge Technologies IT Management: Guide to Managing Vendors IT Management: Navigating Global IT IT Management: Recovery Ahead IT Management: The Future of IT IT Management: The Resourceful Project Manager Innovative Technology Awards 2004 Management: Reinventing IT Microsoft in Transition Mobile/Wireless Leaders and Laggards Mobile/Wireless: The Untethered Worker Mobile/Wireless: Tiny Gadgets, Huge Costs Network Management: Taking Control Networking: Turbulence Ahead! Networking: VoIP Goes Mainstream Operating Systems: In the Slow Lane Operating Systems: Linux Goes Global Operating Systems: Should You Nix Unix? Outsourcing: Offshore Buyer's Guide Outsourcing: Outsourcing Dangers Premier 100 IT Leaders 2004 Best in Class Premier 100 IT Leaders 2005 Best in Class ROI: Do the Math! Salary Survey 2003 Salary Survey 2004 Salary Survey 2005 Security Action Plan Security: Compliance Headaches Security: Proactive Security Security: Risk and Reward Security: Tips From Security Pros Souped-up Security Storage: Battling Complexity Storage: Cheap & Secure Data Stores Storage: Stretching Your Storage Dollars Storage: The Lean Storage Machine Storage: The New Rules of Storage Storage: The Ultimate Backup Guide Supply Chain: Missing Links Supply Chain: RFID Reality Check The Business of Security Web 2.0 Security Wireless: Wireless At Work All Zones Application Performance Zone Business Continuity Zone The File Data Management Zone Security Management Zone ITIL Best Practices Zone The SAS Zone Business Intelligence and Analytics Zone Windows Protection Zone Identity & Security Management Zone See your link here

Computerworld Technology Briefing: An open-source path to optimal virtualization Download this Technology Briefing now! (Source: Novell/IBM/Intel) Virtualization is about a lot more than just lowering total cost of ownership. In fact users that have taken an open source path to virtualization have realized the additional, mission-critical benefit of markedly reduced IT complexity, as well as a more flexible infrastructure that is easier to change to meet shifting, often unpredictable business requirements. Download this executive briefing  From Laggard to Leader: Transforming the Data Center From Laggard to Leader: Transforming the Data Center Register for this complimentary live webcast today! Go to the webcast  Windows Vista®: A Cyber Security Shield Get this paper now! (Source: Dell) Windows Vista® incorporates a number of new and enhanced security features that address spyware, security vulnerabilities and end user naivete. Download this white paper  White Papers Read up on the latest ideas and technologies from companies that sell hardware, software and services. Death to PST: Hidden Cost of Email Mismanagement Extend, Replace, or Convert; which is the best way forward for COBOL Applications? The Trend from Unix to Linux in SAP Data Centers View more whitepapers

• Survival of the fittest
• Sidebar: The Perils of Prediction

• At 10, Google reiterates commitment to CIOs
• iPhone 3G owner sues Apple, AT&T over dropped calls, app crashes
• Microsoft explains Seinfeld-Windows TV ad: just a 'teaser'
• More top stories 

Brought to you by Computerworld and InfoWorld, this virtual conference focuses on key areas such as: Service-oriented architecture (SOA) The event-driven enterprise Software-as-a-service (SaaS) Business architecture Plus many more tools to build, implement and maintain an agile architecture for the enterprise. This new and exciting event happens on September 30th from 10am - 6pm ET / 7am-3pm PT. Register now

Creating Tiered Storage Architecture This paper shows how Brocade StorageX enables tiers of storage across Windows and NetApp storage to align storage practices with organizational objectives. Download this white paper now!

Best Places to Work in IT Where can you earn top dollar, get the best benefits, the latest IT and more? Find out in our 14th annual survey.