Evaluating Project Success, Failure -- and Everything in Between

Computerworld - According to the widely quoted "Chaos Chronicles," a staggering 66% of IT projects prove unsuccessful in some measure, whether they fail completely, exceed their allotted budget, aren't completed according to schedule or are rolled out with fewer features and functions than promised.

Clearly, IT project management could do with some improvement. But project managers themselves are often working so hard, and racing against such tight deadlines, that they simply don't have the time to pause and reflect on the efficacy of their project management practices.

Yet the key to improving the rate of project success may lie right underneath managers' noses in the form of comprehensive project retrospectives. Allotting time for the formal evaluation of a project's success, extracting the primary lessons from the project management process and making actionable recommendations for future projects (or for the future of the project in question) might very well, in the longer term, represent the most efficient use of managers' time and resources.

Graduate students in the master of science in the management of IT program at the University of Virginia's McIntire School of Commerce have, since 1999, collectively performed 72 IT project retrospectives at 57 organizations. These retrospectives have ranged from the examination of relatively small (several hundred thousand dollars) intrafirm projects to very large (over $100 million) mission-critical projects involving multiple external providers.

Each retrospective tells a unique story in its own right and provides a rich understanding of specific project management practices used within a specific time frame. But when viewed as a whole, these 72 projects offer the opportunity for considerable insight into project management at the macro level, revealing trends that may be generalized across a wide spectrum of applications and organizations.

Fundamentally, the project retrospectives showed success itself is difficult to define. In a typical group of stakeholders—i.e., the project manager, team members, product end users, project sponsor and top management—a project's success might, at any given moment, receive very different ratings.

Given this reality, it became clear that an evaluation of project success should include both process and outcome criteria. They used the following process-related criteria:

1. Time: Did the project come in on schedule?
2. Cost: Did the project come in according to budget?
3. Product: Did the project result in a product of acceptable quality and meet other product-related specifications?

The three outcome-related criteria they used were:

1. Use: Were the project's resultant products/services used by its intended constituents?
2. Learning: Did the project increase stakeholder knowledge and better prepare the organization for future challenges?
3. Value Did the project lead directly to the organization's improved efficiency or effectiveness? Common metrics include net present value (NPV), internal rate of return (IRR), economic value added (EVA) and the balanced scorecard.