The roots of failure in software development management

Computerworld - It's no secret that software development is in a sorry state. Many projects fail; most others come in late or over budget—or both—according to industry reports.

The main reason we don't consistently achieve our desired results in managing software development is that we haven't settled the question of what software development is.

If you manage a software development organization, I've got a question for you. Can you explain the "why" of your approach to managing projects without using the word like? As in "Software development is like ..."

Why, after almost 50 years of experience, are we still working from analogy? And why are we comparing software development to activities like a game called Kerplunk, gardening or writing a novel? (Google on "software development is like" if you don't believe me. It gets worse.) Are these analogies supposed to be leading us to effective management strategies for software development?

The earliest software managers had no choice but to work from analogy. In 1956, Herbert Benington presented "a description of the organization and techniques we used in the mid-1950's to produce programs for the SAGE air-defense system," arguably the world's first truly large-scale software development effort.

Benington and his team had to work from analogy. They were pioneers. So they did what was reasonable. They applied lessons learned from what they already knew: hardware. Said Benington, "It is easy for me to single out the one factor that I think led to our relative success: We were all engineers and had been trained to organize our efforts along engineering lines." (Annals of the History of Computing, Vol. 5, No. 4, pg. 300.)

The question is, given all the advances in tools and in the educational system, with all the experience we now have with software development, how did we end up switching from "software development is like hardware development" to the use of analogies such as gardening? Isn't software development really just like hardware development? That question, and the implications of the answer, is what this and my next several columns will explore.

Let's begin our exploration using a model of production that's basic enough that we can be reasonably assured it applies to both hardware and software.

At the end of the production process, there is the Product, the working, usable result of our labor. There doesn't seem to me to be much controversy lurking around this definition. Whether it's a PC or a program that runs on one, it doesn't seem valid to refer to it as Product until it's working and usable in the end-user sense.

At the beginning of the production process, there is the Design. (For now, we'll assume that Requirements exist outside the production process). The Design is complete when it's been validated within the Development process.