It's no secret that software development is in a sorry state. Many projects fail; most others come in late or over budget—or both—according to industry reports.
The main reason we don't consistently achieve our desired results in managing software development is that we haven't settled the question of what software development is.
If you manage a software development organization, I've got a question for you. Can you explain the "why" of your approach to managing projects without using the word like? As in "Software development is like ..."
Why, after almost 50 years of experience, are we still working from analogy? And why are we comparing software development to activities like a game called Kerplunk, gardening or writing a novel? (Google on "software development is like" if you don't believe me. It gets worse.) Are these analogies supposed to be leading us to effective management strategies for software development?
The earliest software managers had no choice but to work from analogy. In 1956, Herbert Benington presented "a description of the organization and techniques we used in the mid-1950's to produce programs for the SAGE air-defense system," arguably the world's first truly large-scale software development effort.
Benington and his team had to work from analogy. They were pioneers. So they did what was reasonable. They applied lessons learned from what they already knew: hardware. Said Benington, "It is easy for me to single out the one factor that I think led to our relative success: We were all engineers and had been trained to organize our efforts along engineering lines." (Annals of the History of Computing, Vol. 5, No. 4, pg. 300.)
The question is, given all the advances in tools and in the educational system, with all the experience we now have with software development, how did we end up switching from "software development is like hardware development" to the use of analogies such as gardening? Isn't software development really just like hardware development? That question, and the implications of the answer, is what this and my next several columns will explore.
Let's begin our exploration using a model of production that's basic enough that we can be reasonably assured it applies to both hardware and software.
At the end of the production process, there is the Product, the working, usable result of our labor. There doesn't seem to me to be much controversy lurking around this definition. Whether it's a PC or a program that runs on one, it doesn't seem valid to refer to it as Product until it's working and usable in the end-user sense.
At the beginning of the production process, there is the Design. (For now, we'll assume that Requirements exist outside the production process). The Design is complete when it's been validated within the Development process.
In between Design/Development and the Product is the Making—the activity of turning the Design into the Product. Because our intent is to determine whether it's reasonable to use the same management approach for both software and hardware, we need to be especially careful here.
 |
  |
|
Bill Walton has 15 years of experience in mostly large software development organizations. He now works as an independent IT consultant specializing in requirements engineering and software test management. He can be reached at bill.walton@jstats.com. |
In the production of hardware, the Making takes the Design and the Parts specified in the Design as inputs and produces, or outputs, the Product. In hardware (think PCs), Making occurs in two environments. Designs are Made into prototypes in the Development environment in order to validate the Design's readiness for release to Manufacturing. Designs are also Made into Products on the manufacturing floor for eventual sale to customers.
In discussing the nature of software development, the question "Where are the Parts?" often comes up. Are the source code statements Parts? The answer is no. A precompiled DLL qualifies as a Part. But source code doesn't because it's not "working and usable in an end-user sense." In software, the input to the Making process is, in most cases, just the Design. That is the thing, and the only thing, that makes software development fundamentally "different" from hardware development.
Given the above, the truth of the following statements seems clear to me. Source code statements are Part Designs, not Parts. The framework/algorithm within which the source code statements exist is the Product Design. Compiling (and Linking) is Making. The executable code is the Prototype or Product depending on whether or not it has been verified as acceptably free from Design defects.
Figure 1 shows what I believe is the appropriate mapping between the hardware and software product processes.
 |
|||||||||||||
|
So I ask again: Is software development really like gardening (or any of the other weird analogies you're likely to Google up)? Or isn't it really just like hardware development? To circle back and answer the question I started with:
Computer software development is a Design/Development phase activity. It is best managed using exactly the same approaches that have proved successful in managing the Design/Development phase of computer hardware.
We'll look at this, and its implications vis-a-vis some of the problems that got me started on this track, in more detail in upcoming columns.
If you work in a hardware manufacturing company, I'd appreciate it if you'd help me out with a little data collection. Shoot me an e-mail at bill.walton@jstats.com and let me know the answer to the following question:
Does your company manage your IT groups' projects and your Product Design/Development groups' projects using the same approach?
If software development is, as I posit, a Design/Development phase activity, the two groups' plans (i.e., timelines) should be very similar. I'll bet you a nickel they're not. And I'll be happy to pay up if they are. Send me some proof, and I'll send you my business card, signed with congratulations and a nickel taped to the back. Either way, don't worry about "airing your company's dirty laundry." I'm always willing to change the names to protect the guilty!
Post Feedback
Want to post a response to this article? Send the editor an e-mail message.
Readers Respond:
Ray essay presents the other side
In Eric Ray's "The Magic Cauldron," good points are made against favorable comparison between manufacturing and writing software. This essay is well worth reading to hear the other side of the argument Bill Walton is making.
This essay comes as part of published book, "The Cathedral and the Bazaar." It is also available online for free.
-- Max Polk
Author responds:
Hi Max,
Thanks for making time to read the article and for your feedback. Ray's article does indeed argue, as I do, against the use of the manufacturing analogy, but he's making a completely different argument.
Ray is arguing that software should not be "valued" based on what it costs to develop, refuting the "you get what you pay for" argument against open source software. It's a good point, but not one, as I'm sure you realize, that I've attempted to tackle in this article. Thanks again.
-- Bill
Read Martin?
Has the writer read any of James Martin's textbooks? Have you come across Information Engineering methodology? I think he wrote about this very topic about 20 years ago...
--L.C.
Author responds:
Hi L.C.,
I have read Martin, although it's been many years. I remember the RAD movement very well. I don't, however, remember Martin making a plain case for it. RAD makes sense, IMO, because software development is a Design activity and the single most important determiner of success in any Design activity is feedback. I don't have copies of his books anymore, so I'll have to see if I can dig them up somewhere and see if he did, in fact, state his case this clearly and I just missed it.
When I wrote the article I did not, still do not, remember every having seen the point made the way I've tackled it. Another reader, however, has pointed me to an article written by Jack W. Reeves in 1992 that did make the point that source code is design. If you're interested, it's here. It seems that he only wrote on the topic once, so I'm planning to try to contact Reeves and find out why he dropped it.
Thanks for making time to read the article and for your feedback. I appreciate both.
Best regards,
-- Bill
Mapping is missing
This is an interesting article.
One of the most direct mappings between hardware design and the software line of source code is to view a line of source code in the same way as a piece of geometry (a line or pad) in the mask design of an integrated circuit.
The collection of geometries that compose the masks of an integrated circuit defines the function of a completed part. So too, a collection of source code lines defines the behaviors of a software component.
One can construct formal mappings between hardware and software structures using these concepts.
Taking these concepts forward, there are a few conceptual problems with the way in which the comments of the article map onto the hardware development process model.
I believe that some of these conceptual problems are at the heart of many difficulties we seem to have with quantifying software development.
I am a computer engineer by training. My course of study began with electrical engineering, so that hardware design and its theoretical models and abstractions are clear under that discipline. My studies then took me into software design, where it was demonstrated that the two are the same.
The engineering process in deciding how to design a product is a matter of chosing which function is to be implemented using which technology, silicon or software.
A very large culture has emerged of software developers who are not trained in electrical engineering. The mappings are missing and this often very clear when one studies designs and project management materials.
I am not saying that it is in any way simpler to design a system on silicon than it is to design a large software system. The mapping, however, forms a basis for applying a mature body of knowledge to facilitate the design of both.
-- Rick Doyle
Reeves addressed same issues
This idea is still as pertinent now as it was in 1992 when Jack W. Reeves addressed the same issues in the article "What is Software Design?".
-- Scott Bellware
It's not like cooking!
Many users (who fund development) and software development managers alike think that Software Development is like cooking a dish. Missing seasoning? Add more!
An education system that primarily teaches young people and experienced professionals only how to program adds to the trouble. In the end, we have a universe of undisciplined users, managers and developers.
Why are we acting surprised that software development is characterized by failure?
-- Srinivas Bangarbale
Analogies are just a start
Sorry Mr. Walton, I think that many of those analogies about software development are true, and I'd even like to add my own. Software development is like making motion pictures. Think about the old Hollywood studio system. Or think about someone doing animation all by themselves. Or maybe software development isn't really like anything else, except, well, software development.
Analogies aren't ever perfect, they're just points of departure on a journey of understanding.
Have fun.
-- Louis Russ
Software parts fundamentally different
This is a comment arguing that software is quite distinct to hardware development.
I agree with Bill's statement about parts, both in hardware and software development. And that they are the objects that make the real difference between hardware and software development or between software development and gardening if you want.
Parts in hardware development are part of the input, of the constructions process, of the output, and they are subject to quality assurance from the beginning to the end. Parts in hardware development (and in gardening) can be objectively measured as input, during construction, and as the final output.
This is fundamentally different to software parts. We still lack such undoubted and comprehensive measures, their scales and tolerances. This induces a much higher degree of uncertainty into the software development process. And as always with a high degree of uncertainty we run into high risks and failure isn't far. Thus, without focusing on these aspects of the parts we won't gain new insights into software project management processes.
-- Wolfram Bartussek