First, "the tame, systematic software engineering that is taught at the universities," like any other type of engineering taught at the universities, doesn't exist anywhere but the universities. I was fortunate enough to work in the PC world when Innovation was its defining characteristic. There's still some going on there, in areas like storage (with SAN, iSCSI, etc.), but not much. And I can tell you without hesitation that it was any thing but tame. Requirements changes are very much a fact of life in any Design/Development effort I've ever participated in, or even heard of. Yes, it's probably true that software typically sees more requests for changes than hardware. Software typically involves more areas of functionality than hardware, so there's more areas where changes can be requested. But I think it would be really interesting to see some research that compares the number of changes, "acre-for-acre" if you will, that are requested for software while it's in development relative to the same measure for hardware. If we're comparing the software Design/Development phase to the hardware Design/Development phase, there's very little difference in the cost of change. Changing a schematic is pretty much just a labor cost, just like changing source code. The real cost in both cases is time-to-market.
Second, we agree that software is different in its non-physical nature. I said in the article that that's the thing, and the only thing that makes software development fundamentally different from hardware development. It loosens the constraints that would otherwise lead the hardware engineers I've worked with to use a process that looks very much like XP; more iterations with smaller increments.
Thanks again, and best regards,
-- Bill
Failure to learn
This article by Bill Walton caught my attention through a disbelief that we are still talking about this. The replies were also interesting and to the point. Overall the article raised a lot of questions that I think can be discussed from the viewpoint of 2004 not 1950 or 1970, etc. I hope the discussion can continue.
One observation. We have often muddled the application development process with the software development process. The application development process (which include the software development process as one step) remains largely a learning process.
When customers are directly involved in the process like an application development within an organization, they and the IT professionals are in a continuous learning process that ultimate results in a compromise -- the specification statement that is eventually implemented in a collection of hardware, software, telecom, procedures, etc. This learning process continues as the specifications are implemented and the customers arrive at actual use-time somewhat, or a lot, ahead of the product. In addition the business, economic, regulatory, etc environment has changed. A mismatch between the product and current reality is guaranteed.
When the customer is a retail customers (buying Windows, OS X, Word, etc, etc, error and mismatches are discovered through use. Vendors re-engineer the product, trusting that the new version, release, etc will correct both errors and functional demands. Learning is just as dynamic on both sides.
Often we confuse this learning with a failure of customers and implementers. We do not take into account the amount of learning that occurs and change that the technology enables; both proceed at a rate much faster than applications and software builders can accommodate. Hence there is a lot of dissatisfaction and charges of incompetence fly. Criticism is the standard.
This problem will solve itself in 40 or 50 years and participants on all sides learn more about the dynamics of the processes and agree on standards to use in going forward. How long did it take to establish accounting standards? How well do they work? How long did it take to learn to build bridges?
Between now and then, we need to have ongoing discussion, argument, trial and error, investment in new practices, and patience.
-- Bob Rouse
Washington University--Saint Louis
Software development is like software development. Period.
Thank you for initiating a conversation on one of the most important topics in our industry.
The problem is not how do we do software development well, but how do we disseminate what we already know to create a truly professional, qualified workforce?
Ample hard data exists on what works and what does not for software development projects. Companies that are process-centric enable average people to do great things, whereas in "cult of the hero," chaotic environments, it takes great people to do even mediocre work.
Given the staggering quantity of data we have on software projects of all sizes and varieties, we can state categorically that software development is like software development. Period. Sure, the analogies with other fields of endeavor are fun and likely to produce some valuable insights, but would you want a gardener running your multi-million dollar CRM project? Further, given the choice between a highly experienced, successful software PM and an equivalent hardware PM, most managers would choose the software PM.
There are a number of institutions and individuals who have made learning best practices fairly simple. The Software Engineering Institute at Carnegie Mellon would be a good start. Also, have a look at Steve McConnell, Capers Jones, and Barry Boehm, all of whom have a lot of clearly written, pragmatic wisdom to share. There are many, many others, but if everyone had their knowledge as a baseline, we would be well on our way to fixing software quality and schedule problems.
Sadly, the Project Management Institute has also fallen prey to this muddled thinking by not differentiating between managing a construction project for a building and managing a software project (go ahead, hire a construction PM to manage that CRM project). We can only hope for a truly useful, representative set of exams and the evolution of a culture where best practices are expected basic knowledge for developers and managers alike.
I remain hopeful that we will continue to improve our methods as well as our ability to educate software practitioners and the businesses we serve.
-- Andre Lockhart