Author responds: Hi Wolfram, Thanks for making the time to read the article and for sending your feedback. I appreciate both. Please see my response to William Simpson. Best regards,
-- Bill
Proper techniques are known, just not practiced
I don't know when many of the posters last did a hardware design, but...
The first part (specification) is a superset of software design because you must not only specify the logical functions and size, but things like power requirements, operating altitudes, etc.
The design and coding phase is...virtually identical!!! The majority of digital hardware is designed by writing code in Verilog, VHDL, or System C. This code looks remarkably like software. Debugging is done almost the same with simulators. Many companies use 3rd party components, also written in Verilog. Physical size constraints are very similar to memory constraints. But there's also power, etc.
The interesting thing is that hardware engineers are learning from software engineers how to do large scale coding projects. I strongly believe that the problem is not that proper software engineering techniques are not known, it's just that they aren't practiced.
-- Larry Lewis
Director of R&D
SAS Institute
Author responds:
Hi Larry,
I completely agree with your comments. I think the reason they aren't practiced is that that requires discipline, and discipline is hard. Much easier to hide behind "software is different." That's why I decided to start the discussion where I did. Software is not different, not from other Design activities characterized by similar degrees of Innovation/Uncertainty. And those projects are all around us if we're willing to look.
Thanks for making the time to read the column and for your feedback. I hope you'll be able to make time for the columns that follow.
Best regards,
-- Bill
It's all about parts
The problem, as I see it is not one of management, it is one of part failure. What is the difference between hardware design and software design? Parts (components if you prefer).
Every hardware part can be fully tested. One can "test" every part in their design by simulation. Every part, from the simple transistor, to the CPU can be fully described and plugged in. In software the parts are not uniform. Building a bridge, by way of illustration, is not too difficult. Do one thing and repeat it over and over. Each part is fully understood and does not change.
Until software parts, be they DLLs or stand-alone objects that other objects can use, are as fully testable and repeatable as hardware components nothing will improve. All the software components, from the simple to the most complex will need to be defined and stabilized to a point of standard usability. Programmers will have to give up their pet code and holier than thou attitude and use only well tested components just like all the other disciplines. Any new part introduced will have to undergo the same scrutiny as new drugs by the FDA. The software engineer/designer, like the other professionals, will earn his/her pay by knowing which components to use, when to use them, and how to put them together for the client's needs.
Doctors don't make up their own medicines, nor do bridge builders use circles instead of triangles to build bridges because they might look "cooler." Software engineers need to stop trying to reinvent the same wheel, pull back their egos, and make things which work the first time because all the components are known to be reliable and repeatable.
One of the great things about software, unlike hardware, is that there will be no bad components. Every one will be exactly the same. No manufacturing imperfections to botch things up. And they can be made in unlimited quantities, instantly.
-- William Simpson
Hi William,
Thanks for making the time to read the article and for sending your feedback. I appreciate both. But, with all due respect, I disagree. I submit that your argument, and those of the other readers who have responded similarly re: parts, is a reflection of the thought pattern instilled by the linguistic trap I've addressed in the article.
As you note, to test hardware parts in the Design stage, you have to simulate them. Ask any hardware engineer and (s)he will tell you that simulation, while helpful, does not replace testing the actual product. In software, because the cost of Making the product is so trivial, we don't have to resort to simulation. We can cost effectively move directly to testing the actual product.
I will admit that the use of parameters makes software parts potentially more complex than some hardware parts. But two points very effectively counter the argument that this makes software development fundamentally different from hardware development. First, there are lots of complex hardware parts and lots of simple software parts, so it's very easy to come up with counter examples. Second, and more importantly, there are well known engineering techniques that can and should be applied to the Design/Development of products in both domains to minimize unnecessary complexity. Unfortunately, we do not see those techniques broadly understood or regularly applied in the software domain.
As to the "fully testable and repeatable" nature of software vs. hardware parts, this is a BIG red herring. In fact, software parts have a huge inherent advantage over hardware parts in this respect. Hardware parts are inherently variable. The thickness of the plating on a resistor's connector, for example, has an impact on the part's output. Check the specs for any hardware parts you choose and you will see that they all suffer this variability to some degree. The inputs and outputs of all hardware parts are given as targets with a range (e.g., 3.5 Ohm +/- .05). Software parts, on the other hand are inherently invariant. The compiler will produce exactly the same machine instruction every time it encounters the same source code statement (software part design) and that machine instruction will perform exactly the same actions each time it is executed with the same set of parameters.
As to "making things that work the first time," I submit this expectation is at the heart of the problem. Software development is entirely a Design/Development activity and no nontrivial Design "works" the first time. Design is inherently a learning process with the amount of learning required depending fundamentally on the amount of Innovation involved in the product and the project. If your definition of "works" includes "does what the customer needs it to do" then you have to accept that customers have to actually use the product to come to a complete and final specification of their needs. They accept that fact in hardware development. We should too.
Thanks again,
-- Bill
Isn't the cost of quality worth it?
I read your article with great interest. I do have one question for you regarding the comparison of software development with the standard manufacturing model.
In the standard manufacturing model, you have a tangible article to hold and look at to determine if it is usable, ergonomic, etc. You do not generally have this with a software model in the same sense, especially where functionality is concerned, as most manufactured articles of this nature are single-purpose, which is seldom the case with software.
When a manufactured article is not functional for some reason or another, you send it back to the design group, who re-establishes contact with the users, for modifications in the process to make it more useable -- for instance case design.
With software, this step is often skipped and the item is sent back (to what would normally be the machine operator in the standard manufacturing model) for changes that often result in other design flaws.
Why is it that in the software manufacturing model, we (industry) are not ready to bite the bullet and go all the way back to design to ensure the quality of the finished product as we do in other manufacturing processes? It does drive up the cost as it does in any process but is the cost not worth the expense in producing a final product that is better to the end user?
-- Mark Beberstein
Author responds:
Hi Mark,
I agree with your comments about the differences in difficulty of getting feedback in hardware vs. software. To get the same level of feedback with software as is possible with hardware, two things are necessary. First, we have to have feedback on the Product (i.e., the compiled and linked source code; working software), not the Design (i.e., the source code or any higher level representation of the Design). Second, we have to deliver that working software in small increments of functionality so as not to overload the feedback loop by overwhelming either the customer providing feedback or the developers incorporating it.
I also agree with your comments about how Design flaws are handled. I think the reason we're not doing what you suggest lies in the expectations we've created; both for our customers and ourselves. We've created a linguistic trap by breaking down the Design phase of the Product life cycle into subphases that use some of the same words. So when we talk about doing "Design" in the SDLC, we and our listeners have an expection about the predictability of next phase (SWEBOK calls it "Construction") that's based on the wrong thing; that's based on the Product life cycle.
The truth is the entire SDLC is the Design phase, in the Product Life Cycle sense. In the Product Life Cycle, changes that occur during Design represent learning and are good. Changes that occur after Design, in the Construction (i.e., brick laying) or Manufacturing phase represent waste. By using the words we have, by not recognizing that all of software development is a Design phase activity, we've convinced both the customer and ourselves that changes to the source code based on feedback (i.e., Testing) are waste. That's not true. The changes we make to the source code, because the source code is the Design, are learning.
And yes, I most certainly agree with you that the cost of quality is worth it.
Thanks for making time to read the column and for sending your feedback. I appreciate it and hope you'll find time to do the same with the upcoming columns.
Best regards,
-- Bill
Requirements should be inside the production process
I take exception to "(For now, we'll assume that Requirements exist outside the production process)." I read this as "assume that the requirements have already been defined".
This assumption effectively turns the chaos of software development in the real world into the tame, systematic software engineering that is taught at the universities. In this light, all that you have concluded is that software engineering should be managed like any other engineering processes - big surprise.
In traditional engineering applications, the cost of change is such that customers must keep changes to a minimum, hence the existence of change control boards. Using agile OO techniques, the cost of change is much smaller for software. Software development has to hit moving targets, so design cannot proceed assuming that all the requirements have been defined.
Instead, design/development should produce working software that can be used to obtain feedback and help users refine requirements and discover new requirements. Requirements, design, and development should be intertwined in software development rather than considered separate phases.
-- Steven A. Gordon, Ph.D.
Manager, Software Factory
Arizona State University
Author responds:
Hi Steven,
Thank you for making time to read the article and for sending your feedback. I appreciate both.
I understand your reaction to the way I dealt with Requirements. I said "For now..." because that topic is at least a column in-and-of itself. The point I was trying to focus on in this first article is that, if people are going to compare software development to anything else, they should at least to compare apples to apples. Software development is a Design/Development (product life cycle) phase activity that, if it must, should be compared to the same phase in other domains. To give a hint of where I'm going with the Requirements issue, hardware development projects discover and deal with the vast majority of Requirements in the Design/Development phase. The Requirements that are known and fixed at the start of that phase are typically of the "Business Case" variety (e.g., the new PC will be Windows-compliant).
A couple of counterpoints your posting ...