Extreme Programming inventor talks about agile development

InfoWorld - Recognized as the inventor of Extreme Programming (XP) and a co-author of the "Manifesto for Agile Software Development," Kent Beck has been a prominent advocate of agile programming and its use of shorter release cycles in software development projects. The founder and director of Three Rivers Institute, which offers various programming services, Beck attended the QCon conference in San Francisco this week, where he answered some questions from InfoWorld editor at large Paul Krill.

Could you talk about the Agile Manifesto and XP?
The fundamental assumption behind XP is that there are certain activities that contribute to software development success. I asked the question, "What happens if we did those as intensely as possible?" Hence the name "Extreme." And it turns out that if you take some practices like technical collaboration [and] testing and push them harder than they had been pushed up to that time, at least in typical development, that there's all these nice consequences. You get very low defect rates, which means you can release software much more frequently. You get a very low cost to getting projects into an initially deployable state, low cost and short time compared to other styles of development. And if you take this notion of incremental design, continually investing in design seriously, you can continue deploying new functionality at a fairly steady rate for a very long time.
That was how XP started, and it turns out that in order to accomplish all those goals, you also have to, as a team and as individuals, learn a bunch of new social skills: integrity, transparency, accountability. [You] get to a certain speed, and the next thing, if you want to go [to] the next step faster, what you've got to do is be able to communicate much more clearly and transparently about what's going on.

You mentioned trends that are driving agile programming: reliability, low cost of change, increased return on investment. Why is the market moving away from the waterfall style to agile development? Or is still just a small portion of developers that are using that method?
Well, it is a small portion of developers that are doing agile, but I think it's growing quite quickly. I don't have numbers to back that up, but that's the sense that I get if you look at the growth of conferences and so on. I think what's driving agile development right now is that it's possible to be much more honest, transparent and accountable if you have short cycles and you decide that that's what you want to accomplish. There's a huge latent market for software development that's just flat-out honest.

Why so?
Because it hadn't been that way for a long, long time.

You mentioned this morning that there's the problem in which you have to make sure your software works. You say that's something that you should be able to take for granted, but that has not been so. Why has that been the case?
Well, I think it's a combination of technical and social factors that leads to all the defects in deployed software. Part of it is the attitude that software is just inherently unreliable, and customers are conditioned to accept that. Developers are conditioned to accept that. Testers are conditioned to accept that. We just decided it was like the weather and there's nothing we could do about it, which isn't a very responsible position because in fact, there's a lot that software developers can do about it. Both technically, [through] test-driven development, automated integration testing, continuous integration and socially both in terms of personal pride of workmanship and integrity, not wanting to send out software that has defects. And in terms of relationships of teams.

There have been these studies that you're familiar with that talk about the fact that most software projects are failing. Is agile really a remedy for that, or is the jury still out?
Is agile a remedy for failing software projects? I think a lot of software projects should fail, and the problem is you just don't know which ones until you're pretty well into it. So is agile a remedy for it? No. I think software projects are still going to fail because there still [will be] the promising ideas that don't work out in practice. One thing that agile development can give you is to make sure those projects fail faster, sooner, cheaper, so you can get on with the next thing.