It's not the coding that's hard, it's the people

Computerworld - The data about software development is sobering. Many projects end up over budget and behind schedule, and one study puts the failure rate at one in five.

The path to better projects may be for software developers to become better people. A recent conference on agile development had "new age"-sounding sessions such as "An Introduction to Non-Violent Communication for Agile Coaches," "Fear Driven Impediments" and "Collaborating with Non-Collaborators."

The stakes involved in getting software projects right can be huge. Development projects costing millions of dollars can become such a mess that businesses turn to people such as Billie Blair, an organizational psychologist and president and CEO of consulting firm Change Strategists Inc., to untangle the disorganization.

In nearly every case, Blair said the source of all project dysfunction is the project manager.

"Anything that goes awry in a company," said Blair, "can always be traced back to the manager."

Often enough, the project manager isn't ready for the job. Engineers, IT professionals and highly skilled technical people can be appointed project managers under the mistaken belief that if they have good technical skills, "then it follows that they will be a good manager," said Blair. Businesses and organizations don't always recognize that management is a skill in itself, she said.

Project managers have to deal well with people, embrace conflicts and not run from them, know how to assist people in sorting things out, and be compelling, said Blair. Engineers and IT professionals "are wonderful at what they do and their skill set, but generally those managing skills are not there and they have to acquire them," she said.

The people part of building software appears to be getting much more attention thanks to agile software development, which emphasizes communications, collaboration, rapid production of code and frequent feedback. It's a less rigid approach that requires a more flexible person at all levels, and not just at the manager level.

That is in contrast to some traditional development methods, waterfall in particular, which calls for getting all the requirements upfront and then moving through the development process one phase after another.

With traditional methods, "it's only at the end that you discover that you have a real problem," said Todd Little, a senior development manager for Landmark Graphics Corp., which makes applications for the oil and gas industry.

With agile development, you are constantly delivering working software at each stage of the process "that's available for discovery," and "each time you are closing in on reducing the overall uncertainty of the project."

To help accomplish that, Little focuses on the workplace culture, emphasizing highly collaborative teams.

When dealing with project problems, "very rarely are they technical challenges; almost always, the challenges are with people," said Little, who was chairman of the Agile 2011 conference in August, where the sessions mentioned above were held.

The Standish Group surveys software development for a study it releases every two years. Its most recent survey, released this year, collected data on 10,000 projects.

It found that 37% of the software development projects in its study were successful, meaning that they came in on time and on budget and that the users accepted them. It categorized 42% of the projects as challenges. These were projects that had problems such as being late, being over budget, or not having everything users wanted. The remaining 21% were failures, meaning they weren't completed or were rejected by the customer.

But the success rate of projects improved by 5% from two years ago, said Jim Johnson, the chairman of Standish. That may be due to more agile projects, as well as the types of projects undertaken during the recession, including fewer ERP installations.