Complexity is our enemy

Multi-million IT project failures are legend, and often the root cause is the most obvious: complexity.

The bigger the undertaking, the more complex it is.  And the more complex it is, the harder it is to make it work as intended.  Should you be able to get a complex system off the ground, the greater the likelihood that it will break often and then the sheer complexity will vastly complicate the job of addressing the inevitable problems.

Studies have shown that systems under a million dollars have a better than 75% chance of success, while the success rate for systems in the $2 million to $3 million range drops to 40% to 50%.

"So the question is not, are we failing?  We know we're failing," says Roger Sessions, CTO of ObjectWatch and an expert in software architecture.  "The question is, why aren't we doing something about it?"

Sessions is doing something about it.  He just received a patent on a way to reduce the complexity of large systems called the Simple Iterative Partitions (SIP) methodology.

It isn't, after all, just a matter of breaking large projects into smaller components.  "Unfortunately, as you minimize the complexity of the system by breaking it down, you increase the complexity of those system interdependencies," he says. "So you're in a no-win situation."

SIP uses a mathematical technique called equivalence relation analysis to find the best possible balance between "making the system small and minimizing functionality related complexity, and keeping the system large and minimizing dependency related complexity," Sessions says.

The result is an optimized number of subprojects that have a far greater chance of success, he says. 

But wait, won't the arrival of cloud computing help simplify the world, or does that just add another layer of complexity? 

"In my opinion, it adds another layer of complexity," he says. "I like the idea of cloud architectures, not for everything, but for many things.  But I think it is a serious mistake to put a large, complex system on the cloud.  That's going to be very difficult to do, very difficult to manage.  And if you're successful, you'll be trapped on that particular vendor. So what you want to do is break these systems up into smaller pieces," and then evaluate the deployment options.  "But you do that after you've removed as much of the complexity as you can by splitting up the project into smaller projects."

This is interesting stuff.  Read a full Q&A with Sessions here.

Read more about infrastructure management in Network World's Infrastructure Management section.

This story, "Complexity is our enemy" was originally published by Network World.

Copyright © 2010 IDG Communications, Inc.

Bing’s AI chatbot came to work for me. I had to fire it.
Shop Tech Products at Amazon