Q&A: Software's advance is so steady, you probably don't even notice it

Computerworld - William Scherlis is a professor of computer science at Carnegie Mellon University and director of the Institute for Software Research there. He specializes in software assurance, software evolution and technology to support software teams. He has a long association with NASA and the U.S. Department of Defense. Scherlis spoke with Computerworld's Gary Anthes about progress in software development.

The performance of hardware -- be it microprocessors, storage systems or networks -- has increased exponentially over the years. Why has progress in software been so slow? Sometimes people think we are at a plateau with software. But I'd like to refute that. It's making enormous strides, and on a pretty steady basis. It's been 50 years since John Backus, the inventor of Fortran, wrote his seminal paper on "automatic programming" to describe the translation from Fortran to machine code. Why "automatic programming"? Because at the time, Fortran seemed so highly descriptive and problem-oriented. Now the old Fortran seems very low-level and mechanistic. This same cycle has happened in the world of information systems and databases -- remember the "automatic programming" promise of [fourth-generation programming languages]?

William Scherlis

Fortran and 4GLs were enormous advancements, but with these kinds of advances, our ambitions correspondingly increase. So the commoditization and standardization never completely take over -- the market drives us to create new value, and so we're improving tools, languages and processes at the top end just as quickly as we "routinize" and automate at the low end. The magic of software is that, because there are no limits of physics, we can keep advancing the technology to meet our ambitions. I call this the "endless value spiral."

What are some more recent advances? Object-oriented programming was a similar leap forward, and that is manifest in C++ and Java and C#. Object- oriented programming has allowed us to do things that previously we couldn't do, and one of the most important is building software frameworks -- application servers, e-commerce frameworks like J2EE and .Net, and the ERP frameworks like SAP's NetWeaver. More recently, there has been a parallel development in an area called functional programming and a language at Microsoft called F#. They have built that into their .Net Framework.

There are also giant-scale programming models for high performance. Google and Yahoo both use the "MapReduce" model to handle parallel processing across multiterabyte data sets, rather than SQL, for example.

What's so important about those things for the future? The framework technologies allow the very clean separation of infrastructure provisioning from infrastructure usage. To a bank CIO, that means that it's possible to outsource the infrastructure and insource just exactly that software development that differentiates the bank from some other bank. In the bad old days, you had to roll your own for the infrastructure as well.