Why Good Programming Projects Go Bad

Fred Brooks wrote the software development classic The Mythical Man-Month almost 40 years ago. In this interview, Brooks explains why managers still make the same mistakes.

In 1964, Fred Brooks found himself in charge of one of the biggest software projects ever attempted--the operating system for the IBM 360 series--a new family of general purpose computers which became known as mainframes. Brooks quickly realized that managing a software project was very different from producing hardware. He documented his observations in the software development classic The Mythical Man-Month, in which Brooks famously declared that “adding manpower to a late software project, makes it later.”

Although it was published almost 40 years ago, The Mythical Man-Month is still astonishingly relevant today. Brooks writes elegantly, astutely, and often entertainingly about the joys and woes of the craft of software development and the psychology of those who make it (one of his famous maxims: “All programmers are optimists”).

Brooks went on to found the Computer Science Department at University of North Carolina and receive the Turing award in 1999. In 2010, he published his latest book, The Design of Design: Essays from a Computer Scientist. Now 82, he talked to Co.Labs about project management, designers, and why managers still make the same mistakes.

How did you first become interested in computers?

I grew up in a small farm market town in Eastern North Carolina--Greenville, North Carolina. I was reading, I think it was Time, in the public library and the cover had a drawing of the Harvard Mark I. I had been interested in manual methods of business data processing since I'd been about eight or nine (I started with a card file on my map collection), edge-notched cards, those kind of tools. So when I read about this I was fascinated, and I decided that's what I wanted to do.

The Mythical Man-Month was based on your experience of managing the OS/360 project at IBM. How did you end up heading up that project?

I went to Harvard in Computer Science. (It was called Applied Mathematics in those days.) I did my dissertation under Howard Aiken, who had built the Harvard Mark I back in 1944. Then I joined IBM working on the Stretch supercomputer for four years. After being in charge of architecture for a new product line which did not fly, I was chosen to manage the System 360 computer family hardware. That was 1961. In 1964 it became evident that the hardware was on track and was being released into the factories on time. We were getting good cost estimates and all that, but the software was in big trouble. So my boss and I decided that I should go run the operating system project and see what I could do about bailing it out.

How big a project was the OS/360 for IBM?

I don't know the total cost, but I've heard numbers anywhere from 300 to 500 million 1960s dollars. Those dollars would be today about 10 times more valuable, somewhere in the neighborhood of $3-5 billion. At the peak the project had a thousand people, but the average was much lower than that--we built up. There were laboratories all over the world--Britain, Germany, France, Sweden, California, and New York State.

You have said that “Everybody quotes it (The Mythical Man-Month), some people read it, and a few people go by it.” Why is that?

It's all about disciplined decisions that are hard from a manager’s point of view. Just look at the software mess over the rollout of the health law. They made almost every mistake in the book. The book has more than 500,000 copies in print and is used in most software engineering curricula, so many people have learnt from those mistakes of the past. But it's evident that if one picks people who are not software engineers to run major software engineering projects, you wouldn't expect them to have studied the subject. Therefore they make the same mistakes again. The biggest mistake with the health law rollout was that there was no one in charge. That's the biggest of all mistakes. That project seems to have had neither architect nor project manager. How bad can you get?

Why is it so important to have both a project manager and an architect on a software project?

I think it's important even with a small team to distinguish the functions of the project manager from the function of the architect. When I was teaching software engineering, which I did 22 times here (The Department of Computer Science at the University of North Carolina, which Brooks founded), I always made even teams of four people choose a separate project manager from the architect, who was responsible for the technical content. The project manager is responsible for schedule, resources, and such. I notice the same division of function occurs in the film industry. A film has a producer who is in charge, but the person whose name is last on the credits is the director. The producer is responsible for making it happen, and the director works for the producer, but the director is responsible for the artistic content. I think that the same separation of function which evolved in that industry applies as well in ours.

The project manager first has to be tough, second place has to be flexible. A motto I consider important is “Never uncertain; always open.” I saw that in Latin (Numquam incertus; semper apertus) on the ceiling of a German fraternity in Heidelberg. It's important to always have a direction and be going there. You can't steer a ship that's not underway. But it's also important to be open to changes in circumstance and direction and not just to be completely bullheaded. A project manager also has to be a people person. Project management is a people function and most of the problems are people problems.

Almost 40 years after the publication of The Mythical Man-Month, why is it still so difficult to estimate how long it will take to make a large piece of software?

The problem is that we are working with ever-new technology on ever-new development. Product development always contains many uncertainties. In building a house we are working with known technologies and pretty well-known specifications. Building a whole new thing like building the first nuclear submarine or building the first space shuttle, you don't know what you're going to run into. Any piece of software is a whole new thing, unless you are just copying somebody else's.

In The Design of Design you say “I believe ‘a science of design’ to be an impossible and indeed misleading goal.” Why?

The difference between science and engineering is not so much what scientists and engineers do as why they do it. The scientist may spend a lot of effort building an apparatus for an experiment, but he builds in order to learn. The engineer may spend a lot of time studying materials or previous designs or users, but he learns in order to build. I think that distinction is crucial. So I believe the National Science Foundation's program hunting for a science of design, or indeed Simon’s original Sciences of the Artificial book, was chasing a thing which isn't--a chimera--because building is different from learning the laws of nature. One can systematize physics; one doesn't necessarily systematize the process by which physicists work. There's not necessarily a science of science in other words, so to expect a science of design, which is really a quite different undertaking, I think is misguided.

You also say that “standard corporate standard design processes do indeed work against great and innovative design” but that process can also improve average performance. How does the designer know when to apply process?

That's the judgment that distinguishes good designers from poor ones, I think. There's no way to tell a person how to do that. It’s possible to train people to do that by putting them through many projects, mentored and advised so that they acquire the feel, the taste, and the judgment. There's an old saying “good judgment comes from experience and experience comes from bad judgment.” I think that applies to the training of designers.

How should designers be trained?

Supervised, mentored design experience in a variety of projects. A rotation between different aspects of the design and construction process. The designer stands between the user and the builder. He needs to understand the user and that means spending time with users doing what users do. He needs to understand what building's about, and that means spending time building stuff.

Most big companies have a pathway for training managers with job rotations, maybe with formal academic training (Brooks dispatched one of his own reports, Edgar Codd, who later invented the relational database, back to grad school mid-career), and then working up multiple levels. In IBM the most promising of the young managers were made aide-de-camps to the chief executives. The chief financial officer, the CEO, the chief operating officer had lieutenants who were considered very promising and might one day inherit those jobs. This mentoring process I think is just as important for designers as it is for managers. The managers are the ones who decide who gets trained. They are aware of the need to train new managers but they don't have any reason to understand or believe intuitively in the need for training the designers.

What did IBM’s CEO teach you about making things?

When I was leaving IBM to come to North Carolina, Mr. Watson (IBM's CEO Thomas Watson Jr.) asked me down to the executive dining room for a one-on-one lunch. You know what that's about--that's an arm-twisting session. He proposed that I stay at IBM. I didn't want to do that. I said, “I really like to make things, and I want to get back closer to the technology”, because I was a third-level manager by this point. He said something very enlightening and very memorable. He said “I like to make things too. Have you looked at Poughkeepsie (IBM’s biggest computer manufacturing complex) recently?” And then I realized that this 10,000-person plant and laboratory complex was his creation. He had persuaded his daddy to get the company into the computer business. I never would have thought of that as making things. He raised my vision a whole lot in one sentence.

[Image: Flickr user Steven Depolo]