The difficulties of grasping complexity
We humans are almost never satisfied for very long by an accomplishment, be it great or not. As soon as we manage to succeed somewhere, we quickly define new goals that challenge our imagination. Degrees certainly differ from one person to the next, but I am intimately convinced it is in our genes.
Gerald Weinberg has suggested in his writings that we face a natural law that he calls the Square Law of Computation: "Human brain capacity is more or less fixed, but software complexity grows at least as fast as the square of the size of the program".
The first response of companies is often to try and hire the smartest, most talented people they can... and I won't claim it is a mistake, but the magic soon evaporates when the ambition rises again. The law is the law!
They thus strive to find innovative ways to overcome this complexity.
All efforts we make to perfectly understand the present, guess and control the future are doomed, but it is not forbidden to build approximations models... As Weinberg (again) states, "we'll never have complete control, but neither are we victims"
We easily distinguish two families of effort:
- The scientific approaches try to build a complete theory describing a coherent system.
Some examples I think about are:
The much younger software industry confusingly tries to imitate the elder-sister, with a mitigated success: the same recipes are disappointing, as the properties and constraints of software development are not exactly the same! Well, maybe we are not so far to attain our next plateau thanks to a subtle combination of XP and Lean, the second permitting to scientifically measure, control and improve the first. Unfortunately, I do not have enough hindsight and experience to go further, but hope to be able to experiment in my current project.
Special thanks to Regis, for our very interesting discussion on the topic, that inspired this post.
Gerald Weinberg has suggested in his writings that we face a natural law that he calls the Square Law of Computation: "Human brain capacity is more or less fixed, but software complexity grows at least as fast as the square of the size of the program".
The first response of companies is often to try and hire the smartest, most talented people they can... and I won't claim it is a mistake, but the magic soon evaporates when the ambition rises again. The law is the law!
They thus strive to find innovative ways to overcome this complexity.
All efforts we make to perfectly understand the present, guess and control the future are doomed, but it is not forbidden to build approximations models... As Weinberg (again) states, "we'll never have complete control, but neither are we victims"
We easily distinguish two families of effort:
- The scientific approaches try to build a complete theory describing a coherent system.
Some examples I think about are:
- MBTI for human psychology
- Newton gravity, and then Einstein relativity for astrophysics
- Taylorism for car industry
- Cascade for software engineering
- Modularity and low coupling for software design (to divide and conquer)
- XP principles and practices as an approach to software projects
- Lean Thinking for continuous improvement in any construction process
The much younger software industry confusingly tries to imitate the elder-sister, with a mitigated success: the same recipes are disappointing, as the properties and constraints of software development are not exactly the same! Well, maybe we are not so far to attain our next plateau thanks to a subtle combination of XP and Lean, the second permitting to scientifically measure, control and improve the first. Unfortunately, I do not have enough hindsight and experience to go further, but hope to be able to experiment in my current project.
Special thanks to Regis, for our very interesting discussion on the topic, that inspired this post.
posted by Pascal at 10:47 AM
6 Comments:
Very fine essay, Pascal! Tres bien!
I would suggest to your readers that they spend a little time studying the history of other engineering disciplines. They will find a wealth of hints and principles that can be adapted to software. Yes, software is different, but human beings are not.
And so, of course, study people, too. Get your nose out of the code for a bit of time every day. You won't regret it.
Thank you very much for your support and your advices!
We can certainly learn a lot from the experience of other, more mature, engineering disciplines. I would be very grateful if you could recommend us a book or a lecture in particular, accessible to non-specialists.
Studying the dynamics of teamwork, as well as the psychology of the individual is another fascinating subject, that we (with friends of mine) have recently been trying to explore. For instance, we are today thinking hard about a recipe for consistently getting a positive, constructive outcome from difficult conflicts between two persons.
Reading from experienced authors (like you) is an invaluable help, but it does not replace the experience and the daily observations in the workplace.
Pascal,
Here's a few really fine books from which software people can learn a bit more about engineering. I hope they're not too hard to find.
Whyte, R. R., ed. Engineering Progress through Trouble. London: The Institution of Mechanical Engineers, 1975.
Bignell, Victor, Goeff Peters, and Christopher Pym. Catastropic Failures. Milton Keynes, England: The Open University Press, 1977.
Petroski, Henry. To Engineer Is Human. New York: St. Marten's Press, 1985.
(Really, anything by Petroski, but start here.)
Holt, L. T. C. A Short History of Machine Tools. Cambridge, MA: MIT Press, 1965.
Roe, Joseph Wickham. English and American Tool Builders -- the Men Who Created Machine Tools: Lindsay Publications Inc., 1987, reprinted from the 1916 Yale University Press Edition.
I find the machine tool industry the closest analogy we have to the rise of the software industry--tools that build tools.
As for handling conflicts, the classic is
Fisher, Roger, and William Ury. Getting to Yes: Negotiating Agreement without Giving In. New York: Penguin Books, 1981.
(There may be a new edition.)
And, of course, a number of my books, but I don't want to make this an advertisement. People can always visit my website if they want to see that junk pile.
Thank you for all those valuable links, Gerald. My bookshelf already contains many books awaiting for my reading (among which 5 are from you), but the list is about to grow again!
I often wonder how some people manage to read loads of books in a year, without sacrificing the quality. Maybe they (consciously or not) apply some speed reading method...
Pascal, my reading method is intended to reduce complexity. It has two major heuristics:
1. I don't read books that aren't worth reading (I rely on my friends to recommend books).
2. I don't finish reading books that don't live up to their promise.
These two speed up my "reading" enormously.
I already rely a lot on your first heuristic, but I find it clearly not sufficient. I believe I now need more lucidity and courage to apply the second; thanks for the advice!
Here is another great suggestion I recently had from my good friend Régis: when the time comes to decide which book to read next, take your time and think hard about which one on your shelf is most likely to respond to your current profound motivation.
Post a Comment
Links to this post:
Create a Link
<< Home