Complexity and why it is at the heart of Agile
It is important to distinguish between two different types of system. One is a complex system, the other is a complicated system. It is unfortunate that the two words are so similar in English. This inevitably leads to confusion.
When discussing systems, these have very distinct meanings as explained in more detail in the Play on the Stacey Matrix.
Complicated
A complicated problem is one which consists of many parts which interrelate in a predictable manner. I find it helpful to visualise this as a clockwork mechanism. Clockwork is the essence of predictability. In a clock the hand moves exactly and predictably once every second. In the difference engine the same inputs will always give the same outputs.
But clockwork systems can be very complicated indeed. Let’s look at the clockwork computing designs of Charles Babbage. His design consists of tiny, intricate cogs and gears. Each spins and interacts with the next. Columns of numbers affect other columns of numbers. It must be incalculably hard to assemble such a masterpiece. Babbage announced his design in 1822 and despite substantial government funding abandoned the project unfinished in 1842.
A complicated system is hard to construct and analyse. It is however predictable.
Complex
What then is a complex system? In a complex system, the key is not just the components but their interaction. The interaction of elements results in an unpredictable behaviour. You cannot use analysis to create a predictable solution to the behaviour. The inter-relation of elements has a significant effect. Therefore deconstructing the system into its elements does not allow you to understand its behaviour.
A system is not the sum of its parts, but the product of their interactions.
Russell Ackoff
Complex is a better description of the environment for software development than complicated. Typically software development has a strong emergent factor where learning comes as part of doing. This seems to be inherent in the value which software development brings.
Classical approaches to managing projects are focussed on tools for complicated environment. They deconstruct the system into small component parts. They then analyse the components. And finally they reconstruct the system on the basis that a complete understanding of the components gives you a complete understanding of the system.
Reductionism
Reductionism is the approach of solving problems by analysis. However, “analysis” means more than just “thinking about a problem”. The term “analysis” comes from two Greek words meaning “breaking apart”. It means taking a problem and breaking it down into component parts. This approach of breaking down a problem to solve it has been used since the times of the ancient Greeks. As a formal approach it is often attributed to thinkers such as Descartes.
Divide each difficulty into as many parts as is feasible and necessary to resolve it.
René Descartes
As it happens, reductionist approaches are very successful for complicated problems. There is a strong linkage between cause and effect. As a result the approach has been central to classical project management. You take a big problem, decompose it into smaller problems, solve those smaller problems and you have solved the big problem. This is a very valid approach when cause and effect are tightly linked.
But what if you have a complex problem? There is less linkage between inputs and outputs and exhibit dynamic and adaptive behaviour. If you deconstruct a complex problem you do not generate trivial tasks which can be performed based on a set of rules. Instead you still have problems which need to be addressed by skilled individuals. In many cases you cannot deconstruct the problem without solving the whole problem. Therefore reductionism isn’t an effective solution.
Agile rejects the idea of reductionism
Does this make agile development impossible? Not at all, although there are challenges inherent in complex environments. But it means that the techniques and approaches which will be effective will be different from classical projects. This is why we need different strategies (or Plays as we might call them) for software development.
An approach for classifying environments to understand which are complex and which are simple or complicated can be seen in the Play on the Stacey Matrix.
If you understand your painting beforehand, you might as well not paint it.
Salvador Dali
Leave a Reply