What is Evolutionary Design?
Evolutionary Design is the practice of growing a system in a natural way, by adding the minimum amount of code to satisfy the business needs in an iterative and incremental approach. When done right, the code structure changes continuously to optimize for change, thus allowing a constant speed of development for longer periods of time.
In a dynamic world we need to adapt our tools and techniques to be resilient to change. Evolutionary Design is one of these techniques that helps us grow a system well by understanding its current characteristics.
When the software industry was born in the 60s it was easy to make a plan for a project. The requirements were clear, well documented and not a lot could change. The development speed was slow and steady, one could predict the delivery time. But after a while when software started to spread around, the complexity of the project grew more and more. It wasn’t possible anymore to know all the requirements and we started having specializations on areas (testing, programming, compiling, etc), but also on system areas (storage, back end, etc). Waterfall could have worked until the systems grew too complex and nobody could have an overview on the whole thing. With waterfall we create a fixed plan and then we implement it, knowing that everything works well on each stage.
But when complexity was too much, the software industry invented the incremental and iterative approach. We build a part of the system with a concrete deployable result. This is one way of reducing complexity. But then the problem is that we need to envision the future additions to the existing system, so we need to build the current part of the system with the future developments in mind. This is thinking about evolution, this is the historical beginning of Evolutionary Design.
Evolutionary Design Tools
After many years of working on more and more complex software systems, the people involved started inventing and using techniques to evolve the system in an evolutionary way. Unit Testing, Test Driven Development, Design Patterns, Continuous Integration, Domain Driven Design are just a few of these techniques that help deal with complexity in an evolutionary way.
Unit Testing is a technique inspired from manufacturing, where each of the smallest units (screws, bolts, etc) to bigger units (iron cast parts, chassis, etc) needs to be tested independently. The reason we test each unit independently is because we don’t want to assemble the whole module and then observe a defect, disassemble it and change the defective part. Unit Testing is a technique of identifying defects up-front with the purpose of minimizing waste.
Test Driven Development (TDD) added on top of Unit Testing the cycle Red -> Green -> Refactor. Because code is easier to mold than metal or concrete, we can refactor it with small cost. So with TDD we introduce already the concept of evolution. We evolve the code with the Red -> Green -> Refactor cycle while using the unit tests to ensure no defects are added to the existing units, that were already tested. Refactoring means changing the existing structure of the code without changing its behavior. Unit Testing is our safety net when refactoring the code with TDD. So TDD is a tool for Evolutionary Design. You can watch here a series of code casts on TDD as if you Meant It. But in order to use TDD right, one needs to understand patterns of growth.
Design Patterns appeared naturally after observing patterns that exist when growing many software systems. They are documented observations of good practices. Using Design Patterns with TDD would help us grow a system to be easier to change and less expensive to maintain.
Patterns of Growth
During the lifecycle of developing a software system, I observed that we take the following steps: normalize growth, optimize growth, maximize growth.
Normalizing Growth is the process of observing irregular developments in a system, and refactoring it so it grows nice and balanced. Imagine you are growing an apple tree and you want it to be productive. Then you need to cut some internal branches, trim some of the rest of the branches so the tree has enough light and air to produce many juicy apples. Now imagine you are growing a software system and you want it to work well for the users. You need to set-up practices so that the unit tests are written in the same way, the data layer is consistent, the UI has an MVC approach, etc. So by applying all these practices you normalize the growth of the system. We setup standards and observe their usage. Every once in a while the standards are not respected by the team and we need to “trim the branch”, or we need to add some mentorship and guidance to the process so we come back to the efficient growth. You can read more about Normalizing Growth here.
Optimizing Growth means we choose a growth outcome. You cannot have an apple tree that grows huge branches and has tons of apple trees. You need to choose: do you want big branches and shadow, or smaller branches and apples. Optimization means compromising, choosing your way. The same way a software system cannot be very easy to change and with great performance. Often you need a balance between the two.
An example of such an approach of optimizing the growth of the system is Selection Pressure, and you can watch a code cast in part 1 and part 2. Selection Pressure means choosing your business metrics even before writing your first line of code and wrapping all the code around your business metrics.
Maximizing Growth means that we want to make the most out of each optimization. Our apple tree might need some fertilizer to maximize the growth of the fruits. But we need to know when to apply the fertilizer, how to apply it and in which quantities. Often the blooming period is critical: we need to give the apple tree enough nutrients and water to grow spectacular fruits.
With our software system we need to understand when that moment arises. If we want to maximize growth when we haven’t even normalized it yet it would be equivalent to pouring out nutrients to a tree’s roots during the winter and hope it will grow faster. Every system has its time when we can maximize the growth, and here we can observe patterns of evolution like Low Coupling, High Cohesion, Testing Strategy, and many others.
Being able to observe the patterns of growth in a software system is essential when using Evolutionary Design techniques. One example of such patterns are the Transformation Priority Premise wrote by Robert Martin.
Evolutionary Design is about using all these tools, patterns, practices, habits and knowledge to be able to create a system that is adapted to the current need, but envisions the future developments. We don’t develop according to a plan, like in waterfall, but we develop according to a goal: adding features to serve the users and the client.