This post originally appeared on Excella Consulting’s Blog.

This is an exciting time to be involved in the Agile community. A number of new models are emerging that allow us to understand our work better and frame our challenges more effectively. One of the most robust is the theory of Complex Adaptive Systems (CAS). A CAS is a collection of unique elements, like people and their tools, that interact in multiple ways, changing, learning, and adapting over time. CAS have several important characteristics:

• Greater than the sum of their parts: a CAS is created by the interaction of its component pieces. It cannot be decomposed and must be considered as a whole.
• Sensitive to time: the specific sequence of events within a CAS is an extremely important factor in shaping its future.
• Sensitive to context: existing patterns and relationships are very influential; generalization is dangerous when working with a CAS.
• Emergent futures: the future of a CAS is uncertain and unpredictable; new patterns emerge from the interaction of component elements.[1]

CAS theory has opened up new avenues for understanding in biology and the physical sciences, and is now being applied profitably to knowledge work.[2]

Complexity and Software Development

Software development is a form of knowledge work. We focus on non-routine problem solving, creative thinking, and the acquisition of new knowledge. That new knowledge is what allows software solutions to solve business problems in creative ways. Without it, we would be stuck automating existing routines, rather than developing new, innovative solutions that transform our clients’ ways of working.

CAS theory offers us a new perspective for managing this knowledge work. In a complex system, control—or influence—is exercised through constraints. Constraints limit the behavior of the individual actors in the system. I like to use examples from the natural world. In a forest, the limits of hot and cold temperatures in the summer and winter, the patterns of rainfall, and the features of the terrain would all be constraints. Along with many others, these constraints influence the animals and plants that dwell in the forest, helping to define how they live and interact. Similar patterns exist in a software team. In a team, the constraints might include a two-week sprint cycle, defined office hours, and the physical boundaries of the office. All these strongly influence how the team interacts and works together.

Triggering New and Innovative Ideas

That’s straightforward enough, but the magic of complex systems is that they are capable of spontaneously introducing new and innovative characteristics through the interaction of individuals within the boundaries of these constraints. In biological systems, this is what triggers evolutionary change and the introduction of new species. The classic example of this is the peppered moth; as the industrial revolution overtook England, soot covered the trees. The soot was a new constraint. Moths with darker coloration were better camouflaged in this environment. Birds didn’t eat them as often, so more of them survived to breed. Eventually, the whole population of moths became darker colored. This was an evolutionary change. When this process couples with an initial genetic mutation, new species can result; the different species of finches in the Galapagos are a well-known example.

The same dynamics can occur within a development team. New constraints can encourage different behavior, leading to evolutionary improvement in the work of the team. Shortening the length of sprints, for example, from three weeks to two will foster different behavior. Stories will become smaller, opportunities for feedback will occur more often, and the team may feel like it is moving more quickly. These differences may trigger fundamentally different ways of working within the team.

Finding the Balance

The challenge in doing this is that constraints have a sweet spot. They are more likely to trigger innovative approaches when they focus the team, but do not overly limit them. If the constraints are too loose, there will be too many degrees of freedom. Team members will lack focus and feel no need to harness new, creative ideas. However, in an overly constrained system, there will be insufficient flexibility and no freedom to introduce innovative concepts. In between these two extremes, constraints will channel the creative energies of team members and focus them on creative solutions.

We see this dynamic at work on effective agile teams. When the team members have a clear sense of their objective, and they are focused on delivering a small, discrete piece of value in the near term, innovation results. The constraints—the sprint goal, the two-week iteration, and a small set of valuable user stories—focus the team and energize them. Their creativity comes to the fore and shows in their work.

Complexity gives us a model that explains why agile works. It also helps us understand why some agile teams struggle. If a team hasn’t found the right balance of constraints—if they aren’t in the sweet spot between the two extremes—they won’t foster the creative energies of their team members and they will find it difficult to come up with new and innovative ideas.

Complexity suggests that the best way to enable the success of agile teams is to keep a close eye on the constraints governing their behavior and to deliberately manipulate them to encourage focus and foster creativity. One reason the Scrum framework has been so widely adopted is because it is an effective vehicle for this. Well-designed Kanban systems that employ visualization and work-in-progress limits are also extremely effective. In the next post in this series, I’ll look in more detail at these two approaches, contrast their use of constraints, and offer more insights for how use constraints in knowledge work.

Read the next post in this series here.

^{[1] For a detailed examination of these ideas, see Embracing Complexity by Jean G. Boulton, Peter M. Allen, and Cliff Bowman.}

^{[2] For a great introduction, see the HBR article, “A Leader’s Framework for Decision Making” by David J. Snowden and Mary E. Boone.}