Complexity in Action: Operating in a Complex World

This post originally appeared on Excella Consulting’s Blog.

In the previous posts in this series, I introduced Complex Adaptive Systems (CAS) and described some basic approaches for using complexity to improve the work of Agile teams. In this post, I’ll delve a bit deeper and explain how complexity can help trigger the mindset shift necessary to create an effective environment for Agile teams.

But first, it’s important to revisit the main characteristics of CAS:

  • Greater than the sum of their parts.
  • Sensitive to time.
  • Sensitive to context.
  • Emergent futures.[1]

Each of these has a bearing on the work of Agile teams.

Greater than the Sum of the Parts

Anyone who has been part of a good team has experienced this dynamic. Good teams become more than a collection of individuals. They create an environment that enhances the effectiveness of team members. They learn how to work together, to communicate seamlessly, and to enhance each other’s skills. Good teams become entities unto themselves; they are complex systems.

Although Agile emphasizes the need to develop cohesive, self-organizing teams, the importance of this is often underestimated. There is a lengthy history in modern management of treating individuals as “resources” and assuming that they can be moved from team to team without significant reductions in effectiveness. CAS theory illustrates how this idea rests on a false assumption. Teams cannot be reduced to their component pieces, taken apart, and reassembled. They need to be considered a whole with unique characteristics that are more than the sum of their components.

To be successful with Agile, we need to foster the development of good teams, create environments that allow them to flourish, and let them become effective at working together. Once they achieve that state, treat them as a team, not a collection of individuals. This means shifting the focus of management. Managers become responsible for creating the necessary environment; they shift from managing individuals to managing teams.

Sensitive to Time and Context

This work is aided by becoming familiar with how sensitive teams are to historical events and their current context. Just like CAS in general, Agile teams are a product of their pasts and current environmental influences. This has several important implications.

It is impossible to “rewind” the clock and return to a previous state in a complex system. Historical events and their impact become embedded in a team; they constrain future potential. Focus efforts like retrospectives on broadening everyone’s perspective and validating the diversity of views within the team; there will be no single “correct” view of the past. Acknowledging diverse perspectives will provide a solid foundation for future improvements and ensure they are informed by the experiences of the entire team.

Context exerts a similar influence. I have worked with dozens of Agile teams around the world, from Europe to Asia and North America. Each of those teams had a slightly different context—different team members, different products, different ways of working—and they all approached Agile slightly differently. To be most effective, I had to adjust my approach, and my message, to allow my efforts to resonate with them.

An effective environment for Agile teams needs to recognize this variability. Just as we cannot expect all individuals on a team to react and behave the same way, we cannot manage all teams the same way. What’s needed is a clear set of expectations—well-defined constraints—for how teams will be evaluated and managed. If those constraints are in the “sweet spot” then the teams will have the freedom to improve, grow, and self-organize, and the leaders of the organization will have enough visibility and control to develop the confidence they need. The key is to remember that the “sweet spot” will vary from one organization to another.

Emergent Futures

One of the most exciting aspects of Agile approaches is waiting until the “last responsible moment” to take action. It’s exciting because identifying the right moment can be a challenge, and because it allows us to defer commitment of energy and time until we have the latest (and best) information.

The future is unpredictable. We all know this. But in CAS, we assume the future will emerge, that unanticipated new ideas and opportunities will arise. By waiting until the “last responsible moment” we increase the probability that we will be able to take advantage of these when they do.

The challenge, of course, is that emergent futures are not predictable, and most organizations crave predictability as a means to reducing risk. From the perspective of CAS, this is quickly exposed as a paradox. If we invest in predictability to reduce risk, we inhibit our ability to defer commitment and seize emergent opportunities. The alternative is to reduce risk by minimizing unknowns. If we can go from starting work on a user story to delivering the associated functionality in two weeks, we keep risk very, very small. Agility offers new ways to reduce risk.

To make Agile work effectively, especially in an environment with multiple teams, a mindset shift is required. CAS theory helps us understand the nature of that shift and provides a new window into understanding why Agile approaches are so effective. In these four posts, I’ve barely scratched the surface. If you’d like to learn more about how to apply complexity in your organization, please contact me or one of the other coaches here at Excella.

[1] For a detailed examination of these ideas, see Embracing Complexity by Jean G. Boulton, Peter M. Allen, and Cliff Bowman.

What are Heuristics and How Can They Accelerate Decision-Making?

This post originally appeared on Excella Consulting’s Blog.

In the earlier posts in this series, I introduced Complex Adaptive Systems (CAS), and contrasted Scrum and Kanban from a CAS perspective. I also briefly touched on the idea of heuristics; in this post, I’ll expand on these ideas and explain why heuristics are advantageous for decision-making in complex systems like software development teams.

A heuristic is a guide that aids decision-making. It is not a set rule, but it is a rule of thumb. I use a simple set of heuristics at my local grocery store to accelerate my checkout process. If I have a basket of items, I will use the self-checkout. If I have a cart, I’ll look for the checkout line that has the least number of people in it. If two lines have the same number of people, I’ll choose the one with the fewest items. These guidelines are simple heuristics and we all use techniques like these, even if we’re not always aware we’re doing so.

Heuristics are powerful because they accelerate the process of making decisions. We don’t have to spend time assessing all the potential options, performing a cost-benefit analysis, and ranking alternatives. Instead, we employ a specific set of guidelines to quickly arrive at what we believe to be the best option.

Heuristics in Complex Systems

Why is this important and how does it relate to complex systems? In an earlier post, I noted how CAS are extremely sensitive to context. The specific ways in which the individuals in the system interact and the constraints that act upon them vary; the nature of these interactions also change over time, altering behavior within the system and its potential for change. This makes complex systems dynamic. To operate within them effectively, flexibility is needed.

Heuristics offer the necessary flexibility. Because heuristics are not set rules, they can be quickly tailored to specific contexts. I can adjust my grocery store heuristics based on circumstances. If I’ve got a bottle of wine in my basket, for example, I might opt for a checkout line over the self-checkout because I know that a cashier will be able to check my age and validate I’m over 21 faster than the attendant overseeing the self-checkout machines. If children are with me, I may look for the line with no candy, regardless of its length. Because I’m using heuristics, I remain sensitive to context and can tailor my decisions quickly and easily.

That’s the power of heuristics. They inform decisions without substituting for them. Heuristics enable human judgment by rapidly framing the problem and allowing our minds to focus on the specifics of the situation. This leads to quick decisions that factor in valuable contextual information without overly complicated and unnecessary rules.

Heuristics in Agile

What’s this got to do with Agile software development? Agile software teams make decisions every day. How can those teams ensure that these decisions happen rapidly and align with their overall goals? One way is to make sure they’re made collaboratively. If teams regularly talk about their work and the impediments they’re facing (like in the daily standup), then their collective knowledge and experience will be brought to bear. However, many decisions are made outside of those collaborative sessions. How can we ensure they’re made rapidly and reflect the team’s objectives?

Heuristics are the answer. By explicitly developing heuristics to guide their decisions, teams can make decisions more rapidly, ensure the outcomes align with their objectives, and leave room for individual judgment and creativity. An excellent heuristic at work in software development is the “rule of three,” a simple heuristic that suggests code can be copied twice, but should be refactored into a new procedure the third time it is used. Another common heuristic builds on the Agile principle emphasizing face-to-face conversation: whenever you have a question, ask it face-to-face; if that’s impossible, use a video conference; and if that’s not available, use the phone.

Try Them Out!

Explicitly defining heuristics can accelerate the decision-making in Agile teams. Because they are sensitive to context, heuristics can adapt to a variety of different circumstances; they won’t age and become obsolete like “agreements” or “rules.” Teams employing heuristics can make decisions more rapidly and be more confident that day-to-day decision-making aligns with their goals. By allowing team members to utilize their knowledge and skill, heuristics foster greater engagement and morale. They fit perfectly into a view of software development as a complex adaptive system.

Try defining some heuristics and see how they can improve the effectiveness of your team! Next in this series, I’ll write more about how you can improve the work of Agile teams by employing ideas from complexity.

 

Effective Use of Constraints: Scrum vs. Kanban

This post originally appeared on Excella Consulting’s Blog.

In the first post in this series, I introduced the concept of Complex Adaptive Systems (CAS) and how constraints can trigger innovative ideas. In this post, I’ll investigate constraints in more detail and contrast how Kanban and Scrum use constraints to enable successful software teams.

Both Scrum and Kanban use constraints to limit work in progress (WIP), foster decentralized decision making, and focus the team on near-term deliverables. However, the two approaches have significant differences.

Constraints in Scrum and Kanban

Purpose of Constraint Scrum Kanban
Limiting WIP Sprint Backlog WIP Limits
Near-term Deadline Sprint Review SLAs for Individual Work Items
Focused objective Sprint Goal Work Visualized on a Kanban Board
Prioritization Buffer Sprint Backlog “Ready” Queue
Roadmap Release and Product Backlog Product and Product Backlog

Scrum: Timeboxes and Ceremonies

In Scrum, the most important constraint is the timeboxed iteration, or Sprint, and the regular cadence of ceremonies. The Sprint limits WIP through the creation of a Sprint Backlog. This backlog is usually estimated in story points and sized based on past performance. If the team has a consistent velocity (e.g., 30 points per Sprint), capacity planning starts with that number and adjusts it based on circumstances.

The Sprint Planning ceremony ensures agreement on the content of the Sprint Backlog and its size. During the iteration, the backlog serves as a prioritization buffer. Outside of that buffer, the Product Owner can make as many reprioritization decisions as necessary, but the Sprint Backlog should only be adjusted in rare circumstances, so that the team can focus on delivering that Sprint’s Goal.

The Sprint Goal is a clear expression of value, the focus for that Sprint. Daily standup meetings allow the team to self-organize and determine how to progress towards the goal. The Sprint Review gives them—and their stakeholders—feedback on whether that goal was met, and informs the objective of the next iteration.

Kanban: WIP Limits and Visualization

In Kanban, limited WIP and visualization are the main constraints. WIP can be limited in many different ways, but the most common is to cap the amount of work in any given state. A maximum of five stories might be permitted “In Progress,” or a maximum of three in “Test.” The WIP constraint in Kanban focuses the team on flow, and not on time.

Attention is focused on the work by use of a visual Kanban board. On the board, everyone can see what is being worked and what state it is in. This triggers self-organization; the board fosters a collaborative understanding of what is most important. Aligned and decentralized decision making flows naturally from this shared understanding. Scrum teams often leverage visualizations like this as well. Scrum does not explicitly call for them; Kanban does.

Rather than up-front estimates of size and complexity, Kanban uses Service Level Agreements (SLAs) based on historical cycle time. These become a constraint that encourages focus. Note that the SLA is not an estimate; it is a prediction based on past performance. SLAs operate on each and every individual item in a Kanban system, allowing a continual flow; work is not batched up into timeboxed iterations as in Scrum.

Kanban also uses a prioritization buffer, called a Ready Queue. This short backlog contains items that are ready to be worked on, but are not yet in progress. Teams and their Product Owners are free to alter the prioritization of items in the Ready Queue at any time, giving them flexibility to rapidly adjust based on new information.

Although there are no specific ceremonies in Kanban, it is customary to employ daily standups, regular grooming sessions, and demonstrations of new functionality. These can happen on a cadence, or they can be triggered by the flow of work. For example, a team might hold a grooming session once they have only two items in the Ready Queue; they might demonstrate new functionality after five items have been completed. An exception is the Sprint Planning event; Kanban teams have no need for them because there is no iteration.

Finding the Sweet Spot

Is one of these approaches superior to the other? Not in all circumstances; remember, CAS are very sensitive to context. However, if we examine the constraints employed by Scrum and Kanban, we gain some important insights. The ability of a CAS to support the creation of new and innovative ideas is called Emergent Potential. Greater Emergent Potential means a higher likelihood that creative approaches will be discovered; Emergent Potential is maximized in that “sweet spot” between constraints that are too restrictive and too loose.

Scrum is very portable and easily adopted because it provides a well-defined structure. The structure is intended to rest in that “sweet spot.” If it isn’t a perfect fit, teams should Inspect and Adapt, tailoring their specific Scrum implementation in light of experience to better fit their needs. However, this can be very difficult.

In situations where the business is unwilling or unable to leave the team focused for the duration of an iteration, the Scrum process can become a hindrance. If a team is responsible for production support, for example, it may be impossible to work effectively in Sprints. Support cannot always be nicely packaged up into iteration-sized chunks.

Mature Agile teams may also have difficulty with Scrum. The iteration boundaries and associated ceremonies can provide less value as the team becomes more effective at working together. In those situations, more flexibility may be needed to allow a team to maximize its potential.

In both of these situations Kanban may be a better fit because the constraints in a Kanban system are more sensitive to context. This gives the team greater flexibility and the promise of increased Emergent Potential.

Recommended Heuristics

If you’re uncertain about whether to employ Scrum or Kanban, consider these factors. Scrum, because it is a well-defined, portable framework, will work well for teams that are newly forming or just learning Agile. Scrum is easy to adopt and can quickly deliver value. More mature teams, who are already familiar with Agile, or who have deeply embedded ways of working, tend to be better off with Kanban. The increased contextual sensitivity of Kanban will offer those teams greater potential.

These guidelines are relatively simple and straightforward, but I’ve found them to be a nice set of heuristics that work well with Agile teams. Learn more about what heuristics are and how to apply them in the next post in this series.

What is Complexity and Why is it Useful?

This post originally appeared on Excella Consulting’s Blog.

This is an exciting time to be involved in the Agile community. A number of new models are emerging that allow us to understand our work better and frame our challenges more effectively. One of the most robust is the theory of Complex Adaptive Systems (CAS). A CAS is a collection of unique elements, like people and their tools, that interact in multiple ways, changing, learning, and adapting over time. CAS have several important characteristics:

  • Greater than the sum of their parts: a CAS is created by the interaction of its component pieces. It cannot be decomposed and must be considered as a whole.
  • Sensitive to time: the specific sequence of events within a CAS is an extremely important factor in shaping its future.
  • Sensitive to context: existing patterns and relationships are very influential; generalization is dangerous when working with a CAS.
  • Emergent futures: the future of a CAS is uncertain and unpredictable; new patterns emerge from the interaction of component elements.[1]

CAS theory has opened up new avenues for understanding in biology and the physical sciences, and is now being applied profitably to knowledge work.[2]

Complexity and Software Development

Software development is a form of knowledge work. We focus on non-routine problem solving, creative thinking, and the acquisition of new knowledge. That new knowledge is what allows software solutions to solve business problems in creative ways. Without it, we would be stuck automating existing routines, rather than developing new, innovative solutions that transform our clients’ ways of working.

CAS theory offers us a new perspective for managing this knowledge work. In a complex system, control—or influence—is exercised through constraints. Constraints limit the behavior of the individual actors in the system. I like to use examples from the natural world. In a forest, the limits of hot and cold temperatures in the summer and winter, the patterns of rainfall, and the features of the terrain would all be constraints. Along with many others, these constraints influence the animals and plants that dwell in the forest, helping to define how they live and interact. Similar patterns exist in a software team. In a team, the constraints might include a two-week sprint cycle, defined office hours, and the physical boundaries of the office. All these strongly influence how the team interacts and works together.

Triggering New and Innovative Ideas

That’s straightforward enough, but the magic of complex systems is that they are capable of spontaneously introducing new and innovative characteristics through the interaction of individuals within the boundaries of these constraints. In biological systems, this is what triggers evolutionary change and the introduction of new species. The classic example of this is the peppered moth; as the industrial revolution overtook England, soot covered the trees. The soot was a new constraint. Moths with darker coloration were better camouflaged in this environment. Birds didn’t eat them as often, so more of them survived to breed. Eventually, the whole population of moths became darker colored. This was an evolutionary change. When this process couples with an initial genetic mutation, new species can result; the different species of finches in the Galapagos are a well-known example.

The same dynamics can occur within a development team. New constraints can encourage different behavior, leading to evolutionary improvement in the work of the team. Shortening the length of sprints, for example, from three weeks to two will foster different behavior. Stories will become smaller, opportunities for feedback will occur more often, and the team may feel like it is moving more quickly. These differences may trigger fundamentally different ways of working within the team.

Finding the Balance

The challenge in doing this is that constraints have a sweet spot. They are more likely to trigger innovative approaches when they focus the team, but do not overly limit them. If the constraints are too loose, there will be too many degrees of freedom. Team members will lack focus and feel no need to harness new, creative ideas. However, in an overly constrained system, there will be insufficient flexibility and no freedom to introduce innovative concepts. In between these two extremes, constraints will channel the creative energies of team members and focus them on creative solutions.

We see this dynamic at work on effective agile teams. When the team members have a clear sense of their objective, and they are focused on delivering a small, discrete piece of value in the near term, innovation results. The constraints—the sprint goal, the two-week iteration, and a small set of valuable user stories—focus the team and energize them. Their creativity comes to the fore and shows in their work.

Complexity gives us a model that explains why agile works. It also helps us understand why some agile teams struggle. If a team hasn’t found the right balance of constraints—if they aren’t in the sweet spot between the two extremes—they won’t foster the creative energies of their team members and they will find it difficult to come up with new and innovative ideas.

Complexity suggests that the best way to enable the success of agile teams is to keep a close eye on the constraints governing their behavior and to deliberately manipulate them to encourage focus and foster creativity. One reason the Scrum framework has been so widely adopted is because it is an effective vehicle for this. Well-designed Kanban systems that employ visualization and work-in-progress limits are also extremely effective. In the next post in this series, I’ll look in more detail at these two approaches, contrast their use of constraints, and offer more insights for how use constraints in knowledge work.

Read the next post in this series here.

[1] For a detailed examination of these ideas, see Embracing Complexity by Jean G. Boulton, Peter M. Allen, and Cliff Bowman.

[2] For a great introduction, see the HBR article, “A Leader’s Framework for Decision Making” by David J. Snowden and Mary E. Boone.