What and When to Automate?

Texas_1928This post springs out of a Twitter conversation with Marc Burgauer and Kim B. They will also be sharing their thoughts on what and when to automate (here and here, respectively).

My simple answer is that automation is most valuable when it can provide rapid feedback into the decisions people make.

When the question came up, I immediately thought about my experiences developing software, and the automation of testing cycles. I have developed an ingrained assumption that some types of automated testing are inherently “good.” It was fortunate that Kim was so pointed in her questioning. I was forced to revisit my assumptions and come at the question in another way in order to respond with a considered answer.

I believe the development of the US Navy’s early surface fire control systems is a useful illustration of effective automation. These systems were intended to allow a moving ship to fire its guns accurately and hit another ship at ranges of five to ten miles or more. At the time these systems were developed—between 1905 and 1918—these were significant distances; hitting a moving target at these ranges was not easy.

The core of these systems was a representative model of the movements of the target. At first, this model was developed manually. Large rangefinders observed the target and estimated its range. Other instruments tracked the target and recorded its bearing. These two—bearing and range—if observed and recorded over time, could be combined to develop a plot of the target’s movements. At first, the US Navy’s preferred approach was to plot the movements of the firing ship and the target separately. This produced a bird’s eye plot which could be used to predict the future location of the target, where the guns would have to be aimed to secure a hit.

Feedback was incorporated into the system to allow it to be successful. At first there was only a single feedback loop. A “spotter” high in the masts of the ship watched the movement of the target and observed the splashes of the shells that missed. To make this process easier, the Navy preferred “salvo fire,” which meant firing all available guns in a battery at once, maximizing the number of splashes. Depending on where these shells landed, the spotter would call for corrections. These corrections would be fed back into the model, in order to improve it.

The process did not work well. Building the model manually required numerous observations and took a lot of time. A different approach was adopted which involved measuring rates of change—particularly the rate at which range was changing—and aiming the guns based on that. This was less desirable, as it was not a comprehensive “model” of the target’s movements. However, automatic devices  could be used to roughly predict future ranges once the current rate of change was known, allowing the future position of the target to be predicted more rapidly.

These “Range Clocks” were a simple form of automation. They took two inputs—the current range and the rate at which it was changing—and gave an output based on simple timing. They reduced workload, but did not provide feedback. They also did not account for situations where the rate at which the range was changing was also changing. Automation would have better been focused on something else, and ultimately it was.

The early fire control systems reached maturity when the model of the target’s movements was automated. The Navy introduced its first system of this type in 1916. Called a “Rangekeeper” this device was a mechanical computer that used the same basic observations of the target (range and bearing, along with estimates of course and speed) to develop a model of its movements.

The great advantage of this approach over previous systems was that the model embedded in the Rangekeeper allowed for the introduction of another level of feedback into the system. The face of the device provided a representation of the target. This representation graphically displayed the computed target heading and speed. Overlaid above this representation were two lines that indicated observed target bearing and observed target range.

If the model computed by the Rangekeeper was accurate, the two lines indicating observed bearing and range would meet above the representation of the target course and speed. This meant that if the model was not accurate—due to a change of course by the target or bad inputs—the operator could recognize it and make the necessary corrections. This made for faster and more accurate refinements of the model. Automation in this case led to faster feedback, better decisions, and ultimately more accurate gunfire.

When we think about automating in software, I believe it is better to concentrate on this type of automation—the kind that can lead to more rapid feedback and better decision-making. Automation of unit tests can allow this, by telling us immediately when a build is broken and many teams use them exactly this way.

When we approach the problem this way, we’re not just providing an automated mechanism for a time-consuming repetitive task. There is some value to that—this is the approach the Navy took with the range clock—but it is more valuable if our automation enable better decisions through faster feedback. Decisions are difficult; often there is less information than we would like. The more we can leverage automation to improve decision-making, the better off we will be. This is the approach the Navy took with the Rangekeeper, and I think it’s a valuable lesson for us today.

Making Sense of “The Good, the Bad, and the Ugly”

I recently attended a Cynefin and Sense-Making Workshop given by Dave Snowden and Michael Cheveldave of Cognitive Edge. It was an excellent course and a useful introduction to how to apply concepts from complex adaptive systems, biology, and anthropology to better understand human approaches to problem solving.

The Cynefin framework is an elegant expression of these ideas. It posits five domains that reflect the three types of systems we encounter in the world. There are ordered systems, in which outcomes are predictable and repeatable. There are chaotic systems, which are inherently unpredictable and temporary; and there are complex systems, in which the system and the actors within it interact to shape an unpredictable future.

We can use the Cynefin framework to help us make sense of our current situation and understand what course of action might be best at a given moment. If we are dealing with an ordered system, then we are in one of the ordered domains, either “Obvious” or “Complicated.” In either of these circumstances, we can reason our way to the right answer, provided we have the necessary experience and expertise. The predictability of the system permits this.

If, however, we are in the “Chaotic” domain, the system is wholly unpredictable. The “Complex” domain embraces complex adaptive systems: those that are governed by some level of constraint yet remain unpredictable. Think of the foot traffic in your local shopping mall, and you can get some idea of how these systems manifest: you can purposefully walk from one end to the other, but if the mall is crowded, you can’t predict the course you’ll have to take to get there.

A fifth domain, “Disorder,” exists to explain those times where our current state is unknown.

To increase our familiarity with how to use the Cynefin framework, we performed a number of exercises. In one of them, my tablemates (including Adam Yuret and Marc Burgauer) and I tried to make sense of the final, climactic scene of “The Good, the Bad, and the Ugly.” Spoilers follow, so if you haven’t seen it, now’s a good time to bail out.

The scene involves a three-way standoff between “Blondie” (Clint Eastwood), “Angel” (Lee Van Cleef), and “Tuco” (Eli Wallach). The three gunslingers stand in a rough triangle at the center of a graveyard. Blondie’s written the location of the treasure on the bottom of a rock, and placed it at the center of the triangle. None of them wants to share the treasure.

At first blush, it seems to be an ideal example of a complex system. As soon as any one of them acts, the others will fire, and the standoff will end, but no one can predict how. That’s why each of them stands there, eyeing one another cautiously, as the tension builds to Ennio Morricone’s music.

But that’s not the truth of the matter. Blondie is no fool. He’d gotten the drop on Tuco and had time to unload Tuco’s weapon. As we watch the scene, we don’t know this, but for Blondie, the situation is well-ordered. All he needs to do is pick the right time to gun Angel down. Blondie knows Tuco’s not a threat.

The other two must deal with more unknowns. It’s not a chaotic system for them. There is a certain level of predictability. Someone will shoot. But the details of who that will be—and when he will fire—are uncertain. What happens after that is anyone’s guess. Both Tuco and Angel want to trigger a specific outcome—their survival and the death of the other two—but exactly how to manage this outcome is impossible to predict given the other elements of the system. It’s a perfect example of a complex adaptive system.

We thought this was an extremely useful example to help us “make sense” of Cynefin and the concepts it embraces. I hope you do too.