Better Living through Algorithms

I’ve been reflecting on an article from June of this year. It’s about Daniel Kahneman’s efforts to help organizations to become better at decision-making. He is a Nobel Prize winning psychologist and behavioral economist who retired from teaching to enter the consulting world. In the article he says he “expected to be awed” by the quality of decision-making in highly competitive organizations. He was not.

“You look at large organizations that are supposed to be optimal, rational. And the amount of folly in the way these places are run, the stupid procedures that they have, the really, really poor thinking you see all around you, is actually fairly troubling.”

Tell us what you really think about your clients, Dr. Kahneman. He added that these organizations do have significant room for improvement. That sounds familiar.

Why Smart People Make Dumb Decisions

People routinely make decisions that are lower quality that their capability, intelligence or experience would suggest. Kahneman and other behavioral economists identify overconfidence, limited attention, cognitive biases and other psychological factors that cause poor decision making. We jump to conclusions, misunderstand situations and make errors in judgment. These biases are not bugs of our species. They are features. While we can be aware of and try to minimize them, many are by nature unconscious and difficult to avoid.

In the article Kahneman reveals that experienced professionals may have a larger well of experience to draw on, but the quality of their decisions are just as variable as that of novices. In addition, even in group settings, decision-making often suffers from social pressures to conform. If people can be trusted individually or in groups to consistently make good decisions, what’s one to do? Rather than continuous improvement targeted at our decision-making, Kahneman suggests recognizing cases where this is not a core competency of humans. We should outsource these decisions to algorithms.

What is an Algorithm?

Algorithms are mathematical concepts. Humanity has employed them for nearly five millennia. In mathematics and computer science, an algorithm is a process or set of rules to follow in calculations or other problem-solving operations. Algorithms are finite sequences of well-defined instructions. When we put data into an algorithm, we always get clear and consistent output from its series of formulas. We can think of an algorithm as a recipe.

Better Living through Applied Mathematics

As consumers, much of the news we hear lately about algorithm is not so positive. We hear that big tech collects vast amounts of data on our personal behaviors. They employ AI, machine learning and other tools. Big tech monetizes this by showing us what we want, or are likely to buy, perhaps even before we know we want it. They are using algorithms to make better decisions about how to market and sell to consumers.

In business and in my personal life, I am eager to make better decision. Even when I believe that I follow good decision-making processes, human nature can undermine me. There are some decisions that humans can’t always be trusted to make at our highest level of thinking. We can set decision-making parameters and automate certain processes. The article makes me think that in the near future, decision-making may be more of a dance. Sometimes the homo sapiens leads and sometimes we let algorithm take the lead.

Algorithms within Lean Thinking

There are various algorithms lurking within lean thinking. The use of flow charts, value steam maps or other process mapping tools resemble algorithms. We input data on the current state, apply a set of rules and calculations, and generate an improved future state.

The fourth of the six practices of Agile kanban is to make process policies explicit. This is an act of creating algorithms, or linked sets of rules. Agile kanban teams decide to how to handle decisions  around WIP limits, when to pull work past various commitment points, how to handle blockers etc. They write these down, follow and improve them.

The DNA of TPS as an Algorithm

Stephen Spear’s description of lean management, from his paper Decoding the DNA of the Toyota Production System, describes an algorithm. In his definition, the essence of the Toyota method is to specify content, timing, sequence and outcome for all work. In addition, people perform work as experiments against a hypothesis. This allows people to find flaws in process rules and to revise them based on changing conditions. We could say that the DNA of TPS is the creation and improvement algorithms for achieving business goals.

The Two Algorithms within Toyota Kata

Toyota kata is a set of practice patterns for developing our scientific thinking skills while solving practical problems. The steps of the improvement kata and the coaching kata are presented as sets of questions. The kata approach is an algorithm for dealing with uncertainty and the experimenting through the unknown. It is not an algorithm for implementing known solutions or proven practices. Its main aim is to develop the thinking and skills to solve problems. As a result, the practice of kata embeds algorithms within human brains in the form of habits and thinking patterns.

Separating Human Work from Machine Work

One of the core tenets of lean management comes to us from the jidoka pillar. It requires that we separate human work from machine work. We shouldn’t make people do work using methods or tools that are substandard or unsafe. We shouldn’t ask people to babysit or stand and watch machines running on auto cycle. Nor should we expect lines of robots to do work that requires thinking, human judgment and adaptability.

We must design processes by using machines when they do the work safer, better or faster. Of course, this must be paired with the idea that we don’t over-design machines and process automation. Lean machines are additive, low-cost, and general purpose. The design is inside-out from fixtures and tooling that do just enough to create the value-added transformation.

The use of algorithms and AI for decision-making is a logical extension of the separation of human work from machine work.

Reducing Decision Variability by 80% – 90%

How do company leaders respond when they learn that they should implement algorithms to guide their decisions? “Not very well,” according to Kahneman. But they change their mind when they learn that there is 50% variability between human experts making a decision and only 5% or 10% when it’s an algorithm.

We don’t hesitate to make investments in proven, labor-saving machines. As more of us learn that we can automate decision-making to make it more reliable, perhaps we will explore this with an open mind. Where can we replace ad hoc and fallible human judgment with algorithms? How can people build better decision-making instruction sets? How can we use the time and energy freed from this to taken on even bigger challenges?

An Industrial Revolution for Decision-Making

We revolutionized our industrial manufacturing a century and a half ago through steam-powered machines, electricity, and progressively intelligent automation. Today we don’t hesitate to make investments in proven, labor-saving machines and information technologies. Yet we attempt to manage these processes and make decisions about how to invest, run and improve them using craftsman-like methods. We rely on bias-riddled, protein-based human judgement.

Perhaps it’s time for the industrialization of management decision-making. We can identify classes of decisions that are burdensome or variable for humans and build algorithms for them. This allows humans to make decisions appropriate for us. Human ingenuity and invention will be necessary in novel situations. Human judgment brings empathy and understanding into the equation. Let the machines do their work, humans ours.

---