Empirical Evidence for the Power of Kata

We’ve long known that embracing failure with an aim toward learning and improvement can create future success.  Organizations that see failure in that light, and support their people with cultures and systems to learn from failure, can become engines of innovation.

Now there’s quantitative evidence via a tested empirical model to support our thoughts on the potential power of failure.  In their article in last month’s issue of Nature, Quantifying the Dynamics of Failure Across Science, Startups, and Security, Yian Yin, Yang Wang, James Evans, and Dashun Wang tackle the subject.

The large-scale datasets they used are rather unique: grant applications to the National Institutes of Health, entrepreneurs trying to obtain funding, and terrorist organizations trying to execute mass casualty events.  Radically different mindsets, perspectives, and activities.  In each situation the authors looked a whether the organizations and individuals were successful, controlled for underlying demographic data, and compared past failure to both the efficiency and quality of future success.

Two key findings from the study were that people who experience early failures often become more successful that those who achieve early success, and that there is a tipping point between failure and eventual success that is driven both by learning and the pace of subsequent failures.

In an interview with Harvard Business Review, Dashun Wang discusses this tipping point between eventual success and ongoing failure, which he calls stagnation.  This is a transcript of an interview, so you’ll have to pardon the imprecise grammar.

As people fail over and over these two groups of people [those that eventually succeed and those that continue to fail] can be quite similar in terms of their learning strategies or their characteristics.

But depending on which side of the tipping point they are in, they could  experience fundamentally different outcomes. One group of people, if you’re below the tipping point and may be failing over and over, you try to get up and try again, but over and over you’re not learning enough to actually achieve success. You’re trying over and over, but you’ll fail to learn enough to initiate an intelligent pattern with the improvement.

On the other side of the tipping point, which is above this threshold, people fail over and over but they fail faster and faster to eventually approach success. So this creates a quite a surprising prediction because what it means then is first of all, not all failures lead to success.

If you take these two groups of people, what you will find is that in the beginning, in their first failure, they essentially have the same kind of a performance and the same kind of characteristics. But as they fail over and over, they start to become two very distinct groups of people because they’re in the two sides of the tipping point.

The authors then created an empirical model and tested it against the three radically different datasets described above.

I think not only I feel like our results support that fail fast mantra that’s very popular in Silicon Valley, but it also actually doubled down on its importance. For example, the way I see it is our results actually now suggest this idea of failing fast is not just prescriptive but also diagnostic.

Kata is a methodical approach to improvement that ensures learning is derived from intentional experimentation and failure.  Without failure there would be no learning, without which there would be no next step to move closer to an improvement goal.  A critical component of kata is having an ultimate objective, and then iterating through defined experiments to break through obstacles and obtain new knowledge.  Kata also prescribes experimentation cycles as short as possible, which improves the pace of failure – and learning.

Kata, in effect, helps you be on the success side of the tipping point discovered by Dashun Wang and his colleagues.

---