How Do You Measure Success?

I recently received the following question from Petri, a reader of LSS Academy.

The issue I wanted to contact you about was if LSSA has ever done work on measuring lean success? I am not sure if there are any tools which are used in this area, or if “Lean companies” have i.e. key KPI’s which are used to track this?

I know that there are auditing systems which measure the level of how far 5S is implemented etc, but I am interested in broader measures.

I find the topic of KPI’s with most successful companies fascinating, and was wondering if there is reference material one could learn from?

Your thoughts on this would be appreciated.

Best regards,
Petri
Finland

This is a great question and one I wanted to put some thought into. I am a believer that you must measure it to improve it. With this said, here are some of my personal thoughts on the matter.

The Wrong Measurements

First of all, I believe many companies get caught measuring the wrong things.

For example, some feel like they can measure the success of their continuous improvement initiative by tracking the number of people trained or certified.

Please don’t get me wrong, I think it’s good to measure these things. But to think that the number of people trained or certified equates to guaranteed success is a gigantic fallacy.

In fact, this type of behavior is the definition of an activity based program.

It’s about one thing: Results

red-arrow.jpgThe opposite of activity based programs are results driven programs whereby things like quick wins and bottom line impact rule the day.

But how can one go about measuring this? Is there a magic scorecard we can all download? Unfortunately, at least as far as I know, the answer is no.

With this said, here are some ideas I have for excellent indicators of the health and vitality of a lean or six sigma initiative.

1. Number of improvement initiatives complete (higher is better)

I’d track this monthly and count any documented (even a one page summary) improvement as an initiative.

2. Average time to complete improvement initiatives (lower is better)

I’d also track this monthly. I despise long, drawn out projects that take ages to complete. If an improvement initiative cannot be done is less than 4 months I say it’s over scoped.

Further, the reason I am such an advocate of kaizen events is the speed at which they are done. When done properly, these quick events become contagious and begin to spread throughout an organization like wild fire.

3. Number of improvement suggestions submitted (higher is better)

Tracked monthly, this metric simply allows an organization to measure how motivated the work force is to make improvements.

4. Number of improvement suggestions implemented (higher is better)

The reason most suggestion box initiatives fail is simple. The employees submit ideas and never hear back from management so they stop suggesting things! So, in order to combat this, management should make it clear how many suggestions – as well as what the suggestion was – were implemented. And for the suggestions that were not implemented a clear and honest explanation with the employee should occur.

5. Sales per Employee (higher is better)

In the end, finishing projects and offering suggestions are great… but if you are in a for profit business one thing is clear – cash is king.

We lean and six sigma bloggers can wax poetic about respecting people and all the rest of it… but if an organization is not profitable the doors will be shut… simple as that.

With this said, my favorite productivity metric is sales per employee. Of course some will think I’m advocating cutting heads in order to drive this metric up. I’m not. I am, however, interested in making sure all employees understand whether the business is making money and whether things are trending up or down.

What metrics do you use?

These are just a few ideas I have for how to measure the success of a lean of six sigma initiative. I’m sure I’ve missed a plethora of metrics so if you know of any I missed please share in the comments section below.

18 Comments

  1. Steve Adams

    November 6, 2008 - 8:47 am

    Nice list. I also like to track inventory. We track WIP per employee, turns, and obsolete and slow moving inventory.

  2. Mark Galvan

    November 6, 2008 - 9:58 am

    The suggested measures are not bad as objectives; however, Shingo and Ohno reiterate repeatedly in their writings that the ultimate objective is to decrease costs, while delivering the highest quality of service, the fastest possible way… therefore, costs are not actually reduced until there is a reduction in labor – literally, the reduction in headcount.

    Even if you reduce overall lead time, increase throughput, eliminate defects and eliminate muri and mura, if it still requires the same labor force without an increase in sales, cost did not go down and profitability did not go up.

    So the ultimate measurement is: number of roles eliminated

    That said, Lean’s first message to the workforce is “the goal is not to eliminate people.” So how do we reconcile the two messages. We know that roles are vacated because people are promoted, people latterly move, people quit, people are fired for performance reasons and people are fired for doing stupid things – therefore; there is enough natural attrition to prevent ever having to RIF a worforce as a result of lean initiatives. The new challenge is, can the lean initiatives keep up with the pace of attrition so that no role has to be backfilled?

  3. Jeff Arnold

    November 6, 2008 - 10:06 am

    My boss (CEO) told me last Monday that my role is to save money, so my performance metric (and our measure for successful implementation) is dollars saved per quarter. The target value is my salary.

    I find that I’m now shifting my focus from developing an infrastructure for LSS, to finding projects with quantifiable hard savings. I do this without regard to whether those projects are aligned with our strategic vision.

    We started a Lean implementation last year, and added Six Sigma this year. We are experiencing record growth and profits. I am the lone BB, recently trained. My title is LSS Champion, a new position for both me and my company. I’ve worked here (steel forging) for 30 years, starting at the bottom.

    Should my main focus now be on high dollar initiatives, or should I follow my gut and continue to build a foundation for LSS, including increased kaizens, even if the dollar savings aren’t as easily quantifiable?

    Sorry for the long post. All comments are welcome and appreciated.

  4. Ron Pereira

    November 6, 2008 - 10:31 am

    Hi Steve – Great ideas!

    Hi Mark – If number of roles reduced means creating a flatter organization I am totally fine with this. But if you mean cutting heads for the sake of short term benefits this is completely wrong and definitely not what Ohno, et al taught. In fact, Toyota recently had workers do extra training during slow times instead of letting them go.

    To your point… I agree sales going up should be our goal with costs coming down but even if sales are flat and we’re able to drive costs down (less defects, less inventory, etc.) things would still be better.

    Hi Jeff – This is a tough one. I purposely left out “savings” to see if anyone would mention it. You did. I figured someone would.

    Unfortunately most of us are forced to deal with the fact we must show savings as a result of projects. I’m personally not a huge fan of this since it’s so hard to do accurately.

    With this said, you must do what you must do and my best advice is to build a STRONG relationship with your financial controller… work together and build a model that you both agree to follow. Benefits to operating income, working capital, and growth (sales) are good things to measure.

    For working capital you may agree to take 15% of any savings and count that as hard savings. For example, $100,000 reduction in inventory could be counted as $15,000 hard savings due to a reduction in holding costs, cost to move it around, etc. Or to simplify it just report it to your boss as $100,000 working capital and be done with it.

    Again, the secret is to start now with your controller and not wait until the project(s) are done.

    Great comments all. If anyone else has thoughts on the matter please share.

  5. Jon Miller

    November 6, 2008 - 11:20 am

    Measuring lean success should be the same as measuring business success. Lean is an enabler of market share growth, profit growth and people growth. There is something called a balanced scorecard which is helpful in measuring not only financial performance but also in tracking operational metrics, people / cultural metrics and customer metrics. Without a balanced approach and a focus on cost, you will gain short-term results only and not build an organization capable of continued improvement. This may be the biggest difference between a six sigma focused approach and the lean approach: train a few experts versus train everyone as experts.

    As to Mark’s comments about it being all about headcount reduction, Ohno’s point was that there is a huge difference between apparent productivity improvement and actual reassignment of people. In the original Japanese there was some wordplay between identical words “shojinka” reduce headcount and “shojinka”.

    Ohno focused on that issue so that people would actually reassign people instead of show on-paper improvements try to hold onto the people in their area for just-in-case situations. He was an advocate of cost reduction across a much broader spectrum of cost: materials, energy, logistics, cost of capital, etc. Many manufacturers today have 80% – 90%+ cost in areas other than labor.

    “Costs exist to be reduced.” – Taiichi Ohno

    “Building things starts with building people.” – One of the Toyota founders

    If you’re not doing both, lean fails. Over the long term, so does the organization itself.

  6. Petri Tuomela

    November 7, 2008 - 12:36 am

    I posted the original question due to internal debate we have in our company. It is also an important question when you deal with change resistance, how do you convince that the initiative is yielding results? Soft benefits like no of suggestions made or projects completed are all nice, but does not alone justify the effort and costs which goes in to implementation of any program.

    I agree that at the end if you follow Ohno’s statement that “all we do is we reduce the time line”, the results will follow. You really cannot do this without removing waste.

    Measuring becomes important also as the business climate changes. You do need to show that sales/wip, sales/employee, margins etc are improving. If you clean i.e. price changes in the market from the equation while improve direct margin of product or service, you are doing something right.

    We are currently in process of developing our internal KPI’s for lean purposes, and these comments give good thoughts for this. I would hope to hear more discussion about this topic in these forums. Failure to implement lean is widely discussed, but if you cannot show results, what else can you expect expect but failure?

  7. Mark Graban

    November 7, 2008 - 7:20 pm

    Ron – I’ll respectfully disagree that measuring the # of events is a #1 key measure. That’s at best a secondary type of measure beyond actual business results: safety, quality, on-time delivery, cost and ultimate long-term profits. Too many companies (and hospitals) get wrapped around “# of events” as something to be proud of.

  8. Ron Pereira

    November 7, 2008 - 10:23 pm

    Hi Mark, the numbering wasn’t meant to be in order of importance.

    Funny thing is, I used to think this wasn’t worth tracking until I went to Japan and saw how these ultra lean companies tracked every improvement… and I do mean every. Kevin actually mentioned it as well over on EE.

    Anyhow, we’ve started to do the same and it’s really benefiting us. People like to see what they’ve done. They’re proud of it. And it also makes them want to do more.

    Obviously I can’t speak for healthcare… but it’s working for us (and the Japanese companies we visited).

    By the way… are you feeling confident about your Wildcats tomorrow? It should be fun to watch… even if it is on TIVO. Go Bucks!

  9. Ron Pereira

    November 7, 2008 - 10:26 pm

    The amazing comments left for this post are why I love blogging and never intend to stop. Thanks all for the great thoughts folks. Once again, I’ve learned from you all. Have a great weekend!

  10. Gail Bray

    November 17, 2008 - 8:48 pm

    Hi Ron

    I am new to Lean and work for a public TAFE here in Melbourne, Australia. I am extremely interested in how you measure the success of Lean. I am a project leader in charge of implementing Lean Prinicples. We commenced our journey early this year and I have been given the task of writing an evaluation strategy and evaluating our success to be delivered to our Executive Team.

    I was not sure where to start, thanks for your measures it is a good place to start. Do you have any suggestions on how to present this information effectively? eg. A3 or Graphs

    Another Question, I am in the process of writing an organizational story about Lean. Any suggestions?, it will be used to ‘sell’ the idea to over 1000 employees.

    All the best from Down Under.

  11. Ron Pereira

    November 18, 2008 - 10:02 pm

    Hi Gail, be sure to check out the recent post “Helping our Friend from Down Under” as I attempted to answer your questions! All the best.

  12. Panu Kinnari

    November 19, 2008 - 1:52 am

    I have started tracking value created in our organization per employee. (Turnover – goods and services bought from outside) / headcount. This gives us average amount each employee creates value and that amount needs to cover all of our expenses.

    I use 12 months sliding scale to even out monthly variances and total hours worked divided by 1700 to get headcount. These hours include office personnel as well as management in addition to production.

    Not necessarily lean metric, but something that I find gives good overall picture on our direction.

  13. Owen Berkeley-Hill

    November 23, 2008 - 2:07 pm

    I understand that every change requires three categories of people, the Change Sponsor, the Change Agent, and the Change Target. Lean discussion threads seem to be almost exclusively between Change Agents. I have yet to see a posting from a change target, the value-adding people from the Gemba. So most of these conversations tend to be about how we can change someone else, rarely about how we can change ourselves.

    If my assumption is correct, then we should be a bit careful about which measurements we apply at each stage of the lean journey. As change agents, if we’re trying to change the culture of the people on the Gemba, some of the measures proposed, no matter how well-intentioned, may be counterproductive and may lead to unexpected consequences.

    I am no great fan of suggestion schemes: they are usually badly designed and discourage rather than encourage. When the Japanese were being benchmarked in the 70s and 80s, one measure was the number of suggestions submitted by each employee. The company I work for (which shall remain anonymous) experienced one suggestion (I repeat one) for every 200 employees annually. This was not because we lacked experience, intelligence or imagination. You were not allowed to submit any suggestions about your own job: it was assumed that you would make these suggestions as a matter of course, even though there was no encouragement and no process to assess and implement them. You can imagine the reaction of the department which received your suggestion.

    I’m also no great fan of focusing on hard savings particularly at the start of any Lean journey. “Turkeys don’t vote for Christmas”, because the easiest hard saving is showing people the door. We are still in awe of the black art of accountancy which has proved so reliable that companies like Enron were allowed to flourish. In the early stages of a cultural change, I would recommend any savings, be they hard or soft, should be valued; no matter how small or even if they are difficult to estimate: success breeds success. As experience grows across the work force, and they get better at seeing and tackling waste, the savings will grow. When I was a Black Belt, Green Belts were only allowed to tackle projects if they realised savings of more than $10,000. One Green Belt I had trained came to me in frustration saying her supervisor would not allow her to work on her proposal because it would only save an estimated $7,000! Can you think of a better way to discourage people from getting involved in kaizen?

    Be it Lean or Six Sigma or any combination of the two, the really hard work is to create kaizen momentum and to see that that momentum remains through succeeding generations of managers. Unfortunately, this requires a clear sense of (Strategic) direction, stamina, patience, a sense of humour, and more patience. All too often, success is declared when the first green shoots of a “kaizen spring” are detected. The trainers/coaches/sensei are dispensed with, and the whole façade goes downhill to a place often worse that the starting point. You now also have disgruntled people in the Gemba. Always remember you are fighting a corrosive form of entropy, and that any system will degrade over time even if it does reach “World Class”.

    I have a suspicion (no more) that the Balanced Scorecard was designed to make MBAs comfortable in leadership positions. This allows them to travel back in time to one of their idols, Alfred Sloan, who developed “management by numbers”. It also allows them to avoid visits to the Gemba where their ignorance of how value is added, could be exposed.

    My apologies if this sounds like a rant, but I hope it helps.

  14. Ron Pereira

    November 23, 2008 - 8:48 pm

    Owen, I truly appreciate your insights. Once again you’ve really made me think about this. Thanks for your excellent contributions. All the best my friend.

  15. Anonymous

    April 28, 2009 - 4:37 pm

    Does success of updating the charts on MDT1 or mini-business area determines the success of Lean