Knee Jerk Statistics

Binary Logistic RegressionThe folks over at Minitab recently wrote an excellent article about how a Six Sigma practitioner leveraged binary logistic regression analysis in order to better understand why associates at a manufacturing plant were quitting.

What they learned

The team discovered that the distance the employee drove to work seemed to impact how likely they were to quit.

Put another way, there seemed to be strong correlation between the distance of the employees commute and how likely they were to quit.

Sure, people can nitpick sample size and whether there truly is causation… but, really, the analysis seems sound.

Revamp the interview process

As a result of what they learned this team decided a potential countermeasure was to revamp the interview process. Specifically:

The manufacturer’s HR department used the results of Parks’ analysis to revamp the interview process for manufacturing positions. They began to look more closely at the potential employee’s commuting distance and took this into account before making hiring decisions.

Are they missing the mark?

When I first read this I couldn’t help but ponder whether this company was really addressing the root cause of the problem.

Sure, driving 45 minutes to work may stink and could lead to folks wanting to find another job.

But is the best answer to simply not hire them? Should the HR team seek some additional countermeasures?

Obviously, an assembler or machine operator can’t really work from home… but could their hours be shifted to avoid traffic? Perhaps there are other things at work here in addition to the long commute?

What do you think?

This is not an easy situation to assess and I’m sure this team put a lot of thought into this particular problem… but is it possible they’re missing the mark with their “don’t hire based on commute length” countermeasure?

What do you think?

37 Comments

  1. Sean Thompson

    April 29, 2013 - 7:16 am

    It is a hard situation to comment on without knowing more. But if the human resources people are simply not hiring someone based on where the person lives I’d think they could be accused of discrimination even if they are only “discriminating” based on how far the person has to drive.

    • Ron Pereira

      April 29, 2013 - 7:47 am

      Thanks for the comment, Sean. I hadn’t thought of this angle… but, I tend to agree… if the HR department tells someone they’re not hired based on where they live they could be asking for trouble.

    • Darryl

      May 1, 2013 - 12:49 pm

      The reaction of the HR department is an indicator of the type of organization we are talking about. It shows a laziness and a tendancy to move towards the convienient, not necessarily the truth. Using a simple tool like the 5 Why’s may help them in this situation. Are they conducting exit interviews?

  2. Carly Barry

    April 29, 2013 - 9:44 am

    Great post, Ron. Thanks for keeping the discussion going!

    • Ron Pereira

      April 29, 2013 - 10:20 am

      No problem, Carly. Thanks for the comment and starting the conversation! 🙂

  3. Ken Cook

    April 29, 2013 - 11:56 am

    This is a great post with lots of opportunity for learning!

    This smells like a case where the statistics are being substituted for the effort required to gain deeper insight, ie substituting statistical validity for insight into the truth. I would suspect that the people who are quitting that also have a long commute are the proverbial “canaries in the mine.” If someone is incurring the time and financial cost of a long commute, they may have less tolerance for other job dissatisfiers and be the first ones to quit, but not the only ones thinking about it. To find out if this hypothesis is true, and of so what the underlying driver is would require some good old fashioned legwork where the management would actually have to talk to their people. Also, acting on the results of findings from such a qualitiative process would require more bravery, as it may be more difficult to get the numbers to back up the decision.

    • Ron Pereira

      April 29, 2013 - 12:24 pm

      Thanks for the comment, Ken. Yes, actually talking to people – preferably before they plan to leave! – is extremely important.

      So while the stats can definitely help us better understand the problem nothing will ever replace going to gemba, or the place the work is done, and actually engaging with each and every associate.

  4. kicab

    April 29, 2013 - 12:11 pm

    I don’t understand what the variables are and how the measurements were taken on those variables.

    Why don’t you just ask (both those who haven’t left and those who did leave) instead of doing statistical analysis?

    • Ron Pereira

      April 29, 2013 - 12:27 pm

      Hi kicab, thanks for the comment.

      As I mentioned in my reply to Ken talking to people, and really engaging them, is definitely the most important thing that can be done in situations like this.

      But, I think there is also a place for more advanced analysis like this… especially when the data exists as it may help us better understand the current state of the problem.

  5. Kevin

    April 29, 2013 - 12:33 pm

    Not hiring someone based on where they live is not legally discrimination – even here in California, the land of “protected classes.” But the point that others make is more appropriate: there is more here than stats can describe, or describe easily, and talking with potential people and people that have left is important. I loved my 45 minute each way commute for nearly a decade – gave me time to reflect, relax, and decompress. But that’s just me, and others could legitimately go nuts. So it’s important to understand the individual.

    • Ron Pereira

      April 29, 2013 - 2:16 pm

      Thanks, Kevin. Interesting that it isn’t discrimination to not hire based on where someone lives… learn something new everyday!

  6. Isaac Curtis

    April 29, 2013 - 3:50 pm

    While it’s hard to tell from the information given in the article, it seems to me that the commuting problem is more of a secondary issue. Obviously, commuting long distances to work is stressful, and I’m pretty sure that most folks would work from home or a very close facility if they could. However, most manufacturing facilities are not located near the suburbs or cities where many folks live, thus different zoning regulations and long commutes. But, if one really enjoys one’s workplace, then taking a long commute to work can be worth it, especially if you are getting paid more than just gas money, which most folks are not. Obviously, most of us know what it is like to work in a manufacturing facility. It is hard work, and except for a few of the great companies, it can be very soul-sucking. Especially when many assembly/labor folks are consistently ignored by upper management and forced to work in sub-human conditions. It’s a wonder many people show up for work at all, given the stressful commute, low-pay, lack of child care options,lack of respect, and the generally unpleasant nature of manufacturing facilities.

    It’s time for the captains of industry to begin promoting and SPENDING on safe, healthy, aesthetically pleasing, comfortable, respectful, and supportive environments for their products to be made. Until that happens “work” will continue to be a monotonous, stressful, and painful endeavor, that if people had a real choice, probably wouldn’t even walk across the street to.

    It’s time to make a quality loom. It is time to create an environment that our grandma’s and mothers can work at without hurting their bodies, their souls, their precious spirit.

  7. Lino Aguirre

    April 30, 2013 - 11:11 am

    To start i believe this “study-case” is an excellent way to nurture curiosity besides giving us good quality Food for thoguhts,

    In general, statistics are a good Tool, and a good way to start an analysis, but I’m really wondering if this variable that these fellows are analyzing is really coming from a thorough analysis and was found to be the main contributor from a 80/20 pareto analysis… Without a question data is king but if the Problem is not well understood and factors/variables are coming as best guesses or part of the “tribal feeling” we might not be giving a good use of the Lean/six sigma tools.

    Bottomline Ron, in my opinion I would suggest to double-check the Analysis that led to this conclusion and revisit your April’s 15 article about “How to Not Become Handcuffed by Lean & Six Sigma Tools”, cause this might be one of those subtle cases. Do you happen to know if they apply the “5 Why’s” to the people that left the company? that could be a good start if they didn’t.

    thanks and congratulations Ron for your awesome insight and good quality articles!

    • Ron Pereira

      April 30, 2013 - 11:25 am

      Thanks for the comment, Lino! Unfortunately I don’t know any more than the Minitab article shared… perhaps Carly (from Minitab) knows more or could help us explore the topic more deeply?

  8. Rob Li

    April 30, 2013 - 11:16 am

    I totally agree with most of the posts, that a deeper understanding of the reasons for people leaving is required. Given the limited information in this story, however, would it not be wise to lean towards a person whose commute is shorter? That is, all other things being equal, education, experience, cultural fit, etc. commuting distance might be a good tie, breaker.

    An important piece of information that I’m not sure was explored is if the commuting distance of the people leaving used for the analysis was that of their date of hire or their date of departure. A person living 5 minutes away at their time of hire might move 60min away for a variety of reasons sometime during their employment. Unless the probability of such a move is known at the time of hire, its weight as a deciding factor is drastically diminished.

    • Ron Pereira

      April 30, 2013 - 11:26 am

      Thanks for the comment, Rob. Isn’t it amazing how many questions pop up with situations like this?

  9. Sophie Breslin

    April 30, 2013 - 11:23 am

    Motivation? Why do we work? Do we enjoy our work? Why do we enjoy our work?

    I travel to Manchester from Banbury every week because I’m passionate about the work I do. I feel supported, respected and that makes the commute worthwhile. I find mechanisms to support my commuting decsisions as long as I feel my efforts are worthwhile.

    Isn’t this really about the employee recognising their drivers and the employer supplying a supportive, responsive and engaging environment in which to work? To me, the word ‘value(d)’ springs to mind. I agree with Isaac’s comments and am retiring to the bar for a glass of wine on company expenses…..

    Have a great evening 🙂

    • Ron Pereira

      April 30, 2013 - 11:31 am

      No fair, Sophie! It’s only 12:31 PM for me… so no wine for me for at least a few more hours. 🙂

      Enjoy your evening and thanks for the comment! CHEERS!

  10. John Avers

    April 30, 2013 - 1:24 pm

    Talk about knee jerk… how about this article. No data, not even a summary. Not much of a context. Just second guess one of the conclusions.

    Ron: Next time provide a link to the source article.

  11. Carly

    April 30, 2013 - 2:01 pm

    Wow – great discussion happening here! I agree with Ron and Rob: it’s amazing how many thoughts and questions come to mind when you really start to think through the entire situation and all the options and info/data that may or not have been available to this manufacturer when making hiring decisions.

    Unfortunately, I do not have any more information about the analyses performed or any further context than what was provided in my original blog post.

  12. Stephen Gallagher

    April 30, 2013 - 4:02 pm

    Hi Ron,
    There are still some questions to asked to gain more clarity: Is distance a feature regardless of the direction travelled (i.e. is one direction more congested, difficult to travel and more likely to result in employees leaving than others)?; for those who did leave, did they go to job that was closer to their home? etc.
    As to the response HR is taking, I think they are acting somewhat precipitously. They might have a identified a correlation with distance, but distance itself may not be the cause. To immediately settle on a solution is part of being human, a trait we need to constantly fight. I agree that going to gemba is vital to understand the actual causes, and that mutliple potential solutions should be generated before narrowing them down to the ones most likely to be effective.

  13. John A

    April 30, 2013 - 7:58 pm

    I find it hard to comprehend how one variable analysis can provide the needed insight into employee motivation. It’s implying that the only reason people are leaving is distance. Then to make a business decision off of this “conclusion” is merely conjecture. Are there people staying who live a long distance away, is the only happening on a particular shift or department, are they older or younger? As many have remarked, you need to go to gemba, and not fall into the trap of only looking for data that support your conclusion.

    Excellent discussion and not a company I would work for since they are guessing on my behavior before they have a chance to show them my work! Pity!

    • Ron Pereira

      May 1, 2013 - 7:32 am

      Thanks for the comment, John!

  14. Mr Owen Berkeley-Hill

    May 1, 2013 - 4:09 am

    I have an allergy when it comes to data and statistical analysis even though I am (technically) a qualified Black Belt. It goes back over four decades when I first had to calculate the standard deviation of a sample using a slide rule. Minitab is a great piece of software but nothing comes with just positive benefits. The Norwegians (when I lived there) had an expression: “Iron men in wooden ships; wooden men in …”. And this is my concern: is Six Sigma’s obsession with data bordering on a form of OCD, and not just in those with belts?

    Many years ago (someone else may have more details) the CEO of Ford (it could have been Caldwell or Petersen) approached his counterpart at Toyota with a list of all the TPS tools that Ford had applied (though perhaps in no sustained way). He assumed that the list was incomplete and that there was a “secret ingredient” that Toyota was not sharing with the rest of the world. I wonder if that secret ingredient is something that we still have not learned: a radically different attitude to leadership and the people they lead.

    Genchi Genbutsu is a way of life which is supposed to alter leadership thinking radically, but I wonder what percentage of supervisors, managers and leaders practice it in its true spirit? I suspect this population is still very small but I may be wrong. Although I am not suggesting that Genchi Genbutsu is a substitute for an obsession with statistical data analysis, it would, I believe, sensitise the leader to see the organisation systemically and to realise that it, like all of us, suffers from entropic decay over time. Genchi Genbutsu would help see the obvious and do something quickly rather than gathering reams of data and looking for some form of causality. The practice would also (assuming the leader has the basic interpersonal skills) bring him/her into contact with the workforce. This might, if he is not seen as a policeman, help him/her understand their issues.

    This discussion has echoes of another in LinkedIn which questioned whether there was something better than the 5 Whys. I can’t help feeling that much of Six Sigma panders to the conventional way of leadership thinking and therefore does not encourage that leader to change. It is for this reason that I question whether the two approaches, Lean and Six Sigma are compatible. I recognise that this is like lighting the blue touch paper and heading for the hills. Perhaps it is the meat from separate discussion.

    • Ron Pereira

      May 1, 2013 - 7:32 am

      Hi Owen, it’s great to hear from you again and thanks for the comment. As always, I think you offer many excellent insights and lots to ponder.

      And, yes, the lean vs. six sigma debate will likely rage on forever and ever… there is no easy answer but, I think, we must all be careful when we assume “our” approach is correct and “their” approach is flawed.

      Put another way, I think there is goodness and value to be found in all forms of continuous improvement.

  15. Mark Welch

    May 1, 2013 - 7:15 am

    Great comments above, especially Ken Cook. Also, as a former HR guy, I must support Kevin’s comment about one’s place of residence not qualifying as a basis for discrimination. It’s simply not in any laws, state or federal, anywhere. It never has been, which is one reason you see it on every job application.

    In the study described at Minitab they found correlation between TWO variables, and a POSSIBLE LINK to distance from work as a cause. If they wanted to get messier, they could have used multiple regression and analyzed several variables simultaneously, but this would just take them further down the road of a statistical exercise best suited for anal mathemeticians who love statistics rather than getting to the heart of the matter. In all likelihood there are several significant causes, and to determine them there is no substitute for going to the gemba and actually talking to the people. Let them draw their statistics from that.

    • Ron Pereira

      May 1, 2013 - 7:58 am

      Thanks for the comment, Mark. I never knew you were an old HR guy!

  16. Mark Welch

    May 1, 2013 - 8:19 am

    Yes, Ron. An old HR guy by fate, not choice. I had been the continuous improvement guy at a manufacturing company who changed plant managers. The new one was old school and had no taste for C.I., so I went to HR in order to remain employed. In a county of 16,000 good jobs are hard to come by. My wife owns a business in the area so uprooting the family wasn’t a viable option, so I did HR for several years until I could get back into C.I. I learned a lot and it was valuable experience to see another viewpoint. Lean is my true calling, especially in healthcare, and I hope to keep doing this throughout my working life.

  17. sandor

    May 1, 2013 - 10:14 am

    Hi,
    I think the analysis was right but the use the company made of it was way too rough. The question to ask imo would have been why the commute distance is playing such a large role? Obviously this can not be answered by this analysis, but the result is generating a really good question, which is the role of statistical analysis anyhow.

    BTW there is an old and very powerful technique called exit analysis, where you interview people who already resigned or even better, already left the company. This statistical analysis could have helped HR to refine their interview -if they had one.

  18. Bill | Leadership Heart Coaching

    May 1, 2013 - 11:38 am

    I wonder if the manufacturing plant being referenced may be here in the Silicon Valley where I live.

    The cost of living in the Bay Area has always been high, and for many people the dream of owning a house was not possible unless they purchased in outlying communities – often 75 or more miles away.

    When this trend started back in the early 90s with the dot com boom, houses in those remote towns were affordable, gas was affordable, and freeways in and out of the bay area were not as congested. Today, that is not the case.

    I do believe most employees leave managers and not companies, but a commute from a Silicon Valley manufacturing plant to home 1.5 to 2 hours away could certainly be the exception. Especially as those remote towns grow and industry begins to come to them.

  19. Isaac Hernández

    May 1, 2013 - 12:29 pm

    This is a good analysis, or at least a breakthrough on we should face the interviewing process at the workplace. We are facing this situation, therefore the information is useful. I am not a believer on the effectiveness of an exit interview. People would often lie on the true about their exit, just to avoid burning a bridge. After reviewing the comments, we will go ahead and start using this approach to gather better information on why people leave the company. If some one has gone through this process as well, I would appreciate all the insight you can provide me.

    Still, it is a great article.
    Thanks for sharing it Ron!

  20. sandor

    May 1, 2013 - 1:25 pm

    Isaac,
    concerning exit interviews, i believe they are a lot more truthful then interviews you would have with normal employees. If you are worried that people might lie to avoid “burning bridges” after they left the company then there might be no sense whatsoever asking people stillnworking there. In this case commuting distance should be the least of your worries.

  21. Sudhir

    May 1, 2013 - 11:36 pm

    To cite a real scenario at my current workplace : The office is located on the outskirts of the city (in altogether another district to be precise). Now majority of the working class live in the heart of the city and it does take around one and half hours for a one-way commute. Although I agree to the view that one should not discriminate based on location, it is a good idea to ask the candidate (during the entry interview) about his/her comfortability and willingness to endure the ‘long’ commute after joining the organization. This way, the expectations are correctly set at the very begining so that any concerns (down the line) regarding commuting are avoided.

  22. Robert Drescher

    May 2, 2013 - 1:12 pm

    Hi Ron

    I agree if all things are equal commuting distance can be a cause, but their article doesn’t discuss other far more relevant information that also drives people to quit. If these associates are treated like the vast majority of production associates are today (lowest possible wage rate in the area, meaningless work, zero involvement in their job, constant demands to produce, minimal benefits etc.), than having to drive any distance will make a simialr job closer to home look great, and after all they are being rational as they are in fact saving themselves money.

    That being said I know of a great many people that drive well over an hour to get to work everyday, yet they have zero desire to leave, in fact their companies have very low employee turnover period. These businesses all have one thing in common they treat their employees fairly, have them involved, and pay them a real wage. If people are leaving a company for any reason in today’s economy, it generally says you are not providing what anyone really thinks are good jobs (good jobs are hard to come by). Human beings generally are motivated by rational self interest, if you want to know why they leave simple logic can answer the question. Just honestly tell me about how you treat and pay your employees, let me look up what the norms are in your area, and I will tell you what your turnover rate is for employees will look like. And you never need statistical analysis to do it.

  23. Per Uke Bjorn

    June 13, 2013 - 3:06 am

    WRONG WRONG WRONG !

    Ron, this is a great article you’ve mentioned, although not as a best practice but as a brilliant example of how NOT to use statistics (and how to not draw conclusions too quickly from them)…

    On the statistics side, the first and main deviation is not taking into account the global population and only the new comers: how many “14 miles away” employees does the company have in its high seniority members ? Then we can also quote other biases (some comenters already mentioned some of them) like is the distance really representative of commute time (not necessarily)

    And lastly, but definitely not the least, their study is simply the wrong one, as the initial definition of their problem was “increasing labor turn over despite downturn period” and I would find very strange that people would classify unemployment as preferable to 13 miles commute (and I agree this is an opinion, not a fact nor a valid statistic !) so there must be a different (or additional) root cause involved in the process…(i.e. they’ve isolated what looks like a major cause, but failed to apply the five why to find the real root cause)

    Long story short: no, their statistical analysis is not sound (at least not according to the way they describe it). The use of minitab gives a mathematically accurate result but doesn’t garantee a statistically correct interpretation. For those interested in the topic, you can watch the TED talk from Peter Donelly “how stats fool juries” which is a brilliant and funny demonstration of how statistics are too often misused/misinterpreted

    PS: hiring should be based on Qualifications only, so a twelve miles distance filter should be considered as a discrimination (e.g. a biker will drive 14 miles in less time than a car driver would take for only 6 miles of congested streets)