Oh Snap! We Sent Bad Product to Our Customer: Fun with the Hypergeometric Distribution

If you enjoy statistics this is the perfect article for you! But even if hearing the words hypergeometric distribution makes you want to yack all over the floor at least give this article a quick skim as it could prove useful for you someday… especially if you ship a product or service to a customer!

To help me with this article I’m going to make up a completely fabricated story.

Let’s say a producer of widgets (replace widgets with the product your company produces if you wish) recently shipped 1,000 units to their customer.

Let’s also say that, prior to shipping these units, the producer randomly inspected 100 widgets.

After inspection, the producer ships the 1,000 widgets to their customer, collects payment, and moves on to the next order.

The Angry Customer

A few weeks later the producer gets a phone call from their customer – who is less than pleased to say the least – explaining that they’ve discovered 4 defective widgets and they hadn’t even looked at all the widgets!

The customer goes on to say that they are sending all 1,000 widgets back and they are giving the producer 10 days to make things right.

Once the widgets arrive back in the plant a team quickly inspects all 1,000 widgets. Once they are complete they realize that 36 of the widgets were actually defective!

Luckily, the root cause of the defective units was quickly identified and an immediate countermeasure was put into place ensuring this problem would never happen again.

The next question that was asked was how this order ever shipped. It was hypothesized that since 100 widgets were 100% inspected at least 1 of the 100 widgets should have been found to defective which would have forced a more thorough inspection of the entire lot.

Not sure how to solve this problem the team asked their local Six Sigma Black Belt to help them determine the probability of catching at least 1 defective product while inspecting 100 units out of a lot of 1,000.

After learning more about the situation the black belt decides to use the Hypergeometric Distribution, which Wikipedia so eloquently defines as a discrete probability distribution that describes the number of successes in a sequence of n draws from a finite population without replacement.

Here is how the black belt set things up in Minitab, which is the statistical software package they use.

In this example we are dealing with a finite population of 1,000 widgets so the “Population size” is 1,000.

Since we know, in this example, 36 widgets were defective and being defective is the “Event” in this case, the “Event count in the population” is 36.

Since the company sampled 100 widgets before shipping the lot, the “Sample size” is 100.

And since, for the Hypergeometric Distribution, X is the number of defective widgets we found in the sample of 100 widgets… X can range from 0 (we found no defective widgets) up to 36 (we found all defective widgets).

Now then, since the Black Belt wants to know the probability of finding at least 1 defective widget, they want to find the probability that X is greater than 0.

So, by using an “Input constant” of 0, they learn that the probability of having no defective widgets in this lot of 1,000 widgets is ~ 2% as shown above.

In other words, there is a 98% chance that at least 1 of the 36 defective widgets should have been detected during the inspection process.

With this information, the team came to the conclusion that their inspection process was broken. There were three theories that needed to be investigated.

So, as is the case with many things related to continuous improvement, answering one question has created several more that need to be addressed.

What do you think of this analysis? Would you have done it another way? What about the conclusions of the team… can you think of another theory of why these widgets were sent to the customer?

Evan Durant
August 3, 2009 - 10:50 am
Reply

This analysis assumes the sample was randomly drawn from the population. I would look at the sampling technique to ensure is wasn’t a cluster, as this would impact the likelihood of finding a defect.
Ron Pereira
August 3, 2009 - 10:54 am
Reply

Great point, Evan. Let’s assume, in this fictitious example, that the samples were randomly drawn. I mentioned that every 10th item was inspected but, to your point, this is not exactly random! So, I may tweak my example accordingly! 😉
Aaron Emerson
August 3, 2009 - 6:47 pm
Reply

Great job explaining something I’m pretty sure my college stats prof would have taken 4 weeks of painful and boring lecturing to explain. Or at least he would have attempted to explain it!
Kathleen
August 4, 2009 - 6:27 am
Reply

Statistics and sampling is definitely an area of weakness for me. To that end, I’m reading _The Drunkard’s Walk: How Randomness Rules Our Lives_. Altho given passages are a bit heavy for me, it’s quite an interesting read explaining how errors such as this occur and how they can be avoided. I suggest the book could be a layman’s introduction for those who are as math challenged as I am.
Ron Jacques
August 4, 2009 - 6:28 am
Reply

My immediate thoughts were similar to Evan’s. Why are we inspecting “randomly” after the fact. Why is there not a sampling plan with control charts at the point of mfg. Maybe the machine went out of control on the last 36 parts and they somehow ended up on the bottom of the random sample and were clustered as suggested. There is no substitute for controlling quality at the source. Inspecting afterward leaves things to chance. 36 defects in 1000 is a level of 36,000 DPMO. Not a very controlled process. Sounds like Poke Yoke needs to be put in place at the point of mfg.
William
August 4, 2009 - 6:29 am
Reply

Forgive me if I am missing something obvious (it’s still early!), but why would X have a range of 0-24 and not 0-36?
Ron Pereira
August 4, 2009 - 8:18 am
Reply

William, you are 100% right! I had initially used 24 but changed it to 36… I’ve since corrected the article. Thanks for catching my defect! It seems my measurement system needs some work as well! 😉
Tom
August 4, 2009 - 9:16 am
Reply

Man, I like it when I learn something useful!
Ed Kemmerling
August 4, 2009 - 9:32 am
Reply

What do I think? I think that this is a perfect example of using the wrong tool at the wrong time to fix a problem.

The team took a lot of time to do a detailed analysis. After their analysis, their conclusion was that the inspection process was broken. Of course it was broken, otherwise we would never have shipped the 36 defective parts.

As a Quality Manager, I experienced problems very similar to this one. The first step is to always protect the customer, and to review our internal inspection process. If it should have found the defects, what went wrong? If it was not capable of finding the defects, what needs to be changed?

Then NEXT step after implementing a more effective inspection process would be to do a detailed hypergeometric distribution analysis to look at our production process and determine what in the process went wrong to produce the defects. The goal is to make the process more robust so that the defects are never produced.

Once we verify through statistical analysis that our process will not produce the defects, we would be in a position to reevaluate our inspection process.

Ed Kemmerling
Chris Simmons
August 4, 2009 - 11:38 am
Reply

Ed, you seem to be missing the point. The question this analysis answers is whether the sampling quantity was sufficient. In other words it confirms that it should have been consistent. It could have gone the other way showing that the sampling size was not sufficient. This is a huge problem as most QA folks have no idea how to go about this. At least that’s been my experience.
Ed Kemmerling
August 4, 2009 - 12:03 pm
Reply

I agree that we should always evaluate sample size. However, when we miss 36 defects out of 1000, I guarantee that sample size is not the issue.
Ron Pereira
August 4, 2009 - 4:36 pm
Reply

Hi everyone, thanks for your great and thought provoking comments. I really appreciate them.
Jeff Hajek
August 8, 2009 - 12:56 am
Reply

This article only talks about one shipment to one customer. The sampling plan is undoubtedly used on all of the shipments. So, out of every fifty shipments, it is likely that one will look good and actually be bad (consumer’s risk, or beta risk).
If this were the only inspection that happened and it had a problem, the most likely scenario is that there is a problem with the inspection process. I would look at that first.
On the other hand, if there were 49 other inspections that turned out right (i.e. properly categorized the lot), the sampling process might not be broken. Sampling is a shortcut, and carries risk. Every sampling plan will, eventually, have this kind of problem. Even with a sample size of 999, there is still a chance, no matter how small, that the one that isn’t inspected will be the one that is defective. That’s why sampling isn’t used for critical functions, like airport security.
There’s another important task, though. We don’t know how long ago the problem happened. Since there is only a 1 in 50 (ish) chance that this happened to the very first customer, the producer probably needs to be looking at prior shipments too.
The good news, though, is that the article mentions that a permanent fix (poka yoke) was put in place. It sounds like the company is on the right track–preventing defects at the source.
Anonymous
August 10, 2009 - 7:20 pm
Reply

If you were making teh widgets in lots of 10, testing every 10th wouldnt be the smartest idea. 10mod10=0
Anonymous
August 10, 2009 - 7:24 pm
Reply

Do defects occur completely randomly or in clusters? If the latter i would think entirely random sampling might not be the most efficient.
Gagandeep S. Datta
August 19, 2009 - 4:56 am
Reply

When we talk of hyper-geometric distribution, we require the following data to establish a probability:
• the number of successes in the sample
• the size of the sample
• the number of successes in the population
• the population size

In excel the formula is entered as HYPGEOMDIST(the number of successes in the sample, the size of the sample, the number of successes in the population, the population size)

Let me first illustrate using a lottery example. For e.g., If we want the probability of getting 3 winning numbers in a lottery of 49 numbers where we draw 6 numbers, it is equivalent to saying, “any 3 from 6 in 49 of 1/49% (2.04%),” which in excel translates to: =HYPGEOMDIST(3,6,6,49) = 1.77%

So in the example stated by you, in this scenario,
– the number of defectives in the sample = 0
– the size of the sample = 100
– the number of defectives in the population = 36
– the population size = 1000

Which in excel translates to: =HYPGEOMDIST(0,100,36,1000) = 2.09676%
Ron Pereira
August 19, 2009 - 6:34 am
Reply

Thanks for the excellent analysis, Gagandeep! I, and I’m sure those without Minitab access, appreciate the time you took to share this!
Gagandeep S. Datta
August 19, 2009 - 9:20 am
Reply

Ron

I have a leading question to this … suppose we did not have the defectives count in the population (which 99.9% is true!). i.e.,

– the number of defectives in the sample = 0
– the size of the sample = 100
– the number of defectives in the population = “unknown”
– the population size = 1000

What I have boserved is that many examples assume ” the number of defectives in the population = the size of the sample” i.e.,

– the number of defectives in the sample = 0
– the size of the sample = 100
– the number of defectives in the population = 100
– the population size = 1000

Why is this so the case?

For e.g., suppose we want the probability of getting 3 winning numbers in a lottery of 49 numbers where we draw 6 numbers. This is equivalent to saying, any 3 from 6 in 49 of 1/49% (2.04%).

The resulting excel formula and result is =HYPGEOMDIST(3,6,6,49) = 1.77%

The reason I am seeking this is because Minitab asks for Sample-Size, Population-Size and Population-Defectives, REALITY is that we have defectives IN samples and NOT in populations (populations which are theoretically and practically infinite!).

Seek clarity.

Gagandeep
Ron Pereira
August 19, 2009 - 9:33 am
Reply

Gagandeep, the key to this analysis – and really any analysis that attempts to “infer” something from a sample of data is to ensure you do a good job sampling.

My example here was made up… but in order to do what I describe for real you’d need to have confidence in your sampling plan… also, you must know how many defects or defectives we’re starting with in order to infer anything.

So, if you don’t have any data on how many defects or defectives a process has this must be your first step. Go to gemba and observe first… Mintab second. Make sense?
Sowrirajan
August 20, 2009 - 8:35 pm
Reply

Hi
There seem to a basic problem with the sampling method they used.
Sampling methods use certain sample size formulae that are developed for well defined static population situations.
Though random sampling can be applied to stable processes we use systematic or sub-group sampling with clear time order to represent the process behaviour precisely.
Representativeness is the most important aspect of sampling.
Ron can correct me if i have faltered in my assessment please.
Robert
September 1, 2009 - 7:15 am
Reply

So why talking about fixing the inspection system? If 36 products out of 1000 are busted, there is room for improvement on the manufacturing side…
Or was this about explaining the distribution in a simple way?