Hypothesis Testing

One of my favorite statistical tools is hypothesis testing. We can use hypothesis testing for many purposes.

For example, we would use the popular 2-sample t-test when we have two samples of variable data and want to understand if they represent different populations, statistically speaking of course.

State the Null and Alternate Hypothesis

The first order of business when completing hypothesis tests is to state the null hypothesis (Ho) and the alternative hypothesis (Ha).

The null hypothesis is the statement of no change. I always remember it like this, “Ho hum… there is no difference here.”

Conversely, the alternative hypothesis is the statement of change. Just remember, “a Ha, there is a change!”

In our 2-sample t-test example, Ho would be that the means are equal and Ha would be that the means are not equal. It’s as simple as that.

Collect Data and Run Test

Next, we need to carefully collect some data and run the actual hypothesis test. You can program spreadsheets to do this test or use a standard off the shelf software package to do it for you.

We Never “Accept” Anything!

When we run the actual hypothesis test we will get a P value. We use this P value to determine whether enough evidence exists to REJECT the null hypothesis.

I emphasize REJECT since the most common error I see people make is when they speak of “accepting” a null hypothesis.

We never accept a null hypothesis for the same reason we never prove someone innocent in the US judicial system. Instead, we prove someone guilty or not guilty. So with hypothesis testing we either reject or fail to reject the null hypothesis.

If P is Low, Ho must Go!

Now then, the P value is the probability of incorrectly rejecting the null hypothesis. Since we are making important decisions with this P value we tend to error on the safe side.

Typically, if the P value is less than 5% we reject the null hypothesis. If the P value is greater than 5% we fail to reject the null hypothesis.

If all this makes your head hurt no worries. Just remember this saying, “If P is low, Ho must go. If P is high, Ho can fly.”

Alpha Risk Explained

Why 5%?  The standard is usually to go with 5% since this the risk most people are willing to take at being wrong. This is also why you often hear about 95% confidence intervals.

If you are sending people to the moon or testing something ultra serious you may consider tightening this “alpha value” as it is called to something like 1% or 2%. I will resort to the response any good Black Belt should give when asked what alpha value to use – it depends!

Until next time, I wish you all the best on your journey towards continuous improvement.

Tagged in:,


  1. robert

    February 27, 2007 - 5:52 am

    Hi Ron

    Hypothesis testing is one of the simpler but most powerful tools which can be deployed. I posted a couple of articles on this subject here: http://tinyurl.com/ypjxp2

    I can’t figure out why it is not more widely deployed throughout industry? Any ideas?


  2. Ron Pereira

    February 27, 2007 - 11:30 am

    Good question. I guess it all comes back to awareness. The more aware we make people the more it will get used.

  3. Jon Miller

    February 27, 2007 - 12:16 pm

    Very useful tidbit. Keep it up.

  4. Ron Pereira

    February 27, 2007 - 1:35 pm

    Thanks Jon. I will do my best!

  5. Joe

    March 1, 2007 - 5:38 am

    Hi Ron

    The article which you have posted was very simple and useful for the beginners like me and hope to get more thoughts on REGRESSION Analysis too,

  6. Ron Pereira

    March 2, 2007 - 2:18 am

    Thanks for the idea Joe. I will write something on regression soon.

  7. Anonymous

    August 8, 2007 - 7:12 am

    Good use of legal system to explain hypothesis test.

  8. Ron Pereira

    August 8, 2007 - 12:39 pm

    Thank you Anonymous. Please stop back again.