Hypothesis Testing

One of my favorite statistical tools is hypothesis testing. We can use hypothesis testing for many purposes.

For example, we would use the popular 2-sample t-test when we have two samples of variable data and want to understand if they represent different populations, statistically speaking of course.

State the Null and Alternate Hypothesis

The first order of business when completing hypothesis tests is to state the null hypothesis (Ho) and the alternative hypothesis (Ha).

The null hypothesis is the statement of no change. I always remember it like this, “Ho hum… there is no difference here.”

Conversely, the alternative hypothesis is the statement of change. Just remember, “a Ha, there is a change!”

In our 2-sample t-test example, Ho would be that the means are equal and Ha would be that the means are not equal. It’s as simple as that.

Collect Data and Run Test

Next, we need to carefully collect some data and run the actual hypothesis test. You can program spreadsheets to do this test or use a standard off the shelf software package to do it for you.

We Never “Accept” Anything!

When we run the actual hypothesis test we will get a P value. We use this P value to determine whether enough evidence exists to REJECT the null hypothesis.

I emphasize REJECT since the most common error I see people make is when they speak of “accepting” a null hypothesis.

We never accept a null hypothesis for the same reason we never prove someone innocent in the US judicial system. Instead, we prove someone guilty or not guilty. So with hypothesis testing we either reject or fail to reject the null hypothesis.

If P is Low, Ho must Go!

Now then, the P value is the probability of incorrectly rejecting the null hypothesis. Since we are making important decisions with this P value we tend to error on the safe side.

Typically, if the P value is less than 5% we reject the null hypothesis. If the P value is greater than 5% we fail to reject the null hypothesis.

If all this makes your head hurt no worries. Just remember this saying, “If P is low, Ho must go. If P is high, Ho can fly.”

Alpha Risk Explained

Why 5%?  The standard is usually to go with 5% since this the risk most people are willing to take at being wrong. This is also why you often hear about 95% confidence intervals.

If you are sending people to the moon or testing something ultra serious you may consider tightening this “alpha value” as it is called to something like 1% or 2%. I will resort to the response any good Black Belt should give when asked what alpha value to use – it depends!

Until next time, I wish you all the best on your journey towards continuous improvement.

---