What is the power of a statistical test and why use it?

Power is the probability of making a correct decision (to reject the null hypothesis) when the null hypothesis is false.

To determine it, one needs to assume a particular alternative hypothesis.

Because power focuses on “making a correct decision” it is natural to use it to report to decision-makers on the results of a test.

Unfortunately, it is not used as often as it should. Instead, major focus has been given to p-values.

A p-value is the probability of observing a result as large or larger than a test result given that the null hypothesis is true. In other words, it provides a measure of evidence supporting the null hypothesis assuming it is true. This is generally a bit harder to explain.

P-value and power are diametrically opposed in that as the p-value gets smaller (suggesting that the null hypothesis is false) power gets larger (suggesting that the alternative hypothesis is true), thus increasing the probability of making a correct decision.

In this video, Julian Parris, Learning Strategy Manager for SAS/JMP software, covers the basics of the power of a statistical test and the factors that affect it.