Hypothesis Tests in Practice

Abstract

Hypothesis tests, such as the t-test, chi-square test, and ANOVA test, yield p-values that represent the probability of observing a particular study result, or a more extreme result, if a pre-specified null hypothesis about the population were true. A p-value threshold, such as 0.05, is typically used to declare statistical significance. Consequently, a hypothesis test may declare a result to be significant when in fact there is no actual difference in the population (type I error) or declare a result to be nonsignificant when in fact there is an actual difference in the population (type II error). Study power, which is the probability of not making a type II error, is influenced by sample size, effect size, variation, and the threshold value for declaring significance.