Choosing the Right Statistical Test Cheat Sheet

Key Facts

• Use a one-sample t test for one sample mean when the population standard deviation is unknown, with $t=\frac{\bar{x}-\mu_0}{s/\sqrt{n}}$.
• Use a two-sample t test to compare two independent means, with $t=\frac{\bar{x}_1-\bar{x}_2}{\sqrt{\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}}}$.
• Use a paired t test when the same subjects are measured twice or matched pairs are used, and test the mean difference with $t=\frac{\bar{d}}{s_d/\sqrt{n}}$.
• Use a one-proportion z test when testing one population proportion, with $z=\frac{\hat{p}-p_0}{\sqrt{\frac{p_0(1-p_0)}{n}}}$.
• Use a two-proportion z test when comparing two independent proportions, with $z=\frac{\hat{p}_1-\hat{p}_2}{\sqrt{\hat{p}(1-\hat{p})\left(\frac{1}{n_1}+\frac{1}{n_2}\right)}}$.
• Use a chi-square goodness-of-fit test for one categorical variable, with $\chi^2=\sum \frac{(O-E)^2}{E}$.
• Use a chi-square test of independence when testing whether two categorical variables are related in a two-way table.
• Use ANOVA to compare $3$ or more group means, where the test statistic is $F=\frac{\text{variation between groups}}{\text{variation within groups}}$.

Vocabulary

Null hypothesis

The null hypothesis, written $H_0$, is the claim that there is no effect, no difference, or no relationship in the population.

Alternative hypothesis

The alternative hypothesis, written $H_a$, is the claim that an effect, difference, or relationship exists.

P-value

The p-value is the probability of getting results at least as extreme as the sample results if $H_0$ is true.

Significance level

The significance level, written $\alpha$, is the cutoff probability for rejecting $H_0$, often $\alpha=0.05$.

Independent samples

Independent samples are groups where the data values in one group are not naturally paired with values in another group.

Paired data

Paired data occur when two measurements are linked, such as before-and-after measurements on the same person.