Tukey HSD is a post-hoc test used after a one-way ANOVA finds evidence that not all group means are equal. It helps students compare every pair of group means while controlling the familywise error rate. This cheat sheet is useful because post-hoc testing can be confusing when there are many groups and many possible comparisons.
It organizes when to use the test, how to compute the statistic, and how to interpret results.
The core idea is to compare the absolute difference between two sample means to a honestly significant difference threshold. For equal sample sizes, the cutoff is . For unequal sample sizes, the Tukey-Kramer form uses .
A pair is statistically significant when is larger than the appropriate critical difference.
Key Facts
- Use Tukey HSD after a statistically significant one-way ANOVA when you want to compare all pairwise group means.
- The familywise error rate is controlled at across all pairwise comparisons, not separately for each individual comparison.
- For equal group sizes, the honestly significant difference is .
- For unequal group sizes, Tukey-Kramer uses for comparing groups and .
- The test statistic for a pair can be written as when all groups have the same sample size.
- Reject for a pair when or when the Tukey-adjusted confidence interval excludes .
- A Tukey confidence interval for equal sample sizes is .
- Tukey HSD assumes independent observations, approximately normal errors within groups, and similar population variances across groups.
Vocabulary
- Post-hoc test
- A follow-up procedure used after ANOVA to identify which specific group means differ.
- Familywise error rate
- The probability of making at least one Type I error across a set of multiple comparisons.
- Studentized range statistic
- A statistic, usually written as , based on the range of sample means divided by an estimated standard error.
- Mean square error
- is the within-group variance estimate from ANOVA used as the pooled error variance.
- Honestly significant difference
- The minimum pairwise mean difference required for significance in Tukey HSD.
- Tukey-Kramer method
- A version of Tukey's procedure that adjusts the standard error for unequal group sample sizes.
Common Mistakes to Avoid
- Using Tukey HSD without first checking the ANOVA context is wrong because the method is designed for pairwise follow-up comparisons after an omnibus ANOVA.
- Comparing each pair with ordinary tests is wrong because repeated unadjusted tests inflate the familywise error rate above .
- Using the wrong degrees of freedom is wrong because the critical value must use the ANOVA error degrees of freedom, not the total sample size.
- Using in the equal-size formula when group sizes are unequal is wrong because unequal samples require .
- Interpreting a significant pair as a large or important effect is wrong because statistical significance does not automatically measure practical importance.
Practice Questions
- 1 A one-way ANOVA has , , , equal group size , and . Compute .
- 2 For the same study, group means are , , , and . Which pairwise differences are significant using your from Question 1?
- 3 In an unequal-size study, , , , , , and . Compute the Tukey-Kramer critical difference for groups and , then decide whether the pair is significant.
- 4 Explain why Tukey HSD is preferred over several unadjusted pairwise tests when comparing all group means after ANOVA.
Understanding Tukey HSD Post-Hoc Test Reference
The special feature of Tukey’s method is the studentized range distribution. A usual two-group test considers one difference at a time. Tukey’s method considers the fact that researchers often inspect the largest gap among several group means.
Large gaps can appear by chance when many groups are present, even if every population mean is truly the same. The studentized range accounts for this selection effect. Its critical value becomes larger when the analysis includes more groups.
It is shaped by the error degrees of freedom as well. More error degrees of freedom usually give a more precise estimate of ordinary within-group variation, so the cutoff can become smaller.
The error term from the ANOVA is central to every comparison. It represents the typical spread of individual scores around their own group mean after group membership has been considered. Tukey uses this shared estimate rather than calculating a separate variance for each pair.
This is sensible when the groups have similar spreads. A small mean difference may be important if scores within groups are tightly clustered.
The same difference may be unconvincing when individual scores vary widely. For example, small differences in average reaction time can be clear in a controlled lab, yet not clear in a class survey with uneven conditions.
Read each comparison in both a statistical and a practical way. The sign of a mean difference gives the direction. A positive difference for group A minus group B means group A had the higher sample mean.
An adjusted confidence interval gives more detail than a simple significant or not significant label. Its endpoints show a range of population differences that remain compatible with the data under the model. An interval far from zero suggests a more clearly separated pair.
An interval close to zero signals uncertainty. Statistical significance does not tell how large or meaningful the difference is in real terms. Report the group means, their units, and a useful effect size when possible.
Check the data before trusting the table of pairwise results. Independence comes from the study design, not from a graph or a software setting. It can fail when one student contributes several scores, when classmates influence each other, or when repeated measurements are treated as separate people.
Look at plots of residuals and group spreads for extreme outliers, strong skew, and one group with much more variation than the others. Unequal sample sizes make these checks more important because a small group can have an unstable mean.
Tukey-Kramer handles differing group sizes, but it does not fix poor sampling, dependent observations, or severely unequal variances. When the design has repeated measures or important variance differences, a different analysis may be needed.