Simpson's Paradox happens when a trend appears in several separate groups but reverses or disappears when the groups are combined. It matters because it shows that averages, percentages, and correlations can be misleading when data are pooled without context. The paradox is common in medicine, admissions, sports, economics, and social science.
It reminds us that statistics must be interpreted with attention to hidden variables and group sizes.
The mechanism usually involves a lurking variable that changes the weighting of groups in the combined data. For example, Treatment A can have a higher success rate than Treatment B for both mild and severe cases, while Treatment B looks better overall if it was used more often on easier cases. The combined rate is a weighted average, so groups with larger sample sizes have more influence.
To analyze the situation correctly, compare within meaningful subgroups and ask whether the grouping variable is related to both the explanatory variable and the outcome.
Key Facts
- Simpson's Paradox: a trend within separate groups reverses when the groups are combined.
- Group success rate = successes in group / total in group.
- Combined success rate = total successes / total cases.
- Weighted mean: x̄ = (w1x1 + w2x2 + ... + wnxn) / (w1 + w2 + ... + wn).
- A lurking variable can distort an apparent relationship if it affects both the explanatory variable and the response variable.
- Stratifying data means analyzing results separately within relevant groups before combining conclusions.
Vocabulary
- Simpson's Paradox
- A statistical pattern in which a relationship seen within multiple groups reverses or changes when the groups are combined.
- Lurking variable
- A hidden or unaccounted variable that influences the relationship between the variables being studied.
- Weighted average
- An average in which some values count more than others because their groups are larger or more important.
- Stratification
- The process of separating data into meaningful subgroups before comparing or modeling results.
- Confounding
- A situation where the effect of one variable is mixed with the effect of another variable, making conclusions unclear.
Common Mistakes to Avoid
- Comparing only the overall percentages, which is wrong because the combined result may hide different subgroup sizes and subgroup patterns.
- Ignoring the lurking variable, which is wrong because a hidden factor such as case severity, age, or difficulty level can drive the apparent trend.
- Treating a higher combined rate as proof that one option is better, which is wrong because the combined rate may be influenced more by where the cases were concentrated than by true performance.
- Assuming Simpson's Paradox is a calculation error, which is wrong because the arithmetic can be perfectly correct while the interpretation is incomplete.
Practice Questions
- 1 Treatment A succeeds in 18 of 20 mild cases and 30 of 100 severe cases. Treatment B succeeds in 80 of 100 mild cases and 2 of 10 severe cases. Find the success rate for each treatment in each subgroup and overall. Which treatment looks better within each subgroup, and which looks better overall?
- 2 A school reports that Program X has 45 acceptances out of 100 applicants, while Program Y has 50 acceptances out of 100 applicants. In Group 1, X accepts 40 of 80 and Y accepts 9 of 20. In Group 2, X accepts 5 of 20 and Y accepts 41 of 80. Compute the subgroup acceptance rates and explain whether the overall comparison matches the subgroup comparisons.
- 3 A data set shows that a new study method improves test pass rates for both beginners and advanced students, but the overall pass rate is lower for students using the new method. Explain how different numbers of beginners and advanced students in each method group could create this result.