Cohen's d is a standardized effect size that describes how far apart two group means are in units of standard deviation. It helps you judge the size of a difference, not just whether a difference is statistically significant. This matters because a tiny difference can be statistically significant in a very large sample, while a meaningful difference may fail to reach significance in a small sample.
Cohen's d gives a scale-free way to compare results across studies that use different units or measurements.
In a diagram, Cohen's d is shown as the horizontal separation between two bell curve centers divided by the typical spread of the data. If the two distributions overlap heavily, d is small, and the groups are not very different in practical terms. If the distributions are farther apart, d is larger, and knowing a person's group gives more information about their likely score.
Common guidelines are d = 0.2 for small, d = 0.5 for medium, and d = 0.8 for large, but the meaning always depends on the context and measurement.
Key Facts
- Cohen's d = (mean of Group B - mean of Group A) / pooled standard deviation.
- Pooled standard deviation: sp = sqrt(((n1 - 1)s1^2 + (n2 - 1)s2^2) / (n1 + n2 - 2)).
- If both groups have the same standard deviation s, then d = (M2 - M1) / s.
- A positive d means Group B has the higher mean if d = (M2 - M1) / sp; a negative d means Group B has the lower mean.
- Rule of thumb: d = 0.2 is small, d = 0.5 is medium, and d = 0.8 is large.
- Statistical significance depends strongly on sample size, while Cohen's d describes practical size in standard deviation units.
Vocabulary
- Effect size
- A numerical measure of how large or important a difference, relationship, or change is.
- Cohen's d
- A standardized effect size that measures the difference between two means in units of standard deviation.
- Mean
- The average value of a data set, found by adding all values and dividing by the number of values.
- Standard deviation
- A measure of how spread out data values are around the mean.
- Statistical significance
- A result is statistically significant when the observed data would be unlikely under a specified null hypothesis.
Common Mistakes to Avoid
- Treating p-values as effect sizes is wrong because a p-value tells how surprising the data are under a null hypothesis, not how large the difference is.
- Ignoring the sign of Cohen's d is a mistake because the sign shows which group has the higher mean based on the order used in the formula.
- Using the raw mean difference alone can be misleading because a difference of 5 units may be large on one scale but small on another.
- Applying the small, medium, and large labels without context is wrong because a small effect can matter in medicine or policy, while a large effect may be trivial in a low-stakes setting.
Practice Questions
- 1 Group A has a mean score of 70 and Group B has a mean score of 78. The pooled standard deviation is 10. Calculate Cohen's d and interpret it using the common rule of thumb.
- 2 Two classes take the same test. Class 1 has n = 20, mean = 82, and s = 6. Class 2 has n = 20, mean = 76, and s = 8. Calculate the pooled standard deviation, then calculate Cohen's d using d = (M1 - M2) / sp.
- 3 A study with 10,000 participants finds p < 0.001 but Cohen's d = 0.05 for a new study method. Explain why the result can be statistically significant but still have little practical importance.