Cohen's d is a standardized effect size that describes how far apart two group means are in units of standard deviation. It helps you judge the size of a difference, not just whether a difference is statistically significant. This matters because a tiny difference can be statistically significant in a very large sample, while a meaningful difference may fail to reach significance in a small sample.
Cohen's d gives a scale-free way to compare results across studies that use different units or measurements.
In a diagram, Cohen's d is shown as the horizontal separation between two bell curve centers divided by the typical spread of the data. If the two distributions overlap heavily, d is small, and the groups are not very different in practical terms. If the distributions are farther apart, d is larger, and knowing a person's group gives more information about their likely score.
Common guidelines are d = 0.2 for small, d = 0.5 for medium, and d = 0.8 for large, but the meaning always depends on the context and measurement.
Understanding Statistics: Effect Size (Cohen's d)
The calculation has two parts. First find the gap between the group averages. Then judge that gap against the ordinary variation among individual scores.
The pooled standard deviation is used when the groups have roughly similar spreads. It combines each group’s variation while giving more weight to the group with more observations. This matters because a standard deviation based on ten students is less stable than one based on one hundred students.
The sign of the result only records direction. If Group B is placed first in the subtraction, a positive result says its average was higher. Reversing the order changes the sign but not the size of the difference.
A result can look large or small partly because of how variable the measurement is. Imagine two classes whose test averages differ by five marks. If most scores lie within two marks of their class average, five marks is a substantial separation.
If scores usually vary by twenty marks, the same five-mark gap is modest. Outliers can increase the standard deviation and make the effect size smaller. A few unusually high or low values deserve checking before any conclusion is made.
Students should inspect a graph of the data, such as a dot plot or box plot, rather than relying on one number. Two groups can have the same average gap but very different shapes, spreads, or clusters.
Cohen's d is useful when raw units do not transfer easily between settings. A difference of three points means little without knowing whether the score is out of ten, out of one hundred, or based on a scale with wide natural variation. In education, it can describe the difference in reading scores after two teaching methods.
In health research, it can describe a change in pain scores after a treatment. In sports science, it can compare improvements in sprint time across training plans. Practical importance still needs real-world knowledge.
A small effect on a cheap, safe action that reaches millions of people may matter greatly. A larger effect may not matter if it is too costly, too risky, or below a useful threshold.
The usual small, medium, and large labels are only rough starting points. Different subjects have different typical effect sizes. A small difference in a medical outcome can be valuable, while a medium difference in a classroom test may not change a student’s learning in a meaningful way.
Cohen's d is most straightforward for two independent groups with measurements that are reasonably well behaved and have similar variation. When the same people are measured before and after an intervention, the paired differences must be handled differently because the scores are linked. When group spreads are very unequal, a pooled standard deviation may be misleading.
Small samples can make the estimated effect size unstable and slightly too large on average. Researchers may use a corrected version called Hedges' g and report a confidence interval to show the range of effect sizes compatible with the data.
Key Facts
- Cohen's d = (mean of Group B - mean of Group A) / pooled standard deviation.
- Pooled standard deviation: sp = sqrt(((n1 - 1)s1^2 + (n2 - 1)s2^2) / (n1 + n2 - 2)).
- If both groups have the same standard deviation s, then d = (M2 - M1) / s.
- A positive d means Group B has the higher mean if d = (M2 - M1) / sp; a negative d means Group B has the lower mean.
- Rule of thumb: d = 0.2 is small, d = 0.5 is medium, and d = 0.8 is large.
- Statistical significance depends strongly on sample size, while Cohen's d describes practical size in standard deviation units.
Vocabulary
- Effect size
- A numerical measure of how large or important a difference, relationship, or change is.
- Cohen's d
- A standardized effect size that measures the difference between two means in units of standard deviation.
- Mean
- The average value of a data set, found by adding all values and dividing by the number of values.
- Standard deviation
- A measure of how spread out data values are around the mean.
- Statistical significance
- A result is statistically significant when the observed data would be unlikely under a specified null hypothesis.
Common Mistakes to Avoid
- Treating p-values as effect sizes is wrong because a p-value tells how surprising the data are under a null hypothesis, not how large the difference is.
- Ignoring the sign of Cohen's d is a mistake because the sign shows which group has the higher mean based on the order used in the formula.
- Using the raw mean difference alone can be misleading because a difference of 5 units may be large on one scale but small on another.
- Applying the small, medium, and large labels without context is wrong because a small effect can matter in medicine or policy, while a large effect may be trivial in a low-stakes setting.
Practice Questions
- 1 Group A has a mean score of 70 and Group B has a mean score of 78. The pooled standard deviation is 10. Calculate Cohen's d and interpret it using the common rule of thumb.
- 2 Two classes take the same test. Class 1 has n = 20, mean = 82, and s = 6. Class 2 has n = 20, mean = 76, and s = 8. Calculate the pooled standard deviation, then calculate Cohen's d using d = (M1 - M2) / sp.
- 3 A study with 10,000 participants finds p < 0.001 but Cohen's d = 0.05 for a new study method. Explain why the result can be statistically significant but still have little practical importance.