Replication and reproducibility are two pillars of reliable science. Replication within a study means repeating measurements, trials, or analyses under the same planned conditions to check whether the result is stable. Reproducibility across studies means independent researchers can obtain similar conclusions using new data, methods, settings, or research teams.
These ideas matter because a single surprising result can be caused by chance, bias, or hidden errors.
Key Facts
- Replication within a study reduces random error by repeating measurements or trials.
- Reproducibility across studies tests whether a conclusion holds beyond one lab, sample, or method.
- Standard error often decreases as sample size increases: SE = s / sqrt(n).
- A p-value is not the probability that the hypothesis is true, and p < 0.05 does not guarantee reproducibility.
- Effect size describes how large a result is, while statistical significance describes how unusual the result is under a null model.
- Transparent methods, shared data, preregistration, and independent confirmation increase scientific reliability.
Vocabulary
- Replication
- Replication is the repetition of measurements, trials, or analyses to see whether a result is consistent under similar conditions.
- Reproducibility
- Reproducibility is the ability of independent researchers or methods to reach similar conclusions from the same or new evidence.
- Random error
- Random error is unpredictable variation in measurements that can make repeated observations differ from one another.
- Effect size
- Effect size is a numerical measure of the strength or magnitude of a relationship, difference, or treatment effect.
- Reproducibility crisis
- The reproducibility crisis refers to concerns that many published scientific findings are difficult to confirm in later studies.
Common Mistakes to Avoid
- Confusing replication with reproducibility is wrong because repeating trials inside one study is not the same as confirming the result in independent studies.
- Treating p < 0.05 as proof is wrong because statistical significance can occur by chance, especially when many tests are performed.
- Ignoring sample size is wrong because small samples often produce unstable estimates and exaggerated effect sizes.
- Changing methods after seeing the data without reporting it is wrong because it can make results look stronger than they really are.
Practice Questions
- 1 A lab measures a reaction time with standard deviation s = 12 ms using n = 9 repeated trials. Calculate the standard error using SE = s / sqrt(n).
- 2 Study A tests 20 people and finds an effect size of 0.80. Study B tests 200 people and finds an effect size of 0.35 for the same effect. Which estimate is likely more stable, and why?
- 3 A published study reports a surprising result with p = 0.03, but three independent labs using larger samples do not find the effect. Explain what this suggests about reproducibility and why the original result might still have occurred.