Reading a research study is a skill for deciding whether a statistical claim is trustworthy. A study may use real data and still lead to a weak conclusion if the sample, design, or analysis is flawed. Good readers look past the headline and inspect how the evidence was collected, measured, and interpreted.
This matters in science, medicine, economics, psychology, and any field where data are used to guide decisions.
A strong reading process checks the research question, sample size, sampling method, study design, variables, possible confounders, effect size, uncertainty, and statistical significance. Randomized experiments can support stronger cause and effect claims than observational studies, but both require careful interpretation. A result can be statistically significant while still being small, biased, or not useful in practice.
The best conclusion matches the evidence without overstating what the study actually shows.
Key Facts
- Sample size matters because larger samples usually reduce random sampling error, but they do not fix biased sampling.
- Margin of error often decreases like 1/sqrt(n), where n is the sample size.
- A p-value is P(data this extreme or more extreme | null hypothesis is true), not the probability that the null hypothesis is true.
- A 95% confidence interval gives a range of plausible values for a population parameter based on the study method.
- Effect size measures how large a difference or relationship is, such as difference in means = mean treatment - mean control.
- Correlation does not prove causation because confounders, reverse causation, or selection effects may explain the pattern.
Vocabulary
- Sample
- A sample is the group of individuals, objects, or cases actually measured in a study.
- Population
- A population is the larger group that the researchers want to draw conclusions about.
- Confounder
- A confounder is a variable related to both the explanatory variable and the outcome that can distort the apparent relationship.
- Statistical significance
- Statistical significance means the observed result would be unlikely under a specified null hypothesis, often judged using a p-value cutoff.
- Effect size
- Effect size is a measure of the magnitude of a difference, association, or treatment impact.
Common Mistakes to Avoid
- Treating a small sample as automatically convincing is wrong because random variation can strongly affect small studies and make results unstable.
- Reading a p-value as the chance the claim is true is wrong because a p-value is calculated assuming the null hypothesis, not the study claim.
- Ignoring the study design is wrong because an observational study usually cannot establish causation without strong assumptions and controls.
- Focusing only on statistical significance is wrong because a tiny effect can be significant in a large sample but still be unimportant in practice.
Practice Questions
- 1 A survey reports that 54 out of 300 students prefer online homework. What sample proportion prefers online homework?
- 2 A treatment group has a mean score of 82 and a control group has a mean score of 76. What is the difference in means, and which group scored higher?
- 3 A study finds that people who drink more coffee have lower rates of a disease. Explain why this result alone does not prove that coffee prevents the disease.