A p-value is a probability used in hypothesis testing to measure how unusual your data would be if the null hypothesis were true. It helps decide whether an observed result is surprising enough to question a default claim, such as no effect or no difference. In science, medicine, engineering, and social science, p-values help separate random variation from evidence of a real pattern.
They do not prove a hypothesis, but they quantify how compatible the data are with a specific model.
Key Facts
- A p-value is P(data as extreme or more extreme than observed | H0 is true).
- A small p-value means the observed result would be unlikely under the null hypothesis.
- Reject H0 when p-value ≤ alpha, where alpha is the chosen significance level.
- For a right-tailed z-test, p-value = P(Z ≥ zobs).
- For a two-tailed z-test, p-value = 2P(Z ≥ |zobs|).
- A p-value is not P(H0 is true | data), and it is not the probability that the result happened by chance.
Vocabulary
- Null hypothesis
- The null hypothesis, written H0, is the default claim being tested, often stating that there is no effect, no difference, or no association.
- Alternative hypothesis
- The alternative hypothesis, written Ha or H1, is the claim that the test looks for evidence to support.
- p-value
- The p-value is the probability of getting a result at least as extreme as the observed result, assuming the null hypothesis is true.
- Significance level
- The significance level, written alpha, is the cutoff probability chosen before the test for deciding when to reject the null hypothesis.
- Test statistic
- A test statistic is a standardized number, such as z or t, that measures how far the observed data are from what the null hypothesis predicts.
Common Mistakes to Avoid
- Saying the p-value is the probability that H0 is true is wrong because the p-value assumes H0 is true and then asks how unusual the data are under that assumption.
- Treating p = 0.049 and p = 0.051 as completely different is wrong because the evidence changes smoothly, while the alpha cutoff is a decision rule.
- Using alpha after seeing the p-value is wrong because alpha should be chosen before the analysis to avoid biased decision making.
- Thinking a small p-value proves a large or important effect is wrong because statistical significance depends on sample size as well as effect size.
Practice Questions
- 1 A right-tailed z-test gives zobs = 1.96. Using P(Z ≥ 1.96) = 0.025, what is the p-value, and would you reject H0 at alpha = 0.05?
- 2 A two-tailed z-test gives zobs = -2.30. Using P(Z ≥ 2.30) = 0.0107, calculate the p-value and decide whether the result is significant at alpha = 0.01.
- 3 A study reports p = 0.03 for a new treatment compared with a placebo. Explain what this p-value means, what it does not mean, and how the conclusion depends on alpha.