In hypothesis testing, statisticians make decisions using sample data, but those decisions can be wrong. Type I and Type II errors describe the two main ways a test can fail when deciding whether to reject a null hypothesis. These ideas matter because every real test, from medical screening to quality control, balances the risk of false alarms against the risk of missed effects.
Understanding both errors helps students interpret test results more carefully.
A Type I error happens when the null hypothesis is actually true, but the test rejects it anyway. A Type II error happens when the null hypothesis is actually false, but the test fails to reject it. The probability of a Type I error is called , and the probability of a Type II error is called .
Test power is , so reducing missed detections usually means increasing power through better design, larger samples, or stronger effects.
Understanding Type I and Type II Errors
When a test is run, the true state of the world is hidden. A decision table separates that hidden reality from the conclusion drawn from a sample. Its rows represent whether the null hypothesis matches reality.
Its columns represent the action taken by the researcher. Two cells show correct decisions, while two cells show mistakes. This layout matters because a statistical result does not reveal reality with certainty.
Failing to reject the null hypothesis is not proof that it is true. It means the data did not provide enough evidence under the rule chosen for the test.
Researchers choose a decision threshold before examining results. That threshold controls how often a false alarm would occur across many repetitions of the same kind of study, if the null hypothesis were actually true. It is a long run rule, not a guarantee about one experiment.
A p value has a different job. It describes how unusual the observed data, or more extreme data, would be if the null hypothesis were true.
It does not state the chance that the null hypothesis itself is true. Students often confuse these ideas, so it helps to keep the decision rule separate from the evidence calculated from the sample.
The balance between the two error types depends on the consequences of each mistake. In airport security, missing a real danger can be very serious, so systems may be designed to detect more possible threats. This can create more false alarms.
In a factory, stopping a production line because of a harmless measurement may be costly. Yet allowing a faulty product through may be worse. Medical screening has the same tension.
A positive screen is usually followed by a more accurate diagnostic test because a screening result alone can produce false positives. The best threshold depends on who could be harmed, what a mistake costs, and whether a second check is available.
Better study design can reduce the need for a harsh tradeoff. Larger samples make estimates less variable, which helps a test notice a real effect without raising the false alarm rate. Clear measurement methods matter for the same reason.
Noisy instruments, biased samples, and uncontrolled differences between groups can hide genuine patterns or create misleading ones. Students should pay attention to the wording of the null and alternative hypotheses, the sample size, and the assumptions behind the test. They should also watch for repeated testing.
If someone tests many possible patterns and reports only one significant result, false positives become more likely. Reporting effect sizes and confidence intervals gives more useful context than a reject or fail to reject decision alone.
Key Facts
- Type I error: reject when is true.
- Type II error: fail to reject when is false.
- P(Type I error) = .
- P(Type II error) = .
- Power = .
- Lowering usually makes rejecting harder and can increase if sample size stays fixed.
Vocabulary
- Null hypothesis
- The default claim, usually written as , that says there is no effect, no difference, or no change.
- Alternative hypothesis
- The competing claim, usually written as or , that says an effect, difference, or change exists.
- Significance level
- The chosen cutoff that sets the maximum tolerated probability of a Type I error.
- Type I error
- A false positive in which the test rejects the null hypothesis even though it is actually true.
- Type II error
- A false negative in which the test does not reject the null hypothesis even though it is actually false.
Common Mistakes to Avoid
- Saying a Type I error means the null hypothesis is false, because a Type I error actually happens when the null hypothesis is true but gets rejected anyway.
- Thinking failing to reject proves is true, because the test may simply lack enough evidence or power to detect a real effect.
- Confusing with , because is the probability of a false positive while is the probability of a false negative.
- Assuming lowering improves everything, because making rejection harder can increase and reduce power when sample size does not change.
Practice Questions
- 1 A medical test uses . What is the probability of a Type I error, and what does that mean in words?
- 2 A hypothesis test has . Calculate the power of the test.
- 3 A researcher lowers from to without increasing sample size. Explain how this change is likely to affect the chances of Type I and Type II errors.