Hypothesis testing is a statistical method for using sample data to evaluate a claim about a population. It helps scientists, doctors, engineers, and social researchers decide whether an observed effect is likely real or could have happened by random chance alone. The process gives a structured way to compare evidence against a default assumption.
This makes conclusions more consistent and transparent.
In a hypothesis test, you begin with a null hypothesis that represents no effect or no difference, and an alternative hypothesis that represents the claim you want to investigate. After choosing a significance level , you calculate a test statistic from the sample and use it to find a -value. The -value measures how unusual the sample result would be if the null hypothesis were true.
If the -value is small enough, you reject the null hypothesis; otherwise, you fail to reject it.
Understanding Hypothesis Testing
A test works only when its starting assumptions fit the data. Before calculating anything, identify the population, the variable, and the sampling method. A random sample is important because it helps the sample represent the wider group.
Independence matters too. One person’s result should not strongly control another person’s result. For tests about averages, the data should be roughly bell shaped or the sample should be large enough for averages to behave predictably.
Outliers need attention because a few extreme values can pull an average far from the typical value. A precise calculation cannot fix biased data collection.
The direction of the alternative claim changes the evidence you count. A two sided test looks for a result either above or below the claimed value. It is used when either direction would matter.
A one sided test looks in only one direction, such as whether a new battery lasts longer than the old design. This choice must be made before viewing the data.
Choosing a one sided test after seeing that the sample average moved in a convenient direction makes the result less trustworthy. Students often lose marks by selecting the wrong tail or by writing a conclusion that does not match the original claim.
A small p-value is not the probability that the null hypothesis is false. It is a probability calculated under the assumption that the null hypothesis is true. It describes results at least as extreme as the observed sample result.
This distinction is important. A p-value of 0.03 does not mean there is a 3 percent chance that the null hypothesis is correct. It means that if the null hypothesis were correct, results this unusual would occur about 3 times in 100 similar samples.
The size of the observed difference matters separately. With a very large sample, even a tiny difference can produce a small p-value. A useful report includes the estimated difference and, when possible, a confidence interval.
Every decision has a risk of error. Rejecting a true null hypothesis is a Type I error. Failing to reject a false null hypothesis is a Type II error.
Lowering the significance level makes false alarms less likely, but it can make real effects harder to detect. The chance of detecting a real effect is called power. Power improves with a larger sample, less variable measurements, or a larger true difference.
This appears in real studies of medicines, school programs, factory quality checks, and online surveys. A conclusion should use careful wording. Say that the data provide evidence against the null hypothesis, or that the data do not provide enough evidence against it.
Failing to reject is not proof that there is no effect. It may simply mean the sample was too small or too noisy to show one.
Key Facts
- Null hypothesis usually states no difference, no effect, or a specific population value.
- Alternative hypothesis states the competing claim, such as , , or .
- Decision rule: reject if -value ; otherwise fail to reject .
- A common significance level is , which is the probability of a Type I error.
- For a test of a mean, .
- For a one-sample test, .
Vocabulary
- Null hypothesis
- The starting claim that there is no effect, no difference, or no change in the population.
- Alternative hypothesis
- The competing claim that says there is an effect, a difference, or a change.
- p-value
- The probability of getting a result at least as extreme as the sample result if the null hypothesis is true.
- Significance level
- The cutoff probability used to decide when evidence is strong enough to reject the null hypothesis.
- Test statistic
- A standardized number computed from sample data that measures how far the sample result is from what the null hypothesis predicts.
Common Mistakes to Avoid
- Saying the p-value is the probability that is true, which is wrong because the p-value assumes is true and measures the probability of the observed data or more extreme data.
- Writing reject when p-value > , which is wrong because results larger than do not provide enough evidence against the null hypothesis.
- Confusing fail to reject with proving true, which is wrong because the test may simply lack enough evidence or sample size to detect a real effect.
- Choosing a one-tailed test after looking at the data, which is wrong because the direction of the test must be set before analysis to avoid biased conclusions.
Practice Questions
- 1 A factory claims the mean battery life is 20 hours. A sample of 36 batteries has mean 18.8 hours. Assume hours and test versus at . Compute the statistic and state the decision using the critical value method or p-value method.
- 2 A school tests whether a new tutoring program changes average test scores. For 25 students, the sample mean is 78, the hypothesized mean is 74, and the sample standard deviation is 10. Test versus at using a one-sample test. Compute the statistic and state whether to reject .
- 3 A study reports -value = 0.08 for a test conducted at . Explain what decision should be made and why this does not mean the null hypothesis has been proven true.