Hypothesis testing is a way to use sample data to judge whether a claim about a population is believable. The null hypothesis is the starting assumption, such as no difference or no effect. A test statistic measures how far the sample result is from what the null hypothesis predicts.
Critical values mark the boundary between results that are considered ordinary under the null and results that are unusual enough to reject it.
The decision depends on the sampling distribution of the test statistic, the significance level, and whether the test is one-tailed or two-tailed. If the test statistic falls in a rejection region beyond a critical value, the result is statistically significant and the null hypothesis is rejected. If it does not, the data do not provide enough evidence to reject the null.
For example, in a two-tailed z test with alpha = 0.05, the critical values are about z = -1.96 and z = 1.96, so a test statistic of z = 2.30 leads to rejecting the null hypothesis.
Understanding Statistics: Test Statistics and Critical Values
A test statistic puts a result on a common scale. A difference of five points can be large in one study but small in another. It depends on how much natural variation is expected and how much data were collected.
The standard error describes the usual amount that a sample estimate changes from sample to sample. A small standard error makes the same difference look more convincing.
Larger samples usually reduce standard error because random highs and lows cancel out more reliably. This is why sample size has such a strong effect on hypothesis tests.
The correct reference distribution matters. A z test is used when the population standard deviation is known or when an approximation is justified by a large sample. In many real studies, that population spread is unknown.
Students estimate it from their sample, which adds uncertainty. A t test accounts for this extra uncertainty. Its distribution has heavier tails than the normal distribution, especially for small samples.
As the sample size grows, the t distribution becomes very close to the normal distribution. This explains why critical values depend on degrees of freedom for t tests.
Choosing one tail or two tails must happen before viewing the data. A two-tailed test checks for a difference in either direction. For example, a school may test whether a new timetable changes average attendance, whether it rises or falls.
A one-tailed test checks one stated direction, such as whether a training program increases typing speed. One-tailed tests place all of the allowed false alarm risk in one end of the distribution. They can detect an effect in that direction more easily, but they should not be chosen simply because the observed result points the preferred way.
The significance level controls the chance of a Type One error over many repeated studies. A Type One error means rejecting a null hypothesis that is actually true. Lowering alpha reduces this risk, though it makes rejection harder.
That can increase the chance of a Type Two error, where a real effect is missed. Researchers often use zero point zero five, but it is a convention rather than a law.
Medical safety studies may need a stricter threshold. Early exploratory work may use a different choice if it is stated clearly and justified.
A result that fails to reject the null does not prove that the null is true. It may mean the sample was too small, the measurements were noisy, or the true effect was modest. Students should check the assumptions behind the test, including random sampling, independent observations, and an appropriate distribution or sample size.
They should examine the size of the estimated effect, not only the reject or fail to reject decision. A tiny difference can be statistically significant with thousands of observations yet have little practical importance. Confidence intervals help show both the likely size of an effect and the uncertainty around it.
Key Facts
- Test statistic = (sample estimate - null value) / standard error
- For a z test of a mean with known population standard deviation, z = (x̄ - μ0) / (σ / √n)
- For a one-sample t test, t = (x̄ - μ0) / (s / √n)
- In a two-tailed z test with alpha = 0.05, the critical values are approximately -1.96 and 1.96
- Reject H0 if the test statistic falls in the rejection region
- The significance level alpha is the probability of rejecting H0 when H0 is actually true
Vocabulary
- Null hypothesis
- The starting claim in a hypothesis test, usually stating no effect, no difference, or a specific population value.
- Test statistic
- A standardized number that shows how far the sample result is from the null hypothesis in standard error units.
- Critical value
- A cutoff point on the test statistic scale that separates the non-rejection region from the rejection region.
- Rejection region
- The part of the sampling distribution where test statistic values are unlikely enough under the null hypothesis to reject it.
- Significance level
- The chosen probability of a Type I error, commonly written as alpha and often set to 0.05.
Common Mistakes to Avoid
- Comparing the raw sample mean to the critical value is wrong because critical values are on the standardized test statistic scale, not the original measurement scale.
- Using a two-tailed critical value for a one-tailed test is wrong because the rejection area is split differently depending on the alternative hypothesis.
- Saying fail to reject H0 proves H0 is true is wrong because the test only shows that the sample evidence was not strong enough to reject it.
- Ignoring the sign of the test statistic in a one-tailed test is wrong because the rejection region may be only on the left or only on the right side of the distribution.
Practice Questions
- 1 A z test has H0: μ = 100, sample mean x̄ = 106, population standard deviation σ = 12, and n = 36. Compute the test statistic z.
- 2 For a two-tailed z test with alpha = 0.05, the critical values are -1.96 and 1.96. If the test statistic is z = -2.14, should you reject H0?
- 3 A study reports a test statistic that is close to 0 and lies near the center of the null distribution. Explain what this suggests about the evidence against the null hypothesis.