Two-sample tests compare the means or proportions of two independent groups to decide whether an observed difference is statistically meaningful. This cheat sheet helps students choose between a two-sample t-test and a two-sample z-test, set up hypotheses, and interpret results. It is useful for classwork, exams, and data investigations where two groups are being compared.
Key Facts
- Use a two-sample t-test for independent means when population standard deviations are unknown, with test statistic .
- Use a two-sample z-test for independent means when population standard deviations are known, with test statistic .
- For two proportions, use , where for a hypothesis test with .
- A confidence interval for the difference of two means is when using sample standard deviations.
- A confidence interval for the difference of two proportions is .
- The null hypothesis usually states no difference, such as or .
- Reject when the p-value is less than the significance level , such as .
- Two-sample tests require independent samples, random sampling or random assignment, and approximately normal sampling distributions.
Vocabulary
- Two-sample test
- A statistical test used to compare a parameter, such as a mean or proportion, between two independent groups.
- Null hypothesis
- The claim being tested that usually says there is no difference, such as .
- Alternative hypothesis
- The claim that represents a difference or direction of change, such as .
- Standard error
- The estimated standard deviation of a sampling distribution, such as for two sample means.
- P-value
- The probability of getting a test statistic at least as extreme as the observed one, assuming the null hypothesis is true.
- Significance level
- The cutoff probability used to decide whether evidence is strong enough to reject the null hypothesis.
Common Mistakes to Avoid
- Using a z-test when population standard deviations are unknown is wrong because sample standard deviations require the t-distribution for means.
- Pooling proportions in a confidence interval is wrong because is used for the hypothesis test under , not for estimating the interval.
- Forgetting to check independence is wrong because two-sample formulas assume the groups do not influence each other.
- Interpreting a large p-value as proof that is true is wrong because it only means the sample did not provide strong enough evidence against .
- Reversing the order of subtraction is wrong when the conclusion depends on direction, since and have opposite signs.
Practice Questions
- 1 Two independent samples have , , , , , and . Compute the two-sample t statistic for testing .
- 2 A survey finds successes out of and successes out of . Compute , , and the pooled proportion for testing .
- 3 For two independent samples, and . Using , find the confidence interval for .
- 4 Explain why a two-sample t-test is usually more appropriate than a two-sample z-test when comparing the average test scores of two classes.
Understanding Two-Sample Tests (t-test, z-test)
A test starts with a difference calculated from the two samples. That difference alone does not settle anything, because samples vary by chance. The standard error measures the amount of variation expected in the difference if the same study were repeated many times.
A small standard error makes a given difference more convincing. A large standard error makes it less clear. Sample size has a major effect here.
Larger groups usually give smaller standard errors because they provide more information. The test statistic compares the observed difference with the difference predicted by the null hypothesis, usually zero. It reports how many standard errors the result lies from that prediction.
The t distribution is used when standard deviations come from the samples. This adds uncertainty, especially with small samples. Compared with the normal distribution used for a z test, the t distribution gives more probability to values far from zero.
Its shape depends on degrees of freedom, which increase as sample sizes increase. With large samples, t and z results become very similar. In most real investigations, population standard deviations are not known, so a two sample t test is common.
Many courses use a version called Welch's t test. It does not assume that both groups have exactly the same spread, which is usually a sensible choice.
The word independent needs careful attention. Measurements are independent when one person's result does not determine another person's result. Comparing test scores from two separate classes can fit this idea if students were sampled fairly.
Comparing a student's score before tutoring with that same student's score after tutoring does not fit. Those measurements are linked, so a paired test is needed instead. Random assignment supports cause and effect claims in experiments.
Random sampling supports conclusions about a larger population. Without either one, a test can describe the studied groups, but its wider conclusion may be weak. Extreme outliers and strongly uneven data can distort results, particularly when groups are small.
A p value is not the chance that the null hypothesis is true. It is the chance of getting a result at least as far from the null prediction if the null hypothesis were true. A small p value is evidence against the null hypothesis, but it does not prove anything with certainty.
When the p value is above the chosen significance level, the correct wording is fail to reject the null hypothesis. It does not mean that no difference exists. Confidence intervals add useful detail because they show a range of plausible differences.
If a confidence interval for a difference does not include zero, it agrees with a two sided test at the matching significance level. Students should report the direction and size of the difference, then consider whether that size matters in real life. A tiny difference can be statistically significant in a huge sample, while an important difference may remain uncertain in a small sample.