Statistical tests are tools for deciding whether patterns in data are likely to be real or could have happened by chance. Each test is built on assumptions about how the data were collected and how the values behave. If those assumptions are badly violated, a p-value or confidence interval can become misleading.
Checking assumptions helps you choose a test that matches the data instead of forcing the data into the wrong method.
Common assumptions include independence, random sampling, approximate normality, and equal variance between groups. Some assumptions are checked by study design, while others can be checked with graphs, summaries, or formal tests. When an assumption fails, the solution might be a different test, a transformation, a robust method, or a clearer statement of the limits of the conclusion.
Good statistical reasoning means treating assumption checks as part of the analysis, not as an optional final step.
Key Facts
- Independence means one observation does not determine or strongly influence another observation.
- For a one-sample or paired t test, the differences should be approximately normal if the sample size is small.
- For a two-sample t test with equal variances, the pooled standard error is SE = sp sqrt(1/n1 + 1/n2).
- Welch's t test is often preferred when group variances or sample sizes are unequal.
- The chi-square test of independence usually requires expected cell counts of at least 5 in most cells.
- Standard error of a sample mean is SE = s/sqrt(n), and it decreases as sample size n increases.
Vocabulary
- Assumption
- An assumption is a condition that should be reasonably true for a statistical test to give reliable results.
- Independence
- Independence means observations are not linked in a way that makes one value predict or control another.
- Normality
- Normality means the data, residuals, or paired differences follow an approximately bell-shaped distribution.
- Equal variance
- Equal variance means the spread of values is about the same across the groups being compared.
- Robust test
- A robust test is a statistical method that still works fairly well when some assumptions are not perfectly met.
Common Mistakes to Avoid
- Checking normality on the raw data for every test is wrong because many tests require normal residuals or paired differences, not necessarily normal original values.
- Ignoring independence is wrong because dependence can make the effective sample size smaller and produce p-values that look too confident.
- Using the equal-variance t test when group spreads are very different is wrong because the standard error can be inaccurate, especially with unequal sample sizes.
- Treating a significant normality test as automatic failure is wrong because large samples can detect tiny, unimportant departures from normality, so plots and sample size should also be considered.
Practice Questions
- 1 Two groups have sample sizes n1 = 12 and n2 = 18, with sample standard deviations s1 = 4.0 and s2 = 9.0. Should you prefer the equal-variance t test or Welch's t test, and why?
- 2 A sample has standard deviation s = 15 and sample size n = 25. Calculate the standard error of the sample mean using SE = s/sqrt(n).
- 3 A class compares test scores from students sitting at the same tables, where students at each table studied together. Explain why independence may be questionable and name one possible remedy or alternative analysis.