Choosing the right hypothesis test helps you turn data into a fair statistical decision. The best test depends on the type of response variable, the number of groups, and whether the groups are independent or paired. A flowchart is useful because it starts with simple questions and narrows the choices step by step.
This prevents using a test that does not match the study design or the kind of data collected.
For numerical data, t-tests compare means for one or two groups, while ANOVA compares means across three or more groups. For categorical data, chi-square tests often examine counts in categories, while proportion tests compare success rates. Paired designs, such as before-and-after measurements on the same people, require tests that account for the link between observations.
The goal is always to match the test statistic and assumptions to the structure of the data.
Understanding Statistics: Choosing the Right Hypothesis Test
A hypothesis test begins before any calculation. First, identify the observational unit. It may be one student, one plant, one machine, or one measurement taken on a particular day.
Then identify the response, which is the outcome recorded for each unit. A test about average height needs numerical measurements. A test about whether students passed needs category counts.
The explanatory variable describes the groups or conditions being compared. Clear variables prevent a common mistake, which is treating labels, rankings, or repeated measurements as if they were separate independent numbers.
Independence is one of the most important ideas in the flowchart. Independent observations come from units whose outcomes do not directly determine each other. For example, results from two separately selected classes may be treated as independent if no student appears in both samples.
Paired observations have a built-in connection. A student measured before and after a training program produces a pair. So do twins, matched patients, or products tested under two settings.
In paired work, the useful data are the changes within pairs. Ignoring the connection usually adds noise and can hide a real pattern.
Every test relies on assumptions. Tests for means work best when the data within each group have no extreme outliers and are roughly balanced in shape, especially with small samples. Larger samples often reduce the impact of mild skewness, but they do not fix poor sampling.
Tests involving proportions or category counts need enough expected observations in each category. A tiny expected count can make the usual approximation unreliable. Graphs should come before formal testing.
Dot plots, histograms, box plots, and bar charts can reveal outliers, gaps, uneven spread, or categories with very few cases. Students should learn to inspect the raw pattern rather than trusting a calculator output alone.
The null hypothesis represents a baseline claim, such as no average change or no association between two categories. The alternative states the pattern the study is looking for. A p-value measures how unusual the observed result would be if the null claim were true.
It does not give the probability that the null claim is true. A small p-value is evidence against the null, though it does not prove a cause. Statistical significance can occur for a very small difference when the sample is large.
Report the size of the difference and a confidence interval when possible. If several group comparisons are made after an overall analysis, the chance of a false positive rises. Planned comparisons and adjustment methods help control that risk.
Key Facts
- One-sample t-test: tests whether a population mean differs from a hypothesized value, using t = (x̄ - μ0) / (s / √n).
- Two-sample t-test: compares the means of two independent groups, such as treatment versus control.
- Paired t-test: compares the mean of paired differences, using d̄ as the average difference within matched pairs.
- One-way ANOVA: compares means across three or more independent groups, using F = variation between groups / variation within groups.
- Chi-square test of independence: tests whether two categorical variables are associated, using χ² = Σ((O - E)² / E).
- Proportion tests: compare categorical success rates, such as one-proportion or two-proportion z-tests, using z = (p̂ - p0) / SE.
Vocabulary
- Hypothesis test
- A statistical procedure that uses sample data to evaluate a claim about a population.
- Response variable
- The outcome variable being measured or classified in a study.
- Independent groups
- Groups are independent when the observations in one group are not naturally linked to observations in another group.
- Paired data
- Paired data occur when two measurements are linked, such as measurements from the same person before and after a treatment.
- Categorical data
- Categorical data place observations into groups or labels, such as yes or no, red or blue, or treatment type.
Common Mistakes to Avoid
- Using a t-test for three or more groups: this is wrong because multiple pairwise t-tests increase the chance of a false positive, so a one-way ANOVA is usually the correct first test.
- Treating paired data as independent: this is wrong because matched observations share information, and ignoring the pairing can hide real effects or distort uncertainty.
- Using a chi-square test on means: this is wrong because chi-square tests are for counts in categories, not numerical averages.
- Choosing a test before identifying the response variable type: this is wrong because numerical responses, categorical responses, and counts often require different test families.
Practice Questions
- 1 A teacher compares the mean exam score of one class to a known district average of 75. The class has n = 30 students, x̄ = 78, and s = 6. Which hypothesis test should be used, and what is the test statistic?
- 2 A researcher compares recovery times for patients in 4 different treatment groups. The response is measured in days, and the groups are independent. Which test should be used to compare the mean recovery times?
- 3 A survey records whether students prefer online or in-person classes and whether they are freshmen, sophomores, juniors, or seniors. Explain which test should be used and why the data type matters.