Bayesian and frequentist statistics are two major ways to use data to make conclusions under uncertainty. This cheat sheet compares how each approach treats probability, parameters, evidence, and conclusions. Students need it because the same data can lead to different wording and reasoning depending on the statistical framework.
Understanding the comparison helps students interpret studies, experiments, polls, and statistical claims more carefully.
Frequentist methods treat parameters as fixed but unknown values and use repeated sampling to describe uncertainty. Bayesian methods treat parameters as uncertain and update beliefs using Bayes' theorem, . Key comparisons include confidence intervals versus credible intervals, -values versus posterior probabilities, and likelihood-based evidence versus prior-updated evidence.
Both approaches use data, models, and assumptions, but they answer different statistical questions.
Key Facts
- Bayes' theorem is , where is a hypothesis and is the observed data.
- In frequentist statistics, an unknown parameter such as is fixed, and uncertainty comes from the random sample.
- In Bayesian statistics, an unknown parameter such as is treated as uncertain, and a probability distribution describes beliefs about its possible values.
- A frequentist confidence interval has the form , where is an estimate and is its standard error.
- A Bayesian posterior distribution is proportional to likelihood times prior, written as .
- A confidence interval means that, over many repeated samples, about of intervals made by the same method would contain the true parameter.
- A credible interval means that, given the model, data, and prior, there is posterior probability that lies in the interval.
- A frequentist -value is , not the probability that is true.
Vocabulary
- Prior distribution
- A probability distribution, written , that represents beliefs about a parameter before observing the current data.
- Likelihood
- The likelihood, written , measures how compatible the observed data are with different parameter values .
- Posterior distribution
- The posterior distribution, written , combines the prior and likelihood to describe updated beliefs after seeing the data.
- Confidence interval
- A confidence interval is a frequentist interval estimate built by a method that captures the true parameter in a stated proportion of repeated samples.
- Credible interval
- A credible interval is a Bayesian interval that contains a stated amount of posterior probability for a parameter.
- P-value
- A -value is the probability, assuming is true, of getting results at least as extreme as the observed results.
Common Mistakes to Avoid
- Saying a confidence interval has a chance of containing the true parameter is wrong because, in frequentist statistics, the parameter is fixed and the interval is random.
- Treating a -value as is wrong because a -value is calculated as .
- Ignoring the prior in a Bayesian analysis is wrong because the posterior depends on both the prior and the likelihood .
- Comparing a credible interval and a confidence interval as if they mean the same thing is wrong because they answer different questions about uncertainty.
- Assuming Bayesian methods are always subjective and frequentist methods are always objective is wrong because both approaches require modeling choices and assumptions.
Practice Questions
- 1 A diagnostic test has , , and . Use Bayes' theorem to find .
- 2 A sample has and . Using , compute the approximate confidence interval .
- 3 A Bayesian analysis gives a posterior distribution with a credible interval of for . Write one correct sentence interpreting this interval.
- 4 A study reports a -value of . Explain what this means in frequentist terms and why it does not prove that the alternative hypothesis is true.
Understanding Bayesian vs Frequentist Comparison
A useful way to see the difference is to imagine estimating the proportion of students who support a school policy. Both methods begin with a sample, such as a survey of two hundred students. The likelihood describes which population proportions would make the observed survey results more or less expected.
A frequentist analysis uses that sample to build a procedure with known long-run behavior. A Bayesian analysis combines the sample evidence with a prior distribution.
The prior can come from earlier surveys, past experiments, expert knowledge, or a deliberately broad starting assumption. After new data arrive, the prior is updated into a posterior distribution.
The role of the prior deserves careful attention. With a large, well collected sample, reasonable priors often have little effect because the data provide strong evidence. With a small sample or noisy measurements, the prior can matter a great deal.
This is not automatically a weakness. Previous knowledge can be relevant, especially in medicine, weather forecasting, and quality control. It can become a problem when a prior is chosen without a clear reason or is selected to favor a preferred conclusion.
Good Bayesian work states the prior openly and checks whether the conclusion changes under other sensible priors. This practice is called sensitivity analysis.
Frequentist results require equally careful interpretation. A p-value measures how unusual the observed result would be if a chosen null hypothesis were correct. A small p-value may count as evidence against that null hypothesis, but it does not tell how large an effect is, whether the effect matters in real life, or whether a result will repeat.
Sample size strongly affects p-values. A tiny difference can look statistically significant in a huge study.
A meaningful difference may fail to reach a common cutoff in a small study. Students should examine the estimated effect, its uncertainty, the sample size, and how the data were collected before focusing on one number.
Intervals often cause the most confusion. A confidence interval describes the reliability of an interval-making method over repeated samples. It does not assign a probability to the one fixed parameter after the data have been observed.
A credible interval does assign probability across possible parameter values, but only within the stated Bayesian model and prior. In both approaches, an interval can be misleading if the sample is biased, the measurements are poor, or the model assumptions do not fit the situation. A poll with voluntary responses may have a narrow interval yet still miss the true population view because its respondents are not representative.
In school science, frequentist methods appear often in experiments that compare groups, test a claimed effect, or report margins of error. Bayesian reasoning appears naturally when evidence is updated over time, such as revising the chance of rain after new radar images or judging a medical test result using the disease rate in a population.
Neither framework removes the need for judgment. The most important habits are defining the population clearly, checking how observations were selected, stating assumptions, and matching the conclusion to what the method actually supports.