Bayesian updating is a method for changing a probability when new evidence arrives. It is useful because many real decisions begin with incomplete information, such as estimating whether a patient has a disease before and after a test result. The starting belief is called the prior, and the updated belief after evidence is called the posterior.
This process helps students see probability as a rational way to learn from data.
The central rule combines the prior probability with the likelihood of observing the evidence under different possibilities. In a medical test example, a rare disease can still have a low posterior probability after a positive result if false positives are common. Bayesian updating works by comparing how well each hypothesis predicts the evidence, then normalizing the results so the probabilities add to 1.
The same idea is used in science, medicine, machine learning, weather forecasting, and everyday reasoning.
Key Facts
- Bayes' theorem: P(H|E) = P(E|H)P(H) / P(E)
- Prior: P(H) is the probability of a hypothesis before using the new evidence.
- Likelihood: P(E|H) is the probability of seeing the evidence if the hypothesis is true.
- Posterior: P(H|E) is the updated probability of the hypothesis after using the evidence.
- Total probability: P(E) = P(E|H)P(H) + P(E|not H)P(not H) for two cases.
- Odds form: posterior odds = prior odds × likelihood ratio.
Vocabulary
- Prior
- A prior is the probability assigned to a hypothesis before considering the new evidence.
- Likelihood
- A likelihood is the probability of observing the evidence assuming a particular hypothesis is true.
- Posterior
- A posterior is the updated probability of a hypothesis after the evidence has been included.
- Base rate
- A base rate is the overall frequency of an event in the population before using case-specific evidence.
- Likelihood ratio
- A likelihood ratio compares how probable the same evidence is under one hypothesis versus another.
Common Mistakes to Avoid
- Confusing P(H|E) with P(E|H). These are different probabilities because the probability of a hypothesis given evidence is not usually the same as the probability of evidence given the hypothesis.
- Ignoring the base rate. A positive test for a rare disease may still leave a low posterior probability if the disease is very uncommon.
- Forgetting to include false positives or false negatives. Test accuracy must include both what happens when the condition is present and what happens when it is absent.
- Failing to normalize probabilities. After multiplying priors by likelihoods, divide by the total probability of the evidence so all posterior probabilities add to 1.
Practice Questions
- 1 A disease affects 2% of a population. A test is positive 95% of the time for people with the disease and positive 10% of the time for people without it. If a person tests positive, what is P(disease|positive)?
- 2 A box is chosen at random. Box A has 80 red and 20 blue marbles, and Box B has 30 red and 70 blue marbles. Before drawing, each box is equally likely. If one red marble is drawn, what is the posterior probability that the box is A?
- 3 A student believes there is a 50% chance that a coin is biased toward heads. After observing 8 heads in 10 flips, explain qualitatively how the posterior should change and why the prior still matters.