Bayesian updating is a method for changing a probability when new evidence arrives. It is useful because many real decisions begin with incomplete information, such as estimating whether a patient has a disease before and after a test result. The starting belief is called the prior, and the updated belief after evidence is called the posterior.
This process helps students see probability as a rational way to learn from data.
The central rule combines the prior probability with the likelihood of observing the evidence under different possibilities. In a medical test example, a rare disease can still have a low posterior probability after a positive result if false positives are common. Bayesian updating works by comparing how well each hypothesis predicts the evidence, then normalizing the results so the probabilities add to 1.
The same idea is used in science, medicine, machine learning, weather forecasting, and everyday reasoning.
Understanding Statistics: Bayesian Updating
One reliable way to understand an update is to imagine a large group rather than starting with a formula. Suppose a disease affects one hundred out of every ten thousand people. A test detects ninety of those one hundred people.
This sounds strong. Now suppose the test gives a positive result to five percent of healthy people. Among the nine thousand nine hundred healthy people, that produces about four hundred ninety five positive results.
There are then about five hundred eighty five positive results in total, but only ninety come from people with the disease. A positive test therefore raises concern, yet the chance of disease is only about fifteen in one hundred. The starting rate in the population matters greatly when a condition is rare.
This counting method shows why the probability of the evidence is important. A positive result can arise through more than one route. It may come from a person who has the disease, or from a healthy person receiving a false positive.
The updated belief depends on the share of all positive results that belong to each route. Tables with rows for the possible conditions and columns for possible test results make this visible. They are often easier to use than symbols at first.
They prevent a common error called reversing the conditional. The chance of a positive test among sick people is not the same as the chance of being sick after a positive test.
Updating can happen many times as information arrives. A doctor may begin with a population rate, then include symptoms, family history, an examination, and test results. Each new piece of evidence should change the current estimate rather than restart the whole calculation.
This only works cleanly when the pieces of evidence are reasonably independent after the hypothesis is known. Two blood tests that measure nearly the same chemical may give similar information.
Treating them as fully separate can count the same clue twice and create too much confidence. Good models pay attention to where data came from, how it was measured, and whether the people in a study resemble the person being assessed.
Bayesian thinking appears outside medicine. An email filter weighs words, links, and sender history before marking a message as spam. Weather forecasts combine computer models with fresh observations.
A student uses it informally when revising a belief after seeing reliable evidence. In every setting, the quality of the update depends on the quality of the assumptions. The prior should come from relevant data when possible, not a personal guess presented as fact.
Likelihoods should be checked against real outcomes. A probability is not a promise about one event. It describes uncertainty across similar cases.
When learning this topic, state the hypothesis clearly, list realistic alternatives, and ask whether the evidence would be common or rare under each alternative. These habits matter more than memorising a rule.
Key Facts
- Bayes' theorem: P(H|E) = P(E|H)P(H) / P(E)
- Prior: P(H) is the probability of a hypothesis before using the new evidence.
- Likelihood: P(E|H) is the probability of seeing the evidence if the hypothesis is true.
- Posterior: P(H|E) is the updated probability of the hypothesis after using the evidence.
- Total probability: P(E) = P(E|H)P(H) + P(E|not H)P(not H) for two cases.
- Odds form: posterior odds = prior odds × likelihood ratio.
Vocabulary
- Prior
- A prior is the probability assigned to a hypothesis before considering the new evidence.
- Likelihood
- A likelihood is the probability of observing the evidence assuming a particular hypothesis is true.
- Posterior
- A posterior is the updated probability of a hypothesis after the evidence has been included.
- Base rate
- A base rate is the overall frequency of an event in the population before using case-specific evidence.
- Likelihood ratio
- A likelihood ratio compares how probable the same evidence is under one hypothesis versus another.
Common Mistakes to Avoid
- Confusing P(H|E) with P(E|H). These are different probabilities because the probability of a hypothesis given evidence is not usually the same as the probability of evidence given the hypothesis.
- Ignoring the base rate. A positive test for a rare disease may still leave a low posterior probability if the disease is very uncommon.
- Forgetting to include false positives or false negatives. Test accuracy must include both what happens when the condition is present and what happens when it is absent.
- Failing to normalize probabilities. After multiplying priors by likelihoods, divide by the total probability of the evidence so all posterior probabilities add to 1.
Practice Questions
- 1 A disease affects 2% of a population. A test is positive 95% of the time for people with the disease and positive 10% of the time for people without it. If a person tests positive, what is P(disease|positive)?
- 2 A box is chosen at random. Box A has 80 red and 20 blue marbles, and Box B has 30 red and 70 blue marbles. Before drawing, each box is equally likely. If one red marble is drawn, what is the posterior probability that the box is A?
- 3 A student believes there is a 50% chance that a coin is biased toward heads. After observing 8 heads in 10 flips, explain qualitatively how the posterior should change and why the prior still matters.