Bayes Theorem

Updating Beliefs with Evidence

Download PNG Save to Pinterest

Related Tools

Related Labs

Bayesian Inference Lab

Related Worksheets

Related Cheat Sheets

Bayes Theorem is a rule for updating probabilities when new evidence appears. It connects what you believed before seeing the evidence to what you should believe after seeing it. This idea matters in medicine, machine learning, weather forecasting, and everyday decision making. It helps students move beyond guessing and toward reasoning with conditional probability.

The theorem combines three pieces: a prior probability, a likelihood, and the overall probability of the evidence. In symbols, $P(A|B) = \frac{P(B|A)P(A)}{P(B)}$ . The numerator measures how well hypothesis $A$ explains evidence $B$ , while the denominator rescales the result so the final probability stays between $0$ and $1$ . A common application is interpreting test results, where the chance of actually having a condition depends not only on the test accuracy but also on how common the condition is.

Key Facts

Bayes Theorem: $P(A|B) = \frac{P(B|A)P(A)}{P(B)}$
Conditional probability: $P(A|B) = \frac{P(A \text{ and } B)}{P(B)}$ , for $P(B) > 0$
Evidence can be expanded as $P(B) = P(B|A)P(A) + P(B|\text{not } A)P(\text{not } A)$
Posterior = likelihood $\times$ prior / evidence
If $A$ and $B$ are independent, then $P(A|B) = P(A)$ , so observing $B$ does not change belief about $A$
Base rate matters: a rare event can still have a low posterior probability even after a positive test

Vocabulary

Prior probability: The probability assigned to a hypothesis before new evidence is observed.
Posterior probability: The updated probability of a hypothesis after taking the evidence into account.
Likelihood: The probability of observing the evidence if a particular hypothesis is true.
Conditional probability: The probability of one event occurring given that another event has already occurred.
Base rate: The overall frequency or prevalence of an event in the population before considering specific evidence.

Common Mistakes to Avoid

Confusing P(A|B) with P(B|A), because these probabilities describe different conditions and are usually not equal. Always read the condition after the vertical bar carefully.
Ignoring the base rate, which leads students to overestimate the chance that a positive result means the hypothesis is true. The prior probability must be included in the calculation.
Forgetting to compute P(B), because the denominator is needed to normalize the result into a valid probability. Use all ways the evidence could happen, not just the favored hypothesis.
Using percentages and decimals inconsistently, because mixing forms can cause arithmetic errors. Convert all values to the same form before substituting into the formula.

Practice Questions

1 A disease affects 2% of a population. A test is positive 95% of the time when a person has the disease and 8% of the time when a person does not have it. If a person tests positive, what is the probability that the person actually has the disease?
2 A factory makes 60% of its parts on Machine A and 40% on Machine B. Machine A produces defective parts 3% of the time, and Machine B produces defective parts 6% of the time. If a randomly chosen part is defective, what is the probability it came from Machine B?
3 Explain why a highly accurate test can still produce many false alarms when the condition being tested is very rare. Use the ideas of prior probability and evidence in your explanation.