Bayesian statistics is a way to update beliefs when new data arrives. Instead of treating probability only as long-run frequency, it also treats probability as a measure of uncertainty about a claim, parameter, or model. This matters in science, medicine, engineering, and artificial intelligence because decisions often must be made with incomplete information.
A Bayesian analysis makes the starting assumptions and the effect of evidence visible.
Key Facts
- Bayes' theorem: P(H|E) = P(E|H)P(H) / P(E)
- Posterior is proportional to likelihood times prior: P(H|E) ∝ P(E|H)P(H)
- Prior belief represents uncertainty before seeing the current evidence.
- Likelihood measures how well the observed evidence is explained by each possible hypothesis or parameter value.
- Posterior belief becomes the new prior if more evidence is collected later.
- For two hypotheses, posterior odds = prior odds × likelihood ratio.
Vocabulary
- Prior
- A prior is the probability distribution that represents what is believed before the current data are used.
- Likelihood
- A likelihood is the probability of observing the data for different possible hypotheses or parameter values.
- Posterior
- A posterior is the updated probability distribution after combining the prior with the likelihood from the evidence.
- Bayes' theorem
- Bayes' theorem is the rule that connects prior probability, evidence, and posterior probability.
- Likelihood ratio
- A likelihood ratio compares how much more likely the evidence is under one hypothesis than under another.
Common Mistakes to Avoid
- Confusing P(H|E) with P(E|H) is wrong because the probability of a hypothesis given evidence is not the same as the probability of evidence given a hypothesis.
- Ignoring the prior is wrong because Bayesian updating always combines the prior with the likelihood, even if the prior is chosen to be weak or broad.
- Treating the posterior as final truth is wrong because it is an updated state of uncertainty that can change when more evidence is added.
- Using a strong prior without justification is wrong because it can overpower the data and make the conclusion depend too much on an unsupported starting belief.
Practice Questions
- 1 A disease affects 2% of a population. A test is positive 95% of the time when a person has the disease and positive 10% of the time when a person does not have it. If a person tests positive, what is P(disease|positive)?
- 2 A machine is either well calibrated or poorly calibrated. Before inspection, P(well calibrated) = 0.70. A sample result is 4 times as likely if the machine is well calibrated than if it is poorly calibrated. Use posterior odds = prior odds × likelihood ratio to find the posterior probability that the machine is well calibrated.
- 3 Explain why two scientists can start with different priors but move toward similar posteriors after collecting a large amount of strong evidence.