Bayesian statistics is a way to update beliefs when new data arrives. Instead of treating probability only as long-run frequency, it also treats probability as a measure of uncertainty about a claim, parameter, or model. This matters in science, medicine, engineering, and artificial intelligence because decisions often must be made with incomplete information.
A Bayesian analysis makes the starting assumptions and the effect of evidence visible.
Understanding Statistics: Bayesian Statistics Introduction
A Bayesian calculation has three moving parts. First comes a set of possible explanations, such as several values for the chance that a coin lands heads. Next comes evidence, such as the results of fifty flips.
The calculation compares how expected those results would be under each possible value. Values that make the data more expected gain support. Values that make the data less expected lose support.
The final result is usually not one certain answer. It is a distribution showing which values remain plausible and how uncertain the estimate is.
The starting distribution matters most when evidence is limited. Imagine estimating the reliability of a new machine after only two tests. A prior based on similar machines can prevent a wild conclusion from a tiny sample.
As more tests arrive, the data usually have greater influence. This does not mean a prior can be chosen carelessly to force a desired result. Good work explains where it came from.
It may use past studies, physical limits, expert knowledge, or a deliberately broad range of possibilities. Analysts often test several reasonable priors to see whether the conclusion changes. If it changes a lot, the available evidence is weak for that conclusion.
Medical testing shows why this approach is useful. A positive test does not by itself tell a patient’s chance of having a disease. That chance depends on how common the disease was before the test, plus the test’s false positive and false negative rates.
For a rare disease, many people tested may be healthy, so even a very accurate test can produce positive results that need follow-up. The same reasoning appears in spam filters, weather forecasts, fault detection in factories, and court evidence. In each case, the evidence must be judged against the background rate of the event being considered.
Bayesian methods can be done with simple tables for a few hypotheses. Real problems often involve many unknown quantities, such as the effectiveness of a treatment across patients of different ages. Computers then use repeated random samples to approximate the posterior distribution.
Students should focus on the meaning of each step rather than memorising a formula. State the possible models clearly. Check whether the data were measured fairly.
Ask whether the likelihood represents the data-generating process well. Report uncertainty with the estimate, since a precise-looking number can hide large uncertainty. A Bayesian result is only as trustworthy as its assumptions, data, and model checks.
Key Facts
- Bayes' theorem: P(H|E) = P(E|H)P(H) / P(E)
- Posterior is proportional to likelihood times prior: P(H|E) ∝ P(E|H)P(H)
- Prior belief represents uncertainty before seeing the current evidence.
- Likelihood measures how well the observed evidence is explained by each possible hypothesis or parameter value.
- Posterior belief becomes the new prior if more evidence is collected later.
- For two hypotheses, posterior odds = prior odds × likelihood ratio.
Vocabulary
- Prior
- A prior is the probability distribution that represents what is believed before the current data are used.
- Likelihood
- A likelihood is the probability of observing the data for different possible hypotheses or parameter values.
- Posterior
- A posterior is the updated probability distribution after combining the prior with the likelihood from the evidence.
- Bayes' theorem
- Bayes' theorem is the rule that connects prior probability, evidence, and posterior probability.
- Likelihood ratio
- A likelihood ratio compares how much more likely the evidence is under one hypothesis than under another.
Common Mistakes to Avoid
- Confusing P(H|E) with P(E|H) is wrong because the probability of a hypothesis given evidence is not the same as the probability of evidence given a hypothesis.
- Ignoring the prior is wrong because Bayesian updating always combines the prior with the likelihood, even if the prior is chosen to be weak or broad.
- Treating the posterior as final truth is wrong because it is an updated state of uncertainty that can change when more evidence is added.
- Using a strong prior without justification is wrong because it can overpower the data and make the conclusion depend too much on an unsupported starting belief.
Practice Questions
- 1 A disease affects 2% of a population. A test is positive 95% of the time when a person has the disease and positive 10% of the time when a person does not have it. If a person tests positive, what is P(disease|positive)?
- 2 A machine is either well calibrated or poorly calibrated. Before inspection, P(well calibrated) = 0.70. A sample result is 4 times as likely if the machine is well calibrated than if it is poorly calibrated. Use posterior odds = prior odds × likelihood ratio to find the posterior probability that the machine is well calibrated.
- 3 Explain why two scientists can start with different priors but move toward similar posteriors after collecting a large amount of strong evidence.