Maximum likelihood estimation, or MLE, is a method for estimating an unknown parameter from observed data. It asks which parameter value would make the data we actually saw most probable. This idea matters because it is used throughout statistics, machine learning, physics, biology, and economics.
The result is called the maximum likelihood estimate and is usually written as θ̂.
Key Facts
- Likelihood means treating the observed data as fixed and the parameter as the variable.
- For independent data, L(θ) = P(x1 | θ)P(x2 | θ)...P(xn | θ).
- The maximum likelihood estimate is θ̂ = arg max L(θ).
- It is often easier to maximize the log-likelihood: ℓ(θ) = ln L(θ).
- For independent data, ℓ(θ) = ln P(x1 | θ) + ln P(x2 | θ) + ... + ln P(xn | θ).
- For a coin with h heads in n flips, the MLE for the probability of heads is p̂ = h/n.
Vocabulary
- Parameter
- A parameter is an unknown number in a statistical model, such as a mean, probability, or rate.
- Likelihood function
- A likelihood function gives how plausible different parameter values are after the data have been observed.
- Maximum likelihood estimate
- A maximum likelihood estimate is the parameter value that gives the largest likelihood for the observed data.
- Log-likelihood
- The log-likelihood is the natural logarithm of the likelihood function, often used because sums are easier to work with than products.
- Independent observations
- Independent observations are data points whose probabilities can be multiplied because one observation does not affect another.
Common Mistakes to Avoid
- Confusing likelihood with probability is wrong because likelihood varies the parameter after the data are fixed, while probability usually varies possible data for a fixed parameter.
- Forgetting to multiply probabilities for independent observations is wrong because the likelihood of the whole data set must combine all observations, not just one data point.
- Maximizing the likelihood instead of the log-likelihood by hand can lead to unnecessary algebra errors because the log-likelihood has the same maximum but is usually simpler.
- Assuming the highest point of a plotted curve is always meaningful without checking the model is wrong because MLE depends on choosing an appropriate statistical model for the data.
Practice Questions
- 1 A coin is flipped 20 times and lands heads 13 times. Using maximum likelihood estimation, what is p̂ for the probability of heads?
- 2 A biased die has probability θ of rolling a 6. In 50 rolls, a 6 appears 8 times. Write the likelihood L(θ) up to a constant factor and find the MLE θ̂.
- 3 Two models give different likelihood curves for the same data. Model A has a sharp high peak, while Model B has a lower but wider peak. Explain what the peak location tells you and why the width of the curve matters for uncertainty.