A mixture distribution describes data that come from two or more sub-populations rather than from one single source. It matters because many real data sets contain hidden groups, such as heights from different age groups, commute times from drivers and train riders, or test scores from students with different preparation levels. When these groups are combined, the overall distribution can have multiple peaks, skew, or unusually wide spread.
Recognizing a mixture helps prevent misleading summaries and poor predictions.
Key Facts
- A finite mixture density is f(x) = w1 f1(x) + w2 f2(x) + ... + wk fk(x).
- Mixture weights must satisfy wi >= 0 and w1 + w2 + ... + wk = 1.
- For a two-component mixture, f(x) = w f1(x) + (1 - w) f2(x).
- The mixture mean is E[X] = w1 μ1 + w2 μ2 + ... + wk μk.
- The mixture variance is Var(X) = Σ wi[σi^2 + (μi - μ)^2], where μ is the mixture mean.
- A mixture distribution can be multimodal, but separate components do not always create separate visible peaks.
Vocabulary
- Mixture distribution
- A probability distribution formed by combining two or more component distributions using weights.
- Component distribution
- One sub-population distribution that contributes to the overall mixture.
- Mixture weight
- The proportion or probability assigned to a component distribution in the mixture.
- Multimodal
- Having more than one peak or local maximum in a distribution.
- Latent variable
- An unobserved variable that indicates which component or group generated a data value.
Common Mistakes to Avoid
- Averaging component means without weights is wrong because larger sub-populations contribute more to the mixture mean.
- Assuming every mixture has multiple peaks is wrong because overlapping components can blend into one visible peak.
- Treating the combined curve as a single normal distribution can be wrong because mixtures may have skew, heavy tails, or several modes.
- Using weights that do not add to 1 is wrong because mixture weights represent probabilities or proportions.
Practice Questions
- 1 A data set is modeled as a mixture of two normal groups: 70% have mean 10 and 30% have mean 20. What is the mixture mean?
- 2 A two-component mixture has f(x) = 0.4 f1(x) + 0.6 f2(x), with μ1 = 5, σ1 = 1, μ2 = 9, and σ2 = 2. Compute the mixture mean and variance.
- 3 Two sub-populations have normal distributions with the same standard deviation but means that are very close together. Explain why the combined mixture may show only one peak instead of two.