The Law of Large Numbers is a central idea in statistics that explains why averages become more reliable when they are based on many trials. If you repeat the same random process again and again, the sample mean tends to move closer to the true expected value. This is why large surveys, repeated experiments, and long-run simulations are usually more trustworthy than small samples.
It matters because it connects random short-term variation with predictable long-term patterns.
In a convergence graph, the sample mean may jump around early because each new observation has a large effect when the sample is small. As the number of trials grows, each additional result changes the average less, so the curve stabilizes near the expected value. The Law of Large Numbers does not say that random outcomes will correct themselves after a streak.
That mistaken belief is the gambler's fallacy, which confuses long-run stability of averages with short-run compensation.
Key Facts
- Sample mean: x̄ = (x1 + x2 + ... + xn) / n
- Expected value: E(X) = Σ x P(x) for a discrete random variable
- Law of Large Numbers: as n increases, x̄ tends to approach μ = E(X)
- For a fair coin coded heads = 1 and tails = 0, E(X) = 0.5
- Early trials can produce large swings in x̄, but later trials usually cause smaller changes
- The Law of Large Numbers describes long-run convergence, not a guarantee for any short run
Vocabulary
- Law of Large Numbers
- A statistical principle stating that the sample mean tends to get closer to the expected value as the number of independent trials increases.
- Sample Mean
- The average of the observed values in a sample, found by adding the values and dividing by the number of observations.
- Expected Value
- The long-run average value of a random variable over many repeated trials.
- Independent Trials
- Trials are independent when the result of one trial does not change the probabilities of results on another trial.
- Gambler's Fallacy
- The mistaken belief that a random process is due to produce a certain outcome because of what happened in recent trials.
Common Mistakes to Avoid
- Expecting exact equality after many trials is wrong because the sample mean approaches the expected value but does not have to equal it exactly.
- Thinking a streak must be balanced immediately is wrong because independent trials do not remember past outcomes.
- Using too small a sample to estimate a probability is wrong because early random variation can strongly distort the sample mean.
- Confusing total counts with proportions is wrong because the Law of Large Numbers stabilizes relative frequencies and averages, not necessarily the difference between counts.
Practice Questions
- 1 A fair coin is flipped 100 times and heads occurs 57 times. If heads = 1 and tails = 0, what is the sample mean, and how far is it from the expected value?
- 2 A six-sided die is rolled 200 times. The sum of all rolls is 690. Find the sample mean and compare it with the expected value of a fair die, E(X) = 3.5.
- 3 A basketball player makes 80 percent of free throws. After missing 3 shots in a row, a fan says the next shot is more likely to go in because the player is due. Explain why this reasoning is the gambler's fallacy and how the Law of Large Numbers should be interpreted instead.