The Law of Large Numbers is a central idea in statistics that explains why averages become more reliable when they are based on many trials. If you repeat the same random process again and again, the sample mean tends to move closer to the true expected value. This is why large surveys, repeated experiments, and long-run simulations are usually more trustworthy than small samples.
It matters because it connects random short-term variation with predictable long-term patterns.
In a convergence graph, the sample mean may jump around early because each new observation has a large effect when the sample is small. As the number of trials grows, each additional result changes the average less, so the curve stabilizes near the expected value. The Law of Large Numbers does not say that random outcomes will correct themselves after a streak.
That mistaken belief is the gambler's fallacy, which confuses long-run stability of averages with short-run compensation.
Understanding Statistics: The Law of Large Numbers
The key idea comes from cancellation. Individual results can sit above or below the value expected in the long run. A few unusual results can pull an early average far from its usual level.
With more observations, high and low departures tend to balance each other more effectively. They do not balance in a neat pattern or on a fixed schedule. Instead, the total error is spread across a growing number of observations.
A single extra result has weight one divided by the number of trials in the average. After ten trials, one result has much more influence than it has after one thousand trials.
Statistics describes this shrinking uncertainty with a quantity called the standard error. For many common situations, the typical size of the error in a sample average decreases roughly in proportion to one divided by the square root of the sample size. This has an important consequence.
Making a sample four times as large cuts the typical error by about half, not by four. To make an estimate much more precise, researchers may need far more data than students first expect. This is why a poll based on four hundred people is useful but still has noticeable uncertainty, while a poll based on four thousand people is usually more stable if both samples are collected fairly.
The law needs suitable conditions. The observations should come from the same underlying process, with an expected value that stays meaningful over the period studied. They should be independent, or close enough to independent for the dependence not to dominate the result.
Repeated measurements from one person, for example, do not provide the same information as measurements from many unrelated people. A large biased sample remains biased. If a school survey is answered mainly by students from one friendship group, adding thousands more answers from that group does not reveal the view of the whole school.
More data reduces random noise. It does not fix poor sampling, faulty measuring tools, missing groups, or a changing population.
Students meet this principle whenever they estimate a probability from repeated trials. A simulation of rolling dice becomes more informative when it runs many times. A factory checks items from a production line to estimate the defect rate.
Weather services combine many observations because any single reading can be noisy. In science practical work, repeated measurements help reduce the effect of small reading errors. When studying graphs, pay attention to the scale of the horizontal axis.
A line that looks settled may still move enough to matter for the decision being made. It is useful to compare several separate runs, not just one. Different runs can follow different paths, even when they all become reasonably close to the same long run value.
Key Facts
- Sample mean: x̄ = (x1 + x2 + ... + xn) / n
- Expected value: E(X) = Σ x P(x) for a discrete random variable
- Law of Large Numbers: as n increases, x̄ tends to approach μ = E(X)
- For a fair coin coded heads = 1 and tails = 0, E(X) = 0.5
- Early trials can produce large swings in x̄, but later trials usually cause smaller changes
- The Law of Large Numbers describes long-run convergence, not a guarantee for any short run
Vocabulary
- Law of Large Numbers
- A statistical principle stating that the sample mean tends to get closer to the expected value as the number of independent trials increases.
- Sample Mean
- The average of the observed values in a sample, found by adding the values and dividing by the number of observations.
- Expected Value
- The long-run average value of a random variable over many repeated trials.
- Independent Trials
- Trials are independent when the result of one trial does not change the probabilities of results on another trial.
- Gambler's Fallacy
- The mistaken belief that a random process is due to produce a certain outcome because of what happened in recent trials.
Common Mistakes to Avoid
- Expecting exact equality after many trials is wrong because the sample mean approaches the expected value but does not have to equal it exactly.
- Thinking a streak must be balanced immediately is wrong because independent trials do not remember past outcomes.
- Using too small a sample to estimate a probability is wrong because early random variation can strongly distort the sample mean.
- Confusing total counts with proportions is wrong because the Law of Large Numbers stabilizes relative frequencies and averages, not necessarily the difference between counts.
Practice Questions
- 1 A fair coin is flipped 100 times and heads occurs 57 times. If heads = 1 and tails = 0, what is the sample mean, and how far is it from the expected value?
- 2 A six-sided die is rolled 200 times. The sum of all rolls is 690. Find the sample mean and compare it with the expected value of a fair die, E(X) = 3.5.
- 3 A basketball player makes 80 percent of free throws. After missing 3 shots in a row, a fan says the next shot is more likely to go in because the player is due. Explain why this reasoning is the gambler's fallacy and how the Law of Large Numbers should be interpreted instead.