Surveys are useful only when the people measured are a good match for the population being studied. Two common threats are undercoverage bias and nonresponse bias, both of which make some groups count less than they should. These biases can shift results even when the sample size is large.
Learning to spot them helps students judge polls, research studies, and data reports more carefully.
Undercoverage happens before data collection when the sampling frame leaves out part of the target population. Nonresponse happens after people are selected when some sampled individuals do not answer or cannot be reached. Bias becomes serious when the missing people differ in important ways from the people included.
Researchers reduce these problems by improving the sampling frame, using repeated contact attempts, offering multiple response modes, and weighting results when justified.
Understanding Statistics: Nonresponse and Undercoverage Bias
Missing data changes an estimate because survey results are built from the answers that arrive, not from the answers that were hoped for. Imagine a school survey about time spent on homework. If students who have heavy jobs after school are less available to respond, the completed surveys may come mostly from students with more free time.
The reported average can then be too low. The problem is not simply that fewer answers were collected.
It is that the pattern of missing answers is connected to the subject being measured. A sample of ten thousand can still give a misleading result when one kind of person is consistently absent.
Undercoverage often comes from the way a list is made. An online survey link shared through a school learning platform reaches students who use that platform, yet it may miss students who have limited internet access or who are absent often. A poll based only on landline numbers misses many households that use mobile phones only.
A study of public transport riders conducted at midday may leave out commuters who travel early in the morning or late at night. Every method has boundaries. Students should identify who had a real chance to appear in the sample, then compare that group with the group the study claims to describe.
Nonresponse has several causes. People may be busy, distrust the sender, find questions too personal, lack the language support needed, or never receive the invitation. Follow up messages can raise the response rate, but they do not automatically remove bias.
People who answer after several reminders can differ from people who answer immediately. People who never answer may differ more.
Researchers sometimes compare early respondents with late respondents, or compare known features of respondents with population records. For instance, if a town knows the age distribution of its residents, it can check whether young adults are represented too little in the final data.
Weighting is one tool for adjusting a sample after collection. If young adults make up one fifth of a population but only one tenth of respondents, each young adult response may be given more influence in the estimate. This can help when the adjustment groups are known and the people within each group are reasonably similar.
It cannot fully repair a group that is absent, or fix missing differences that were never measured. When reading a survey report, pay attention to how people were contacted, who could not be reached, how many selected people responded, and whether results were adjusted. Those details reveal how much confidence the numbers deserve.
Key Facts
- Target population = the full group the study wants to describe.
- Sampling frame = the list or method used to identify people who can be sampled.
- Undercoverage bias occurs when some members of the target population have little or no chance of being selected.
- Nonresponse rate = number of sampled nonrespondents / total number sampled.
- Response rate = number of respondents / total number sampled.
- Bias means the method tends to overestimate or underestimate the true population value in a systematic way.
Vocabulary
- Target population
- The complete group of individuals or objects that a statistical study is trying to learn about.
- Sampling frame
- The actual list, map, database, or procedure used to choose members of the sample.
- Undercoverage bias
- A systematic error caused when some groups in the target population are missing or poorly represented in the sampling frame.
- Nonresponse bias
- A systematic error caused when selected individuals who do not respond differ from those who do respond.
- Response rate
- The fraction or percentage of sampled individuals who provide usable data.
Common Mistakes to Avoid
- Confusing undercoverage with nonresponse: undercoverage happens when people cannot be selected in the first place, while nonresponse happens after people are selected but do not answer.
- Assuming a large sample removes bias: a huge sample can still be biased if it misses an important group or if nonrespondents differ from respondents.
- Using only the easiest people to reach: convenience contact methods often exclude people with different schedules, technology access, income levels, or opinions.
- Reporting only the number of responses: the response rate and the sampling method are needed to judge whether the results may be biased.
Practice Questions
- 1 A city wants to estimate support for a new bus route and calls 1,000 landline phone numbers. Only 620 people answer. What is the response rate, and name one possible source of undercoverage.
- 2 A school surveys 500 randomly selected students about cafeteria food, but 125 selected students do not complete the survey. What is the nonresponse rate? If many nonrespondents skip lunch at school, explain how the estimate could be biased.
- 3 A researcher uses an online survey to study the opinions of all adults in a rural county. Explain whether undercoverage, nonresponse, or both may be present, and describe one improvement to reduce the bias.