Survey bias happens when a survey method systematically pushes results away from the truth. This cheat sheet helps students identify the most common types of bias before interpreting survey conclusions. It is useful for statistics problems involving polls, questionnaires, sampling plans, and real-world claims.
Recognizing bias helps students decide whether survey results are trustworthy.
The core idea is that a good survey should represent the population and collect honest, clear responses. Important concepts include selection bias, undercoverage, nonresponse bias, voluntary response bias, response bias, and wording bias. Students should also know that random sampling reduces bias but does not automatically fix poor question wording or low response rates.
A useful measure is response rate, given by .
Key Facts
- Selection bias occurs when the sampling method favors some members of the population over others.
- Undercoverage is a type of selection bias where part of the population has little or no chance of being included.
- Voluntary response bias occurs when people choose whether to participate, often causing strong opinions to be overrepresented.
- Nonresponse bias occurs when selected people do not respond and the nonresponders differ from responders in an important way.
- Response bias occurs when answers are inaccurate because people lie, forget, feel pressured, or misunderstand the question.
- Wording bias occurs when a question is leading, confusing, emotionally loaded, or suggests a preferred answer.
- The response rate is .
- A simple random sample gives every group of size from a population of size an equal chance of being selected.
Vocabulary
- Population
- The population is the entire group of individuals or objects that a survey wants to describe.
- Sample
- A sample is the smaller group selected from the population to provide data.
- Selection Bias
- Selection bias is a systematic error caused by choosing a sample that does not fairly represent the population.
- Nonresponse Bias
- Nonresponse bias happens when people chosen for the survey do not respond and their missing answers would change the results.
- Response Bias
- Response bias happens when the answers given are inaccurate because of pressure, memory errors, dishonesty, or misunderstanding.
- Wording Bias
- Wording bias happens when the wording of a question influences people toward a particular answer.
Common Mistakes to Avoid
- Calling every bad survey random error is wrong because bias is systematic and tends to push results in a consistent direction.
- Assuming a large sample removes bias is wrong because a large biased sample can still give a very inaccurate estimate.
- Ignoring who was left out is wrong because undercoverage can make the sample different from the target population.
- Treating voluntary online polls as representative is wrong because people with strong opinions are more likely to respond.
- Blaming only the sample when the question is leading is wrong because wording bias can affect responses even with a well-chosen sample.
Practice Questions
- 1 A school emails a survey to students, and respond. Find the response rate using .
- 2 A city survey calls landline phone numbers to estimate how many residents support a new bus route. Identify the most likely type of bias and explain why.
- 3 In a poll of website visitors, vote in favor of a new rule. What percent of the voluntary respondents support the rule?
- 4 A survey asks, "Do you agree that our excellent school lunch program should continue?" Explain why this question may produce biased results.
Understanding Survey Bias Types Reference
Bias is about the path from a large population to a final number. Each step can change who is heard and what they say. First, researchers define the population, such as all students at a school or all voters in a state.
Then they build a list or method for reaching people. This is called a sampling frame. A school email list misses students without active accounts.
A landline phone list misses many younger households. These gaps matter because the missing group may have different experiences or views. Randomly choosing names from an incomplete list is still random, but it cannot represent people who were never on the list.
Sampling methods have different strengths. A simple random sample is often the basic model in statistics because selection is based on chance. A stratified sample divides the population into important groups, then randomly samples within every group.
This can help ensure that small groups are included. A cluster sample randomly selects whole groups, such as several classrooms, then surveys everyone in those classrooms. It is practical, but chosen clusters may differ from the wider population.
A convenience sample uses people who are easy to reach, such as students leaving one cafeteria. It can be useful for quick feedback, but it should not support broad claims about every student.
Low participation creates a second problem after selection. Suppose one thousand people are invited and four hundred respond. The response rate is forty percent, found by dividing the number of responses by the number selected, then multiplying by one hundred percent.
A higher rate is usually helpful, though it does not prove the answers are representative. What matters most is whether nonresponders differ from responders. For example, students who feel disconnected from school may be less likely to complete a school climate survey.
Sending reminders, offering several ways to respond, and surveying at different times can reduce this problem. Researchers should report how many people were contacted, how many replied, and who was excluded.
The wording and setting of a survey affect the data even when the sample is well chosen. A question asking whether a school should stop wasting money on an activity pushes respondents toward one view. A better question uses neutral language and names the activity clearly.
Questions should ask one idea at a time. Asking whether students support longer lunches and shorter school days gives no clear answer when someone supports only one change. Response options need to fit realistic answers, including choices such as not sure or not applicable when needed.
Privacy matters too. People may give more honest answers about cheating, health, or family income when responses are anonymous. When evaluating a survey claim, students should trace the process from population, to sample, to participation, to wording before trusting the final percentage.