Question wording bias happens when the wording, order, or tone of a survey question pushes people toward certain answers. This matters because surveys are often used to make decisions in science, business, health, education, and public policy. Even a small wording change can shift percentages enough to change the conclusion of a study.
A good survey question measures what people think, not what the question leads them to say.
Biased wording can appear as loaded language, leading phrases, confusing negatives, or answer choices that do not cover all reasonable responses. Order effects can also influence answers because earlier questions can frame how later questions are interpreted. Social desirability bias occurs when people answer in a way that makes them look responsible, kind, or socially acceptable.
Researchers reduce these problems by using neutral wording, balanced answer choices, random question order, and pilot testing.
Key Facts
- Question wording bias occurs when the phrasing of a question changes the distribution of responses.
- A leading question suggests a preferred answer, such as Do you agree that this helpful program should continue?
- A neutral rewrite removes judgment words, such as Should this program continue?
- Response rate = number of completed surveys / number invited.
- Observed percent = number choosing an answer / total number of respondents x 100%.
- Bias is different from random error because bias systematically pushes results in one direction.
Vocabulary
- Question wording bias
- Question wording bias is a systematic change in survey answers caused by the way a question is phrased.
- Leading question
- A leading question is a question that hints at or encourages a particular answer.
- Order effect
- An order effect is a change in responses caused by the sequence in which questions or answer choices are presented.
- Social desirability bias
- Social desirability bias occurs when respondents give answers they think will make them look good to others.
- Neutral wording
- Neutral wording uses balanced, clear language that does not suggest which answer is preferred.
Common Mistakes to Avoid
- Using emotional words in a survey question, such as dangerous, wasteful, or excellent, is wrong because these words add an opinion before the respondent answers.
- Treating a leading question as unbiased data is wrong because the response may reflect the prompt more than the respondent's true belief.
- Ignoring question order is wrong because an earlier question can make certain ideas more available and influence later answers.
- Comparing two surveys with different wording as if they measure the same thing is wrong because wording changes can create different response patterns.
Practice Questions
- 1 A biased survey asks, Do you support the mayor's irresponsible plan to raise parking fees? Rewrite the question using neutral wording.
- 2 In Survey A, the question says, Should the city improve public transportation by adding bus lanes? and 62 out of 100 people answer yes. In Survey B, the question says, Should the city remove car lanes to add bus lanes? and 48 out of 100 people answer yes. Calculate the yes percentage for each survey and describe what the difference suggests.
- 3 A researcher asks students first whether cheating is a serious problem at their school, then asks whether teachers should use stricter test monitoring. Explain how the first question could influence answers to the second question.