Data Analytics & Research
Seven questionnaire design mistakes that ruin survey data
GOSPELTRADER Research Desk · 1 October 2026 · 6 min read
Quick answer
The most damaging survey mistakes are double-barrelled questions, leading wording, unbalanced scales, overlapping answer ranges, missing 'not applicable' options, inconsistent scale direction and skipping a pilot test. Each one produces data you cannot analyse honestly.
Analysis cannot repair a badly worded question. By the time a dataset reaches us, the only honest option for a flawed item is often to drop it. These are the seven problems we remove most often.
The seven mistakes
- • Double-barrelled items: "Is the service fast and friendly?" asks two things at once.
- • Leading wording: "How much did you enjoy…" assumes the respondent enjoyed it.
- • Unbalanced scales: three positive options and one negative skew every mean.
- • Overlapping ranges: age bands of 18–25 and 25–35 put 25-year-olds in both.
- • No 'not applicable' option: forces guesses that look like real answers.
- • Flipping scale direction mid-survey: respondents keep ticking the same side.
- • No pilot: ten pilot respondents catch most of the above in an afternoon.
Plan the analysis while you write the questions
For every question, write down which hypothesis it serves and which test you will run on it. Questions with no analysis attached are the first to cut — shorter surveys get more complete responses.