How to write survey questions backwards, starting from the decision

Write survey questions from the decision backwards: one question per fact that could change it, asked about recent behavior, with opt-outs and a pretest.

Instant Expert EditorialPublished Updated 6 min read

The best way to write survey questions is backwards. Start with the decision the survey will inform, list the few facts that would change it, and write one plain question per fact. If a flipped answer wouldn't change what you do, cut the question.

Most weak surveys fail before the wording: they ask about everything the team is curious about and return a pile of percentages nobody can act on. The Nielsen Norman Group's survey best practices puts "ask only questions that you need answered" first, and says to keep surveys short.

1. Write the decision in one sentence

Name the choice, the options and the threshold. "Learn about our customers" is not a decision. "Build the lien-waiver feature before the change-order feature if at least a third of respondents handled lien waivers in the past month" is.

AAPOR's best practices start in the same place: specific, unambiguous objectives, a check for data other surveys already collected, and whether another method fits better. AAPOR also says surveys should not be used to produce predetermined results or to sell. If the real goal is a quote for your pitch deck, you will write leading questions without meaning to.

2. Cross off what you already know

List the facts that would move the decision, then cross off anything you can get elsewhere. NN/g advises against asking questions you can answer yourself, such as demographics you could look up for existing customers. Your product analytics, CRM or order history often already hold the "how often" and "how much."

3. Ask about last week, not next year

NN/g compares three ways to ask the same thing:

  • Future prediction (unreliable): "How likely are you to use this product?"
  • Present estimate (still shaky): "How often do you use this product in an average week?"
  • Past estimate (better): "Approximately how many times did you use this product in the past 7 days?"

People are poor predictors of their own behavior, and "an average week" invites them to remember an unusual one. Ask about a specific recent period, include "approximately," and offer ranges.

4. Match the format to the fact

Counts and categories need closed questions with complete, non-overlapping options. Opinions need a balanced rating scale, priorities a ranking or forced trade-off. Answers you can't predict need an open-ended question, used sparingly. See types of survey questions and Likert scale questions.

5. Write in the respondent's words

Pew's guide to writing survey questions asks for simple, concrete language, one question at a time, no double negatives and no unfamiliar abbreviations or jargon. "Change order" means something to a construction subcontractor. "CO workflow module" means something to your product team, and only on a good day.

Pew also notes that researchers sometimes run open-ended pilots first and build answer lists from the most common responses. For a small business, a handful of conversations does the same job and keeps your assumptions out of the answer options.

Then run each question past the biased survey questions checklist.

6. Give people a way out

NN/g recommends an opt-out on every multiple-choice question ("Not applicable," "Don't know," "Other," "Prefer not to answer"), kept as separate options because they mean different things. It also recommends making most questions optional: a required question someone can't answer gets a random answer or a dropout, and AAPOR agrees that requiring answers raises the risk of people leaving early.

7. Watch real people answer before you send it

AAPOR recommends pretesting with people similar to your respondents, often through cognitive interviews where they explain how they understood each question and reached an answer. Three or four sessions can expose a question people read differently from the way you meant it.

A worked example: a subcontractor billing survey

This example is hypothetical. A founder building billing software for electrical and plumbing subcontractors must choose which of two features to build first. The decision sentence: "Build lien-waiver tracking first if at least a third of respondents handled one in the past month and more than half of those say it caused a payment delay."

The questions follow directly:

  1. "In the past 30 days, approximately how many lien waivers did your company send or receive?" None / 1-5 / 6-20 / More than 20 / Don't know
  2. (If any) "In the past 30 days, was any payment to your company delayed because of a missing or incorrect lien waiver?" Yes / No / Don't know
  3. "In the past 30 days, approximately how many change orders did your company submit?" None / 1-5 / 6-20 / More than 20 / Don't know
  4. (If any) "In the past 30 days, was any payment delayed because a change order was disputed or unsigned?" Yes / No / Don't know
  5. "Is there anything else about getting paid on jobs that you'd like us to know?" (optional, open)

Suppose 150 people respond and 60 handled a lien waiver: 40%, above the one-third threshold. If 36 of those 60 report a delay, that is 60%, above half. Build lien-waiver tracking first.

Five questions, one decision. The founder skipped "Would you use lien-waiver tracking?" because a yes costs the respondent nothing.

Interview first when you can't write the answer options yet

NN/g's article Should you run a survey? places surveys as the one common research method that is both quantitative and attitudinal: good at how many people hold a view, weak at "why" and at observing behavior. NN/g also warns against running a survey just to give a skeptical executive a big number when the underlying question is qualitative.

Interview first when:

  • You can't yet write the answer options, because you don't know how people handle the problem.
  • The question is "why": why deals stall, why customers churn, why a workaround persists.
  • The decision depends on how a purchase is approved and who is involved.
  • Your audience is small and specialized, such as finance leads at mid-size subcontractors, and a survey would reach too few of them to count.

In those cases, five to eight 30-minute conversations often tell you more than a survey and give you the vocabulary and answer options for a survey later. See interviews vs. surveys for the choice and customer interview questions for wording.

Your next step

Write your decision sentence and the threshold that would change it. List the three to six facts that matter, cross off what you already have, and draft one question per fact.

If the people you need are not on your list, Instant Expert can find matches for a description such as controllers at specialty subcontractors (finance professionals in construction is one example). You review who it finds, it sends your invitations, and you pay only for calls that get booked.

From decision to survey question

  1. 1

    Decision

    Write the choice, the options and the threshold in one sentence.

  2. 2

    Facts

    List what would move the decision and cross off what you already know.

  3. 3

    Question

    Ask about recent behavior in the respondent's words, one fact per question.

  4. 4

    Format

    Pick closed, rating, ranking or open format to match the fact, and add opt-outs.

  5. 5

    Pretest

    Watch three or four similar people answer and explain how they read each question.

Every question traces back to a fact that could change the decision. Anything that can't is cut.