Your market research survey should come second, after a few interviews
A market research survey counts answers you already understand. Run a few interviews first, write simple questions, pretest, then call the surprises.
A market research survey should come second, after a few interviews. A survey asks a defined group the same questions so you can count how common each answer is, which works once you know what to ask and whom. As a first step it fails, because it can only measure the answer options you thought of, and it can't ask a follow-up question.
So the order is: a few interviews to learn the words and options people use, a survey to count them, then a handful of calls to explain anything surprising.
One made-up example runs through this page
Here is a made-up example. You run a regional food distributor with about 600 independent restaurants on your customer list. Most orders arrive by phone call or text message, some of them at hours you would rather not know about. You are considering an online ordering portal and want to know who places orders at each restaurant, when, and what goes wrong.
1. Check that a survey is the right tool
AAPOR's best practices for survey research start before any question is written. They ask whether your objectives are specific, whether other surveys already collected the data, whether another method fits better, and whether you also need records.
Don't ask what you can already count. The distributor's records show when orders arrive and through which channel. The survey is for what records can't show: who places the order, what goes wrong and how much that matters.
Write the decision the survey supports in one sentence. For example: "Build the portal if a large share of restaurants report ordering problems that a portal would fix." Cut every question that doesn't help answer it.
2. Write questions people can answer accurately
Pew Research Center's guide to writing survey questions and the AAPOR page agree on the rules that matter most for business surveys:
- Ask about one thing at a time. A question that combines ordering and delivery gets answers you can't interpret.
- Make answer options complete and non-overlapping. Include "don't know" or "does not apply" where people may need them.
- Keep lists short. Pew suggests four or perhaps five answer choices at most for most opinion questions, since people struggle to keep more in mind.
- Avoid agree/disagree statements. Pew notes some respondents tend to agree with whatever they are shown, and recommends a choice between alternative statements instead.
- Ask yes/no per item instead of "select all that apply." Pew found forced-choice questions tend to give more accurate answers and generally avoids select-all lists.
| Instead of | Ask |
|---|---|
| "How satisfied are you with ordering and delivery?" | Two questions: one about ordering, one about delivery |
| "Would you use an online ordering portal?" | "How did you place your last order with us?" Phone call / Text message / Email / Through a sales rep / Other (please specify) |
| "Do you agree that ordering takes too long?" | "Which comes closer to your view? Placing an order takes about the right amount of time / Placing an order takes too long" |
| "Which of these problems have you had? (select all)" | A yes/no question for each problem, such as "In the past month, did an order arrive with a missing item?" |
Ask what someone did recently, not what they would do. A predicted "yes, I'd use a portal" costs the respondent nothing.
3. Build the answer options from interviews
The options you offer shape the result. In a Pew experiment, 58% of respondents chose the economy as their top voting issue when it appeared on a list, but only 35% volunteered it when asked without a list. In the open-ended version, 43% gave an answer that wasn't on the closed list at all.
That is why Pew notes researchers sometimes run an open-ended pilot first and build closed options from the most common responses. For a small business, five or six short interviews do the same job. Customer interview questions has wording for those first conversations.
4. Put easy, general questions first and keep it short
AAPOR recommends general questions before specific ones, easy questions first, a short questionnaire, and letting people skip or answer "don't know," because forcing an answer makes them more likely to quit. Pew adds that demographic questions shouldn't open a survey unless they are needed to check eligibility.
For the distributor: first how the restaurant placed its last order and who placed it, then which problems happened recently, and only at the end a description of the portal and a question about it. Describe the portal first and it colors every answer after it.
End with one optional question: "May we contact you for a 15-minute follow-up call?" You will want that list later.
5. Watch a few people fill it in before you send it
AAPOR recommends pretesting with people similar to your respondents, often through cognitive interviews, where the person explains how they understood each question and arrived at an answer. Then it recommends a pilot run of the whole process.
Sit with three or four restaurant managers while they fill it in. They will find the questions they read differently than you meant.
6. Read the numbers as a picture of who answered
AAPOR calls the list you draw respondents from the sampling frame. The distributor's list covers only its customers. Restaurants that buy from a competitor aren't on it, so the survey can't tell you how the whole local market orders.
Pew explains that a simple random sample of 1,067 people has a margin of error of about plus or minus 3 percentage points, while a subgroup of about 160 has about plus or minus 8. To compare full-service and quick-service restaurants, each group needs enough responses on its own.
A survey emailed to a customer list and answered by whoever chooses to respond isn't a random sample either. AAPOR notes such nonprobability samples need special care in analysis and reporting. Treat the results as a rough picture of your respondents, and say so when you share them. Quantitative market research covers sample planning in more depth.
7. Call people to find out why
A survey tells you how many, rarely why. Book follow-up calls when:
- A result surprises you. Suppose 40% of restaurants say the chef orders by text late at night. Is that a preference, or the only time anyone is free?
- Many people choose "Other" and write similar things. That is an answer option you missed.
- Two groups answer very differently and you need to know what separates them.
- You need to know how a decision gets made, such as who would approve paying for a portal.
Call five to eight people from the "may we contact you" list and ask about a recent, specific event, not their opinion of the topic. Interviews vs. surveys explains how the two methods fit together.
Your next step
Write the decision your survey supports in one sentence, then list the three to five questions that answer it. Run a few interviews before you draft any answer options.
If the people you need aren't on your own list, such as purchasing managers at restaurants that buy from someone else, Instant Expert can find people who match a description you write. You review who it finds, it sends your invitations, and you pay only for calls that get booked. The directory page for procurement professionals in restaurants is one place to start.