Piloting a Local Promotion Survey Before You Trust Its Results

What to take away

A pilot tests a survey on a few real people before it goes to everyone. For a local promotion, that covers the questions, the skip logic, the consent text and the export file. Small businesses often skip this step, then act on numbers that came from a confusing form. This article shows how to run the pilot and when to run it again.

Lead with a decision

Write the choice, alternatives, owner, deadline, threshold, and evidence that would change the action. Rank assumptions by uncertainty and consequence. Use the least burdensome credible method for the highest-risk gap.

For a local promotion, the choice is usually to run the offer, change it, or drop it. Write the pilot's job in one sentence, such as checking whether a loyalty question is understood, before you recruit anyone.

Use existing evidence first

Before writing a survey, check the sources you already have: sales records, the local chamber of commerce, public business counts from the Census Bureau, and competitors' public offers. Save the source, date, geography and calculation for each one, so a later reader can see what the numbers rest on.

Define the population

Specify the buyer, the user, the occasion, the alternative, the geography and the exclusions. Recruitment should reach people who can answer the question, including lost customers where relevant. Report the groups you missed. A biased sample will mislead even a well-written survey.

Recruit the pilot group

Recruit 5 to 10 people from the target audience for each survey version. Add about five more for each subgroup you plan to read separately, such as new versus repeat customers. Pilot respondents should match the people who will get the real survey, not your staff, because staff already know what the questions mean.

Run a pilot script

Run every pilot session the same way, in this order:

Test the whole path

Walk the whole path: the invitation, the screen wording, skip logic, timing, consent text and the export of responses. Google Forms and SurveyMonkey both let you preview a form and export responses, so test the same tool you will use for fieldwork. Check each tool's current plan limits on its own pricing page before you commit.

Note every question where a respondent hesitates, asks what you mean, or picks an answer that does not fit.

Consider a coffee shop testing a loyalty-promotion survey. One question reads "Do you order dairy or oat milk?" A respondent who orders only oat milk has no honest answer and drops out. This is a hypothetical example, but a pilot would catch the same problem. The fix is two separate questions, one for each milk type, and that branch needs its own re-pilot.

Re-pilot after any change to wording, order, routing, mode or translation. Do not change a live questionnaire silently, because answers before and after the change will not be comparable.

Pilot interview guides and focus groups

The same logic applies to interview guides and focus-group scripts. Run the guide with two or three people, time it, and note which prompts lead people to an answer. Change the wording before the real sessions, and record which version each participant received.

Protect participants

Explain the purpose, what respondents do, whether anything is recorded, what the incentive is, how the data is used and kept, and how to withdraw. If you offer an incentive, write its amount and total cost into the plan.

The FTC's guide, Protecting Personal Information: A Guide for Business, advises collecting only the personal data you need, protecting it and disposing of it securely. Write the retention period and the deletion step into the consent text and the data plan. Sensitive topics, children and regulated data need extra review before the pilot begins.

Separate evidence from inference

Preserve quotations, observations, calculations, the codebook, missing values, bases, counterexamples and researcher judgments. Do not convert interviews with a few customers into market percentages. A pilot of ten people shows whether questions work. It does not show what share of the market holds an opinion.

Triangulate and report

Seek independent evidence from official counts, sales behavior, interviews, competitors and a contained test of the promotion itself. Investigate disagreement rather than averaging it away. Report the sample, recruitment, question wording, field dates, method, uncertainty, limitations, next action and the conditions that would reverse the recommendation.

Keep a decision record

Archive the brief, the evidence, the exclusions, the alternatives considered, the decision, the owner and the review date. Refresh inputs that depend on competitors, prices, season or customer behavior.

Keep the record to a short field list:

Research best-practice record

Assign the practice

Assign one owner for the pilot and one for the data. The pilot owner signs off on the findings before fieldwork starts. The data owner confirms what gets stored, where it sits and how long it is kept, which is where the privacy guidance above applies.

Common questions

What comes before the survey?

Write the decision, population, the question it must answer, the threshold, the cost of a wrong answer, the owner, the deadline, and the evidence gap, before choosing questions.

Why should a questionnaire be piloted?

A pilot can expose misunderstood wording, missing options, order effects, accessibility barriers, mode problems, burden, routing errors, and unusable data before full fieldwork.

How should uncertainty be reported?

State what was observed, what was inferred, who and what were covered, the method, missing evidence, contradictory findings, quantitative uncertainty where valid, and the consequence for the decision.