Check survey sample size for decision confidence

Find out if your survey response count is reliable enough to make business decisions or if the margin of error is too wide.

You getReliability Verdict, margin of Error Estimate and practical Implications

Prompt

Prompt
You are a quantitative researcher. Evaluate the survey sample data provided below to determine if the sample size warrants making strategic business decisions.

- Calculate the estimated margin of error at a standard 95% confidence level based on the population and sample size given.
- Explain plainly what this margin of error means for interpreting percentage results (for example, whether a 52% preference is trustworthy).
- State whether the response volume is adequate for aggregate decisions, segment-level decisions, or exploratory guidance only.
- Recommend the minimum additional responses required to achieve an acceptable decision threshold.

Output format:
- Reliability Verdict (High, Moderate, or Insufficient)
- Margin of Error Estimate (plain breakdown of confidence range)
- Practical Implications (2-3 bullet points on how to use these results safely)
- Recommended Next Steps

Here are the survey details:

Example output

Example for: 42 responses from an active user base of 1,200 customers

Reliability Verdict
Moderate for general directional feedback; Insufficient for critical business bets or pricing changes.

Margin of Error Estimate
With 42 completed surveys from an audience of 1,200, the estimated margin of error is approximately ±14.9% at a 95% confidence level.

Practical Implications

  • If 60% of respondents state they want a new workflow, the true proportion in your full customer base likely lies somewhere between 45% and 75%.
  • Breaking this sample down into sub-groups (such as power users versus occasional users) produces buckets of 10 to 15 people, which will produce random results.
  • Use these findings to gather qualitative quotes and identify problem themes, not to forecast feature adoption rates.

Recommended Next Steps
Collect about 47 more responses (89 in total) to bring your margin of error to roughly ±10%, or about 250 more (292 in total) for ±5% before committing engineering budget. These figures assume responses are a random sample of your customers.

Written by AI from this prompt. Your results will vary.

Paste your total customer population size, number of completed responses, and key survey question results into Claude, ChatGPT, or Gemini. As a follow-up, ask what response rate incentive could safely expand the sample.

Make it repeatable

Keep using this prompt without pasting it in every time.

Keep exploring

More in Data Analysis