Thinking about sample size: a practical guide (what decisions matter)

Sample-size calculations often feel like a math exam. In practice, they are a device for making trade-offs explicit. This guide helps you turn uncertainty, cost, and decision impact into a defensible sampling plan.

Start with the decision not the number

Ask: what will we do differently if the experiment shows an effect of the size we consider important? If the answer is 'nothing,' then detecting that effect is not worth the cost. Define the smallest effect that would change your decision — that's your minimum detectable effect (MDE).

Key inputs for sample planning

1. Minimum Detectable Effect (MDE): the smallest difference that matters practically.
2. Baseline or variance estimate: past measurements, pilot data, or literature give you the variability you'll see.
3. Alpha: how much risk of a false positive you will tolerate (commonly 0.05).
4. Power: the desired chance of detecting the MDE if it truly exists (commonly 80–90%).

Rules of thumb and when to get help

For many small laboratory comparisons, a medium effect (something that shifts means by roughly half a standard deviation) may require a few dozen samples per group. For very noisy measurements, required samples can rise quickly. These are only starting points — use a proper calculator or a statistician for anything that will drive major decisions or where risk is high.

Pitfalls to avoid

• No clear MDE: defaulting to statistical significance without practical meaning wastes resources.
• Using unrealistic variance estimates: optimistic SDs understate needed samples; pessimistic estimates may waste resources. If unsure, run a small pilot.
• Multiple outcomes: having many primary tests inflates false-positive risk; plan adjustments in advance or choose a single primary endpoint.
• Underpowered exploratory studies: exploratory work is fine, but report it as such and avoid definitive claims from underpowered tests.

Practical next steps

1. Write down the decision that depends on the experiment.
2. Estimate an MDE tied to that decision.
3. Gather variance estimates from pilot data or the literature.
4. Use an off-the-shelf calculator (G*Power, R packages) or consult a statistician to convert these inputs into a sample-size recommendation.
5. If resources are limited, consider sequential designs, pilot-to-full workflows, or clearly labelled exploratory experiments instead of underpowered confirmatory studies.

Remember: the goal is a defensible plan that connects measurement, uncertainty, and decision-making. Sample size is a tool to manage that link — not an end in itself.