Tool

ROI & Sample Size Calculator

An interactive, saveable calculator that helps teams estimate potential ROI for experiments and determine minimum sample sizes for A/B tests. Includes guided inputs for conversion and continuous metrics, sensible defaults, and clear interpretation guidance so teams can prioritize tests and plan efficient pilots.

Interactive Tool

ROI & Sample Size Calculator

Use this tool to capture experiment assumptions and calculate ROI and sample-size guidance.

This interactive form collects the numbers you normally use in spreadsheets and saves them for sharing or iteration. It includes two practical sections: ROI estimator (useful for prioritizing experiments) and Sample-size guidance (for conversion and continuous metrics). The form does not execute statistical calculations in-platform; it stores your inputs so you can reuse them in spreadsheets, dashboards, or experiment trackers. Guidance and formulas are provided below each section to help you compute results or paste values into your analysis tool.

How to use: Enter the best available estimates. If you are unsure, enter a conservative value and save. After submission you can copy values into a spreadsheet or request a run by an analyst. Use the guidance fields to record key assumptions and decisions.

This section captures the basic drivers used to estimate an experiment's potential return. Leave fields blank when not applicable.
Average visitors or sessions in the period you're modeling (e.g., monthly visitors).
Current conversion rate for the metric you'll change (enter as percent, e.g., 2.5).
Average value of a conversion in your currency. For non-commerce, use the monetary value of a success or expected lifetime value per conversion.
Anticipated percent improvement from the experiment (e.g., enter 10 for a 10% uplift). Use realistic or conservative estimates.
Portion of additional revenue that contributes to gross profit (e.g., 40). Use 100% for pure-margin cases or lower if costs scale with volume.
Include development, design, data analysis, promotional costs, or tooling needed to run the test. Currency units should match revenue fields.
Period over which you expect to realize uplift (used to annualize or horizon-limit ROI).
Use these formulas in a spreadsheet to compute estimated annualized ROI and payback: 1) Incremental conversions per period = baseline_traffic * (baseline_conversion_pct / 100) * (expected_lift_pct / 100) 2) Incremental revenue per period = incremental_conversions_per_period * baseline_revenue_per_conv 3) Incremental gross profit per period = incremental_revenue_per_period * (incremental_margin_pct / 100) 4) ROI (simple) = (incremental_gross_profit_per_horizon - experiment_cost_total) / experiment_cost_total Adjust time horizon to annualize: multiply per-period profit by (12 / period_months) when appropriate. These are starting-point calculations — treat numbers as estimates and run sensitivity checks for multiple expected_lift_pct values.
Choose whether your primary metric is a binary conversion or a continuous metric. Provide the baseline and the minimum effect you care about detecting.
Select whether the metric is a binary conversion (e.g., purchase) or a continuous metric (e.g., revenue per user).
Enter baseline conversion as percent (e.g., 2.5). Leave blank for continuous metrics.
Average value per user for continuous metrics (same units as MDE). Leave blank for conversion metrics.
Standard deviation of the metric across users (needed to compute sample size for means). If unknown, estimate conservatively or measure from historical data.
Smallest relative change you care about detecting (e.g., 10 for 10%). For conversions this will be applied to baseline rate; for means, multiply baseline_mean by this percent to get absolute MDE.
Type I error rate. Common choices: 0.05 (95% confidence) or 0.01.
Probability of detecting the effect if it exists. Common defaults: 0.8 (80%) or 0.9.
Percent of traffic to assign to the treatment group (commonly 50 for equal split).
Practical formulas you can paste into a spreadsheet or run in a stats tool: For conversion (binary) approximation (pooled): 1) p1 = baseline_rate, p2 = baseline_rate * (1 + mde_pct/100) 2) pooled variance = p1*(1-p1) + p2*(1-p2) 3) z_alpha = inverse_norm(1 - alpha/2), z_beta = inverse_norm(power) 4) n_per_group = (z_alpha*sqrt(pooled_variance) + z_beta*sqrt(p1*(1-p1)+p2*(1-p2)))^2 / (p2 - p1)^2 For continuous (means): 1) absolute_mde = baseline_mean * (mde_pct/100) 2) n_per_group = 2 * (z_alpha + z_beta)^2 * (baseline_std^2) / (absolute_mde^2) These formulas assume independent samples, two-sided tests, and approximate normality. Use them for planning and sanity checks. When in doubt, prefer larger sample sizes or run a pilot to measure baseline variance.
Notes on interpreting results and choosing a test strategy.
1) Small MDEs require large samples — test feasibility before committing. 2) Consider running rapid, lower-powered exploratory tests for early learning, but avoid making definitive decisions from underpowered tests. 3) Use sensitivity checks: compute ROI and sample size across a range of lift scenarios (optimistic, expected, conservative). 4) Include experiment cost and operational overhead when deciding whether a test is worth running. 5) Record assumptions (baseline, MDE, horizon) so results can be revisited and audited.
Write the key assumptions your team used (e.g., data source, time period, expected seasonality). These notes are saved with the inputs so others can review them.
You can explore this tool now. Sign in or create an account to save your responses and return to them later.
Sign In to Save Create an Account
Make this tool part of your work

Save a personal copy, bring it to your team, or tailor the questions and workflow to fit what you are hungry to improve.

Member customization and team collaboration are coming soon.
Discuss Ask the Librarian

Discussion

Comments and conversation will live here.