Template

A/B Test Template for Shopfloor Process Experiments

A practical, interactive experiment plan template for controlled shopfloor A/B tests. Guides hypothesis formation, measurement, sampling, randomization, acceptance rules, data collection, and the decision to stop, scale, or iterate.

Interactive Tool

A/B Test Template — Shopfloor Process Experiments

Use this interactive template to design and record small, defensible shopfloor experiments. The form focuses on clear hypotheses, reliable baseline measurement, simple randomization or control, practical sample-size awareness, and explicit decision rules so improvements produce trustworthy, actionable learning.

Quick tips: keep the change small and isolated, record who/when/what for each observation, avoid introducing other changes during the test, and prefer randomized or alternating assignment to prevent bias.

Example

Hypothesis: A visual checklist during changeovers will reduce average changeover time from 25 to 21 minutes. Baseline window: last 10 changeovers. Treatment: use checklist for designated runs. Sample: target 30 changeovers per group. Acceptance: at least 15% median reduction and clear shift on a run chart; if unclear after the planned duration, iterate.

Write a clear, testable hypothesis. Example: 'If we do X, then metric Y will improve by Z% because...'.
Name the numeric metric (e.g., 'changeover time (minutes)', 'first-pass yield (%)', 'inspections per hour'). Include units.
Describe recent baseline value(s) and the measurement window (e.g., 'avg 25 min over last 10 changeovers' or 'FPY 92% across last 2 weeks').
How will you run the control? (e.g., current standard procedure, same operators, same shifts). Include any constraints to keep the control consistent.
Describe the exact change to apply for treatment runs. Be specific enough that another operator could implement it the same way.
Choose how runs will be assigned to control or treatment to avoid bias.
A simple target for how many observations you plan per condition. If you need a formal power calculation, consult a statistician. Practical rule: plan for at least 20–30 observations per group for continuous metrics when possible.
How long you'll run the experiment. Consider production cadence and how long it takes to collect the planned sample size.
Define concrete criteria that will lead you to accept, reject, or iterate (e.g., '% reduction, shift in run chart, or statistical test threshold'). Include practical business thresholds, not just p-values.
List what you'll record for each observation: date/time, part/lot, operator, shift, run id, metric value, notes, photos, and source (MES, manual log). Specify who will collect and how data will be stored.
Select how you will analyze results. Choose conservative, easy-to-interpret methods suitable for frontline teams.
Describe exactly when you will stop the test early, scale the treatment, or iterate (e.g., 'Stop early if safety incidents increase; scale if acceptance rules met for two consecutive weeks').
List factors that could bias the result (different operators, concurrent process changes, material lots, seasonality) and how you'll control or record them.
Name and role of the person accountable for running the test and data integrity.
Use YYYY-MM-DD or descriptive (e.g., 'Week of May 10').
Describe safety, quality, or production limits that would force immediate stop (e.g., 'If scrap rate increases >2% absolute, stop immediately').
Any additional context, links to photos, work instructions, or data exports. Attachments can be referenced by URL or platform attachment id.
You can explore this tool now. Sign in or create an account to save your responses and return to them later.
Sign In to Save Create an Account
Make this tool part of your work

Save a personal copy, bring it to your team, or tailor the questions and workflow to fit what you are hungry to improve.

Member customization and team collaboration are coming soon.
Discuss Ask the Librarian

Discussion

Comments and conversation will live here.