Template

Predictive Quality Pilot Scoping Worksheet

An interactive scoping worksheet to decide whether process and sensor data can predict quality escapes early enough to act, and to capture the practical guardrails — data readiness, labeling, validation criteria, alert thresholds, operator workflows, KPIs and success criteria — needed for an actionable pilot.

Interactive Tool

Predictive Quality Pilot Scoping Worksheet

This worksheet helps teams scope a focused predictive-quality pilot that is practical, measurable, and actionable on the shop floor. Use it to confirm data readiness, choose a labeling strategy, set validation and action thresholds, define operator actions on alerts, and tie results to shop-floor KPIs (escapes, rework, first-pass yield). Complete this before any data science work so the pilot delivers usable alerts rather than distracting false positives.

Save the form to store your scoping record. Helpful hints appear with many fields.

A short descriptive name (e.g., 'Paint Defect Early Warning - Line 2').
Where the pilot will run (site, building, production line or cell).
Person who will shepherd the pilot and make decisions about operator actions and thresholds.
Choose the type of defect or escape you want to predict (pick the best match).
Short description of the escape mode and where/when it is detected.
How many minutes (or fraction of hour) of warning are needed for an operator or system to act and prevent the escape? Example: 10 means you need alerts at least 10 minutes before the escape would be detected downstream.
What should happen when an alert fires? Pick the primary planned action.
Select all signals you could reasonably access for the pilot. Add others in 'Other signals' below.
If you selected 'Other', list specific signals, tag names, or data tables.
Historical records (timestamped) are often required for initial model training. 'Yes' means you can access at least several weeks/months of aligned data depending on defect rate.
Enter an approximate count of records or sample size. If unknown, leave blank.
Choose how you will label examples of defects for supervised modeling.
Who will create or validate labels (QA inspectors, operators, engineers) and how many hours can be dedicated during scoping/pilot?
Estimate the average cost when a defect escapes (scrap, rework, warranty, customer impact). If unknown, estimate conservatively.
Operator attention is limited. What percent of alerts can be false positives before trust erodes? Example: 10 means 10% false positives is acceptable.
What percent of escapes may be missed during the pilot while still being acceptable? Lower is better for safety-critical items.
Precision = true positives / all positives. If you require high trust, set a higher target (e.g., 80).
Recall = true positives / all actual defects. Balance with precision depending on costs.
List how you will validate model performance and pilot success. Tie to measurable KPIs such as % reduction in escapes, reduction in rework hours, improvement in FPY, or operator time per alert.
Record current performance so you can measure improvement. Include time window (e.g., escapes/month).
Describe the step-by-step action an operator will take when an alert arrives (inspect, adjust, hold, escalate). Include who owns the check and acceptable turnaround time.
How will the alert be delivered to the person/system who will act?
List systems the pilot will integrate with (MES, OPC tags, historian, QA dashboard). Note any required IT support.
Any restrictions on data usage, retention, export, or sharing that the project must respect?
High-level timeline (scoping, data collection, model development, validation, pilot run, go/no-go decision). Include target dates where possible.
Examples: insufficient labeled examples, poor signal alignment, operator overload, late alerts. Suggest concrete mitigations.
State the threshold(s) that must be met to move from pilot to limited deployment (e.g., precision >= 75% and escapes reduced >= 30%).
List the stakeholders who must approve this scoping (plant manager, QA lead, IT, operations) and their sign-off status.
Anything else the team should know before starting the pilot.
You can explore this tool now. Sign in or create an account to save your responses and return to them later.
Sign In to Save Create an Account
Make this tool part of your work

Save a personal copy, bring it to your team, or tailor the questions and workflow to fit what you are hungry to improve.

Member customization and team collaboration are coming soon.
Discuss Ask the Librarian

Discussion

Comments and conversation will live here.