AI Pilot Evaluation Scorecard

{ "Title": "AI Pilot Evaluation Scorecard", "IntroductionHtml": "

This scorecard helps teams evaluate industrial AI pilots objectively and consistently. Use it to capture key evidence, compute a weighted score, and produce a clear recommendation. The form focuses on shop-floor outcomes: downtime reduction, yield improvement, scheduling gains, and inspection automation.

How to use: complete the fields below, score each dimension on a 1–5 scale (higher is better), calculate the weighted score using the weights shown in the help text, then enter the weighted score (0–100) and select a recommendation.

Decision thresholds: Go >= 70 (clear path to production), Further work 50–69 (needs additional data, refinement, or operator trials), No-Go < 50 (insufficient value or too risky/costly).

", "SubmitLabel": "Save evaluation", "SuccessMessage": "Evaluation saved. Consider sharing results with stakeholders and recording artifacts (data diagnostics, model artifacts, integration plan).", "DataType": "ai-pilot-eval-scorecard", "SchemaVersion": "1.0", "Fields": [ {"Key": "pilotName", "Type": "text", "Label": "Pilot name / ID", "HelpText": "Descriptive name or identifier for the pilot (e.g., 'Line 2 Vibration Anomaly - Mar 2026')", "Required": true}, {"Key": "plant", "Type": "text", "Label": "Plant / Location", "HelpText": "Where the pilot ran (site, line, cell)"}, {"Key": "evaluator", "Type": "text", "Label": "Evaluator name", "HelpText": "Person completing this evaluation", "Required": true}, {"Key": "date", "Type": "text", "Label": "Date", "HelpText": "YYYY-MM-DD or preferred local format", "Required": true}, {"Key": "businessImpact", "Type": "scale", "Label": "Business impact", "HelpText": "Potential measurable impact on key KPIs (e.g., downtime, throughput, first-pass yield). 1 = negligible, 5 = transformational. Weight: 25%", "Min": 1, "Max": 5, "Required": true}, {"Key": "accuracyPrecision", "Type": "scale", "Label": "Accuracy & precision", "HelpText": "Model performance relative to operational needs (false positives/negatives matter). 1 = poor, 5 = excellent. Weight: 15%", "Min": 1, "Max": 5, "Required": true}, {"Key": "leadTimeToAction", "Type": "scale", "Label": "Lead-time to action", "HelpText": "How quickly predictions produce an actionable window for operators or systems. 1 = too slow, 5 = immediate. Weight: 15%", "Min": 1, "Max": 5, "Required": true}, {"Key": "dataSufficiency", "Type": "scale", "Label": "Data sufficiency & quality", "HelpText": "Volume, labeling, completeness, and drift risk. 1 = insufficient, 5 = production-ready. Weight: 15%", "Min": 1, "Max": 5, "Required": true}, {"Key": "integrationFeasibility", "Type": "scale", "Label": "Integration feasibility", "HelpText": "Ease of connecting outputs to MES/SCADA/controls and embedding into workflows. 1 = high friction, 5 = seamless. Weight: 10%", "Min": 1, "Max": 5, "Required": true}, {"Key": "operatorAcceptance", "Type": "scale", "Label": "Operator acceptance & UX", "HelpText": "Operator trust, explainability, and ergonomics of suggested actions. 1 = unlikely to be used, 5 = likely to be adopted. Weight: 10%", "Min": 1, "Max": 5, "Required": true}, {"Key": "costScalability", "Type": "scale", "Label": "Cost & scalability", "HelpText": "Estimated total cost of ownership and ability to scale across lines or sites. 1 = costly/limited, 5 = low cost/highly scalable. Weight: 10%", "Min": 1, "Max": 5, "Required": true}, {"Key": "weightedScore", "Type": "number", "Label": "Weighted score (0–100)", "HelpText": "Compute weighted score using: BusinessImpact*25 + Accuracy*15 + LeadTime*15 + DataSufficiency*15 + Integration*10 + Operator*10 + Cost*10. Each dimension score is 1–5; compute weighted average and multiply to scale 0–100. Example: if weighted average = 3.6 then score = 3.6/5 * 100 = 72. Enter the final number.", "Required": true}, {"Key": "recommendation", "Type": "select", "Label": "Recommendation", "HelpText": "Use thresholds: Go >= 70, Further work 50–69, No-Go < 50. Add rationale in comments.", "Options": [{"Value": "go", "Label": "Go to production"}, {"Value": "further", "Label": "Further work / iterate pilot"}, {"Value": "no-go", "Label": "No go / discontinue"}], "Required": true}, {"Key": "comments", "Type": "textarea", "Label": "Evaluator comments & evidence", "HelpText": "Summarize key evidence (metrics, sample confusion matrices, operational observations), risks, required next steps, and stakeholders to involve."}, {"Key": "artifacts", "Type": "textarea", "Label": "Linked artifacts (URLs or paths)", "HelpText": "Location of model artifacts, code repo, data diagnostics, dashboards, integration plan, SOPs, or test logs."} ] }