AI pilot scoring card (impact, data readiness, risk)

{ "Title": "AI pilot scoring card (impact, data readiness, risk)", "IntroductionHtml": "

Use this scoring card to compare and prioritize AI pilot ideas by capturing standardized scores, relative weights, rationale, sample size and acceptance criteria. Run this during AI opportunity workshops or complete it when assessing individual proposals. Default weights are suggested — adjust to match your organization’s priorities.

How scoring works: Each dimension is scored 1–5 (1 = poor/high risk/low value, 5 = excellent/low risk/high value). To compute a weighted score out of 100, use: Weighted score = sum((score_i / 5) * weight_i), where weight_i are percentages that total 100.

", "SubmitLabel": "Save scoring", "SuccessMessage": "Scoring saved. Use the results to rank pilots and gather evidence for selection discussions.", "DataType": "AIPilotScoringCard", "SchemaVersion": 1, "Fields": [ { "Key": "pilot_name", "FieldType": "text", "Label": "Pilot name / short description", "Required": true, "HelpText": "A short, clear name that identifies the pilot idea." }, { "Key": "owner", "FieldType": "text", "Label": "Proposed by / owner", "Required": false, "HelpText": "Team or person proposing the pilot." }, { "Key": "date", "FieldType": "text", "Label": "Date", "Required": false, "HelpText": "YYYY-MM-DD (optional)." }, { "Key": "weight_business_impact", "FieldType": "number", "Label": "Weight — Business impact (%)", "Required": true, "HelpText": "Relative importance of this dimension. Default 30. Weights should sum to 100 across dimensions.", "Min": 0, "Max": 100, "Default": 30 }, { "Key": "score_business_impact", "FieldType": "scale", "Label": "Business impact (1–5)", "Required": true, "HelpText": "Estimate of revenue, cost avoidance, or strategic value (1 = negligible, 5 = transformative).", "Min": 1, "Max": 5, "Step": 1 }, { "Key": "rationale_business_impact", "FieldType": "textarea", "Label": "Rationale — Business impact", "HelpText": "Brief supporting evidence (e.g., estimated $ impact, affected SKUs, customer benefit)." }, { "Key": "weight_data_readiness", "FieldType": "number", "Label": "Weight — Data availability & quality (%)", "Required": true, "HelpText": "Default 25. Consider whether data exists, is labeled, accessible and high-enough quality for the pilot.", "Min": 0, "Max": 100, "Default": 25 }, { "Key": "score_data_readiness", "FieldType": "scale", "Label": "Data availability & quality (1–5)", "Required": true, "HelpText": "1 = poor or unavailable data, 5 = representative, labeled, accessible, and trustworthy.", "Min": 1, "Max": 5, "Step": 1 }, { "Key": "rationale_data_readiness", "FieldType": "textarea", "Label": "Rationale — Data readiness", "HelpText": "Examples: sample size available, labeling effort, data latency, missing safety signals. Attach or reference representative samples when possible." }, { "Key": "weight_integration_effort", "FieldType": "number", "Label": "Weight — Integration & implementation effort (%)", "Required": true, "HelpText": "Default 15. Measure of technical and process work required to integrate and operate the pilot." }, { "Key": "score_integration_effort", "FieldType": "scale", "Label": "Integration effort — ease (1–5)", "Required": true, "HelpText": "Score for ease (1 = very hard / high complexity, 5 = very easy / low integration work). Clarify dependencies on ERP/MES/PLC, APIs, or middleware.", "Min": 1, "Max": 5, "Step": 1 }, { "Key": "rationale_integration_effort", "FieldType": "textarea", "Label": "Rationale — Integration effort", "HelpText": "Note required systems, custom work, and estimated development or operations effort." }, { "Key": "weight_operational_risk", "FieldType": "number", "Label": "Weight — Operational risk / safety (%)", "Required": true, "HelpText": "Default 20. Consider safety, compliance, human-in-the-loop needs, and potential for operator impact.", "Min": 0, "Max": 100, "Default": 20 }, { "Key": "score_operational_risk", "FieldType": "scale", "Label": "Operational risk / safety (1–5)", "Required": true, "HelpText": "1 = high risk or unresolved safety concerns, 5 = low risk with clear mitigations and operator acceptance.", "Min": 1, "Max": 5, "Step": 1 }, { "Key": "rationale_operational_risk", "FieldType": "textarea", "Label": "Rationale — Operational risk", "HelpText": "Describe safety concerns, failure modes, and planned mitigations or review needs." }, { "Key": "weight_time_to_value", "FieldType": "number", "Label": "Weight — Time to value (%)", "Required": true, "HelpText": "Default 10. How quickly the pilot is expected to show measurable value.", "Min": 0, "Max": 100, "Default": 10 }, { "Key": "score_time_to_value", "FieldType": "scale", "Label": "Time to value (1–5)", "Required": true, "HelpText": "1 = long or uncertain timeline, 5 = short time to measurable results (weeks to a few months).", "Min": 1, "Max": 5, "Step": 1 }, { "Key": "rationale_time_to_value", "FieldType": "textarea", "Label": "Rationale — Time to value", "HelpText": "Expected timeline, key milestones, and earliest measurable KPIs." }, { "Key": "representative_sample_size", "FieldType": "number", "Label": "Representative sample size (rows, hours, events)", "Required": false, "HelpText": "Estimate of sample size available to validate the model (e.g., 10,000 records; 100 hours).", "Min": 0 }, { "Key": "acceptance_criteria", "FieldType": "textarea", "Label": "Acceptance criteria for pilot", "Required": true, "HelpText": "Concrete success criteria the pilot must meet to be considered successful (e.g., >85% precision in defect detection, 10% reduction in cycle time)." }, { "Key": "calculated_weighted_score", "FieldType": "number", "Label": "Calculated weighted score (0–100)", "Required": true, "HelpText": "Compute using: sum((score_i / 5) * weight_i). Example: if business impact score=4 and weight=30, contribution = (4/5)*30 = 24. Sum contributions across dimensions.", "Min": 0, "Max": 100, "Default": 0 }, { "Key": "overall_rationale", "FieldType": "textarea", "Label": "Overall rationale / notes", "HelpText": "Any additional context, dependencies, or open questions. Reference files or links in your project tracker." }, { "Key": "recommendation", "FieldType": "select", "Label": "Recommendation", "Required": true, "HelpText": "Team recommendation based on score and readiness.", "Options": [ { "Value": "proceed_pilot", "Label": "Proceed to pilot" }, { "Value": "pilot_with_mitigations", "Label": "Pilot if mitigations addressed" }, { "Value": "defer", "Label": "Defer / gather more data" }, { "Value": "reject", "Label": "Reject - not a good fit" } ] } ] }