Model Governance & MLOps Checklist for OT

{ "Title": "Model Governance & MLOps Checklist for OT", "IntroductionHtml": "

This checklist helps OT teams deploy machine learning models in a controlled, auditable way. Use it to record the model identity, owners, deployment environment, readiness checks, monitoring configuration, rollback and operator‑override plans, retention and lineage references, and governance signoff. Complete one form per model deployment or promotion (e.g., staging → production). Where evidence is required (tickets, metrics, registry links), paste links or brief references in the related fields.

Guidance: keep answers concise. If an item is 'No' or 'Not ready', pause the deployment and follow your change control process.

", "SubmitLabel": "Save checklist", "SuccessMessage": "Checklist saved. This record can be used as an audit artifact for the deployment.", "DataType": "mlops-ot-checklist-submission", "SchemaVersion": 1, "Fields": [ { "Key": "model_name", "Type": "text", "Label": "Model name (registry ID)", "Required": true, "HelpText": "Exact name or registry identifier for the model being deployed." }, { "Key": "model_version", "Type": "text", "Label": "Model version / artifact ID", "Required": true, "HelpText": "Version, commit, or artifact identifier to ensure immutable traceability." }, { "Key": "model_description", "Type": "textarea", "Label": "Short model description and purpose", "HelpText": "One or two sentences describing what the model predicts/controls and the intended operator action." }, { "Key": "owner_name", "Type": "text", "Label": "Model owner (name)", "Required": true, "HelpText": "Person accountable for the model in production (team lead, ML owner, or engineer)." }, { "Key": "owner_role", "Type": "text", "Label": "Owner role / team", "HelpText": "Role or team (e.g., Controls Eng, Data Science, Reliability)." }, { "Key": "deployment_environment", "Type": "select", "Label": "Deployment environment", "Required": true, "Options": [ { "Value": "test", "Label": "Test / Lab" }, { "Value": "staging", "Label": "Staging / Pilot" }, { "Value": "production", "Label": "Production / Shop floor" } ], "HelpText": "Where the model will run. Production deployments require stricter controls." }, { "Key": "deployment_date", "Type": "text", "Label": "Planned deployment date", "HelpText": "YYYY-MM-DD or approximate date for the deployment/promotion." }, { "Key": "approval_status", "Type": "radio", "Label": "Deployment approval", "Required": true, "Options": [ { "Value": "approved", "Label": "Approved" }, { "Value": "rejected", "Label": "Rejected" }, { "Value": "pending", "Label": "Pending" } ], "HelpText": "Record the result of the deployment gate review." }, { "Key": "approver_name", "Type": "text", "Label": "Approver name", "HelpText": "Name of the person who approved the deployment gate." }, { "Key": "data_readiness_confirmed", "Type": "yesno", "Label": "Data readiness confirmed", "HelpText": "Have the input data schemas, quality checks, and sample rates been validated for the target environment?", "Required": true }, { "Key": "training_metrics_summary", "Type": "textarea", "Label": "Training / validation metrics (baseline)", "HelpText": "Key metrics and their values from training/validation (e.g., accuracy, MAE, F1). Include expected range." }, { "Key": "baseline_performance_reference", "Type": "text", "Label": "Baseline performance reference (link/ticket)", "HelpText": "Link to experiment results, model card, or ticket containing metric artifacts." }, { "Key": "monitoring_metrics", "Type": "checkbox", "Label": "Monitoring metrics to enable", "Required": true, "Options": [ { "Value": "data_drift", "Label": "Data drift (input distribution)" }, { "Value": "concept_drift", "Label": "Concept drift (label / target shift)" }, { "Value": "latency", "Label": "Latency / response time" }, { "Value": "throughput", "Label": "Throughput / calls per sec" }, { "Value": "accuracy", "Label": "Accuracy / error rate / KPI" }, { "Value": "resource", "Label": "Resource usage (CPU/GPU/memory)" }, { "Value": "business_kpi", "Label": "Selected business KPI(s)" } ], "HelpText": "Select at least those metrics necessary to detect silent degradation or safety issues." }, { "Key": "alerting_thresholds", "Type": "textarea", "Label": "Alerting thresholds (brief)", "HelpText": "Describe numeric thresholds or change conditions that should trigger alerts (e.g., accuracy drop > 5%, latency > 300 ms, drift score > 0.2)." }, { "Key": "monitoring_frequency", "Type": "select", "Label": "Monitoring frequency", "Options": [ { "Value": "realtime", "Label": "Real-time / stream" }, { "Value": "5min", "Label": "Every 5 minutes" }, { "Value": "15min", "Label": "Every 15 minutes" }, { "Value": "hourly", "Label": "Hourly" }, { "Value": "daily", "Label": "Daily" } ], "HelpText": "Choose how often monitoring signals are evaluated and reported." }, { "Key": "alert_notification_channel", "Type": "checkbox", "Label": "Alert notification channels", "Options": [ { "Value": "dashboard", "Label": "Operational dashboard" }, { "Value": "email", "Label": "Email" }, { "Value": "sms", "Label": "SMS" }, { "Value": "pagerduty", "Label": "PagerDuty / on-call" }, { "Value": "operator_console", "Label": "Operator console alarm" } ], "HelpText": "Where should alerts be surfaced so operators and owners can act." }, { "Key": "rollback_ready", "Type": "yesno", "Label": "Rollback plan available", "Required": true, "HelpText": "Is there a tested, documented rollback or safe‑stop plan that can be executed quickly?" }, { "Key": "rollback_steps", "Type": "textarea", "Label": "Rollback / safe‑stop steps (brief)", "HelpText": "Concise, ordered steps to revert the model or place it into a safe state (who, how, expected time). Include required tickets or console commands." }, { "Key": "operator_override_procedure", "Type": "textarea", "Label": "Operator override / manual action procedure", "HelpText": "How operators should respond to alarms caused by the model (stop, bypass, manual control). Keep language clear for floor staff." }, { "Key": "model_registry_link", "Type": "text", "Label": "Model registry / artifact link", "HelpText": "Link to the model artifact in your model registry or version control." }, { "Key": "data_lineage_reference", "Type": "text", "Label": "Data lineage / source reference", "HelpText": "Where inputs come from ( historian tag names, schema version, dataset ID )." }, { "Key": "data_retention_days", "Type": "number", "Label": "Input / metric retention (days)", "HelpText": "How long raw inputs, features and monitoring metrics will be retained for audits and retraining." }, { "Key": "audit_log_retention_days", "Type": "number", "Label": "Audit log retention (days)", "HelpText": "How long model decision logs, alerts and sign‑off records will be kept." }, { "Key": "change_control_ticket", "Type": "text", "Label": "Change control / deployment ticket ID", "HelpText": "Reference to the deployment/change ticket governing this rollout." }, { "Key": "risk_level", "Type": "select", "Label": "Operational risk level", "Options": [ { "Value": "low", "Label": "Low" }, { "Value": "medium", "Label": "Medium" }, { "Value": "high", "Label": "High" } ], "HelpText": "Estimated risk if the model misbehaves (safety, quality, production impact)." }, { "Key": "safety_constraints", "Type": "checkbox", "Label": "Safety constraints / hard limits enforced", "Options": [ { "Value": "yes_hard_limits", "Label": "Yes — hardware or PLC hard limits" }, { "Value": "no_hard_limits", "Label": "No — model outputs are advisory only" }, { "Value": "operator_override", "Label": "Operator override always available" } ], "HelpText": "How the model is constrained from taking unsafe actions." }, { "Key": "governance_signoff_name", "Type": "text", "Label": "Governance sign‑off (name)", "HelpText": "Name of the person who signs off for production (e.g., Safety Officer, OT Manager).", "Required": true }, { "Key": "governance_signoff_date", "Type": "text", "Label": "Sign‑off date", "HelpText": "YYYY-MM-DD when governance sign‑off occurred." }, { "Key": "additional_notes", "Type": "textarea", "Label": "Additional notes / implementation links", "HelpText": "Any other context, links to runbooks, test results, or monitoring dashboards." } ] }