AI Model Monitoring & Post‑Deployment Runbook

Purpose and scope

This runbook helps clinical teams keep deployed AI models safe, fair, and effective. It describes what to monitor, how to detect meaningful change, who should act, and step‑by‑step actions to contain, mitigate, and learn from incidents. Use this playbook for predictive or prescriptive models used in clinical workflows, decision support, triage, or operational automation.

Audience

Model owners, ML engineers, data engineers, clinical subject matter experts (SMEs), quality & safety officers, compliance teams, and operations staff responsible for model performance and patient safety.

Prerequisites

• Versioned model artifacts and training data snapshots.
• Production logging of inputs, outputs, timestamps, request/response metadata, and model version IDs.
• Routine access to labeled outcomes (or a plan for delayed labels) for ground‑truth comparison.
• Contact list for key stakeholders and an escalation channel (pager, Slack channel, incident management queue).

Monitoring categories & target signals

Group your checks into these categories. For each signal define: metric, monitoring frequency, alert threshold, and owner.

1. Model performance (labelled data)

• Metrics: AUC, sensitivity, specificity, positive predictive value (PPV), calibration (e.g., calibration slope or Brier score).
• Cadence: weekly or monthly depending on label lag; daily if near real time with rapid labels.
• Trigger examples: relative drop >5–10% in primary metric vs. baseline; statistically significant degradation on recent window.

2. Data & input distribution (unlabelled, real‑time)

• Metrics: feature distribution shifts (Population Stability Index, PSI), KS test p‑values, missingness rates, schema changes, range/units anomalies.
• Cadence: daily (real‑time streams) or weekly for slower flows.
• Trigger examples: PSI >0.2 for key features; missingness increase >5% from baseline; new categorical levels observed.

3. Prediction behavior & confidence

• Metrics: distribution of predicted probabilities, sudden changes in mean/variance, spike in near‑0.5 probabilities, confidence degradation.
• Triggers: concentration of outputs at extremes or collapse of predictive spread; drop in average predicted risk without corresponding label change.

4. Fairness & subgroup performance

• Metrics: subgroup AUC, sensitivity/specificity, false positive/negative rates by protected attributes (where available and permitted).
• Triggers: subgroup metric deficit exceeds an agreed threshold relative to overall performance or baseline.

5. Operational health

• Metrics: inference latency, throughput, error rates, queue/backlog size, feature pipeline failures.
• Triggers: error rate spike, latency above SLA, persistent pipeline failures.

Daily / Weekly Checklist (practical)

Capture this as a living checklist owners sign off on.

1. Confirm model version and artifact integrity (match hashes) in production logs.
2. Verify data pipeline: recent ingestion timestamps, no schema drift alerts, missingness within expected bounds.
3. Check key performance short‑term indicators (if labels available): rolling window stats vs. baseline.
4. Inspect prediction distribution and average confidence for unexpected shifts.
5. Review subgroup performance monitors for any emerging disparities.
6. Examine system health: latency, error logs, infrastructure alerts.
7. Record anomalies and assign tickets for investigation when any trigger exceeded.

Drift detection approaches (practical options)

• Statistical tests: KS test, chi‑square for categorical features, PSI for continuous features. Interpret results with context—small p‑values may be expected on large samples.
• Model‑based detectors: train small surrogate models to predict time or cohort and watch accuracy increase (indicates systematic change).
• Embedding / distance methods: use feature embeddings and monitor centroid/MD distances over time.
• Label‑aware drift: compare recent labeled outcomes to historical patterns (best when labels lag is manageable).
• Operational heuristics: sudden rise of nulls, new categories, or values outside expected ranges.

Incident triage and response (step‑by‑step)

When an alert fires, follow a short checklist to contain risk quickly.

1. Detect & verify — Confirm alert is valid (remove noise, reproduce metric). Owner: Monitoring Lead.
2. Assess risk — What clinical decisions are affected? Estimate potential patient impact. Owner: Model Owner + Clinician SME.
3. Contain — If risk is high, temporarily disable automated actions or route to human review. Use a feature flag or configuration toggle. Owner: Ops Engineer.
4. Mitigate — Choose remediation: rollback to previous model version, enable a rule‑based fallback, or apply input sanitization.
5. Communicate — Notify stakeholders (see templates below) and open an incident ticket with priority and owner.
6. Investigate — Collect data windows, logs, sample inputs, and perform root‑cause analysis. Owner: Data Engineer + ML Engineer + Clinician SME.
7. Remediate & validate — Retrain, patch feature engineering, or adjust thresholds in a controlled environment (shadow/canary) before redeploying.
8. Postmortem — Document timeline, causes, corrective actions, and monitoring improvements. Share lessons with teams and governance bodies.

Rollback and mitigation checklist

• Confirm safe rollback candidate (previous model ID is tested and approved).
• Check rollback dependencies (feature pipelines, preprocessing code must match rollback version).
• Switch traffic using canary or feature flag; monitor for immediate improvement.
• If rollback not possible, enable human‑in‑the‑loop or rule‑based stopgap until issue resolved.

Stakeholder notification templates

Email / Message: Initial alert

Subject: [ALERT] Model X performance/health anomaly detected — action required

Body: We observed a [brief description: e.g., 8% drop in AUC / PSI >0.25 for feature Y / outage] for Model X in production at [time]. Current impact estimate: [low/medium/high]. Immediate action: [containment action taken]. Assigned owner: [name]. Next update in [timeframe].

Email / Message: Incident update

Subject: [UPDATE] Model X incident — containment in progress

Body: Summary of findings so far, actions taken (rollback, disable automation, opened ticket ID), and estimated time to next update.

Roles & responsibilities (suggested)

• Model Owner: accountable for monitoring plan, thresholds, and remediation decisions.
• ML Engineer: implements detectors, runs investigations, prepares fixes and retraining pipelines.
• Data Engineer: ensures pipeline health, data snapshots, and logging completeness.
• Clinician SME / Safety Officer: assesses clinical risk, approves containment and mitigation steps affecting care pathways.
• Ops / SRE: performs rollbacks, feature flag changes, and infrastructure actions.
• Compliance / Legal: engaged for data/privacy or regulatory implications.

Retraining & redeployment guidance

Decide to retrain when degradation is due to data drift, label distribution change, or a systematic upstream change. Follow this pattern:

1. Collect recent labeled data or curate a representative dataset for retraining.
2. Train candidate model(s) and evaluate on holdout and subgroup metrics.
3. Run candidate in shadow mode on live traffic and compare decisions.
4. Plan a canary rollout with monitoring and quick rollback capability.

Post‑incident review checklist

• Timeline of detection and actions.
• Root cause and contributing factors (data, model, pipeline, user behavior).
• Corrective actions taken and preventive controls planned or implemented.
• Updates to monitoring thresholds, new detectors, or process changes.
• Communication and training needs for clinical staff.

Practical notes & suggested starting thresholds

Thresholds must be contextual. Use these as starting points and tune with historical data and SME judgment:

• AUC drop relative to baseline: 5–10% (treat as signal, not absolute failure).
• PSI >0.2 for a feature — investigate; PSI >0.3 — likely actionable.
• Missingness increase >5% for critical features — create immediate ticket.
• Subgroup performance gap >10% relative to overall — escalate to clinician SME.

Documentation & audit artifacts

Keep an auditable record for each event: model and data versions, monitoring graphs, incident ticket, decisions made, rollback artifact IDs, test results, and postmortem document.

Next steps to operationalize this playbook

• Map this runbook to your monitoring dashboards and implement the key signals as automated alerts.
• Establish an on‑call rotation for the Monitoring Lead role.
• Run tabletop drills for model incidents with clinician participation.
• Schedule periodic reviews of thresholds and subgroup metrics (quarterly or after significant deployment changes).

Use this playbook as a living document: tune metrics, thresholds, and responsibilities to your clinical context. The goal is timely detection, safe containment, clear communication, and rapid recovery while preserving trust and patient safety.