AI Model Monitoring & Post‑Deployment Surveillance Runbook

Purpose and scope

This runbook describes the operational checks, detection methods, escalation steps, and corrective actions required to monitor clinical AI models after deployment. It is intended for ML engineers, data engineers, clinical safety officers, quality teams, and on-call clinicians responsible for model safety and effectiveness. Use this runbook to detect drift, performance degradation, bias or safety signals early and to respond in a consistent, auditable manner.

Key outcomes (Hungers)

• Detect model drift, data pipeline issues, and safety signals early.
• Provide a reproducible triage and investigation workflow.
• Define clear thresholds and escalation routes to mitigate patient risk quickly.
• Ensure auditability and evidence for regulatory and quality reviews.

Definitions

• Data drift: change in input feature distributions versus baseline.
• Concept drift: change in the relationship between inputs and target (model performance declines despite similar inputs).
• Label drift: shifts in outcome prevalence or coding practice.
• Performance decay: drop in model metrics (AUC, sensitivity, calibration) against labeled data.

Roles & responsibilities

• Model Owner: accountable for monitoring design, thresholds, and remediation decisions.
• Data/ML Engineer: implements telemetry, dashboards, and automated checks; runs triage steps.
• Clinical Safety Lead: assesses clinical risk, approves mitigations and rollbacks, communicates with stakeholders.
• Quality/Compliance: retains audit logs and ensures procedures meet regulatory requirements.
• On-call Team: initial responder for alerts; executes triage checklist.

Baseline performance & acceptance criteria

Document the pre-deployment baseline for each monitored metric and the acceptable operating envelope. Typical baseline items:

• Primary metric (example: AUC, sensitivity) with confidence intervals.
• Secondary metrics (PPV, NPV, calibration slope, false positive rate).
• Throughput and latency limits for inference.
• Data schema and feature cardinalities.
• Fairness checks (performance by subgroups such as age, sex, race where ethically and legally permissible).

Monitoring plan — what to check and how often

Define automated checks and their frequencies:

• Hourly sanity checks: availability, inference latency, error rates.
• Daily checks: input-feature distribution comparisons, missing value rates, population composition.
• Weekly checks: performance on any newly labeled data, calibration plots, subgroup performance.
• Event-driven checks: code deployments, ETL changes, EHR schema changes trigger immediate validation runs.

Detection methods & telemetry

• Statistical tests for feature drift (e.g., KL divergence, population stability index) with configured windows.
• Performance monitoring using near-real-time labeled data where available, or periodic chart review sampling when labels lag.
• Uncertainty and confidence monitoring (prediction confidence distribution shifts).
• Data quality metrics: null rates, value ranges, distribution of categorical levels.
• Operational telemetry: request volumes, latency percentiles, error traces.

Alert thresholds & severity levels

Configure alerts with clear severity and routing. Example structure:

• Severity 1 (Critical): Primary metric drop beyond predefined absolute delta (e.g., AUC drop > 0.08) or any model outputs causing immediate patient risk. Action: stop automated recommendations, notify clinical safety lead and on-call within 15 minutes.
• Severity 2 (High): Significant drift in key input distributions (PSI > threshold) or sustained latency spikes. Action: investigate within 4 hours; may throttle model if degradation confirmed.
• Severity 3 (Informational): Minor distributional changes or non-critical increases in error logs. Action: review in routine weekly meeting.

Alert routing & escalation

1. Alert page creates an incident ticket and posts to the on-call channel.
2. On-call responder follows the Initial Triage Checklist below within the SLA for the alert severity.
3. If unresolved within defined timeboxes, escalate to Model Owner and Clinical Safety Lead.
4. For Severity 1, initiate emergency stand-up and consider immediate rollback or disablement per rollback procedure.

Initial triage checklist (stepwise)

1. Confirm alert validity: check timestamps, duplicate alerts, and downstream incident correlation.
2. Check operational telemetry: recent deployments, feature pipeline errors, and data schema changes.
3. Compare current feature distributions to baseline and recent historical windows.
4. Pull a representative sample of inputs and predictions for quick clinical review (where privacy permits).
5. If labels exist, compute recent performance on labeled samples; otherwise, schedule chart review sampling.
6. Document findings in the incident ticket and determine next action: monitor, apply mitigation, or rollback.

Investigation & root-cause template

Capture the following fields for every investigation:

• Incident ID and alert details (metric, threshold, observed value).
• Time window and affected population.
• Recent code, config, or data pipeline changes.
• Evidence (plots, queries, logs, sample records).
• Hypothesis for root cause and validation tests run.
• Decision and actions taken (mitigation, rollback, patch), with timestamps and approvers.

Mitigation options & rollback procedures

• Throttle or disable automated scoring/decision pathways while preserving logging.
• Fall back to prior validated model version (document model version and activation time).
• Enable human-in-the-loop review for affected cases until resolution.
• Apply hotfixes to data preprocessing or feature extraction if root cause is an ETL bug.

Rollback checklist: identify target version, confirm artifacts in model registry, run smoke tests against a staging dataset, record rollback approval by Model Owner and Clinical Safety Lead, implement and monitor for stabilization.

Auditability & logging

Store the following with secure retention and access controls for audits:

• Raw inputs, model outputs, decision logs, model version IDs, and relevant metadata (de-identified as required).
• All alerts, tickets, triage notes, root-cause analyses, and approvals.
• Dashboard snapshots and query results used during investigation.

Dashboards & sample KPIs

• Primary performance metric trend with confidence bands.
• Feature drift heatmap and PSI over time.
• Subgroup performance comparison.
• Latency p50/p95 and error rate trends.
• Volume by cohort and missing-value trends.

Governance, reviews, and continuous improvement

Schedule periodic reviews (monthly performance review; quarterly safety audit). Maintain a change-control process for model updates that requires test results, monitoring plan updates, and clinical sign-off before production release.

Appendix — practical notes

Sample quick queries, plotting conventions, and suggested retention windows belong in the model-specific appendix. For models with delayed labels, include a chart-review protocol and sampling plan to estimate real-world performance rapidly.

Next steps

Adopt this runbook into the model onboarding checklist; create a versioned monitoring plan per model and instrument the telemetry and dashboards before release. Use the incident template for every alert to build organizational memory.