AI Incident Response & Forensics — Runbook Template

Purpose

This runbook helps teams quickly and confidently respond to incidents where AI outputs, model behavior, or model-driven automation cause harm, incorrect actions, privacy breaches, reputational risk, or operational disruption. Use this playbook to triage, contain, preserve evidence for forensics, communicate with stakeholders, remediate root causes, and capture lessons.

Scope & When to Use

Use this runbook for incidents that arise from AI components or where AI outputs materially contributed to an adverse outcome. Examples include unsafe recommendations, large-scale inaccurate outputs, data leakage, model drift causing incorrect decisions, biased outputs affecting users, automated actions causing system failures, and exposed sensitive data in responses.

Roles & Responsibilities

• Incident Commander (IC): Owns incident coordination, severity decisions, stakeholder updates, and post-incident review scheduling.
• AI Forensics Lead: Leads evidence collection, model & data snapshot, and technical root-cause analysis.
• Model Owner / ML Engineer: Provides model versions, training metadata, feature definitions, and deployment configs.
• Data Engineer: Provides data lineage, recent datasets, data pipeline logs, and access to storage snapshots.
• SRE / Platform: Executes containment actions (disable endpoints, rollback deployments, apply throttles/feature flags) and provides system logs and telemetry.
• Legal & Compliance: Advises on regulatory reporting, privacy obligations, and public statements.
• Communications / Product: Drafts internal and external messages and coordinates customer communications.

Quick Start Triage Checklist (first 15–30 minutes)

1. Record discovery time, reporter identity, and initial summary of observed issue.
2. Assign Incident Commander and briefly notify core responders (Forensics Lead, Model Owner, SRE).
3. Classify suspected scope: single user, single subsystem, multiple customers, public exposure.
4. Capture a volatile snapshot (see Forensics Snapshot) immediately if possible; do not overwrite logs or restart systems until evidence is secured.
5. Decide initial containment action: disable endpoint, rollback to last known-good model, apply rate limits, or switch to human-in-the-loop mode.
6. Open an incident channel (chat, ticket, war room) and begin a timeline log of actions and observations.

Severity Levels (suggested)

• Severity 1 (Critical): Safety, legal, or large-scale customer impact (e.g., data leak, harmful medical advice, financial loss at scale). Immediate containment required and likely regulator notification.
• Severity 2 (High): Significant customer impact, sensitive incorrect outputs, or cascading system failures affecting multiple users.
• Severity 3 (Moderate): Isolated incorrect outputs, degraded performance, or reproducible but limited errors requiring remediation.
• Severity 4 (Low): Cosmetic or minor issues with little user impact; schedule remediation in normal backlog.

Containment Actions (choose least-disruptive effective action)

• Disable or throttle the offending model endpoint via feature flag or API gateway.
• Rollback to the previous model version or configuration known to be stable.
• Switch to a safe fallback (human review, rule-based system, or lower-risk model).
• Stop automated actions triggered by model outputs (pause downstream workflows).
• If sensitive data is exposed, isolate storage, revoke keys, rotate credentials, and consult Legal/Privacy.

Forensics Snapshot Checklist (preserve evidence immediately)

Make read-only copies whenever possible. Document the chain of custody for any extracted data.

• Model artifacts: model ID, version, commit hash, container image tag, build metadata.
• Configuration: serving config, hyperparameters, temperature/seeding, prompt templates, safety filters, rate limits.
• Input/output logs: raw inputs, full outputs, timestamps, user/session IDs, request headers.
• System logs & telemetry: application logs, API gateway logs, latency/error rates, CPU/memory, autoscaling events.
• Data snapshots: recent training/validation datasets, feature stores or last-inserted examples, data pipeline snapshots.
• Environment: deployment environment, dependencies, OS/container state, infrastructure-as-code versions.
• Repro scripts: note any seeds or pseudo-random state if applicable and the steps needed to reproduce the output.
• Checksums & metadata: compute hashes for all saved artifacts, note who collected them and when.

Triage Flow (short)

1. Confirm AI involvement. If AI uncertain, collect evidence and escalate to Forensics Lead.
2. Classify severity and select containment action.
3. Implement containment and verify mitigation effects.
4. Collect a full forensic snapshot before further changes.
5. Conduct root-cause analysis and implement short-term fixes and long-term remediation plan.

Evidence Handling & Preservation

Preserve original logs and artifacts in immutable storage when possible. Limit access and record all access. Use checksums to prove integrity. If human review is required, create copies to avoid altering originals.

Communication Templates (editable)

Internal (short)

Subject: [INCIDENT] AI model incident — [summary]

Severity: [1/2/3/4]

Summary: Brief description of observed behavior, affected customers/systems, containment action taken.

Action: Owners and next steps. Incident channel: [link].

Customer-facing (example)

We are contacting you about an issue affecting [service/feature]. Our AI system produced results that were [inaccurate/exposed data/other]. We have temporarily [disabled/rolled back/throttled] the feature while we investigate. We will provide an update by [timeframe]. If you believe you were impacted, please [contact/support link].

Regulator / Legal (example)

Do not send regulator-level notices without Legal review. Prepare a factual timeline, scope of impact, data types involved, containment actions, and remediation plans for Legal to review and approve.

Root Cause & Remediation

After containment and evidence collection, run an RCA that addresses:

• How and why the model produced the harmful output (data drift, training issue, prompt change, feature bug, third-party model change).
• Gaps in monitoring, alerting, testing, or deployment safeguards that allowed the issue to reach production.
• Short-term fixes (patch, rollback, tightened filters) and long-term fixes (retraining, new tests, stronger gating, additional instrumentation).

Post-Incident Review & Tracking

1. Document timeline with timestamps for discovery, containment, evidence collection, and key actions.
2. Host a blameless post-incident review within 1–2 weeks that includes technical staff, product, legal, and communications.
3. Create remediation tickets with owners, acceptance criteria, and target dates. Track remedial actions to closure.
4. Update runbooks, tests, monitoring, and training datasets as part of remediation closure.

Monitoring & Prevention Recommendations

• Instrument model inputs/outputs for drift detection, distribution shifts, and safety-related signals.
• Set alerts on error spikes, unusual output patterns, and confidence-score anomalies.
• Maintain a model registry with immutable versions, metadata, and quick rollback capability.
• Implement automated pre-deployment checks, unit and integration tests for prompts and output constraints, and staged rollouts with telemetry gates.

KPIs & Postmortem Metrics

• Time to detect
• Time to contain
• Time to remediate
• Number of affected customers/sessions
• Number of recurrence incidents after remediation

Runbook Ownership & Review Cadence

Runbook Owner: [Team/Role]. Review this playbook quarterly or after any Severity 1–2 incident. Update contact lists, commands, and templates as systems change.

Appendix: Quick Checklist (one-page)

1. Assign IC and open incident channel.
2. Preserve volatile evidence (logs, inputs/outputs, model version).
3. Contain (disable/rollback/fallback).
4. Notify Legal & Communications if sensitive data or public impact.
5. Collect full forensic snapshot.
6. Perform RCA, create remediation tickets, and schedule postmortem.

For teams that want to operationalize this runbook: consider converting the Quick Checklist and Forensics Snapshot into interactive incident intake and evidence-collection forms so responses and artifacts are stored with the incident record.