Agent & Automation Design Template — Expanded (use case, controls, escalation)

Purpose

This design template helps teams document and evaluate conversational agents and automation workflows before development and deployment. It focuses on clinical safety, human-in-the-loop checkpoints, access and data controls, failure modes, logging and audit needs, and clear escalation pathways.

How to use

Fill each section with concise, actionable information. Use the examples where helpful. After completing the template, run a cross-functional review (clinical lead, IT/security, privacy, quality/risk, operations) and score the risk. Use the testing checklist and acceptance criteria before production release. Keep this document versioned and attach test logs and audit examples.

Template fields and guided prompts

1. Short use-case title

   (One line): e.g., “Auto-schedule MRI prior authorization request.”

2. Use-case description

   What this automation/agent does, who it serves, where it runs (EHR, portal, phone IVR, internal bot), and the value it creates. Include measurable outcomes (time saved, tasks avoided, error reduction targets).

3. Primary users & stakeholders

   List roles (e.g., scheduler, radiology tech, ordering clinician, patient) and responsible approvers/owners.

4. User journey / interaction flow

   Step-by-step happy-path flow and branches. Note explicit points where the user must confirm or may override an automated decision.

5. Triggers & frequency

   What starts the workflow (event, schedule, API call, message) and expected volume.

6. Data inputs & outputs

   List all data elements used and produced. Mark which are PHI, sensitive, or regulated. Include expected formats and maximum retention of outputs.

7. Access controls & authentication

   Who can call, view, modify, or approve. Describe role-based restrictions, audit logging requirements, and any least-privilege design.

8. Decision logic & explainability

   Describe the rules, models, thresholds, or decision matrices used. Explain how the system will present rationale to a clinician or user.

9. Human-in-the-loop checkpoints

   Explicitly list each step where a person must intervene, confirm, or can override. Define acceptable response times and default behavior if the person does not respond.

10. Failure modes & mitigations

    Identify likely failure scenarios (missing data, service outage, wrong patient match, incorrect decision) and the fallback behavior (safest default, queued for human review, fail closed/open). For each failure, include detection signals and immediate mitigations.

11. Logging & audit requirements

    Which events must be logged (inputs, outputs, decisions, overrides, user IDs, timestamps), retention period, log integrity (WORM, immutable storage if required), and where logs are stored. Note any regulatory reporting needs.

12. Testing plan & acceptance criteria

    Define test types (unit, integration, clinical simulation, shadow mode, staged rollout), pass/fail metrics, required clinical signoffs, and performance targets (latency, accuracy). Include a rollback plan and success conditions for production rollout.

13. Escalation pathways & contacts

    Who gets notified on critical failures (on-call clinician, operations, security), notification method (SMS, email, pager), and expected response SLAs. Define who can authorize temporary disabling of the automation.

14. Monitoring & KPIs

    Key metrics to track (error rate, override rate, time saved, patient impact), monitoring frequency, alert thresholds, and dashboard owners.

15. Maintenance & review cadence

    Who owns updates, how model/rule changes are validated, and review frequency (e.g., quarterly), including re-scoring of risk after major changes.

16. Regulatory & privacy notes

    Any relevant regulations, internal policies, or consent required. Data sharing constraints and de-identification needs.

Risk scoring rubric (simple, actionable)

Score probability and impact on 1–5 scales then multiply to create a risk score. Use this to prioritize controls, testing, and human review.

• Probability (1 = rare, 5 = almost certain)
• Impact (1 = negligible, 5 = catastrophic patient harm or major legal/regulatory exposure)

Risk score = Probability × Impact

Interpreting the score:

• 1–4: Low — standard controls, routine testing.
• 5–9: Moderate — require clinician review of decisions, increased monitoring.
• 10–15: High — require human approval for final action, shadow testing prior to release.
• 16–25: Critical — do not automate; require design rework and senior clinical governance signoff.

Testing checklist (pre-production)

1. Unit tests for decision logic and input validation.
2. Integration tests with EHR, scheduling, or external services (including simulated latency and error conditions).
3. Clinical scenario tests (representative patient cases including edge cases and incorrect inputs).
4. Shadow mode run comparing automated decisions to human decisions for a defined period/volume.
5. Failover and outage simulation (service disabled, degraded inputs).
6. Security and privacy review, including penetration test if required.
7. Logging/audit verification: confirm all required events are recorded and retrievable.
8. User acceptance testing with intended end-users (clinicians, schedulers, patients where applicable).
9. Approval from clinical safety, IT/security, and operations with documented acceptance criteria met.

Acceptance criteria (examples)

• Override rate below X% during shadow trial and no adverse patient events attributable to the automation.
• Decision accuracy ≥ agreed threshold (e.g., match human decision in ≥ 95% of validated cases).
• System responds within SLA (e.g., <2 seconds for UI actions, <30s for background decisions).
• All PHI is stored and transmitted according to policy and encrypted in transit and at rest.
• Alerting and escalation tested and acknowledged by on-call staff.

Escalation examples and default behaviors

Design explicit behavior for common escalation scenarios:

• Missing critical data: pause automation, notify responsible clinician and queue for manual review.
• External service outage: enter safe-fail mode (disable automated sends), log incident, and notify ops.
• High-impact disagreement detected in shadow mode: escalate to clinical governance for rule review.

Example (brief, filled)

Title: Auto-notify primary care when high-risk lab result detected.

Use-case: Bot scans lab feed for critical potassium levels, generates message to clinician inbox with recommended actions. Human-in-loop: clinician must confirm treatment before order auto-sent. Probability=2, Impact=5 → Score=10 (High) → Require clinician approval, 24/7 on-call coverage, immutable logging of decision, shadow run 2 weeks before rollout.

Versioning & record keeping

Save each completed template as a versioned artifact linked to change history, approvals, test results, and production incidents. Retain for the time required by policy or applicable regulation.

When to convert this template into an interactive form

If your team will design multiple agents, convert this template into an interactive form that: collects structured fields, stores submissions (for audit and metrics), computes risk score automatically, and ties designs to owners and review workflows. That allows reporting (counts of high-risk automations), scheduled review reminders, and aggregated monitoring dashboards.

Quick reference: fields included

use-case description, user journey, data inputs/outputs, failure modes, access controls, human-in-the-loop checkpoints, logging/audit requirements, testing plan, risk scoring (probability & impact), escalation pathways, monitoring KPIs, acceptance criteria, maintenance & review cadence.

Keep this document living. Re-score after any substantive change to logic, data sources, or integration points.