Checklist

AI Model Ethics & Fairness Review Checklist

An actionable, evidence-capturing checklist for assessing fairness, bias, ethics, and safety risks before deploying clinical AI. The interactive form guides reviewers through data representativeness, subgroup performance, explainability, clinical validation, governance and monitoring, mitigation options, and stakeholder sign-off while saving structured responses for audits and follow-up.

Interactive Tool

AI Model Ethics & Fairness Review Checklist

This interactive checklist helps teams systematically evaluate bias, fairness, ethical risks, and safety concerns before deploying a clinical AI model. Use it to capture evidence, mitigation plans, residual risk, and formal sign-off. Each item includes a space for supporting notes or links to documentation. Save the review to preserve an audit trail and support follow-up actions.

Suggested reviewers: model developer, clinical lead, quality/safety officer, data governance representative, patient advocacy or equity lead (when appropriate).

Confirm the training and evaluation datasets reflect the populations where the model will be used. Check available provenance and known gaps.
Summarize dataset sources, inclusion/exclusion criteria, known sampling biases, and links to dataset documentation or dataset card.
List attributes such as age bands, sex, race/ethnicity, language, socioeconomic proxies, disability status, clinical subgroups, etc., that could affect fairness or safety.
Provide the list of subgroups and why each is relevant to the model's use case.
Indicate whether key metrics (e.g., AUROC, sensitivity, specificity, calibration, PPV/NPV) were computed for each subgroup.
Capture numerical results or paste a summary table. Note any gaps where sample sizes are insufficient.
Document whether differences in outcomes or predicted actions across subgroups were analyzed, and whether potential clinical harms were mapped.
Enter the pre-agreed allowable difference between subgroups (e.g., 5). Leave blank if no threshold defined.
Confirm whether the model provides the level of explanation required for safe clinical use (global or local explanations, feature attributions, clinical rationale).
Describe the explanation techniques used and any limitations or residual interpretability concerns.
Decide whether clinicians must review model outputs before action is taken.
Describe how clinicians interact with outputs, escalation paths, and when overrides are allowed.
Indicate the types of validation completed and whether prospective or real-world pilot data exist.
Summarize study design, population, endpoints, and key results (including failure cases).
List scenarios where the model could fail and planned protections (e.g., hard stops, conservative defaults, alerts).
Describe mitigations, fallback plans, and how to detect and respond to failures.
Confirm compliance with HIPAA and internal policies and whether secondary uses were permitted.
Document data retention, de-identification, and any patient consent or opt-out considerations.
List IRB, regulatory filings, institutional approvals, or third-party certifications as applicable.
Ensure the model card includes intended use, performance by subgroup, limitations, training data summary, versioning and contact information.
Paste stable links or repository locations for artifacts referenced above.
Define metrics, subgroup monitoring, alert thresholds, drift detection, and reporting cadence.
Who will monitor which metrics, how often, and what actions are triggered by alerts.
Choose any mitigation(s) planned prior to deployment.
Describe how mitigation will be implemented, tested, and measured, and the timeline for doing so.
1 = negligible, 5 = high. Consider potential for harm to vulnerable subgroups and likelihood of occurrence.
1.0 10.0
Choose the reviewer recommendation for deployment readiness.
Links to dashboards, notebooks, validation reports, or ticket numbers for related work.
Full name of reviewer
Role (e.g., Clinical Lead, Data Scientist, Quality Officer)
YYYY-MM-DD or system date returned on submit
Any closing notes, next steps, or required actions for deployment.
Typed name acts as formal sign-off. Optionally capture additional approvers in final comments.
You can explore this tool now. Sign in or create an account to save your responses and return to them later.
Sign In to Save Create an Account
Make this tool part of your work

Save a personal copy, bring it to your team, or tailor the questions and workflow to fit what you are hungry to improve.

Member customization and team collaboration are coming soon.
Discuss Ask the Librarian

Discussion

Comments and conversation will live here.