Audit

Reproducible analysis checklist

An actionable audit checklist to verify code, data, environment, and documentation for reproducible analyses — capture evidence, assign owners, set priorities, and record remediation actions.

Interactive Tool

Reproducible analysis checklist

Use this structured audit to quickly assess common reproducibility gaps in analysis pipelines. For each item, record evidence links, assign an owner, set priority, and note required actions. Submissions are saved to the project record so teams can track remediation progress.

Include whether both code and dataset snapshots are under version control. Examples: Git repo URL, DVC-tracked dataset, S3 snapshot with version id.
Paste links, commit hashes, dataset IDs, or short instructions for locating the exact revision used.
Examples: Dockerfile + image tag, Conda environment.yml, pip/poetry lockfile, or list of system dependencies.
Link to Docker image, CI build logs, or a file path in the repo that reproduces the environment.
Rate from 1 (no provenance) to 5 (complete provenance and machine-readable metadata). Consider lineage, collection method, preprocessing steps, and identifiers.
1.0 10.0
Summarize missing fields, undocumented transformations, or opportunities to add metadata (schema, units, sample descriptions).
Indicate whether an analysis plan exists and how accessible it is.
Link to the analysis protocol, preregistration, or the file/section describing planned outcomes and decision rules.
Unit tests, integration tests, or lightweight smoke tests that validate core steps and expected outputs.
Describe what is tested, how to run tests (commands or CI links), and current pass/fail status.
A short statement describing how to access code, data, environment, and any restrictions (licenses, sensitive data protections).
Paste the statement or a link to the manuscript section / repository file.
Assessor judgment of how likely the analysis is to be reproduced by an independent team given current artifacts.
1.0 10.0
Person or role responsible for implementing fixes or improvements.
Set priority so teams can schedule follow-up work.
If yes, describe the remediation below and assign follow-up steps.
Describe what will be done, by whom, and by when. Examples: add environment capture, tag data snapshot, create tests, publish reproducibility statement.
Starter suggestions: 1) Tag and snapshot data; 2) Add Dockerfile or environment lockfile; 3) Add CI tests for core pipeline; 4) Publish reproducibility statement and access instructions; 5) Pre-register analysis or document decision rules.
Use YYYY-MM-DD or a short note on timing. Stored as text for portability.
List links to PRs, issues, storage objects, or DOI records useful to reproduce the analysis.
Name or identifier of the person completing this audit.
You can explore this tool now. Sign in or create an account to save your responses and return to them later.
Sign In to Save Create an Account
Make this tool part of your work

Save a personal copy, bring it to your team, or tailor the questions and workflow to fit what you are hungry to improve.

Member customization and team collaboration are coming soon.
Discuss Ask the Librarian

Discussion

Comments and conversation will live here.