Measuring reproducibility: KPIs and a simple dashboard

Measurement keeps improvement accountable. These KPIs are pragmatic and observable for most research teams. Use them to track progress across audits and to report improvements to stakeholders.

Recommended KPIs

• Audit risk score (median) — Median overall reproducibility risk across recent audits (scale 1–5). Tracks trend over time.
• % Projects with canonical protocol version — Share of audited projects that have a single canonical protocol location and version history.
• % Analyses in version control — Share of audited projects with analysis code in a version-control system and environment spec.
• Mean time to remediate top issue — Average days between issue identification and remediation completion for prioritized items.
• % Experiments with required metadata — Portion of experiments that include the agreed minimum metadata fields.
• QC pass rate — Percent of routine runs meeting predefined QC acceptance criteria.

Dashboard layout (recommended)

1. Top row: Median audit risk (trend sparkline), number of audits completed this quarter, % remediation completed.
2. Middle row: Bar chart by category (Documentation, Data, Analysis, Instruments, Training) showing count of open issues.
3. Bottom row: Table of top 10 open issues with owner, impact, effort, and target date.

Notes on measurement

Start small. Capture a handful of KPIs consistently and avoid adding metrics that are hard to observe. Automate what you can (for example, checking repo presence via scripts), but do not wait for perfect automation to begin measurement.

Using KPIs responsibly

KPIs should guide improvement, not punish teams. Use them as a diagnostic tool in collaborative huddles and to allocate resources for remediation rather than as a punitive scorecard.