Reproducibility & Quality Audit

Preview Cards

Here are the first 5 questions. Create a conversation to invite someone and discuss each card.

1. <section> <h2>Why a Reproducibility & Quality Audit matters — and how to get started</h2> <p>Reproducible research is not a nice-to-have; it is the groundwork of trustworthy results, efficient teams, and reliable progress. Many organizations discover that the problem isn’t one dramatic failure but dozens of small gaps: missing protocol versions, untracked data transforms, ad-hoc file sharing, or inconsistent training. Those small gaps add up to wasted time, irreproducible results, and lost confidence.</p> <p>This audit resource helps you find those gaps quickly, judge which ones matter most, and move from discovery to practical fixes. You can use it for a one-off health check, a quarterly quality review, or as the backbone of a continuous improvement rhythm.</p> <h3>Who this is for</h3> <ul> <li>Principal investigators and lab leads who want dependable results</li> <li>Research managers and operations staff improving lab efficiency</li> <li>Data scientists and analysts who need trustworthy inputs and code provenance</li> <li>Quality and compliance professionals who need a repeatable, documented audit process</li> </ul> <h3>What you’ll get from the resource</h3> <ul> <li>A ready-to-run interactive audit you can fill out during a walk-through or remote review</li> <li>A prioritization worksheet to turn findings into owner-assigned remediation tasks</li> <li>A practical guide explaining how to scope, score, and run the audit without creating unnecessary bureaucracy</li> <li>A short quick-wins checklist for immediate impact</li> <li>Recommended KPIs and monitoring practices to track progress over time</li> </ul> <h3>How to begin right now</h3> <ol> <li>Run the interactive audit during a 60–90 minute walk-through with one team and one sample project.</li> <li>Capture the top 3 findings in the Remediation Worksheet and assign an owner and target date.</li> <li>Start with one quick-win from the checklist (for example: centralize protocol versions) and measure before/after.</li> </ol> <p>Use this resource as a living tool: rerun the audit after fixes, track KPIs, and repeat. The next items will help you run, prioritize, and measure those improvements.</p> </section>
2. { "FormType": "InteractiveForm", "Title": "Reproducibility & Quality Audit — Walk‑through Form", "IntroductionHtml": "<p>Use this interactive audit during a 60–90 minute walk-through or remote review. Answer each question honestly and add short notes or links to evidence. The form collects the structured responses so you can prioritize fixes afterwards.</p>", "SubmitLabel": "Save Audit Responses", "SuccessMessage": "Audit saved. Use the Remediation Worksheet to prioritize your top findings.", "DataType": "ReproducibilityAuditResponse", "SchemaVersion": 1, "Fields": [ { "Key": "audit_title", "Label": "Audit title or project name", "HelpText": "Name the project, lab, or workflow you are auditing (e.g., 'Cell-growth assay — Team B').", "FieldType": "text", "IsRequired": true, "Placeholder": "Project or workflow name", "Minimum": null, "Maximum": null, "Options": [] }, { "Key": "audit_date", "Label": "Audit date", "HelpText": "Date of the walk-through", "FieldType": "text", "IsRequired": true, "Placeholder": "YYYY-MM-DD", "Minimum": null, "Maximum": null, "Options": [] }, { "Key": "auditor_name", "Label": "Person completing the audit", "HelpText": "Name and role of the auditor or lead reviewer", "FieldType": "text", "IsRequired": true, "Placeholder": "Name (Role)", "Minimum": null, "Maximum": null, "Options": [] }, { "Key": "documentation_complete", "Label": "Are key protocols, methods, and SOPs documented and versioned?", "HelpText": "Includes lab protocols, computational methods, and standard operating procedures with clear version history.", "FieldType": "yesno", "IsRequired": true, "Placeholder": "", "Minimum": null, "Maximum": null, "Options": [] }, { "Key": "documentation_accessible", "Label": "Are those documents easy to find and accessible to the team?", "HelpText": "Consider access controls, central storage, and discoverability.", "FieldType": "yesno", "IsRequired": true, "Placeholder": "", "Minimum": null, "Maximum": null, "Options": [] }, { "Key": "protocol_changes_tracked", "Label": "Are protocol changes tracked and dated (change control)?", "HelpText": "Look for change logs, who approved the change, and when it was applied.", "FieldType": "yesno", "IsRequired": true, "Placeholder": "", "Minimum": null, "Maximum": null, "Options": [] }, { "Key": "raw_data_versioned", "Label": "Is raw data versioned and preserved separately from processed data?", "HelpText": "Is there a clear separation and retention of raw vs processed data with file-level metadata?", "FieldType": "yesno", "IsRequired": true, "Placeholder": "", "Minimum": null, "Maximum": null, "Options": [] }, { "Key": "data_processing_transparent", "Label": "Are data-processing steps documented and reproducible (scripts, notebooks, workflows)?", "HelpText": "Look for scripts in version control, workflow definitions, and instructions to re-run analysis.", "FieldType": "yesno", "IsRequired": true, "Placeholder": "", "Minimum": null, "Maximum": null, "Options": [] }, { "Key": "code_reproducible", "Label": "Is analysis code in version control and accompanied by environment information?", "HelpText": "Does the repo include environment specs, container images, or package lists to recreate the environment?", "FieldType": "yesno", "IsRequired": true, "Placeholder": "", "Minimum": null, "Maximum": null, "Options": [] }, { "Key": "instrumentation_calibrated", "Label": "Are instruments and equipment calibrated, and are calibration records maintained?", "HelpText": "Check calibration logs, scheduled maintenance, and recent service records.", "FieldType": "yesno", "IsRequired": true, "Placeholder": "", "Minimum": null, "Maximum": null, "Options": [] }, { "Key": "metadata_collected", "Label": "Does the team consistently capture experiment metadata (who, when, reagents, batches)?", "HelpText": "Metadata improves interpretability and reproducibility. Look for consistent fields, templates, or forms.", "FieldType": "yesno", "IsRequired": true, "Placeholder": "", "Minimum": null, "Maximum": null, "Options": [] }, { "Key": "training_records", "Label": "Are training and competency records for personnel up to date?", "HelpText": "Look for evidence that people running protocols were trained on the exact version of the procedure.", "FieldType": "yesno", "IsRequired": true, "Placeholder": "", "Minimum": null, "Maximum": null, "Options": [] }, { "Key": "quality_checks_defined", "Label": "Are there defined quality checks or acceptance criteria for routine experiments?", "HelpText": "E.g., QC samples, control charts, or automated checks in analysis pipelines.", "FieldType": "yesno", "IsRequired": true, "Placeholder": "", "Minimum": null, "Maximum": null, "Options": [] }, { "Key": "archiving_practices", "Label": "Are results and analysis outputs archived with clear retention policies?", "HelpText": "Where are final results stored and who is responsible for retention?", "FieldType": "yesno", "IsRequired": true, "Placeholder": "", "Minimum": null, "Maximum": null, "Options": [] }, { "Key": "cultural_concerns", "Label": "How concerned is the team about cultural barriers to reproducibility?", "HelpText": "Select the level of concern observed during interviews and walk-throughs.", "FieldType": "select", "IsRequired": true, "Placeholder": "", "Minimum": null, "Maximum": null, "Options": [ { "Value": "low", "Label": "Low concern (team values reproducibility)" }, { "Value": "moderate", "Label": "Moderate concern (some inconsistent practices)" }, { "Value": "high", "Label": "High concern (practices routinely compromise reproducibility)" } ] }, { "Key": "overall_risk_score", "Label": "Overall reproducibility risk (1 = low, 5 = high)", "HelpText": "Your assessment of how likely a routine experiment would be reproducible today.", "FieldType": "scale", "IsRequired": true, "Placeholder": "", "Minimum": 1.0, "Maximum": 5.0, "Options": [] }, { "Key": "top_issues_summary", "Label": "Top 3 issues observed (brief)", "HelpText": "Summarize the most important findings you observed during the audit.", "FieldType": "textarea", "IsRequired": false, "Placeholder": "1) ...\n2) ...\n3) ...", "Minimum": null, "Maximum": null, "Options": [] }, { "Key": "evidence_links", "Label": "Links or locations of evidence (docs, repos, file paths)", "HelpText": "Paste links, repository names, or location paths that support answers above.", "FieldType": "textarea", "IsRequired": false, "Placeholder": "https://... or /share/protocols/...", "Minimum": null, "Maximum": null, "Options": [] }, { "Key": "auditor_recommendation", "Label": "High-level recommendation", "HelpText": "Single-sentence recommendation: e.g., 'Centralize protocol versions and require code repos for all analyses.'", "FieldType": "textarea", "IsRequired": false, "Placeholder": "Recommendation", "Minimum": null, "Maximum": null, "Options": [] } ] }