Version control & reproducible workflows runbook

Purpose

This runbook explains actionable practices to keep code, analyses, environments, containers, data, and model outputs versioned and auditable so research results can be reproduced, validated, and trusted.

Scope

Applies to analysis code, Jupyter/RMarkdown notebooks, scripts, container images, environment specifications, datasets (small and large), model artifacts, and CI/automation that produce results.

Principles (why these matter)

• Small, traceable changes: make each change reviewable and attributable.
• Immutable artifacts: store checksums or content-addressed identifiers for datasets and models.
• Captured environment: record exact environment used to run an experiment (packages, OS, container digest).
• Reproducible pipelines: prefer scripted, parameterized runs over manual GUI steps.
• Provenance metadata: keep a manifest describing inputs, code commit, parameters, date/time, and random seeds.

Who owns this runbook

Recommended owners: project lead or research software engineer (RSE) for a project; lab lead or data platform team at organizational scale. Ownership includes curating repository layout, CI templates, and onboarding documentation.

Git workflow: recommended patterns

Use a simple, consistent branching strategy that supports review and reproducible releases.

Branch naming

• main (or master): always stable and releasable.
• develop (optional): integration branch for ongoing work.
• feature/ — new analyses or features.
• fix/- — bug fixes or corrections.
• release/vMAJOR.MINOR.PATCH — preparation for a reproducible release.

Tagging and releases

Tag commits that produced published results with a semantic tag and include metadata in the release notes: code commit SHA, container digest, dataset versions, parameters, and CI artifact links. Example tag: v1.2.0 or v2025-08-15-commitsha.

Example git commands

git checkout -b feature/add-analysis-x
# work, commit small incremental changes
git commit -m "add: data cleaning step for X with tests"
git push origin feature/add-analysis-x
# open a pull request, review, merge to main
git tag -a v1.0.0 -m "Release for paper draft"

Notebooks: maintainability & versioning

• Prefer script-first development (scripts + small notebooks for visualization) or keep notebooks paired with scripts using jupytext so diffs are readable.
• Strip outputs before committing, or use git filters to remove outputs automatically.
• Use nbdime for notebook diffs and reviews.
• Parameterize notebooks with papermill or papermill-like approaches so runs are reproducible and parameter values are recorded.
• Record notebook metadata including kernel, python version, and package list for each run.

Environment & containerization

• Record environment specifications: conda environment.yml, pip requirements.txt, or pip-lock style files (pip-tools / pip freeze / pip-lock).
• Prefer lockfiles (exact package hashes) for reproducibility (conda-lock, pip-tools, poetry lock).
• Build container images for reproducible runtime. Capture container image digest (sha256) and push to a registry.

  Include labels in Dockerfile with metadata (commit SHA, build date, maintainer):

  LABEL org.opencontainers.image.revision="${GIT_COMMIT}" \
        org.opencontainers.image.created="${BUILD_DATE}"

• For HPC or sensitive environments consider Singularity/Apptainer images built from the same Dockerfile.

Data & artifact versioning patterns

Design patterns depending on dataset size:

• Small datasets: keep in repo under data/ with a manifest and checksum (sha256).
• Large datasets: store in an external artifact store (S3, object store, institutional data repository). Keep only pointers (URN, S3 URI) plus immutable object hash in the repo.
• Use data-versioning tools where appropriate: DVC, git-annex, Quilt, Delta Lake, or a dataset registry. Always record dataset version IDs and checksums alongside experiments.
• Model outputs: store models as immutable artifacts in a model registry (MLflow, S3 with manifests, or institutional registry) and record the artifact URI and checksum in the experiment metadata.

Provenance & metadata capture

For each experiment/run capture a manifest that includes at minimum:

• code_repo: URL and commit SHA
• container: image name + digest (sha256) or environment lockfile
• inputs: dataset URIs and checksums
• parameters: all non-default parameters used
• random_seeds: explicit seed values
• outputs: artifact URIs and checksums
• timestamp and actor (who ran it)
• CI build ID or run link when automated

Store this manifest in the artifact storage alongside results and include it in the release tag or publication supplementary materials.

Continuous integration and checks

• CI should run linting, tests, and at least one smoke reproducibility check (small dataset or mock inputs) for pull requests.
• As part of release CI, record resulting artifacts (container digest, environment lockfile, artifact URIs) into the release manifest.
• Use CI secrets carefully for credentials to artifact stores; avoid committing credentials to code.

Reproducibility checklist (adoptable)

1. Code under version control with clear branch and PR history.
2. Release tagged with commit SHA and release notes including manifest.
3. Notebooks paired with scripts or committed with outputs stripped and diffs readable.
4. Environment captured with a lockfile or container with recorded digest.
5. Datasets referenced by immutable URIs and checksums; large data not stored directly in the repo.
6. Model artifacts stored in a registry with version identifiers and checksums.
7. Experiment manifest saved with code commit, container digest, inputs, parameters, seeds, outputs, timestamp, and actor.
8. CI reproduces a minimal end-to-end run and publishes artifacts used for full runs.

Common mistakes and how to avoid them

• Committing large binary data into git — instead use DVC/git-annex or external stores and store pointers/checksums in git.
• Committing notebooks with outputs — enable a pre-commit hook to strip outputs or use jupytext pairing.
• Not recording environment versions — prefer lockfiles or container images with digests.
• Using floating dependency versions (e.g., pandas>=1.0) without lockfiles — produces unreproducible runs.

Example minimal release manifest (JSON snippet)

{
  "repo": "git@example/research.git",
  "commit": "3a1f2b4",
  "tag": "v1.0.0",
  "container": "registry.example.org/research:1.0.0@sha256:...",
  "datasets": [{"uri": "s3://project/datasets/X.csv", "sha256": "..."}],
  "parameters": {"alpha": 0.1, "n_iter": 1000},
  "outputs": [{"uri": "s3://project/models/m1.pkl", "sha256": "..."}],
  "timestamp": "2026-08-15T12:00:00Z",
  "actor": "alice@example.org"
}

Tools & templates (recommended)

• Source control: Git + GitHub / GitLab / Bitbucket
• Notebooks: jupytext, nbdime, papermill
• Data versioning: DVC, git-annex, Quilt
• Environment lock: conda-lock, pip-tools, poetry
• Containers: Docker, Singularity/Apptainer
• Model registries: MLflow, S3 with manifest + registry layer
• CI: GitHub Actions / GitLab CI / Jenkins with reproducibility job templates

Next steps & tailoring

Adopt the checklist as a baseline, then tailor these patterns to your institutional policies (data residency, security, storage quotas). Create repository templates and CI job templates that enforce these checks for new projects.

Quick-start checklist for a new project

1. Create repo from template that includes: README, CODE_OF_CONDUCT, CONTRIBUTING, .gitignore, pre-commit hooks to strip notebook outputs.
2. Add environment lockfile and example container Dockerfile with label placeholders for commit and build metadata.
3. Create a data/manifest pattern and document how to register large datasets in the project’s artifact store.
4. Add CI smoke test that runs a minimal, deterministic pipeline and uploads a manifest to the artifact store on release.

If you want, we can convert the checklist and templates into interactive onboarding forms, repository templates, CI job templates, and an audit that validates repos for reproducibility.