Experiment Failure & Retrospective Guide

Purpose

This playbook helps teams turn failed or inconclusive experiments into clear, usable learning. Use it to reduce repeat mistakes, preserve organizational memory, redesign better follow-up experiments, and keep team morale and curiosity intact.

When to use this

Run a short retrospective when an experiment ends with an unexpected result, ambiguous outcome, or clear failure to reach success criteria. Use after the immediate facts are still available but emotions have cooled enough for constructive conversation (often 24–72 hours).

Quick facilitator checklist (prep)

• Invite key contributors and one neutral facilitator (can be the PI if trained in facilitation).
• Collect and attach artifacts (see Evidence Capture checklist below).
• Schedule 45–90 minutes depending on experiment complexity.
• Share the retrospective goal: learn, not blame. Clarify desired output (actions, redesigned experiment, knowledge saved).
• Prepare a shared workspace (document, whiteboard, or the platform’s interactive form).

Suggested agenda (60 minutes)

1. Welcome & purpose (5 minutes) — set norms and psychological safety.
2. Quick factual recap (10 minutes) — read the evidence summary and measurements.
3. Timeline reconstruction (10 minutes) — place major events, decisions, and parameter changes on a timeline.
4. Root-cause exploration (20 minutes) — use directed prompts and a selected RCA technique.
5. Action definition & experiment redesign (10 minutes) — assign owners, due dates, and verification steps.
6. Close & follow-up plan (5 minutes) — who does what, where results are stored, and next check-in date.

Pre-retrospective data collection (what to gather)

Having concrete artifacts makes the session factual and fast. Collect:

• Primary experiment protocol and version used.
• Raw measurements, processed data files, plots, and statistical outputs.
• Instrument logs, timestamps, and environmental data (temperatures, reagent batches, lot numbers).
• Code, scripts, configuration files, and container/virtual environment details.
• Photos, video, or screenshots of setup and results.
• Deviation notes, incident reports, and any troubleshooting steps taken during the run.
• Prior related experiment summaries and expected success criteria.

Timeline reconstruction (practical method)

On a shared timeline, capture:

• Key decisions and who made them.
• Parameter changes or unplanned events (e.g., power fluctuation, reagent spiking).
• When measurements were taken and by whom.
• When anomalies first appeared.

Use colored markers or tags for: (a) facts, (b) assumptions, and (c) hypotheses.

Facilitation script & prompts (use as-is or adapt)

Open with norms: “We’re here to understand what happened so our next experiment is wiser. We’ll focus on facts first, then interpretations.”

1. Factual recap (ask): “Who ran the experiment, when, and which protocol version?” Record short bullets.
2. Data review (ask): “What measurements and outputs do we have? Do they match our recorded success criteria?”
3. Timeline prompts: “What happened just before the anomaly? What changed in the setup or environment?”
4. Evidence vs assumption: For each claim, ask: “Is this a recorded fact, or an assumption?”
5. Root-cause prompts: Use the RCA options below to structure investigation.
6. Action prompts: “Given what we know, what specific experiment redesign or check should we do next? Who will own it and how will we verify?”

Root-cause analysis options

5 Whys (fast)

Pick one clear failure outcome and ask “Why?” up to five times to move from symptom to a plausible root cause. Document each step and whether it’s supported by evidence.

Fishbone / Ishikawa (broader)

Use categories such as Materials, Methods, Equipment, Measurement, People, Environment. Populate each branch with observed facts and testable hypotheses. This is useful when multiple contributing factors are likely.

Evidence check

For each candidate cause, list the evidence that supports or refutes it and the simplest test that would confirm or reject the hypothesis.

Evidence capture checklist (what to save and tag)

• Data files with clear filenames and experiment ID.
• Protocol version and any deviations annotated inline.
• Instrument logs, serial numbers, and calibration status.
• Reagent lot numbers, expiration dates, and storage notes.
• Photographs of the setup and anomalies (with timestamps).
• Who ran which steps and any ad-hoc notes they made.
• Link to the retrospective summary and the action items in your knowledge base.

Action assignment & experiment redesign checklist

Every action should be SMART and verifiable. Use a table or card for each action with these fields:

• Action description (clear, short)
• Owner (single person)
• Due date
• Verification metric or pass/fail test
• Dependency notes (what must be ready first)
• Where results will be stored (experiment ID, folder, notebook link)

Redesign checklist for the next run:

• Explicit success criteria and measurement plan.
• Control runs and boundary tests to isolate suspected causes.
• Smaller stepwise changes rather than multiple simultaneous changes.
• Instrumentation check/calibration before starting.
• Versioned protocol and code deployment (tagged).
• Pre-register the verification test so success is objective.

Follow-up cadence and organizational memory

Assign a short follow-up check-in (1–2 weeks after action due dates). Store the retrospective summary, artifacts, and action outcomes in a searchable location, tag with experiment ID, people, and process area. Consider an annual review of recurring failures to identify systemic problems.

Facilitator tips for psychological safety

• Model curiosity: start with “I’m curious what the data shows” rather than blame.
• Name the difference between fact and interpretation explicitly.
• Encourage brief, equal airtime; call for quick written notes if one voice dominates.
• Celebrate small wins (what did we learn?) before listing actions.

Sample outputs (what success looks like)

• A one-page learning summary with a clear hypothesis about root cause, supporting evidence, and two prioritized follow-up experiments.
• Assigned actions with owners, deadlines, and verification steps recorded in the organizational knowledge base.
• Versioned protocol and code updates linked to the retrospective.

How to convert this playbook into an interactive tool (capability notes)

This playbook maps naturally to an interactive retrospective form where teams submit evidence, build the timeline collaboratively, and capture actions. Use the platform’s interactive form rendering and data submission capabilities to:

• Collect pre-retrospective artifacts and structured metadata (experiment ID, protocol version, reagent lots).
• Render timed agenda and guided prompts for facilitators.
• Save retrospective submissions (actions, evidence links, hypotheses) to the content item data store for tracking and analytics.

Next steps

Use this playbook as a template. Start by running a single retrospective and saving the output as a reusable template in your domain. After two or three runs, adapt the prompts and evidence checklist to your lab’s common failure modes and consider packaging the set as a reusable Experiment Retrospective Toolkit for other teams.