Card

Causal Inference Design Card (quick reference)

A concise one‑page design card to help analysts pick defensible causal approaches when experiments aren't possible. For each method it lists the core identification assumption, typical data needs, common threats, minimal diagnostics to run, and a short documentation template. Includes a practical checklist to decide whether a randomized experiment is feasible.

Causal Inference Design Card — Quick Reference

When randomized experiments aren’t possible, choose the clearest quasi‑experimental design that matches your question, data, and threats. Use this card to compare designs, run minimal diagnostics, and document your identification strategy so decisions remain transparent and prudent.

How to use this card

  1. Match your practical constraints (timing, units, treatment rollout, data frequency) to candidate designs.
  2. State the causal question and the identifying assumption you must accept for each candidate.
  3. Run the minimal diagnostics listed for that design. If diagnostics fail, downgrade confidence or seek alternatives.
  4. Pre‑specify what you will report and the robustness checks you’ll present.

Designs at a glance

Randomized Controlled Trial (Gold standard)

  • When to use: Feasible random assignment of treatment to units without unacceptable ethical or practical constraints.
  • Key assumption: Randomization achieves exchangeability — treated and control groups comparable in expectation.
  • Typical data: Pre/post outcomes, baseline covariates, enough sample size for power.
  • Main threats: Noncompliance, attrition, spillovers/contamination.
  • Minimal diagnostics: Balance tests on pre‑treatment covariates, compliance rates, attrition analysis, pre‑registered analysis plan.
  • Documentation template: question, unit, randomization scheme, stratification/clustering, power calculation, primary outcome, pre‑specified estimand.

Regression Discontinuity (RD)

  • When to use: Treatment assignment changes sharply at a known cutoff (score, date, eligibility threshold).
  • Key assumption: No precise manipulation of the running variable around the cutoff; units just above and below are comparable.
  • Typical data: High density of observations near cutoff; continuous running variable; outcome measured close to threshold.
  • Main threats: Sorting/manipulation at cutoff, functional form misspecification, poor bandwidth choice.
  • Minimal diagnostics: McCrary or density test for manipulation; covariate continuity plots; robustness to bandwidth and polynomial/order choices.
  • Documentation template: cutoff definition, running variable, bandwidth method, local fit, continuity tests, local average treatment interpretation.

Difference‑in‑Differences (DiD)

  • When to use: A group receives treatment while another similar group does not, with repeated pre/post observations.
  • Key assumption: Parallel trends — in absence of treatment, treated and control groups would have evolved similarly.
  • Typical data: Panel or repeated cross‑section data with multiple pre‑ and post‑periods.
  • Main threats: Diverging pre‑trends, differential shocks, compositional changes, anticipatory effects.
  • Minimal diagnostics: Pre‑trend visual checks and formal tests, event‑study estimates, robustness to covariates and alternative control groups.
  • Documentation template: treated/control definitions, timing, number of pre/post periods, event‑study plot, parallel trends diagnostics.

Interrupted Time Series (ITS)

  • When to use: A population experiences a discrete intervention at a point in time and you have frequent time series data.
  • Key assumption: No other coincident policy or structural break explains the change at the intervention time.
  • Typical data: Many time points before and after intervention, consistent measurement process.
  • Main threats: Seasonality, autocorrelation, concurrent interventions, slowly changing trends.
  • Minimal diagnostics: Model autocorrelation (AR errors), seasonal controls, pre‑intervention trend fit, falsification dates.
  • Documentation template: series description, intervention date, model (level/slope change), autocorrelation handling, placebo checks.

Matching / Covariate Adjustment

  • When to use: Rich covariate data exist and treated/control groups can be made comparable on observed confounders.
  • Key assumption: Conditional exchangeability — no unobserved confounding after conditioning on covariates.
  • Typical data: Individual‑level covariates that strongly predict treatment and outcome; sufficient overlap/common support.
  • Main threats: Hidden confounding, poor overlap, model dependence.
  • Minimal diagnostics: Covariate balance checks before/after matching, overlap plots, sensitivity analysis for unobserved confounding.
  • Documentation template: covariates used, matching algorithm, balance metrics, trimmed/weighted sample description.

Instrumental Variables (IV)

  • When to use: There’s a variable (instrument) that affects treatment assignment but not the outcome directly except through treatment.
  • Key assumption: Relevance (instrument strongly predicts treatment) and exclusion (instrument affects outcome only via treatment); monotonicity often assumed.
  • Typical data: Instrument, treatment, outcomes, and covariates; need enough variation in instrument.
  • Main threats: Invalid exclusion restriction, weak instruments, heterogeneous treatment effects complicating interpretation.
  • Minimal diagnostics: First‑stage F statistic for strength, overidentification tests (when multiple instruments), plausibility argument for exclusion, sensitivity checks.
  • Documentation template: instrument description, first‑stage strength, exclusion argument, local average treatment effect interpretation.

Minimal reporting checklist (pre‑specify before seeing outcomes)

  1. Research question and estimand (what exactly you will estimate).
  2. Chosen design and core identifying assumption stated in plain language.
  3. Data sources, sample, inclusion/exclusion rules, and timing.
  4. Primary model, pre‑specified covariates, and estimation method.
  5. Minimal diagnostics to be run and thresholds for concern.
  6. How results will be interpreted for decision making (decision threshold, practical significance).

Quick feasibility checklist for a randomized experiment

Consider an RCT if answers to most of these are 'yes'.

  • Can you ethically and legally randomize the intervention?
  • Is there a clear unit of randomization (individual, clinic, plant, region) and low risk of contamination?
  • Is sample size sufficient for a credible power calculation?
  • Can you pre‑register the outcome and analysis plan and secure cooperation for follow‑up?
  • Are costs and logistics of randomization acceptable compared with the value of stronger evidence?

Practical tips

  • Prefer simpler designs that make assumptions explicit and testable.
  • Always plot raw data: trends, pre/post means, and covariate balance graphs.
  • Report uncertainty and practical effect sizes, not only p‑values.
  • Use placebo/falsification tests where possible (fake cutoffs, leads/lags, unaffected outcomes).
  • Document limitations candidly and describe how findings should inform decisions under uncertainty.

Keep this card handy when framing causal questions — clear assumptions, transparent diagnostics, and a pre‑specified documentation plan are often more valuable than a novel estimator run without context.

Discuss Ask the Librarian

Discussion

Comments and conversation will live here.