Causal Discovery & Inference — Practitioner Primer

Welcome — why causal thinking matters now

Teams often ask whether a change actually caused an outcome or whether an observed association is a misleading coincidence. Sound causal reasoning turns observational data into actionable decisions—helping product teams prioritize features, operations leaders decide process changes, clinicians interpret non‑randomized evidence, and policymakers evaluate interventions—without mistaking correlation for cause.

What this primer helps you do

Learn to pose clear causal questions, choose feasible quasi‑experimental strategies, document and test assumptions, run pragmatic validity checks, and decide when to call a causal specialist. This is a practical guide—aimed at analytics, experimentation, operations, and product teams—so you can raise the baseline of causal reasoning where randomized trials are unavailable, impractical, or incomplete.

Start with a clear causal question

Good causal work begins with a specific, testable question and a target population. Examples:

• Product: Did enabling the new checkout flow increase completed purchases among mobile users in Region A?
• Operations: Did a new training program reduce the mean time-to-repair on Line 3?
• Health services: Did extending clinic hours reduce no‑show rates among patients aged 65+?

Write the counterfactual plainly: "What would have happened to the same users/units if we had not introduced X?" This clarifies the causal estimand and the data you need.

Core causal concepts (plain language)

• Counterfactual: The outcome that would have occurred without the intervention. Causal inference compares observed outcomes to plausible counterfactuals.
• Confounder: A variable that affects both treatment and outcome (creates spurious association if not accounted for).
• Selection bias: When treated and untreated groups differ systematically for reasons related to the outcome.
• Directed acyclic graph (DAG): A simple causal diagram that helps make assumptions explicit and reveal potential confounders.
• External validity: Whether results generalize beyond your study population, time, or context.

Quick overview of practical designs

Below are common quasi‑experimental approaches, when they are useful, and what to watch for.

Difference‑in‑Differences (DiD)

Compare pre/post changes between treated and comparison groups. Works well when a plausible untreated comparison exists and trends would have been parallel without treatment.

Key checks: pre‑trend tests (visual and statistical), balance on covariates, robustness to varying pre‑ and post‑windows, and placebo (fake treatment dates).

Regression Discontinuity (RD)

Use a sharp eligibility cutoff (e.g., score, date) where units just above and below the threshold are assumed comparable. Estimates local causal effects around the cutoff.

Key checks: continuity of covariates, McCrary test for bunching/manipulation, sensitivity to bandwidth and polynomial order, graphical inspection of the jump.

Instrumental Variables (IV)

Use an instrument—an external variable that affects treatment but not the outcome except through treatment—to isolate exogenous variation. Example instruments: random assignment to encouragement, distance to facility.

Key checks: first-stage strength (F‑statistic), plausibility of exclusion restriction, monotonicity assumption, overidentification tests when multiple instruments exist.

Interrupted Time Series (ITS) & Synthetic Control

ITS: Analyze long series before and after a discrete intervention, adjusting for trends and seasonality. Synthetic control: build a weighted combination of untreated units to approximate the treated unit's counterfactual.

Key checks: model fit before intervention, placebo tests on donors, sensitivity to donor pool and weighting choices.

Matching and Weighting

Construct treated and control groups with similar observed covariates (propensity scores, coarsened exact matching). Matching helps reduce observable confounding but cannot fix unobserved confounders.

Key checks: covariate balance diagnostics, overlap/positivity, sensitivity to caliper and matching algorithm.

Practical stepwise plan for a causal analysis

1. Define the causal estimand precisely: population, treatment, outcome, time window.
2. Draw a simple causal diagram (DAG) and list observable and unobservable confounders.
3. Identify feasible designs given your data and context (DiD, RD, IV, ITS, synthetic control, matching).
4. Pre-register or document your analysis plan: choice of outcome, estimation window, covariates, and robustness checks.
5. Run the main specification and stepwise robustness checks (below).
6. Write a short findings section that: states assumptions, shows main estimate(s), presents sensitivity tests, and outlines limitations and implications for action.

Practical validity checks and sensitivity analyses

• Pre‑trend and placebo tests: check for differences before treatment or using false treatment times.
• Covariate balance: show that observed characteristics are similar between groups (before and after weighting/matching).
• Sensitivity to specification: vary functional forms, time windows, bandwidths (RD), donor pools (synthetic control).
• Instrument diagnostics: report first‑stage strength (F > 10 rule of thumb), and discuss instrument plausibility.
• Permutation or randomization inference: compute placebo treatment effects by randomly assigning treatment to donors to assess significance.
• Quantify unobserved confounding sensitivity: Rosenbaum bounds, E‑values, or other sensitivity metrics where appropriate.
• Heterogeneity checks: test whether effects vary across meaningful subgroups (but correct for multiple testing).

Common pitfalls and how to avoid them

• Mistaking correlation for causation — always state and evaluate key assumptions explicitly.
• P‑hacking and specification searching — predefine primary specification and treat other analyses as exploration.
• Ignoring selection into treatment — document why treated units might not be comparable and choose a design that addresses selection.
• Overgeneralizing localized estimates — RD and DiD often estimate local effects; be cautious when scaling to other populations or times.
• Weak instruments — a weak first stage leads to biased IV estimates; prefer alternative designs or stronger instruments.

A compact practitioner checklist

1. Is my causal question precise? (population, intervention, outcome, timing)
2. Which plausible quasi‑experiment fits the context?
3. Have I drawn a DAG and listed major confounders?
4. Did I predefine the primary specification and key robustness checks?
5. Do pre‑treatment trends/balance support my design?
6. Did I run sensitivity analyses (bandwidths, placebo dates, alternative controls)?
7. Have I reported assumptions, limitations, and the practical decision implications?

Deliverables your team should produce

• Short one‑page causal question + DAG + chosen design.
• Main results table and concise effect size in practical units (e.g., percent increase, tickets saved/day).
• Key diagnostic plots (pre‑trends, RD jump plot, balance tables, synthetic control fit).
• Robustness and sensitivity appendix with clear interpretation.
• Decision note: what action (if any) is justified given assumptions and evidence?

When to call a causal specialist

Bring in a specialist when:

• The design requires advanced identification (complex IV strategies, staggered DiD with heterogeneous effects, panel data fixed‑effects subtleties).
• The instrument plausibility or exclusion restriction is contested and needs careful justification.
• Regulatory, safety, or clinical risks mean errors could cause harm.
• The analysis influences major budgetary or policy decisions and needs expert defensibility under scrutiny.
• You need formal sensitivity quantification, causal mediation analysis, or transportability/external validity adjustments.

Next practical steps for your team

1. Pick a current decision that depends on non‑experimental data and draft the one‑page causal question and DAG.
2. Map available data to the variables in your DAG and identify likely confounders and instruments.
3. Run a simple diagnostic (pre‑trend plot or balance table) to see whether a basic DiD or matching approach is feasible.
4. Document your assumptions and, if results look promising and consequential, schedule a review with a causal specialist.

Further reading & tools

Suggested topics to explore when you want to deepen skills: causal DAGs and do‑calculus, advanced DiD estimators for staggered rollouts, local randomization RD, synthetic control practical guides, permutation inference, and sensitivity analysis methods (Rosenbaum bounds, E‑values). Consider short workshops or pairing with an applied causal analyst for your first few projects.

Use this primer to raise the baseline of causal reasoning in your analytics and experimentation teams. The goal is not to make everyone a causal econometrician, but to ensure teams ask the right questions, document critical assumptions, apply feasible quasi‑experimental designs responsibly, and only act when evidence and assumptions justify operational change.