Statistical Tests Quick Reference Card

How to use this card

This compact reference helps you map a practical question and data type to an appropriate statistical test, recall the main assumptions to check, and interpret typical outputs. Use it as a quick decision aid — not a substitute for a formal analysis plan, power calculation, or statistical consulting for high-stakes decisions.

Quick decision checklist

• Outcome continuous, two independent groups → Independent two-sample t-test (or Welch's t-test)
• Outcome continuous, two paired measurements → Paired t-test
• Outcome continuous, 3+ groups → One-way ANOVA (+post-hoc) or Kruskal–Wallis
• Outcome ordinal or continuous non-normal, two independent groups → Mann–Whitney U (Wilcoxon rank-sum)
• Two categorical variables → Chi-square test of independence (or Fisher's exact if small counts)
• Two continuous variables, linear association → Pearson correlation; if monotonic but nonlinear or ranks preferred → Spearman

Independent two-sample t-test (including Welch's)

Purpose: Compare means of a continuous outcome between two independent groups.

When to use: Outcome is continuous and approximately normal within groups, groups are independent. If variances differ, use Welch's t-test.

Assumptions: independent observations, approximate normality of the outcome in each group (t is robust with moderate n), similar variances (unless using Welch).

Sample-size pointers: For medium effects, ~30 per group is often adequate; smaller effects require larger samples. Always run a power calculation for planning.

Example H0/H1: H0: meanA = meanB. H1: meanA ≠ meanB.

Typical outputs & interpretation: t-statistic, degrees of freedom, p-value, CI for mean difference. If p < α, reject H0. Report effect size (Cohen's d) and CI.

Common mistakes & alternatives: Applying t-test to highly skewed small samples (consider Mann–Whitney or bootstrap); ignoring unequal variances (use Welch's); treating non-independent data as independent.

Paired (dependent) t-test

Purpose: Compare means of paired measurements (e.g., before/after on same subjects).

When to use: Paired observations and differences are approximately normally distributed.

Assumptions: differences between paired observations are roughly normal; pairs are independent of other pairs.

Typical outputs: t-statistic on differences, df, p-value, CI for mean difference.

Common mistakes: Treating paired data as independent; using paired t-test when the pairing is not meaningful.

One-way ANOVA

Purpose: Test whether three or more group means are equal.

When to use: Continuous outcome, 3+ independent groups, roughly normal within-group distributions.

Assumptions: independence, normality within groups (ANOVA is robust with moderate samples), homogeneity of variances (check with Levene's test).

Sample-size pointers: Balance across groups improves power; use power analysis for planned contrasts or post-hoc comparisons.

Typical outputs & interpretation: F-statistic, df (between/within), p-value. A significant F indicates at least one group mean differs; follow up with post-hoc tests (Tukey, Bonferroni) or planned contrasts.

Common mistakes & alternatives: Ignoring multiple comparisons; using ANOVA when distributions are clearly non-normal with small samples (consider Kruskal–Wallis or transformations).

Chi-square test (χ²)

Purpose: Test association between two categorical variables (contingency table) or goodness-of-fit to expected categories.

When to use: Both variables categorical; expected cell counts should generally be ≥5 for validity of χ² approximation.

Assumptions: independent observations, adequate expected cell counts; for small samples use Fisher's exact test.

Typical outputs & interpretation: χ² statistic, df, p-value. Large χ² and small p suggest dependence or deviation from expected distribution. Examine standardized residuals to identify cells driving the result.

Common mistakes & alternatives: Applying χ² with low expected counts (use Fisher's exact), interpreting association as causation, ignoring ordered categories (consider trend tests).

Mann–Whitney U (Wilcoxon rank-sum)

Purpose: Compare two independent groups when the outcome is ordinal or continuous but not assumed normal.

When to use: Two independent groups, distributional assumptions of t-test violated, data at least ordinal.

Assumptions: independent observations, distributions have similar shapes if you want the test to reflect medians rather than general distributional shifts.

Typical outputs & interpretation: U statistic, p-value, and rank-sum summaries. A significant result indicates a difference in central tendency or distributional location.

Common mistakes & alternatives: Interpreting it strictly as a median test when distributions differ in shape; for matched pairs use Wilcoxon signed-rank; for 3+ groups use Kruskal–Wallis.

Correlation — Pearson & Spearman

Purpose: Measure strength and direction of association between two variables.

Pearson: measures linear association between two continuous variables; assumes bivariate normality (or approximate linearity and no large outliers).

Spearman: rank-based measure for monotonic relationships; robust to outliers and non-normality.

Typical outputs & interpretation: correlation coefficient (r or ρ), p-value, confidence interval. |r| near 0 little linear association; near 1 strong linear association. Always visualize (scatterplot) to check linearity and outliers.

Common mistakes & alternatives: Interpreting correlation as causation; ignoring nonlinearity (consider transformations or nonparametric measures); reporting correlation without scatterplots.

Practical sample-size & planning notes

• Always perform a power analysis during study planning using expected effect size, α (commonly 0.05), and desired power (commonly 0.8–0.9).
• Rule-of-thumb minimums (very rough): >= 20 per group for many t-test situations; but small samples with strong effects can work and large samples can detect trivial effects — both contexts require thought.
• For categorical tests, ensure expected cell counts are adequate; consider collapsing rare categories or using Fisher's exact for small samples.

General interpretation tips

• Report effect sizes and confidence intervals, not just p-values.
• Check assumptions (normality, homogeneity of variance, independence) before relying on parametric test results; when in doubt, visualize the data and consider robust or nonparametric alternatives.
• Adjust for multiple comparisons when making many pairwise tests.
• Context matters: statistical significance ≠ practical importance.

When to consult a statistician

Use this card for quick decisions and reviews. Consult statistical expertise when experiments are complex (blocking, clustering, repeated measures, missing data), when regulatory or legal decisions depend on results, or when planning studies with critical power or safety implications.

Reminder: This reference is a practical cheat sheet, not a substitute for domain-specific standards, detailed analysis plans, or professional statistical consultation.