Statistical Thinking Quick Reference

Statistical Thinking — Quick Reference

Purpose: fast, practical reminders analysts and decision-makers repeatedly need — what to check, which test to consider, how to interpret results, and quick formulas to guide sample-size and effect-size thinking. Use this as a starting point, not a substitute for domain-specific standards or professional consultation.

Quick orientation — Before you test

• What is the question? (estimation, difference, association, prediction)
• What are the variables? (continuous, ordinal, categorical, counts)
• Is the design independent or paired/repeated?
• What is an acceptable margin of error or meaningful effect?
• What assumptions must hold (normality, independence, equal variance)?

Distributions — short reminders

• Normal: symmetrical, mean = median; many parametric tests assume approximate normality for residuals.
• Binomial: yes/no outcomes (proportion); use for counts of successes out of n.
• Poisson: counts per unit (rare events, varying exposure).
• CLT rule of thumb: sample means approach normality; n ≥ 30 is a common rough guide but depends on skewness.

Central tendency & variability (formulas)

• Mean: x̄ = (Σxi) / n
• Median: middle value (useful when skewed)
• Variance: s² = Σ(xi - x̄)² / (n-1) (sample variance)
• Standard deviation (SD): s = sqrt(s²)
• Interquartile range (IQR): Q3 − Q1 (robust to outliers)
• Standard error of the mean (SE): SE = s / sqrt(n)

Confidence intervals — interpretation & quick formula

Interpretation: a 95% CI around an estimator is the range that, under repeated sampling and the same procedure, would contain the true parameter about 95% of the time. It expresses uncertainty about estimation; it is not a probability statement about a single fixed value.

Quick formula (mean): CI ≈ x̄ ± z* × SE (z* ≈ 1.96 for 95%). For small samples and unknown variance, use t* with df = n−1.

p-values — practical interpretation

• A p-value is the probability of observing data as extreme (or more) than what you saw, assuming the null hypothesis is true. It is not the probability the null is true.
• Don't treat thresholds as magic; report exact p and CI. Consider effect size and context.
• Pre-specify alpha (commonly 0.05). For many comparisons, control the familywise error (Bonferroni) or use FDR methods.

Effect sizes — quick measures & interpretation

• Cohen's d (difference in means): d = (x̄1 − x̄2) / s_pooled. Benchmarks: small ≈ 0.2, medium ≈ 0.5, large ≈ 0.8 (context matters).
• Correlation r: strength benchmarks: small ≈ 0.1, medium ≈ 0.3, large ≈ 0.5.
• Odds ratio (OR): OR = (a/b) / (c/d) — useful for binary outcomes; interpret multiplicatively.
• Always report CIs for effect sizes — they show practical importance beyond statistical significance.

Power & sample-size rules of thumb

• Typical target power: 80% (0.8). Higher power needed for smaller effects or stricter decisions.
• For a two-sample t-test detecting a medium effect (d ≈ 0.5) with 80% power and α=0.05, you typically need ~64 total (≈32 per group). For small effects (d ≈ 0.2), n grows quickly (several hundred per group).
• Sample size for a proportion (approx): n ≈ (Z² × p(1−p)) / ME² (Z=1.96 for 95% CI). Use p=0.5 for conservative estimate.
• For mean: n ≈ (Z × s / ME)², where ME is desired margin of error and s is estimated SD.

Which test? A compact decision flow

• Outcome continuous, two groups, independent: t-test (use Welch's t if unequal variances).
• Outcome continuous, two groups, paired/repeated: paired t-test (or Wilcoxon signed-rank if non-normal).
• Outcome continuous, >2 groups: one-way ANOVA (follow with contrasts); Kruskal–Wallis if non-parametric.
• Outcome binary, two groups: chi-square or Fisher's exact (if small counts); logistic regression to adjust covariates.
• Association between two continuous variables: Pearson (linear, normal residuals) or Spearman (monotonic, nonparametric).
• Counts/rates: Poisson or negative binomial regression for rate data with exposure.
• Nonparametric alternatives: Mann–Whitney U (independent), Wilcoxon signed-rank (paired).
• Adjust for covariates: regression models (linear, logistic, Poisson) rather than many stratified t-tests.

Common pitfalls

• Multiple comparisons without correction → inflated false positives.
• Selection bias, survivorship bias, or analyzing a non-representative sample.
• P-hacking: repeated testing, selective reporting, and stopping rules that inflate type I error.
• Assumption violations: independence, normality (of residuals), equal variance.
• Confusing statistical with practical significance — always contextualize effect sizes and costs/benefits.
• Overreliance on single test: consider estimation, visualization, and robustness checks.

Quick checklist before reporting

1. State question, data source, and design (paired/independent, randomized/observational).
2. Summarize distribution and outliers visually (histogram, boxplot) and numerically (median, IQR, mean, SD).
3. Choose test based on measurement level & design; check assumptions.
4. Report effect sizes with CIs and exact p-values; note sample sizes and missing data.
5. If multiple tests, state correction method or rationale for not correcting.
6. Offer a simple practical interpretation (what would change if effect is real?).

Further reading & tools

Use this sheet as a practical reminder. For critical decisions, consult domain standards or a statistician. Consider simple calculators for sample size, power, and effect-size conversion when planning studies.

Short note of caution

This cheat sheet is a practical reference, not a substitute for professional statistical consultation, domain-specific standards, or validated procedures. Avoid copying definitions into enterprise systems without tailoring to local master data and governance.