Analytics Glossary & Quick Reference

Welcome — how to use this reference

This glossary is a single, practical place for consistent analytics vocabulary across teams. Each entry gives a short definition, why the term matters in everyday work, and a one-line example you can adapt. Use it for onboarding, meeting alignment, code comments, reports, and documentation. These are concise, operational definitions — treat them as a shared starting point and adapt them to your domain or local standards before embedding into systems.

How this resource helps

• Reduce miscommunication by agreeing on concise meanings and examples.
• Speed onboarding with clear one-line examples tied to practice.
• Choose the right tests and formulas quickly using the quick-reference sections below.

Core terms (alphabetical)

A/B test

Definition: A controlled experiment that compares two variants (A and B) to measure which performs better against a chosen metric.

Why it matters: Proves whether a change causes improved outcomes rather than relying on intuition.

Example: Run an A/B test to compare two checkout flows and measure conversion rate difference.

Accuracy

Definition: Fraction of correct predictions out of all predictions (for classification).

Why it matters: Simple overall measure but can be misleading with imbalanced classes.

Example: A model that predicts 'no fault' for almost all parts might show high accuracy if faults are rare.

Bias

Definition: Systematic error or deviation in data, measurement, sampling, or model outcomes that causes results to be consistently off-target.

Why it matters: Biased inputs or methods produce unreliable decisions and unfair outcomes.

Example: Training data collected only from daytime shifts may bias a model to ignore night-shift patterns.

Cohort

Definition: A group of subjects/events that share a defined characteristic or start time used for comparison over time.

Why it matters: Cohort analysis reveals changes and trends that aggregate metrics can hide.

Example: Compare retention of users who signed up in January vs. February.

Confidence interval (CI)

Definition: A range of values, calculated from data, that likely contains the true value of a parameter (e.g., mean) at a chosen confidence level (commonly 95%).

Why it matters: Shows uncertainty around estimates — useful for decisions that depend on precision.

Example: The average downtime is 12 ± 3 minutes (95% CI), meaning we are 95% confident the true mean lies in that range.

Confusion matrix

Definition: A table showing true positives, false positives, true negatives, and false negatives for a classifier.

Why it matters: Breaks down errors so you can choose metrics (precision, recall) aligned to business risk.

Example: Use the confusion matrix to prioritize reducing false negatives for safety-critical detection.

Correlation vs causation

Definition: Correlation measures association between variables; causation implies one variable influences another.

Why it matters: Correlation alone cannot justify interventions; causal analysis or experiments are needed for action.

Example: Sales and marketing spend may correlate, but only an experiment can show that increased spend caused sales to rise.

Data lineage

Definition: The record of data origin, transformations, and movement through systems from source to reporting.

Why it matters: Essential for trust, reproducibility, debugging, and compliance.

Example: Use lineage to trace a discrepancy in a KPI back to a change in an ETL script.

ELT / ETL

Definition: ETL (Extract, Transform, Load) transforms data before loading; ELT (Extract, Load, Transform) loads raw data first then transforms it in-place.

Why it matters: Your architecture decision affects latency, storage, and the ability to reprocess data.

Example: Use ELT for flexible analytics when you retain raw data in a cloud data warehouse.

Feature engineering

Definition: Creating or transforming input variables (features) to improve model performance.

Why it matters: Often more impactful than model selection; good features encode domain insight.

Example: Convert timestamp to hour-of-day and weekday flags for demand forecasting.

Overfitting

Definition: When a model learns noise or idiosyncrasies in training data and fails to generalize to new data.

Why it matters: An overfit model performs well in development but poorly in production.

Example: A model that memorizes rare defect IDs will not predict defects on new product lines.

P-value

Definition: The probability of observing data as extreme as (or more than) the observed, assuming the null hypothesis is true.

Why it matters: Helps evaluate evidence against a null hypothesis, but should not be the only decision rule.

Example: A p-value of 0.02 suggests the observed effect is unlikely under the null, but check practical significance and assumptions.

Precision & Recall

Definition: Precision = TP / (TP + FP). Recall = TP / (TP + FN).

Why it matters: Choose between them depending on whether false positives or false negatives are more harmful.

Example: In fraud detection, high precision avoids false alarms; in disease screening, high recall avoids missing cases.

ROC / AUC

Definition: ROC curve plots true positive rate vs false positive rate across thresholds; AUC is the area under that curve.

Why it matters: AUC summarizes ranking quality independent of a particular threshold.

Example: Use AUC to compare models' discrimination ability when class proportions vary.

Seasonality

Definition: Regular, predictable patterns in time-series data occurring at fixed intervals (daily, weekly, yearly).

Why it matters: Forecasting and anomaly detection must model seasonality to avoid false signals.

Example: Sales spike every Saturday — include weekly seasonality in the demand model.

Statistical power

Definition: The probability an experiment detects a true effect of a specified size (1 − Type II error).

Why it matters: Low power means you may miss real, actionable effects; affects sample sizing.

Example: Increase sample size to achieve 80% power to detect a 5% lift in conversion.

Type I / Type II error

Definition: Type I (false positive) rejects a true null; Type II (false negative) fails to reject a false null.

Why it matters: Balance errors based on costs of incorrect decisions.

Example: In safety alarms, a Type II (missed alarm) may be costlier than a Type I (false alarm).

Data drift

Definition: Change in data distribution over time that can degrade model performance.

Why it matters: Requires monitoring and retraining to maintain accuracy and reliability.

Example: Customer behavior shifts after a new product launch, causing prediction errors.

Feature store

Definition: A central repository for storing, serving, and governing machine-learning features across teams.

Why it matters: Promotes feature reuse, consistency between training and production, and governance.

Example: Store a canonical "customer_lifetime_value" feature used by multiple models.

Canary

Definition: A small-scale release or deployment used to validate changes before wider rollout.

Why it matters: Limits blast radius and provides early detection of problems in production.

Example: Deploy a new scoring model to 5% of traffic and monitor key metrics before full rollout.

Quick formulas

• Precision = TP / (TP + FP)
• Recall = TP / (TP + FN)
• Accuracy = (TP + TN) / (TP + TN + FP + FN)
• F1 score = 2 * (Precision * Recall) / (Precision + Recall)
• MAE (mean absolute error) = mean(|pred − obs|)
• RMSE = sqrt(mean((pred − obs)^2))
• MAPE = mean(|(pred − obs) / obs|) × 100%
• 95% CI (approx) = estimate ± 1.96 × SE (when sampling distribution is approximately normal)

Choosing a statistical test: a short guide

Pick tests based on data type, distribution, and design. This is a pragmatic guide — consult a statistician for complex cases.

1. Comparing two group means (continuous, approximately normal): t-test (paired or unpaired as appropriate).
2. Comparing more than two group means: ANOVA (use post-hoc tests to find which groups differ).
3. Non-normal continuous data: Mann–Whitney U (two groups) or Kruskal–Wallis (multiple groups).
4. Categorical association: Chi-square test (large samples) or Fisher’s exact test (small samples).
5. Paired categorical data: McNemar’s test.
6. Correlations: Pearson (linear, both variables approx. normal) or Spearman (monotonic, not necessarily normal).
7. Regression for prediction/adjustment: linear regression (continuous outcome), logistic regression (binary outcome), Poisson/negative binomial (count outcomes).

Common pitfalls & best practices

• Don’t rely on a single metric — pick measures aligned to business risk and interpret them together.
• Check assumptions: normality, independence, equal variance — tests can be invalid if assumptions are violated.
• Watch for data leakage: ensure that training data cannot contain future information from the prediction target.
• Avoid p-hacking and multiple comparisons without correction; pre-specify primary outcomes when possible.
• Monitor models in production for drift and performance degradation; log predictions, inputs, and outcomes where feasible.
• Document definitions and units (e.g., what "conversion" means) to prevent semantic drift across teams.

Apply locally — adaptation checklist

Before embedding terms into operational systems or governance artifacts, run this quick checklist:

• Map terms to local master data definitions and units.
• Confirm acceptable thresholds, confidence levels, and error-cost tradeoffs with stakeholders.
• Document preferred tests and formulas for your domain (export control, regulated industries, safety-critical contexts need specialist review).
• Design simple monitoring for models and metrics (canaries, alerts, retraining triggers).

Where to go next

Consider turning this glossary into an interactive, searchable glossary with usage examples, code snippets, and links to standardized definitions in your enterprise master data. See capability notes below for suggestions on adding interactivity, submission forms for new terms, and usage analytics to understand what your teams rely on most.