Checklist

Synthetic Data & Simulation — Synthetic Data Generation Checklist

Interactive checklist to help teams decide whether to use synthetic data, choose appropriate generation methods and fidelity, run essential privacy and quality checks, and document governance, limitations, and next steps.

Interactive Tool

Synthetic Data Generation Checklist

This interactive checklist helps teams plan and evaluate synthetic datasets and simulations for experiments, model testing, product development, and operations. Use it to be explicit about purpose, required fidelity, privacy controls, validation steps, governance, and known limitations. Saving responses creates an auditable record you can revisit before moving toward production or real-data validation.

Project or experiment name

Short name so reviewers can trace this checklist to related artifacts.

Owner / responsible person

Who is responsible for the dataset, testing, and follow-up actions.

Date

YYYY-MM-DD or other convenient format.

Primary purpose for synthetic data

Select the most important reason you’re generating synthetic data.

If other, describe purpose

Short description if 'Other' was chosen above.

Will results from synthetic data be used to make production decisions?

If yes, stronger validation and governance are required.

Required fidelity for your purpose

Choose the minimum acceptable realism for your experiments.

Planned generation methods (select all that apply)

Multiple methods are common — record what you will use.

Rule-based / heuristics

Domain simulator / agent-based model

Model-based generative (GANs, VAEs, diffusion, language models)

Differential privacy mechanisms

Hybrid / augmentation of real records

Privacy controls applied (select all that apply)

Be explicit — 'no controls' requires strong justification and governance.

Differential privacy

K-anonymity / l-diversity

Record suppression / masking

No privacy-specific controls applied

Have you tested for leakage of real records or re-identification risk?

Examples: nearest-neighbor comparisons, membership inference tests, MAE to real records.

Leakage / re-identification test notes and results

Summarize test methods, key metrics, and pass/fail criteria. Attach artifacts to project records if available.

Have you validated distributional similarity and key statistics?

Consider KS tests, MMD, pairwise correlations, marginal distributions, and conditional behavior.

Key similarity metrics and thresholds (optional)

Record the metrics you used (e.g., KS p-values, MMD values) and the thresholds you consider acceptable.

Have you tested downstream model or process performance on synthetic vs. real data?

Train/evaluate models on synthetic data and compare to real-data baselines where possible.

Downstream validation summary

Summarize performance differences, failure modes, and whether synthetic results would change decisions.

Have you documented limitations, known biases, and blind spots?

Be explicit about scenarios where synthetic data is unreliable.

Limitations, biases, and conditions where synthetic data is unreliable

Examples: rare event frequencies, chained correlations, temporal drift, policy-sensitive fields.

Governance review status

Indicate whether privacy, legal, or domain governance reviews are complete.

Estimated risk level of using this synthetic data for decision-making

1 = very low risk (safe experimentation only), 5 = high risk (could mislead production decisions).

1.0 10.0

Is explicit approval required before using results in production?

If yes, record approver(s) in next field.

Approver(s) (if required)

Names or roles expected to sign off (privacy, legal, product, ops).

Next steps and recommended follow-up

Examples: real-data validation plan, additional leakage tests, governance sign-off, production pilot.

You can explore this tool now. Sign in or create an account to save your responses and return to them later.

Make this tool part of your work

Save a personal copy, bring it to your team, or tailor the questions and workflow to fit what you are hungry to improve.

Member customization and team collaboration are coming soon.

Discuss Ask the Librarian

Discussion

Comments and conversation will live here.

Synthetic Data & Simulation — Synthetic Data Generation Checklist

Synthetic Data Generation Checklist

Discussion

Services

Explore

Contact