Synthetic & Privacy-Preserving Data Assessment Template

{"Title":"Synthetic & Privacy-Preserving Data Assessment","IntroductionHtml":"

This interactive assessment helps teams evaluate whether synthetic data or other privacy-preserving techniques are appropriate for a specific use case. Use it to record context, score privacy risks, compare methods, define validation tasks, and capture an implementation and monitoring plan. The form collects structured answers you can save, share, and iterate on.

Guidance: Answer honestly, include concrete examples, and attach any external validation artifacts to your project record. If you select differential privacy or other configurable methods, record your chosen parameters and rationale.

","SubmitLabel":"Save assessment","SuccessMessage":"Assessment saved. Use the results to plan validation experiments, risk mitigation, and monitoring.","DataType":"SyntheticPrivacyAssessment","SchemaVersion":"1.0","Fields":[{"Key":"use_case_summary","Label":"Brief use-case summary (purpose, consumers, high-level workflow)","FieldType":"textarea","HelpText":"Describe the analytic goals, who will consume the results, whether models are for internal use, shared with partners, or productized. Mention downstream tasks the data must support (e.g., predictive model training, aggregate reporting, exploratory analysis).","Required":true},{"Key":"contains_personal_data","Label":"Does the dataset contain personal data or direct identifiers?","FieldType":"yesno","HelpText":"Direct identifiers include name, SSN, exact address, phone numbers; personal data may include pseudonymous IDs linked elsewhere."},{"Key":"sensitivity_of_attributes","Label":"How sensitive are the attributes (business/health/financial/rare events)?","FieldType":"scale","ScaleMin":0,"ScaleMax":10,"ScaleStep":1,"HelpText":"0 = not sensitive, 10 = extremely sensitive (medical, criminal, financial details, or rare event signals)."},{"Key":"need_for_high_fidelity","Label":"Does the use case require high-fidelity, row-level realism for model performance?","FieldType":"scale","ScaleMin":1,"ScaleMax":5,"ScaleStep":1,"HelpText":"1 = coarse aggregates only, 5 = requires near-identical row-level patterns and rare event fidelity."},{"Key":"regulatory_constraints","Label":"Regulatory or contractual constraints that apply","FieldType":"checkbox","Options":[{"Value":"gdpr","Label":"GDPR"},{"Value":"hipaa","Label":"HIPAA"},{"Value":"ccpa","Label":"CCPA"},{"Value":"other_regulatory","Label":"Other (specify below)"}],"HelpText":"Select any that apply. If Other, note details in Implementation Notes."},{"Key":"external_sharing","Label":"Will data or models be shared outside the organization?","FieldType":"select","Options":[{"Value":"no","Label":"No, internal only"},{"Value":"partners_limited","Label":"Yes — limited partners under contract"},{"Value":"public_release","Label":"Yes — public or broad third-party release"}],"HelpText":"Sharing scope affects acceptable risk tolerance and validation requirements."},{"Key":"privacy_risk_identifiability","Label":"Privacy risk: identifiability / linkability (0 low — 10 high)","FieldType":"scale","ScaleMin":0,"ScaleMax":10,"ScaleStep":1,"HelpText":"Consider ease of linking records to external sources, uniqueness of combinations, and presence of quasi-identifiers."},{"Key":"privacy_risk_impact","Label":"Privacy risk: impact of re-identification if it occurs (0 low — 10 high)","FieldType":"scale","ScaleMin":0,"ScaleMax":10,"ScaleStep":1,"HelpText":"Consider harm to individuals, legal exposure, and reputational damage."},{"Key":"privacy_risk_exploitability","Label":"Privacy risk: exploitability (attacker access level, incentives, known external datasets) (0 low — 10 high)","FieldType":"scale","ScaleMin":0,"ScaleMax":10,"ScaleStep":1,"HelpText":"Higher if many external linking datasets exist or adversaries are motivated and capable."},{"Key":"preferred_methods","Label":"Methods under consideration (select all that apply)","FieldType":"checkbox","Options":[{"Value":"naive_anonymization","Label":"Naive anonymization (removal of direct identifiers)"},{"Value":"k_anonymity","Label":"k-anonymity / l-diversity / t-closeness"},{"Value":"differential_privacy","Label":"Differential privacy (DP) — noise/DP mechanisms"},{"Value":"synthetic_generation_generative","Label":"Synthetic generation — generative models (GANs, diffusion, language models)"},{"Value":"synthetic_statistical","Label":"Statistical synthesis / sampling-based methods"},{"Value":"hybrid","Label":"Hybrid (synthetic + anonymization or DP)"}],"HelpText":"Select the approaches you plan to evaluate. You may select multiple for comparison."},{"Key":"dp_configuration_notes","Label":"If Differential Privacy is considered: planned privacy budget and rationale","FieldType":"textarea","HelpText":"Record planned epsilon/delta, mechanism (Gaussian/Laplace), accounting method, and acceptance criteria. If DP is not considered, leave blank."},{"Key":"generator_details","Label":"If synthetic generation is considered: model family and training constraints","FieldType":"textarea","HelpText":"Note model family (GAN, VAE, diffusion, LLM, task-specific tabular synth), training data partitioning, random seeds, and mechanisms to avoid memorization (e.g., remove outliers, regularization)."},{"Key":"utility_metrics","Label":"Planned utility metrics to validate (select all that apply)","FieldType":"checkbox","Options":[{"Value":"distributional_similarity","Label":"Distributional similarity (marginals, joint distributions)"},{"Value":"predictive_performance","Label":"Downstream predictive model performance"},{"Value":"correlation_structure","Label":"Feature correlation / covariance preservation"},{"Value":"rare_event_support","Label":"Preservation of rare events / tail behaviour"},{"Value":"aggregate_statistics","Label":"Aggregate statistics and reporting consistency"}],"HelpText":"Choose metrics that reflect the real tasks the synthetic data must support."},{"Key":"validation_tasks","Label":"Validation dataset and tasks (describe held-out real tests, tasks and success criteria)","FieldType":"textarea","HelpText":"Example: hold out 20% of real data; train model on synthetic, test on held-out real; require <= X% degradation in AUC and no substantial shift in key aggregates."},{"Key":"validation_holdout_size","Label":"Size of held-out real validation set (rows)","FieldType":"number","HelpText":"Enter the planned number of rows or fraction of dataset reserved for validation."},{"Key":"adversarial_reid_tests","Label":"Adversarial re-identification testing planned?","FieldType":"radio","Options":[{"Value":"yes","Label":"Yes"},{"Value":"no","Label":"No"},{"Value":"partial","Label":"Partial / limited tests planned"}],"HelpText":"Adversarial tests might include record linkage attacks, membership inference, or nearest-neighbor memorization checks."},{"Key":"caveats_checklist","Label":"Implementation caveats & mandatory checks (tick as completed)","FieldType":"checkbox","Options":[{"Value":"validate_against_real","Label":"Validate synthetic models against held-out real data"},{"Value":"adversarial_reid","Label":"Run adversarial re-id and memorization tests"},{"Value":"stakeholder_risk_review","Label":"Stakeholder & legal risk review completed"},{"Value":"provenance_and_versioning","Label":"Provenance recording and versioning controls"},{"Value":"parameter_documentation","Label":"Document all generation/privacy parameters and seeds"},{"Value":"regulatory_documentation","Label":"Regulatory / contractual compliance check"}],"HelpText":"These are minimum operational controls to complete before deployment or sharing."},{"Key":"implementation_plan","Label":"Implementation checklist and caveats (action items, owners, ETA)","FieldType":"textarea","HelpText":"List concrete actions, responsible owners, dates, and acceptance criteria. E.g., create synth pipeline, DP config, run validation, legal sign-off."},{"Key":"monitoring_plan","Label":"Monitoring for leakage and drift in production (what to monitor and frequency)","FieldType":"textarea","HelpText":"Include metrics, alert thresholds, frequency (daily/weekly/monthly), and response playbook. Consider membership inference detection and data drift signals."},{"Key":"overall_recommendation","Label":"Overall recommendation","FieldType":"select","Options":[{"Value":"use_synthetic_with_validation","Label":"Proceed with synthetic data, subject to validation and controls"},{"Value":"use_dp_limited_synthetic","Label":"Use synthetic with differential privacy safeguards"},{"Value":"use_anonymization_and_governance","Label":"Prefer anonymization + governance for sharing"},{"Value":"dont_use_synthetic","Label":"Do not use synthetic for this use case (too high risk or inadequate utility)"},{"Value":"pilot_required","Label":"Run a constrained pilot and re-evaluate"}],"HelpText":"Choose the recommendation the team can act on."},{"Key":"estimated_privacy_risk_score","Label":"Estimated composite privacy risk score (0 low — 100 high)","FieldType":"number","HelpText":"Optionally calculate a composite score using the privacy risk scales above (example: weighted sum normalized to 100) and enter the result here."},{"Key":"final_notes_and_followups","Label":"Final notes, open questions, and next steps (owners & timelines)","FieldType":"textarea","HelpText":"Record unresolved risks, experiments to run, required approvals, and the next review date."}],"HelpText":"Save this assessment as part of your project record. Repeat after any method changes or new validation results."}