Citizen Data Scientist Handbook & Guardrails

Welcome

This handbook helps domain experts—product managers, operations leads, clinicians, shop foremen, marketers, educators, and others—use data responsibly to answer questions, test ideas, and improve outcomes without creating risk or duplicate work. It defines what you can do, how to do it safely, what to document, and when to bring in analytics, data engineering, or ML teams.

Scope and Goals

Use these guardrails to:

• Produce decision-focused, reproducible analyses that are understandable to colleagues.
• Protect privacy, security, and enterprise reporting integrity.
• Prevent orphaned models, duplicated effort, and hidden assumptions.
• Provide clear handoffs when specialized skills, infrastructure, or governance are required.

Who May Do What (Role-Aware Allowed Analyses)

The following matrix explains typical permissions and responsibilities. Local organizations should adapt these roles and map them to named people or job codes.

• Domain Expert (non-analyst): Explore data with approved tools, run standard templates, create descriptive charts and summary tables, and submit results with documented assumptions. Must follow data access rules and validation checklist.
• Power User: All Domain Expert privileges plus ability to join datasets at defined keys, run simple regressions or A/B analyses using templates, and propose experiments. Results must be peer-reviewed before operational changes.
• Citizen Data Scientist (trained): Can perform hypothesis-driven analyses, build reproducible notebooks from approved templates, and run low-risk models on approved datasets. Required to register projects in the team registry and follow review gates.
• Analytics/ML Team: Responsible for complex modeling, production model deployment, data engineering tasks, ensuring lineage, and high-risk approvals. Must assist handoffs and create enterprise-grade artifacts when work scales.

Data Access Rules

Follow these mandatory rules whenever you access organizational data.

• Use only approved datasets and query endpoints listed in the data catalog. If needed data is not available, request it through the data-access request process—do not create local copies of raw source systems.
• Handle any personally identifiable information (PII) or protected health information (PHI) according to privacy policy. If your analysis touches sensitive fields, obtain explicit approval and anonymize or use safe-views before analysis.
• Work in approved environments (sandbox, governed notebooks, or BI tool workspaces). Avoid downloading entire datasets to uncontrolled devices.
• Record data provenance: dataset name, version, extraction date/time, filters applied, and any transformations. This must accompany every shared result.
• Respect licensing and third-party data contracts. Check with data governance if unsure.

Simple Validation Checklist (useful as an interactive form)

Before sharing results or acting on them, confirm these items. This checklist is intentionally short so it can be used as a quick gating step.

1. Purpose: Is the question or decision you’re supporting clearly stated and linked to action?
2. Data provenance: Are source names, versions, and extraction dates documented?
3. Appropriate data: Are sensitive fields handled correctly or removed/obfuscated?
4. Sanity checks: Do basic aggregates (counts, sums, means) match known reporting or expectations?
5. Bias & confounding: Have obvious confounders been considered and noted?
6. Statistical checks: Were tests appropriate for the sample size and variable types? (See cheat sheet for common pitfalls.)
7. Reproducibility: Can a colleague rerun the analysis using the provided notebook/script/template and get the same results?
8. Documentation: Is a short README included with question, data sources, transformations, assumptions, and next steps?
9. Escalation: If work could change production systems, customer experience, billing, safety, compliance, or high-volume automation, escalate to Analytics/ML/Data Engineering.

Each checklist item should be recorded (yes/no/comment) and stored with the project entry in the team registry.

Approved Tools & Templates

Use only approved tooling and share outputs consistently. Examples include:

• Governed BI workspaces for dashboards (tool-name placeholder)
• Reviewed notebook templates (Python/R) with data-access connectors and logging
• Pre-built analysis templates: descriptive summary, cohort comparison, A/B analysis, time-series checklist
• Version-controlled artifact storage (git or enterprise alternative) and readme templates

Templates should include standard header metadata: project name, owner, date, data sources, version, and validation checklist results.

When to Involve Data Engineering, Analytics, or ML Teams

Involve specialists when work meets any of these criteria:

• High impact: potential to change revenue, safety, legal status, patient care, or core operational processes.
• Productionization: the analysis or model will be automated, scheduled, or embedded into production systems.
• Complex data needs: requires joining multiple sensitive sources, large-scale processing, or new data ingestion pipelines.
• Model risk: training predictive models affecting customers, compliance, or decisions with regulatory exposure.
• Unclear reproducibility: results cannot be reproduced using provided artifacts and environment descriptions.

When escalating, include the validation checklist, reproducible notebook or query, data provenance, and a short description of the desired outcome and risks.

Lightweight SLAs and Review Steps

To avoid blocking innovation while protecting the enterprise, use lightweight SLAs:

• Low-risk requests: auto-approval or 3 business day review window.
• Medium-risk requests: 5 business day review with designated reviewer from analytics or data governance.
• High-risk requests: formal review meeting within 10 business days with required sign-offs.

Documentation & Provenance Template

Every project should include a single README with these fields:

• Project title and short purpose statement
• Owner and collaborators
• Data sources and extraction queries
• Transformations and feature logic
• Validation checklist results and known limitations
• Actionable recommendation or next steps
• Escalation status (if any) and links to review notes

Training Plan (short, modular)

Training should be role-based and pragmatic. Suggested modules:

• Intro to Responsible Citizen Data Science: scope, hunger, and guardrails (1 hour)
• Working with the Data Catalog and Provenance (1 hour)
• Using Approved Templates & Tools (hands-on, 2 hours)
• Simple Statistical Thinking & Validation Checklist (2 hours)
• Escalation Process & SLAs (30 minutes)

Certification can be lightweight—complete modules, pass a short quiz, and submit a supervised mini-project.

Quick Start Cheat Sheet: Common Statistical Pitfalls

This one-page guide is a quick desk reference for common traps.

• Small samples: be cautious about percentages; report counts and confidence intervals.
• Multiple comparisons: adjust expectations when running many tests; risk of false positives.
• Correlation vs causation: correlations can suggest hypotheses but not prove causality without design or adjustment.
• Selection bias: consider how the sample was collected and who is excluded.
• Confounding variables: check for obvious alternative explanations and include them when possible.
• Outliers: investigate before removing—outliers can be data issues or real signals.
• Data leakage in models: ensure training data does not include future information that wouldn’t be available in production.

Practical Next Steps & Implementation Notes

To operationalize this handbook consider:

• Converting the validation checklist into an interactive form that stores responses with each project (recommended capability).
• Publishing approved templates in a version-controlled library tied to the data catalog.
• Running a pilot certification workshop with a small group of domain experts and measuring outcomes.

Keep the handbook living: collect feedback, track near-misses, and improve templates and SLAs based on experience.

Appendix: Useful Links and Placeholders

• Data catalog (link placeholder)
• Approved templates library (link placeholder)
• Project registry / submission form (link placeholder)
• Analytics & ML team contact (link placeholder)