Exploratory Data Analysis Validation Checklist & Hypothesis Log

{ "Title": "EDA Validation Checklist & Hypothesis Log", "IntroductionHtml": "

Use this form to record a reproducibility-focused EDA session: verify data quality and provenance, document checks and assumptions, and capture each promising observation as a discrete hypothesis with a test plan, priority, owner and status. Save one hypothesis per submission and repeat as needed. Responses are stored so teams can review, export, and hand off experiments to validation or A/B testing workflows.

How to use: Complete the validation checklist items to the extent possible, then enter a single hypothesis. Submit to save both the checklist status and a hypothesis record. Return to add more hypotheses or to update status as tests progress.

", "SubmitLabel": "Save Checklist & Hypothesis", "SuccessMessage": "Saved. You can add more hypotheses, update this record later, or export saved responses for reporting.", "DataType": "EDAValidationChecklistWithHypothesis", "SchemaVersion": "1.0", "Fields": [ { "Key": "session_name", "FieldType": "text", "Label": "Session name or ID", "HelpText": "Give this EDA session a short name so related submissions can be tracked (e.g., 'Churn_EDA_2026-08-26').", "Required": true }, { "Key": "analyst_name", "FieldType": "text", "Label": "Analyst / recorder", "HelpText": "Person completing this checklist.", "Required": true }, { "Key": "data_sources_verified", "FieldType": "yesno", "Label": "Primary data sources verified", "HelpText": "Have you confirmed the origin, extraction logic, and last-refresh timestamps for each primary source?", "Required": true }, { "Key": "data_dictionary_present", "FieldType": "yesno", "Label": "Data dictionary or schema available", "HelpText": "Is there documentation describing fields, types, units and known transforms?", "Required": true }, { "Key": "missingness_assessed", "FieldType": "yesno", "Label": "Missing values and patterns assessed", "HelpText": "Have you checked for missingness by variable, by segment, and over time?", "Required": true }, { "Key": "outliers_and_ranges_checked", "FieldType": "yesno", "Label": "Outliers, ranges and plausibility checked", "HelpText": "Have you inspected variable ranges, flagged impossible values, and investigated extreme points?", "Required": true }, { "Key": "sampling_and_selection_bias_considered", "FieldType": "yesno", "Label": "Sampling, selection bias and representativeness considered", "HelpText": "Is the sample representative of the target population? Have you identified likely biases?", "Required": true }, { "Key": "preprocessing_logged", "FieldType": "yesno", "Label": "Preprocessing, transformations and filters logged", "HelpText": "Are the exact preprocessing steps (joins, filters, imputation, feature creation) recorded so others can reproduce the analysis?", "Required": true }, { "Key": "random_seed_and_versions", "FieldType": "yesno", "Label": "Random seeds, libraries and code versions recorded", "HelpText": "Were random seeds, library versions and the analysis script or notebook version noted?", "Required": true }, { "Key": "multiple_comparisons_considered", "FieldType": "yesno", "Label": "Multiple comparisons and p-hacking risk considered", "HelpText": "Have you noted where multiple testing might inflate false positives or where selective reporting could occur?", "Required": true }, { "Key": "sensitivity_checks_planned", "FieldType": "yesno", "Label": "Sensitivity or robustness checks planned", "HelpText": "Are there clear follow-up checks (e.g., alternate definitions, holdout splits, stratified tests) you will run to validate findings?", "Required": true }, { "Key": "validation_notes", "FieldType": "textarea", "Label": "Notes on any validation gaps or action items", "HelpText": "Record missing documentation, data quality concerns, or steps needed to make this observation testable.", "Required": false }, { "Key": "hypothesis_title", "FieldType": "text", "Label": "Hypothesis title", "HelpText": "A short, testable hypothesis statement (e.g., 'Users who see promo A have 8% higher conversion than control').", "Required": true }, { "Key": "hypothesis_statement", "FieldType": "textarea", "Label": "Hypothesis (detailed)", "HelpText": "Describe the observation, why you think it might be true, and the expected direction or effect size if relevant.", "Required": true }, { "Key": "evidence_observed", "FieldType": "textarea", "Label": "Evidence / EDA findings", "HelpText": "Summarize the charts, aggregates or patterns that produced this observation (include file names, notebook cells, or figure references).", "Required": true }, { "Key": "proposed_test", "FieldType": "textarea", "Label": "Proposed test or validation plan", "HelpText": "How will you test this hypothesis? (e.g., A/B test, holdout validation, difference-in-differences, predictive model evaluation). Include metrics, success criteria, and sample size notes if known.", "Required": true }, { "Key": "priority", "FieldType": "select", "Label": "Priority", "HelpText": "How important is this hypothesis to business outcomes or risk reduction?", "Required": true, "Options": [ { "Value": "low", "Label": "Low" }, { "Value": "medium", "Label": "Medium" }, { "Value": "high", "Label": "High" } ] }, { "Key": "owner", "FieldType": "text", "Label": "Owner", "HelpText": "Person responsible for converting the hypothesis into an experiment or validation activity.", "Required": true }, { "Key": "target_date", "FieldType": "text", "Label": "Target date for test / next step", "HelpText": "Optional — date or sprint when this should be validated (YYYY-MM-DD or sprint name).", "Required": false }, { "Key": "status", "FieldType": "select", "Label": "Status", "HelpText": "Current status of this hypothesis record.", "Required": true, "Options": [ { "Value": "proposed", "Label": "Proposed" }, { "Value": "in_planning", "Label": "In planning" }, { "Value": "in_testing", "Label": "In testing" }, { "Value": "validated", "Label": "Validated" }, { "Value": "rejected", "Label": "Rejected" } ] }, { "Key": "additional_notes", "FieldType": "textarea", "Label": "Additional notes or links", "HelpText": "Links to notebooks, dashboards, SQL queries, data slices, or tickets that help reproduce or act on this hypothesis.", "Required": false } ] }