AI Pilot Data Readiness Checklist (Vision, PdM, Scheduling)

{ "Title": "AI Pilot Data Readiness Checklist (Vision, PdM, Scheduling)", "IntroductionHtml": "

This interactive checklist helps teams confirm whether their dataset is credible for an AI pilot. Use it to document dataset size, labeling quality, temporal and feature coverage, leakage checks, privacy/IP constraints, and a reserved holdout strategy. Answer honestly — gaps found here should be fixed before building models. Use the notes fields to capture evidence, file paths, and next steps.

", "SubmitLabel": "Save Readiness Assessment", "SuccessMessage": "Readiness assessment saved.", "DataType": "ai_pilot_data_readiness", "SchemaVersion": 1, "Fields": [ {"Key":"dataset_name","Label":"Dataset name or ID","FieldType":"text","Required":true,"HelpText":"Unique name, path, or table identifying the dataset."}, {"Key":"pilot_type","Label":"Pilot type","FieldType":"select","Required":true,"Options":[ {"Label":"Vision (image / visual inspection)","Value":"vision"},{"Label":"Predictive Maintenance (PdM)","Value":"pdm"},{"Label":"Scheduling / planning","Value":"scheduling"},{"Label":"Other","Value":"other"}]}, {"Key":"purpose","Label":"Pilot objective (brief)","FieldType":"textarea","HelpText":"What measurable outcome will the pilot evaluate? e.g., detect defects at X% precision/recall; predict failures Y days ahead; improve schedule adherence by Z%."}, {"Key":"total_labeled_examples","Label":"Total labeled examples (all classes)","FieldType":"number","HelpText":"Enter the number of labeled instances (images, events, rows). Useful for estimating statistical power and training feasibility."}, {"Key":"min_examples_per_class","Label":"Minimum labeled examples per target class (if applicable)","FieldType":"number","HelpText":"If vision: examples per defect type. For PdM: number of failure events per failure mode. If uneven, note imbalance strategy."}, {"Key":"class_balance_ok","Label":"Class balance acceptable?","FieldType":"yesno","HelpText":"If NO, plan for resampling, weighting, augmentation, or more collection."}, {"Key":"imbalance_strategy","Label":"Planned mitigation for class imbalance","FieldType":"select","Options":[{"Label":"Collect more data","Value":"collect"},{"Label":"Resample / oversample","Value":"resample"},{"Label":"Use class weighting","Value":"weight"},{"Label":"Synthetic augmentation","Value":"augment"},{"Label":"Modeling strategy (anomaly detection)","Value":"anomaly"},{"Label":"Not decided","Value":"undecided"}]}, {"Key":"labeling_process_documented","Label":"Labeling process documented and reproducible?","FieldType":"yesno","HelpText":"Includes labeling instructions, examples, and quality checks (who labeled, when, tooling)."}, {"Key":"inter_annotator_agreement","Label":"Inter-annotator agreement (if measured, 0–1)","FieldType":"number","HelpText":"Enter Cohen’s kappa, percent agreement, or NA if not measured."}, {"Key":"label_quality_issues","Label":"Known label quality issues","FieldType":"textarea","HelpText":"Describe ambiguous labels, labeling drift, unclear categories, or missing label types."}, {"Key":"temporal_coverage_months","Label":"Temporal coverage (months of history)","FieldType":"number","HelpText":"How many months or cycles does the dataset span? Ensure seasonality, production cycles, and maintenance windows are included."}, {"Key":"horizon_coverage","Label":"Horizon coverage (prediction window)","FieldType":"text","HelpText":"For scheduling: forecast horizons covered (e.g., 4 weeks). For PdM: look-ahead days available for features and labels."}, {"Key":"signal_continuity_ok","Label":"Signal continuity and sensor uptime adequate?","FieldType":"yesno","HelpText":"For time-series/PdM ensure sampling rates, gaps, and outages are understood and documented."}, {"Key":"feature_completeness_percent","Label":"Estimated feature completeness (%)","FieldType":"number","HelpText":"Percent of records with required features present. Low completeness usually needs feature engineering or more collection."}, {"Key":"missing_values_policy","Label":"Missing value policy","FieldType":"select","Options":[{"Label":"Impute (describe)","Value":"impute"},{"Label":"Drop records","Value":"drop"},{"Label":"Add missing indicator features (flag)","Value":"flag"},{"Label":"Not decided","Value":"undecided"}]}, {"Key":"data_leakage_checks_done","Label":"Data leakage checks completed?","FieldType":"yesno","HelpText":"Checks include future data in features, target leakage via timestamps, and duplicated records across splits."}, {"Key":"data_leakage_notes","Label":"Data leakage notes / findings","FieldType":"textarea","HelpText":"Document any leaky features found and the mitigation applied (e.g., drop columns, shift windows)."}, {"Key":"train_val_test_split","Label":"Planned train/validation/test split (%)","FieldType":"text","HelpText":"E.g., 70/15/15, or temporal holdout details. For time-series, prefer temporal splits rather than random sampling."}, {"Key":"holdout_reserved","Label":"Separate holdout / blind test set reserved?","FieldType":"yesno","HelpText":"A reserved holdout unseen during model selection provides realistic evaluation and reduces overfitting risk."}, {"Key":"holdout_details","Label":"Holdout set details","FieldType":"textarea","HelpText":"Describe selection method, size, selection dates, and storage location/path."}, {"Key":"synthetic_test_cases","Label":"Synthetic or edge-case test cases prepared?","FieldType":"yesno","HelpText":"Useful for rare defects, extreme scheduling scenarios, or failure modes that are underrepresented."}, {"Key":"synthetic_test_description","Label":"Synthetic test case description","FieldType":"textarea"}, {"Key":"privacy_ip_reviewed","Label":"Privacy and IP review completed?","FieldType":"yesno","HelpText":"Includes customer data, identifiable images, vendor data, and licensing of third-party datasets."}, {"Key":"privacy_notes","Label":"Privacy or IP restrictions / actions required","FieldType":"textarea"}, {"Key":"data_storage_location","Label":"Data storage location / catalog path","FieldType":"text"}, {"Key":"data_owner_contact","Label":"Dataset owner / contact","FieldType":"text"}, {"Key":"baseline_defined","Label":"Baseline model or rule defined for comparison?","FieldType":"yesno","HelpText":"A simple baseline (heuristic, rules, or trivial model) helps judge whether the ML model adds value."}, {"Key":"success_metrics_defined","Label":"Primary success metrics defined","FieldType":"yesno","HelpText":"List metrics to decide pilot success (e.g., precision/recall, lead time accuracy, MAE, schedule adherence)."}, {"Key":"go_no_go_criteria","Label":"Go / No‑Go criteria (brief)","FieldType":"textarea","HelpText":"Concrete thresholds that determine whether to scale the pilot (e.g., >X% precision and stable for Y weeks)."}, {"Key":"overall_readiness","Label":"Overall readiness (1=Not ready, 5=Ready)","FieldType":"scale","HelpText":"Subjective summary after completing the checklist."}, {"Key":"readiness_notes","Label":"Additional notes, risks, and next steps","FieldType":"textarea"} ] }