Analyst–AI Collaboration Guide & Prompt Patterns

Welcome — Why this guide matters

Analysts want useful, repeatable insights faster without trading away accuracy, traceability, or decision ownership. This guide gives practical collaboration patterns, reusable prompt templates, concrete validation checks, and governance guardrails so you can design, pilot, and operate analyst-facing LLMs and agents with confidence.

How to use this guide

Pick the patterns and templates that match your task (exploration, summarization, hypothesis generation, transformation to dashboards). Adapt placeholders to your data, add local acceptance criteria, and capture the prompt + context and outputs as part of each run so results can be reproduced and audited.

Collaboration patterns

Safe prompt templates and constraints

Good prompts are explicit about role, input data, output format, constraints, and validation requirements. Use templates with stable placeholders so prompts remain reproducible.

General EDA prompt (replace placeholders)

"You are a data analyst. Input: dataset name: {{DATASET}}, time range: {{DATE_RANGE}}, key metrics: {{METRICS}}. Task: Provide a concise exploratory summary highlighting main trends, outliers, missingness, and three testable hypotheses. Output format: JSON with keys 'summary', 'data_issues', 'hypotheses' (each hypothesis should include a suggested validation query). Do not assume data not provided. If uncertain, list what additional data is needed."

Dashboard conversion prompt

"You are a dashboard designer. Input: insight: {{INSIGHT_TEXT}}; stakeholders: {{STAKEHOLDERS}}; desired actions: {{ACTIONS}}. Produce a dashboard spec in JSON: { 'tiles': [ { 'title','metric','visualization_type','filters','update_frequency','owner'} ] }. Include suggested thresholds and a short notes field explaining why each tile supports the decision."

Operational constraints to set in the tool or prompt: temperature <= 0.2 for factual outputs, max tokens suitable to capture structured JSON, and explicit instruction to return only the requested structured format to simplify parsing.

Output validation checklist & provenance capture

Every analytic output produced with an LLM or agent should be accompanied by a short validation record and provenance metadata. Capture these automatically when possible.

• Provenance metadata to record: model version & provider, prompt text (including placeholders filled), input data identifiers and query used, timestamp, user who initiated run, agent/toolchain steps, and output hash or ID.
• Validation checklist — For each output, confirm: (a) Data alignment: results align to the specified dataset and time range; (b) Sanity checks: totals and aggregates match quick SQL checks; (c) Missingness: noted and handled; (d) Statistical viability: tests or confidence statements present when claims are probabilistic; (e) Reproducibility: the prompt and inputs are saved; (f) Reviewer approval: a named human reviewer signs off.
• Automated checks to run: run the simple SQL/aggregation queries recommended by the model’s hypotheses, check for nulls/outliers, and compare model-suggested aggregates with direct queries.

Example prompts and expected output shapes

Design your prompts to return structured shapes so outputs are machine-actionable. Example expected shapes:

Prompt patterns for specific analyst tasks

• Data cleaning helper: Ask for suspicious value detection rules and sample SQL to normalize or flag rows.
• Query generation: Provide table schema and desired KPI; request parameterized SQL with explain plan recommendations or index suggestions.
• Narrative drafting: Provide numbered data facts and ask for concise, stakeholder-tailored paragraphs with explicit citation to the fact numbers.

Governance guardrails & human-in-loop rules

• Designate tasks allowed for automated execution vs tasks requiring explicit human approval (e.g., automated alerts ok, monthly strategic reports require human sign-off).
• Require model version, prompt, inputs, and output artifact to be stored with each produced insight for auditability.
• Define acceptance criteria for accuracy and reproducibility before automated actions (e.g., thresholds for acceptable mismatch between model-suggested aggregates and ground-truth queries).
• Mandate periodic review cycles for prompt templates and model performance, including sampling outputs for quality checks and drift detection.
• Escalation rules: if model indicates high uncertainty or conflicting evidence, require human review before any action.

Common mistakes and how to avoid them

• Blind acceptance of fluent narratives — always map claims to a reproducible validation query.
• Using vague prompts — prefer templates with explicit input schema and output schema.
• Lack of versioning — store prompt + model version to avoid irreproducible future behavior.
• No feedback loop — capture reviewer corrections so templates and checks can improve over time.

Quick-start checklist (what to do on your first run)

• Pick a well-scoped dataset and a simple question.
• Use an EDA template prompt and request JSON output with suggested validation queries.
• Run the model, save prompt + inputs + model version, execute the suggested validation queries, and compare results.
• Have a human reviewer sign off and record the decision, recommended actions, and dashboard spec if relevant.

Next steps and capability opportunities

Start by converting frequently used templates into controlled interactive forms so analysts can fill parameter values and have the system capture provenance automatically. Over time, build small agent playbooks that orchestrate: run prompt, execute validation queries, compare results, and surface only human-actionable exceptions for review.

Note: This guide is intentionally practical and adaptable. Treat the templates as starting points — refine them against your data, tooling, privacy requirements, and reviewer conventions.