Playbook

AI Opportunity Scan Playbook

A practical, step-oriented playbook for discovering, evaluating, and prioritizing AI opportunities that balance value, feasibility, and risk — including data-readiness checks, a reproducible scoring model, a rapid prototyping plan, and an early ethics review.

Welcome

This playbook helps teams discover where AI can create measurable value in products, services, or operations while avoiding common traps such as chasing hype, launching unscoped pilots, or neglecting data, ethics, and governance. Use it as a living template: adapt checklists, scoring weights, and success criteria to your organization and the problem context.

Who this is for

Product leads, operations managers, business analysts, data engineers, research teams, and innovation leads who want a practical, low-risk path from opportunity discovery through an initial experiment and decision about scaling.

What you will produce

A prioritized list of AI pilot candidates, with a recommended first experiment for each top candidate plus clear success criteria, data-readiness notes, feasibility risks, and an early ethics screening.

Opportunity discovery and candidate mapping

Gather a short list of candidate problems or processes where AI might help. Sources include customer feedback, frontline staff observations, operational metrics, quality incidents, product backlogs, and competitive scanning.

Candidate capture template (use one row per candidate)

Candidate name (short)
Clear problem statement — who is affected and what harm or missed opportunity exists
Desired outcome — what change would count as success
Primary user or stakeholder
Current baseline metric(s) (if known)
Estimated potential impact (qualitative/quantitative)
Notes on current systems and data sources

Data readiness checklist

Assessing data readiness early prevents wasted effort. For each candidate, evaluate these items and capture short answers.

Does relevant data exist? (yes/no/partial) — Describe location and ownership.
Is the data accessible for development and testing? (APIs, exports, logs)
Is data volume sufficient for the approach you expect to use? (examples, sample size)
Is the data labeled, or can labels be generated cheaply? (manual labeling, weak labels)
Are there known data quality issues? (missingness, bias, duplicates)
Are privacy or regulatory constraints likely to block access or use?
Can a realistic synthetic or surrogate dataset be created if needed?

Feasibility and ROI scoring model

Use a simple scoring rubric that balances value, feasibility, and risk. Keep it transparent so stakeholders understand choices. Suggested dimensions and point ranges are below. Adjust weights to match your strategic priorities.

Scoring dimensions (example)

Value (0–10) — likely business impact, revenue, cost savings, or customer experience improvement.
Data & Tech Feasibility (0–10) — data readiness, integration effort, existing platforms.
Implementation Effort (0–10) — engineering, labeling, infra, change management.
Ethical/Compliance Risk (0–10) — privacy, fairness, regulatory exposure (lower score for higher risk).
Strategic Fit (0–5) — alignment with business priorities or capability building.

Example weighted score = (Value * 0.35) + (Feasibility * 0.30) + ((10 - Effort) * 0.15) + ((10 - EthicalRisk) * 0.15) + (StrategicFit * 0.05).

Record both raw dimension scores and the weighted composite score. Use thresholds or bucket candidates into: Quick Win pilots, Explore (requires investment), or Defer/Not Recommended.

Rapid prototyping plan and success criteria

Frame each top candidate as a constrained experiment that aims to answer the riskiest unknowns quickly and cheaply. A good experiment reduces at least one major uncertainty about value, feasibility, or integration.

Experiment brief (one page)

Hypothesis: what you expect to change and why.
Primary success metric(s): measurable and tied to baseline.
Minimum viable approach: a scaled-down model or manual proxy to simulate AI output.
Data & tooling needed and how you will access them.
Duration and budget (keep short and small — typically days to a few weeks).
Who will run it and required approvals.
Decision rule: what measured result leads to scale, iterate, or stop.

Consider human-in-the-loop prototypes: human operators augmented with model suggestions can validate value before automating.

Ethical and safety checklist for early screening

Screen opportunities for foreseeable harms before heavy investment. Capture issues and mitigation ideas.

Could the system unfairly disadvantage a group? (bias risks)
Are there privacy or personally identifiable information concerns?
Could errors cause safety, legal, or reputational harm?
Is transparency or explainability required for users or regulators?
Are there auditability or logging requirements to support later governance?

For any moderate-or-high-risk item, list mitigations (e.g., restricted scope, human review, differential privacy, record-keeping) and determine if mitigation is acceptable before proceeding.

Output and recommended next steps

Deliverables from a scan include:

A prioritized list of candidates with composite scores and short rationales.
An experiment brief for each top candidate with clear success criteria and decision rules.
Data readiness notes and immediate actions (data access, labeling plan, quality fixes).
Ethics screening summary and required mitigations or approvals.

Use the experiment results to update scores and either: continue to build and integrate, iterate with new hypotheses, or archive the idea with reasons for deferment.

Practical tips and common mistakes

Prefer measurable outcomes over vague promises. Define metrics before building.
Use simple baselines (rule-based or human proxies) to validate value before model development.
Avoid ambitious scope in early experiments — remove integration and change-management assumptions when testing value.
Log decisions and failures: learning from what didn't work is essential for future discovery.

Where this Playbook can evolve

Once validated in practice, consider turning repeated artifacts (candidate templates, scoring forms, experiment briefs, and ethics checklists) into interactive templates that collect results, enable dashboards, and support reuse across teams.

Output (concise)

Prioritized list of AI pilot candidates with recommended first experiments, measurable success criteria, data-readiness actions, and initial ethics mitigations.