Rapid Evidence Review Workflow

Rapid Evidence Review — Workflow

Purpose: Quickly gather, assess, and synthesize existing evidence so teams can make better-informed, time‑sensitive decisions while being explicit about limits and uncertainty. Use this workflow when you have a clear decision horizon (e.g., 1 day, 3 days, 1 week) and need an evidence-informed input for experiments, procurement, pilots, policy choices, or product decisions.

Quick overview of phases

1. Frame the question (problem and decision)
2. Design the search strategy (sources, scope, stop rules)
3. Fast screening (title/abstract heuristics)
4. Extraction (concise structured template)
5. Synthesis and evidence weighing (narrative + confidence)
6. Produce an evidence memo for stakeholders

Time budgets (examples)

Adapt to your time-box. Below are starting budgets; scale proportionally.

• 1 working day (8 hours): Frame 30m, Search 2h, Screen 2.5h, Extract 1.5h, Synthesize & memo 1.5h
• 3 working days: Frame 1h, Search 4h, Screen 6h, Extract 4h, Synthesize & memo 5h
• 1 working week: Frame 2h, Search 1 day, Screen 1.5 days, Extract 1 day, Synthesize & memo 1 day

Document the chosen time-box at the start and stop strictly at the deadline. The review is intended to inform near-term decisions, not replace full systematic reviews.

Phase detail and practical heuristics

1. Frame the question (30–120 minutes)

• Define the decision you need to inform (who, what decision, deadline, criteria for success).
• Write a short focused question using PICO/PICo or plain language depending on context (Population, Intervention, Comparator, Outcome). Example: "For small retail stores, does automated reorder software reduce stockouts within 3 months compared to manual reorder?"
• Specify inclusion boundaries: date range, languages, settings, study designs to include (e.g., evaluations, case studies, implementation reports).
• Identify critical outcomes or measures the decision cares about (e.g., cost impact, implementation complexity, safety, lead time reductions).

2. Design the search strategy (20–120 minutes)

• Choose 3–6 high-yield sources to cover different evidence types: academic databases (PubMed, Scopus), preprint servers, industry research, credible grey literature (government reports, major NGOs), and targeted expert or vendor material.
• Create 3–6 keyword strings combining synonyms and key concepts. Keep them simple and reproducible. Example: "(automated reorder OR inventory automation) AND (small retail OR independent store)"
• Set pragmatic stop rules: e.g., stop after 200 hits per source, or when 20 consecutive records add no new relevant information.
• Record search log: date, source, search string, number of hits, stop rule reached.

3. Fast screening (title/abstract; 30–180 minutes)

• Use a two-stage rapid screen: title/abstract triage, then quick full-text check for those that pass.
• Apply fast heuristics for exclusion (document reason): wrong population, irrelevant outcome, commentary without data, duplicate, or inaccessible full text.
• Use a single screener with spot-checks by a second reviewer when time allows — document who screened and any disagreements.
• Keep a log of excluded-but-potentially-relevant items if follow-up research is planned.

4. Extraction template (concise, structured)

Capture only fields the decision needs. Use consistent short entries. Example extraction fields:

1. Unique ID / citation
2. Type of evidence (RCT, quasi-experiment, observational, case study, vendor report)
3. Setting and population
4. Intervention or exposure (brief)
5. Comparator (if any)
6. Key outcomes and numeric results (effect size, direction, CI if reported)
7. Study limitations (1–2 short bullets)
8. Relevance to decision (1 sentence: why this matters)
9. Confidence rating (High / Moderate / Low using simple criteria below)

Keep extraction entries to a few lines each. Aim to extract the minimum that supports synthesis and recommendations.

Confidence rating (simple rule)

• High: multiple credible studies with consistent effects, low risk of major bias for the outcome in question.
• Moderate: limited number of studies or some risk of bias, but findings are coherent and plausible.
• Low: single small studies, high risk of bias, inconsistent findings, or heavy reliance on vendor/grey reports.

5. Synthesis & evidence weighing (narrative + explicit weight)

• Create a short narrative (200–400 words) that answers the framed question directly, referencing the highest-confidence evidence first.
• Provide an evidence table summary: number of studies per evidence type, direction of effect, typical magnitude, and overall confidence.
• Explicitly note gaps, contradictory findings, and any contextual factors that alter applicability (scale, geography, timeline).
• When numeric pooling isn't feasible, use qualitative weight language and a small rubric to summarize degree of support (e.g., Strong / Moderate / Weak / Conflicting).

6. Produce a concise evidence memo for stakeholders

Format: 1 page executive summary + 1–2 pages supporting evidence table and notes.

Suggested structure and word guidance:

1. Headline answer (1 sentence)
2. Bottom-line recommendation for the decision (1–2 sentences)
3. Key findings & confidence (3–6 bullets)
4. Key evidence supporting recommendation (1 short sentence each for top 3 items)
5. Limitations & uncertainties (2–4 bullets)
6. Suggested next steps (e.g., pilot, monitoring metrics, further research needed)

Documentation & transparency checklist

• Question and decision context recorded
• Search strings and sources saved
• Screening log with reasons for exclusion
• Extraction entries and confidence ratings stored
• Evidence memo with date, author, and time-box identified

Common pitfalls and guardrails

• Don't let the time-box shrink methodological transparency — document shortcuts and their likely impact.
• Avoid over-reliance on vendor materials for effectiveness claims without independent corroboration; flag conflicts of interest.
• Do not conflate absence of evidence with evidence of absence — explicitly say when there is simply no data.
• When findings are low-confidence, pair recommendations with small pilots and measurement plans rather than large irrevocable investments.

Optional adaptations

• For operational teams, convert the extraction template into a shared spreadsheet or an interactive form so multiple reviewers can work in parallel.
• When the domain will run repeated rapid reviews, build a reusable domain collection with standardized templates, training snippets, and curated source lists.

Next steps template (at review end)

1. Decision recommended: (Accept / Pilot / Defer / Reject)
2. If pilot: scope, duration, success metrics
3. Who will monitor outcomes and when
4. Suggested follow-up evidence work (full systematic review, targeted evaluation, vendor due diligence)

Use this workflow to produce quick, transparent, decision-ready evidence summaries. Document your assumptions and limits so decision-makers can weigh the evidence appropriately and choose pragmatic next steps.