Rapid Evidence Review — Template & Protocol

Welcome — purpose and practical hunger

This Rapid Evidence Review protocol helps teams produce defensible, usable evidence fast when decisions or experiments are imminent. It balances speed with rigor by standardizing question framing, search scope, extraction, confidence scoring, and a one‑page memo that decision makers can act on. Use it for quick technology scans, intervention choices, prototype evidence, or pre‑experiment literature checks.

When to use this protocol

• Time horizon: complete in a few hours to 3 days.
• Goal: inform an experiment, pilot, or near‑term decision rather than produce a full systematic review.
• Not appropriate for high‑stakes regulatory claims, exhaustive clinical guidelines, or when exhaustive synthesis is required.

Rapid Protocol — overview and timebox

Core steps (typical time per step for a 1‑person rapid review; scale for team):

1. Define the question (15–45 min): scope the decision, define outcomes, and pick primary evidence types.
2. Set inclusion/exclusion criteria (15–30 min): transparency prevents hindsight bias.
3. Search sources & run searches (30–90 min): prioritize high‑value sources and use documented search strings.
4. Screen and select evidence (30–120 min): title/abstract screen, then quick full‑text check for candidates.
5. Extract key facts (60–180 min): fill the extraction table for selected items.
6. Rate quality/confidence (30–60 min): apply a simple, reproducible rubric.
7. Synthesize findings & implications (30–90 min): short narrative + recommended next steps.
8. Write one‑page evidence memo (15–30 min): concise summary for decision makers.

Step 1 — Define the question (practical framing)

Write a single decision question in plain language. Then translate it into an evidence question using a light PICO or equivalent framework:

• Population / Context: who or what context matters?
• Intervention / Technology / Action: what are you examining?
• Comparator (if relevant): what alternative?
• Outcome(s): what evidence would change your decision or experiment design?

Example: "For mid‑sized manufacturing sites (Context), does installing low‑cost IIoT sensors (Intervention) vs. manual inspections (Comparator) improve early fault detection (Outcome) within 6 months?"

Step 2 — Inclusion & exclusion criteria (template)

Use explicit criteria. Example fields:

• Study types: empirical trials, observational studies, case studies, simulations, vendor white papers (flag but lower confidence).
• Publication date range: e.g., last 7 years (adjust by domain speed).
• Languages: English only or include others and how to translate.
• Context/setting filters: industry, scale, geography.
• Outcome measures acceptable and thresholds.
• Exclude: opinion pieces without data, advertising materials lacking methods.

Step 3 — Search strategy (sources & tips)

Prioritize high‑yield sources to save time. Example shortlist:

• Academic: Google Scholar, PubMed, IEEE Xplore, SSRN (domain dependent).
• Industry and practice: professional association reports, standards bodies.
• Grey literature: vendor evaluations (use cautiously), case studies, government reports.
• Internal data or prior experiments in your organization.
• Recent systematic reviews or meta‑analyses — use their references.

Search tips: document exact search strings, date ranges, and where results were limited (e.g., by database or language). Save search results or export citations when possible.

Step 4 — Extraction table (required template)

Copy the table below and use it for every included item. Consistent fields enable synthesis and traceability.

Step 5 — Confidence scoring (simple reproducible rubric)

Use a compact rubric to make tradeoffs explicit. Example 4‑level scale:

• High: Credible methods, low risk of bias, direct context match, measured outcomes with effect sizes reported.
• Moderate: Reasonable methods but some limitations (small sample, short follow‑up, minor indirectness).
• Low: Observational, high risk of confounding, vendor‑sponsored without independent validation, or indirect context.
• Very low: Anecdote, single‑case report, or descriptive claims without transparent methods.

Document the primary reason for downgrading (risk of bias, indirectness, imprecision, inconsistency).

Step 6 — Synthesis (practical approaches)

Keep synthesis short, evidence‑graded, and actionable. Options:

• Narrative synthesis: group evidence by outcome and confidence level; report typical effect sizes and where evidence is missing.
• Simple quantitative aggregation: where studies are comparable, report median effect and interquartile range; avoid complex meta‑analysis unless you have time and expertise.
• Evidence matrix: create a one‑page table mapping key findings to confidence and relevance.

Step 7 — Implications for experiments and decisions

Translate evidence into tangible next steps. For each recommendation include:

• Proposed action or experiment (what to test).
• Why the action is supported by the evidence (cite items and confidence levels).
• Key uncertainties and how the proposed experiment will reduce them.
• Suggested metrics, sample size guidance, and minimum success criteria.

Template — One‑page Evidence Memo

Use this memo to brief decision makers. Keep to one page or a single screen.

Documentation & reproducibility checklist

• Record the decision question and protocol before searching.
• Save search strings and date run.
• Record screening decisions (include/exclude with reason).
• Store completed extraction table and one‑page memo in the project repo with timestamps.
• Note conflicts of interest (vendor funding, author affiliations).

Team roles (minimal)

• Owner/Lead: frames question, makes final call on scope, writes memo.
• Searcher: runs searches, exports citations.
• Extractor: completes extraction table and proposes confidence ratings.
• Reviewer (ideally independent): spot‑checks 10–20% of extractions and ratings.

Bias reduction tips

• Predefine inclusion/exclusion and the confidence rubric before looking at results.
• Prefer independent double‑checks for high‑influence items.
• Flag vendor or sponsor involvement and downgrade confidence accordingly.

When to stop and when to escalate

If the evidence is inconsistent, scarce, or the decision is high‑stakes, escalate to a fuller systematic approach or consult domain experts. Use this rapid protocol only when speed is essential and participants accept residual uncertainty.

File naming & metadata suggestion

Example: RapidReview_[YYYYMMDD]_[ShortQuestion]_[OwnerInitials]. Store: searchStrings.txt, extractionTable.csv, memo.pdf, and a README.md describing protocol choices.

Quick start checklist

1. Write the decision question and timebox.
2. Set inclusion/exclusion and confidence rubric.
3. Run prioritized searches and save outputs.
4. Screen, extract, and rate evidence into the table.
5. Write the one‑page memo and share for rapid review.

Use this template as a living protocol: adapt inclusion rules, dates, and search sources to your domain and organizational standards. Preserve transparency so the evidence can later be re‑used or expanded.