Rapid Evidence Review Protocol

Purpose

Use this protocol to convert scattered literature and data into a concise evidence brief within a tight timeframe while making tradeoffs transparent. Rapid evidence reviews are intended to inform near‑term decisions and hypotheses, not to replace full systematic reviews. Document limits, uncertainty, and recommended follow‑up research.

When to use a rapid review

• Decision horizon is days–weeks and action cannot wait for a full systematic review.
• You need a defensible, reproducible summary of the best-available evidence to shape experiments, pilot design, procurement, or policy options.
• Resources or access to expertise for a full review are limited but transparent tradeoffs are possible.

Core principles

1. Scope tightly: define the question and a small set of critical outcomes or decision criteria.
2. Pre-specify inclusion/exclusion rules and timelines to avoid ad-hoc selection.
3. Document every step (searches, screening decisions, extraction, quality judgments).
4. Balance speed and rigor through explicit, auditable shortcuts (e.g., date limits, single‑reviewer screening with verification rules).
5. Report uncertainty and mark where further primary research or a full review is needed.

Protocol overview (timeboxed workflow)

Typical small rapid review: 1–2 weeks. Larger topics: 3–4 weeks. Adjust according to team capacity and stakes.

1. Plan (0–1 day): Clarify question, decision context, team and deadlines.
2. Search (1–2 days): Run prioritized searches and harvest citations.
3. Screen (1–3 days): Title/abstract and full-text screening per pre-specified rules.
4. Extract & rate quality (1–3 days): Populate extraction table and apply a simple quality rubric.
5. Synthesize (0.5–1 day): Fill one‑page evidence synthesis and recommended actions.
6. Review & deliver (0.5–1 day): Quick peer check, finalize brief and append methods log.)

Team roles (minimal)

• Lead (owner of question, stakeholders): sets scope and uses findings.
• Reviewer(s): run searches, screen, extract, and rate quality.
• Verifier: second‑checks a random sample of exclusions and extractions (recommended 10–20% unless stakes require more).
• Writer: composes the one‑page synthesis and documents methods.

Scope & inclusion/exclusion template (example)

Define these before searching and keep them simple:

• Population: Adults employed in frontline service roles.
• Intervention/Exposure: Short training modules (≤ 2 hours) delivered online.
• Comparator: Usual training or no training.
• Outcomes: Measurable behavior change or performance within 3 months.
• Study types: RCTs, controlled before-after, observational studies; exclude opinion pieces and editorials.
• Date limit: 2015–present (justify if used).
• Language: English (note as limitation).

Search strategy checklist

• List 3–5 core databases relevant to the question (e.g., PubMed, Scopus, Web of Science, PsycINFO, IEEE Xplore).
• Define 6–12 search terms including synonyms and controlled vocabulary (sample Boolean string included below).
• Set date & language limits only if necessary and record justification.
• Search grey literature selectively (technical reports, gov sites, major conference abstracts) if time allows.
• Export results into a reference manager or spreadsheet and remove duplicates.
• Record the full search string and date searched for reproducibility.

Sample Boolean string: ("online training" OR "e-learning") AND ("behavior change" OR performance) AND (workplace OR employees)

Screening & selection - pragmatic rules

• Title/abstract screening: single reviewer with a verifier checking 15–20% of exclusions. If disagreement > 20%, switch to dual screening.
• Full-text screening: dual check recommended for higher‑stakes questions; otherwise single reviewer with verification of exclusions.
• Log reasons for exclusion at full-text level (brief code list such as wrong population, wrong outcome, no primary data).
• Keep a simple PRISMA flow log: records identified, screened, included, excluded (with reasons).

Extraction table (template)

Simple quality‑rating rubric (practical)

Use a compact High/Moderate/Low scale. Apply conservatively and document reasons.

• High — Robust design (RCT or strong quasi-experimental), appropriate analysis, low risk of bias, outcomes measured objectively.
• Moderate — Some design or reporting limitations (small samples, some bias risk), but effect plausible and measured.
• Low — Observational or poorly reported studies with high risk of bias or inconsistent outcome measurement.

One‑page evidence synthesis template

Use this as the front‑page deliverable for decision makers (keep it to one page):

1. Question & context (1–2 lines)
2. Search summary: databases, date range, total screened, included (n)
3. Key findings: 3 bullet points summarizing strongest evidence and effect sizes
4. Confidence: overall certainty (High/Moderate/Low) with short justification
5. Decision implications: recommended immediate actions (clear, observable next steps)
6. Limitations & uncertainty: what this review cannot tell you
7. Research & monitoring needs: suggested follow‑up or evaluation measures

Tips for balancing speed and rigor

• Pre-specify shortcuts (e.g., date limits, English-only) and record why you used them.
• Use single-reviewer screening with verification rules to accelerate low‑risk topics.
• Prioritize highest-quality evidence first (RCTs, systematic reviews) and stop searching once consistent, high-confidence evidence is found for the decision at hand.
• Use automation where available: deduplication tools, basic keyword filters, citation chaining.
• Document a minimal reproducibility log (search strings, databases, dates, screening decisions) — this is the core defensibility evidence.

Deliverables

• One‑page evidence summary (required)
• Methods appendix: search strings, databases, dates, inclusion/exclusion criteria, PRISMA flow
• Extraction table and quality ratings (spreadsheet or saved form)
• Recommendations and suggested evaluation measures

Using platform capabilities (recommendations)

To increase reproducibility and reuse, consider:

• Rendering the screening and extraction templates as interactive forms so reviewers can save structured data and the platform stores submissions for later analysis.
• Capturing the PRISMA flow and method metadata via the submission API to build an audit trail and dashboard.

Common pitfalls to avoid

• Changing inclusion rules mid-review without recording the reason.
• Selective citation of studies that support a favored action — include all eligible studies and report dissenting evidence.
• Overstating certainty; label recommendations according to the actual evidence confidence.

Quick checklist (one‑minute scan)

1. Clear question and decision context documented
2. Search date, databases, and strings saved
3. Screening performed with verification rules
4. Extraction complete for included studies
5. Quality ratings applied and recorded
6. One‑page synthesis and methods appendix delivered

Adapt these templates to your context, record tradeoffs, and use the review as evidence to shape experiments or policy decisions rather than as final proof.