Systematic review starter kit

Welcome — quick orientation

This starter kit helps you launch a reproducible systematic review or evidence synthesis without rebuilding core artifacts. It focuses on practical decisions you must make early (protocol, search strategy, screening plan, extraction fields, bias assessment, and reporting) and gives concrete examples, thresholds, and tools you can adapt to your topic and team.

Why this matters

Well-planned reviews save time, reduce bias, and prevent duplicated effort. Use this kit to make your review defensible, auditable, and easier to extend later (updates, living reviews, or meta-analysis).

Core components included

• Protocol template with objectives, PICO/PECO, inclusion/exclusion criteria, outcomes, and analysis plan
• Search strategy checklist (databases, grey literature, search documentation) and sample search strings
• Screening matrix (title/abstract and full-text) plus inter-rater agreement guidance and conflict-resolution process
• Data-extraction spreadsheet template with recommended fields and examples
• Risk-of-bias checklist mapping (RoB2, ROBINS-I, Newcastle-Ottawa, QUADAS-2) and PRISMA-style flow diagram template

Step-by-step starter workflow

1. Define scope and register the protocol. Write a short protocol describing the question (use PICO/PECO), primary and secondary outcomes, eligibility criteria, databases to search, planned synthesis methods, and handling of missing data. Register on PROSPERO or post on OSF to timestamp the plan.
2. Design and test search strategies. Choose databases (e.g., PubMed/MEDLINE, Embase, Cochrane Library, Web of Science, Scopus) and bibliographic tools (Zotero, EndNote, Mendeley). Document exact search strings, dates, filters, and save results and exports. Include grey literature sources and trial registries where relevant.
3. Run deduplication and citation mapping. Export records, remove duplicates (document method and software), and perform forward/backward citation chasing (Google Scholar, Scopus, Web of Science, Lens.org) to catch missed studies.
4. Screen in two stages. Title/abstract screening followed by full-text screening. Pilot the screening on a random subset (e.g., 50–100 records) to align reviewers. Keep a screening log and reasons for exclusion at full-text.
5. Extract data and assess risk of bias. Use the provided extraction template. Assign two reviewers for at least a subset of studies to check consistency. Use appropriate RoB tools by study design.
6. Synthesize and report. Decide whether meta-analysis is feasible. If not, prepare a structured narrative synthesis. Report using PRISMA and include a completed flow diagram and search appendices.

Practical guidance & examples

Sample PubMed search string (adapt to your topic)

("diabetes mellitus"[MeSH] OR diabetes[tiab]) AND (telemedicine[tiab] OR telehealth[tiab] OR "remote monitoring"[tiab])

Note: translate MeSH to database-specific syntax and document every run (date, database, platform, filters).

Search strategy checklist

• Pick primary electronic databases and justify choices.
• Plan grey-literature sources (conference proceedings, dissertations, clinical trial registries, government reports).
• Record exact strings, date executed, platform/provider, and number of records retrieved.
• Export and archive full search results and save intermediate files (RIS, CSV).
• Document deduplication approach and software.

Screening & inter-rater agreement

• Pilot on 50–100 items to align criteria and improve consistency.
• Use two independent reviewers for screening; resolve disagreements by discussion or a third reviewer.
• Calculate agreement (Cohen's kappa or percent agreement). Practical thresholds: kappa > 0.6 acceptable, > 0.8 strong. If below target, refine criteria and re-pilot.
• Keep a screening log with exclusion reasons at full-text level (required for PRISMA).

Recommended extraction fields (spreadsheet columns)

• Citation (author, year)
• Study design
• Population and eligibility
• Sample size
• Intervention/exposure and comparator
• Primary and secondary outcomes (definitions and measurement timepoints)
• Effect estimates, variance, and units
• Funding sources/conflicts of interest
• Notes on missing data, imputation, or special methods
• Risk-of-bias judgments by domain
• Raw files or links to full-text PDFs

Risk-of-bias tools (pick by study type)

• Randomized trials: Cochrane RoB2
• Non-randomized interventions: ROBINS-I
• Observational cohort/case-control: Newcastle-Ottawa Scale
• Diagnostic accuracy studies: QUADAS-2

Reporting and transparency

Follow PRISMA for structure and include:

• Complete search strings and dates as appendices
• PRISMA flow diagram (numbers at each stage and reasons for full-text exclusions)
• Protocol registration ID or link
• A table mapping included studies to outcomes and risk-of-bias ratings

Tools & software that speed work

• Reference managers: Zotero, EndNote, Mendeley
• Screening platforms: Rayyan, Covidence, DistillerSR (support blinded dual screening)
• Data extraction: shared spreadsheets (Google Sheets) or specialized forms in review software
• Citation mapping and analytics: Scopus, Web of Science, Lens.org, VOSviewer for bibliometrics
• Automation & AI: semi-automated deduplication and prioritization tools exist—use cautiously and document decisions for reproducibility

Quick starter checklist

1. Draft protocol (PICO/PECO, outcomes, eligibility) and register it.
2. Select databases and grey sources; draft and test search strings.
3. Export results, deduplicate, and archive raw exports.
4. Pilot title/abstract screening with two reviewers; finalize screening form.
5. Screen full texts, record exclusions with reasons.
6. Extract data into the recommended template and assess risk of bias.
7. Decide on synthesis approach and prepare PRISMA flow diagram and search appendix.

Included downloadable artifacts

• Protocol template (editable)
• Search strategy checklist and sample strings
• Screening matrix template and inter-rater guidance
• Data-extraction spreadsheet template with recommended fields
• Risk-of-bias checklist mapping and PRISMA-style flow diagram template

Next steps & customization

Adapt the templates to your discipline, registry expectations, and organizational standards. If you expect to update the review regularly, plan a living-review workflow and consider automation only where it can be validated and fully documented.

Want help tailoring this kit to your team or turning extraction forms into shared, savable forms? Consider converting the extraction and screening templates into interactive forms so teams can save responses, track progress, and export structured JSON for analysis.

Common pitfalls to avoid

• Skipping protocol registration or failing to document changes
• Using untested search strings or failing to document database/platform differences
• Inconsistent screening criteria without piloting
• Performing extraction without double-checks for critical fields
• Over-reliance on opaque automation without human oversight

Contact & adaptation notes

If your organization wants a tailored toolkit (site-specific protocols, saved form templates, or interactive extraction and screening forms), this starter kit is designed to be adapted into an owned collection that teams can copy and evolve.