A strong systematic review begins long before the database search. Cochrane defines systematic reviews as reviews that use explicit, systematic methods documented in advance to answer a focused question, and PRISMA 2020 was updated precisely because many reviews still report these methods incompletely. If you want a publishable review, the real task is not simply “finding studies.” It is building a defensible workflow from question to update.
The timeline above reflects the sequence emphasized by Cochrane, PRISMA 2020, and recent search-methods guidance: scoping, searching, screening, synthesis, reporting, and updating should be planned as one integrated process, not as isolated tasks.
| Common mistake | Why it matters | Corrective action | Practical example or template |
|---|---|---|---|
| Using the wrong review type or a vague question | Authors often launch a systematic review when they actually need a scoping review, or they ask a question too broad for reproducible eligibility criteria. | Define a decision-ready question with PICO or a suitable framework for your review type before searching. | PICO template: Population, Intervention, Comparator, Outcomes, Setting, Study design. |
| Skipping a protocol or changing methods on the fly | Protocols reduce bias by locking scope and methods in advance. Undocumented deviations weaken trust. | Write a protocol, register prospectively when appropriate, and document amendments in the final report. | Add a protocol log with: date, section changed, rationale, impact on findings. |
| Building a narrow search strategy | No single database is complete; relying on one source or a few keywords risks missing eligible studies. | Search multiple relevant databases, trial registries, and targeted grey literature sources. Involve an information specialist if possible. | Search MEDLINE + Embase + CENTRAL + trial registry for intervention reviews. |
| Failing to make the search reproducible | PRISMA-S was created because search reporting is often too thin to reproduce or update. | Report database names, platforms, dates, limits, full strategies, and update methods. Peer review the strategy where feasible. | Keep a search appendix with exact Ovid or PubMed lines and run dates. |
| Screening with weak calibration | Inclusion decisions are highly influential and judgement-heavy; single-person full-text decisions increase error and bias. | Pilot eligibility criteria on a sample, then use two independent reviewers for full texts and a preplanned conflict-resolution process. | Run a 50-record calibration exercise before formal screening. |
| Not linking multiple reports from the same study | Duplicate or companion reports can lead to double counting in meta-analysis. | Match reports by registry ID, authors, setting, dates, intervention details, and sample characteristics. | Add a “study family ID” field in your screening log. |
| Using an unpiloted extraction form | Reviews depend on accurate, complete, accessible data. Unclear forms produce missing or inconsistent inputs. | Pilot the form, train extractors, and create decision rules for ambiguous cases and multiple reports. | Core fields: study ID, design, participants, intervention, comparator, outcomes, timepoints, effect estimates, funding, conflicts. |
| Extracting outcome data with only one reviewer | Cochrane treats duplicate outcome extraction as mandatory because it lowers error risk. | Use two independent extractors for outcome data and reconcile disagreements against the source report. | Add columns for extractor A, extractor B, final value, and resolution note. |
| Using reporting checklists as appraisal tools | Risk-of-bias assessment is not the same as “study quality” scoring or checklist completion. RoB 2 and ROBINS-I are result-focused and domain-based. | Use RoB 2 for randomized trials and ROBINS-I for non-randomized intervention studies. Avoid summary quality scores. | RoB 2 domains: randomization, deviations, missing data, outcome measurement, selective reporting. |
| Pooling studies that should not be pooled | Clinical, methodological, and statistical heterogeneity can make a pooled estimate misleading. Random-effects models do not “solve” heterogeneity. | Check whether populations, interventions, outcomes, and designs are sufficiently comparable before meta-analysis; preplan subgroup and sensitivity analyses. | Report both fixed-effect and random-effects as sensitivity analyses when justified, especially with few studies. |
| Ignoring missing results and weak synthesis methods | Selective non-reporting can exaggerate benefits and understate harms. Reviews without meta-analysis still need transparent synthesis. | Assess risk of bias due to missing evidence, inspect asymmetry cautiously, and use SWiM when narrative synthesis replaces pooling. | If no meta-analysis is possible, state grouping logic, standardized metric, direction of effect, and synthesis limitations. |
| Underreporting results and letting the review go stale | PRISMA 2020 expects transparent selection, flow diagrams, and complete reporting; Cochrane recommends rerunning searches within 12 months, preferably 6, before publication. Living reviews need explicit update reporting. | Report the PRISMA flow, excluded-study reasons, certainty of evidence, and last search date. Plan update triggers for fast-moving topics. | Add a “Search last rerun on [date]” line and a one-sentence update policy in Methods. |
Reusable Mini-Templates
Illustrative search-string template
textCopy
("type 2 diabetes" OR T2D OR "diabetes mellitus, type 2")
AND
(semaglutide OR GLP-1 OR "glucagon-like peptide-1")
AND
(randomized OR randomised OR trial*)
NOT
(animal* NOT human*)
This is only a starting scaffold. A production search should be database-specific, use controlled vocabulary where available, and be translated carefully across platforms.
Minimum data-extraction fields
textCopy
Study ID
Citation
Country/setting
Design
Eligibility criteria
Population/sample size
Intervention details
Comparator
Outcomes and timepoints
Effect measure and numeric result
Analysis population
Funding source
Conflicts of interest
Notes on multiple reports
These fields support synthesis, risk-of-bias assessment, transparency, and future updates.
The practical lesson is simple: most systematic review failures are process failures. The best reviews are conservative about claims, obsessive about documentation, and explicit about uncertainty. If a method cannot be reproduced, updated, or audited, it is probably not strong enough for a systematic review.
FAQ
What is the biggest mistake in a systematic review?
Usually, it is starting with an unfocused question or the wrong review type, because every later step inherits that error.
Is PRISMA a method for conducting a review?
No. PRISMA is primarily a reporting guideline. For conduct, authors still need methodological guidance such as the Cochrane Handbook and appropriate appraisal tools.
Should every systematic review be registered?
A written protocol is essential, and prospective registration is widely recommended when appropriate. PRISMA-P and PROSPERO support that workflow.
Can I use one reviewer for screening or extraction?
Cochrane allows some flexibility for title and abstract screening, but full-text inclusion decisions and outcome extraction are stronger with duplicate independent review.
When should I update my review?
At minimum, rerun searches close to publication; Cochrane’s conduct standards say the most recent search should be within 12 months, preferably 6. For rapidly changing topics, consider a living review model.