Inclusion and Exclusion Criteria for Systematic Reviews: A Comprehensive Guide

Systematic reviews require precise eligibility criteria to ensure valid, unbiased results. Inclusion criteria define the essential characteristics a study must have (e.g. Population, Intervention, Comparator, Outcomes, Study design – PICOS) to be counted, while exclusion criteria list features that disqualify a study. This guide shows how to develop PICOS-based criteria, with step-by-step examples from medicine, public health, education, and social sciences. It warns of common pitfalls (overly broad/narrow scope, ambiguous terms, unjustified filters) and how to avoid bias when applying language, publication type, and quality filters. We discuss handling non-English papers and grey literature in light of Cochrane and PRISMA guidelines, and provide PRISMA/Cochrane citations.

What Are Inclusion and Exclusion Criteria?

Definitions: Inclusion criteria are the attributes or conditions that must be met for a study to be included in the review. They typically reflect the review question’s PICO(S) elements – for example, the target population (age, condition, setting), the interventions/exposures of interest, the comparison or control condition, the outcomes measured, and eligible study designs. Exclusion criteria, by contrast, are specific characteristics that disqualify a study, even if it otherwise matches the inclusion criteria. For example, many reviews exclude animal studies, publications in the wrong language or time period, or studies with critically low sample size. PRISMA Item 5 explicitly instructs authors to “specify the inclusion and exclusion criteria for the review” and to justify any grouping of studies. Cochrane likewise emphasizes that eligibility criteria should be defined before screening begins, based on the review’s PICO elements plus study types.

Importance: Clear criteria focus the review question and minimize selection bias. By pre-specifying what will and won’t be considered, researchers avoid subjective or post-hoc decisions that could skew results. In fact, Cochrane notes that one hallmark of a systematic review is the a priori specification of eligibility criteria. If criteria are too broad, the review may include irrelevant studies and become unmanageable; if too narrow, it may miss key evidence or introduce bias. Well-defined inclusion/exclusion rules also make the screening process more reliable – for instance, multiple reviewers can apply them independently to decide on each study. It is best practice to pilot the criteria on a few studies first to ensure they work as intended.

Developing PICOS Eligibility Criteria (Step-by-Step)

A structured framework (e.g. PICOS) helps translate a review question into concrete rules. Cochrane and other guides recommend using PICOS (Population, Intervention, Comparator, Outcomes, Study design) as a checklist when drafting eligibility. Below are steps to define each element:

1. Population (P): Specify the target group (characteristics, setting, time). Include relevant age range, health status or demographic features. Example: “Adults (≥18 years) with Type 2 diabetes treated in primary care.” Avoid vague terms (e.g. “adults”) by being precise. If your topic spans subgroups, define them (e.g. “Adults 18–65; subgroup: patients ≥65”). Also note setting or geography if relevant (e.g. “inpatients” vs. “community-dwelling”). An example from practice defined population as “young people aged 10–24 in sub-Saharan Africa”. Exclude populations not relevant to the question (e.g. “Exclude studies on high-risk subgroups such as sex workers”).
2. Intervention/Exposure (I): Define exactly what is being tested or studied. For interventions, specify the type, dosage, timing, and mode of delivery. For example: “Behavioral interventions (education or counselling) focused on sexual health”. If it’s an exposure (in observational studies), define the exposure metric (e.g. pollutant X level, policy Y). Exclude studies that do not evaluate the intervention/exposure of interest (for example, pharmacological interventions if studying exercise programs). The comparator (C) can be “standard care”, placebo, or another intervention; state what counts as a valid comparison. Cochrane guidance suggests stating comparison precisely (e.g. “placebo, no treatment, or usual care”). If multiple comparators exist, note them. Exclusion: e.g. “Exclude studies with no control group or only post-intervention data.”
3. Outcomes (O): Decide which outcomes will guide inclusion. Ideally, include studies regardless of which outcomes they report, to minimize outcome reporting bias. Cochrane advises not using outcomes as strict eligibility criteria – include all studies of the interventions even if they don’t report your outcome. However, you may exclude studies that did not measure any outcome of interest at all. List key outcomes (primary and secondary) that the review will focus on (e.g. clinical events, behavioral changes). Example: “Inclusion requires reporting at least one outcome (e.g. HIV incidence, pregnancy, or behavior) measured ≥3 months post-intervention”. Exclude studies that only measure irrelevant or surrogate outcomes.
4. Study Design (S): State what study types will be considered. For an intervention review, you might include randomized controlled trials and possibly quasi-experimental designs (e.g. controlled before-after studies), while excluding case reports or qualitative research. Define acceptable designs explicitly: e.g. “Randomized controlled trials, and non-randomized controlled studies with concurrent or pre-post comparisons”. If limiting to RCTs, specify (e.g. “include cluster RCTs, but exclude crossover trials”). Cochrane warns to base this on design features, not just labels. Exclusion examples: “Exclude uncontrolled before-after studies and case series.” Remember that reviews of other topics (diagnosis, prevalence, qualitative) may use different frameworks (see Other frameworks below).
5. Other Criteria: Consider additional filters such as publication date (e.g. “studies from 2000 onward”), language, and setting (e.g. “hospital vs community”). These can be handled here or under “Other”. For example: “Include studies published in English between 2010–2025” (if a date range is justified by changes in practice). However, any such restriction should be carefully justified (see Language and Publication Filters below). It can also be useful to specify sample size thresholds (e.g. “sample ≥50 participants”) if very small studies would be too imprecise to include.

Documenting criteria: All criteria should be predefined in the protocol and written in unambiguous terms. It is helpful to list them in table form (PICOS columns) in the protocol. For example, Smith et al. (2015) tabulated PICOS criteria with inclusion and exclusion details (see Table above). Maintain a list of excluded studies (at full-text) with reasons for exclusion, as required by PRISMA. This transparency allows readers to see how criteria were applied.

Other Frameworks (Beyond PICOS)

While PICOS is common in health reviews, other frameworks exist for different evidence types:

• PEO (Population, Exposure, Outcome): Often used for observational or epidemiology reviews where “intervention” is replaced by exposure (risk factor). Example: “Urban adults (P) exposed to air pollution (E), with outcome asthma incidence (O).”
• SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type): Used for qualitative or mixed-methods syntheses. Focuses on participant group (sample), experiences (phenomenon), study design, and how findings are evaluated. Example: “High school students (Sample) experiencing inclusive education (Phenomenon); qualitative or mixed-design studies (Design); outcomes evaluated via interviews (Evaluation).”
• ECLIPSE, CLARe, CoCoPop, etc.: Variants exist for policy reviews, diagnostic accuracy, or prevalence (e.g. CoCoPop: Condition, Context, Population). Use the framework that best matches the question. If a single framework doesn’t fit, you can adapt PICOS by adding elements (like T for Time, or S for Setting) or mixing. Always justify your choice.

Examples of Inclusion/Exclusion Criteria in Different Fields

The following table illustrates how PICOS criteria might look across four example domains. Each column shows sample inclusion rules and corresponding exclusion examples (not exhaustive).

Common Pitfalls and How to Avoid Them

• Overly Broad or Vague Criteria: If inclusion criteria are too general (e.g. “all studies on health”), the review may become unmanageable and include irrelevant studies. Conversely, too-narrow criteria may miss key evidence. Solution: Iteratively refine criteria. Pilot-test on a subset of studies to see if they select appropriate papers. Be explicit (e.g. define “adolescent” as age 10–19, not “teenagers”).
• Ambiguous Terms: Using loose language (“seniors”, “young people”, “low-income”) can lead to inconsistent screening. Solution: Specify exact values or thresholds (ages, income levels) or definitions.
• Post-hoc Changes/Scope Creep: Changing criteria after screening can introduce bias. PRISMA/Cochrane mandate justifying any changes made after protocol. Solution: Fix criteria in protocol. If new issues arise (e.g. an unforeseen study type), decide transparently and conduct sensitivity analyses if needed.
• Unjustified Exclusions: Common mistakes include excluding studies just because they are unpublished, in a non-English language, or have negative results. Cochrane explicitly warns against excluding unpublished/grey literature and calls for careful consideration of language restrictions. Solution: Only apply filters that are necessary (e.g. exclude non-English only if translation is impossible, but note this decision and consider bias implications).
• Quality Bias at Screening: Excluding studies solely for low quality or small size can bias results (e.g. small negative trials). Some guidelines suggest not using quality to filter out studies at selection stage. Instead, include all eligible studies and handle quality in appraisal or sensitivity analysis. Solution: If truly excluding for quality, define clear, stringent thresholds beforehand (e.g. “exclude studies with N<50 if planning to combine only large trials”), but better to include and flag risk of bias later.
• Inconsistent Application: Without a clear process, different reviewers might apply criteria inconsistently. Solution: Use at least two independent screeners on titles/abstracts and full texts. Create a screening guide or form with examples of “yes/no” decisions. Discuss and resolve discrepancies by consensus or a third reviewer.
• Incomplete Documentation: Not keeping track of excluded studies or reasons undermines transparency. Follow PRISMA by logging each excluded study at full text with at least one exclusion reason. This also helps in writing the review’s methods and flow diagram.

Language, Publication-Type, and Quality Filters

• Language Restrictions: Excluding studies by language (e.g. English only) is common but can introduce language bias. Cochrane notes that reviews should consider the bias and equity issues of restricting languages. If resources allow, include all languages and use translation tools or multilingual team members. If a restriction is necessary, justify it (e.g. “no translation available for languages beyond team capacity”) and consider its impact on bias. A methodology study found language restrictions can lower credibility of reviews.
• Publication Status: Systematic reviews should ideally include grey literature (conference abstracts, theses, reports) and unpublished data to reduce publication bias. Cochrane’s MECIR Standards require including studies “irrespective of publication status” unless justified (MECIR C12). If excluding e.g. conference abstracts due to poor data, note this decision. Remember unpublished studies may be hard to find, but excluding them entirely often biases results towards positive findings.
• Publication Type: Decide which document types count. Commonly exclude non-research items (letters, editorials, books, news). Specify in criteria (e.g. “only primary empirical research”). If including systematic reviews of interest, clarify that they will only be used to find primary studies, not included themselves. For quality, you might exclude extremely poor data reports (e.g. pure qualitative opinions) if not meeting your study design criteria.
• Quality and Bias Thresholds: It is generally advised not to preclude studies based solely on quality in the eligibility stage. Instead, assess risk of bias after inclusion (e.g. using Cochrane RoB tool) and interpret results accordingly. However, some reviews adopt minimum design levels as criteria (e.g. include only RCTs or only cohort studies). If doing so, state the rationale: for example, a review may exclude any design below Level 3 evidence to ensure validity. Always align with your question: don’t exclude high-bias studies just to get “higher quality” results — this can itself introduce bias.

In practice, one approach is to flag low-quality studies rather than drop them, then perform sensitivity analyses excluding them or down-weighting their evidence. The Cochrane MECIR standard C40 reinforces this: do not exclude studies for unusable outcome data; include them in the dataset and address missing data in synthesis.

Handling Non-English Studies and Grey Literature

To be comprehensive, plan for non-English studies and grey literature:

• Non-English Studies: A truly systematic review should consider studies in all languages relevant to the topic. Use database filters to capture languages, enlist colleagues, or use machine translation (even abstracts) to screen. For example, many Asian or European public health studies might be in local languages. If found, decide whether to include them (e.g. extract data from English abstract or translate key parts). If resource constraints force a language restriction, report it as a limitation. PRISMA recommends stating languages included/excluded in the methods.
• Grey Literature: Search beyond databases: look at trial registries, conference proceedings, dissertations, government reports, and organizational websites. Gray sources can reveal negative or niche studies. If including, decide on a plan to appraise their often-variable quality. If excluding grey lit (e.g. non–peer-reviewed), justify (e.g. “focus was on published literature”), but understand this may bias results toward positive findings. Always document whether grey literature was considered and how.
• Time Frames for Non-Indexed Work: Sometimes older or regional studies are only in grey form. Cochrane Chapter 3 advises careful searching for unpublished/grey studies to reduce bias. Don’t exclude them simply for lack of journal citation; instead, track down reports or authors.
• Documentation: Whether or not grey/non-English items are included, detail your approach in the protocol. For each excluded non-English or grey item, document reason (e.g. “no translator available” or “abstract only, no data”). This transparency helps readers judge completeness.

Aligning with PRISMA, Cochrane, and Best Practice

Our approach should align with established guidelines:

• PRISMA 2020: Item 5 of PRISMA 2020 requires authors to “specify the inclusion and exclusion criteria for the review and how studies were grouped for the syntheses.”. This means stating exactly what qualifies a study and how different PICOS elements are handled (e.g. grouping similar interventions). The PRISMA flowchart (not shown here) will illustrate how many studies were excluded at each stage and for what reason, so clear criteria ensure the flow diagram is meaningful.
• Cochrane Handbook: Cochrane Chapter 3 emphasizes that eligibility criteria should be predefined based on the PICO elements plus study design. It also notes that outcomes normally are not used to exclude studies (except when no relevant outcome is measured). Cochrane’s MECIR (C39–C42) standards reinforce duplicating study selection with ≥2 reviewers and including studies even with incomplete data. We follow these by planning dual screening and by not rejecting studies merely for missing some outcome data.
• Quality Standards: If using quality thresholds (e.g. minimum study design), refer to recognized hierarchies (e.g. CRD or NHMRC tables) as guidance. Otherwise, simply assess all studies with a risk-of-bias tool as part of analysis.
• Document and Justify: As required by both PRISMA and Cochrane, any eligibility decisions must be documented and justified. Keep in mind that the protocol should contain the criteria and that any changes made during the review must be explained.

By citing these sources and building our criteria accordingly, we ensure methodological rigor and transparency for readers and editors.

Conclusion

Clear, transparent inclusion and exclusion criteria are the foundation of a high-quality systematic review. By systematically defining PICOS elements (or alternate frameworks) and explicitly stating filters for language, publication type, and study quality, reviewers draw valid boundaries around their research question. Following PRISMA and Cochrane guidance ensures these criteria are justified and consistently applied. Avoid common mistakes (like vague scopes or unjustified exclusions) by piloting your criteria and using independent screening. Document all decisions and rationale so that readers can understand exactly how studies were chosen (or not). Doing so not only improves the credibility of your review but also its discoverability (SEO) and relevance to your audience. Ultimately, rigorous eligibility criteria lead to more trustworthy evidence syntheses that better inform practice and policy worldwide.