Screening is the critical study selection phase of a systematic review where researchers systematically evaluate identified records against pre-defined eligibility criteria to determine which studies should be included in the evidence synthesis. The screening process occurs in two stages: title and abstract screening (first pass) and full-text screening (second pass). How rigorously and consistently you screen directly determines the validity of your review. Errors at this stage propagate through every subsequent phase, from data extraction to synthesis and conclusions.
Screening is also where the largest volume of work occurs: systematic reviews often screen thousands of records, making efficiency, consistency, and documentation essential.
The Two-Stage Screening Process
Stage 1: Title and Abstract Screening
In the first stage, two or more independent reviewers evaluate each record's title and abstract against the eligibility criteria. The goal is to rapidly exclude obviously irrelevant records while retaining all potentially relevant records for full-text assessment.
Key principles for title and abstract screening:
- Be inclusive: When in doubt, include the record for full-text review. It is better to retrieve unnecessary full texts than to miss relevant studies.
- Apply criteria consistently: Every record must be evaluated against the same eligibility criteria.
- Work independently: Reviewers must not consult each other during screening to avoid bias.
- Document decisions: Record include/exclude decisions for every record.
Stage 2: Full-Text Screening
Records that pass title and abstract screening are retrieved as full-text reports and assessed in detail against the eligibility criteria. At this stage, each exclusion must be accompanied by a specific reason (e.g., wrong population, wrong intervention, wrong outcome, wrong study design).
Full-text screening exclusion reasons are reported in your PRISMA flow diagram. Document these carefully, as they are among the most common PRISMA diagram errors identified during peer review.
Best Practices for Title and Abstract Screening
1. Pilot Your Eligibility Criteria
Before beginning full screening, pilot your eligibility criteria on a random sample of 50-100 records. Both reviewers independently screen the sample, then compare results and discuss disagreements. This calibration step:
- Identifies ambiguities in eligibility criteria that need clarification
- Establishes shared understanding between reviewers
- Measures inter-rater agreement (calculate Cohen's kappa)
- Reduces disagreements during full screening
Aim for a Cohen's kappa of at least 0.80 (substantial agreement) before proceeding. If agreement is below 0.60, revise your criteria and pilot again.
2. Create a Screening Decision Guide
Develop a written screening decision guide (separate from your eligibility criteria) that provides specific, operationalized rules for common edge cases. For example:
- "If the abstract does not mention the intervention but the title suggests relevance → INCLUDE for full-text review"
- "Conference abstracts → INCLUDE if full publication cannot be identified"
- "Protocols with no results → EXCLUDE with reason 'protocol only'"
- "Studies in languages we cannot read → INCLUDE and attempt translation"
This guide reduces subjective judgment and ensures consistency across thousands of screening decisions.
3. Use Independent Dual Screening
The gold standard for systematic review screening is independent dual screening, where two reviewers independently assess every record. This approach:
- Reduces the risk of relevant studies being missed (single-reviewer screening misses approximately 8-10% of relevant studies)
- Enables measurement of inter-rater reliability
- Is expected by Cochrane, JBI, and most journal guidelines
- Is a key quality indicator assessed by AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews)
Single-reviewer screening should only be used in rapid reviews or when explicitly justified as a limitation.
4. Establish a Clear Conflict Resolution Process
When reviewers disagree, use a pre-specified resolution process:
Option A: Discussion to consensus. The two reviewers discuss the record and reach agreement. This is the most common approach for most disagreements.
Option B: Third reviewer. A senior researcher or subject matter expert provides the deciding vote. Use this for persistent disagreements or when the original reviewers cannot reach consensus.
Option C: Default to inclusion. When doubt remains, include the record for full-text review. This is the conservative approach that minimizes the risk of missing relevant studies.
Document which resolution method was used and how many conflicts required resolution, as this information demonstrates the rigor of your screening process.
5. Maintain Blinding Where Possible
Consider blinding reviewers to:
- Author names and institutional affiliations (to prevent reputation bias)
- Journal names (to prevent impact factor bias)
- Publication dates (to prevent recency bias)
Some screening tools support blinding features. While not always practical or required, blinding reduces potential sources of screening bias.
6. Screen in Batches with Quality Checks
Rather than screening all records in a single session, work in batches of 200-500 records. Between batches:
- Compare inter-rater agreement on the completed batch
- Discuss any patterns in disagreements
- Refine the screening decision guide if needed
- Check for screening fatigue or drift from eligibility criteria
This iterative quality monitoring catches consistency problems before they propagate through thousands of records.
Choosing a Screening Tool
Several software platforms support systematic review screening. Your choice should consider team size, budget, volume of records, and integration with other review stages.
Rayyan: Free web-based platform with AI-assisted screening recommendations. Good for small teams. Supports collaboration, conflict resolution, and PRISMA diagram generation.
Covidence: Comprehensive systematic review platform with screening, extraction, and quality assessment. Institutional pricing. Strong Cochrane integration.
DistillerSR: Enterprise-level platform with AI-assisted screening, customizable forms, and advanced reporting. Higher price point.
EPPI-Reviewer: Developed by UCL, supports text mining and machine learning for screening prioritization. Free for some users.
ASReview: Open-source active learning tool that prioritizes records most likely to be relevant, potentially reducing screening workload by 70-95%.
For a broader comparison of systematic review tools, see our comparison of PRISMA flow diagram tools.
Handling Large Volumes of Records
When your search yields thousands of records, these strategies improve efficiency without compromising rigor:
Machine Learning-Assisted Screening
Tools like ASReview and Abstrackr use active learning to prioritize records most likely to be relevant. After manually screening an initial training set, the algorithm reorders remaining records by relevance probability. This can reduce screening workload significantly, but all records must still be screened; ML tools prioritize, they do not replace human judgment.
Liberal Accelerated Screening
One reviewer screens all records; a second reviewer screens only excluded records. This approach reduces workload while maintaining sensitivity for relevant studies. The rationale: the critical error is false exclusion (missing a relevant study), not false inclusion (sending an irrelevant record to full-text review).
Text Mining for Deduplication
Before screening, use automated or semi-automated deduplication to remove exact and near-exact duplicates. This can reduce the screening volume by 20-40% with minimal manual effort.
Common Screening Mistakes
Mistake 1: Applying Criteria Too Strictly at Title/Abstract
At the title/abstract stage, information is often incomplete. Exclude only when a record CLEARLY fails a criterion. If uncertain, include for full-text review.
Mistake 2: Inconsistent Application of Criteria
Criteria interpretation may drift over time, especially in large reviews. Regular calibration checks and a written decision guide prevent this.
Mistake 3: Not Documenting Exclusion Reasons at Full-Text
Every full-text exclusion needs a specific, documented reason. Generic reasons like "did not meet criteria" are insufficient and are a frequent cause of peer review revision requests.
Mistake 4: Single Reviewer for Initial Screening
Using one reviewer to screen titles and abstracts misses approximately 8-10% of relevant records. If resources are truly limited, use liberal accelerated screening rather than single-reviewer screening.
Mistake 5: Not Piloting Before Full Screening
Skipping the pilot phase leads to inconsistent screening, higher disagreement rates, and potentially having to re-screen records after clarifying criteria.
Reporting Screening in Your PRISMA Diagram
Your PRISMA 2020 flow diagram documents the screening process with specific numbers at each stage:
- Records screened: Total unique records after deduplication
- Records excluded: Number excluded at title/abstract stage
- Reports sought for retrieval: Number advancing to full-text
- Reports not retrieved: Number of full texts you could not obtain
- Reports assessed for eligibility: Number of full texts evaluated
- Reports excluded with reasons: Number excluded at full-text stage, broken down by reason
These numbers must be internally consistent: records screened minus records excluded must equal reports sought for retrieval. Use our free tool to build your PRISMA flow diagram and verify your numbers with real-time validation.
For the complete systematic review workflow from question formulation through reporting, see our comprehensive guide to conducting a systematic review.
Frequently Asked Questions
How many reviewers are needed for screening?
A minimum of two independent reviewers is the accepted standard for systematic reviews. Cochrane and JBI guidelines require dual independent screening. Single-reviewer screening is acceptable only in rapid reviews and must be declared as a limitation.
What is an acceptable inter-rater agreement for screening?
A Cohen's kappa of 0.80 or higher indicates substantial agreement and suggests eligibility criteria are clear and consistently applied. Kappa values between 0.60-0.80 indicate moderate agreement and may warrant criteria refinement. Below 0.60 indicates problematic disagreement requiring criteria revision.
Should I use AI-assisted screening tools?
AI-assisted tools can significantly improve screening efficiency, particularly for large reviews. However, they should complement, not replace, human screening. Use AI tools to prioritize records (screen relevant records first) or for workload allocation, but ensure all records are eventually screened by human reviewers.
How do I handle records in languages I cannot read?
Include foreign-language records that appear potentially relevant based on their English title and abstract. Arrange translation for full-text assessment. If translation is not feasible, document these as "reports not retrieved" with the reason "language" in your PRISMA flow diagram. Describe language restrictions in your methods and discuss potential language bias.
What if I find additional relevant studies after screening is complete?
If additional studies are identified (e.g., through citation searching of included studies), process them through the same screening and eligibility assessment. Document them separately in the "other sources" column of your PRISMA flow diagram if you defined your protocol to include these sources. This is one reason to use a template that includes other sources. See our guide on PRISMA 2020 flow diagram templates.