Skip to Main Content
Libraries
askus Ask us
 

Systematic Reviews & Evidence Synthesis

A guide for researchers undertaking a systematic review or for those interested in evidence synthesis

When is a systematic review necessary?

Systematic reviews should be done:

  • When there is a large body of primary research on a specific research question
  • When a transparent search methodology and replicability are needed
  • When multiple published studies point to contradictory outcomes
  • When an existing systematic review is outdated (consider updating the existing review)
  • When no ongoing or existing systematic review addresses your research question

Before beginning a systematic review, researchers should address these questions:

  • Is there is enough literature published on the topic to warrant a review? 
    Systematic reviews are designed to distill the evidence from many studies into actionable insights. Is there a body of evidence available to analyze, or does more primary research need to be done?
     
  • Can your research question be answered by a systematic review?
    Systematic review questions should be specific and clearly defined. Questions that fit the PICO (problem/patient, intervention, comparison, outcome) format are usually well-suited for the systematic review methodology. The research question determines the search strategy, inclusion criteria, and data that you extract from the selected studies, so it should be clearly defined at the start of the review process.
     
  • Do you have a protocol outlining the review plan?
    The protocol is the roadmap for the review project. A good protocol outlines study methodology, includes the rationale for the systematic review, and describes the key question broken into PICO components. It is also a good place to plan out inclusion/exclusion criteria, databases that will be searched, data abstraction and management methods, and how the studies will be assessed for methodological quality.
     
  • Do you have a team of experts?
    A systematic review is team effort. Having multiple reviewers minimizes bias and strengthens analysis. Teams or often composed of  subject experts, two or more literature screeners, a librarian to conduct the search, and a statistician to analyze the data. 
     
  • Do you have the time that it takes to properly conduct a systematic review? 
    Systematic reviews typically take 12-18 months. 
     
  • Do you have a method for discerning bias?  
    There are many types of bias, including selection, performance, & reporting bias, and assessing the risk of bias of individual studies is an important part of your study design.
     
  • Can you afford to have articles in languages other than English translated?
    You should include all relevant studies in your systematic review, regardless of the language they were published in, so as to avoid language bias. 

Source: Reproduced with gratitude from the University of Michigan's Systematic Review research guide

Has another systematic review been done on your topic?

Search the following databases in your subject area to find if a recent systematic review has already been done. 

A good strategy is to find one on a closely related topic and build on its search strategy. Note: This is not a comprehensive list and you may need to search in other databases in your field.

  • PROSPERO
    This international database includes records for health and social care systematic reviews that have been registered but not yet published. Key features from the review protocol are recorded and maintained as a permanent record in PROSPERO.
  • Joanna Briggs Institute Registered Systematic Reviews

    Titles and main points of systematic reviews registered with the international Joanna Briggs Institute.

  • Pubmed Clinical Queries

    Locates Systematic Reviews in the Medline database

  • Epistemonikos

    Epistemonikos connects systematic reviews and their included studies, and thus allows clustering of systematic reviews based on the primary studies they have in common.

  • Cochrane Database of Systematic Reviews (CDSR)

    Full-text access to regularly updated systematic reviews by the Cochrane Collaboration. Includes completed systematic reviews and review protocols in development.

  • Collaboration for Environmental Evidence

    Systematic reviews related to the environment or conservation.

  • Campbell Collaboration

    For information on conducting systematic reviews in the social sciences, consult the Campbell Collaboration.

  • EPPI-Centre Systematic Reviews

    Systematic reviews focused on health promotion, public health, education and social policy, and related areas.

Is a systematic review the right type of review for your research project?

Use these tools to decide if you should complete a systematic review or another form of evidence synthesis.

  • Developed by librarians at the University of Manitoba Libraries, this interactive survey will help you determine which types of review might be suitable for your project: What Review is Right for You?
  • Review Methodology Decision Tree by Cornell University is a static document that visually lays out the review options available to you.

This table highlights key differences between systematic reviews and literature reviews (sometimes called narrative reviews):

Category Systematic Review Literature Review
Purpose

To find the answer to a specific research question.

This question is developed and may be registered before the systematic review begins.
Need not answer a question. Answers the question “What do we know about _?” Critical; synthesis of theories and approaches to a problem or topic. Conceptual categories
Methodology As prescribed precisely by PRISMA. Documentation of methodology must be included in the review. Varies by discipline and topic
Criteria for evidence Pre-defined and confirmed by 2+ raters. Empirical, qualitative evidence; may include discussion of and analysis of theoretical frameworks, etc.
Criteria for inclusion may not be stated.
Type of publications retrieved from search Primary research, e.g. peer-reviewed journal articles, clinical trials, conference proceedings. Varies by discipline and topic but may include primary sources (e.g. archival materials, datasets), as well as monographs, journal articles and proceedings.
Database(s) search terms and strategies Documented, and forms part of most systematic reviews, typically as an appendix; aim is replicability Documentation of database search(es) not typically required in completed review.


Source: Reproduced with gratitude from University of British Columbia Systematic and Scoping Reviews Search Methodology

What resources are needed to complete a systematic review?

Time, Teamwork, Tools

  • Time
    Systematic reviews can be very time intensive (up to 18 months, by some estimates), and other reviews may be more appropriate for you if you have limited time, or are working alone. The PredicTER tool can give you an estimate of how much time may be needed for your review. 

    The Cochrane Handbook for Systematic Reviews of Interventions suggests the following timeline to complete a Cochrane Review:

    Month

    Activity

    1 - 2

    Preparation of protocol

    3 - 8

    Searches for published & unpublished studies

    2 - 3

    Pilot test of eligibility criteria

    3 - 8

    Inclusion assessments

    3

    Pilot test of 'Risk of bias' assessment

    3 - 10

    Validity assessments

    3

    Pilot test of data collection

    3 - 10

    Data collection

    3 - 10

    Data entry

    5 - 11

    Follow up of missing information

    8 - 10

    Analysis

    1 - 11

    Preparation of review report

    12 -

    Keeping the review up to date

    (Higgins & Thomas, 2019)
     
  • Teamwork
    A systematic review cannot be typically done alone. Results must be screened and appraised by a minimum of two reviewers, ideally with a third available to settle any disagreements.
     
  • Tools 
    Systematic review management software tools are specifically tailored to the needs of systematic review teams. These tools can help with data extraction, performing a meta-analysis, tracking team progress, and facilitating communication between members. Some of these tools are fee-based while others are free. View the tab titled Management and Software Tools in this guide for more information. You should also keep in mind that not every tool is appropriate for every kind of synthesis or review. 

Works Cited

Higgins, J., & Thomas, J. (2019). Cochrane Handbook for Systematic Reviews of Interventions. https://training.cochrane.org/handbook/current

Institute of Medicine. (2011). Finding what works in health care: standards for systematic reviews. Eden, J. Levit, L., Berg, A., & Morton, S. (Eds.) Washington, D.C.: National Academies Press.

 

Creative Commons License
This work by The University of Victoria Libraries is licensed under a Creative Commons Attribution 4.0 International License unless otherwise indicated when material has been used from other sources.