Systematic Reviews: A How-To Guide

Determine if a Systematic Review is Needed

Use these tools to decide if you should complete a systematic review or another form of evidence synthesis.

• Developed by the Knowledge Translation Program, this interactive resource will help you determine which types of review might be suitable for your project: What Review is Right for You? 
• Review Methodology Decision Tree by Cornell University is a static document that visually lays out the review options available to you.

Systematic reviews should be done:

• When there is a large body of primary research on a specific research question
• When a transparent search methodology and replicability are needed
• When multiple published studies point to contradictory outcomes
• When an existing systematic review is outdated (consider updating the existing review)
• When no ongoing or existing systematic review addresses your research question

Understand the Differences Between a Systematic Review and a Literature Review

This table highlights key differences between systematic reviews and literature reviews (sometimes called narrative reviews):

Source: Reproduced with gratitude from University of British Columbia Systematic and Scoping Reviews Search Methodology

Additional Considerations

Before beginning a systematic review, researchers should address these questions:

1. Is there is enough literature published on the topic to warrant a review? 
   Systematic reviews are designed to distill the evidence from many studies into actionable insights. Is there a body of evidence available to analyze, or does more primary research need to be done?
    
2. Can your research question be answered by a systematic review?
   Systematic review questions should be specific and clearly defined. Questions that fit the PICO (problem/patient, intervention, comparison, outcome) format are usually well-suited for the systematic review methodology. The research question determines the search strategy, inclusion criteria, and data that you extract from the selected studies, so it should be clearly defined at the start of the review process.
    
3. Do you have a protocol outlining the review plan?
   The protocol is the roadmap for the review project. A good protocol outlines study methodology, includes the rationale for the systematic review, and describes the key question broken into PICO components. It is also a good place to plan out inclusion/exclusion criteria, databases that will be searched, data abstraction and management methods, and how the studies will be assessed for methodological quality.
    
4. Do you have a team of experts?
   A systematic review is team effort. Having multiple reviewers minimizes bias and strengthens analysis. Teams or often composed of  subject experts, two or more literature screeners, a librarian to conduct the search, and a statistician to analyze the data. 
    
5. Do you have the time that it takes to properly conduct a systematic review? 
   Systematic reviews typically take 12-18 months. 
    
6. Do you have a method for discerning bias?  
   There are many types of bias, including selection, performance, & reporting bias, and assessing the risk of bias of individual studies is an important part of your study design.
    
7. Can you afford to have articles in languages other than English translated?
   You should include all relevant studies in your systematic review, regardless of the language they were published in, so as to avoid language bias. 

Source: Reproduced with gratitude from the University of Michigan's Systematic Review research guide

Research Already Completed Systematic Reviews

Search the following databases in your subject area to find if a recent systematic review has already been done.

A good strategy is to find one on a closely related topic and build on its search strategy. Note: This is not a comprehensive list and you may need to search in other databases in your field.

• PROSPERO
  This international database includes records for health and social care systematic reviews that have been registered but not yet published. Key features from the review protocol are recorded and maintained as a permanent record in PROSPERO.
• Joanna Briggs Institute Registered Systematic Reviews

  Titles and main points of systematic reviews registered with the international Joanna Briggs Institute.

• Pubmed Clinical Queries

  Locates Systematic Reviews in the Medline database

• Epistemonikos

  Epistemonikos connects systematic reviews and their included studies, and thus allows clustering of systematic reviews based on the primary studies they have in common.

• Cochrane Database of Systematic Reviews (CDSR)

  Full-text access to regularly updated systematic reviews by the Cochrane Collaboration. Includes completed systematic reviews and review protocols in development.

• Collaboration for Environmental Evidence

  Systematic reviews related to the environment or conservation.

• Campbell Collaboration

  For information on conducting systematic reviews in the social sciences, consult the Campbell Collaboration.

• EPPI-Centre Systematic Reviews

  Systematic reviews focused on health promotion, public health, education and social policy, and related areas.

Perform a Scoping Search

Once you have a clear question in mind, the next step is to perform a scoping search.

These are the main reasons for performing a scoping review:

1. If you are doing a systematic review, to identify if a systematic review has already been published, or is currently in process, covering the question you are considering. If it has you will need to come up with a different question. However, if the review is more than five years old you can perform an update of the same question.
2. To establish that there is enough literature available on your topic in order to be able to perform a review. If there isn't you will need to make your question broader or come up with a new one.
3. Alternatively, a scoping search may show you that you need to make your research question more specific as there may be vast amounts of literature available on your topic.
4. To get an idea of the search terms that are being used to retrieve information on your topic. This can be done by checking titles and abstracts of records you find and references used by key papers.

Source: Reproduced with modifications with gratitude from the University of London: Using a framework to structure your question Research Guide.

Determine Needed Resources

Time, Teamwork, Tools

• Time
  Systematic reviews can be very time intensive (up to 18 months, by some estimates), and other reviews may be more appropriate for you if you have limited time, or are working alone. The PredicTER tool can give you an estimate of how much time may be needed for your review. 
  
  The Cochrane Handbook for Systematic Reviews of Interventions suggests the following timeline to complete a Cochrane Review:

  Month	Activity
  1 - 2	Preparation of protocol
  3 - 8	Searches for published & unpublished studies
  2 - 3	Pilot test of eligibility criteria
  3 - 8	Inclusion assessments
  3	Pilot test of 'Risk of bias' assessment
  3 - 10	Validity assessments
  3	Pilot test of data collection
  3 - 10	Data collection
  3 - 10	Data entry
  5 - 11	Follow up of missing information
  8 - 10	Analysis
  1 - 11	Preparation of review report
  12 -	Keeping the review up to date

  (Higgins & Thomas, 2019)
   
• Teamwork
  A systematic review cannot be typically done alone. Results must be screened and appraised by a minimum of two reviewers, ideally with a third available to settle any disagreements.
   
• Tools 
  Systematic review management software tools are specifically tailored to the needs of systematic review teams. These tools can help with data extraction, performing a meta-analysis, tracking team progress, and facilitating communication between members. Some of these tools are fee-based while others are free. View the tab titled Management and Software Tools in this guide for more information. You should also keep in mind that not every tool is appropriate for every kind of synthesis or review. 

Higgins, J., & Thomas, J. (2019). Cochrane Handbook for Systematic Reviews of Interventions. https://training.cochrane.org/handbook/current

Institute of Medicine. (2011). Finding what works in health care: standards for systematic reviews. Eden, J. Levit, L., Berg, A., & Morton, S. (Eds.) Washington, D.C.: National Academies Press.