Skip to Main Content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.
Libraries
askus Ask us
 

Systematic Reviews & Evidence Synthesis

A guide for researchers undertaking a systematic review or for those interested in evidence synthesis

Step 1: Develop a Research Question

Developing your research question is one of the most important steps in the evidence synthesis process. At this stage in the process, you and your team have identified a knowledge gap in your field and are aiming to answer a specific question.

Formulating a clear, well-defined research question of appropriate scope is key to a successful evidence synthesis. The research question will be the foundation of your synthesis and from it your research team will identify 2-5 possible search concepts. These search concepts will later be used to build your search strategy. 

Step 2: Select Databases

The databases you choose will depend on your research question and the disciplines in which relevant research may be conducted. Every database works differently. Librarians can help with designing complex searches using the specialized syntax of individual databases.

Step 3: Select Gray Literature Sources

Gray (or grey) literature is literature produced by individuals or organizations outside of commercial and/or academic publishers. This can include information such as government reports, conference proceedings, graduate dissertations, unpublished clinical trials, and much more.  The sources you select will be informed by your research question and field of study, but should likely include, at a minimum, theses and dissertations. While not peer-reviewed, gray literature represents a valuable body of information that is critical to consider when synthesizing and evaluating all available evidence.

Step 4: Write a Search Strategy

Writing a successful search strategy takes an intimate knowledge of bibliographic databases. Several search techniques are common to a variety of licensed databases - subject headings, truncation, Boolean operators, and limits. Depending on your topic, there may also be search filters available to apply to one or more databases. 

In general, it is recommended that you work with a librarian to help you design comprehensive search strategies across a variety of databases.

Step 5: Translate Search Strategies

Evidence synthesis methods require authors to search multiple databases, and not all databases accept the same search "syntax." Each individual database requires use of specialized search syntax, and therefore evidence synthesis search strategies must be 'translated' between databases. 

For example: A search for vitamin D[tiab] in PubMed will show you all citations with the phrase "vitamin D" in the title, abstract, or keywords, but a search for vitamin D[tiab] in Web of Science will not work at all. 

Below is a template that you can use to document your search strategy translations and results. 

Step 6: Register a Protocol

An evidence synthesis protocol states your rationale, hypothesis, and planned methodology.  Members of the team then use the protocol as a guide to conducting the research. It is recommended that you register your protocol prior to conducting your review. This will improve transparency and reproducibility, but will also ensure that other research teams do not duplicate efforts. 

Though not comprehensive, below is a list of registries to consider:

Step 7: Citation Management

A citation management program will save you a lot of time when doing your evidence synthesis.  Programs like Endnote, Zotero or Mendeley will store and organize the citations collected during your screening, de-duplicate the results and automatically format in-text citations and bibliographies in your manuscript.

You will likely retrieve multiple versions of the same study as you search many databases and will need to de-duplicate your results before article screening.

Note: The evidence synthesis tool, Covidence, automatically de-duplicates your results.

Step 8: Article Screening

The purpose of article screening to remove studies that are clearly not related to your topic. Use your inclusion/exclusion criteria to first screen the title and abstracts of your studies and determine whether they are relevant to your research question. Once titles and abstracts have been screened, the full text must be retrieved and screened to definitely decide whether the study fits the eligibility criteria of your synthesis.

It is highly recommended that two independent reviewers screen all studies, resolving areas of disagreement by consensus or by a third party who is an expert in the field. 


Tools for Article Screening

  • Covidence is an online systematic review management tool that allows for independent title/abstract screening, full text screening, data extraction and risk of bias assessment. UVic has an institutional license to Covidence. Sign-up here
  • Rayyan is a free online tool that can be used for independent screening and coding of studies in an evidence synthesis. Rayyan will pre-populate inclusion and exclusion criteria, but you can customize these criteria. It also uses tagging and filtering to code and organize references.  Title and abstract screening can be conducted in one project, while full text screening can be conducted in a second project.

Step 9: Risk of Bias Assessment

Risk of bias assessment (sometimes called "quality assessment" or "critical appraisal") helps to establish transparency of evidence synthesis results and findings. A risk of bias assessment is often performed for each included study in your review.  Evidence syntheses strive to eliminate bias in their findings.  Individual studies that are included in a synthesis may include biases in their results or conclusions, for example design flaws that raise questions about validity of findings or an overestimate of intervention effect.  Risk of bias assessment generally is not required with evidence synthesis methods outside of systematic reviews. 

Step 10: Data Extraction

Whether you plan to perform a meta-analysis or not, you will need to establish a regimented approach to extracting data. Researchers often use a form or table to capture the data they will then summarize or analyze. The amount and types of data you collect, as well as the number of collaborators who will be extracting it, will dictate which extraction tools are best for your project. Programs like Excel or Google Spreadsheets may be the best option for smaller or more straightforward projects, while systematic review software platforms can provide more robust support for larger or more complicated data.


Tools for Data Extraction

  • Excel
    Excel is the most basic tool for the management of the screening and data extraction stages of the systematic review process. 

  • Covidence
    Covidence is a software platform built specifically for managing each step of a systematic review project, including data extraction. Read more about how Covidence can help you customize extraction tables and export your extracted data.  

  • RevMan
    RevMan is free software used to manage Cochrane reviews. For more information on RevMan, including an explanation of how it may be used to extract and analyze data, watch Introduction to RevMan - a guided tour.

  • The Systematic Review Toolbox
    The SR Toolbox is a community-driven, searchable, web-based catalogue of tools that support the systematic review process across multiple domains. Use the advanced search option to restrict to tools specific to data extraction. 

Step 11: Synthesize Results

Once the extraction and appraisal of individual included studies is completed, a narrative synthesis is required.  The purpose of the synthesis is to bring together the principal findings of the systematic review in a narrative (i.e. text) form.  Reviewers should also consider including a "Summary of Findings" table to support the narrative in a clear format.

Works Cited

Source: Adapted with gratitude from Cornell University's A Guide to Evidence Synthesis

Creative Commons License
This work by The University of Victoria Libraries is licensed under a Creative Commons Attribution 4.0 International License unless otherwise indicated when material has been used from other sources.