EBM hierarchies rank study types based on the strength and precision of their research methods. Different hierarchies exist for different question types, and even experts may disagree on the exact rank of information in the evidence hierarchies. Still, most agree that current, well designed systematic reviews and meta-analyses are at the top of the pyramid, and that expert opinion and anecdotal experience are at the bottom.
(Szajewska, 2018)
The next step in the EMB protocol is called critical appraisal.
Critical appraisal is the process of carefully and systematically assessing the outcome of scientific research (evidence) to judge its trustworthiness, value and relevance in a particular context. Critical appraisal looks at the way a study is conducted and examines factors such as internal validity, generalization and relevance.
Some initial appraisal questions you could ask are:
1. Is the evidence from a known, reputable source?
2. Has the evidence been evaluated in any way? If so, how and by whom?
3. How up-to-date is the evidence?
Secondly, you could look at the study itself and ask the following general appraisal questions:
1. How was the outcome measured?
2. Is that a reliable way to measure?
3. How large was the effect size?
4. What implications does the study have for your practice? Is it relevant?
5. Can the results be applied to your organization?
If the answer to any of these questions is “no”, you can save yourself the trouble of reading the rest of the article. Keep searching!
There are a number of different resources available to help with critical appraisal.
The CAT (Critically Appraised Topic) Manager App for Android and iOS helps managers and consultants critically appraise the trustworthiness of scientific studies published in academic journals.
The following are links to appraisal questionnaires (checklists) for specific study designs:
The following (see below) are downloadable Word document worksheets for specific study designs:
Szajewska, H. (2018). Evidence-based medicine and clinical research: both are needed, neither is perfect. Annals of Nutrition & Metabolism, 72 (suppl 3):13–23. doi:10.1159/000487375
Ask us