Systematic Reviews

Critical appraisal is an integral part of the process of Evidence Based Practice (EBP). Critical appraisal aims to identify methodological flaws in the literature and provide consumers of research evidence the opportunity to make informed decisions about the quality of the research evidence. When critically appraising the evidence, make sure to look at the study methodology, the statistical tests and methods used, and the results. If you have questions, find the answers to those questions!

• Julian PT Higgins, Douglas G Altman and Jonathan AC Sterne on behalf of the Cochrane Statistical Methods Group and the Cochrane Bias Methods Group. Assessing risk of bias in included studies. In Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011). Chapter 8. Available from http://handbook-5-1.cochrane.org.
• RoB 2 tool: For risk of bias in individually randomized, parallel group trials; cluster randomized, parallel group trials; and individually randomized, cross-over trials.

• ROBINS-I tool: For risk of bias in non-randomized quantitative studies (including cohort-like designs and quasi-randomized studies) estimating effectiveness of an intervention.
• Newcastle-Ottawa Scale: For quality assessment of nonrandomised studies (including case-control and cohort studies) in meta-analyses.

• Downes MJ, Brennan ML, Williams HC, Dean RS. Development of a critical appraisal tool to assess the quality of cross-sectional studies (AXIS). BMJ Open. 2016 Dec 8;6(12):e011458. doi: 10.1136/bmjopen-2016-011458. PMID: 27932337; PMCID: PMC5168618.

• QUADAS-2

• COSMIN Checklist: Risk of bias checklist for assessing the quality of studies involving outcome measurement instruments (mainly for patient reported outcome measures, or PROMs).
• QODA Checklist: For critically appraising questionnaire origin (sources from which questionnaire elements are made) and initial development (process of making the questionnaire from its elements).

• Mixed Methods Appraisal Tool

• Baethge C, Goldbeck-Wood S, Mertens S. SANRA-a scale for the quality assessment of narrative review articles. Res Integr Peer Rev. 2019 Mar 26;4:5. doi: 10.1186/s41073-019-0064-8. PMID: 30962953; PMCID: PMC6434870.

• Wolff RF, Moons KGM, Riley RD, Whiting PF, Westwood M, Collins GS, Reitsma JB, Kleijnen J, Mallett S; PROBAST Group†. PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies. Ann Intern Med. 2019 Jan 1;170(1):51-58. doi: 10.7326/M18-1376. PMID: 30596875.

• Noyes J, Booth A, Flemming K, Garside R, Harden A, Lewin S, Pantoja T, Hannes K, Cargo M, Thomas J. Cochrane Qualitative and Implementation Methods Group guidance series-paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings. J Clin Epidemiol. 2018 May;97:49-58. doi: 10.1016/j.jclinepi.2017.06.020. Epub 2017 Dec 13. PMID: 29247700.

• AMSTAR 2: For critically appraising systematic reviews of randomised and non-randomised studies of healthcare interventions.
• ROBIS: For risk of bias in systematic reviews.

• Health Evidence Appraisal Tools