Systematic Reviews: Knowing Where the Evidence Came From

Systematic Reviews are a key part of any research you do on a topic, because as the name implies, the evidence is reviewed systematically, with a formal plan, and the reader gets to know what the plan is, and how it was executed. Hopefully you will be getting more than someone's opinion about the medical literature, but instead an evaluation using specific criteria of quality. The biggest challenge with Systematic Reviews is that since they are so thorough, they can take a long time to be published, so the topic you are interested in might not have any, or be out of date.

A Narrative Review article is much more common. An author gives a summary or overview of the evidence, but doesn't have to tell you how they searched for the evidence, how many articles they looked at, or what their criteria was in deciding which articles to include.

TABLE 1. Contrasting systematic review and the methods of other types of reviews¹

Systematic review (minimum criteria) Other types of reviews A protocol (ie, working plan for the systematic review) is developed, along with a clearly formulated question. A protocol is not used, and the question may not be clear and concise. The literature is searched broadly by using many different methods (eg, searching multiple relevant databases, contacting experts, or scanning reference lists). The search strategy, including databases and years searched and search terms employed, is transparently reported in the systematic review manuscript. The literature may not be searched, or only the literature that the authors are aware of is searched. The search strategy is not reported or is not fully reported in the review manuscript. To determine study eligibility, the literature is screened by using criteria set a priori. Ideally, 2 independent reviewers screen all material and resolve conflicts through discussion. The eligibility criteria, including the number of articles that were excluded and the reasons for exclusion, are transparently reported in the systematic review manuscript. No eligibility criteria are set, and the authors are free to TAB and choose which studies should be included in the review. The inclusion criteria and the number of excluded articles are not reported in the manuscript. The risk of bias of the included studies is assessed by using validated and applicable study appraisal instruments to determine the validity of the study results. Ideally, 2 reviewers independently appraise the quality of all included studies and resolve conflicts through discussion. The quality of the included studies is not assessed, or unvalidated instruments are used to appraise the quality of included studies. Data are abstracted consistently from all included studies by using a previously defined data abstraction form. Ideally, 2 reviewers independently abstract data from all included studies and resolve conflicts through discussion. The data abstraction form is transparently described in the systematic review manuscript. The authors are free to TAB and choose which results from the included studies to report. Data may not be abstracted consistently from all included studies. The data abstraction form is not described. The results are synthesized by using the totality of evidence. A meta-analysis may be conducted if the included studies are deemed homogenous in terms of study population, study design, exposure or intervention examined, comparators studied, and outcomes assessed. Studies are summarized on the basis of the results that are most appealing to the authors. A meta-analysis may be performed with studies that are not homogenous. The discussion section provides an overall summary of the strengths and weaknesses of the included studies and a summary of the strengths and weaknesses of the systematic review itself. The discussion section may not provide an overall summary of the weaknesses of included studies or of the weaknesses of the review itself.

¹ One way to determine whether a study is a systematic review or another type of review, such as a narrative review, is to examine whether there is a methods section in the report. If a methods section is missing from the study report, chances are the study was not a systematic review, and, thus, it is susceptible to considerable bias in the results.

Related Resource:

Article on how to read a Review Paper

Introduction to Systematic Reviews