??♂️ What are the stages of a systematic review?
??♀️ What does 'systematic' actually mean?
??♂️ Why don't all systematic reviews use meta-analysis?
??♀️ What does meta-analysis do?
Systematic reviewer pro-tip
??♀️ What are the other ways to synthesise evidence?
Conclusion
Systematic review and meta-analysis are two terms that you might see used interchangeably. Each term refers to research about research, but there are important differences!
A systematic review is a piece of work that asks a research question and then answers it by summarising the evidence that meets a set of pre-specified criteria. Some systematic reviews present their results using meta-analysis, a statistical method that combines the results of several trials to generate an average result. Meta-analysis adds value because it can produce a more precise estimate of the effect of a treatment than considering each study individually ?.
Let’s take a look at a few related questions that you might have about systematic reviews and meta-analysis.
A systematic review starts with a research question and a protocol or research plan. A review team searches for studies to answer the question using a highly sensitive search strategy. The retrieved studies are then screened for eligibility using the inclusion and exclusion criteria (this is done by at least two people working independently). Next, the reviewers extract the relevant data and assess the quality of the included studies. Finally, the review team synthesises the extracted study data (perhaps using meta-analysis) and presents the results. The process is shown in figure 1.
Covidence helps researchers complete systematic review quickly and easily! It supports reviewers with study selection, data extraction and quality assessment. Data exported from Covidence can be saved in Excel for reliable transfer to your choice of data analysis software or, if you’re writing a Cochrane Review, to RevMan 5.
In this context, systematic means that the methods used to search for and analyse the data are
transparent, reproducible and defined before searching begins. This is what differentiates a systematic review from a descriptive review that might be based on, for example, a subset of the literature that the author is familiar with at the time of writing. Systematic reviews strive to be as thorough and rigorous as possible to minimise the bias that would result from cherry-picking studies in a non-systematic way. Systematic reviews sit at the top of the evidence hierarchy because it is widely agreed that studies with rigorous methods are those best able to minimise the risk of bias on the results of the study. This is what makes systematic reviews the most reliable form of evidence (see figure 2).
Meta-analysis can improve the precision of an effect estimate. But it can also be misleading if it is performed with data that are not sufficiently similar, or with data whose methodological quality is poor (for example, because the study participants were not properly randomized). So it’s not always appropriate to use meta-analysis and many systematic reviews do not include them. Reviews that do not contain meta-analysis can still synthesise study data to produce something that has greater value than the sum of its parts.
Meta-analysis produces a more precise estimate of treatment effect. There are several types of effect size and the most suitable type is chosen by the review team based on the type of outcomes and interventions under investigation. Typical effect sizes in systematic reviews are the odds ratio, the risk ratio, the weighted mean difference and the standardized mean difference. The results of a meta-analysis are displayed using a forest plot like the one in figure 3.
Some meta-analyses also include subgroup analysis or meta-regression. These techniques are used to explore a factor (for example, the age of the study participant) that might influence the relationship between the treatment and the intervention. Plans to analyse the data using these techniques should be described and justified before looking at the data, ideally at the research plan or protocol stage, to avoid introducing bias. Like meta-analysis, subgroup analysis and meta-regression are advisable only in certain circumstances.
Think carefully before you plan subgroup analysis or meta-regression and always ask a methodologist for advice
Systematic reviews combine study data in a number of ways to reach an overall understanding of the evidence. Meta-analysis is a type of statistical synthesis. Narrative synthesis combines the findings of multiple studies using words. All systematic reviews, including those that use meta-analysis, are likely to contain an element of narrative synthesis by summarising in words the evidence included in the review. But narrative synthesis doesn’t just describe the included studies: it also seeks to explain the gathered evidence, for example by looking at similarities and differences between the study findings and by exploring possible reasons for those similarities and differences in a systematic way. Narrative synthesis should not be confused with narrative review, which is a term sometimes used for a non-systematic review of the literature (for example in a textbook chapter) where there is no systematic attempt to address issues of bias.
There are many types of systematic review. What they all have in common is the use of transparent and reproducible methods that are defined before the search begins. There is no ‘best’ way to synthesise systematic review evidence, and the most suitable approach will depend on factors such as the nature of the review question, the type of intervention and the outcomes of interest.
Covidence is a web-based tool that saves you time at the screening, selection, data extraction and quality assessment stages of your review. It provides easy collaboration across teams and a clear overview of task status, helping you to efficiently complete your review. Sign up for a free trial today!