{'fraction_non_alphanumeric': 0.031607402786442085, 'fraction_numerical': 0.011228945726762321, 'mean_word_length': 4.593023255813954, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Interpreting meta-analysis in systematic reviews

A meta-analysis estimates an average effect of 2 or more studies that address the same question in more or less the same way. In this guide to interpreting meta-analyses, we use a systematic review by Bravata and colleagues (1) on the effects of pedometers to increase physical activity to illustrate these concepts. A good systematic review includes a thorough search for all studies, an appraisal of their quality, and selection of the better studies for answering the question. We won't go over the appraisal here but will focus on reading the combined results. Reading a meta-analysis can be broken down into 4 basic steps (2): 1. What is the summary measure? 2. What does the forest plot show? 3. What does the pooled effect (average effect) mean? 4. Was it valid to combine studies? 1. What is the summary measure? Read the horizontal scale of the forest plot (Figure) to check which measure of effect has been used and which side is good and bad for the treatment. The effect in each study can be summarized in different ways (Table). In the pedometer review, the outcome of interest is the mean difference in the number of steps per day. 2. What does the forest plot show? A forest plot is the usual method of graphic display for meta-analyses. It presents the findings for each study plus (usually) the combined results. This allows readers to visualize how much uncertainty surrounds the results. The Figure presents a forest plot and identifies its basic components. A vertical line of no effect (in the center of the graph) represents the result if the intervention had no impact. Each included study is shown as a horizontal line with a square in the middle, which corresponds to the study's effect and the 95% CI. The size of the central square is roughly equal to the size of the study (or more exactly, the proportion of the weight that study contributes to the combined effect). Notice that, depending on the outcome being measured, a positive effect will mean that the square and the 95% CI lie on the left or the right of the line of no effect. In the pedometer review, an increase in steps taken is seen as a positive outcome, and therefore we expect the positive studies to lie on the right-hand side of the Figure. The combined (average) effect appears as a diamond at the bottom of the forest plot. The center of the diamond is equal to the average effect, and the extremes of the diamond are equal to the 95% CI of the average effect. In the pedometer review, the average effect is positive, with the intervention increasing the number of steps by 2500 on average. 3. What does the pooled effect (average effect) mean? The diamond, or average effect, is obtained by combining the summary measures collected for all included studies. This gives a single (more precise) value of the effect of the intervention. The idea is simple: Each study provides some information about the real effect, and by combining all of them, we are more likely to be near the truth. Because the information provided by the trials differs, pooling information usually involves obtaining some form of weighted average. Larger weights are given to those studies that are believed to give more information (usually larger studies). 4. Was it valid to combine studies? Obtaining a result from the combination of several studies raises an obvious question: Was combining these studies valid? No 2 studies are exactly alike: They all vary in their PICOspopulation, intervention, comparator, or outcomein some minor or major way. For this reason, the question has to be addressed in 3 steps: Are the studies (a) clinically similar and (b) methodologically similar, and (c) are the findings statistically similar (homogeneity)? For the first step, content expertise is needed, while for the second step, critical appraisal skills are necessary. The final step, assessing statistical homogeneity, uses a statistic to test if the variation is larger than expected by chance. The i-squared is the most commonly used measure and assesses the degree of heterogeneity. As a rule of thumb, an i-squared >50% signals that combining studies is not valid (although this depends on the number and size of the studies included>. In the pedometer review, the heterogeneity for the main outcome (increase in step counts) was high: Increase in steps in the intervention group was 2491 (95% CI 1098 to 3885), i-squared 91%. To deal with this, the study investigators removed a study (7) that reported a much higher increase in physical activity than the other studies. This reduced the effect to 2004 steps (CI 878 to 3129); however, the authors did not provide the i-squared statistic for this result, nor did they give clinical or methodological reasons for exclusion of the study.