IJE Advance Access published online on April 18, 2008
International Journal of Epidemiology, doi:10.1093/ije/dyn065
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Sensitivity of between-study heterogeneity in meta-analysis: proposed metrics and empirical evaluation
1Clinical Trials and Evidence-Based Medicine Unit and Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45110, Greece.
2Institute for Clinical Research and Health Policy Studies, Tufts-New England Medical Center, Tufts University School of Medicine, Boston, MA 02111, USA.
3Biomedical Research Institute, Foundation for Research and Technology-Hellas, Ioannina 45110, Greece.
*Corresponding author. Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45 110, Greece. E-mail: jioannid{at}cc.uoi.gr
 |
Abstract |
|---|
Background Several approaches are available for evaluating heterogeneity in meta-analysis. Sensitivity analyses are often used, but these are often implemented in various non-standardized ways.
Methods We developed and implemented sequential and combinatorial algorithms that evaluate the change in between-study heterogeneity as one or more studies are excluded from the calculations. The algorithms exclude studies aiming to achieve either the maximum or the minimum final I2 below a desired pre-set threshold. We applied these algorithms in databases of meta-analyses of binary outcome and 
4 studies from Cochrane Database of Systematic Reviews (Issue 4, 2005, n = 1011) and meta-analyses of genetic associations (n = 50). Two I2 thresholds were used (50% and 25%).
Results Both algorithms have succeeded in achieving the pre-specified final I2 thresholds. Differences in the number of excluded studies varied from 0% to 6% depending on the database and the heterogeneity threshold, while it was common to exclude different specific studies. Among meta-analyses with initial I2 > 50%, in the large majority [19 (90.5%) and 208 (85.9%) in genetic and Cochrane meta-analyses, respectively] exclusion of one or two studies sufficed to decrease I2 < 50%. Similarly, among meta-analyses with initial I2 > 25%, in most cases [16 (57.1%) and 382 (81.3%), respectively) exclusion of one or two studies sufficed to decrease heterogeneity even <25%. The number of excluded studies correlated modestly with initial estimated I2 (correlation coefficients 0.52–0.68 depending on algorithm used).
Conclusions The proposed algorithms can be routinely applied in meta-analyses as standardized sensitivity analyses for heterogeneity. Caution is needed evaluating post hoc which specific studies are responsible for the heterogeneity.
Keywords Heterogeneity, sensitivity analysis, sequential algorithm, meta-analysis
Accepted 12 March 2008
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  M. D Coory Comment on: heterogeneity in meta-analysis should be expected and appropriately quantified Int. J. Epidemiol., April 6, 2009; (2009) dyp157v1. [Full Text] [PDF]
|
|
|
|
|
|
N. A Patsopoulos and J. P A Ioannidis Author's: Response Heterogeneity metrics: not perfect, but would not abandon Int. J. Epidemiol., April 6, 2009; (2009) dyp158v1. [Full Text] [PDF]
|
|
|
|
|
|
N. A Patsopoulos, E. Evangelou, and J. P. Ioannidis Heterogeneous views on heterogeneity Int. J. Epidemiol., October 21, 2008; (2008) dyn235v1. [Full Text] [PDF]
|
|
|
|
|
|
G. D. Smith How do we know, what do we know and what can knowledge do? From John Brownlee to translational medicine Int. J. Epidemiol., October 1, 2008; 37(5): 911 - 913. [Full Text] [PDF]
|
|
|
|
|
|
J. P T Higgins Commentary: Heterogeneity in meta-analysis should be expected and appropriately quantified Int. J. Epidemiol., October 1, 2008; 37(5): 1158 - 1160. [Full Text] [PDF]
|
|