IJE Advance Access originally published online on April 18, 2008
International Journal of Epidemiology 2008 37(5):1148-1157; doi:10.1093/ije/dyn065
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Sensitivity of between-study heterogeneity in meta-analysis: proposed metrics and empirical evaluation
1 Clinical Trials and Evidence-Based Medicine Unit and Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45110, Greece.
2 Institute for Clinical Research and Health Policy Studies, Tufts-New England Medical Center, Tufts University School of Medicine, Boston, MA 02111, USA.
3 Biomedical Research Institute, Foundation for Research and Technology-Hellas, Ioannina 45110, Greece.
* Corresponding author. Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45 110, Greece. E-mail: jioannid{at}cc.uoi.gr
 |
Abstract |
|---|
Background Several approaches are available for evaluating heterogeneity in meta-analysis. Sensitivity analyses are often used, but these are often implemented in various non-standardized ways.
Methods We developed and implemented sequential and combinatorial algorithms that evaluate the change in between-study heterogeneity as one or more studies are excluded from the calculations. The algorithms exclude studies aiming to achieve either the maximum or the minimum final I2 below a desired pre-set threshold. We applied these algorithms in databases of meta-analyses of binary outcome and 
4 studies from Cochrane Database of Systematic Reviews (Issue 4, 2005, n = 1011) and meta-analyses of genetic associations (n = 50). Two I2 thresholds were used (50% and 25%).
Results Both algorithms have succeeded in achieving the pre-specified final I2 thresholds. Differences in the number of excluded studies varied from 0% to 6% depending on the database and the heterogeneity threshold, while it was common to exclude different specific studies. Among meta-analyses with initial I2 > 50%, in the large majority [19 (90.5%) and 208 (85.9%) in genetic and Cochrane meta-analyses, respectively] exclusion of one or two studies sufficed to decrease I2 < 50%. Similarly, among meta-analyses with initial I2 > 25%, in most cases [16 (57.1%) and 382 (81.3%), respectively) exclusion of one or two studies sufficed to decrease heterogeneity even <25%. The number of excluded studies correlated modestly with initial estimated I2 (correlation coefficients 0.52–0.68 depending on algorithm used).
Conclusions The proposed algorithms can be routinely applied in meta-analyses as standardized sensitivity analyses for heterogeneity. Caution is needed evaluating post hoc which specific studies are responsible for the heterogeneity.
Keywords Heterogeneity, sensitivity analysis, sequential algorithm, meta-analysis
Accepted 12 March 2008
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  M. D Coory Comment on: heterogeneity in meta-analysis should be expected and appropriately quantified Int. J. Epidemiol., April 6, 2009; (2009) dyp157v1. [Full Text] [PDF]
|
|
|
|
|
|
N. A Patsopoulos and J. P A Ioannidis Author's: Response Heterogeneity metrics: not perfect, but would not abandon Int. J. Epidemiol., April 6, 2009; (2009) dyp158v1. [Full Text] [PDF]
|
|
|
|
|
|
N. A Patsopoulos, E. Evangelou, and J. P. Ioannidis Heterogeneous views on heterogeneity Int. J. Epidemiol., October 21, 2008; (2008) dyn235v1. [Full Text] [PDF]
|
|
|
|
|
|
G. D. Smith How do we know, what do we know and what can knowledge do? From John Brownlee to translational medicine Int. J. Epidemiol., October 1, 2008; 37(5): 911 - 913. [Full Text] [PDF]
|
|
|
|
|
|
J. P T Higgins Commentary: Heterogeneity in meta-analysis should be expected and appropriately quantified Int. J. Epidemiol., October 1, 2008; 37(5): 1158 - 1160. [Full Text] [PDF]
|
|