IJE Advance Access originally published online on September 29, 2008
International Journal of Epidemiology 2009 38(1):276-286; doi:10.1093/ije/dyn179
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Can trial sequential monitoring boundaries reduce spurious inferences from meta-analyses?
1 Copenhagen Trial Unit, Center for Clinical Intervention Research, Copenhagen University Hospital, Department 3344, Blegdamsvej 9, DK-2100 Copenhagen, Denmark.
2 Faculty of Health Sciences, Department of Clinical Epidemiology and Biostatistics, McMaster University, CLARITY, L8N 3Z5 Hamilton, Ontario, Canada.
3 Department of Hygiene and Epidemiology, Clinical and Molecular Epidemiology Unit, University of Ioannina School of Medicine, Ioannina, 45110, Greece.
4 Faculty of Health Sciences, Department of Clinical Epidemiology Centre for Evaluation of Medicines, McMaster University, Biostatistics/FSORC, St Joseph's Healthcare Hamilton, Hamilton ON L8N 4A6, Ontario, Canada.
* Corresponding author. Copenhagen Trial Unit, Center for Clinical Intervention Research, Copenhagen University Hospital, Department 3344, Blegdamsvej 9, DK-2100 Copenhagen, Denmark. E-mail: kthorlund{at}ctu.rh.dk
 |
Abstract |
|---|
Background Results from apparently conclusive meta-analyses may be false. A limited number of events from a few small trials and the associated random error may be under-recognized sources of spurious findings. The information size (IS, i.e. number of participants) required for a reliable and conclusive meta-analysis should be no less rigorous than the sample size of a single, optimally powered randomized clinical trial. If a meta-analysis is conducted before a sufficient IS is reached, it should be evaluated in a manner that accounts for the increased risk that the result might represent a chance finding (i.e. applying trial sequential monitoring boundaries).
Methods We analysed 33 meta-analyses with a sufficient IS to detect a treatment effect of 15% relative risk reduction (RRR). We successively monitored the results of the meta-analyses by generating interim cumulative meta-analyses after each included trial and evaluated their results using a conventional statistical criterion (
= 0.05) and two-sided Lan-DeMets monitoring boundaries. We examined the proportion of false positive results and important inaccuracies in estimates of treatment effects that resulted from the two approaches.
Results Using the random-effects model and final data, 12 of the meta-analyses yielded P > = 0.05, and 21 yielded P 
= 0.05. False positive interim results were observed in 3 out of 12 meta-analyses with P > = 0.05. The monitoring boundaries eliminated all false positives. Important inaccuracies in estimates were observed in 6 out of 21 meta-analyses using the conventional P = 0.05 and 0 out of 21 using the monitoring boundaries.
Conclusions Evaluating statistical inference with trial sequential monitoring boundaries when meta-analyses fall short of a required IS may reduce the risk of false positive results and important inaccurate effect estimates.
Keywords Meta-analysis, information size, monitoring boundaries, spurious inferences
Accepted 30 July 2008
CiteULike 
Connotea 
Del.icio.us What's this?
Related articles in Int. J. Epidemiol.:
- Apparently conclusive meta-analyses may be inconclusive—Trial sequential analysis adjustment of random error risk due to repetitive testing of accumulating data in apparently conclusive neonatal meta-analyses
- Jesper Brok, Kristian Thorlund, Jørn Wetterslev, and Christian Gluud
Int. J. Epidemiol. 2009 38: 287-298.[Abstract] [FREE Full Text]
This article has been cited by other articles:
|
|
 |
|
  M. Monami, N. Marchionni, and E. Mannucci Glucagon-like peptide-1 receptor agonists in type 2 diabetes: a meta-analysis of randomized clinical trials Eur. J. Endocrinol., June 1, 2009; 160(6): 909 - 917. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. Brok, K. Thorlund, J. Wetterslev, and C. Gluud Apparently conclusive meta-analyses may be inconclusive--Trial sequential analysis adjustment of random error risk due to repetitive testing of accumulating data in apparently conclusive neonatal meta-analyses Int. J. Epidemiol., February 1, 2009; 38(1): 287 - 298. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
E. Nuesch and P. Juni Commentary: Which meta-analyses are conclusive? Int. J. Epidemiol., February 1, 2009; 38(1): 298 - 303. [Full Text] [PDF]
|
|