International Journal of Epidemiology 2002;31:115-123
© International Epidemiological Association 2002
Special Theme: Systematic Reviews and Meta-Analysis |
Direction and impact of language bias in meta-analyses of controlled trials: empirical study
a Medical Research Council Health Services Research Collaboration, Department of Social Medicine, University of Bristol, UK.
b Department of Social and Preventive Medicine, University of Berne, Switzerland.
Matthias Egger, Department of Social Medicine, Canynge Hall, Whiteladies Road, University of Bristol, Bristol BS8 2PR, UK. E-mail: m.egger{at}bristol.ac.uk
Abstract
Background Excluding clinical trials reported in languages other than English from meta-analyses may introduce bias and reduce the precision of combined estimates of treatment effects. We examined the influence of trials published in languages other than English on combined estimates and conclusions of published meta-analyses.
Methods We searched journals and the Cochrane Database of Systematic Reviews for meta-analyses of at least five trials with binary outcomes that were based on comprehensive literature searches without language restrictions. We compared estimates of treatment effects from trials published in languages other than English to those from trials published in English, and assessed the impact of restricting meta-analyses to trials published in English.
Results We identified 303 meta-analyses: 159 (52.4%) employed comprehensive literature searches of which 50 included 485 English and 115 non-English language trials. Non-English language trials included fewer participants (median 88 versus 116, P = 0.006) and were more likely to produce significant results at P < 0.05 (41.7% versus 31.3%, P = 0.033). The methodological quality of non-English language trials tended to be lower than that of trials published in English. Estimates of treatment effects were on average 16% (95% CI : 3–26%) more beneficial in non-English-language trials than in English-language trials. In 29 (58.0%) meta-analyses the change in effect estimates after exclusion of non-English language trials was less than 5%. In the remaining meta-analyses, 5 (10.0%) showed more benefit and 16 (32.0%) less benefit after exclusion of non-English language trials.
Conclusions This retrospective analysis suggests that excluding trials published in languages other than English has generally little effect on summary treatment effect estimates. The importance of non-English language trials is, however, difficult to predict for individual systematic reviews. Comprehensive literature searches followed by a careful assessment of trial quality are required to assess the contribution of all relevant trials, independent of language of publication.
Keywords Meta-analysis, randomized-controlled trials, publication bias, language bias, meta-regression
Accepted 2 October 2001
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  A. X. Garg, D. Hackam, and M. Tonelli Systematic Review and Meta-analysis: When One Study Is Just not Enough Clin. J. Am. Soc. Nephrol., January 1, 2008; 3(1): 253 - 260. [Full Text] [PDF]
|
|
|
|
 |
|
 C. B. Polis, K. Schaffer, K. Blanchard, A. Glasier, C. C. Harper, and D. A. Grimes Advance Provision of Emergency Contraception for Pregnancy Prevention: A Meta-Analysis Obstet. Gynecol., December 1, 2007; 110(6): 1379 - 1388. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J J Kingma, R de Knikker, H M Wittink, and T Takken Eccentric overload training in patients with chronic Achilles tendinopathy: a systematic review Br. J. Sports Med., June 1, 2007; 41(6): e3 - e3. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. S. Gangji, T. Cukierman, H. C. Gerstein, C. H. Goldsmith, and C. M. Clase A Systematic Review and Meta-Analysis of Hypoglycemia and Cardiovascular Events: A comparison of glyburide with other secretagogues and with insulin Diabetes Care, February 1, 2007; 30(2): 389 - 394. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 E. Zintzaras and A. G. Kaditis Sleep-Disordered Breathing and Blood Pressure in Children: A Meta-analysis Arch Pediatr Adolesc Med, February 1, 2007; 161(2): 172 - 178. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. Nicholson, H. Kuper, and H. Hemingway Depression as an aetiologic and prognostic factor in coronary heart disease: a meta-analysis of 6362 events among 146 538 participants in 54 observational studies Eur. Heart J., December 1, 2006; 27(23): 2763 - 2774. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 D. S. Liebeskind, C. S. Kidwell, J. W. Sayre, and J. L. Saver Evidence of publication bias in reporting acute stroke clinical trials. Neurology, September 26, 2006; 67(6): 973 - 979. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. F. Costello, M. Chapman, and U. Conway A systematic review and meta-analysis of randomized controlled trials on metformin co-administration during gonadotrophin ovulation induction or IVF in women with polycystic ovary syndrome Hum. Reprod., June 1, 2006; 21(6): 1387 - 1399. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. M. Richman, S. S. Liu, G. Courpas, R. Wong, A. J. Rowlingson, J. McGready, S. R. Cohen, and C. L. Wu Does Continuous Peripheral Nerve Block Provide Superior Pain Control to Opioids? A Meta-Analysis Anesth. Analg., January 1, 2006; 102(1): 248 - 257. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. Marx, D. Pewsner, M. Egger, R. Nuesch, H. C. Bucher, B. Genton, C. Hatz, and P. Juni Meta-Analysis: Accuracy of Rapid Tests for Malaria in Travelers Returning from Endemic Areas Ann Intern Med, May 17, 2005; 142(10): 836 - 846. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. S. Doig, F. Simpson, and A. Delaney A Review of the True Methodological Quality of Nutritional Support Trials Conducted in the Critically Ill: Time for Improvement Anesth. Analg., February 1, 2005; 100(2): 527 - 533. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. Eng, J. A. Krishnan, J. B. Segal, D. T. Bolger, L. J. Tamariz, M. B. Streiff, M. W. Jenckes, and E. B. Bass Accuracy of CT in the Diagnosis of Pulmonary Embolism: A Systematic Literature Review Am. J. Roentgenol., December 1, 2004; 183(6): 1819 - 1827. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. Kashyap, G. A. Wells, and Z. Rosenwaks Insulin-sensitizing agents as primary therapy for patients with polycystic ovarian syndrome Hum. Reprod., November 1, 2004; 19(11): 2474 - 2483. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M Delgado-Rodriguez and J Llorca Bias J. Epidemiol. Community Health, August 1, 2004; 58(8): 635 - 641. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. Kashyap, D. Moher, M. F. K. Fung, and Z. Rosenwaks Assisted Reproductive Technology and the Incidence of Ovarian Cancer: A Meta-Analysis Obstet. Gynecol., April 1, 2004; 103(4): 785 - 794. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 H. H. B. Yoo, T. T. Queluz, M. E. Kreider, and D. A. Lipson Locating and Selecting Appraisal Studies for Reviews Chest, February 1, 2004; 125(2): 798 - 799. [Full Text] [PDF]
|
|
|
|
|
|
M. F. Gallo, D. A. Grimes, K. F. Schulz, and F. M. Helmerhorst Combination Estrogen-Progestin Contraceptives and Body Weight: Systematic Review of Randomized Controlled Trials Obstet. Gynecol., February 1, 2004; 103(2): 359 - 373. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 B. M. Block, S. S. Liu, A. J. Rowlingson, A. R. Cowan, J. A. Cowan Jr, and C. L. Wu Efficacy of Postoperative Epidural Analgesia: A Meta-analysis JAMA, November 12, 2003; 290(18): 2455 - 2463. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. M. King, K. Alperovitz-Bichell, P. C. Rowe, and H. P. Lehmann Is Epinephrine Efficacious in the Treatment of Bronchiolitis? Arch Pediatr Adolesc Med, October 1, 2003; 157(10): 965 - 968. [Full Text] [PDF]
|
|
|
|
|
|
A. M. Sawka and L. Thabane Effect of Industry Sponsorship on the Results of Biomedical Research JAMA, May 21, 2003; 289(19): 2502 - 2503. [Full Text] [PDF]
|
|
|
|
|
|
J. E. Bekelman, C. P. Gross, and Y. Li Effect of Industry Sponsorship on the Results of Biomedical Research--Reply JAMA, May 21, 2003; 289(19): 2503 - 2503. [Full Text] [PDF]
|
|
|
|
 |
|
 M. Egger, S. Ebrahim, and G. D. Smith Where now for meta-analysis? Int. J. Epidemiol., February 1, 2002; 31(1): 1 - 5. [Full Text] [PDF]
|
|
|
|
|
|
M. Clarke Commentary: Searching for trials for systematic reviews: what difference does it make? Int. J. Epidemiol., February 1, 2002; 31(1): 123 - 124. [Full Text] [PDF]
|
|