IJE Advance Access published online on May 21, 2007
International Journal of Epidemiology, doi:10.1093/ije/dym087
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials
1The Nordic Cochrane Centre, Rigshospitalet, DK.
2Department of Biostatistics, University of Copenhagen, DK.
3Centre for Statistics in Medicine, Oxford, UK.
*Corresponding author. The Nordic Cochrane Centre, Rigshopitalet, 3343 Blegdamsvej 9, DK-2100 Copenhagen, Denmark. E-mail: jp{at}cochrance.dk
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
Background Randomized trials without reported adequate allocation concealment have been shown to overestimate the benefit of experimental interventions. We investigated the robustness of conclusions drawn from meta-analyses to exclusion of such trials.
Material Random sample of 38 reviews from The Cochrane Library 2003, issue 2 and 32 other reviews from PubMed accessed in 2002. Eligible reviews presented a binary effect estimate from a meta-analysis of randomized controlled trials as the first statistically significant result that supported a conclusion in favour of one of the interventions.
Methods We assessed the methods sections of the trials in each included meta-analysis for adequacy of allocation concealment. We replicated each meta-analysis using the authors' methods but included only trials that had adequate allocation concealment. Conclusions were defined as not supported if our result was not statistically significant.
Results Thirty-four of the 70 meta-analyses contained a mixture of trials with unclear or inadequate concealment as well as trials with adequate allocation concealment. Four meta-analyses only contained trials with adequate concealment, and 32, only trials with unclear or inadequate concealment. When only trials with adequate concealment were included, 48 of 70 conclusions (69%; 95% confidence interval: 56–79%) lost support. The loss of support mainly reflected loss of power (the total number of patients was reduced by 49%) but also a shift in the point estimate towards a less beneficial effect.
Conclusion Two-thirds of conclusions in favour of one of the interventions were no longer supported if only trials with adequate allocation concealment were included.
Keywords Bias (epidemiology), double-blind method, methods, randomized controlled trials, meta-analysis
Accepted 27 March 2007
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
Concealment of treatment allocation and blinding are important safeguards against bias in randomized controlled trials.1 Allocation concealment serves to avoid selection bias by concealing what treatment the next patient will receive, if enrolled. Without concealment the person in charge of enrolment might channel patients with a better prognosis into his or her preferred treatment, for instance by influencing whether a patient enters the trial at all. The blinding of key trial persons (e.g. patients, treatment providers and data collectors) is sometimes referred to as double-blinding. However, ‘double-blinding’ is inconsistently interpreted. For instance, some believe it means that patients and treatment providers are blinded, others that patients and data collectors are blinded.2,3
Empirical studies show that the effects of experimental interventions measured as odds ratios are exaggerated on average by 21% [ratio of odds ratios (ROR): 0.79, 95% confidence interval (CI): 0.66–0.95], if allocation concealment is unclear or inadequate, and by 18% (ROR: 0.82, 95% CI 0.71–1.05), if trials are not reported as ‘double-blind’.4–10
To reduce the risk of introducing interventions based on trial results that are inflated, a simple approach would be to rely only on trials with clear indication of adequate allocation concealment. A similar exclusion of trials not reported as ‘double-blind’ would be more difficult to interpret; since, as opposed to allocation concealment that can and should always be performed, blinding may be impossible or the feasibility of it doubtful. In addition, the term ‘double-blind’ is inconsistently used. Thus, our primary aim was to estimate the fraction of conclusions based on statistically significant results in meta-analyses that would no longer be supported if only trials with reported adequate allocation concealment were included. Our secondary aim was to assess the impact of absence vs presence of reported adequate allocation concealment in trials on the effect estimates. We also studied how double-blinding—as a possible confounder—was related to allocation concealment and how absence of double-blinding affected the treatment effect estimate.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
Identification and selection of reviews
We aimed to retrieve 70 published reviews containing one or more meta-analyses, with about half of them being Cochrane Reviews. This number was arbitrarily chosen based on our anticipation of how informative the meta-analyses would be and the workload. Reviews were eligible if the authors concluded that one of the assessed interventions was superior to the other and if this preference was supported by the first statistically significant result of a meta-analysis reported in the abstract. When a review presented several meta-analyses, only the meta-analysis identified as described was included. Reviews were excluded if:
- The first statistically significant result of a meta-analysis reported in the abstract was not for a binary outcome.
- Substantial uncertainty existed concerning what the authors of the review perceived as experimental and conventional treatment.
- There were more than 40 trials in the first statistically significant meta-analysis reported in the abstract.
- A genuine meta-analysis was not performed.
- The abstract of the review explicitly stated that it was partly based on non-randomized trials.
These selection criteria were applied on reviews identified as follows: in the Cochrane Library 2003, issue 2 all reviews containing at least one meta-analysis with a binary outcome were numbered randomly by an IT specialist not otherwise involved in the project. Reviews were assessed for eligibility in the random order. The PubMed database, years 2001 and 2002 was searched with the strategy:
(randomi* OR controlled OR blind* OR placebo OR ‘controlled ? trial’) ANDReviews identified in PubMed were assessed for eligibility in the order of publication date.(meta-analysis OR metaanalysis Field: All Fields, Limits: Meta-Analysis)
One author (J.P.) assessed all abstracts of reviews from both sources for eligibility. Those abstracts that did not meet any exclusion criteria were assessed in duplicate by two authors as to whether the conclusion favoured one of the interventions. If they independently agreed that this was the case, then the full text of the review was retrieved and reassessed for eligibility including independent assessments of whether classification of interventions as experimental or conventional was unequivocal. Disagreements led to exclusion.
Retrieval of trials and assessment of methodology
Original articles were retrieved for all the trials in the included meta-analyses. If the methods section in a trial report referred to another report for details, this was also obtained. A student not otherwise involved in the project photocopied the methods sections of the reports of the included trials. Pairs of authors assessed the adequacy of allocation concealment and of double-blinding independently and blinded to the results sections. We used the criteria reported by Schulz et al.4 for classification of allocation concealment:
Trials with ‘adequate concealment’ employed central randomization including pharmacy-controlled randomization (where a pharmacy remote from the clinical ward allocated the treatment); numbered or coded bottles or containers; serially numbered, opaque, sealed envelopes; or the trialists presented other descriptions that implied convincing concealment. Methods were deemed to provide ‘inadequate concealment’ if it was obvious to which treatment the next patient would be allocated (alternation, use of case record numbers, dates of birth, etc.). Trials with ‘unclear concealment’ did not report on allocation concealment or reported an approach that did not clearly fall into one of the other categories.
Trials were categorized as ‘double-blind’ if described as double-blind; or if patients and treatment providers were explicitly reported as blinded. Trials reported to be placebo-controlled without any indication that the treatments might be distinguishable or that investigators might have become unblinded before the onset of the treatment were also categorized as double-blind. ‘Patients and assessor’-blinding was not categorized as double-blinding because lack of blinding of treatment providers might increase the risk that trials with unclear reporting on allocation concealment in reality had inadequate concealment.
Extraction of 2 x 2 data for each meta-analysis
Most of the trial 2 x 2 data necessary for our replication of the individual meta-analyses were extracted from the reviews. When absent in the review, data were sought in the trial reports. If the methods sections of the reviews and trial reports did not allow unequivocal identification of the exact numbers, we contacted the authors of the reviews.
Data analysis and statistics
For the primary analysis we first replicated the meta-analyses using the review authors’ method of analysis while retaining all trials in the analysis. When 2 x 2 data were unobtainable, we used the point estimates and standard errors from the meta-analyses and did an inverse variance analysis11 using a fixed or random effects model depending on the authors’ original method. Next, we redid each meta-analysis, but included only those trials that had reported adequate allocation concealment (primary analysis). Here, and for all later analyses, we had to give up the distinction between unclear and manifestly inadequate concealment because the latter was too rare. We did not correct any of the review authors’ errors of analysis (counting the same control group twice, etc.), as we wished to isolate the impact of requiring reported adequate allocation concealment. We defined the review authors’ conclusions not to be supported by the reduced data if our estimates were not statistically significant at a two-sided 5% significance level.
Loss of statistical support might here be due to loss of data or to differentially biased treatment effects in the discarded vs retained trials. This was investigated among the meta-analyses with a mixture of trials with and without reported adequate concealment in the following way: we assumed that the point estimates obtained in the reduced meta-analyses still had the statistical precision of the unreduced analyses, and calculated whether or not that would restore the original statistical support. Thus, an adjusted z-value was defined as the z-value in the reduced data set x 
[(total number of participants in the original data set)/(total number of participants in the reduced data set)]. If the adjusted z-value was below 1.96 then the loss of statistical significance was interpreted as being caused by a shift in the point estimate rather than loss of statistical power.
For the secondary analysis we used an approach described by Sterne et al.12 Briefly, we estimated the ROR by comparing trials with absence vs presence of reported adequate allocation concealment in a univariate random effects meta-regression in each meta-analysis. To ensure consistency we recalculated the effect estimates where necessary, so that all results were expressed as undesirable events (e.g. presence of symptoms rather than absence of symptoms). Since the odds in the numerator of the odds ratio of each trial were for the experimental treatment, OR<1 represents superiority of the experimental intervention. The ROR estimates were then combined in a random effects generic inverse variance meta-analysis. We used the I2 quantity to report the percentage of variability between meta-analyses that could not be ascribed to chance.13 Reviews for which 2 x 2 data could not be obtained, or where all trials either were with or without reported adequate allocation concealment, did not contribute to this analysis. Corresponding analyses were employed to estimate the impact of absence vs presence of double-blinding on the treatment effect estimate. Analyses were performed using Stata software, version 8 (StataCorp) and RevMan software, version 4.2.3 (Nordic Cochrane Centre, The Cochrane Collaboration: available from http://www.cochrane.org).
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
Characteristics of included meta-analyses and component trials
The selection process for Cochrane Reviews and other reviews is outlined in Figure 1. After two rounds, 181 Cochrane Reviews and 128 other reviews had been assessed, and 38 Cochrane Reviews and 32 other reviews were included. A larger fraction of Cochrane Reviews was excluded because no preference was stated in the conclusion, rate difference 28% (95% CI 14–42%), P = 0.0002. The 38 Cochrane Reviews comprised a total of 202 trials and a median of four trials per meta-analysis (10th to 90th percentile range: 2–12).14–51 The 32 other reviews comprised 297 trials with a median of seven trials included per meta-analysis (10th to 90th percentile range: 4–17).52–84 Two reviews, based on 6 and 17 trials, respectively, had three trials in common. As the overlap was very small and the reviews addressed different outcomes, no data were excluded.
|
Overall, only 82/499 (16%) of trials had adequate allocation concealment, including 51% of participants; 379 trials (76%) had unclear and 38 trials (8%) had inadequate concealment. More trials in Cochrane Reviews reported adequate allocation concealment (25% vs 10%). A larger fraction of trials with adequate allocation concealment also had double-blinding. Trials with adequate allocation concealment and trials with double-blinding tended to be larger, more recent, and include fewer non-English reports than their counterparts (Tables 1 and 2).
|
|
We compared our assessment of adequacy of allocation concealment to that of the Cochrane review authors. When we used the same instructions that Cochrane Review authors are expected to follow, the only extra detail required was for the ‘numbered coded vehicles’-method to be reported with the vehicles as being sequentially administered. Yet, we found that among 202 trials we categorized 92 trials (46%) differently than the review authors; mainly because the review authors did not use the criteria or applied them more laxly. The latter caused an overestimation of adequacy of allocation concealment in 75 trials (36%).
Although all reviews purportedly included randomized trials only, our reassessments revealed that seven trials were not randomized. However, as this is a pragmatic study the reviews containing these trials were not excluded, and the few non-randomized trials were treated in the analyses as having inadequate allocation concealment.
In the meta-analyses that contained a mixture of trials with and without reported adequate concealment, we were able to reproduce the meta-analyses to within two rounding units (e.g. 0.02 when an odds ratio scale was used) with two exceptions, but even here the discrepancy was less than a tenth of the associated standard error.
Conclusions no longer supported
In total, 34 meta-analyses included a mixture of trials with and without reported adequate allocation concealment, whereas 32 meta-analyses exclusively contained trials with unclear or inadequate concealment, and four meta-analyses exclusively contained trials with adequate concealment. Forty-eight of 70 conclusions (69%; 95% CI 56–79%) lost support when only trials with adequate concealment were included.
Among the 34 meta-analyses with a mixture of trials with and without reported adequate concealment, 16 (47%) lost support. P-values and average precision before and after exclusion of trials without reported adequate concealment in these meta-analyses are shown in Table 3. Five of these 16 meta-analyses (31%) were estimated to loose support if the statistical power had been the same, i.e. they lost support because of a shift in the point estimate.
|
Overestimation of treatment benefit
Of the 34 meta-analyses containing trials with as well as without reported adequate concealment, four did not contribute to the analysis because 2 x 2 data were unobtainable and one because it contained only two trials. In the remaining 29 meta-analyses the pooled estimate of the ratio of the treatment effect estimates from trials with unclear or inadequate allocation concealment compared to those from trials with adequate concealment was an ROR of 0.90 (95% CI: 0.81–1.01; P = 0.08, I2 = 0%) (Figure 2). Hence, there was a non-significant trend towards a seemingly more beneficial effect of the experimental treatment in the trials without reported adequate allocation concealment.
|
Among the 20 meta-analyses comprising a mixture of trials without and with double-blinding, the corresponding ROR was similar: ROR 0.94 (95% CI: 0.80–1.10; P = 0.41, I2 = 2.5%) (Figure 3).
|
We explored how our result adds to the current evidence of the impact of lack of reported adequate concealment using a random effects generic inverse variance meta-analysis.4–9 The overall estimate was an ROR of 0.82 (95% CI: 0.71–0.94); P = 0.006, I2 = 84.1%), implying an 18% better outcome (when expressed on the OR scale) in the experimental treatment groups in trials without reported adequate allocation concealment; i.e. an 18% lower failure rate when treatment failures are few (Figure 4). However, the individual estimates were heterogeneous (P < 0.00001). The corresponding estimate for absence vs presence of double-blinding was an ROR of 0.91(95% CI 0.83–1.00; P = 0.05, I2 = 21.2%) (Figure 5).
|
|
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
Two-thirds of the conclusions that favoured one of the interventions based on a meta-analysis lost support if only trials with adequate allocation concealment were included. This was mainly because of loss of statistical power but also because estimates of treatment effects tended to be less beneficial.
Strengths and limitations of this study
We aimed to achieve consistent and valid assessments of allocation concealment and double-blinding by retrieving all the original trial publications and reassessing the adequacy of these components in duplicate and blinded to the results sections. We required a convincing mechanism for allocation concealment to be described in order for a trial report to be classified as having adequate concealment, whereas for double-blinding, a statement that the trial was double-blind was sufficient for it to be classified as such. It could also have been interesting to assess whether a convincing mechanism for double blinding had been employed in the trials described as double-blind; however, trial authors frequently do not provide information on who was blinded by which means.85,3
Most randomized controlled trials have low statistical power and a level of reported bias protection comparable with those included in the random sample of meta-analyses in our study.87–89 Accordingly, our primary finding can probably be generalized to the evidence that supports current health care interventions.
We used P < 0.05 as the cut-off point for deciding whether a preference in a conclusion was supported by the data; more conclusions would have lost support if we had been able to adjust for the trials authors’ multiple testing. However, the level of reporting (i.e. unspecified primary outcomes and uncertain number of tested outcomes) in the reviews did not allow such adjustments.
The estimated impact of adequacy of allocation concealment and double-blinding on the intervention effect was protected against confounding by disease area and type of intervention because it was based on meta-analyses. However, it may have been confounded by other trial characteristics potentially associated with the level of bias susceptibility as shown in Tables 1 and 2. Furthermore, a statistical interaction between allocation concealment and double-blinding may exist. However, the meta-analyses were too few and too small to permit exploration of these possibilities.
Relation of our findings to those of other studies
The range of proportions of health care interventions that are supported by randomized controlled trials is wide. It depends on specialty and varies between 11% and 65%.90 That most conclusions lost support in our study is consistent with the high prevalence of trial reports with unclear or inadequate allocation concealment.91 Furthermore, authors of systematic reviews frequently do not take the assessed level of bias protection into account in the analysis and interpretation of their results.92 Fewer trials in our sample had adequate allocation concealment (25% of trials in Cochrane Reviews and 10% of trials in other reviews) than in other similar studies (36–40%).4–8 This may partly be due to differences in the applied criteria, e.g. Kjaergard et al.6 did not require that envelopes had to be serially numbered or opaque, whereas Schulz et al.4 and Moher et al.5 did. Egger et al.7 relied on the quality assessment by the authors of Cochrane reviews who tended to apply the criteria more laxly than Schulz et al.4, Moher et al.5 and us. Another difference is that Schulz et al.4, Egger et al.7 and Als-Nielsen et al.9 selected meta-analyses with at least one trial with adequate allocation concealment, and Kjaergard et al.6 selected meta-analyses with at least one trial comprising more than 1000 patients. Thus, trials with adequate allocation concealment might be overrepresented in these studies.
We found that 69% of meta-analyses lost statistical significance when trials with unclear or inadequate allocation concealment were excluded, and 47% when only the meta-analyses with a mixture of trials were considered. An estimate corresponding to the latter was reported in another study to be 38%, which might to a larger extent have been due to a shift in the point estimate rather than loss of power, since this study showed a larger impact of allocation concealment on the treatment effect estimate, and had a larger fraction of trials with adequate concealment.7
Our estimate of the impact of unclear or inadequate allocation concealment was less than those reported in the first four similar studies (Figure 4).1 A fifth study by Balk et al.8 found an ROR of 0.95 (0.83–1.09), but it has been questioned because one of the inclusion criteria was statistically significant heterogeneity between the included trials in each meta-analysis,93 which could introduce too much noise to allow detection of the full effect of lack of reported adequate concealment. However, this does not apply to our study or to a sixth study, that found an ROR of 1.02 (0.90–1.16).9 Several explanations for these varying findings are possible. First, confounders may have differentially influenced the results of the studies, e.g. the individual bias protection components might have been correlated with each other to a different extent in the different studies. Whether (and how) this was taken into account varied. Secondly, the apparent impact of absence of a bias protection component might differ according to subgroups, which might be differentially represented in the different studies. For example, the impact of unclear allocation concealment might be less in a cohort where drug trials with double-blinding comprise a large subgroup, because an adequate method for allocation concealment (numbered coded vehicles) is frequently employed in these trials, but often not explicitly described in the trial report.94 Thirdly, the studies used slightly different criteria for adequate allocation concealment and different strategies for statistical analysis.
Implications for research, clinicians and policy makers
Most conclusions favouring an intervention would lose support if trials with unclear or inadequate allocation concealment were excluded from the meta-analysis. This may seem too radical, especially since the bias associated with these trials appears to be smaller and less consistent than previously thought. Furthermore, the remaining trials might still be affected by other sources of bias, for instance selective reporting of significant outcomes.89 Yet, results of meta-analyses should always be accompanied by sensitivity analyses presenting the results with and without trials with unclear or inadequate bias protection. While sensitivity analyses will allow the reader to gauge the possible impact of bias, decisions still have to be made whether or not the investigated interventions should be implemented. To guide such decisions, further research on the size and direction of different types of bias under different circumstances is warranted.
In addition, steps to prevent bias and avoid uncertainty regarding the level of bias protection should be taken. First, the gatekeepers of trial protocols (primarily drug-regulatory authorities and research ethics committees) should insist on description of methods to ensure allocation concealment and sanction only protocols with adequate methods. Secondly, trial protocols should be publicly available to facilitate critical appraisal of trials and thirdly, the CONSORT statement,91 which requires explicit and appropriate reporting on measures taken to protect a trial against bias, should be broadly enforced.
|
Acknowledgements |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
J. Hedegaard is thanked for secretarial support, Rasmus Moustgaard for generating the random list of eligible Cochrane Reviews. J.P., A.H., K.J.J. and P.C.G. are funded by Copenhagen Hospital Corporation, D.G.A. by Cancer Research UK. The funding organizations had no role in any aspect of the study, the manuscript or the decision to publish.
Conflict of interest: None declared.
| KEY MESSAGES Already known on this topic:
What this study adds:
|
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
1 Jüni P, Altman DG, Egger M. Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ (2001) 323:42–46.
2 Devereaux PJ, Manns BJ, Ghali WA, et al. Physician interpretations and textbook definitions of blinding terminology in randomized controlled trials. JAMA (2001) 285:2000–3.
3 Haahr MT, Hrobjartsson A. Who is blinded in randomized clinical trials? A study of 200 trials and a survey of authors. Clin Trials (2006) 3:360–65.
4 Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA (1995) 273:408–12.
5 Moher D, Pham B, Jones A, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet (1998) 352:609–13.[CrossRef][Web of Science][Medline]
6 Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med (2001) 135:982–89.
7 Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess (2003) 7:1–76.
8 Balk EM, Bonis PA, Moskowitz H, et al. Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA (2002) 287:2973–82.
9 Als-Nielsen B, Chen W, Gluud LL, Sierma V, Hilden J, Gluud C. Are trial size and reported methodological quality associated with treatment effects? Program & Abstract Book, 12th Cochrane Colloquium (2004) 102–3.
10 Als-Nielsen B, Gluud LL, Gluud C. Methodological quality and treatment effects in randomised trials: a review of six empirical studies. Program #38; Abstract Book, 12th Cochrane Colloquium (2004) 88–89.
11 Deeks JJ, Altman DG, Bradburn MJ. Statistical methods for examining heterogeneity and combining results from several studies in meta-analysis. In: Systematic Reviews in Health Care: Meta-Analysis in Context.—Egger M, Davey Smith G, Altman DG, eds. (2001) 2nd. London: BMJ Publication Group.
12 Sterne JA, Juni P, Schulz KF, Altman DG, Bartlett C, Egger M. Statistical methods for assessing the influence of study characteristics on treatment effects in ‘meta-epidemiological’ research. Stat Med (2002) 21:1513–24.[CrossRef][Web of Science][Medline]
13 Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ (2003) 327:557–60.
14 Soares-Weiser K, Brezis M, Tur-Kaspa R, Leibovici L. Antibiotic prophylaxis for cirrhotic patients with gastrointestinal bleeding. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
15 Guidugli F, Castro AA, Atallah AN. Antibiotics for preventing leptospirosis. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
16 Liberati A, D’Amico R, Pifferi S, et al. Antibiotics for preventing respiratory tract infections in adults receiving intensive care. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
17 Sheikh A, Hurwitz B, Cave J. Antibiotics versus placebo for acute bacterial conjunctivitis. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
18 Sandercock P, Mielke O, Liu M, Counsell C. Anticoagulants for preventing recurrence following presumed non-cardioembolic ischaemic stroke or transient ischaemic attack. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
19 Bacaltchuk J, Hay P, Trefiglio R. Antidepressants versus psychological treatments and their combination for bulimia nervosa. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
20 Sandercock P, Gubitz G, Foley P, Counsell C. Antiplatelet therapy for acute ischaemic stroke. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
21 Brocklehurst P, Volmink J. Antiretrovirals for reducing the risk of mother-to-child transmission of HIV infection. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
22 Marshall M, Lockwood A. Assertive community treatment for people with severe mental disorders. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
23 King JF, Flenady VJ, Papatsonis DNM, Dekker GA, Carbonne B. Calcium channel blockers for inhibiting preterm labour. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
24 Wong R, Malthaner R. Combined chemotherapy and radiotherapy (without surgery) compared with radiotherapy alone in localized carcinoma of the esophagus. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
25 Filippini G, Brusaferri F, Sibley WA, et al. Corticosteroids or ACTH for acute exacerbations in multiple sclerosis. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
26 Phelps DL, Lakatos L, Watts JL. D-Penicillamine for preventing retinopathy of prematurity in preterm infants. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
27 Rowe BH, Spooner C, Ducharme FM, Bretzlaff JA, Bota GW. Early emergency department treatment of acute asthma with systemic corticosteroids. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
28 Amaragiri SV, Lees TA. Elastic compression stockings for prevention of deep vein thrombosis. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
29 Smith LA, Oldman AD, McQuay HJ, Moore RA. Eletriptan for acute migraine. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
30 Jolliffe JA, Rees K, Taylor RS, Thompson D, Oldridge N, Ebrahim S. Exercise-based rehabilitation for coronary heart disease. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
31 Jeffery GM, Hickey BE, Hider P. Follow-up strategies for patients treated for non-metastatic colorectal cancer. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
32 Farquhar C, Basser R, Hetrick S, Lethaby A, Marjoribanks J. High dose chemotherapy and autologous bone marrow or stem cell transplantation versus conventional chemotherapy for women with metastatic breast cancer. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
33 Myers RP, Regimbeau C, Thevenot T, et al. Interferon for acute hepatitis C. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
34 Acuin J, Smith A, Mackenzie I. Interventions for chronic suppurative otitis media. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
35 Chaisewikul R, Privitera MD, Hutton JL, Marson AG. Levetiracetam add-on for drug-resistant localization related (partial) epilepsy. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
36 Fouque D, Wang P, Laville M, Boissel JP. Low protein diets for chronic renal failure in non diabetic adults. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
37 Durkan A, Hodson E, Willis N, Craig J. Non-corticosteroid treatment for nephrotic syndrome in children. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
38 Wilkinson D, Ramjee G, Tholandi M, Rutherford G. Nonoxynol-9 for preventing vaginal acquisition of HIV infection by women from men. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
39 Whitelaw A. Postnatal phenobarbitone for the prevention of intraventricular hemorrhage in preterm infants. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
40 Smyth A, Walters S. Prophylactic antibiotics for cystic fibrosis. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
41 McGuire W, Clerihew L, Austin N. Prophylactic intravenous antifungal agents to prevent mortality and morbidity in very low birth weight infants. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
42 Askie LM, Henderson-Smart DJ. Restricted versus liberal oxygen exposure for preventing morbidity and mortality in preterm or low birth weight infants. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
43 Miller RG, Mitchell JD, Lyon M, Moore DH. Riluzole for amyotrophic lateral sclerosis (ALS)/motor neuron disease (MND). In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
44 Oldman AD, Smith LA, McQuay HJ, Moore RA. Rizatriptan for acute migraine. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
45 Sinclair JC. Servo-control for maintaining abdominal skin temperature at 36C in low birth weight infants. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
46 Gibson JNA, Grant IC, Waddell G. Surgery for lumbar disc prolapse. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
47 Jette NJ, Marson AG, Hutton JL. Topiramate add-on for drug-resistant partial epilepsy. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
48 McGuire W, McEwan P. Transpyloric versus gastric tube feeding for preterm infants. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
49 Carroli G, Bergel E. Umbilical vein injection for management of retained placenta. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
50 Soares KVS, McGrath JJ. Vitamin E for neuroleptic-induced tardive dyskinesia. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
51 Chadwick DW, Marson AG. Zonisamide add-on for drug-resistant partial epilepsy. In: The Cochrane Library, Issue 2. (2003) Oxford: Update Software.
52 Tournaye H, Verheyen G, Albano C, et al. Intracytoplasmic sperm injection versus in vitro fertilization: a randomized controlled trial and a meta-analysis of the literature. Fertil Steril (2002) 78:1030–37.[CrossRef][Web of Science][Medline]
53 van der Vyver M, Halpern S, Joseph G. Patient-controlled epidural analgesia versus continuous infusion for labour analgesia: a meta-analysis. Br J Anaesth (2002) 89:459–65.
54 Sylvester RJ, van der Meijden AP, Lamm DL. Intravesical bacillus Calmette-Guerin reduces the risk of progression in patients with superficial bladder cancer: a meta-analysis of the published results of randomized clinical trials. J Urol (2002) 168:1964–70.[CrossRef][Web of Science][Medline]
55 Weisman SM, Graham DY. Evaluation of the benefits and risks of low-dose aspirin in the secondary prevention of cardiovascular and cerebrovascular events. Arch Intern Med (2002) 162:2197–202.
56 Fiddian P, Sabin CA, Griffiths PD. Valacyclovir provides optimum acyclovir exposure for prevention of cytomegalovirus and related outcomes after organ transplantation. J Infect Dis (2002) 186(Suppl 1):S110–15.[CrossRef][Web of Science][Medline]
57 Eberhart LH, Morin AM, Kranke P, Geldner G, Wulf H. Transient neurologic symptoms after spinal anesthesia. A quantitative systematic overview (meta-analysis) of randomized controlled studies. Anaesthesist (2002) 51:539–46.[CrossRef][Web of Science][Medline]
58 Eccleston C, Morley S, Williams A, Yorke L, Mastroyannopoulou K. Systematic review of randomised controlled trials of psychological therapy for chronic pain in children and adolescents, with a subset meta-analysis of pain relief. Pain (2002) 99:157–65.[CrossRef][Web of Science][Medline]
59 Akai M, Kawashima N, Kimura T, Hayashi K. Electrical stimulation as an adjunct to spinal fusion: a meta-analysis of controlled clinical trials. Bioelectromagnetics (2002) 23:496–504.[CrossRef][Web of Science][Medline]
60 Grant AM. Open mesh versus non-mesh repair of groin hernia: meta-analysis of randomised trials based on individual patient data. Hernia (2002) 6:130–36.[CrossRef][Medline]
61 Roffi M, Chew DP, Mukherjee D, et al. Platelet glycoprotein IIb/IIIa inhibition in acute coronary syndromes. Gradient of benefit related to the revascularization strategy. Eur Heart J (2002) 23:1441–48.
62 Papadimitropoulos E, Wells G, Shea B, et al. Meta-analyses of therapies for postmenopausal osteoporosis. VIII. Meta-analysis of the efficacy of vitamin D treatment in preventing osteoporosis in postmenopausal women. Endocr Rev (2002) 23:560–69.
63 Cranney A, Tugwell P, Zytaruk N, et al. Meta-analyses of therapies for postmenopausal osteoporosis. VI. Meta-analysis of calcitonin for the treatment of postmenopausal osteoporosis. Endocr Rev (2002) 23:540–51.
64 Cranney A, Tugwell P, Adachi J, et al. Meta-analyses of therapies for postmenopausal osteoporosis. III. Meta-analysis of risedronate for the treatment of postmenopausal osteoporosis. Endocr Rev (2002) 23:517–23.
65 Cranney A, Wells G, Willan A, et al. Meta-analyses of therapies for postmenopausal osteoporosis. II. Meta-analysis of alendronate for the treatment of postmenopausal women. Endocr Rev (2002) 23:508–16.
66 Chang CH, Chen KY, Lai MY, Chan KA. Meta-analysis: ribavirin-induced haemolytic anaemia in patients with chronic hepatitis C. Aliment Pharmacol Ther (2002) 16:1623–32.[CrossRef][Web of Science][Medline]
67 San Miguel R, Guillen F, Cabases JM, Buti M. Meta-analysis: combination therapy with interferon-alpha 2a/2b and ribavirin for patients with chronic hepatitis C previously non-responsive to interferon. Aliment Pharmacol Ther (2002) 16:1611–21.[CrossRef][Web of Science][Medline]
68 Barker FG 2nd. Efficacy of prophylactic antibiotic therapy in spinal surgery: a meta-analysis. Neurosurgery (2002) 51:391–400. discussion 400-1.[CrossRef][Web of Science][Medline]
69 Kern JW, Shoemaker WC. Meta-analysis of hemodynamic optimization in high-risk patients. Crit Care Med (2002) 30:1686–92.[CrossRef][Web of Science][Medline]
70 Edmonds ML, Camargo CA Jr, Pollack CV Jr, Rowe BH. The effectiveness of inhaled corticosteroids in the emergency department treatment of acute asthma: a meta-analysis. Ann Emerg Med (2002) 40:145–54.[CrossRef][Web of Science][Medline]
71 Salas M, Ward A, Caro J. Are proton pump inhibitors the first choice for acute treatment of gastric ulcers? A meta analysis of randomized clinical trials. BMC Gastroenterol (2002) 2:17–23.[CrossRef][Medline]
72 Bow EJ, Laverdiere M, Lussier N, Rotstein C, Cheang MS, Ioannou S. Antifungal prophylaxis for severely neutropenic chemotherapy recipients: a meta analysis of randomized-controlled clinical trials. Cancer (2002) 94:3230–46.[CrossRef][Web of Science][Medline]
73 Haugh M, Helou S, Boissel JP, Cribier BJ. Terbinafine in fungal infections of the nails: a meta-analysis of randomized clinical trials. Br J Dermatol (2002) 147:118–21.[CrossRef][Web of Science][Medline]
74 Crystal E, Connolly SJ, Sleik K, Ginger TJ, Yusuf S. Interventions on prevention of postoperative atrial fibrillation in patients undergoing heart surgery: a meta-analysis. Circulation (2002) 106:75–80.
75 Camma C, Schepis F, Orlando A, et al. Transarterial chemoembolization for unresectable hepatocellular carcinoma: meta-analysis of randomized controlled trials. Radiology (2002) 224:47–54.
76 D'Souza AL, Rajkumar C, Cooke J, Bulpitt CJ. Probiotics in prevention of antibiotic associated diarrhoea: meta-analysis. BMJ (2002) 324:1361–66.
77 Newby LK, Califf RM, White HD, et al. The failure of orally administered glycoprotein IIb/IIIa inhibitors to prevent recurrent cardiac events. Am J Med (2002) 112:647–58.[CrossRef][Web of Science][Medline]
78 The Artemether-Quinine Meta-analysis Study Group. A meta-analysis using individual patient data of trials comparing artemether with quinine in the treatment of severe falciparum malaria. Trans R Soc Trop Med Hyg (2001) 95:637–50.[CrossRef][Web of Science][Medline]
79 Singer MA, Nelson RL. Primary repair of penetrating colon injuries: a systematic review. Dis Colon Rectum (2002) 45:1579–87.[CrossRef][Web of Science][Medline]
80 Turpie AG, Bauer KA, Eriksson BI, Lassen MR. Fondaparinux vs enoxaparin for the prevention of venous thromboembolism in major orthopedic surgery: a meta-analysis of 4 randomized double-blind studies. Arch Intern Med (2002) 162:1833–40.
81 Berghmans T, Paesmans M, Lafitte JJ, Mascaux C, Meert AP, Sculier JP. Role of granulocyte and granulocyte-macrophage colony-stimulating factors in the treatment of small-cell lung cancer: a systematic review of the literature with methodological assessment and meta-analysis. Lung Cancer (2002) 37:115–23.[CrossRef][Web of Science][Medline]
82 Sewnath ME, Karsten TM, Prins MH, Rauws EJ, Obertop H, Gouma DJ. A meta-analysis on the efficacy of preoperative biliary drainage for tumors causing obstructive jaundice. Ann Surg (2002) 236:17–27.[CrossRef][Web of Science][Medline]
83 Chaiyakunapruk N, Veenstra DL, Lipsky BA, Saint S. Chlorhexidine compared with povidone-iodine solution for vascular catheter-site care: a meta-analysis. Ann Intern Med (2002) 136:792–801.
84 Chaiyakunapruk N, Veenstra DL, Lipsky BA, Saint S. Effectiveness of inhaled corticosteroids in the emergency department treatment of acute asthma: a meta-analysis. Ann Emerg Med (2002) 40:145–54.[CrossRef][Web of Science][Medline]
85 Schulz KF, Chalmers I, Altman DG. The landscape and lexicon of blinding in randomized trials. Ann Intern Med (2002) 136:254–9.
86 Kjaergard LL, Gluud C. Funding, disease area, and internal validity of hepatobiliary randomized clinical trials. Am J Gastroenterol (2002) 97:2708–713.[CrossRef][Web of Science][Medline]
87 Mulward S, Gøtzsche PC. Sample size of randomized double-blind trials 1976–1991. Dan Med Bull (1996) 43:96–8.[Web of Science][Medline]
88 Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA (2004) 291:2457–465.
89 Kyriakidi M, Ioannidis JP. Design and quality considerations for randomized controlled trials in systemic sclerosis. Arthritis Rheum (2002) 47:73–81.[CrossRef][Web of Science][Medline]
90 Matzen P. How evidence-based is medicine? A systematic literature review. Ugeskr Laeger (2003) 165:1431–5.[Medline]
91 Altman DG, Schulz KF, Moher D, et al. The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med (2001) 134:663–694.
92 Moja LP, Telaro E, D’Amico R, Moschetti I, Coe L, Liberati A. Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study. BMJ (2005) 330:1053–55.
93 Jüni P, Egger M. Allocation concealment in clinical trials. JAMA (2002) 288:2407–8. author reply 2408–9.[Web of Science][Medline]
94 Pildal J, Chan AW, Hróbjartsson A, Forfang E, Altman DG, Gøtzsche PC. Comparison of descriptions of allocation concealment in trial protocols and the published reports: cohort study. BMJ (2005) 330:1049–53.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  E. Nuesch, S. Trelle, S. Reichenbach, A. W S Rutjes, E. Burgi, M. Scherer, D. G Altman, and P. Juni The effects of excluding patients from the analysis in randomised controlled trials: meta-epidemiological study BMJ, September 7, 2009; 339(sep07_1): b3244 - b3244. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. Liberati, D. G. Altman, J. Tetzlaff, C. Mulrow, P. C. Gotzsche, J. P.A. Ioannidis, M. Clarke, P. J. Devereaux, J. Kleijnen, and D. Moher The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration Ann Intern Med, August 18, 2009; 151(4): W-65 - W-94. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. G Moreno, A. J Sutton, E. H Turner, K. R Abrams, N. J Cooper, T. M Palmer, and A E Ades Novel methods to deal with publication biases: secondary analysis of antidepressant trials in the FDA trial registry database and related journal publications BMJ, August 7, 2009; 339(aug07_1): b2981 - b2981. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. Liberati, D. G Altman, J. Tetzlaff, C. Mulrow, P. C Gotzsche, J. P A Ioannidis, M. Clarke, P J Devereaux, J. Kleijnen, and D. Moher The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration BMJ, July 21, 2009; 339(jul21_1): b2700 - b2700. [Abstract] [Full Text]
|
|
|
|
 |
|
 the Study to Prospectively Evaluate Reamed Intrame Randomized Trial of Reamed and Unreamed Intramedullary Nailing of Tibial Shaft Fractures J. Bone Joint Surg. Am., December 1, 2008; 90(12): 2567 - 2578. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 I. Needleman, D. Moher, D.G. Altman, K.F. Schulz, D.R. Moles, and H. Worthington Improving the Clarity and Transparency of Reporting Health Research: a Shared Obligation and Responsibility Journal of Dental Research, October 1, 2008; 87(10): 894 - 895. [Full Text] [PDF]
|
|
|
|
 |
|
 M. J. London Quo Vadis, Perioperative Beta Blockade? Are You "POISE'd" on the Brink? Anesth. Analg., April 1, 2008; 106(4): 1025 - 1030. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||