IJE Advance Access originally published online on February 14, 2007
International Journal of Epidemiology 2007 36(2):422-430; doi:10.1093/ije/dym001
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Evidence from crossover trials: empirical evaluation and comparison against parallel arm trials
1 Clinical Trials and Evidence-Based Medicine Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece.
2 Institute for Clinical Research and Health Policy Studies, Department of Medicine, Tufts University School of Medicine, Boston, MA, USA.
* Corresponding author. Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45110, Greece. E-mail: jioannid{at}cc.uoi.gr
 |
Abstract |
|---|
 Top Abstract MethodsResultsDiscussionReferences |
|---|
Background We aimed to evaluate empirically how crossover trial results are analysed in meta-analyses of randomized evidence and whether their results agree with parallel arm studies on the same questions.
Methods We used a systematic sample of Cochrane meta-analyses including crossover trials. We evaluated the methods of analysis for crossover results and compared the concordance of the estimated effect sizes in crossover vs parallel arm trials.
Results Of 334 screened reviews, 62 had crossover trials. Of those, 33 meta-analyses performed quantitative syntheses involving two-arm two-period crossover trials. There was large variability on how these trials were analysed; only one of the 33 meta-analyses stated that they used the data from both the first and second period with an appropriate paired approach. Nine meta-analyses used the first period data only and 14 gave no information at all on what they had done. Twenty-eight meta-analyses had both crossover (n = 137, sample size n = 7162) and parallel arm (n = 132, sample size n = 11 398) trials. Effect sizes correlated well with the two types of designs (
= 0.72). Differences on whether the summary effect had a P < 0.05 or not were common due to limited sample sizes. The summary relative odds ratio for parallel arm vs crossover designs for favourable outcomes was 0.87 (95% CI, 0.74–1.02).
Conclusions Crossover designs may contribute evidence in a fifth of systematic reviews, but few meta-analyses make use of their full data. The results of crossover trials tend to agree with those of parallel arm trials, although there was a trend for more conservative treatment effect estimates in parallel arm trials.
Keywords Crossover studies, parallel studies, meta-analysis, empirical evaluation
Accepted 20 December 2006
Randomized trials may be conducted with either a parallel arm design or variants of crossover designs. Parallel arm trials are far more popular in the literature. They are simpler to design and analyse and are generally feasible. We also have well-established and accepted methods for their meta-analysis.1 Crossover trials are less common, but not rare. An appraisal of issue 1, 2001 of the Cochrane Database of Systematic Reviews found that 184 out of 1000 reviews referred to crossover trials and 151 included data from crossover designs.2 A Medline search of crossover studies published between 2000 and 2003 in five major medical journals found 40 crossover trials.3
Crossover trials are difficult or inappropriate to apply for many research questions. Moreover, they represent relatively complex designs. In a two-treatment crossover study each patient may be perceived as his own control: ideally one might simply compare the effect of treatment of the first period to that of the second period. However, treatment-period interaction and carry over effects jeopardize the validity of such simple inferences. Dealing with these problems poses analytical challenges.4,5 There are several different approaches for the analysis of data from crossover trials. Diverse options apply to the primary analysis of the data; the problem is further accentuated in meta-analyses, where information to perform some types of analyses may be missing in published reports. One may use the first period results only, but this will waste all the data after crossover and will drastically diminish the power to detect any intervention effects. Alternatively, one may use the first and second period data as two different parallel studies, but this approach is biased, because it ignores treatment-period interaction. Finally, one may use outcomes from paired analyses. However, this information is often missing from published reports.3 Then one can try to approximate paired analysis by using imputed correlation coefficients from previous studies with sufficient amount of data.6
This lack of unanimity and use of suboptimal methods may introduce bias in the analysis of crossover studies. We considered that it would be useful to obtain empirical evidence on the handling of the data and the influence of data from crossover trials in meta-analyses of randomized evidence. We tried to answer the following questions. What is the relative contribution of crossover trials to randomized evidence for diverse medical interventions? How are crossover trials handled in current meta-analyses? Do the results of crossover trials agree with those of parallel studies in the same meta-analysis, or do they find strong or weaker treatment effects? To answer these questions we analysed data from a systematic sample of meta-analyses.
|
Methods |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
Selection of meta-analyses, crossover studies and outcomes
We perused every fifth review out of the 1669 included in the Cochrane Library Issue 2, 2003 (in the order listed in the Cochrane Database) to identify reviews which included crossover randomized trials in quantitative syntheses (meta-analyses). The spacing of the evaluated reviews was chosen arbitrarily. We excluded all non-randomized and pseudo-randomized trials; and selected only two treatments, two period crossover studies (AB/BA design). Comparisons of more than two treatments are rare and their interpretation is rather complicated anyhow. At a second step, we focused on meta-analyses with at least one parallel and at least one crossover trial so as to evaluate the concordance of results obtained with these two designs.
We examined both binary and continuous outcomes. In every systematic review we selected the primary outcome as stated by the reviewers. If a primary outcome was not clearly identified, we selected what we deemed to be the most clinically important outcome; if this was also not clear, we selected the one with the larger number of studies; if there were ties, we selected the one with the larger number of randomized patients; and then the meta-analysis with fewer zero cells in the 2 x 2 tables of the constituent studies (for binary outcomes). Whenever there were two or more independent meta-analyses from the same systematic review (e.g. on different, non-overlapping types of patients), we considered them as separate meta-analyses. Two investigators (D.L. and T.A.T.) independently selected eligible meta-analyses and crossover studies. Disagreements were referred to the third investigator (J.P.A.I.).
Data extraction on primary trials
For every eligible meta-analysis we extracted information on the specific comparison, and outcome, and the number of parallel and crossover trials. For each trial we recorded the first author and year of publication, the design (parallel vs. crossover), the way the meta-analysts utilized data from crossover trials [first period only, second period only, ‘combined’ data (and if so how combined), or no comment/unclear], the approach of the meta-analysis to the carry-over effect in each crossover trial, and information necessary for the calculation of the effect sizes (2 x 2 tables for binary outcomes, and number of patients, mean response and SD of the responses in each arm for continuous outcomes).
Statistical methods
Meta-analyses
Quantitative analyses were performed in meta-analyses that included both crossover and parallel studies. For binary outcomes we used the odds ratio (OR) as the metric of choice. Continuous outcomes were quantified using standardized mean differences (SMDs), as expressed by Hedges's g metric. SMDs express the magnitude of the treatment effects relative to the within group standard deviations and can be used to synthesize studies that have quantified treatment effects in different scales.7
All comparisons were coined so that the experimental treatment is compared vs the standard/older treatment or no treatment (or placebo) for a good, favourable outcome. For example, if the outcome used by the original trials and their meta-analysis was therapeutic success, we kept the data as is; however if it was therapeutic failure, then we took the complementary counts, i.e. we inversed the OR. If the outcome was measured on a continuous scale, where health was better with higher scores, we kept the data as is; if the scale meant worse health with higher scores, then we inversed the sign of the difference between arms in the calculation of the SMD. Therefore an OR > 1 or SMD > 0 suggests that the experimental treatment fares better than the comparator.
For every topic we calculated the summary effect size of crossover and parallel studies with both fixed and random effects inverse variance syntheses.8,9 Fixed effects analyses assume that the true effect of treatment is the same across synthesized studies, while random effects analyses allow for between-study heterogeneity (dissimilarity) and incorporate it in the calculations. For binary outcomes, if any trial arms in the analysed studies had zero observed events we added 0.5 to all cells of the pertinent two by two table.
We tested for between-study heterogeneity using the 
2 distributed Q statistic and quantified its extent with the I2 statistic.10 I2 ranges between 0% and 100%, and values above 75% imply very high heterogeneity.
Assessment of concordance
We evaluated the agreement between parallel and crossover designs. Concordance was measured by assessing the Spearman correlation coefficient for the effect size estimates between the two designs across all meta-analyses; how often the two designs showed effects in the same direction; how often the two designs agreed with respect to having P < 0.05 for the summary effect; and whether the effect size estimates of the two designs differed beyond chance.11 For the latter assessment, we calculated the relative odds ratio (ROR) by dividing the summary OR in parallel trials by the summary OR in crossover trials and also estimated the variance and 95% confidence intervals (CIs) of the ROR.12 We transformed the weighted SMDs of the continuous outcomes to ORs13 using the following formulae: 
with 
. When ROR > 1, parallel trials yield larger OR estimates than crossover trials, i.e. the parallel trials give a more favourable picture for the experimental intervention than the crossover trials, and vice versa when ROR < 1.
Finally, we assessed whether there was a systematic difference in the estimated summary effects between the two designs across all topics. The summary ROR was calculated across all topics with random effects syntheses and heterogeneity in the ROR estimates across meta-analyses was measured with the Q and I2 statistics.
Subgroup and sensitivity analyses
The main analyses included all topics, regardless of the way data from the crossover studies were reported to be analysed. In separate analyses we estimated the summary ROR across the two designs considering only the crossover studies where only data from the first period were used; and estimated the summary ROR considering all other crossover studies. We also performed a sensitivity analyses by excluding the largest meta-analysis that included almost half of the crossover studies in our empirical evaluation.
Analyses were conducted in Intercooled Stata 8.2 and in SPSS 13.0. All P-values are two tailed. Hypothesis-testing for between-study heterogeneity is done at 
= 0.10.14
|
Results |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
Eligible meta-analyses
Figure 1 shows the flow of the systematic screening of the Cochrane Library. We screened 334 systematic reviews and excluded 272 of them, as they did not have any crossover trials. Of the remaining 62 systematic reviews, 26 were finally selected.15–40 These pertained to 28 independent meta-analyses, because one review contained eligible data for three different meta-analyses (on three different types of study populations).
|
Analysis of data from crossover trials
A variety of approaches were used to incorporate the results of crossover trials in the 28 meta-analyses that included both types of designs and 12 out of these 28 meta-analyses did not mention at all their approach towards crossover trials results. Nine meta-analyses used only the first period results of crossover studies. Three meta-analyses combined data from first and second period, but only one of them described the method of combination, which was the paired analysis proposed by the Cochrane group. One meta-analysis used data from the second period only. Finally, three meta-analyses did not have a consistent approach towards analysis of crossover trials, using first period data in some trials and data from both periods in others. Only four of the 28 meta-analyses discussed the problem of carry over effect: three of them assumed that there was no carry-over effect in the analysed crossover studies and one relied on the carry over test results of crossover studies as reported in their initial publication.
In the five meta-analyses that had included only crossover trials, two meta-analyses did not mention at all what they had done, one approximated a paired analysis as described in the Reviewers’ Handbook of the Cochrane Collaboration, one used data from the second period, and one combined data from both periods without clarifying precisely the analysis methods. Only one of these meta-analyses stated that it assumed that there was no carry-over effect to the analysed crossover studies, while the remaining four meta-analyses did not comment at all on the potential problem of carry-over effect.
Characteristics of crossover and parallel arm trials
In the 28 meta-analyses that included both types of designs (Table 1), there were 142 crossover trials. We excluded five crossover trials from two meta-analyses (four trials analysed more than two groups and one was a partial crossover trial where many patients continued the first period's treatment in the second period). Finally, we analysed 137 crossover studies with a total sample size of 7162 included in the analyses and 132 parallel studies with 11 398 study subjects. In one topic (effects of low sodium diet vs high sodium diet on blood pressure, and biochemical indices29) three different outcomes were analysed that included 91 crossover analyses with a total sample size of 4518 and 30 parallel studies with 4467 subjects. These trials represented a large proportion of the overall database and thus we performed sensitivity analyses excluding this topic.
|
A wide variety of diseases and interventions were represented. The most common targeted categories of diseases were joint diseases (n = 7, of which n = 5 for rheumatoid arthritis) and respiratory (n = 6, of which n = 3 for chronic obstructive pulmonary disease). Parallel trials had a larger analysed sample size than crossover trials (median 46 vs 36, P = 0.006). The crossover trials contributed anywhere between 5% and 79% of the total sample size of analysed subjects across these meta-analyses. In 16 of 28 meta-analyses, the total sample size in the parallel trials was larger than the sample size analysed in the crossover trials. There was no difference in the year of publication of crossover vs parallel trials (median 1990 vs 1991, P = 0.36). Of the 28 meta-analyses, 18 used continuous outcomes (Table 1).
Treatment effects and between-study heterogeneity
Summary SMDs expressed by Hedges's g for continuous outcomes and summary OR for binary outcomes and I2 for the extent of heterogeneity are shown in Table 2. These data refer to all studies and separately to crossover and parallel studies. Between-study heterogeneity did not seem to be clearly explained by differences between crossover and parallel trials. Only one meta-analysis found very-large between-study heterogeneity overall; one found very large between-study heterogeneity when separate analyses were performed for crossover trials; and two when separate analyses were performed for parallel arm trials.
|
Concordance between crossover and parallel trial results
The summary effect sizes for crossover trials were highly correlated to the summary effects in parallel trials across all 28 meta-analyses (Spearman correlation coefficient 0.72, P < 0.001). The results were similar, but based on more sparse data, separately for meta-analyses of continuous (
= 0.68, P = 0.002) and binary ( = 0.68, P = 0.030) outcomes. In 23 of the 28 meta-analyses, the point summary estimates were in the same direction with both designs (15/18 continuous outcomes, 6/8 binary outcomes). When the overall results had P < 0.05, the two types of designs always found estimates of effects in the same direction.
In six meta-analyses, evidence from both designs gave P < 0.05 in the summary results, and in 12 meta-analyses both designs gave P's 
0.05. However, in 10 meta-analyses the two designs differed in the presence or not of P < 0.05 (in seven only the crossover data resulted in P < 0.05, in 3 only the parallel trials data resulted in P < 0.05). Overall the kappa coefficient was 0.27 (95% CI, –0.07 to 0.62).
ROR analyses
ROR analyses showed that the difference in the effect size observed in the two types of designs was beyond chance in only one of the 28 meta-analyses (Figure 2). This meta-analysis evaluated distal loop diuretics in preterm infants with chronic lung disease: there was only one small parallel trial (n = 21 infants) and one even smaller crossover trial [n = 10 infants, counted twice in the analysis (for the two periods)] that found opposite results. The crossover trial found even formally statistically results, despite its small sample size.
|
There was considerable variability in the point estimates of the ROR per meta-analysis and 12 of the ROR point estimates were either larger than two or smaller than 0.5 (Figure 2). However, the large majority of RORs had wide CIs, because usually the amount of evidence was not extensive with either or both designs. Overall, there was no between-meta-analysis heterogeneity in the ROR estimates (P = 0.46, I2 = 0%). The summary ROR was 0.87 and its 95% CI marginally encompassed 1.00 (ROR = 0.87, 95% CI 0.74–1.02), suggesting a trend for more conservative treatment effects in parallel arm trials than in crossover trials.
Exclusion of the largest meta-analysis yielded similar estimates, with a summary ROR of 0.88. The summary ROR estimates were similar in meta-analyses that clearly specified that they had used consistently only the first period data (ROR = 0.87) and the other meta-analyses (ROR = 0.87). The ROR estimates were 0.94 for binary and 0.86 for continuous data, respectively and these did not differ beyond chance (P = 0.74). Finally, the ROR was still 0.91 (95% CI 0.73–1.13) when limited to meta-analyses where the analysed sample size of the crossover designs was smaller than the sample size of the parallel arm trials (Table 3).
|
|
Discussion |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
Our empirical evaluation shows that crossover designs may contribute evidence on the effectiveness of medical interventions in about a fifth of systematic reviews. We found that their results correlate with those of parallel arm trials in broad terms. However, if anything, there was a trend for more conservative treatment effect estimates in parallel arm trials. This trend should be seen with caution. Perhaps more importantly, the majority of meta-analyses do not mention at all their approach towards the carry-over effect problem and none of them proceeded to reanalysis of data from all crossover periods with testing for carry-over effect. Systematic reviewers usually do not state how exactly they have handled the results of such studies, while a common practice is to use only the first period results, thus wasting the second period information.
For some disciplines, such as rheumatology, respiratory medicine and dermatology, crossover trials may be encountered quite commonly in the literature. This has been demonstrated also in empirical evaluations of large samples of trials in these disciplines.41–44 Sometimes medical interventions may have been evaluated only with crossover trials. It is well appreciated that crossover designs should not be performed for interventions that have substantial carry-over effects. One might reasonably question whether this prerequisite is fulfilled for several of the topics in Table 1, like for example the use of clomiphene citrate vs placebo for unexplained subfertility. Therefore, the caveats associated with the design and conduct of these designs should be better understood. This applies both to the design and conduct of the original trials and to their incorporation into systematic reviews and meta-analyses.
Although treatment effects correlated well between parallel arm and crossover trials, there was a trend for crossover trials to give more favourable results for the experimental intervention. The trend could be due to chance. Alternatively, it may reflect the fact that crossover trials tended to be smaller and it is common for small trials to give more impressive estimates of treatment effects than larger trials on the same topic.45 However, the number of trials per meta-analysis was too small to allow examining small study effects within each meta-analysis in a meaningful way. A second possibility is that crossover trials may have shorter follow-up than parallel trials and sometimes treatment effects may wane over time with longer follow-up.46 However, follow-up data were not available in sufficient detail to allow us to explore this explanation further. Overall, the difference between the two designs, even if truly present, does not seem large enough that it would suggest that one type of design in particular yields consistently biased results.
Some other caveats should be discussed. The number of meta-analyses that we included in the ROR estimations is not large, so there was considerable accompanying uncertainty in the summary ROR estimates. Moreover, for the large majority of the meta-analyses included here, the amount of evidence was limited both for parallel and crossover studies. This is not atypical of randomized evidence in general in the medical sciences and it leaves considerable uncertainty about the credibility of the results47,48 obtained with either study design. Finally, we only examined meta-analyses published in the Cochrane Library. This may be an advantage, as these meta-analyses follow specific instructions and standards,43 but also a disadvantage, as the RevMan statistical package used in the Cochrane reviews cannot easily analyse crossover trials data.49 Surprisingly, only one of the analysed meta-analyses in our study used the approach proposed by Reviewers’ Handbook of the Cochrane Collaboration for incorporation of data of crossover studies in meta-analyses. It is difficult to extrapolate these findings to meta-analyses that have been published in scientific journals. Non-Cochrane meta-analyses may be quite differently reported compared with Cochrane reviews, partly because of space limitations.
It would be inappropriate to consider the results of parallel arm trials as the perfect gold standard against which the crossover trials are judged—or vice versa. Thus agreement between the two study designs does not guarantee that the estimates are unbiased. In particular for small trials, there may be room for biased results with either type of designs. In all, our empirical evaluation highlights the need for better understanding and improved use of crossover designs and their data.
Conflict of interest: None declared.
KEY MESSAGES
|
|
References |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
1 Deeks JJ, Altman DG, Bradburn MJ. Statistical methods for examining heterogeneity and combining results from several studies in meta-analysis. In: Systematic Reviews in Health Care. Meta-Analysis in Context—Egger M, Davey Smith G, Altman DG, eds. (2001) 2nd edn. London: BMJ Books.
2 Elbourne DR, Altman DG, Higgins JPT, Curtin F, Worthington HV, Vail A. Meta-analyses involving cross-over trials: methodological issues. Int J Epidemiol (2002) 31:140–49.
3 Garcia R, Benet M, Arnau C, Cobo E. Efficiency of the crossover design: an empirical estimation. Stat Med (2004) 23:3773–80.[CrossRef][Web of Science][Medline]
4 John JA, Russell KG, Whitaker D. Crossover: an algorithm for the construction of efficient crossover designs. Stat Med (2004) 23:2645–58.[CrossRef][Web of Science][Medline]
5 Senn S. Misunderstandings regarding clinical crossover trials. Stat Med (2005) 24:3675–78.[CrossRef][Web of Science][Medline]
6 Cochrane Collaboration. Reviewers’ Handbook. 8.11.3.
7 Rosenthal R. Parametric measures of effect size. In: The Handbook of Research Synthesis—Cooper H, Hedges LV, eds. (1994) New York: Russell Sage Foundation.
8 DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled Clin Trials (1986) 7:177–88.[CrossRef][Web of Science][Medline]
9 Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F. Methods for Meta-Analysis in Medical Research (2004) London: Wiley.
10 Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med (2002) 21:1539–58.[CrossRef][Web of Science][Medline]
11 Ioannidis JPA, Cappeleri JC, Lau J. Issues in comparisons of meta-analyses and large trials. JAMA (1998) 279:1089–93.
12 Sterne JA, Juni P, Schulz KF, Altman DG, Bartlett C, Egger M. Statistical methods for assessing the influence of study characteristics on treatment effects in ‘meta-epidemiological’ research. Stat Med (2002) 21:1513–24.[CrossRef][Web of Science][Medline]
13 Hasselblad V, Hedges LV. Meta-analysis of screening and diagnostic tests. Psychol Bull (1995) 117:167–78.[CrossRef][Web of Science][Medline]
14 Fleiss JL. Analysis of data from multiclinic trials. Controlled Clin Trials (1986) 7:267–75.[CrossRef][Web of Science][Medline]
15 Gotzsche PC, Johansen HK. Short-term low dose corticosteroids vs placebo and non-steroidal anti-inflammatory drugs in rheumatoid arthritis (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
16 Suarez-Almazor ME, Belseck E, Shea B, Wells G, Tugwell P. Methotrexate for treating rheumatoid arthritis (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
17 Ferreira IM, Brooks D, Lacasse Y, Goldstein RS, White J. Nutritional supplementation for stable chronic obstructive pulmonary disease (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
18 Pinelli J, Symington A. Non nutritive sucking for promoting physiologic stability and nutrition in preterm infants (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
19 White J, Cates C, Wright J. Continuous positive airways pressure for obstructive sleep apnoea (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
20 Poustie VJ, Rutherford P. Dietary interventions for phenylketonuria (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
21 Suarez-Almazor ME, Spooner C, Belseck E. Azathioprine for treating rheumatoid arthritis (Cochrane review). In: The Cochrane Librar (2003) (Issue 2). Oxford: Update Software.
22 Brion LP, Primhak RA, Ambrosio-Perez I. Diuretics acting on the distal renal tubule for preterm infants with (or developing) chronic lung disease (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
23 Lemyre B, Davis PG, De Paoli AG. Nasal intermittent positive pressure ventilation (NIPPV) versus nasal continuous positive airway pressure (NCPAP) for apnea of prematurity (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
24 Lonergan E, Luxenberg J, Colford J. Haloperidol for agitation in dementia (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
25 Jones A, Fay JK, Burr M, Stone M, Hood K, Roberts G. Inhaled corticosteroid effects on bone metabolism in asthma and mild chronic obstructive pulmonary disease (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
26 Green S, Buchbinder R, Barnsley L, Hall S, White M, Smidt N, Assenbelft W. Non-steroidal anti-inflammatory drugs (NSAIDS) for treating lateral elbow pain in adults (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
27 Riemsma RP, Kirwan JR, Taal E, Rasker JJ. Patient education for adults with rheumatoid arthritis (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
28 McCrory DC, Brown CD. Anti-cholinergic bronchodilators versus beta2-sympathomimetic agents for acute exacerbations of chronic obstructive pulmonary disease (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
29 Jurgens G, Graudal NA. Effects of low sodium diet versus high sodium diet on blood pressure, renin, aldosterone, catecholamines, cholesterols and triglyceride (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
30 Toweed TE, Judd MJ, Hochberg MC, Wells G. Acetaminophen for osteoarthritis (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
31 Hughes E, Collins J, Vandekerckhove P. Clomiphene citrate for unexplained subfertility in women (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
32 Cheine M, Ahonen J, Wahlbeck K. Beta-blocker supplementation of standard drug treatment for schizophrenia (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
33 Clarke CE, Speller JM. Pergolide versus bromocriptine for levodopa-induced complications in Parkinson's disease (Coharane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
34 Ortiz Z, Shea B, Suarez-Almazor ME, Moher D, Wells G, Tugwell P. Folic acid and folinic acid for reducing side effects in patients receiving methotrexate for rheumatoid arthritis (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
35 Higgins JPT, Flicker L. Lecithin for dementia and cognitive impairment (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
36 Wiffen P, Collins S, McQuay H, Carroll D, Jadad A, Moore A. Anticonvulsant drugs for acute and chronic pain (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
37 Melchart D, Linde K, Fischer P, et al. Acupuncture for idiopathic headache (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
38 Sultana A, Reilly J, Fenton M. Thioridazine for schizophrenia (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
39 Proctor ML, Smith CA, Farquhar CM, Stones RW. Transcutaneous electrical nerve stimulation and acupuncture for primary dysmenorrhoea (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
40 Hood WB Jr, Dans AL, Guyatt GH, Jaeschke R, McMurray JJV. Digitalis for treatment of congestive heart failure in patients in sinus rhythm (Cochrane review). In: The Cochrane Library (2003) (Issue 2). Oxford: Update Software.
41 Karassa FB, Tatsioni A, Ioannidis JP. Design, quality, and bias in randomized controlled trials of systemic lupus erythematosus. J Rheumatol (2003) 30:979–84.
42 Kyriakidi M, Ioannidis JPA. Design and quality considerations for randomized controlled trials in systemic sclerosis. Arthritis Rheum (2002) 47:73–81.[CrossRef][Web of Science][Medline]
43 Jadad AR, Moher M, Browman GP, et al. Systematic reviews and meta-analyses on treatment of asthma: critical evaluation. Br Med J (2000) 320:537–40.
44 Van Coevorden AM, Coenraads PJ, Svensson A, et al. European Dermato-Epidemiology Network (Eden). Overview of studies of treatments for hand eczema-the EDEN hand eczema survey. Br J Dermatol (2004) 151:446–51.[CrossRef][Web of Science][Medline]
45 Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. Br Med J (1997) 315:629–34.
46 Ioannidis JP, Cappelleri JC, Sacks HS, Lau J. The relationship between study design, results, and reporting of randomized clinical trials of HIV infection. Control Clin Trials (1997) 18:431–44.[CrossRef][Web of Science][Medline]
47 Sterne JA, Davey Smith G. Sifting the evidence-what's wrong with significance tests? Br Med J (2001) 322:226–31.
48 Ioannidis JP. Why most published research findings are false. PLoS Med (2005) 2:e124.[CrossRef][Medline]
49 Senn S. The quality of systematic reviews. Review is biased. Br Med J (2000) 321:297.[CrossRef][Web of Science][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||