International Journal of Epidemiology 2002;31:150-153
© International Epidemiological Association 2002
Special Theme: Systematic Reviews and Meta-Analysis |
Development of a highly sensitive search strategy for the retrieval of reports of controlled trials using PubMed
a Department of Epidemiology, Johns Hopkins University Bloomberg School of Public Health.
b Department of Community Health, Brown University.
Karen A Robinson, Department of Epidemiology, Johns Hopkins University Bloomberg School Public Health, 2024 E. Monument Street, Suite 2–503, Baltimore, MD 1287, USA. E-mail: krobin{at}jhmi.edu
Abstract
Objective To develop, through revision of the Cochrane Collaboration search strategy for OVID-MEDLINE, a highly sensitive search strategy to retrieve reports of controlled trials using PubMed.
Methods The original highly sensitive Cochrane strategy was revised to take into account additional Medical Subject Headings (MeSH) and other terminology as well as the current unique features of PubMed. We compared the retrieval of the revised strategy with that of the original Cochrane strategy before and after translation of the strategies into PubMed format. Finally, we used a gold standard database of reports of controlled trials identified by electronic and hand search of selected journals to test the revised strategy in PubMed format.
Results The revised strategy included a search statement modified for increased precision, and added ‘Cross-over Studies’ as a MeSH term and the term ‘latin square’ as a text word. Compared to the original Cochrane strategy, the revised strategy identified 53 additional reports of controlled trials accessing MEDLINE through OVID. When the revised strategy and original Cochrane strategy were translated into PubMed format, the revised strategy retrieved 90 reports of controlled trials not identified by the original strategy. Finally, the revised strategy in PubMed format retrieved all of the reports of controlled trials in the gold standard database. Ninety-eight per cent of the gold standard reports of controlled trials were retrieved by Phase 1 of the optimal PubMed search strategy.
Conclusions Failure to identify all relevant trials for systematic review could result in bias. We developed a highly sensitive search strategy for the retrieval of reports of controlled trials for use with PubMed that retrieves more relevant citations (greater sensitivity) and fewer non-relevant citations (greater precision) than the original Cochrane search strategy.
Keywords Search strategies, controlled trials, systematic reviews, MEDLINE, PubMed, Cochrane Collaboration
Accepted 5 October 2001
Failure to identify all possibly relevant reports of controlled trials for systematic review could result in bias. As part of the effort to identify all such reports, the Cochrane Collaboration has been using a ‘highly sensitive search strategy’ developed in 1993 for the retrieval of reports of controlled trials from MEDLINE.1
Many reviewers do not have access to fee-based versions of MEDLINE and will increasingly turn to the free web-based searching provided by PubMed as a means of identifying reports of controlled trials. Therefore, we set out to revise the original Cochrane strategy to develop a highly sensitive search strategy for the retrieval of reports of controlled trials from MEDLINE using PubMed.
Methods
The following search services were used: OVID-MEDLINE (Version 4.0), a non-graphical vendor software, and PubMed. Unless otherwise stated, searches included publication years 1966–1998.
Three stages were completed: Stages one and two were completed in 1998 and stage three was completed in 2000.
Stage one: Revision of original Cochrane strategy
Because we hypothesized that use of a longer word stem would result in greater precision, the search statement ‘clin$ adj25 trial$’ from the original Cochrane search strategy was changed to ‘clinic$ adj25 trial$’ in the revised strategy. Next tested was the addition of MeSH term ‘Cross-Over Studies’ to the original Cochrane search strategy. The third revision considered was the addition of ‘latin[tw] AND square[tw]’.
Revisions were made to the original Cochrane search strategy and the retrieval of the original and revised strategies were compared. The number of uniquely retrieved citations was noted and these were reviewed by a trained searcher to identify reports of controlled trials.
Stage two: Translation of original and revised strategy into PubMed format
The original and revised strategies were translated into PubMed format and, as in stage one, the difference in the total number of citations retrieved was noted and a trained searcher reviewed the unique citations to determine the number of reports of controlled trials retrieved.
Stage three: Testing of revised strategy in PubMed format
A gold standard database of reports of controlled trials was created using CENTRAL, the Cochrane Collaboration database that contains electronic and hand searching results. The reports of controlled trials in CENTRAL (Issue 4, 2000) from 11 general US medical journals for 1998 were downloaded into a database (n = 308). PubMed was then searched using the revised strategy and results were downloaded for comparison to the gold standard database.
Results
Stage one: Revision of original Cochrane strategy
When we truncated ‘clinical’ with a longer word stem, 29 fewer citations were retrieved, none of which was classified as a report of a controlled trial. This more precise longer word stem was used in Phase 2 of the revised strategy. Separately adding the term ‘latin[tw] AND square[tw]’ resulted in 153 additional citations, of which 119 (78%) were reports of controlled trials. This term was added to Phase 2 of the revised search strategy. Separately adding the term ‘Cross-Over Studies[mh]’ to the strategy resulted in 227 additional citations but this term was low in precision (4% identified as reports of controlled trials) and so was added to the least precise Phase 3 of the search strategy. Figure 1
shows the revised strategy for OVID-MEDLINE.
|
The retrieval of the revised strategy was then compared to the retrieval of the original Cochrane search strategy. There were no changes made to Phase 1 and thus no unique citations retrieved by either strategy. For Phase 2, the revised strategy retrieved 119 additional reports of controlled trials and did not retrieve 29 non-relevant citations retrieved by the original Cochrane strategy. In Phase 3, the original Cochrane strategy retrieved 74 reports of controlled trials not retrieved by Phase 3 of the revised strategy, however all of these reports were already retrieved by Phase 2 of the revised strategy. Also in Phase 3, the revised strategy identified 10 (14% of unique citations) additional reports of controlled trials. Using all phases, the revised strategy for OVID-MEDLINE retrieved 53 (44% of unique citations) additional reports and no reports of controlled trials were missed.
Stage two: Translation of revised strategy into PubMed format
The original and revised search strategies in PubMed format are shown in Figure 2. In PubMed, the revised strategy retrieved all citations retrieved by the original Cochrane strategy. The revised strategy identified 161 additional citations, 90 (55%) of which were classified as reports of controlled trials (73 additional reports of controlled trials in Phase 2 and 17 in Phase 3).
|
Stage three: Testing of revised strategy in PubMed format
The revised strategy in PubMed format retrieved all of the citations in the gold standard database. Phase 1 alone retrieved 301 (98%) and adding Phase 2 retrieved 307 of the 308 reports of controlled trials in the gold standard database.
Discussion
To avoid bias, it is essential that a systematic review identify and consider all possibly relevant trials. An integral component of a comprehensive search plan is an effective MEDLINE search. Compared to the original Cochrane strategy, the revised strategy retrieved more reports of controlled trials (increased sensitivity), thereby minimizing the potential for bias caused by the omission of possibly relevant trials. Because the revised strategy was also more precise, its use limits the amount of non-relevant citations retrieved and thus lowers the amount of work in reviewing citations for eligibility. The revised strategy in PubMed format is an essential tool for reviewers.
Since its initial description,2 this highly sensitive strategy for the retrieval of reports of controlled trials using PubMed has been used as part of comprehensive search plans for a number of systematic reviews in a variety of fields.3–7 In each of these reviews, the revised strategy in PubMed format was combined with a topic-specific strategy designed to balance sensitivity and precision.
Additional testing of the revised strategy in PubMed format may be warranted. For instance, retrieval may differ for citations published in other years, such as those published in 1975 or earlier when MEDLINE did not include abstracts. Also, the gold standard database was relatively small and limited to US general medical journals for practical reasons. The revised strategy could have different sensitivity and precision for journals published outside of the US or in specific health fields. Citations from such journals may include different terms or be indexed with MeSH differently. Finally, CENTRAL was not analysed in terms of validity and completeness before using it as the gold standard database. Further testing with a verified gold standard database covering additional journals and years, as well as an examination of any additional citations retrieved by the revised strategy, may be useful.
To continue to be an effective and efficient strategy, the revised strategy should be examined periodically to take into account new features available on PubMed, as well as developments in indexing by the National Library of Medicine. For instance, additional guidance on the use of the MeSH term ‘Cross-Over Studies’ has been recently published.8 In addition, the consequences of current features of PubMed, such as the inability to use modifiers, need further examination.
KEY MESSAGES
|
Acknowledgments
Phyllis Lansing and Patricia Hinegardner, Health Sciences and Human Services Library, University of Maryland, Baltimore provided assistance in preliminary strategy development and testing and were co-presenters of the poster at the Cochrane Colloquium.
Notes
Part of this project was completed while the authors were at the Baltimore Cochrane Center, University of Maryland, Baltimore. Preliminary results were presented at the Sixth Cochrane Colloquium, Baltimore, MD in October 1998.
References
1
Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. Br Med J 1994;309(6964):1286–91.
2 Robinson KA, Hinegardner PG, Lansing P. Development of an optimal search strategy for the retrieval of controlled trials using PubMed [abstract]. Proceedings of the 6th International Cochrane Colloquium; 1998 Oct 22–26. Abstr Workshops Sci Sess Int Cochrane Coll 1998;6: 85.
3 Lehmann HP, Andrews JA, Robinson KA, Holloway VL, Goodman SG. Evidence Report on Management of Acne. AHRQ Publication No. 01-E019. Rockville, MD: Agency for Healthcare Research and Quality, 2001.
4 Schein OD, Friedman DS, Fleisher LA et al. Evidence Report on Anesthesia Management During Cataract Surgery. AHRQ Publication No. 00-E015. Rockville, MD: Agency for Healthcare Research and Quality, 2000.
5 Jampel HD, Lubomski LH, Friedman DS et al. Evidence Report on Treatment of Coexisting Cataract and Glaucoma. AHRQ Publication No. 01-E050. Rockville, MD: Agency for Healthcare Research and Quality, 2001.
6 Friedman DS, Bass EB, Lubomski LH et al. Synthesis of the literature on the effectiveness of regional anesthesia for cataract surgery. Ophthalmology 2001;108: 519–29.[CrossRef][Web of Science][Medline]
7 Friedman DS, Bass EB, Lubomski LH et al. The methodologic quality of clinical trials on regional anesthesia for cataract surgery. Ophthalmology 2001;108: 530–41.[CrossRef][Web of Science][Medline]
8 Lefebvre C, Clarke MJ. Identifying randomized trials. In: Egger M, Davey Smith G, Altman DG (eds). Systematic Reviews in Health Care: Meta-Analysis in Context, 2nd Edn. London: BMJ Books, 2001.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  A. T. M. van de Water and C. M. Speksnijder Efficacy of Taping for the Treatment of Plantar Fasciosis: A Systematic Review of Controlled Trials J Am Podiatr Med Assoc, January 1, 2010; 100(1): 41 - 51. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. Bosteels, S. Weyers, P. Puttemans, C. Panayotidis, B. Van Herendael, V. Gomel, B. W.J. Mol, C. Mathieu, and T. D'Hooghe The effectiveness of hysteroscopy in improving pregnancy rates in subfertile women without other gynaecological symptoms: a systematic review Hum. Reprod. Update, January 1, 2010; 16(1): 1 - 11. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. P Verhagen, L. Damen, M. Y Berger, J. Passchier, and B. W Koes Lack of benefit for prophylactic drugs of tension-type headache in adults: a systematic review Fam. Pract., December 22, 2009; (2009) cmp089v1. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. Vavken, J. Holinka, J. D. Rompe, and R. Dorotka Focused Extracorporeal Shock Wave Therapy in Calcifying Tendinitis of the Shoulder: A Meta-Analysis Sports Health: A Multidisciplinary Approach, March 1, 2009; 1(2): 137 - 144. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M A M Pijpers, M M Tabbers, M A Benninga, and M Y Berger Currently recommended treatments of childhood constipation are not evidence based: a systematic literature review on the effect of laxative treatment and dietary measures Arch. Dis. Child., February 1, 2009; 94(2): 117 - 131. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 W. Hauser, K. Bernardy, N. Uceyler, and C. Sommer Treatment of Fibromyalgia Syndrome With Antidepressants: A Meta-analysis JAMA, January 14, 2009; 301(2): 198 - 209. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. S. Wiener, D. C. Wiener, and R. J. Larson Benefits and Risks of Tight Glucose Control in Critically Ill Adults: A Meta-analysis JAMA, August 27, 2008; 300(8): 933 - 944. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 E. Martin-Rendon, S. J. Brunskill, C. J. Hyde, S. J. Stanworth, A. Mathur, and S. M. Watt Autologous bone marrow stem cells to treat acute myocardial infarction: a systematic review Eur. Heart J., August 1, 2008; 29(15): 1807 - 1818. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. Penagos, G. Passalacqua, E. Compalati, C. E. Baena-Cagnani, S. Orozco, A. Pedroza, and G. W. Canonica Metaanalysis of the Efficacy of Sublingual Immunotherapy in the Treatment of Allergic Asthma in Pediatric Patients, 3 to 18 Years of Age Chest, March 1, 2008; 133(3): 599 - 609. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. Esteban, N. D. Ferguson, M. O. Meade, F. Frutos-Vivar, C. Apezteguia, L. Brochard, K. Raymondos, N. Nin, J. Hurtado, V. Tomicic, et al. Evolution of Mechanical Ventilation in Response to Clinical Research Am. J. Respir. Crit. Care Med., January 15, 2008; 177(2): 170 - 177. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. G. P. Roberts, E. Guallar, and A. Rodriguez Efficacy and Safety of Statin Monotherapy in Older Adults: A Meta-Analysis J Gerontol A Biol Sci Med Sci, August 1, 2007; 62(8): 879 - 887. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. Vidal, A. Gafter-Gvili, S. Borok, A. Fraser, L. Leibovici, and M. Paul Efficacy and safety of aminoglycoside monotherapy: systematic review and meta-analysis of randomized controlled trials J. Antimicrob. Chemother., August 1, 2007; 60(2): 247 - 257. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 B. L Woodley, R. J Newsham-West, G D. Baxter, M Kjaer, and M S Koehle Chronic tendinopathy: effectiveness of eccentric exercise * COMMENTARY 1 * COMMENTARY 2 Br. J. Sports Med., April 1, 2007; 41(4): 188 - 198. [Full Text] [PDF]
|
|
|
|
 |
|
 A. S. Gangji, T. Cukierman, H. C. Gerstein, C. H. Goldsmith, and C. M. Clase A Systematic Review and Meta-Analysis of Hypoglycemia and Cardiovascular Events: A comparison of glyburide with other secretagogues and with insulin Diabetes Care, February 1, 2007; 30(2): 389 - 394. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 K. A. McKibbon, N. L. Wilczynski, and R. B. Haynes Developing Optimal Search Strategies for Retrieving Qualitative Studies in PsycINFO Eval Health Prof, December 1, 2006; 29(4): 440 - 454. [Abstract] [PDF]
|
|
|
|
 |
|
 J.-P. Collet, G. Montalescot, M. Le May, M. Borentain, and A. Gershlick Percutaneous Coronary Intervention After Fibrinolysis: A Multiple Meta-Analyses Approach According to the Type of Strategy J. Am. Coll. Cardiol., October 3, 2006; 48(7): 1326 - 1335. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. M Kearney, C. Baigent, J. Godwin, H. Halls, J. R Emberson, and C. Patrono Do selective cyclo-oxygenase-2 inhibitors and traditional non-steroidal anti-inflammatory drugs increase the risk of atherothrombosis? Meta-analysis of randomised trials. BMJ, June 3, 2006; 332(7553): 1302 - 1308. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 H. Szajewska, A. Horvath, and B. Koletzko Effect of n-3 long-chain polyunsaturated fatty acid supplementation of women with low-risk pregnancies on pregnancy outcomes and growth measures at birth: a meta-analysis of randomized controlled trials Am. J. Clinical Nutrition, June 1, 2006; 83(6): 1337 - 1344. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L Damen, J Bruijn, A. Verhagen, M. Berger, J Passchier, and B. Koes Prophylactic Treatment of Migraine in Children. Part 2. A Systematic Review of Pharmacological Trials* Cephalalgia, May 1, 2006; 26(5): 497 - 505. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L Damen, J Bruijn, B. Koes, M. Berger, J Passchier, and A. Verhagen Prophylactic Treatment of Migraine in Children. Part 1. A Systematic Review of non-Pharmacological Trials* Cephalalgia, April 1, 2006; 26(4): 373 - 383. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. A. Alghamdi, B. W. McCrindle, and G. S. Van Arsdell Physiologic Versus Anatomic Repair of Congenitally Corrected Transposition of the Great Arteries: Meta-Analysis of Individual Patient Data Ann. Thorac. Surg., April 1, 2006; 81(4): 1529 - 1535. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. C. McCrory and S. Z. Lewis Methodology and Grading of the Evidence for the Diagnosis and Management of Cough: ACCP Evidence-Based Clinical Practice Guidelines Chest, January 1, 2006; 129(1_suppl): 28S - 32S. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 K A de Waal, W Baerts, and M Offringa Systematic review of the optimal fluid for dilutional exchange transfusion in neonatal polycythaemia Arch. Dis. Child. Fetal Neonatal Ed., January 1, 2006; 91(1): F7 - F10. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M.-L. B Lenssinck, A. C Frijlink, M. Y Berger, S. M. Bierma-Zeinstra, K. Verkerk, and A. P Verhagen Effect of Bracing and Other Conservative Interventions in the Treatment of Idiopathic Scoliosis in Adolescents: A Systematic Review of Clinical Trials Physical Therapy, December 1, 2005; 85(12): 1329 - 1339. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. Damen, J. K.J. Bruijn, A. P. Verhagen, M. Y. Berger, J. Passchier, and B. W Koes Symptomatic Treatment of Migraine in Children: A Systematic Review of Medication Trials Pediatrics, August 1, 2005; 116(2): e295 - e302. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 N. L Wilczynski and R B. Haynes Optimal Search Strategies for Detecting Clinically Sound Prognostic Studies in EMBASE: An Analytic Survey JAMIA, July 1, 2005; 12(4): 481 - 485. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. M.C. Welschen, E. Bloemendal, G. Nijpels, J. M. Dekker, R. J. Heine, W. A.B. Stalman, and L. M. Bouter Self-Monitoring of Blood Glucose in Patients With Type 2 Diabetes Who Are Not Using Insulin: A systematic review Diabetes Care, June 1, 2005; 28(6): 1510 - 1517. [Full Text] [PDF]
|
|
|
|
|
|
R B. Haynes, K A. McKibbon, N. L Wilczynski, S. D Walter, S. R Werre, and for the Hedges Team Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytical survey BMJ, May 21, 2005; 330(7501): 1179. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. O. Friedrich, N. Adhikari, M. S. Herridge, and J. Beyene Meta-Analysis: Low-Dose Dopamine Increases Urine Output but Does Not Prevent Renal Dysfunction or Death Ann Intern Med, April 5, 2005; 142(7): 510 - 524. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A.-W. Chan and D. G Altman Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors BMJ, April 2, 2005; 330(7494): 753. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 G. G. Biondi-Zoccai, P. Agostoni, A. Abbate, L. Testa, and F. Burzotta A simple hint to improve Robinson and Dickersin's highly sensitive PubMed search strategy for controlled clinical trials Int. J. Epidemiol., February 1, 2005; 34(1): 224 - 225. [Full Text] [PDF]
|
|
|
|
|
|
K. A Robinson Authors' response Int. J. Epidemiol., February 1, 2005; 34(1): 225 - 225. [Full Text] [PDF]
|
|
|
|
 |
|
 S. S. Gorin and J. E. Heck Meta-Analysis of the Efficacy of Tobacco Counseling by Health Care Providers Cancer Epidemiol. Biomarkers Prev., December 1, 2004; 13(12): 2012 - 2022. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. J. Pronovost, D. C. Angus, T. Dorman, K. A. Robinson, T. T. Dremsizov, and T. L. Young Physician Staffing Patterns and Clinical Outcomes in Critically Ill Patients: A Systematic Review JAMA, November 6, 2002; 288(17): 2151 - 2162. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Egger, S. Ebrahim, and G. D. Smith Where now for meta-analysis? Int. J. Epidemiol., February 1, 2002; 31(1): 1 - 5. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||