IJE Advance Access originally published online on April 30, 2007
International Journal of Epidemiology 2007 36(3):666-676; doi:10.1093/ije/dym018
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography
1Primary Care Genetics, General Practice and Primary Care Research Unit, University of Cambridge and Public Health Genetics Unit, Cambridge,
2Public Health Genetics Unit, Cambridge.
3MRC Biostatistics Unit, Cambridge and Public Health Genetics Unit, Cambridge, UK.
4Present address: PBSE, Hoffman-La Roche, Basel, Switzerland.
*Corresponding author and guarantor. Strangeways Research Labs, Worts Causeway, Cambridge CB1 8RN, UK. E-mail: simon.sanderson{at}srl.cam.ac.uk
 |
Abstract |
|---|
Background Assessing quality and susceptibility to bias is essential when interpreting primary research and conducting systematic reviews and meta-analyses. Tools for assessing quality in clinical trials are well-described but much less attention has been given to similar tools for observational epidemiological studies.
Methods Tools were identified from a search of three electronic databases, bibliographies and an Internet search using Google®. Two reviewers extracted data using a pre-piloted extraction form and strict inclusion criteria. Tool content was evaluated for domains potentially related to bias and was informed by the STROBE guidelines for reporting observational epidemiological studies.
Results A total of 86 tools were reviewed, comprising 41 simple checklists, 12 checklists with additional summary judgements and 33 scales. The number of items ranged from 3 to 36 (mean 13.7). One-third of tools were designed for single use in a specific review and one-third for critical appraisal. Half of the tools provided development details, although most were proposed for future use in other contexts. Most tools included items for selection methods (92%), measurement of study variables (86%), design-specific sources of bias (86%), control of confounding (78%) and use of statistics (78%); only 4% addressed conflict of interest. The distribution and weighting of domains across tools was variable and inconsistent.
Conclusion A number of useful assessment tools have been identified by this report. Tools should be rigorously developed, evidence-based, valid, reliable and easy to use. There is a need to agree on critical elements for assessing susceptibility to bias in observational epidemiology and to develop appropriate evaluation tools.
Keywords Observational studies, epidemiological studies, quality, bias, checklist, scales
Accepted 29 January 2007
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  S. L. Sheridan, A. J. Viera, M. J. Krantz, C. L. Ice, L. E. Steinman, K. E. Peters, L. A. Kopin, D. Lungelow, and for the Cardiovascular Health Intervention Researc The Effect of Giving Global Coronary Risk Information to Adults: A Systematic Review Arch Intern Med, February 8, 2010; 170(3): 230 - 239. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. L. Westlake, A. N. Colebatch, J. Baird, P. Kiely, M. Quinn, E. Choy, A. J. K. Ostor, and C. J. Edwards The effect of methotrexate on cardiovascular disease in patients with rheumatoid arthritis: a systematic literature review Rheumatology, February 1, 2010; 49(2): 295 - 307. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. Liberati, D. G Altman, J. Tetzlaff, C. Mulrow, P. C Gotzsche, J. P A Ioannidis, M. Clarke, P J Devereaux, J. Kleijnen, and D. Moher The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration BMJ, December 4, 2009; 339(jul21_1): b2700 - b2700. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. Hernandez-Rodriguez, M. C. Cid, A. Lopez-Soto, G. Espigol-Frigole, and X. Bosch Treatment of Polymyalgia Rheumatica: A Systematic Review Arch Intern Med, November 9, 2009; 169(20): 1839 - 1850. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 G. Robinson, J. E. McNulty, and J. S. Krasno Observing the Counterfactual? The Search for Political Experiments in Nature Political Analysis, October 1, 2009; 17(4): 341 - 357. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. Liberati, D. G. Altman, J. Tetzlaff, C. Mulrow, P. C. Gotzsche, J. P.A. Ioannidis, M. Clarke, P. J. Devereaux, J. Kleijnen, and D. Moher The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration Ann Intern Med, August 18, 2009; 151(4): W-65 - W-94. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M Mindlin, R Jenkins, and C Law Maternal employment and indicators of child health: a systematic review in pre-school children in OECD countries J Epidemiol Community Health, May 1, 2009; 63(5): 340 - 350. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 N. Simunovic, S. Sprague, and M. Bhandari Methodological Issues in Systematic Reviews and Meta-Analyses of Observational Studies in Orthopaedic Research J. Bone Joint Surg. Am., May 1, 2009; 91(Supplement_3): 87 - 94. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. Russell, M. Chung, E. M Balk, S. Atkinson, E. L Giovannucci, S. Ip, A. H Lichtenstein, S. T. Mayne, G. Raman, A C. Ross, et al. Opportunities and challenges in conducting systematic reviews to support the development of nutrient reference values: vitamin A as an example Am. J. Clinical Nutrition, March 1, 2009; 89(3): 728 - 733. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. G Mulholland, L. J Murray, and M. M Cantwell Glycemic index, glycemic load, and chronic disease risk Am. J. Clinical Nutrition, August 1, 2008; 88(2): 475 - 476. [Full Text] [PDF]
|
|
|
|
|
|
E. v. Elm, D. G Altman, M. Egger, S. J Pocock, P. C Gotzsche, J. P Vandenbroucke, and STROBE Initiative Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies BMJ, October 20, 2007; 335(7624): 806 - 808. [Full Text] [PDF]
|
|
|
|
|
|
E. von Elm, D. G. Altman, M. Egger, S. J. Pocock, P. C. Gotzsche, J. P. Vandenbroucke, and for the STROBE Initiative The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies Ann Intern Med, October 16, 2007; 147(8): 573 - 577. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. Ebrahim and M. Clarke STROBE: new standards for reporting observational epidemiology, a chance to improve Int. J. Epidemiol., October 1, 2007; 36(5): 946 - 948. [Full Text] [PDF]
|
|