IJE Advance Access originally published online on November 14, 2006
International Journal of Epidemiology 2006 35(6):1588-1589; doi:10.1093/ije/dyl226
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Letters to the Editor |
A method to automate probabilistic sensitivity analyses of misclassified binary variables
Epidemiology and Biostatistics, School of Population Health, University of Auckland, New Zealand. E-mail: rj.marshall{at}auckland.ac.nz
Fox et al.1 examine the relationship between odds ratios for a binary measured exposure variable X and the underlying true exposure E. One form of the relationship2,3 that the authors do not consider is:
 | 1 |
, 
) are quality indices of misclassification in cases (controls). Here Q1 is sensitivity (SE) re-scaled according the measured prevalence of exposure PX i.e. Q1 = (SE – PX)/(1 – PX) and Q0 is specificity (SP) re-scaled according the measured prevalence of non-exposure 
i.e. 
. The Q indices are sometimes known as chance corrected sensitivity and specificity.4
From (1) the bias in using ORX is
 |
and 
, or even if they are simply proportional.
Consider, for example, using the lung cancer data in Fox et al. The measured prevalence of exposure in cases is 45/139 = 0.32 and in controls it is 257/1202 = 0.21. When specificity and sensitivity are non-differential and both equal 0.9, which is one situation considered by Fox, the Q indices are Q1 = 0.85, 
, Q0 =0.69, and 
. The Q1 values indicate good agreement in people truly exposed and they are non-differential. But the Q0 indices differ substantially and are much lower. Bias arises because the Q0 indices differ although it is not too severe, B = 0.69 x 0.87/0.52 x 0.85 = 1.33 (which agrees with the ratio of the odds reported ratios ORE/ORX = 2.34/1.8 from Fox's Tables 2 and 3).
Consider though, when specificity and sensitivity are non-differential and both equal 0.8. Then Q1 = 0.71, 
, Q0 = 0.38 and 
. The discrepant Q0 and now lead to substantial bias, B = 6.3 using (1), or the ratio 11.0/1.8 from Table 2 and 3 in Fox. Here is hopelessly low, because the probability of observing non-exposure in controls truly not exposed, (specificity 0.8), is not substantially different from the measured probability of non-exposure 0.79.
Fox et al.'s approach is to ask how much bias there is for given values of sensitivity and specificity. An alternative view is to ask: for what values of sensitivity and specificity is there little or no bias? From (1), and by definition of the Q indices, if any two of the four sensitivity and specificity parameters are fixed, the relationship between the remaining two is linear. Turning to the lung cancer data again, fix the sensitivity and specificity in controls at 0.9. Corresponding values of sensitivity and specificity in cases which yield no bias fall along the straight line in the Figure 1. Above the line B < 1, below it B > 1. The shaded areas either side of the line show where the bias is ±10% i.e. 0.91 < B < 1.1 and where it is ±50% i.e. 0.66 < B < 1.5. Only for a narrow range of case specificity and sensitivity will bias not be too serious (<10%). Superimposed on the Figure are values of bias at selected points.
|
It is worth noting that the V-shaped boundaries converge at the point (1–0.32, 0.32), 0.32 being the prevalence of exposure in cases. With these values of case sensitivity and specificity, Q0 and Q1 are both zero and correction for bias cannot be done, at least, using this approach.
In3 I investigated the magnitude of sensitivities, specificities, quality indices and bias in a meta-analysis of many studies in which exposure measurements had been validated against an ostensibly ‘true’ measure. Unlike sensitivity and specificity, Q indices are often roughly non-differential. There was no systematic bias in either direction and the magnitude of the empirical bias did not, in most cases, reach statistical significance. This does not mean that misclassification bias should not be seriously considered. However, adopting scenario situations based on specified sensitivity and specificity values can be misleading and pessimistic; specificity and sensitivity are not absolutes, they are restricted in their range of possible values, depending on the measured exposure prevalence. The key to considering whether bias is present is not in terms of ‘differentiality’ of specificity and sensitivity but in terms of differentiality of their chance-corrected counterparts.
A Stata program to draw graphs like the one in the Figure is available from me on request.
References
1 Fox MP, Lash TI, Greenland S. A method to automate probabilistic sensitivity analyses of misclassified binary variables. Int J Epidemiol 2005;34:1370–76.
2 Marshall RJ. Assessment of exposure misclassification bias in case-control studies using validation data. J Clin Epidemiol 1997;50:15–19.[CrossRef][Web of Science][Medline]
3 Marshall RJ. An empirical investigation of exposure measurement bias in case-control studies. J Clin Epidemiol 1999;52:547–50.[CrossRef][Web of Science][Medline]
4 Jamart J. Chance corrected sensitivity and specificity for three-zone diagnostic tests. J Clin Epidemiol 1992;45:1035–39.[CrossRef][Web of Science][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||