This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Diamond, G. A. Search for Related Content PubMed Articles by Diamond, G. A. Social Bookmarking                   What's this?

Affirmative Actions

Can the Discriminant Accuracy of a Test Be Determined in the Face of Selection Bias?

Clinical estimates of test efficacy can be distorted by the differential referral of positive and negative test responders for outcome verification. Accordingly, a series of computer simu lations was performed to quantify the effects of various degrees of this selection bias on the observed true-positive rate, false-positive rate, and discriminant accuracy of a hypothetical test. The error in observed true- and false-positive rates was positive with respect to diag nosis, and negative with respect to prognosis. The magnitude of error was highly correlated with the magnitude of bias associated with the test response (primary selection bias), but not with the magnitude of bias associated with additional independent factors (secondary selection bias). Mathematical correction for preferential referral based on the test response using a previously published algorithm completely removed the correlation with primary selection bias for both diagnosis and prognosis. Although a significant correlation with sec ondary selection bias persisted at intermediate base rates, its magnitude was small. Dis criminant accuracy was assessed in terms of area under a receiver operating characteristic (ROC) curve. Biased values of true- and false-positive rates were distributed along the curve defined by the actual true- and false-positive rates of the test for both diagnosis and prognosis. As a result, the areas under ROC curves calculated from biased true- and false-positive rates were within 2% of the areas calculated from the actual rates. Only when the primary and secondary observations were independent with respect to one outcome and dependent with respect to the other outcome did a systematic error appear in ROC area. These data indicate that: 1) selection bias significantly distorts the determination of diagnostic and prog nostic test accuracy in directionally opposite ways; 2) the distortion can be partially offset by a previously published mathematical algorithm; and 3) the area under an ROC curve is insensitive both to the primary bias associated with the test response itself and to the secondary bias associated with concomitant clinical information under a variety of circum stances. Key words: diagnosis; prognosis; referral bias; ROC curves; sensitivity; specificity; verification bias; workup bias. (Med Decis Making 1991;11:48-56)

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

This article has been cited by other articles:

				A. P. Morise, G. A. Diamond, R. Detrano, M. Bobbio, and E. Gunel The Effect of Disease-prevalence Adjustments on the Accuracy of a Logistic Prediction Model Med Decis Making, June 1, 1996; 16(2): 133 - 142. [Abstract] [PDF]

				Y. T. Van Der Schouw, H. Straatman, and A. L.M. Verbeek ROC Curves and the Areas under Them for Dichotomized Tests: Empirical Findings for Logistically and Normally Distributed Diagnostic Test Results Med Decis Making, October 1, 1994; 14(4): 374 - 381. [Abstract] [PDF]

				G. A. Diamond Off Bayes: Effect of Verification Bias on Posterior Probabilities Calculated Using Bayes' Theorem Med Decis Making, February 1, 1992; 12(1): 22 - 30. [Abstract] [PDF]

				G. A. Diamond Scotched on the ROCs Med Decis Making, August 1, 1991; 11(3): 198 - 200. [Abstract] [PDF]