This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Phelps, C. E. Articles by Hutson, A. Search for Related Content PubMed PubMed Citation Articles by Phelps, C. E. Articles by Hutson, A. Pubmed/NCBI databases Medline Plus Health Information Laboratory Tests Social Bookmarking                         What's this?

Estimating Diagnostic Test Accuracy Using a "Fuzzy Gold Standard"

Alan Hutson, MS

This study uses Monte Carlo methods to analyze the consequences of having a criterion standard ("gold standard") that contains some error when analyzing the accuracy of a diagnostic test using ROC curves. Two phenomena emerge: 1) When diagnostic test errors are statistically independent from inaccurate ("fuzzy") gold standard (FGS) errors, estimated test accuracy declines. 2) When the test and the FGS have statistically dependent errors, test accuracy can become overstated. Two methods are proposed to eliminate the first of these errors, exploring the risk of exacerbating the second. Both require a probabilistic (rather than binary) gold-standard statement (e.g., probability that each case is abnormal). The more promising of these, the "two-truth" method, selectively eliminates those cases where the gold standard is most ambiguous (probability near 0.5). When diagnostic test and FGS errors are independent, this approach can eliminate much of the downward bias caused by FGS error, without meaningful risk of overstating test accuracy. When the test and FGS have dependent errors, the resultant upward bias can cause test accuracy to be overstated, in the most extreme cases, even before the offsetting "two-truth" approach is employed. Key words: ROC curves; diagnostic test accuracy; technology assessment. (Med Decis Making 1995;15:44-57)

Medical Decision Making, Vol. 15, No. 1, 44-57 (1995)
DOI: 10.1177/0272989X9501500108


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				C. M. Wilson, K. D. Cocker, M. J. Moseley, C. Paterson, S. T. Clay, W. E. Schulenburg, M. D. Mills, A. L. Ells, K. H. Parker, G. E. Quinn, et al. Computerized Analysis of Retinal Vessel Width and Tortuosity in Premature Infants Invest. Ophthalmol. Vis. Sci., August 1, 2008; 49(8): 3577 - 3585. [Abstract] [Full Text] [PDF]

				N. R. Cook Statistical Evaluation of Prognostic versus Diagnostic Models: Beyond the ROC Curve Clin. Chem., January 1, 2008; 54(1): 17 - 23. [Abstract] [Full Text] [PDF]

				N. V. Greidanus, B. A. Masri, D. S. Garbuz, S. D. Wilson, M. G. McAlinden, M. Xu, and C. P. Duncan Use of Erythrocyte Sedimentation Rate and C-Reactive Protein Level to Diagnose Infection Before Revision Total Knee Arthroplasty. A Prospective Evaluation J. Bone Joint Surg. Am., July 1, 2007; 89(7): 1409 - 1416. [Abstract] [Full Text] [PDF]

				D. L. Phelps It's Plus Disease, Isn't It? Arch Ophthalmol, July 1, 2007; 125(7): 963 - 964. [Full Text] [PDF]

				K. H. Zou, A. J. O'Malley, and L. Mauri Receiver-Operating Characteristic Analysis for Evaluating Diagnostic Tests and Predictive Models Circulation, February 6, 2007; 115(5): 654 - 657. [Full Text] [PDF]

				M. B. Patwardhan, D. C. McCrory, D. B. Matchar, G. P. Samsa, and O. T. Rutschmann Alzheimer Disease: Operating Characteristics of PET-- A Meta-Analysis Radiology, April 1, 2004; 231(1): 73 - 80. [Abstract] [Full Text] [PDF]

				P. Whiting, A. W.S. Rutjes, J. B. Reitsma, A. S. Glas, P. M.M. Bossuyt, and J. Kleijnen Sources of Variation and Bias in Studies of Diagnostic Accuracy: A Systematic Review Ann Intern Med, February 3, 2004; 140(3): 189 - 202. [Abstract] [Full Text] [PDF]

				C. Bowd, L. M. Zangwill, C. C. Berry, E. Z. Blumenthal, C. Vasile, C. Sanchez-Galeana, C. F. Bosworth, P. A. Sample, and R. N. Weinreb Detecting Early Glaucoma by Assessment of Retinal Nerve Fiber Layer Thickness and Visual Function Invest. Ophthalmol. Vis. Sci., August 1, 2001; 42(9): 1993 - 2003. [Abstract] [Full Text] [PDF]

				L. M. Zangwill, C. Bowd, C. C. Berry, J. Williams, E. Z. Blumenthal, C. A. Sanchez-Galeana, C. Vasile, and R. N. Weinreb Discriminating Between Normal and Glaucomatous Eyes Using the Heidelberg Retina Tomograph, GDx Nerve Fiber Analyzer, and Optical Coherence Tomograph Arch Ophthalmol, July 1, 2001; 119(7): 985 - 993. [Abstract] [Full Text] [PDF]

				K. Nanda, D. C. McCrory, E. R. Myers, L. A. Bastian, V. Hasselblad, J. D. Hickey, and D. B. Matchar Accuracy of the Papanicolaou Test in Screening for and Follow-up of Cervical Cytologic Abnormalities: A Systematic Review Ann Intern Med, May 16, 2000; 132(10): 810 - 819. [Abstract] [Full Text] [PDF]

				R. Parasuraman, A. J. Masalonis, and P. A. Hancock Fuzzy Signal Detection Theory: Basic Postulates and Formulas for Analyzing Human and Machine Performance Human Factors: The Journal of the Human Factors and Ergonomics Society, January 1, 2000; 42(4): 636 - 659. [Abstract] [PDF]

				M. Hellmich, K. R. Abrams, and A. J. Sutton Bayesian Approaches to Meta-analysi of ROC Curves Med Decis Making, August 1, 1999; 19(3): 252 - 264. [Abstract] [PDF]

				D. E Shapiro The interpretation of diagnostic tests Statistical Methods in Medical Research, April 1, 1999; 8(2): 113 - 134. [Abstract] [PDF]

				C. E. Schwartz, T. Vollmer, and H. Lee Reliability and validity of two self-report measures of impairment and disability for MS Neurology, January 1, 1999; 52(1): 63 - 63. [Abstract] [Full Text] [PDF]