This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Morise, A. P. Articles by Gunel, E. Search for Related Content PubMed PubMed Citation Articles by Morise, A. P. Articles by Gunel, E. Social Bookmarking                   What's this?

The Effect of Disease-prevalence Adjustments on the Accuracy of a Logistic Prediction Model

George A. Diamond, MD

Robert Detrano, MD, PhD

Marco Bobbio, MD

Erdogan Gunel, PhD

The accuracy of a logistic prediction model is degraded when it is transported to pop ulations with outcome prevalences different from that of the population used to derive the model. The resultant errors can have major clinical implications. Accordingly, the authors developed a logistic prediction model with respect to the noninvasive diagnosis of coronary disease based on 1,824 patients who underwent exercise testing and coronary angiography, varied the prevalence of disease in various "test" populations by random sampling of the original "derivation" population, and determined the accu racy of the logistic prediction model before and after the application of a mathematical algorithm designed to adjust only for these differences in prevalence. The accuracy of each prediction model was quantified in terms of receiver operating characteristic (ROC) curve area (discrimination) and chi-square goodness-of-fit (calibration). As the prevalence of the test population diverged from the prevalence of the derivation pop ulation, discrimination improved (ROC-curve areas increased from 0.82 ± 0.02 to 0.87 ± 0.03; p < 0.05), and calibration deteriorated (chi-square goodness-of-fit statistics increased from 9 to 154; p < 0.05). Following adjustment of the logistic intercept for differences in prevalence, discrimination was unchanged and calibration improved (maximum chi-square goodness-of-fit fell from 154 to 16). When the adjusted algorithm was applied to three geographically remote populations with prevalences that differed from that of the derivation population, calibration improved 87%, while discrimination fell by 1 %. Thus, prevalence differences produce statistically significant and potentially clinically important errors in the accuracy of logistic prediction models. These errors can potentially be mitigated by use of a relatively simple mathematical correction al gorithm. Key words: logistic prediction model; accuracy; prevalence adjustments; model transportability. (Med Decis Making 1996;16:133-142)

Medical Decision Making, Vol. 16, No. 2, 133-142 (1996)
DOI: 10.1177/0272989X9601600205


CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

This article has been cited by other articles:

				W. J. McIsaac, R. Moineddin, and S. Ross Validation of a Decision Aid to Assist Physicians in Reducing Unnecessary Antibiotic Drug Use for Acute Cystitis Arch Intern Med, November 12, 2007; 167(20): 2201 - 2206. [Abstract] [Full Text] [PDF]

				R. Rohrig, A. Junger, B. Hartmann, J. Klasen, L. Quinzio, A. Jost, M. Benson, and G. Hempelmann The Incidence and Prediction of Automatically Detected Intraoperative Cardiovascular Events in Noncardiac Surgery Anesth. Analg., March 1, 2004; 98(3): 569 - 577. [Abstract] [Full Text] [PDF]

				W. J. McIsaac, D. E. Low, A. Biringer, N. Pimlott, M. Evans, and R. Glazier The Impact of Empirical Management of Acute Cystitis on Unnecessary Antibiotic Use Arch Intern Med, March 11, 2002; 162(5): 600 - 605. [Abstract] [Full Text] [PDF]

				W. J. McIsaac, V. Goel, T. To, and D. E. Low The validity of a sore throat score in family practice Can. Med. Assoc. J., October 1, 2000; 163(7): 811 - 815. [Abstract] [Full Text] [PDF]