|
Sign In to gain access to subscriptions and/or personal tools.
|
Medical Decision Making, Vol. 26, No. 6,
565-574 (2006)
DOI: 10.1177/0272989X06295361
Decision Curve Analysis: A Novel Method for Evaluating Prediction Models
Andrew J. Vickers, PhD
Department of Epidemiology and Biostatistics, Department of Urology, and Department of Medicine, Memorial Sloan-Kettering Cancer Center, New York; vickersa{at}mskcc.org
Elena B. Elkin, PhD
Department of Epidemiology and Biostatistics, Memorial Sloan-Kettering Cancer Center, New York.
Background. Diagnostic and prognostic models are typically evaluated with measures of accuracy that do not address clinical consequences. Decision-analytic techniques allow assessment of clinical outcomes but often require collection of additional information and may be cumbersome to apply to models that yield a continuous result. The authors sought a method for evaluating and comparing prediction models that incorporates clinical consequences, requires only the data set on which the models are tested, and can be applied to models that have either continuous or dichotomous results. Method. The authors describe decision curve analysis, a simple, novel method of evaluating predictive models. They start by assuming that the threshold probability of a disease or event at which a patient would opt for treatment is informative of how the patient weighs the relative harms of a false-positive and a false-negative prediction. This theoretical relationship is then used to derive the net benefit of the model across different threshold probabilities. Plotting net benefit against threshold probability yields the "decision curve." The authors apply the method to models for the prediction of seminal vesicle invasion in prostate cancer patients. Decision curve analysis identified the range of threshold probabilities in which a model was of value, the magnitude of benefit, and which of several models was optimal. Conclusion. Decision curve analysis is a suitable method for evaluating alternative diagnostic and prognostic strategies that has advantages over other commonly used measures and techniques.
Key Words: prediction models • multivariate analysis • decision analysis
References
- Freedman AN, Seminara D, Gail MH, et al. Cancer risk prediction models: a workshop on development, evaluation, and application. J Natl Cancer Inst. 2005;97:715–723.[Abstract/Free Full Text]
- Das SK, Baydush AH, Zhou S, et al. Predicting radiotherapyinduced cardiac perfusion defects. Med Phys. 2005;32:19–27.[CrossRef][ISI][Medline]
[Order article via Infotrieve]
- Hendriks DJ, Broekmans FJ, Bancsi LF, Looman CW, De Jong FH, Te Velde ER. Single and repeated GnRH agonist stimulation tests compared with basal markers of ovarian reserve in the prediction of outcome in IVF. J Assist Reprod Genet. 2005;22: 65–73.[CrossRef][ISI][Medline]
[Order article via Infotrieve]
- Cindolo L, Patard JJ, Chiodini P, et al. Comparison of predictive accuracy of four prognostic models for nonmetastatic renal cell carcinoma after nephrectomy. Cancer. 2005;104:1362–1371.[CrossRef][ISI][Medline]
[Order article via Infotrieve]
- Hunink M, Glasziou P, Siegel J. Decision-Making in Health and Medicine: Integrating Evidence and Values. New York: Cambridge University Press; 2001.
- Djulbegovic B, Desoky AH. Equation and nomogram for calculation of testing and treatment thresholds. Med Decis Making. 1996;16:198–199.[Free Full Text]
- Loomes G, Mckenzie L. The use of QALYs in health care decision making. Soc Sci Med. 1989;28:299–308.[CrossRef][ISI][Medline]
[Order article via Infotrieve]
- Van Osch SM, Wakker PP, Van Den Hout WB, Stiggelbout AM. Correcting biases in standard gamble and time tradeoff utilities. Med Decis Making. 2004;24:511–517.[Abstract]
- Weinstein MC, Fineberg HV. Clinical Decision Analysis. Philadelphia: W. B. Saunders; 1980.
- Pauker SG, Kassirer JP. The threshold approach to clinical decision making. N Engl J Med. 1980;302:1109–1117.[Abstract]
- Djulbegovic B, Hozo I, Lyman GH. Linking evidence-based medicine therapeutic summary measures to clinical decision analysis. Medgenmed. 2000;2:E6-E6.[Medline]
[Order article via Infotrieve]
- Zlotta AR, Roumeguere T, Ravery V, et al. Is seminal vesicle ablation mandatory for all patients undergoing radical prostatectomy? A multivariate analysis on 1283 patients. Eur Urol. 2004;46:42–49.[CrossRef][ISI][Medline]
[Order article via Infotrieve]
- Guzzo TJ, Vira M, Wang Y, et al. Preoperative parameters, including percent positive biopsy, in predicting seminal vesicle involvement in patients with prostate cancer. J Urol. 2006;175: 518–521.[CrossRef][ISI][Medline]
[Order article via Infotrieve]
- Peirce CS. The numerical measure of the success of predictions. Science. 1884;4:453–454.
- Kattan MW, Eastham JA, Stapleton AM, Wheeler TM, Scardino PT. A preoperative nomogram for disease recurrence following radical prostatectomy for prostate cancer. J Natl Cancer Inst. 1998;90: 766–771.[Abstract/Free Full Text]
- Steuber T, Karakiewicz PI, Augustin H, et al. Transition zone cancers undermine the predictive accuracy of partin table stage predictions. J Urol. 2005;173:737–741.[CrossRef][ISI][Medline]
[Order article via Infotrieve]
- Pepe MS, Etzioni R, Feng Z, et al. Phases of biomarker development for early detection of cancer. J Natl Cancer Inst. 2001;93: 1054–1061.[Free Full Text]
- Moons KG, Stijnen T, Michel BC, et al. Application of treatment thresholds to diagnostic-test evaluation: an alternative to the comparison of areas under receiver operating characteristic curves. Med Decis Making. 1997;17:447–454.[Abstract/Free Full Text]
- Parmigiani G. Modeling in Medical Decision Making. New York: John Wiley; 2002.
- Habbema JDF, Hilden J. The measurement of performance in probabilistic diagnosis: IV. Utility considerations in therapeutics and prognostics. Meth Inform Med. 1978;17:238–246.[ISI][Medline]
[Order article via Infotrieve]
- Hilden J. Evaluation of diagnostic tests: the schism. Soc Med Decis Making Newsletter. 2004;16:5–6.
 CiteULike  Connotea  Del.icio.us  Digg  Reddit  Technorati What's this?
This article has been cited by other articles:
|
|
|
|
 
D. Ulmert, A. M. Serio, M. F. O'Brien, C. Becker, J. A. Eastham, P. T. Scardino, T. Bjork, G. Berglund, A. J. Vickers, and H. Lilja
Long-Term Prediction of Prostate Cancer: Prostate-Specific Antigen (PSA) Velocity Is Predictive but Does Not Improve the Predictive Accuracy of a Single PSA Measurement 15 Years or More Before Cancer Diagnosis in a Large, Representative, Unscreened Population
J. Clin. Oncol.,
February 20, 2008;
26(6):
835 - 841.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
E. W. Steyerberg and A. J. Vickers
Decision Curve Analysis: A Discussion
Med Decis Making,
February 1, 2008;
28(1):
146 - 149.
[PDF]
|
|
|
|
