This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Steyerberg, E. W. Articles by Habbema, J. D. F. Search for Related Content PubMed PubMed Citation Articles by Steyerberg, E. W. Articles by Habbema, J. D. F. Pubmed/NCBI databases Medline Plus Health Information Heart Attack Social Bookmarking                   What's this?

Prognostic Modeling with Logistic Regression Analysis

In Search of a Sensible Strategy in Small Data Sets

Center for Clinical Decision Sciences, Department of Public Health, Erasmus University, Rotterdam, the Netherlands

Marinus J. C. Eijkemans, MSc

Center for Clinical Decision Sciences, Department of Public Health, Erasmus University, Rotterdam, the Netherlands

Frank E. Harrell, Jr, PhD

Division of Biostatistics and Epidemiology, Department of Health Evaluation Sciences, University of Virginia, Charlottesville, Virginia

J. Dik F. Habbema, PhD

Center for Clinical Decision Sciences, Department of Public Health, Erasmus University, Rotterdam, the Netherlands

Clinical decision making often requires estimates of the likelihood of a dichotomous outcome in individual patients. When empirical data are available, these estimates may well be obtained from a logistic regression model. Several strategies may be followed in the development of such a model. In this study, the authors compare alternative strategies in 23 small subsamples from a large data set of patients with an acute myocardial infarction, where they developed predictive models for 30-day mortality. Evaluations were performed in an independent part of the data set. Specifically, the authors studied the effect of coding of covariables and stepwise selection on discriminative ability of the resulting model, and the effect of statistical "shrinkage" techniques on calibration. As expected, dichotomization of continuous covariables implied a loss of information. Remarkably, stepwise selection resulted in less discriminating models compared to full models including all available covariables, even when more than half of these were randomly associated with the outcome. Using qualitative information on the sign of the effect of predictors slightly improved the predictive ability. Calibration improved when shrinkage was applied on the standard maximum likelihood estimates of the regression coefficients. In conclusion, a sensible strategy in small data sets is to apply shrinkage methods in full models that include well-coded predictors that are selected based on external information.

Key Words: regression analysis • logistic models • bias • variable selection • prediction

Medical Decision Making, Vol. 21, No. 1, 45-56 (2001)
DOI: 10.1177/0272989X0102100106


CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

This article has been cited by other articles:

				U. Corra, A. Mezzani, A. Giordano, E. Bosimini, and P. Giannuzzi Exercise haemodynamic variables rather than ventilatory efficiency indexes contribute to risk assessment in chronic heart failure patients treated with carvedilol Eur. Heart J., April 30, 2009; (2009) ehp138v1. [Abstract] [Full Text] [PDF]

				P. Royston, K. G M Moons, D. G Altman, and Y. Vergouwe Prognosis and prognostic research: Developing a prognostic model BMJ, March 31, 2009; 338(mar31_1): b604 - b604. [Full Text]

				K. E. Freedland, R. L. Reese, and B. C. Steinmeyer Multivariable Models in Biobehavioral Research Psychosom Med, February 1, 2009; 71(2): 205 - 216. [Abstract] [Full Text] [PDF]

				E. W Steyerberg and H. F Lingsma Validating prediction models BMJ, April 12, 2008; 336(7648): 789 - 789. [Full Text] [PDF]

				N. Frasure-Smith and F. Lesperance Depression and Anxiety as Predictors of 2-Year Cardiac Events in Patients With Stable Coronary Artery Disease Arch Gen Psychiatry, January 1, 2008; 65(1): 62 - 71. [Abstract] [Full Text] [PDF]

				D. N. Wijeysundera, K. Karkouti, J.-Y. Dupuis, V. Rao, C. T. Chan, J. T. Granton, and W. S. Beattie Derivation and Validation of a Simplified Predictive Index for Renal Replacement Therapy After Cardiac Surgery JAMA, April 25, 2007; 297(16): 1801 - 1809. [Abstract] [Full Text] [PDF]

				P. H. Wirtz, S. Elsenbruch, L. Emini, K. Rudisuli, S. Groessbauer, and U. Ehlert Perfectionism and the Cortisol Response to Psychosocial Stress in Men Psychosom Med, April 1, 2007; 69(3): 249 - 255. [Abstract] [Full Text] [PDF]

				M. Smits, D. W.J. Dippel, E. W. Steyerberg, G. G. de Haan, H. M. Dekker, P. E. Vos, D. R. Kool, P. J. Nederkoorn, P. A.M. Hofman, A. Twijnstra, et al. Predicting Intracranial Traumatic Findings on Computed Tomography in Patients with Minor Head Injury: The CHIP Prediction Rule Ann Intern Med, March 20, 2007; 146(6): 397 - 405. [Abstract] [Full Text] [PDF]

				R. Souza, C. Jardim, C. Carvalho, G. Rubenfeld, A. Fijalkowska, A. Torbicki, and M. Kurzyna The Role of NT-proBNP as a Prognostic Marker in Pulmonary Hypertension. Chest, November 1, 2006; 130(5): 1627 - 1628. [Full Text] [PDF]

				J. Zhang, R. Niaura, J. R. Dyer, B.-J. Shen, J. F. Todaro, J. M. McCaffery, A. Spiro III, and K. D. Ward Hostility and Urine Norepinephrine Interact to Predict Insulin Resistance: The VA Normative Aging Study. Psychosom Med, September 1, 2006; 68(5): 718 - 726. [Abstract] [Full Text] [PDF]

				B. J. Cowling, M. P. Muller, I. O. L. Wong, L.-M. Ho, S.-V. Lo, T. Tsang, T. H. Lam, M. Louie, and G. M. Leung Clinical prognostic rules for severe acute respiratory syndrome in low- and high-resource settings. Arch Intern Med, July 24, 2006; 166(14): 1505 - 1511. [Abstract] [Full Text] [PDF]

				A. S. Karlamangla, B. H. Singer, and T. E. Seeman Reduction in Allostatic Load in Older Adults Is Associated With Lower All-Cause Mortality Risk: MacArthur Studies of Successful Aging Psychosom Med, May 1, 2006; 68(3): 500 - 507. [Abstract] [Full Text] [PDF]

				E. W. Steyerberg Local Applicability of Clinical and Model-Based Probability Estimates Med Decis Making, November 1, 2005; 25(6): 678 - 680. [PDF]

				M. Bower, B. Gazzard, S. Mandalia, T. Newsom-Davis, C. Thirlwell, T. Dhillon, A. M. Young, T. Powles, A. Gaya, M. Nelson, et al. A Prognostic Index for Systemic AIDS-Related Non-Hodgkin Lymphoma Treated in the Era of Highly Active Antiretroviral Therapy Ann Intern Med, August 16, 2005; 143(4): 265 - 273. [Abstract] [Full Text] [PDF]

				R. Jin, G. L. Grunkemeier, A. Starr, and Providence Health System Cardiovascular Study Grou Validation and Refinement of Mortality Risk Models for Heart Valve Surgery Ann. Thorac. Surg., August 1, 2005; 80(2): 471 - 479. [Abstract] [Full Text] [PDF]

				T. Rakow, C. Vincent, K. Bull, and N. Harvey Assessing the Likelihood of an Important Clinical Outcome: New Insights from a Comparison of Clinical and Actuarial Judgment Med Decis Making, May 1, 2005; 25(3): 262 - 282. [Abstract] [PDF]

				N. Frasure-Smith and F. Lesperance Reflections on Depression as a Cardiac Risk Factor Psychosom Med, May 1, 2005; 67(Supplement_1): S19 - S25. [Abstract] [Full Text] [PDF]

				A. C. J. W. Janssens, Y. Deng, G. J. J. M. Borsboom, M. J. C. Eijkemans, J. Dik. F. Habbema, and E. W. Steyerberg A New Logistic Regression Approach for the Evaluation of Diagnostic Test Results Med Decis Making, March 1, 2005; 25(2): 168 - 177. [Abstract] [PDF]

				B Bressler, R Pinto, D El-Ashry, and E J Heathcote Which patients with primary biliary cirrhosis or primary sclerosing cholangitis should undergo endoscopic screening for oesophageal varices detection? Gut, March 1, 2005; 54(3): 407 - 410. [Abstract] [Full Text] [PDF]

				E Meijer, D E Grobbee, and D Heederik A strategy for health surveillance in laboratory animal workers exposed to high molecular weight allergens Occup. Environ. Med., October 1, 2004; 61(10): 831 - 837. [Abstract] [Full Text] [PDF]

				G. M. Leung, T. H. Rainer, F.-L. Lau, I. O.L. Wong, A. Tong, T.-W. Wong, J. H.B. Kong, A. J. Hedley, T.-H. Lam, and for the Hospital Authority SARS Collaborative Grou A Clinical Prediction Rule for Diagnosing Severe Acute Respiratory Syndrome in the Emergency Department Ann Intern Med, September 7, 2004; 141(5): 333 - 342. [Abstract] [Full Text] [PDF]

				M. R. van Dijk, E. W. Steyerberg, S. P. Stenning, and J. D. F. Habbema Identifying subgroups among poor prognosis patients with nonseminomatous germ cell cancer by tree modelling: a validation study Ann. Onc., September 1, 2004; 15(9): 1400 - 1405. [Abstract] [Full Text] [PDF]

				C.C. Hunault, J.D.F. Habbema, M.J.C. Eijkemans, J.A. Collins, J.L.H. Evers, and E.R. te Velde Two new prediction rules for spontaneous pregnancy leading to live birth among subfertile couples, based on the synthesis of three previous models Hum. Reprod., September 1, 2004; 19(9): 2019 - 2026. [Abstract] [Full Text] [PDF]

				H. Jin, Y. Lu, S. T. Harris, D. M. Black, K. Stone, M. C. Hochberg, and H. K. Genant Classification Algorithms for Hip Fracture Prediction Based on Recursive Partitioning Methods Med Decis Making, August 1, 2004; 24(4): 386 - 398. [Abstract] [PDF]

				M. A. Babyak What You See May Not Be What You Get: A Brief, Nontechnical Introduction to Overfitting in Regression-Type Models Psychosom Med, May 1, 2004; 66(3): 411 - 421. [Abstract] [Full Text] [PDF]

				D. S. Lee, P. C. Austin, J. L. Rouleau, P. P. Liu, D. Naimark, and J. V. Tu Predicting Mortality Among Patients Hospitalized for Heart Failure: Derivation and Validation of a Clinical Model JAMA, November 19, 2003; 290(19): 2581 - 2587. [Abstract] [Full Text] [PDF]

				A. V. Hernandez, Y. Vergouwe, E. W. Steyerberg, and M. Moss Reporting of Predictive Logistic Models Should Be Based on Evidence-Based Guidelines Chest, November 1, 2003; 124(5): 2034 - 2035. [Full Text] [PDF]

				N. Frasure-Smith and F. Lesperance Depression and Other Psychological Risks Following Myocardial Infarction Arch Gen Psychiatry, June 1, 2003; 60(6): 627 - 636. [Abstract] [Full Text] [PDF]

				D. L. Tirschwell, W.T. Longstreth Jr, K. J. Becker, R. E. Gammans Sr, L. A. Sabounjian, S. Hamilton, and L. B. Morgenstern Shortening the NIH Stroke Scale for Use in the Prehospital Setting Stroke, December 1, 2002; 33(12): 2801 - 2806. [Abstract] [Full Text] [PDF]

				F. Kee, C. C. Patterson, A. E. Wilson, J. M. McConnell, S. M. Wheeler, and J. D. Watson Judgment Analysis of Prioritization Decisions within a Dialysis Program in One United Kingdom Region Med Decis Making, April 1, 2002; 22(2): 140 - 151. [Abstract] [PDF]

				F. Lesperance, N. Frasure-Smith, M. Talajic, and M. G. Bourassa Five-Year Risk of Cardiac Mortality in Relation to Initial Severity and One-Year Changes in Depression Symptoms After Myocardial Infarction Circulation, March 5, 2002; 105(9): 1049 - 1053. [Abstract] [Full Text] [PDF]