This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Heckerling, P. S. Articles by Wigton, R. S. Search for Related Content PubMed PubMed Citation Articles by Heckerling, P. S. Articles by Wigton, R. S. Social Bookmarking                         What's this?

Reproducibility of Predictor Variables from a Validated Clinical Rule

Roger C. Conant

Thomas G. Tape

Robert S. Wigton

It has been suggested that clinical prediction rules are not reproducible, and that the most important variables frequently do not appear in replicate models. The authors studied the reproducibility of a validated rule for predicting radiographic evidence of pneumonia (ROC areas for the training and validation cohorts, 0.816 and 0.821, respectively). Two hundred replicate samples of size 250 and size 500 were generated by sampling without replacement from the original training cohort of 905 patients with a 14.6% prevalence of pneumonia. Forward selection was performed among 31 candidate variables by stepwise logistic regres sion. Using as reproducibility criteria: 1) inclusion of all five variables from the original model in the original order; 2) inclusion of all five variables in any order; 3) inclusion of the first three variables; 4) inclusion of the first two variables; 5) inclusion of the first variable; and 6) inclusion of any of the five variables: 2.5%, 13.5%, 48.5%, 85.5%, 98.0%, and 100% of replicate models of sample size 500, respectively, met the criteria, whereas 0%, 0%, 16.5%, 49.0%, 71.5%, and 97.5% of models of sample size 250 met the criteria (all comparisons by sample size p < .0001 except for criteria 1 and 6, p = 0.07). Mean ROC areas in the training and validation samples were 0.829 and 0.791 for replicate models of sample size 500, and 0.831 and 0.779 for models of sample size 250. There was no significant difference in ROC areas between training and validation cohorts for 80.5% of models of sample size 500, and for 75.3% of models of sample size 250. It is concluded that the most important predictor variables from a validated rule were included in the majority of replicate models, although the rule itself was reproduced in only a small minority of cases. In addition, the replicate models demonstrated good discriminatory power, and met statistical criteria for validation in an independent testing sample in a high percentage of cases. Key words: clinical prediction rules; replicate models; validation. (Med Decis Making 1992;12:280-285)

Medical Decision Making, Vol. 12, No. 4, 280-285 (1992)
DOI: 10.1177/0272989X9201200406


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				A. C. Justice, K. E. Covinsky, and J. A. Berlin Assessing the Generalizability of Prognostic Information Ann Intern Med, March 16, 1999; 130(6): 515 - 524. [Abstract] [Full Text] [PDF]

				R. K. Orr Use of a Probabilistic Neural Network to Estimate the Risk of Mortality after Cardiac Surgery Med Decis Making, April 1, 1997; 17(2): 178 - 185. [Abstract] [PDF]

				A. P. Morise, G. A. Diamond, R. Detrano, M. Bobbio, and E. Gunel The Effect of Disease-prevalence Adjustments on the Accuracy of a Logistic Prediction Model Med Decis Making, June 1, 1996; 16(2): 133 - 142. [Abstract] [PDF]

				R. K. Orr, B. S. Maini, F. D. Sottile, E. M. Dumas, and P. O'Mara A Comparison of Four Severity-Adjusted Models to Predict Mortality After Coronary Artery Bypass Graft Surgery Arch Surg, March 1, 1995; 130(3): 301 - 306. [Abstract] [PDF]

				J. A. Knottnerus Prediction Rules: Statistieal Reprodueibility and Clinical Similarity Med Decis Making, December 1, 1992; 12(4): 286 - 287. [PDF]