Advanced Search

Journal Navigation

Journal Home

Subscriptions

Archive

Contact Us

Table of Contents

CiteULike is a free service for managing and discovering scholarly references - click here to get started.

Sign In to gain access to subscriptions and/or personal tools.
Medical Decision Making
This Article
Right arrow Full Text (PDF)
Right arrow References
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to Saved Citations
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Request Reprints
Right arrow Add to My Marked Citations
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Right arrow Citing Articles via Scopus
Google Scholar
Right arrow Articles by Weller, S. C.
Right arrow Articles by Mann, N. C.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Weller, S. C.
Right arrow Articles by Mann, N. C.
Social Bookmarking
Add to CiteULike   Add to Complore   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati   Add to Twitter  
What's this?

Assessing Rater Performance without a "Gold Standard" Using Consensus Theory

Susan C. Weller

N. Clay Mann

This study illustrates the use of consensus theory to assess the diagnostic perform ances of raters and to estimate case diagnoses in the absence of a criterion or "gold" standard. A description is provided of how consensus theory "pools" information pro vided by raters, estimating rater competencies and differentially weighting their re sponses. Although the model assumes that raters respond without bias (i.e., sensitivity = specificity), a Monte Carlo simulation with 1,200 data sets shows that model esti mates appear to be robust even with bias. The model is illustrated on a set of elbow radiographs, and consensus-model estimates are compared with those obtained from follow-up data. Results indicate that with high rater competencies, the model retrieves accurate estimates of competency and case diagnoses even when raters' responses are biased. Key words: clinical competence; interobserver variation; diagnostic evalu ation ; models—mathematical; consensus theory. (Med Decis Making 1997;17:71- 79)

Medical Decision Making, Vol. 17, No. 1, 71-79 (1997)
DOI: 10.1177/0272989X9701700108
Add to CiteULike CiteULike   Add to Complore Complore   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to Technorati Technorati   Add to Twitter Twitter    What's this?


This article has been cited by other articles:


Home page
 Hum Exp ToxicolHome page
S Hoffmann and T Hartung
Toward an evidence-based toxicology
Human and Experimental Toxicology, September 1, 2006; 25(9): 497 - 513.
[Abstract] [PDF]

Home page
 Toxicol SciHome page
S. Hoffmann and T. Hartung
Diagnosis: Toxic! - Trying to Apply Approaches of Clinical Diagnostics and Prevalence in Toxicology Considerations
Toxicol. Sci., May 1, 2005; 85(1): 422 - 428.
[Abstract] [Full Text] [PDF]

Home page
 Field MethodsHome page
V. Reyes-Garcia, E. Byron, V. Vadez, R. Godoy, L. Apaza, E. P. Limache, W. R. Leonard, and D. Wilkie
Measuring Culture as Shared Knowledge: Do Data Collection Formats Matter? Cultural Knowledge of Plant Uses Among Tsimane' Amerindians, Bolivia
Field Methods, May 1, 2004; 16(2): 135 - 156.
[Abstract] [PDF]