This Article Full Text (PDF) All Versions of this Article: 0272989X08327110v1 0272989X08327110v2 29/2/247    most recent References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Matheny, M. E. Articles by Resnic, F. S. Search for Related Content PubMed PubMed Citation Articles by Matheny, M. E. Articles by Resnic, F. S. Social Bookmarking                         What's this?

Validation of an Automated Safety Surveillance System with Prospective, Randomized Trial Data

Division of General Medicine, Brigham & Women's Hospital, Boston, MA, michael.matheny{at}vanderbilt.edu, Decision Systems Group, Department of Radiology, Brigham & Women's Hospital, Boston, MA

David A. Morrow, MD, MPH

TIMI Study Group, Brigham & Women's Hospital, Boston, MA, Division of Cardiology

Lucila Ohno-Machado, MD, PhD

Division of Health Sciences & Technology, Massachusetts Institute of Technology, Cambridge, MA, Decision Systems Group, Department of Radiology, Brigham & Women's Hospital, Boston, MA

Christopher P. Cannon, MD

TIMI Study Group, Brigham & Women's Hospital, Boston, MA, Division of Cardiology, Brigham & Women's Hospital, Boston, MA

Marc S. Sabatine, MD, MPH

TIMI Study Group, Brigham & Women's Hospital, Boston, MA, Division of Cardiology, Brigham & Women's Hospital, Boston, MA

Frederic S. Resnic, MD, MS

Division of Cardiology, Brigham & Women's Hospital, Boston, MA, Decision Systems Group, Department of Radiology, Brigham & Women's Hospital, Boston, MA

Objective. We sought to validate 3 methods for automated safety monitoring by evaluating clinical trials with elevated adverse events. Methods. An automated outcomes surveillance system was used to retrospectively analyze data from 2 randomized, TIMI multicenter trials. Trial A was stopped early due to elevated 30-day mortality rates in the intervention arm. Trial B was not stopped early, but there was transient concern regarding 30-day intracranial hemorrhage rates. We compared statistical process control (SPC), logistic regression risk adjusted SPC (LR-SPC), and Bayesian updating statistic (BUS) methods with a standard prospective 2-arm event rate analysis. Each method compares observed event rates to alerting boundaries established with previously collected data. In this evaluation, the control arms approximated prior data, and the intervention arms approximated the observed data. Results. Trial A experienced elevated 30-day mortality rates beginning 7 months after the start of the trial and continuing until termination at month 14. Trial B did not experience elevated major bleeding rates. Combining the alerting performance of each method across both trials resulted in sensitivities and specificities of 100% and 85% for SPC, 0% and 100% for BUS, and 100% and 93% for both LR-SPC models, respectively. Conclusion. Both SPC and LR-SPC methods correctly identified the majority of months during which the cumulative event rates were elevated in trial A but were susceptible to false positive alerts in trial B. The BUS method did not result in any alerts in either trial and requires revision.

Key Words: risk adjustment • risk stratification • decision support techniques • cardiology.

This version was published on March 1, 2009

Medical Decision Making, Vol. 29, No. 2, 247-256 (2009)
DOI: 10.1177/0272989X08327110


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?