Author + information
- Eric D. Peterson, MD, MPH⁎ ()
- ↵⁎Reprint requests and correspondence:
Dr. Eric D. Peterson, Duke Clinical Research Institute, 2400 Pratt Street, Durham, North Carolina 27705
- American College of Cardiology National Cardiovascular Data Registry CathPCI Registry
- hierarchical risk prediction models
- percutaneous coronary intervention
- predictive models
In today's “transparent” society, the public is increasingly demanding more information on the quality and outcomes of medical care (1,2). Most would agree that patients need more data regarding their health care providers' results, yet the devil is always in the details. For outcome comparisons to be fair and valid, they first need to be adjusted for underlying patient risks (3). For percutaneous coronary intervention (PCI), outcome metrics (e.g., acute mortality) must account for multiple potential confounding factors including patient demographics, disease severity, comorbid illness, and, most importantly, procedural acuity.
The science of outcomes risk assessment needs to continually evolve to keep up with an ever-changing medical field and care environment (4). Historically, hospital comparisons were based on claims (or billing) data. Although claims data were ubiquitous, they often proved inaccurate and lacked many critical factors needed for proper risk adjustment (5). During the past 2 decades, multicenter clinical registries evolved and spread, helping to accelerate our understanding of PCI procedural risk. In 1998, a group of 8 pioneer PCI databases were used to identify key variables associated with PCI risk (6). Since then, the majority of U.S. PCI hospitals have consolidated data submission into a single National Cardiovascular Data Registry (NCDR). The NCDR expanded the number of data elements collected, standardized their definitions, and improved their completeness and accuracy (7). The resulting larger, higher quality database has facilitated the development of more precise PCI risk models. The current NCDR risk model adjusts for >21 independent factors and accurately identifies those who will survive a PCI hospitalization and those who will die in 93% of cases (c-index 0.93) (8).
In this issue of the Journal, Resnic et al. (9) assess whether PCI risk algorithms could be further improved by refining the variables used to summarize procedural acuity. Specifically, a previous case review had demonstrated that most PCI deaths occurred among patients entering the catheterization lab in extremis (10). Although the NCDR data capture many indicators of disease acuity (e.g., ST-segment elevation myocardial infarction, heart failure, shock), some of these life-threatening factors were missed. To address this gap, Resnic et al. (9) studied the incremental impact of 3 novel risk data elements on predicting PCI mortality: 1) cardiopulmonary resuscitation; 2) patients in a coma; or 3) those with a ventricular assist device or extracorporeal bypass device. These 3 factors were combined into a single composite term entitled compassionate use (CU) PCI. Resnic et al. (9) found that while PCI was rarely performed for CU indications (0.03% of all PCI cases), these CU cases did indeed face very high mortality risks, with in-hospital mortality rates approaching 70%. They also found that the CU indicator added independent prognostic information beyond traditional clinical risk factors within Massachusetts' state PCI database.
Measuring how these data affect patient risk stratification, hospital performance assessment, and resource requirements deserves further discussion. From a statistical standpoint, the gains in risk prediction provided by the inclusion of CU in the PCI risk model were modest. Overall model discrimination (c-index) changed little (0.03), and <10% of patients were reclassified correctly to a higher (or lower) risk group when CU was added (9).
Beyond patient-level risk prediction, one also needs to consider how inclusion of the CU variable affects hospital-level risk adjustment. With regards to performance assessment, Resnic et al. (9) found individual hospital risk-adjusted mortality rates were fairly static before and after adding the CU indicator. In fact, in its first year of implementation, consideration of CU had no effect on the classification of centers rated as superior or inferior “outliers.” These findings were not surprising because centers (on average) treated ≤3 CU cases per year (9). Thus, while the risk models were incrementally improved with inclusion of CU, these changes had limited measurable impact on aggregated provider-level conclusions.
Additionally, one should consider the costs associated with CU data implementation. Human resource costs include the need to educate sites regarding the definition of CU, the personal time required for CU data collection, and the subsequent central adjudication effort needed to validate each reported CU case. For example, Resnic et al. (9) found that in Massachusetts, despite state-wide training, up to one-third of investigator-reported CU procedures were later reversed after external audit. This stringent audit system used in the state further contributed to delays in release of the outcomes information to the public. As a result, the Massachusetts PCI mortality model reports were released >2 years after the last procedures were performed (11).
Despite these potential statistical and operational challenges, the work of Resnic et al. (9) points out an extremely important consequence of provider profiling programs; namely, how provider perceptions of the PCI risk adjustment process itself can affect their subsequent clinical actions. Faced with fears of appearing on the front page of their local newspapers, interventionalists in Massachusetts may have thought twice before attempting PCI on very high-risk patients (12). In fact, Resnic et al. (9) found empirical support for this hypothesis. In the first 3 years after Massachusetts' decision to publicly report PCI mortality, the use of coronary revascularization procedures in shock patients decreased by 50%. However, after the Massachusetts Board began adjusting for CU factors, this trend reversed, and interventionalists reverted to their pre-profiling procedure levels. Although such trends will need ongoing evaluation, these initial findings are provocative indicators of the influence of profiling methodology on provider case selection.
Can the Massachusetts CU experience be generalized to the NCDR PCI database? Perhaps. Currently, the NCDR remains a voluntary registry whose goal is to provide clinicians with confidential feedback on their comparative care and outcomes. In an era without the pressures of public reporting, hospitals have limited incentive to up-code or “game” their reported PCI risk data. Looking toward the future, however, the world may be radically different. Beginning last fall, the Society of Thoracic Surgeons has agreed to publicly release hospitals' bypass surgery outcomes and care quality information to Consumer Reports (13). This decision will put pressure on the NCDR and other clinical registries to also release their data to the public. Although such increased transparency is welcomed by policy leaders who have advocated for these types of changes (1,2), it will also likely (at least temporarily) increase the anxiety of practicing clinicians. Thus, if and when professional society registries are also used as public report cards, then the requirements for data accuracy and verification will be just as high as those currently faced by Massachusetts.
Therefore, the study by Resnic et al. (9) has helped our profession prepare in multiple ways for this near-term future. First, Resnic et al. (9) have identified a rare but important set of high-risk factors not previously collected in national registries. Second, the investigators have shown that these data variables were potentially feasible to collect, providing that an external validation is in place to ensure accuracy. Third, and perhaps most important, they demonstrated that the extra data collection and validation may be considered worthwhile, given the positive impact of this process on provider behaviors.
Many have and will debate the relative merits and unintended consequences of releasing outcomes data to the public (14). Nevertheless, if such provider profiling is to be implemented, I strongly believe that this profiling must be used compassionately, in a manner that demonstrates concern for accuracy and fairness to the physicians affected by its critique. As a practicing coronary care unit physician, I rely on the skills and judgment of my interventionalist partners. When I call on my colleagues to consider taking an extremely sick patient to the laboratory, I do not want them worrying about how a case may adversely affect their outcomes score card; rather, I want them to compassionately consider what can be done to save the patient's life.
Dr. Peterson has received research grants from the American College of Cardiology, Bristol-Myers Squibb Company, Eli Lilly & Company, Johnson & Johnson, Merck & Co., Sanofi-Aventis, Society of Thoracic Surgeons, and the American Heart Association; and is a consultant to Bristol-Myers Squibb, Merck & Co., Tethysbio, and AstraZeneca.
↵⁎ Editorials published in the Journal of the American College of Cardiology reflect the views of the authors and do not necessarily represent the views of JACC or the American College of Cardiology.
- American College of Cardiology Foundation
- Krumholz H.M.,
- Brindis R.G.,
- Brush J.E.,
- et al.
- Block P.C.,
- Peterson E.D.,
- Krone R.,
- et al.
- Brindis R.G.,
- Fitzgerald S.,
- Anderson H.V.,
- Shaw R.E.,
- Weintraub W.S.,
- Williams J.F.
- Peterson E.D.,
- Dai D.,
- DeLong E.R.,
- et al.
- Resnic F.S.,
- Normand S.L.T.,
- Piemonte T.C.,
- et al.
- Resnic F.S.,
- Welt F.G.
- Massachusetts Data Analysis Center
- Califf R.M.,
- Peterson E.D.