Author + information
- Received April 7, 2016
- Accepted May 9, 2016
- Published online August 9, 2016.
- Robert L. McNamara, MD, MHSa,∗ (, )
- Kevin F. Kennedy, MSb,
- David J. Cohen, MD, MScc,
- Deborah B. Diercks, MD, MScd,
- Mauro Moscucci, MD, MBAe,f,
- Stephen Ramee, MDg,
- Tracy Y. Wang, MD, MHS, MSch,
- Traci Connolly, BSN, MSi and
- John A. Spertus, MD, MPHc
- aYale University School of Medicine, New Haven, Connecticut
- bMid America Heart Institute, Kansas City, Missouri
- cSaint-Luke’s Mid America Heart Institute and University of Missouri–Kansas City School of Medicine, Kansas City, Missouri
- dUniversity of Texas Southwestern Medical Center, Dallas, Texas
- eSinai Hospital of Baltimore, Baltimore, Maryland
- fUniversity of Michigan Health System, Ann Arbor, Michigan
- gOchsner Medical Center, New Orleans, Louisiana
- hDuke University Medical Center and Duke Clinical Research Institute, Durham, North Carolina
- iAmerican College of Cardiology, Washington, DC
- ↵∗Reprint requests and correspondence:
Dr. Robert L. McNamara, Department of Internal Medicine, Yale University School of Medicine, PO Box 208017, New Haven, Connecticut 06520-8017.
Background As a foundation for quality improvement, assessing clinical outcomes across hospitals requires appropriate risk adjustment to account for differences in patient case mix, including presentation after cardiac arrest.
Objectives The aim of this study was to develop and validate a parsimonious patient-level clinical risk model of in-hospital mortality for contemporary patients with acute myocardial infarction.
Methods Patient characteristics at the time of presentation in the ACTION (Acute Coronary Treatment and Intervention Outcomes Network) Registry–GWTG (Get With the Guidelines) database from January 2012 through December 2013 were used to develop a multivariate hierarchical logistic regression model predicting in-hospital mortality. The population (243,440 patients from 655 hospitals) was divided into a 60% sample for model derivation, with the remaining 40% used for model validation. A simplified risk score was created to enable prospective risk stratification in clinical care.
Results The in-hospital mortality rate was 4.6%. Age, heart rate, systolic blood pressure, presentation after cardiac arrest, presentation in cardiogenic shock, presentation in heart failure, presentation with ST-segment elevation myocardial infarction, creatinine clearance, and troponin ratio were all independently associated with in-hospital mortality. The C statistic was 0.88, with good calibration. The model performed well in subgroups based on age; sex; race; transfer status; and the presence of diabetes mellitus, renal dysfunction, cardiac arrest, cardiogenic shock, and ST-segment elevation myocardial infarction. Observed mortality rates varied substantially across risk groups, ranging from 0.4% in the lowest risk group (score <30) to 49.5% in the highest risk group (score >59).
Conclusions This parsimonious risk model for in-hospital mortality is a valid instrument for risk adjustment and risk stratification in contemporary patients with acute myocardial infarction.
Mortality from cardiovascular disease has decreased dramatically over the past few decades (1), in part because of improvements in acute myocardial infarction (AMI) management (2). In-hospital mortality has decreased from 29% in 1969 (3) to <7% today (4,5). However, more than 100,000 people continue to die after AMIs in the United States each year (1), and in-hospital mortality varies substantially across hospitals (5), suggesting an opportunity for improvement. Adjustment for the variation in patient risk across hospitals is essential to enable a more accurate assessment of each hospital’s performance and opportunity to improve.
Although many risk models of in-hospital mortality have been developed for patients with AMI (6–13), few have included a representative sample from routine clinical care. In 2011, a simple, validated risk model was developed using data from the ACTION (Acute Coronary Treatment and Intervention Outcomes Network) Registry–GWTG (Get With the Guidelines), which included patients from more than 300 hospitals (14). Since that time, ACTION Registry–GWTG collection has been expanded to identify patients presenting after cardiac arrest at the time of AMI presentation. Being able to adjust for cardiac arrest is critical because it is a well-documented predictor of mortality (10,15). Moreover, continued improvement in AMI care mandates periodic updates to the risk models so that hospitals can assess their quality as contemporary care continues to evolve.
To update the existing ACTION–GWTG mortality risk model, we rebuilt the ACTION Registry–GWTG in-hospital mortality risk model using data from January 2012 through December 2013. We also sought to build a parsimonious risk score that could be used prospectively for risk stratification. These tools are designed to be used to further support quality improvement and to aid in clinical management during an AMI.
ACTION Registry–GWTG is a voluntary, hospital-based registry that receives data on consecutive patients admitted with AMI, either ST-segment elevation myocardial infarction (STEMI) or non–ST-segment elevation myocardial infarction (NSTEMI), from participating hospitals across the United States. The ACTION Registry–GWTG design and methods have been described previously (16). Briefly, participating hospitals collect data through retrospective chart review using standardized data collection tools that do not require direct patient contact. Collected data include patient demographics; presenting features; pre-hospital, in-hospital, and hospital discharge therapy; timing of treatments; laboratory tests; procedures; and in-hospital outcomes. On the basis of individual site determinations, this registry was either approved by an Institutional Review Board or considered a quality assurance effort and thus not subject to Institutional Review Board approval (16). The National Cardiovascular Data Registry also has a data quality program, including data abstraction training, data quality thresholds for inclusion, site data quality feedback reports, independent auditing, and data validation (17). Data auditing has demonstrated accurate representation with agreement with chart review of 93% (18).
Between January 2012 and December 2013, a total of 254,066 patients with AMI (NSTEMI and STEMI) from 665 participating hospitals were included in the registry (Figure 1). To ensure more complete outcome data, patients were excluded if they were transferred out of participating hospitals (n = 10,626), leaving a final analytic sample of 243,440 patients. This cohort was randomly divided into a derivation cohort (60% [n = 145,952]) and a validation cohort (40% [n = 97,288]).
Standard definitions have been established for the data elements captured in the ACTION Registry–GWTG database. Mortality was defined as all-cause mortality during hospitalization. Of particular relevance to this updated analysis, cardiac arrest was defined as “evaluated by EMS (Emergency Medical System) or ED (Emergency Department) personnel and either: 1) received attempts at external defibrillation (by lay responders or emergency personnel) or chest compressions by organized EMS or ED personnel; or 2) were pulseless at the time of presentation.” Vital signs were determined at the time of first medical presentation. Baseline creatinine clearance was estimated using the Cockcroft-Gault formula (19). Baseline troponin ratio was defined as the baseline troponin value divided by the local laboratory-specific upper limit of normal. This approach accounted for the different local laboratory troponin assays using different reference ranges and has been used previously to investigate the association of maximum troponin ratio with outcomes (20). Electrocardiograms at presentation were interpreted locally.
The study cohort was divided into derivation and validation cohorts by random number generation from a uniform (0,1) distribution. Initial candidate variables for the model were identified from the list of all ACTION Registry–GWTG data elements that would be known at the time of initial hospital presentation (variables in Table 1). Data are expressed as mean ± SD for continuous variables and as number (percentage) for categorical variables. Extreme values for continuous variables were set to outer limits on the basis of clinical judgment.
The unadjusted association between each candidate variable and in-hospital mortality was tested with Student t tests for continuous variables and chi-square tests for categorical variables. Hierarchical logistic regression was used with site as a random effect to generate the risk model from the selected variables, with odds ratios and 95% confidence intervals shown. To establish a parsimonious model, a backward selection process using the variables listed in Table 1 was performed until 90% of the full model R2 value was retained (21). Discrimination and calibration in the validation cohort were then tested by computing the C statistic and calibration slope and intercept, with a slope of 1 and an intercept of 0 indicating perfect calibration. The computed risk (on the logit scale) was used as a predictor variable in the validation cohort, and the slope and intercept were tested on the model (22). The final model is based on the derivation cohort only. Model performance in pre-specified subgroups was also examined. Finally, an in-hospital risk score was created by assigning weighted values to the variables identified by the final model. The risk score was then calculated by adding all the individual weighted values. All analyses were performed in SAS version 9.4 (SAS Institute, Cary, North Carolina).
This research was conducted in compliance with federal guidelines, including the Common Rule (45 CFR 46). Chesapeake Research Review served as the internal review board. ACTION Registry–GWTG has submitted a protocol to the internal review board and has been granted a waiver of informed consent.
In terms of baseline characteristics for the derivation (n = 145,952) and validation (n = 97,288) cohorts (Table 1), no important differences were observed between the 2 groups. Selected in-hospital management is shown in Table 2.
The bivariate relationships between patient characteristics and in-hospital mortality are shown for continuous variables in Table 3 and for categorical variables in Table 4. In multivariate analysis (Table 5), 9 variables were independently associated with in-hospital mortality: age; presenting heart rate and systolic blood pressure; presentation after cardiac arrest, in cardiogenic shock, in heart failure, and with STEMI; creatinine clearance; and troponin ratio.
The final ACTION Registry–GWTG in-hospital mortality model had high discrimination in both the derivation and validation populations, with a C statistic of 0.88 for both. There was also excellent calibration of the model in the validation cohort (Figure 2), with a slope of 0.996 (p = 0.74) and an intercept of −0.025 (p = 0.42). Additionally, the model performed well in various important subgroups, including patients with STEMI (Figure 3A) or NSTEMI (Figure 3B) as well as patients with (Figure 3C) and without (Figure 3D) cardiac arrest. Other clinically important subgroups are shown in Table 6.
The Central Illustration provides a simple integer score, based on the final risk model, that can be calculated for prospective risk stratification soon after patient presentation. The Central Illustration also shows mortality risk in groups on the basis of risk score in both the derivation and validation cohorts. The observed mortality rates in patients with risk scores <30, 30 to 39, 40 to 49, 50 to 59, and >59 were 0.4%, 1.7%, 5.5%, 18.5%, and 49.5%, respectively.
Using data collected from 243,440 consecutive patients presenting at 665 hospitals participating in the national ACTION Registry–GWTG from January 2012 through December 2013, we developed and validated a contemporary risk model to predict in-hospital mortality for patients after AMI. The final parsimonious model included age, heart rate, systolic blood pressure, presentation after cardiac arrest, presentation in cardiogenic shock, presentation in heart failure, presentation with STEMI, creatinine clearance, and troponin ratio as factors for risk adjustment. The model performed well in an independent validation cohort, as well as in various subgroups stratified by cardiac arrest and other clinical factors. A simplified integer score based on this model also performed well and can potentially serve as a foundation for prospective risk stratification at the point of care.
This work built upon and extended prior mortality risk models developed for patients with AMI. Although valuable for the selected populations at the time of their original development, changes in patient profiles and AMI management demand updating these models for accurate comparisons across hospitals and for prospective risk stratification. For instance, the GRACE (Global Registry of Acute Coronary Events) score was developed on nonconsecutive patients (10) in select international clinical sites, and the TIMI (Thrombolysis In Myocardial Infarction) and GUSTO (Global Use of Strategies to Open Occluded Arteries) scores were developed in clinical trial populations of patients with STEMI (21,22) or NSTEMI or unstable angina (8). In addition, since the creation of these models, significant advances have been made in the diagnosis and care of patients with AMI. Moreover, the publicly reported measure for 30-day AMI mortality used by the Centers for Medicare and Medicaid Services incorporates administrative rather than clinical data, is restricted to patients older than 65 years (23), and is not amenable to clinical use. Given the inherent differences in the populations and goals of these models, the present ACTION Registry–GWTG model was not tested against these prior models.
The new risk model compared favorably with a previous risk model developed using ACTION Registry–GWTG data from 2007 and 2008 (14), which has been subsequently used for quality feedback to participating hospitals. Many of the data elements used for risk adjustment were identical, including age, presenting systolic blood pressure, and troponin ratio. Not surprisingly the main difference in risk adjustment for the new model was the ability to include presentation after cardiac arrest, which was not available at the time the previous model was created. Splines and interactions were no longer significant, resulting in a simpler model for prospective use. Other subtle differences between the present and previous models include the use of creatinine clearance rather than serum creatinine level (which was less predictive) and the separation of heart failure and cardiogenic shock at the time of presentation, which were both independently associated with in-hospital mortality.
Cardiac arrest has been shown to be an important predictor of AMI mortality in multiple previous studies (10,15,24–26). The National Cardiovascular Data Registry CathPCI risk model (27) includes these patients in the cohort, and presentation after cardiac arrest is an important factor in risk adjustment. However, inclusion of patients with cardiac arrest in mortality comparisons in the setting of percutaneous coronary intervention has been controversial, as many believe that the models are inadequate to fully adjust for the risk for these events, given their heterogeneity in clinical severity, and inclusion of these patients in hospital scorecards for percutaneous coronary intervention can result in unintended consequences to withhold aggressive treatment (28–30). Because our model was developed for all patients with AMI, not only those taken to the cardiac catheterization laboratory, this consideration is less relevant. In fact, inclusion of these patients may drive more appropriate care, because their outcomes are clearly poor without acute revascularization, and more aggressive hospitals will have better risk-adjusted outcomes if their use of revascularization improves survival of these patients.
The new ACTION Registry–GWTG mortality risk model discriminated very well in both the derivation and validation cohorts (C statistic = 0.88), results that compare favorably with those from the prior model (C statistic = 0.85 and 0.84, respectively). Furthermore, these C statistics align well with other clinical risk models in patients with AMI (8,10,31,32). Importantly, the model performed well across a broad range of patient subgroups, including those presenting after cardiac arrest. The C statistics are high in this model in part because overall mortality is relatively low, except for some patients with very high risk characteristics, such as presentation in cardiogenic shock and presentation with cardiac arrest. Thus, the model may be more useful for benchmarking mortality outcomes, rather than prospective clinical decision making, given that it is often not difficult to identify patients with these high-risk characteristics.
Although this model represents a robust, parsimonious approach to contemporary risk adjustment methodology for in-hospital mortality after AMI, it should be considered in the context of some potential limitations. First, ACTION Registry–GWTG is a voluntary registry; the contributing hospitals tend to be larger referral centers and are more likely to have PCI capabilities than the average U.S. hospital. In-hospital outcomes at these centers may not be generalizable to all hospitals caring for these patients.
Second, patients transferred from 1 hospital to another present a question of outcome attribution. For this model, patients transferred from other hospitals were included, and outcomes were attributed to the receiving hospital. It is reassuring that the model performed similarly in these patients compared with patients not transferred in. Patients transferring out of the participating hospital were excluded, as outcome ascertainment was not possible. Patients transferred out were a small minority (3.3%) of the overall study population, and their exclusion seems unlikely to have affected the model substantially.
Third, the candidate risk adjustment variables were limited to those available in ACTION Registry–GWTG. In particular, incorporating more information on patients presenting after cardiac arrest, such as whether the arrest was witnessed and presenting neurological status, would likely create a more robust model. In addition, including more information regarding patient baseline health status, such as frailty, and other acute noncardiac conditions, such as pneumonia, would increase robustness as these are likely to be strongly associated with mortality (33).
Fourth, ACTION Registry–GWTG only assesses in-hospital outcomes. Fixed-time outcomes, such as 30-day mortality, would not depend on length of stay, which is a potential confounder.
Finally, the model has been validated using only ACTION Registry–GWTG data; it has not been validated on an external dataset.
The new ACTION Registry–GWTG in-hospital mortality risk model and risk score represent robust, parsimonious, and contemporary risk adjustment methodology for use in routine clinical care and hospital quality assessment. The addition of risk adjustment for patients presenting after cardiac arrest is critically important and enables a fairer assessment across hospitals with varied case mix. This new model should enable improved assessment of hospital quality and enhance research into best practices to further reduce mortality in patients with AMI.
COMPETENCY IN PRACTICE-BASED LEARNING AND IMPROVEMENT: In patients with AMI, older age, high heart rate, low systolic blood pressure, cardiac arrest, cardiogenic shock, heart failure, ST-segment elevation, low creatinine clearance, and high troponin ratio were independently associated with in-hospital mortality.
TRANSLATIONAL OUTLOOK: This risk model for in-hospital mortality could facilitate comparisons of hospital performance and evaluate interventions to improve the outcomes of patients with AMI.
Dr. Cohen has received research grant support from Abbott Vascular, AstraZeneca, Biomet, Boston Scientific, Cardiovascular Systems, Inc., Daiichi-Sankyo, Edwards Lifesciences, Eli Lilly, Medtronic, Merck; has received consulting income from Edwards Lifesciences, Medtronic, and AstraZeneca; and speaking honoraria from AstraZeneca. Dr. Diercks has received consulting honoraria from Janssen, Novartis, and Siemens; and research grant support from Roche. Dr. Moscucci has received book royalties from Lippincott Williams and Wilkins; and owns stocks of Gilead Sciences. Dr. McNamara serves on a clinical trials endpoint adjudication committee for Pfizer. Dr. Ramee receives consulting honoraria from Edwards and has an ownership interest in Large Bore, Ocular Therapeutics, and Marvenray. Dr. Wang reports research grant support to her institution from AstraZeneca, Boston Scientific, Bristol-Myers Squibb, Eli Lilly/Daiichi-Sankyo Alliance, Gilead, GlaxoSmithKline, Regeneron; and consulting honoraria from AstraZeneca, Eli Lilly, and Premier. Dr. Spertus has received research grants and contracts from the American College of Cardiology Foundation, Eli Lilly, Gilead, and Genentech; has consulted for Novartis, Regeneron, Bayer, and Amgen; holds the copyright to the SAQ, KCCQ, and PAQ; and has an equity interest in Health Outcomes Sciences. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose. Ralph D’Agostino, MD, served as Guest Editor for this paper.
- Abbreviations and Acronyms
- acute myocardial infarction
- non–ST-segment elevation myocardial infarction
- ST-segment elevation myocardial infarction
- Received April 7, 2016.
- Accepted May 9, 2016.
- American College of Cardiology Foundation
- Mozaffarian D.,
- Benjamin E.J.,
- Go A.S.,
- et al.
- de Vreede J.J.,
- Gorgels A.P.,
- Verstraaten G.M.,
- et al.
- Kontos M.C.,
- Rennyson S.L.,
- Chen A.Y.,
- Alexander K.P.,
- Peterson E.D.,
- Roe M.T.
- Krumholz H.M.,
- Chen J.,
- Wang Y.,
- Radfod M.J.,
- Chen Y.T.,
- Marciniak T.A.
- Dorsch M.,
- Lawrance R.A.,
- Sapsford R.J.,
- et al.
- Chin C.T.,
- Chen A.Y.,
- Wang T.Y.,
- et al.
- Alahmar A.E.,
- Nelson C.P.,
- Snell K.I.,
- et al.
- Peterson E.D.,
- Roe M.T.,
- Rumsfeld J.S.,
- et al.
- Krone R.J.,
- Shaw R.E.,
- Klein L.W.,
- et al.
- Messenger J.C.,
- Ho K.K.,
- Young C.H.,
- et al.
- Harrell F.E. Jr..
- Krumholz H.M.,
- Wang Y.,
- Mattera J.A.,
- et al.
- Brennan J.M.,
- Curtis J.P.,
- Dai D.,
- et al.
- Moscucci M.,
- Eagle K.A.,
- Share D.,
- et al.
- Resnick F.R.,
- Welt F.G.P.
- Hannah E.L.
- Morrow D.A.,
- Antman E.M.,
- Charlesworth A.,
- et al.
- Califf R.M.,
- Pieper K.S.,
- Lee K.L.,
- et al.
- Lichtman J.H.,
- Spertus J.A.,
- Reid K.J.,
- et al.