Author + information
- Received September 16, 2013
- Accepted September 25, 2013
- Published online March 4, 2014.
- John A. Dodson, MD∗,
- Matthew R. Reynolds, MD, MSc†,
- Haikun Bao, PhD‡,
- Sana M. Al-Khatib, MD, MHS§,
- Eric D. Peterson, MD, MPH§,
- Mark S. Kremers, MD‖,
- Michael J. Mirro, MD¶,
- Jeptha P. Curtis, MD‡∗ (, )
- ∗Division of Aging, Department of Medicine, Brigham and Women's Hospital/Harvard Medical School, Boston, Massachusetts
- †Division of Cardiology, Lahey Hospital and Medical Center, Burlington, Massachusetts
- ‡Section of Cardiovascular Medicine, Department of Medicine, Yale University School of Medicine, New Haven, Connecticut
- §Duke Clinical Research Institute, Division of Cardiology, Department of Medicine, Duke University School of Medicine, Durham, North Carolina
- ‖Mid Carolina Cardiology, Charlotte, North Carolina
- ¶Fort Wayne Cardiology, Parkview Health System, Fort Wayne, Indiana
- ↵∗Reprint requests and correspondence:
Dr. Jeptha P. Curtis, Section of Cardiovascular Medicine, Yale University School of Medicine, 333 Cedar Street, New Haven, Connecticut 06520.
Objectives To better inform patients and physicians of the expected risk of adverse events and to assist hospitals' efforts to improve the outcomes of patients undergoing implantable cardioverter-defibrillator (ICD) implantation, we developed and validated a risk model using data from the NCDR (National Cardiovascular Data Registry) ICD Registry.
Background ICD prolong life in selected patients, but ICD implantation carries the risk of periprocedural complications.
Methods We analyzed data from 240,632 ICD implantation procedures between April 1, 2010, and December 31, 2011 in the registry. The study group was divided into a derivation (70%) and a validation (30%) cohort. Multivariable logistic regression was used to identify factors associated with in-hospital adverse events (complications or mortality). A parsimonious risk score was developed on the basis of beta estimates derived from the logistic model. Hierarchical models were then used to calculate risk-standardized complication rates to account for differences in case mix and procedural volume.
Results Overall, 4,388 patients (1.8%) experienced at least 1 in-hospital complication or death. Thirteen factors were independently associated with an increased risk of adverse outcomes. Model performance was similar in the derivation and validation cohorts (C-statistics = 0.724 and 0.719, respectively). The risk score characterized patients into low- and-high risk subgroups for adverse events (≤10 points, 0.3%; ≥30 points, 4.2%). The risk-standardized complication rates varied significantly across hospitals (median: 1.77, interquartile range 1.54, 2.14, 5th/95th percentiles: 1.16/3.15).
Conclusions We developed a simple model that predicts risk for in-hospital adverse events among patients undergoing ICD placement. This can be used for shared decision making and to benchmark hospital performance.
Randomized clinical trials have demonstrated that implantable cardioverter-defibrillators (ICD) prolong life in patients at high risk of sudden cardiac death (SCD) (1). Over the past decade, ICD implantation rates have increased. However, despite the proven benefits of these devices, their implantation is associated with a risk of procedure-related adverse events that are associated with prolonged hospital length of stay (2), higher cost (2), and decreased 6-month survival (3). To collect information about characteristics and outcomes of patients undergoing ICD implantation in routine clinical practice, the ICD Registry was developed by the American College of Cardiology NCDR (National Cardiovascular Data Registry) in partnership with the Heart Rhythm Society (4). One goal of the registry is to provide information about adverse events that can be used for quality improvement efforts (5).
Each participating hospital whose submission passes data quality thresholds receives quarterly feedback on quality metrics, including the incidence of adverse events at their hospital compared with average performance for all participating hospitals. To date, these reports have not adjusted for differences in patient or procedural characteristics between participating hospitals. As a result, currently reported variations in crude event rates may in part reflect differences in case mix rather than hospital performance. To address this limitation, we developed a risk model for adverse events (complications or mortality) that uses ICD Registry data to calculate hospitals' risk standardized complication rates. Hospitals can use this information to identify whether their performance is above or below average and to target quality improvement efforts accordingly. In addition, the model provides a mechanism for providing individualized estimates of procedural risk that can be used to facilitate shared decision making and to determine the appropriate level of care following the procedure.
Details of the ICD Registry have been previously described (4). In brief, the registry currently collects data on over 90% of ICD placed in the United States (6). Information about patients undergoing ICD implantation is collected using standardized definitions and submitted by participating hospitals to the ICD Registry via a secure website. Submitted data then undergo quality checks and are returned to hospitals for cleaning and resubmission if they do not pass criteria for completeness (7). For the current study, we used version 2 data, which were collected beginning in April 1, 2010. Full elements are available on the NCDR website (5). For our study group, we included all patients undergoing a new ICD implantation or an ICD replacement from April 1, 2010 to December 31, 2011 with the exclusion of those undergoing epicardial lead placement, lead-only procedures, and any procedure involving lead extraction.
Selection of candidate variables
To determine the optimal outcomes to include in the risk model, a working group was established (S.A., J.C., J.D., M.K., M.M., E.P., M.R.) and it met on a bimonthly basis. The group reviewed all complications collected by the registry and excluded those whose occurrence was either exceedingly rare (venous obstruction: 0.03%, conduction block: 0.02%, peripheral embolus: 0.01%, valve injury: <0.01%, peripheral nerve injury: <0.01%), or unlikely to be related to the procedure (drug reaction: 0.05%). On the basis of the working group's consensus, the final primary outcome (any adverse event) consisted of a composite measure of procedure-related complications including any of the following: cardiac arrest; cardiac perforation; coronary venous dissection; hemothorax; device-related infection; lead dislodgement; myocardial infarction; pericardial tamponade; pneumothorax; stroke/transient ischemic attack; urgent cardiac procedure; hematoma; or set screw problem. Given its importance, mortality was included in the composite endpoint with the understanding that it may not always have been directly attributable to the procedure.
For candidate covariates in risk model development, we initially screened all relevant demographic, clinical, and procedural characteristics available from the registry. The working group determined that certain covariates would be excluded for the following reasons: potential collinearity with other covariates (diastolic blood pressure, ischemic heart disease); low frequency (post-transplant, syndrome with high risk of SCD); clinical judgment of factors deemed unlikely to influence complication risk (PR-interval duration, family history of SCD); subjective reporting (life expectancy <1 year); as well as variables that could reflect potential disparities in care (race, insurance status). To assess the potential impact of excluding race from the risk model, we performed a sensitivity analysis that included race and found that it did not significantly change our results.
We present continuous variables as mean ± SD and categorical variables as percentages. For continuous variables, we examined their distribution and established clinically relevant cut points reflecting their association with adverse events. Bivariate comparisons of patients with and without the primary outcome were performed using the t test (continuous variables) and the chi-square test (categorical variables).
Data were missing for <1% of variables except for cardiac arrest (1.5%), laboratory values (glomerular filtration rate: 1.3%, potassium: 1.4%, sodium: 1.6%, blood urea nitrogen: 1.8%, hemoglobin: 2.3%), and left ventricular ejection fraction (9.0%). For cardiac arrest, we assumed “not present” if missing. We imputed glomerular filtration rate to the sex-specific median consistent with previous NCDR models (8). Missing data for left ventricular ejection fraction were imputed to the median, and we added a dummy variable as missing indicator. Missing continuous laboratory values were imputed to the median.
We then randomly split our sample into derivation and validation cohorts (70% derivation/30% validation). In the derivation cohort, we developed a risk model using logistic regression with backward selection of candidate variables. We generated odds ratios (OR) with 95% confidence intervals (CI) to determine the strength of association for covariates that remained significant. We then evaluated model discrimination in the derivation and validation cohorts using C-statistics. For validation, we applied coefficients of the models from the derivation cohort to the validation cohort assessing the predicted versus observed rate of adverse events within deciles of predicted adverse event risk. We also generated C-statistics for primary versus secondary ICD in the derivation cohort, and in 3 other clinically meaningful subgroups: 1) first-time ICD implantations (in patients without previous ICD or pacemaker); 2) previous ICD excluding those undergoing generator replacements; and 3) cardiac resynchronization therapy with defibrillation (CRT-D) devices.
For the purposes of clinical application, we developed a parsimonious risk score that could be used to calculate patient risk at the bedside. To develop the score, we removed variables from the full model until the adjusted R2 of the parsimonious model was 95% of the full model. The performance of the parsimonious model was evaluated in the derivation and validation cohorts by C-statistics and validation plots. We also evaluated model discrimination and calibration in 3 clinically meaningful subgroups (as with the full model): first-time ICD recipients, previous ICD excluding those undergoing generator replacements, and CRT-D devices. We based the risk score system on the beta coefficients for the risk factors in the parsimonious model. Categorical variables were assigned numeric values proportional to their associated beta coefficients. Continuous variables of the risk score model were classified into several categories and these categories were assigned numeric values proportional to the product of the beta coefficient and the distance from the base category to these categories (9). For each patient, the ICD complication risk score was calculated as the simple arithmetic sum of point values assigned to each risk factor. We calculated the adverse event rates observed within the population based on risk score values by intervals of 10 in both the derivation and validation cohorts to evaluate the performance of risk scores.
Data from derivation and validation cohorts were then combined to calculate risk-standardized complication rates. We used hierarchical logistic regression models to account for clustering of patient admissions within hospitals (10,11). The assumption with this approach is that after adjusting for patient risk factors, the remaining variation is attributable to hospital-level factors. The predicted number of complications at each hospital was estimated given its patient mix and using its own hospital-specific intercept, and the expected number of complications in each hospital was estimated using its patient mix and average hospital-specific intercept on the basis of all hospitals in the sample. The risk-standardized complication rate for each hospital was computed by the ratio of number of predicted complications to the number of expected complications, multiplied by the unadjusted overall complication rate. We computed a 95% interval estimate of the risk-standardized complication rate to characterize the level of uncertainty around the point estimate using bootstrapping simulation. The risk-standardized complication rates and associated interval estimates can be used to characterize and compare hospitals' performance with the registry average. As an illustrative example, we used the risk-standardized complication rate and interval estimates to characterize hospital performance as better than registry average, no different than the registry average, and worse than registry average.
A p value of <0.05 was considered statistically significant for all tests. Analyses were performed using SAS (version 9.2, SAS institute, Cary, North Carolina).
During the study period, 263,284 procedures were performed. Procedures involving lead-only placement (n = 9,855, 3.7%), epicardial lead placement (n = 5,677, 2.2%), and lead extraction (n = 7,120, 2.9%) were excluded, leaving an analytic sample of 240,632 procedures. Of these, the majority (59.5%) represented first-time ICD implantations. Over three-quarters of implantations (76.6%) were for primary prevention indications. Mean age of the study group was 67.3 years, and 27.3% were women. The most common medical conditions were hypertension (78.3%), previous myocardial infarction (50.7%), heart failure (42.4%), diabetes (37.9%), and chronic lung disease (21.6%). The most common device type was CRT-D (42.5%), followed by dual-chamber ICD (38.3%), then single-chamber ICD (19.0%).
Overall, 4,388 patients (1.8%) experienced an adverse event. The characteristics of patients with and without adverse events are shown in Table 1. On average, patients who experienced an adverse event were older (68.4 vs. 67.3 years, p < 0.0001), more often female (33.2% vs. 27.2%, p < 0.0001), and less likely to have “ICD implantation” listed as the primary reason for admission (52.7% vs. 73.6%, p < 0.0001). Patients with adverse events were also more likely to have experienced a hospital stay for heart failure within the last 6 months (25.4% vs. 15.6%, p < 0.0001).
The individual in-hospital adverse events (complications or mortality) are listed in Table 2. The most common events were lead dislodgement (n = 1,580, 0.66%), hematoma (n = 723, 0.30%), and pneumothorax (n = 573, 0.24%). In-hospital death occurred in 637 patients (0.26%).
The full model for adverse events after ICD implantation included 21 variables (Table 3). The greatest strength of association was seen with procedure type. Patients undergoing an initial ICD implantation were over 3 times as likely to experience complications as those undergoing a generator replacement for end of battery life (OR: 3.57, 95% CI: 3.13 to 4.08), and placement of either a CRT-D or dual-chamber device were more likely to experience complications compared with single-chamber devices (CRT-D vs. single-chamber: OR: 1.73, 95% CI: 1.51 to 1.98; dual-chamber vs. single-chamber: OR: 1.45, 95% CI: 1.28 to 1.64). Increasing severity of heart failure was also strongly associated with complications (New York Heart Association functional class IV vs. class I/II: OR: 2.01, 95% CI: 1.71 to 2.36). The model demonstrated good discrimination in both the derivation and validation cohorts (C-statistics: 0.724 and 0.722 for derivation and validation cohorts, respectively), and for primary and secondary prevention subgroups in the derivation cohort (C-statistics: 0.713 and 0.763, respectively).
Among first-time ICD implantation patients (no previous ICD or pacemaker, n = 128,749) and patients with a previous ICD, excluding those undergoing generator replacements (n = 30,630), model discrimination was lower (C-statistics: 0.666 and 0.639, respectively). Among the subgroup of patients receiving CRT-D devices (n = 102,279), the C-statistic was 0.703.
The parsimonious risk model retained 12 characteristics from the full model (Table 4). The parsimonious model had similar discrimination (C-statistics: 0.721 and 0.718 for derivation and validation cohorts, respectively). A point system was developed on the basis of regression coefficients. The highest number of points was attributable to procedure type; for example, a patient received 18 points for undergoing device relocation (compared with 0 points for an end-of-battery-life generator replacement). The distribution of risk scores among the study group is shown in Figure 1. On the basis of this distribution, patients with a risk score of 10 or less were at very low risk of complications (0.3%). In contrast, patients with a risk score of 30 or more had a considerably increased risk of complications (4.2%) (Fig. 2).
The distribution of risk-standardized complication rates among participating hospitals (n = 1,428) is shown in Figure 3. The median risk-standardized complication rate was 1.77 (IQR: 1.54 to 2.14, 95% CI: 1.16 to 3.15, respectively). Compared with the registry average, 54 hospitals (3.8%) were worse than expected, and 15 hospitals (1.1%) were better than expected.
We developed a risk model to predict in-hospital adverse events after ICD implantation using clinical characteristics that are readily available at the time of procedure. Our efforts had 2 purposes. First, the risk model can be used to benchmark hospital complication rates and therefore can be a useful tool in quality improvement efforts. Second, the risk score derived from the model can be used to facilitate shared decision making with patients by incorporating a subjects expected risks of ICD implantation.
Our model provides adverse event rates that are risk standardized (risk-standardized complication rates) that we believe will have higher clinical significance for participating sites. To date, hospitals submitting data to the ICD Registry have received quarterly feedback reports listing their crude ICD complication rates compared with average rates for participating hospitals, and this approach has failed to account for differences in the case mix. Sites may therefore dismiss higher crude complication rates as due to patient complexity rather than physician or system-wide factors that may be targeted for quality improvement. The use of risk-standardized complication rates in future reports should have higher face validity and allow a more compelling argument for internal evaluation of the quality of care among sites that are low performing.
The strongest risk factors for adverse events in our model were procedure type (any procedure other than simple generator replacement), ICD type (other than single lead), heart failure severity, renal dysfunction, and nonelective admission. Using our risk model, a patient with all 5 risk factors (e.g., undergoing initial device implantation, dual-chamber device, New York Heart Association functional class IV, blood urea nitrogen = 45, admission for primary reason other than device placement) would have an expected complication risk of at least 4%. Conversely, other patients (e.g., elective admission for a single-lead ICD with mildly symptomatic heart failure and normal renal function) would have an expected complication risk of <1%. Our model may therefore be used for informed discussions between clinicians and patients about the individualized risk of ICD implantation. In addition, the model may potentially be used to identify particularly low-risk patients who may require less intensive post-procedure monitoring (e.g., same-day discharge).
Several other findings are worthy of note. Our observed adverse event rate (1.8%) was lower than in most previous studies, including those using ICD Registry data (12–14). For example, a study of 268,701 ICD Registry patients undergoing procedures from 2006 to 2008 found a complication rate of 3.2% (12). Older studies generally reported even higher in-hospital complication rates—up to 10% in several reports (13,14). The low rate of adverse events in our study sample may represent differences between our datasets and others (i.e., some previous studies using claims data rather than ICD Registry data), or a secular trend toward improved in-hospital outcomes among ICD recipients over time. We do not think our low adverse event rate would negatively affect the performance of our risk model; however, because data are self-reported from sites, with the potential for under-reporting of complications at certain hospitals, we believe it is premature for our risk-standardized complication rates to be used for public reporting. However, the information can be used by hospitals to promote internal quality improvement efforts to improve the outcomes associated with their ICD implantations.
An additional finding of interest is that patient age did not enter into our parsimonious risk model, as its strength of association with adverse events was weak. On the basis of our investigation and others (12,13,15), age does not appear to be a strong predictor of in-hospital adverse events. Therefore, older patients with few comorbidities may be expected to have reasonably good post-procedural outcomes.
Our results were limited to in-hospital adverse events, and therefore our model is unable to predict risk of post-discharge complications (such as device infection) that may require rehospitalization. Also, as some hospitals do not submit data to the registry, or submit poor quality data, the in-hospital adverse event rate in practice may be higher than was reported in our study sample, and there may be risk factors for adverse events that we were unable to identify. In addition, our model only provides information about the expected risk of device implantation, not potential benefits. Nevertheless, we believe this information will prove useful in promoting shared decision making. Another potential barrier to widespread use of the risk score is that there are 12 covariates in our parsimonious model, which is more than several other risk scores. Nevertheless, these characteristics are generally readily available on all patients undergoing ICD implantation. Furthermore, the growth of electronic medical records may facilitate the calculation of a risk score by identifying risk factors in an automated fashion. Finally, given that data are self-reported, enhanced efforts to verify complication rates with external chart audits of participating hospitals may improve the validity of our observations.
We developed a simple model that predicts risk for in-hospital adverse events among patients undergoing ICD placement. This can be used for both shared decision making and to benchmark hospital performance for quality improvement efforts.
This research was supported by the American College of Cardiology Foundation's NCDR (National Cardiovascular Data Registry). The views expressed in this manuscript represent those of the authors and do not necessarily represent the official views of the NCDR or its associated professional societies identified at www.ncdr.com. Dr. Dodson was supported by a Training Grant in Epidemiologic Research on Aging (#T32 AG000158-24) from the National Institutes of Health/National Institute on Aging and a Clinical Research Loan Repayment Award from the National Institutes of Health/National Heart, Lung, and Blood Institute during the writing of this manuscript and is currently supported by the National Institute on Aging grant (#R03AG045067) and a T. Franklin Williams Scholarship Award (funding provided by: Atlantic Philanthropies, Inc.; the John A. Hartford Foundation; the Alliance for Academic Internal Medicine-Association of Specialty Professors; and the American College of Cardiology). Dr. Reynolds has received consulting fees from Medtronic. Dr. Peterson has received honoraria from Janssen, Eli Lilly, and Boehringer Ingelheim. Dr. Kremers has received consulting fees from Medtronic; has served on the Speakers' Bureau of Boston Scientific; and owns equity in Boston Scientific. Dr. Mirro has received consulting fees from Zoll Medical, iRhythm, and McKesson; and has received grant support from St. Jude Medical. Dr. Curtis receives salary support from the American College of Cardiology National Cardiovascular Data Registry. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose. ICD Registry is an initiative of the American College of Cardiology Foundation and the Heart Rhythm Society.
- Abbreviations and Acronyms
- confidence interval(s)
- cardiac resynchronization therapy with defibrillation
- implantable cardioverter-defibrillator(s)
- odds ratio(s)
- sudden cardiac death
- Received September 16, 2013.
- Accepted September 25, 2013.
- American College of Cardiology Foundation
- Reynolds M.R.,
- Cohen D.J.,
- Kugelmass A.D.,
- et al.
- Lee D.S.,
- Krahn A.D.,
- Healey J.S.,
- et al.,
- for the Investigators of the Ontario ICD Database
- ↵NCDR ICD Registry. Available at: https://www.ncdr.com/webncdr/icd/home/datacollection. Accessed December 12, 2013.
- Messenger J.C.,
- Ho K.K.,
- Young C.H.,
- et al.,
- for the NCDR Science and Quality Oversight Committee Data Quality Workshop
- Mehta S.K.,
- Frutkin A.D.,
- Lindsey J.B.,
- et al.,
- for the National Cardiovascular Data Registry
- Krumholz H.M.,
- Wang Y.,
- Mattera J.A.,
- et al.
- Curtis J.P.,
- Geary L.L.,
- Wang Y.,
- et al.
- Haines D.E.,
- Wang Y.,
- Curtis J.
- Al-Khatib S.M.,
- Greiner M.A.,
- Peterson E.D.,
- Hernandez A.F.,
- Schulman K.A.,
- Curtis L.H.
- Mandawat A.,
- Curtis J.P.,
- Mandawat A.,
- Njike V.Y.,
- Lampert R.