Author + information
- Received May 20, 2012
- Revision received June 21, 2012
- Accepted July 3, 2012
- Published online October 23, 2012.
- Beau M. Hawkins, MD⁎,
- Kevin F. Kennedy, MS†,
- Jay Giri, MD, MPH⁎,
- Adam J. Saltzman, MD⁎,
- Kenneth Rosenfield, MD⁎,
- Douglas E. Drachman, MD⁎,
- Christopher J. White, MD‡,
- John A. Spertus, MD, MPH† and
- Robert W. Yeh, MD, MSc⁎,⁎ ()
Reprint requests and correspondence:
Dr. Robert W. Yeh, Cardiology Division, GRB 8-843, Massachusetts General Hospital, 55 Fruit Street, Boston, Massachusetts 02114
Objectives We developed and internally validated a risk score to predict in-hospital stroke or death after carotid artery stenting (CAS).
Background A tool that accurately assesses CAS risk could aid clinical decision making and improve patient selection.
Methods Patients undergoing CAS without acute evolving stroke from April 2005 through June 2011 as part of the NCDR Carotid Artery Revascularization and Endarterectomy (CARE) Registry were included. In-hospital stroke or death was modeled using logistic regression with 35 candidate variables. Internal validation was achieved with bootstrapping, and model discrimination and calibration were assessed.
Results A total of 271 (2.4%) primary endpoint events occurred during 11,122 procedures. Independent predictors of stroke or death included impending major surgery, previous stroke, age, symptomatic lesion, atrial fibrillation, and absence of previous ipsilateral carotid endarterectomy. The model was well calibrated with moderate discriminatory ability (C-statistic: 0.71) overall, and within symptomatic (C-statistic: 0.68) and asymptomatic (C-statistic: 0.72) subgroups. The inclusion of available angiographic variables did not improve model performance (C-statistic: 0.72, integrated discrimination improvement 0.001; p = 0.21). The NCDR CAS score was developed to support prospective risk quantification.
Conclusions The NCDR CAS score, comprising 6 clinical variables, predicts in-hospital S/D after CAS. This tool may be useful to assist clinicians in evaluating optimal management, share more accurate pre-procedural risks with patients, and improve patient selection for CAS.
Carotid endarterectomy (CEA) effectively reduces stroke risk in symptomatic and asymptomatic patients with carotid stenosis compared with medical therapy (1,2). Modern trials have demonstrated similar composite outcomes with carotid artery stenting (CAS) compared with CEA in standard and high-surgical risk patients (3,4). Accordingly, CAS has emerged as a comparable therapy in certain clinical settings, and its use is now increasing (5,6).
In conditions for which comparable therapies exist, it is crucial for clinicians to accurately gauge the risks associated with treatment options and to communicate these risks to patients. This transfer of knowledge facilitates shared decision making and permits selection of the therapeutic strategy that is most congruent with patient and clinician expectations (7). With carotid stenosis, developing a more refined understanding of patient risk is particularly critical because some factors may increase the risk of both CAS and CEA, others may influence the respective procedural risks in opposite directions, and some risk factors may be modifiable (8). Additionally, certain characteristics that influence outcomes with CAS may be distinct from those with CEA, and it is clear that CAS is not merely a safer, less invasive alternative to CEA for all patients (3).
Previous studies reported the clinical and procedural characteristics associated with adverse outcomes after CAS (9), and risk scores derived from single-center populations with a limited number of adverse events have been described (10,11). At present, however, there are no existing prediction models or risk scores that can be used to quantify the risk of adverse events in a broad representative population of patients undergoing CAS. Using data from the CARE (Carotid Artery Revascularization and Endarterectomy) Registry, we developed and validated a pre-procedural risk model and created a simplified version to support clinical use and prospective risk stratification.
The CARE Registry is part of the National Cardiovascular Data Registry (NCDR) and enrolls patients with carotid stenosis who have undergone revascularization with either CEA or CAS (12). The registry was created to monitor clinical practice, assess patient outcomes, and provide a framework for quality improvement initiatives. As of August 2012, the registry included 13,613 CAS procedures performed at 173 hospitals.
All patients undergoing CAS from April 2005 through June 2011 were initially evaluated for inclusion in this analysis. Those with acute evolving stroke were excluded. The primary outcome of interest was the occurrence of in-hospital stroke or death (S/D). Stroke was defined as a new neurological deficit persisting for >24 h. All outcomes were abstracted by trained data collectors using standardized definitions. Quality checks were implemented before incorporation of the data into the CARE Registry, and problems were resolved by sites before submission.
For all patients, we examined a broad list of 35 variables relating to sociodemographic characteristics, cardiovascular history, neurological history, and angiographic characteristics (Tables 1 and 2).⇓ We conducted bivariate comparisons of characteristics of patients with and without in-hospital S/D using chi-square tests for dichotomous variables and t tests for continuous variables. We initially sought to develop 2 separate prediction models: one incorporating only those characteristics known before angiography (clinical model) that could be used to risk-stratify patients before the procedure and a second that included additional variables gained at the time of angiography relating to lesion characteristics (angiographic model).
We conducted multivariable logistic regression to generate prediction models for the primary endpoint of S/D incorporating clinical variables. We then conducted stepwise selection of variables using a criterion for retention of p < 0.05. To account for the possibility that risk factors for S/D had different effects in asymptomatic and symptomatic patients, we included interaction terms for symptomatic status and each of the variables that were included in the selected clinical model. After fitting the model in the entire dataset, we conducted internal validation by refitting the model in 1,000 bootstrap samples with replacement. This method of model validation has been found to have lower variability and lower bias potential compared with traditional split-sample validation and k-fold cross-validation (13). All measures of model performance were corrected for optimism, and the final reported model was recalibrated based on a “shrinkage” factor derived from the calibration slope. To further assess model calibration, we fitted a smoothed line showing the relationship between predicted and observed risk of in-hospital S/D based on the final model (14). We additionally performed validation of this clinical model using a separate cohort of CAS procedures performed from July 2011 through December 2011 (n = 1,544) within the CARE Registry, which became available after development of the original model.
A second angiographic model including all potential clinical and angiographic features was constructed in a similar fashion. To test the incremental improvement that angiographic variables added to prediction, we compared C-statistics using methods by DeLong et al. (15) and computed the integrated discrimination improvement (16). To support routine clinical use, we developed a risk score based on a points system with weights based on the coefficients in the final clinical model (17). Predictive performance of the risk score was then characterized using discrimination and calibration.
We also generated 2 additional models with different primary endpoints. The first used a primary endpoint of in-hospital myocardial infarction, stroke, or death (major adverse cardiac event), and the second used a primary endpoint of in-hospital ipsilateral S/D. Finally, within the CARE registry, 30-day outcomes are available in a subset of patients. To assess the ability of our primary model to predict 30-day S/D, we assessed the discrimination of our model in this subset of patients. Values of p < 0.05 (2 tailed) were considered to indicate statistical significance. All statistical analyses were performed using SAS version 9.2 (SAS Institute, Cary, North Carolina).
A total of 271 (2.4%) primary endpoint events occurred during 11,122 procedures. Stroke occurred in 242 patients (2.2%), and death occurred in 52 patients (0.5%). Of the 242 strokes, 165 (68.2%) were in the ipsilateral carotid territory, 32 (13.2%) were in the contralateral carotid territory, and 45 (18.6%) were in the vertebrobasilar territory or the location was unknown.
Bivariate analysis demonstrated individuals with S/D were older and more likely to have upcoming cardiac or other major surgery. Patients with S/D were more likely to have experienced previous neurological events and more often had symptomatic target lesions. Additionally, atrial fibrillation, smoking, renal impairment, dementia, and chronic lung disease were more common in the group with S/D (Table 1).
After multivariable regression, 6 variables were retained to form the final clinical model: impending major surgery (defined as cardiac, vascular, or other surgery planned within 8 weeks); previous stroke; age; symptomatic target lesion within the previous 6 months; atrial fibrillation; and previous ipsilateral CEA (Table 3). The patient characteristics most strongly associated with S/D were impending major surgery (OR: 2.20; 95% CI: 1.34 to 3.61) and previous stroke (OR: 2.03; 95% CI: 1.53 to 2.70). Previous ipsilateral CEA was associated with a reduced risk of S/D (OR: 0.63; 95% CI: 0.42 to 0.94).This model was found to have a C-statistic of 0.71 in the overall population and was well calibrated (Hosmer-Lemeshow p = 0.51) (Fig. 1). Validation of this model in the separate population of patients undergoing CAS from July 2011 through December 2011 yielded similar performance characteristics (in-hospital S/D C-statistic: 0.68; Hosmer-Lemeshow p = 0.38. Model discrimination and calibration within the asymptomatic and symptomatic subgroups were similar to those observed in the overall population (C-statistic: 0.72; Hosmer-Lemeshow p = 0.83; C-statistic: 0.68, Hosmer-Lemeshow p = 0.67, respectively). The inclusion of interaction terms did not improve model performance and were thus excluded from the final model (data not shown).
Among angiographic variables, only lesion length was a significant predictor of S/D (Table 2) (OR: 1.11; 95% CI: 1.05 to 1.17 per 5 mm additional length). The addition of this variable to the existing clinical model did not improve model performance (C-statistic 0.72, integrated discrimination improvement 0.001; p = 0.21). Similarly, compared with the clinical model, the models using major adverse cardiac events and ipsilateral S/D as primary endpoints had similar risk predictors and performance characteristics (data not shown).
Weighted clinical model
Because the ORs of the 6 clinical variables were disparate, a weighted points system was ascribed to each variable based on the magnitude its effect to create a risk score (Table 4,Fig. 2A) (17). The discriminatory ability of the risk score was unchanged (C-statistic: 0.71). The risk score was then applied to the CARE population with available 30-day outcomes data (n = 8,550). The risk score had similar discriminatory ability in predicting 30-day events (C-statistic: 0.67) and was able to stratify patients into different levels of risk (Fig. 2B). Online Figure 1 demonstrates how the risk score may be applied to individual patients for clinical use.
Within a large representative population of patients undergoing CAS in the United States, we developed and validated a risk score that predicts in-hospital S/D after carotid stenting. This model consists of a small number of readily available pre-procedural clinical variables. Importantly, in the subset of patients with 30-day outcome data available, the model maintains its discriminatory ability. We demonstrated that there is a broad distribution of procedural risk among patients undergoing CAS, further emphasizing the need for accurate risk stratification.
In an environment where 2 carotid revascularization strategies each pose unique risks influenced by patient characteristics, a schema for risk assessment is necessary to assist patients in selecting the most appropriate therapy (7). The NCDR CAS risk score may enable clinicians to identify those patients at excessive risk of CAS so that medical therapy or CEA may be offered as alternatives. Similarly, in patients with prohibitive risk with CEA, this risk score is helpful in identifying patients with acceptable CAS risk.
In accordance with previous observations, age, symptomatic status, and previous stroke each independently increased risk in this analysis (8,18–20). Similarly, previous CEA was found to be protective, which also is consistent with published data (8). Atrial fibrillation and impending major surgery have not been routinely associated with increased risk after carotid stenting. Atrial fibrillation is inherently associated with cardioembolic stroke risk (21), and this risk may be higher periprocedurally due to the need to discontinue oral anticoagulation to safely perform invasive procedures. Impending major surgery likely reflects selection of a uniquely higher risk patient population because these are individuals receiving percutaneous carotid revascularization in preparation for major cardiac, vascular, or other types of surgery. Dual antiplatelet therapy may be prematurely discontinued in these individuals in preparation for surgery, which may potentially increase perioperative risk as well. Finally, selection bias may prevent patients with impending major surgery from receiving CAS due to perceived excess risk, and some studies have excluded patients with atrial fibrillation from trial participation (3).
Although many of the risk factors identified in this analysis are nonmodifiable, it is conceivable that strategies could be implemented to decrease CAS risk. As an example, in patients with atrial fibrillation, more aggressive anticoagulation bridging practices or stenting via alternative access sites without warfarin discontinuation may reduce stroke risk. Similarly, in those patients needing urgent surgery, use of intravenous or reversible antiplatelet agents may pose less risk for stent-related complications perioperatively.
For individuals wishing to use the NCDR CAS risk score, the CAS acronym facilitates clinical use. The letters of the CAS acronym correspond to each variable (previous ipsilateral CEA, Age, Atrial fibrillation, impending major Surgery, Symptomatic target lesion, previous Stroke), and the letter position (C-1, A-2, S-3) identifies the number of risk variables beginning with that letter.
The NCDR CAS score in the context of historical risk stratification
The CEA literature has traditionally referred to patients as being either high or standard surgical risk based on the presence of any number of risk factors including clinically significant cardiac disease, severe pulmonary disease, contralateral occlusion, previous neck radiation, laryngeal nerve palsy, post-CEA restenosis, and age >80 years (22). However, we believe that in the development of the NCDR CAS score, we have created an instrument that can be used to more precisely quantify risk in a fashion that accounts for the differential impact that risk factors contribute, as well as the additive impact that multiple risk factors may impart. In addition, the creation of this risk score allows clinicians to readily stratify patients into widely accepted risk categories, including those based on cutoffs of 3% and 6% in asymptomatic and symptomatic patients, respectively (Fig. 2B) (23).We would caution, however, that such risk categories were defined in an era before the aggressive use of antiplatelet and lipid-lowering therapy and before the routine use of embolic protection devices for CAS (24). Major adverse cardiac event rates with CAS are decreasing (18,25), and it is unclear whether these historical stratification criteria maintain merit with modern revascularization techniques (26).
Several limitations warrant discussion. First, prediction of S/D using this model is most applicable to the in-hospital period only because 30-day outcome data are not uniformly available for all participants in CARE. Second, we did not adjust for all variables that influence CAS-related risk including embolic protection device use, filter time, operator experience, and other patient- or lesion-specific characteristics (27–29). Most operators may, in fact, not consider CAS for patients with complex arch anatomy or severe tortuosity. Additionally, the angiographic information collected within the CARE registry is limited, and it is possible that angiographic variables not captured in this registry are important predictors of outcomes. Finally, the generated model has moderate discriminatory ability (C-statistic: 0.71) and is not meant to substitute for clinical judgment.
We developed and internally validated a simple, easy-to-use risk score to predict in-hospital S/D after CAS. This model may provide valuable information about procedural risk for patients considering this therapy. The application of this tool into clinical practice has the potential to improve outcomes by optimizing patient selection for this procedure.
For a figure demonstrating how the risk score may be applied to individual patients for clinical use, please see the online version of this article.
This research was supported by the American College of Cardiology Foundation's National Cardiovascular Data Registry (NCDR). The views expressed in this manuscript represent those of the author(s), and do not necessarily represent the official views of the NCDR or its associated professional societies identified at www.ncdr.com. The CARE Registry is an initiative of the American College of Cardiology Foundation, The Society for Cardiovascular Angiography and Interventions, the Society of Interventional Radiology, the American Academy of Neurology, the American Association of Neurological Surgeons/Congress of Neurological Surgeons, the Society for Vascular Medicine, and the Society of Vascular and Interventional Neurology. Dr. Yeh is an investigator at the Harvard Clinical Research Institute and has served as a consultant for the Kaiser Permanente Division of Research. Dr. Spertus has an ACCF contract to serve as an analytic center for the CARE registry. Dr. White is an investigator for the CABANA trial from Boston Scientific Corp. Dr. Rosenfield has received research grants from Abbott Vascular, Bard Peripheral Vascular, Medtronic/Invatec, and Atrium; has received consulting/advisory board fees from Abbott Vascular, Boston Scientific Corp., Complete Conference Management, Harvard Clinical Research Institute, Contego, Micell, and Becker Ventures; has equity in Lumen Biomedical, Medical Stimulation Corp., Angioguard (Cordis), and Micell; and has served on the board of directors for VIVA Physicians (501C3). Dr. Drachman has received research grant support from iDev Technologies, Inc. and Lutonix/Bard; is member of the clinical events committee of PLC Medical Systems, Inc.; and is a member of the Data and Safety Monitoring Board of Prarie Education & Research Cooperative. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.
- Abbreviations and Acronyms
- carotid artery stenting
- Carotid Artery Revascularization and Endarterectomy
- carotid endarterectomy
- confidence interval
- National Cardiovascular Data Registry
- odds ratio
- stroke or death
- Received May 20, 2012.
- Revision received June 21, 2012.
- Accepted July 3, 2012.
- American College of Cardiology Foundation
- Goodney P.P.,
- Travis L.L.,
- Malenka D.,
- et al.
- Touze E.,
- Trinquart L.,
- Chatellier G.,
- Mas J.L.
- Roubin G.S.,
- Iyer S.,
- Halkin A.,
- Vitek J.,
- Brennan C.
- Hofmann R.,
- Niessner A.,
- Kypta A.,
- et al.
- Setacci C.,
- Chisci E.,
- Setacci F.,
- Iacoponi F.,
- de Donato G.,
- Rossi A.
- Harrell F.E.
- Gray W.A.,
- Chaturvedi S.,
- Verta P.
- Wolf P.A.,
- Abbott R.D.,
- Kannel W.B.
- Brott T.G.,
- Halperin J.L.,
- Abbara S.,
- et al.
- Moore W.S.,
- Barnett H.J.,
- Beebe H.G.,
- et al.
- Safian R.D.
- Reimers B.,
- Corvaja N.,
- Moshiri S.,
- et al.
- Safian R.D.,
- Bresnahan J.F.,
- Jaff M.R.,
- et al.