Author + information
- Received December 21, 2015
- Revision received January 27, 2016
- Accepted January 28, 2016
- Published online April 12, 2016.
- S073510971600509X-70e45dab4535a2b0c1f185898d127c00Taku Inohara, MD, PhDa,b,
- S073510971600509X-9e2e727eb2988aa8193e88bffb322ce3Shun Kohsaka, MD, PhDa,∗ (, )
- S073510971600509X-f76bcc529f1d9c406614bab1af2c0e6bHiroaki Miyata, PhDc,
- S073510971600509X-94ed9d49c8e37c36ebc355589674244fIkuko Ueda, PhDa,
- S073510971600509X-93cdfd7f9ee766d94ba0378aa7b5c98aYuichiro Maekawa, MD, PhDa,
- S073510971600509X-55176efc77345416188b4f9a99296ba0Keiichi Fukuda, MD, PhDa,
- S073510971600509X-a4aaf6b179b190d6b511588942ae7899David J. Cohen, MD, MScd,
- S073510971600509X-ad108df8f66fe9308e363eefcd6fd135Kevin F. Kennedy, MSd,
- S073510971600509X-52080fbc148c722b3cb0e10afa840ab7John S. Rumsfeld, MD, PhDe and
- S073510971600509X-ecebf6f50775c4f8fb21d63614c53baeJohn A. Spertus, MD, MPHd
- aDepartment of Cardiology, Keio University School of Medicine, Tokyo, Japan
- bDepartment of Cardiology, Hiratsuka City Hospital, Hiratsuka, Japan
- cDepartment Health Policy and Management, Keio University School of Medicine, Tokyo, Japan
- dSaint Luke’s Mid America Heart Institute, University of Missouri-Kansas City School of Medicine, Kansas City, Missouri
- eDenver VA Medical Center, Denver, Colorado
- ↵∗Reprint requests and correspondence:
Dr. Shun Kohsaka, Department of Cardiology, Keio University School of Medicine, 35 Shinanomachi, Shinjuku-ku, Tokyo 160-8582, Japan.
Background Stratifying patient risk for acute kidney injury (AKI) prior to percutaneous coronary intervention (PCI) can enable clinicians to tailor their approach to minimize AKI. The National Cardiovascular Data Registry (NCDR) CathPCI Registry recently developed 2 prediction models: for AKI and AKI requiring dialysis (AKI-D).
Objectives This study sought to externally validate the NCDR AKI and AKI-D models in a Japanese population. Determining the generalizability of the U.S. model could support quality improvement efforts in Japan.
Methods The NCDR prediction models were applied to 11,041 consecutive patients in the Japanese multicenter PCI registry. AKI was defined as an absolute increase ≥0.3 mg/dl or a relative increase of 50% in serum creatinine, in accordance with the definition of AKI Network criteria; AKI-D was defined as initiation of dialysis after PCI. Discrimination and calibration of the NCDR models were tested in the Japanese cohort. If the model was perfectly calibrated, the slope and intercept would equal 1.0 and 0.0, respectively.
Results In the Japanese PCI cohort, AKI and AKI-D occurred in 10.5% and 1.5% of patients, respectively. The NCDR AKI prediction model showed good discrimination (c-statistic = 0.76) and calibration (slope = 0.93 and intercept = –0.10) in both acute and nonacute PCI. The AKI-D prediction model had good discrimination (c-statistic = 0.92), but while the calibration slope was good (1.04), the intercept was significantly underestimated (0.96). However, this was corrected with recalibration (slope = 1.04 and intercept = –0.087).
Conclusions In a Japanese population, the NCDR AKI models validly predict post-procedural AKI and, with recalibration, AKI-D. Prospective use of these models to inform clinical decision making should be tested as a means of reducing AKI after PCI in Japan. (Japan Cardiovascular Database, Percutaneous Coronary Intervention Registry; UMIN R000004736).
Acute kidney injury (AKI) is the most common noncardiac complication of percutaneous coronary intervention (PCI) and is associated with increased risk of morbidity as well as both short- and long-term mortality (1–3). AKI can be prevented by adequate hydration, limiting contrast, and potentially through pre-procedural statin use (4–6). Importantly, the beneficial effects of such interventions are directly proportional to pre-procedural AKI risk, whereby interventions are most effective when used in higher-risk patients.
In the United States, the National Cardiovascular Data Registry (NCDR) CathPCI Registry recently created prediction models for AKI and AKI requiring dialysis (AKI-D) to provide risk-adjusted feedback to centers for quality assessment and prospective risk estimation to support quality improvement strategies and minimize the incidence of AKI (7). However, whether these risk models are valid in non-U.S. populations is unknown. Ethnic differences exist between different regions, especially Asia and North America, with regard to AKI, and different practice patterns might alter the relationship between patient characteristics and AKI incidence (8–12).
Given these differences in both patient profiles and treatment strategies, the predictive accuracy of the U.S.-derived risk model for AKI in the Japanese population is unknown. If valid in Japan, this model could be more rapidly applied into Japanese quality improvement efforts and could support international comparisons to define best practices for AKI prevention. If not valid, however, then a new risk model would need to be built on an exclusively Japanese population. Therefore, we aimed to validate the NCDR AKI and AKI-D risk models in Japan.
Data to validate the NCDR AKI risk models were derived from the JCD-KiCS (Japan Cardiovascular Database Keio interhospital Cardiovascular Studies) registry, which is a prospective 16-center registry designed to collect clinical variables and outcome data on consecutive PCI patients, with dedicated clinical research coordinators assigned to each site (13,14). The full list of the investigators and coordinators are provided in the Online Appendix. Approximately 200 variables were collected from each patient. The clinical variables and in-hospital outcomes of the JCD-KiCS registry were defined in accordance with the NCDR version 4.1 (15,16). The JCD-KiCS registry includes 16 institutes within the metropolitan Tokyo area; mostly are large tertiary care referral centers (>200 beds; n = 12), but a few midsized satellite hospitals (<200 beds; n = 4) also were included. Participating hospitals were instructed to record and register data from consecutive hospital visits for PCI using an internet-based data collection system. All PCI procedures performed with any commercially available coronary device were included. The data entered were checked for completeness and internal consistency. Quality assurance was achieved through automatic system validation and reporting of data completeness and through education and training of dedicated clinical research coordinators specifically trained for this PCI registry. The senior study coordinator (I.U.), and extensive on-site auditing by the investigator (S.K.), ensured proper registration of each patient.
All the patients undergoing PCI in the JCD-KiCS from September 2008 to May 2014 (N = 14,356) were eligible. Exclusion criteria included insufficient baseline information (n = 162; 1.1%), lack of either a pre- or post-procedural serum creatinine level (n = 1,838; 12.8%), multiple PCIs during a single hospitalization (n = 804; 5.6%), and dialysis prior to PCI (n = 511; 3.6%). After these exclusions, 11,041 patients were enrolled in the study.
The post-procedural creatinine value was defined as the highest value within 30 days after the index procedure. Congruent with the NCDR definition, if more than 1 post-procedural creatinine level was measured, the highest value was used for determining AKI. The timing of the creatinine measurement in the core hospital center (Keio University Hospital; n = 979) showed that most such measurements were made in the first day following PCI (Online Figure 1). Timing of peak creatinine was congruent with previously published studies (17). Similar to the NCDR, we defined AKI per the Acute Kidney Injury Network criteria (18). According to pre-procedural and post-procedural creatinine values, patients were classified as follows: 1) AKI stage 1: 0.3 mg/dl absolute or 1.5- to 2-fold relative increase in serum creatinine; 2) AKI stage 2: >2- to 3-fold increase in serum creatinine level; and 3) AKI stage 3: >3-fold increase in serum creatinine level or serum creatinine level >4.0 mg/dl, with an acute increase of >0.5 mg/dl or a need for renal replacement therapy. AKI requiring dialysis (AKI-D) was identified by using a pre-defined JCD-KiCS data element for acute or worsening renal failure necessitating new renal dialysis after PCI. Patients with AKI-D were included in the AKI group, but were examined separately, too, given the greater clinical impact of dialysis.
Calculation of NCDR AKI and AKI-D risk estimates
NCDR AKI and AKI-D risk models were developed in 2014 and the methodology has been previously described (7). In the NCDR AKI risk model, a total of 11 variables are used for predicting AKI: age, baseline renal impairment (categorized as mild, moderate, and severe), prior cerebrovascular disease, prior heart failure, prior PCI, presentation (non–acute coronary syndrome [ACS] vs. non–ST-segment elevation myocardial infarction [NSTEMI]/unstable angina [UA] vs. ST-segment elevation myocardial infarction [STEMI]), diabetes, chronic lung disease, hypertension, cardiac arrest, anemia, heart failure on presentation, pre-procedural intra-aortic balloon pump insertion, and cardiogenic shock. The NCDR AKI-D model uses 5 variables: baseline renal impairment (categorized as mild, moderate, or severe), presentation (non-ACS vs. NSTEMI/UA vs. STEMI), diabetes, heart failure on presentation, and cardiogenic shock. The same data definitions were used in both the NCDR and JCD-KiCS registries, therefore permitting the calculation of NCDR-estimated AKI and AKI-D risk for each patient in the JCD-KiCS dataset.
Patients were stratified by AKI stage, and their pre-procedural characteristics were compared using analysis of variance for continuous variables and chi-square tests for categorical variables.
The performance of the NCDR AKI models in the Japanese population was examined using c-statistics to assess discrimination and by comparing the slopes and intercepts across deciles of risk to evaluate calibration. If the model was perfectly calibrated, the intercept and slope would equal 0 and 1, respectively. We additionally accounted for clustering within sites by using study site as a random effect in a hierarchical logistic regression model.
After assessing discrimination and calibration across the entire JCD-KiCS cohort, we examined model performance for clinically distinct patient populations. These included stable angina and ACS, and by baseline renal function using estimated glomerular filtration rate values above and below the median. Since renal dysfunction at the time of procedure was one of the strongest risk factors for AKI, certifying the sufficient performance of the models, regardless of baseline renal function, would support the robustness of the NCDR models across the spectrum of PCI patients.
Among the 11,041 patients in the Japanese PCI cohort, 1,117 (10.1%) experienced AKI and 162 patients (1.5%) experienced AKI-D after PCI. Patients developing AKI tended to be older and have more heart failure, pre-procedural renal dysfunction, and a greater number of comorbidities (Table 1). Acute indications (STEMI and NSTEMI/UA), cardiogenic shock, and cardiac arrest on presentation were also associated with AKI.
Compared with patients in the U.S. cohort, derived from the NCDR dataset (Online Table 1), those in the Japanese cohort were older, had lower body mass index, and were more likely to be male and smokers. While many comorbidities were more frequently observed in the U.S. cohort, the prevalence of diabetes and renal dysfunction were higher in the Japanese cohort. Furthermore, in Japan, coronary intervention has been more commonly performed for nonacute conditions and more aggressively implemented in severely ill patients receiving mechanical support.
Performance of NCDR AKI and AKI-D risk models
In the entire cohort, the AKI prediction model showed good discrimination (c-statistic = 0.76) and calibration (slope = 0.93 and intercept = –0.10) (Central Illustration). In the highest-risk group, the predicted and observed incidence of AKI was 34% and 36%, respectively, while in the lowest decile the risk was <2%, indicating a wide range of predicted risk. Similar performance was observed in patients with ACS (c-statistic = 0.73; calibration slope = 0.83; and intercept = –0.16) and without ACS at the time of PCI (c-statistic = 0.69; calibration slope = 0.97; and intercept = –0.26) (Figures 1A and 1B). Furthermore, the AKI prediction model’s performance was consistent in those with lower baseline renal function (c-statistic = 0.76; calibration slope = 0.85; and intercept = –0.16) as well as higher baseline renal function (c-statistic = 0.73; calibration slope = 1.04; and intercept = 0.11) (Figures 1C and 1D).
The AKI-D prediction model demonstrated very good discrimination when applied to the Japanese population (c-statistic = 0.92). The calibration slope was very close to 1.0 (slope = 1.04); however, the intercepts of the calibration plots were significantly different between the Japanese (intercept = 0.96) and U.S. PCI populations (intercept = 0.03). This was primarily due to underestimation of dialysis risk in the highest-risk patients (Central Illustration), where the observed and predicted AKI-D incidences were 9% and 4%, respectively. By recalibrating the intercept to the Japanese population, both the slope and intercept improved (calibration slope = 1.04 and intercept = –0.087) (Central Illustration). The final model was able to stratify risk, across deciles, from <0.5% in the lowest 5 deciles to 10% in the highest decile.
To our knowledge, this is the first example of testing a national risk model developed for 1 country with its performance in a second, highlighting the opportunity for more rapid adoption of validated risk models in countries seeking to improve procedural quality. In this study, we examined the performance of U.S. AKI models in Japan, with very different clinical and procedural approaches for patients undergoing PCI (e.g., more advanced age, lower body mass index, and greater use of transradial approaches in Japan , as well as more use of thrombus aspiration  and intra-aortic balloon pumps  than in the United States). Moreover, AKI and AKI-D rates were much higher in Japan than the United States (10.1% and 1.5%, respectively, in Japan vs. 7.3% and 0.3%, respectively, in the United States) (7). Despite these differences, the U.S.-derived AKI prediction model performed very well. In fact, the discrimination of the model in Japanese patients was even better than that in the U.S. cohort for whom it was developed (c-statistic = 0.76 vs. 0.71) (7).
The model also performed well within clinically important subgroups of the population, including those with and without ACS or renal dysfunction at the time of PCI. While the predictive accuracy of the AKI-D model was very good, recalibration of the intercept was needed for it to provide valid prediction estimates in this Japanese cohort. Collectively, the data support the use of the NCDR models in Japan and encourage their use for both quality assessment/improvement purposes and as a foundation for prospective risk stratification and the reduction of AKI.
While numerous AKI prediction models for PCI have been developed (19–22), only a few models have been externally and internationally validated (23). The NCDR AKI prediction model was derived and validated in a large national registry and includes 11 pre-procedural variables with a simplified integer risk score. Hence, the model was ideal for implementation in our patient dataset compared with other risk models that included procedure-related variables (19,20) that could not be known prior to the procedure. Improved pre-procedural stratification of patients at risk for PCI-related complications, including AKI and bleeding, could lead to an increased use of preventive strategies and lower complication rates (24,25).
The AKI-D prediction model showed very good discrimination, but its calibration, particularly the intercept term, was not zero, indicating that the model would need to be recalibrated for optimal performance. This suggests a difference in practice patterns for patients with higher degrees of AKI after PCI. These findings were in accordance with a previous report, which included several causes of AKI other than undergoing PCI, indicating that for patients with AKI, dialysis was initiated earlier in Japan than in other nations, including the United States (26). The good discrimination of the AKI-D model, despite the need for recalibration, suggests that dialysis is initiated for lesser degrees of AKI in Japan, but that the factors that predict the need for dialysis are similar. Whether the long-term outcomes of AKI-D patients differ between Japan and the United States should be determined to help define the better strategy for initiating dialysis (the more liberal Japanese approach or the more conservative U.S. approach).
Using the NCDR AKI risk model opens several opportunities to improve the safety and consistency of PCI in Japan. First, it can enable that risk-adjusted performance reports are provided to hospitals so that they can better understand how their rates of AKI or AKI-D compare with their peers. Sites with higher than predicted rates can use this information to compare their processes of care with their peers to learn strategies that might minimize their kidney injury rates. A second opportunity from the validation of this model is to use it as a foundation for prospective risk stratification and personalized care. Knowing a patient’s individual risk for AKI enables the provider to better inform a patient of his or her risk and to tailor treatment to risk. For example, a patient with multivessel coronary disease who is at high risk for AKI/AKI-D may benefit from a staged procedure, rather than multivessel revascularization in a single PCI procedure, to minimize contrast exposure at any 1 point in time. Testing the utility of these risk models for reducing AKI or AKI-D is an important area for future research.
First, not all hospitals that perform PCI in Japan participate in the JCD-KiCS registry, and the sampling bias and generalizability of the study results to Japan is a potential concern. Our registry, however, is multicenter and includes a relatively large number of procedures. We believe this is one of the most representative Japanese databases of PCI patients and our results comprise the most complete assessment of current practice patterns in Japan. When compared with the national PCI registration data of 2014, the patient characteristics were similar (Online Table 2), supporting the generalizability of our findings across Japan. Furthermore, our registry was built with the cooperation of the NCDR CathPCI Registry in 2009 incorporating common variable definitions, including the definitions for AKI; thus, we had an ideal situation for validating its prediction models. The fact that the performance of the NCDR AKI models was good in a completely different set of patients under different practice patterns could support their solid performance and lead to international generalizability. A second limitation is that the definition of “highest post-procedural creatinine” differed slightly in the JCD-KiCS registry than in the NCDR CathPCI registry. In the United States, post-procedural creatinine is defined as the highest value measured during the hospitalization for the indexed procedure; in the JCD-KiCS registry, it is also extracted from the laboratory values obtained during the entire 30-day follow-up period after the indexed procedure. Given the short length of stay for many procedures, particularly non-ACS cases in the United States, and the fact that peak creatinine levels are often observed 3 to 5 days after contrast exposure (17), the NCDR registry may have underestimated the true rate of AKI in its model development. Nevertheless, the excellent model performance in Japan, with its more complete assessment of AKI rates, further supports the validity of the NCDR model in U.S. patients.
We externally and internationally validated the NCDR AKI and AKI-D prediction models using a Japanese cohort. To improve the quality of care in patients undergoing PCI, the U.S. AKI prediction model could be applied in Japan and, perhaps, in other countries as well. Importantly, explicitly testing a model’s performance can identify discrepancies that warrant recalibration, as seen in the AKI-D model. Given the validity of the NCDR AKI models in Japan, future research in how to best leverage these insights of patient risk to minimize AKI is warranted.
COMPETENCY IN PATIENT CARE AND PROCEDURAL SKILLS: The NCDR AKI models that were developed based on the U.S. population also predict post-procedural AKI in patients undergoing PCI in Japan.
TRANSLATIONAL OUTLOOK: Further research is needed to determine how best to apply the NCDR and other AKI risk prediction tools to preserve renal function in patients undergoing contrast angiography around the world.
The authors appreciate the contributions of all the investigators and clinical coordinators involved in the JCD-KiCS registry, who are listed in the Online Appendix.
The present study was funded by the Grants-in-Aid for Scientific Research from Japan Society for the Promotion of Science (Grant Nos. 25460630, 80571398) and Pfizer Health Research Foundation. The funders had no role in the conduct of the study; in the collection, management, analysis, and interpretation of the data; or in the preparation or approval of the manuscript. Dr. Kohsaka has received unrestricted research grants for the Department of Cardiology, Keio University School of Medicine, from Pfizer Japan, Inc. and Bayer Pharmaceutical Co., Ltd. Dr. Rumsfeld is the Chief Science Officer of the National Cardiovascular Data Registry. Dr. Spertus is the principal investigator of a contract from the American College of Cardiology Foundation to analyze the National Cardiovascular Data Registry data; and has an equity interest in Health Outcomes Sciences. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose. Peter A. McCullough, MD, served as Guest Editor for this paper.
- Abbreviations and Acronyms
- acute coronary syndrome
- acute kidney injury
- acute kidney injury requiring dialysis
- non–ST-segment elevation myocardial infarction
- percutaneous coronary intervention
- ST-segment elevation myocardial infarction
- unstable angina
- Received December 21, 2015.
- Revision received January 27, 2016.
- Accepted January 28, 2016.
- American College of Cardiology Foundation
- Rihal C.S.
- Tepel M.,
- Aspelin P.,
- Lameire N.
- Brown J.R.,
- Robb J.F.,
- Block C.A.,
- et al.
- Tsai T.T.,
- Patel U.D.,
- Chang T.I.,
- et al.
- Mizuno A.,
- Kohsaka S.,
- Miyata H.,
- et al.
- Inohara T.,
- Kohsaka S.,
- Miyata H.,
- et al.
- Brindis R.G.,
- Fitzgerald S.,
- Anderson H.V.,
- et al.
- McCullough P.A.,
- Sandberg K.R.
- Mehran R.,
- Aymong E.D.,
- Nikolsky E.,
- et al.
- Gurm H.S.,
- Seth M.,
- Kooiman J.,
- Share D.
- Spertus J.A.,
- Decker C.,
- Gialde E.,
- et al.