Author + information
- Received May 14, 1997
- Revision received March 11, 1998
- Accepted April 9, 1998
- Published online July 1, 1998.
- Peter C Block, MD, FACCa,* (, )
- Eric C Peterson, MD, MPH∗,
- Ronald Krone, MD, FACC†,
- Karen Kesler, MS∗,
- Edward Hannan, PhD‡,
- Gerald T O’Connor, PhD, DSc§ and
- Katherine Detre, MD, DrPH, FACC∥
- ↵*Address for correspondence: Dr. Peter C. Block, Providence St. Vincent Medical Center, 9205 SW Barnes Rd., Portland, Oregon 97225
Objectives. Our objectives were to identify and define a minimum set of variables for interventional cardiology that carried the most statistical weight for predicting adverse outcomes. Though “gaming” cannot be completely avoided, variables were to be as objective as possible and reproducible and had to be predictive of outcome in current databases.
Background. Outcomes of percutaneous coronary interventions depend on patient risk characteristics and disease severity and acuity. Comparing results of interventions has been difficult because definitions of similar variables differ in databases, and variables are not uniformly tracked. Identifying the best predictor variables and standardizing their definitions are a first step in developing a universal stratification instrument.
Methods. A list of empirically derived variables was first tested in eight cardiac databases (158,273 cases). Three end points (in-hospital death, in-hospital coronary artery bypass graft surgery, Q wave myocardial infarction) were chosen for analysis. Univariate and multivariate regression models were used to quantify the predictive value of the variable in each database. The variables were then defined by consensus by a panel of experts.
Results. In all databases patient demographics were similar, but disease severity varied greatly. The most powerful predictors of adverse outcome were measures of hemodynamic instability, disease severity, demographics and comorbid conditions in both univariate and multivariate analyses.
Conclusions. Our analysis identified 29 variables that have the strongest statistical association with adverse outcomes after coronary interventions. These variables were also objectively defined. Incorporation of these variables into every cardiac dataset will provide uniform standards for data collected. Comparisons of outcomes among physicians, institutions and databases will therefore be more meaningful.
Comparison of outcomes of coronary interventions from different medical centers requires consideration of the risk characteristics of the different patient populations served as well as the severity and acuity of the disease itself (1–3). In addition, accurate evaluation of the anticipated risk of a procedure is essential for planning treatment strategies (4). Comparing the results obtained in various databases (1,5–9) has been difficult because the definitions of seemingly similar variables differ (3), and the variables are not uniformly tracked. Standardizing variable definitions as well as identifying variables that carry most of the statistical weight in the analysis would be an important first step in developing a minimum dataset. This principle has been developed for cardiac surgery by Jones et al. (10). This report identifies a standardized stratification instrument that should be captured by databases designed to assess performance of catheter-based coronary interventions.
A working group with expertise in epidemiology, biostatistics and coronary interventions was convened to develop and define a list of variables relevant to coronary interventional procedures. Variables to be tested were proposed by the participants on the basis of empirical results from individual databases and knowledge of published scientific reports.
Eight databases ranging in size from 2,431 to 66,358 cases were used for this analysis (Table 1). Five of the eight databases were single- or multi-institutional clinical databases including all types of coronary interventional cases performed between 1990 and 1996. The National Heart, Lung, and Blood Institute (NHLBI) database was a multi-institution registry performed during 1985 to 1986. The New Approaches to Cardiac Intervention (NACI) Registry database was a multi-institutional clinical registry that included cases involving newer coronary interventional device technologies other than conventional balloon angioplasty only. The Cooperative Cardiac Project (CCP) database was a multi-institutional database limited to patients ≥65 years undergoing coronary interventional procedures. In total, our final analysis dataset included 158,273 cases.
Definition of variables
Whenever possible, variables were defined similarly to those used in the Cooperative CABG [coronary artery bypass graft surgery] Database Project (10). For the remainder, definitions were developed by consensus of the working group. Definitions were written to be objective in an effort to minimize “gaming.” Nevertheless, variables for potential subjective interpretation include unstable angina, Canadian Cardiovascular Society (CCS) class, congestive failure and urgency of the procedure. Three end points were chosen: 1) In-hospital death (death within the same hospital period from any cause after a coronary interventional procedure). “Cardiac death” was considered but is more subjective, leading to possible “gaming” (11). 2) In-hospital CABG (any CABG during the same hospital period as the coronary intervention). This end point includes “elective” CABG done after an intervention. 3) Q wave myocardial infarction (MI) (new Q waves in two contiguous electrocardiographic leads and a twofold increase in serum creatine kinase levels within the same hospital period as the coronary intervention). Non–Q wave MI was not used because of the wide variability of definitions for this entity in existing databases.
Within each database attempts were made to match existing variable definitions to those set forth by the working group. If a database manager considered that a database variable definition differed significantly, it was not included in the analysis. The availability of predictor variables varied among databases. Missing data were handled by sites in one of two ways: If the variable was missing in >50% of cases within a database, it was deleted from analysis. If the variable was missing in a minority of cases, it was imputed to the median for continuous variables and the lowest risk category for categoric data (e.g., missing ejection fractions at Duke were replaced with a score of 56 [the median ejection fraction for the Duke data], whereas missing comorbid variables [e.g., renal insufficiency] were imputed to the “none” level). These imputation methods are conservative and tend to underestimate the predictive power of a variable. For outcomes, all databases had information on in-hospital death and CABG; only four had information on in-hospital Q wave MI. We also tested whether a variable’s effect on outcome varied among the eight data sources using a standard test for homogeneity (12).
The first step was to construct univariate logistic regression models predicting each of the three outcomes. Each database reported univariate coefficients and standard errors for each variable. To consolidate this large amount of information, these estimates were then combined using meta-analytic methods. The databases were independently analyzed, and then pooled odds ratios from these analyses were developed. Specifically, using a random-effects empirical Bayes methodology model, we calculated a “pooled estimate” for the univariate odds ratio and the confidence interval surrounding these point estimates. This random-effects model provides a more conservative estimate of a variable’s effect on outcome (13,14).
Because many of the candidate variables describe similar aspects of the patients’ disease, multivariate logistic regression models were next used to distinguish between “competing” specifications of risk. Each database was asked to place all the variables it had collected into a “full” model and report the coefficients and standard errors for each variable. After fitting this model, each database used an automated stepwise selection algorithm to fit a smaller “stepwise” model that contained only variables significant at an alpha level of 0.05. Finally, to better assess the discrimination abilities of the variables in question, the C-index or area under the ROC curve was computed for each full and stepwise multivariate model (15).
As part of an international conference to further the development of data standards held in Bethesda, Maryland on June 27 to 28, 1996, the variables were reviewed to create explicit, objective definitions of each variable by consensus.
Baseline demographics, disease severity and comorbidity characteristics of the 158,273 cases entered into the eight databases are shown in Table 2. Patient demographics were similar, but disease severity varied greatly among databases (e.g., the proportion of patients with triple-vessel disease ranged from 9.6% to 17.5%, acute MI within 24 h from 0.7% to 12.4% and congestive failure from 4.0% to 14.7%). Significant variability existed among the more subjectively defined variables (e.g., the frequency of unstable angina at the time of procedure varied from 9.6% to 71.8%, and procedures classified as “urgent” varied from 2.0% to 43.1%). For the three outcomes, overall rates were similar across databases (Table 3). The one exception to this was a generally higher event rate in the CCP database, reflecting the inclusion of only older patients in this database.
Univariate predictors of events
Tables 4, 5 and 6⇓⇓⇓display the significant univariate predictors of in-hospital death, in-hospital CABG and in-hospital Q wave MI, respectively. The tables display the number of sites for which the variable was tested, the number of databases for which the variable was significant at the p <0.05 level and the overall pooled univariable odds ratio and 95% confidence interval around this odds ratio. We indicate for which variables the test for homogeneity of risk effect was p < 0.001. In-hospital death had the highest number of clinical predictors and measures of hemodynamic instability (e.g., cardiogenic shock and use of an intraaortic balloon pump) were the most powerful. Measures of disease severity (such as number of diseased vessels, lesion type, mitral regurgitation and ejection fraction) were also strong predictors of mortality. Finally, patient demographics (e.g., age, gender and body habitus) and a number of comorbid conditions (e.g., diabetes, vascular disease and renal function) were also found to predict in-hospital death.
We also found that the test for homogeneity was significant for certain variables, indicating that the risk factor predictive effect varied significantly between individual sites. However, given our large individual site sample sizes, this test statistic may be overly sensitive and not indicate clinically meaningful differences in the predictive power of a risk factor (12).
Although many of these same baseline clinical features also predicted in-hospital CABG and Q-wave MI (Tables 5 and 6), these variables tended to be less powerful predictors of these other end points (i.e., lower odds ratios). In addition, certain variables that predicted a higher mortality risk (e.g., ejection fraction) predicted a lower risk for CABG or were not significant predictors of CABG (e.g., congestive heart failure).
Multivariate predictors of events
Tables 7, 8 and 9⇓⇓⇓display the variables chosen in stepwise multivariate analysis by each database for the three end points. The variables chosen by each database are organized in the tables into four subgroups for classification purposes: 1) demographics; 2) coronary disease severity; 3) comorbidity; 4) catheterization/intervention data. There was a high degree of consistency in the significant multivariate predictors of outcomes across the databases, as was noted in the univariate analysis.
Although the attempt of this analysis was not to develop the ideal predictive instrument, we did ask the databases to report the discrimination abilities of their final stepwise multivariate models (in terms of the area under the receiver operating characteristics curve or C-index). For in-hospital death, all databases were able to discriminate which patients were likely to live from those who were likely to die, the C-index for in-hospital mortality ranging from 0.79 to 0.93. The models using variables for CABG within the same hospital period and Q wave MI were less robust, with CABG model C-indexes of 0.60 to 0.70 and Q wave MI model C-indexes of 0.60 to 0.69.
These data were then used to select the most important prognostic variables for coronary interventional outcomes. This comparison process provided a list of 29 variables that demonstrated an odds ratio >2.0 (or <0.5) in ≥50% of the databases in the univariate analysis or were a significant predictor identified in a majority of the multivariate analyses. The variables were divided into the categories of demographics, heart disease severity, acuity, comorbidity and procedure details. Consensus definitions for these variables are provided in Table 10.
Selection of variables
This report presents guidelines for a data-supported minimum set of variables required to risk stratify patients undergoing catheter based coronary interventions that could be incorporated into all interventional databases. A uniform minimum dataset, permitting risk adjustment of performance could thus be recorded by any institution, permitting simple responses to requests for information and comparisons between databases. For any database, the actual selection of variables is problematic and is closely related to the goals of data collection. Databases can be used for other analyses (e.g., process measures, cost) that demand collection of other variables. The purpose of the present study was to strictly limit the variables to those which actually provide risk information. Completely objective definitions limit “gaming” but are not possible for all variables. Thus, in multi-institutional databases some form of auditing will likely be needed to allow consistent interpretation of data across institutions. It is also important that the variable is reproducible. For example, although lesion characteristics appear to be predictive in individual databases, Botas et al. (16), using Bypass Angioplasty Revascularization Investigation (BARI) data showed that reproducibility of readings by core laboratory interpreters was unacceptable for features such as tortuosity, eccentricity, discrete, diffuse or tubular lesions or calcification.
Another problem is the choice of end points. Adverse outcomes are relatively few (3), so there is a desire to increase the end points, thus improving the robustness of any model by combining several outcomes. Usually, in-hospital mortality, emergency CABG and Q wave infarctions are combined. However, this study points out that individual variables may have interactions opposite in direction to the individual end points, so that their important role is not apparent. This is seen in the case of the variables “depressed ejection fraction” and “percutaneous transluminal coronary angioplasty (PTCA) of a graft,” which were associated with increased in-hospital death but not associated with in-hospital CABG.
Definition of end points
The methods used to define the end points chosen for analysis deserve comment. Short-term mortality is defined as “in-hospital death.” “Death within 30 days of an intervention” may be better but is at present impractical because of difficulty in obtaining outpatient follow-up information. “All-cause mortality” is not gameable and was therefore chosen. This minimum data set for coronary interventional procedures is not meant to be predictive for all patients undergoing all types of coronary interventional procedures. However, as much as possible we attempted to remain congruent with definitions chosen for CABG (10) so that if outcomes by patient characteristics were compared, data would be available.
Which variables best predict outcome?
It should be emphasized that we intended to identify a minimum dataset for catheter-based interventional cardiology outcomes. Data elements that reflect longer term effects on cardiac disease (e.g., hypertension, lipid profile, family history) are important in any database where the intent is to evaluate longer term outcomes. We realize that other variables that were not tested in this report may be equally predictive of short-term outcomes. The American College of Cardiology database includes our minimum data elements and is an example of a larger, more comprehensive database that might be used to evaluate both short-term and longer range outcomes (17).
Limitations of the study
By necessity, the data on the variables included in this report were retrospectively gathered. Existing variable definitions were matched as closely as possible to the consensus variable definitions. However, our pooled estimates of a specific variable’s effect size should be considered only as a relative approximation because this effect varied somewhat across individual sites for certain variables. In addition, some important variables, such as jeopardy score (18) or lesion characteristics defined by coronary ultrasound, were not included because they were not available in the databases surveyed. The data were also collected at various times, and most data were collected before the introduction of the new atherectomy devices, stents, and glycoprotein IIb/IIIA platelet receptor antagonists, which may have a significant impact on the importance of lesion specific variables (e.g., thrombus, eccentricity). We used data that predominantly referred to PTCA. As new devices are used, device-specific risk factors may be identified. Some variables amenable to gaming had to be included because of their predictive power (1). In some respects trying to control for gaming may seem “unfairly harsh” in some instances (e.g., in-hospital CABG includes all same-admission CABG, even elective), and even the broad definition of in-hospital death will obviously include some patients with a nonprocedure-related death. We only analyzed variables predicting adverse outcomes and do not imply that databases should be limited to that end point only.
By analyzing interventional outcomes across several large cardiac databases we were able to select 29 variables that had the strongest statistical association with outcomes. The variables selected are generally objective, easily obtainable and definable. Incorporation of these elements in every cardiac dataset will permit laboratories to be able to provide the information required in this era of managed care and preferred provider contracts as well as permit comparisons of the outcome of coronary interventions.
- Coronary artery bypass graft surgery
- Cooperative cardiac project
- myocardial infarction
- percutaneous transluminal coronary angioplasty
- Received May 14, 1997.
- Revision received March 11, 1998.
- Accepted April 9, 1998.
- American College of Cardiology
- Ellis S.G,
- Omoigui N,
- Bittl J.A,
- et al.
- 5.Kimmel SE, Berlin JB, Strom BL, Laskey WK, for the Registry Committee of the Society for Cardiac Angiography and Interventions. Development and validation of a simplified predictive index for major complications in contemporary percutaneous transluminal coronary angioplasty practice. J Am Coll Cardiol 1995;26:931–8.
- Ritchie J.L,
- Phillips K.A,
- Luft H.S
- O’Connor G.T,
- Malenka D.J,
- Robb J.G,
- et al.
- Jones R.H,
- Hannan E.L,
- Hammermeister K.E,
- DeLong E.R,
- O’Connor G.T,
- Luepker R.V,
- et al.
- Pratt C.M,
- Greenway P.S,
- Schoenfeld M.H,
- Hibben M.L,
- Reiffel J.A
- Hedges L.R,
- Olkin I
- Hasselblad V,
- McCrory D.C
- Eddy D.M,
- Hasselblad C,
- Schachter R
- Botas J,
- Stadius M.L,
- Bourassa M.G,
- et al.
- Ryan T.J,
- Bauman W.B,
- Kennedy J.W,
- et al.