The Problem With Composite End Points in Cardiovascular StudiesThe Story of Major Adverse Cardiac Events and Percutaneous Coronary Intervention
Author + information
- Kevin E. Kip, PhD⁎,⁎ (kkip{at}health.usf.edu),
- Kim Hollabaugh, RN, MSN†,
- Oscar C. Marroquin, MD, FACC‡ and
- David O. Williams, MD, FACC§
- ↵⁎Reprint requests and correspondence:
Dr. Kevin E. Kip, University of South Florida, College of Nursing, MDC 22, Room 2010, 12901 Bruce B. Downs Boulevard, Tampa, Florida 33612.
Abstract
Objectives Our purpose was to evaluate the heterogeneity and validity of composite end points, major adverse cardiac events (MACE) in particular, in cardiology research.
Background The term MACE is a commonly used end point for cardiovascular research. By definition, MACE is a composite of clinical events and usually includes end points reflecting safety and effectiveness. There is no standard definition for MACE, as individual outcomes used to make this composite end point vary by study. This inconsistency calls into question whether use of MACE in cardiology research is of value.
Methods We conducted a 2-phase literature review on the use of MACE as a composite end point: 1) studies that have compared use of bare-metal versus drug-eluting stents; and 2) studies published in the Journal in calendar year 2006. We subsequently tested 3 different definitions of MACE during 1-year of follow-up among 6,922 patients in the DEScover registry who received at least 1 drug-eluting stent.
Results The review identified substantial heterogeneity in the study-specific individual outcomes used to define MACE. Markedly different results were observed for selected patient subsets of acute myocardial infarction (MI) (vs. no MI) and multilesion stenting (vs. single-lesion stenting) according to the various definitions of MACE.
Conclusions Varying definitions of composite end points, such as MACE, can lead to substantially different results and conclusions. Therefore, the term MACE, in particular, should not be used, and when composite study end points are desired, researchers should focus separately on safety and effectiveness outcomes, and construct separate composite end points to match these different clinical goals.
The term MACE, defined as “major adverse cardiac events,” is arguably the most commonly used composite end point in cardiovascular research. Historically, the term MACE appears to have originated in the mid-1990s with its use restricted primarily to in-hospital complications related to percutaneous coronary interventions (PCIs) (1,2). Today, however, even though there is no standard definition of MACE, it is routinely used and reported for procedural, short-term, and long-term outcome evaluations, and may involve other cardiovascular treatments.
By definition, MACE, as well as all other composite end points, include multiple types of clinical events of varying degrees of relatedness. At the broadest level, definitions of MACE in use today include end points that reflect both the safety and effectiveness of various treatment approaches. This apparent mixing of “apples and oranges” and inconsistency calls into question whether use of MACE is of value. The purpose of this study was to evaluate the validity and utility of MACE as a composite research study end point, with the corresponding results presumably applicable to composite end points at large.
Methods
To evaluate the usefulness of MACE as an end point, we conducted a literature review followed by an empirical analysis. The literature review was performed to determine the heterogeneity of MACE. The empirical analysis tested the influence of different definitions on the interpretation of clinical investigations.
Literature review and data abstraction
We arbitrarily considered, a priori, 2 unrelated approaches of inquiry. In the first approach, we sought to query clinical trials that addressed a timely and relevant research question. We identified randomized clinical trials (RCTs) that have compared the use of drug-eluting stents (DES) to bare-metal stents (BMS) in PCI, and have reported MACE as either a primary or secondary end point. To achieve this, we searched the MEDLINE and Cochrane Central Register of Controlled Trials databases using the medical terms: bare-metal stents, drug-eluting stents, sirolimus, everolimus, rapamycin, paclitaxel, taxane, and taxol for calendar years 2002 to 2007. We then used the search terms MACE, major adverse cardiac events, or major clinical adverse events for the same years. We combined the searches with the limit term of clinical trial. We eliminated any report that did not describe primary trial results. Additionally, the bibliographies of the selected papers were reviewed along with those of several meta-analyses and pooled analysis articles for relevant trials (3–9). A total of 20 RCTs comparing any DES and BMS and reporting MACE as an end point were identified (10–29).
In the second approach, we sought to investigate the use of MACE at large (i.e., irrespective of a specific research question). Thus, we searched the MEDLINE database using the terms MACE, major adverse cardiac events, or major clinical adverse event. For practical reasons, we restricted this search to the Journal (J Am Coll Cardiol) for calendar year 2006. This resulted in the identification of 27 articles (23,29–54).
For each review, the individual outcomes that made up the definition of MACE were identified and abstracted independently by 2 reviewers (K.K., K.H.). Reviewer discrepancies in the individual outcomes of MACE were resolved by consensus.
Empirical analysis
Based on the results of the literature review, we constructed 3 definitions of MACE relevant to the setting of PCI. Included was a definition postulated to relate primarily to safety (death, myocardial infarction [MI], or stent thrombosis [ST]), and 2 definitions postulated to relate to both safety and effectiveness: 1) death, MI, ST, or target vessel revascularization (TVR); and 2) death, MI, ST, or any repeat revascularization).
Using the various constructed definitions, we evaluated 1-year risk of MACE in the DEScover registry (55). Briefly, DEScover is a prospective, multicenter, observational study designed to characterize PCI patients from a broad sampling of 140 hospitals across the U.S. The only exclusion criteria were patient refusal or inability to provide written informed consent and/or Health Insurance Portability and Accountability Act of 1996 authorization. Baseline clinical and angiographic characteristics and procedural and clinical in-hospital events were recorded for the enrolled patients. Follow-up was obtained from patients at 1, 6, and 12 months by a central telephone facility. For those patients reporting an event, a specially trained research coordinator then obtained additional information.
The present investigation was restricted to the 6,922 patients in DEScover who received at least 1 DES and did not initially present with cardiogenic shock. We assessed variation in 1-year risk of MACE using the different definitions and 2 types of patient subset comparisons. In the first type, we sought to compare patient subsets at increased risk for adverse events. In this scenario, we were interested in the influence of various definitions in regard to long-term safety, but not necessarily effectiveness. Thus, we selected and compared patients presenting with versus without acute MI. For the second subgroup analysis, we sought to compare definitions among patient subsets who might be at greater risk for lesion recurrence. This analysis would assess the impact of definitions on long-term effectiveness, but not necessarily safety. Thus, we selected and compared patients with multiple versus 1 lesion treated with PCI. We had initially considered comparing patients treated with BMS versus DES, but in DEScover, only 5% of patients received BMS and their baseline characteristics are very different (i.e., potential selection bias) from those who received DES.
For each comparison, adjusted hazard ratios were estimated by use of Cox proportional hazards regression. Covariates adjusted for included age, gender, urgent or emergent presentation, smoking status, number of diseased vessels, and history of diabetes, congestive heart failure, peripheral vascular disease, renal dysfunction or dialysis, or pulmonary disease. Analyses were performed using the SAS system version 9.0 (SAS Institute, Cary, North Carolina).
Results
Literature review and data abstraction
As seen in Table 1 in the RCTs that have compared DES with BMS, all included death as a component of MACE; however, the balance between the use of all-cause mortality versus cardiac-only mortality was about equal. In addition, MI was included as part of MACE in all trials with only 1 trial restricting the definition to Q-wave MI. For the remaining components of MACE, the trials were variable with a minority including ST, all including either target lesion revascularization (TLR) or TVR, and a few including coronary artery bypass grafting or stroke. Thus, even with a very specific research question (DES vs. BMS), it is clear that there was no consensus definition of MACE.
RCTs of BMS Versus DES With MACE as an Outcome
In the review of publications in the Journal for calendar year 2006, the results were more variable (Table 2). Like the RCT review, death and MI were included in all definitions of MACE, although in the Journal review, nearly all included all-cause mortality rather than cardiac-only mortality. The remaining components of MACE were highly variable, with most including a repeat revascularization component (e.g., TLR, TVR) and a range of other outcomes including stroke, acute coronary syndrome, restenosis on angiographic follow-up evaluation, and congestive heart failure hospitalization. Thus, in the absence of a specific research question (i.e., DES vs. BMS), the definition of MACE was even more heterogeneous.
Use of the Term MACE as an Outcome in the Journal (2006)
Empirical investigation
For the 3 constructed definitions of MACE, the numbers of incident events during 1-year follow-up in the DEScover registry were as follows: 1) death, MI, or ST (362 events); 2) death, MI, ST, or TVR (674 events); and 3) death, MI, ST, or any repeat revascularization (868 events). Thus, when TVR was added to the safety end point of death, MI, or ST, the number of incident events nearly doubled. When any repeat revascularization was added to the safety definition of MACE, the number of incident events increased 2.4-fold.
In the first comparison (Fig. 1, top), patients presenting with acute MI (vs. no acute MI) were at significantly higher adjusted risk of the safety only definition of MACE (death, MI, or ST) (adjusted hazard ratio 1.75, 95% confidence interval [CI] 1.31 to 2.34). In contrast, when TVR was added to the definition of MACE, the adjusted risk associated with the presentation of acute MI was substantially attenuated (diluted) and no longer statistically significant (adjusted hazard ratio 1.20, 95% CI 0.95 to 1.51). Similar results were observed when any repeat revascularization rather than TLR was part of the MACE definition (adjusted hazard ratio 1.14, 95% CI 0.92 to 1.40).
Adjusted Hazard Ratios for Different MACE Definitions
Adjusted hazard ratios of different definitions of major adverse cardiac events (MACE) comparing acute myocardial infarction (MI) versus nonacute MI patients (top) and patients with multilesion versus single-lesion percutaneous coronary intervention (bottom). Filled center circles depict the adjusted hazard ratios, filled circles at the left and right ends depict the lower and upper 95% confidence limits. Revasc = revascularization; ST = stent thrombosis; TVR = target vessel revascularization.
In the second comparison (Fig. 1, bottom), the use of multilesion PCI versus single-lesion PCI was not significantly associated with the safety only definition of MACE (death, MI, or ST) (adjusted hazard ratio 1.06, 95% CI 0.77 to 1.48). In contrast, when TVR was added to the definition of MACE, the adjusted hazard ratio associated with multilesion PCI was substantially higher and statistically significant (adjusted hazard ratio 1.41, 95% CI 1.13 to 1.75).
Discussion
In this analysis, we have shown that in clinical investigations, there is significant heterogeneity in the individual outcomes used to define composite end points such as MACE, and depending on the particular set of clinical outcomes used, widely different results and conclusions may be obtained even within a single study. In particular, in the setting of PCI, when mixing outcomes presumed to relate to safety (i.e., death, MI, ST) with outcomes presumed to relate to procedural effectiveness (e.g., TVR or repeat revascularization at large), the influence of postulated risk factors (e.g., acute MI) and procedural strategies (e.g., multilesion PCI) may be substantially attenuated or increased. Collectively, these observations warrant a reappraisal of the use of MACE and composite end points at large.
Given the seemingly obvious heterogeneity in individual safety and effectiveness outcomes used to construct MACE, one wonders why its use has become so pervasive in cardiology research. Importantly, when the term MACE was initially used, its purpose was to evaluate the “net effect,” that is, both the potential utility (effectiveness) and hazardousness (safety) of a new acute intervention. In that setting, its use was warranted in order to assess the true overall acute effect of a particular intervention. Unfortunately, over the years the use of MACE has extended to reporting of intermediate- and long-term outcomes in which both effectiveness and safety outcomes are put together, even when one is not necessarily interested in assessing the “net effect” of a particular intervention. Thus, this use of MACE, as shown in our analysis, can lead to different conclusions and interpretations of the “effect” of a particular intervention.
A second primary reason that may explain the widespread use of MACE in cardiology research is a general belief that using multiple outcomes to construct a definition of MACE will yield a large number of incident events that, in turn, will increase statistical power to demonstrate statistical significance. While this may occur in many circumstances, it is by no means absolute since, as shown in our analysis, some definitions of MACE may substantially attenuate relative risks (i.e., effect of acute MI), which would tend to bias results away from statistical significance.
A third possible reason, as stated in a recent editorial (56), is the potential for certain sponsors to “game” their trials by construction and use of a particular composite outcome. For example, as recently demonstrated (57), individual component outcomes that are considered minor (e.g., nonfatal angina) or moderate (e.g., coronary revascularization) tend to occur more frequently than deaths or other critical outcomes (e.g., cardiac arrest). Therefore, an apparent positive effect of a particular therapy may be present primarily, or exclusively, for only the individual outcomes that are of lesser clinical importance (57,58). By analogy, this circumstance was observed in our analysis of the effect of multilesion versus single-lesion PCI when the outcome of repeat revascularization (i.e., lesser clinical importance and more common than death, MI, or ST) was added to the definition of MACE, and statistically significant results emerged.
The potential for misleading conclusions depending on the study-specific definition of MACE used is not trivial. Consider a comparison of BMS to DES in which the definition of MACE includes TVR; if the latter had a lower hazard ratio of having MACE, one could erroneously conclude that DES are significantly better at reducing death, MI, ST, and TVR in totality, even though the “significant” effect on MACE would likely be being driven primarily or solely by a reduction in TVR.
In light of the approximate prior 15 years use of the term MACE and its wide heterogeneity in definition and research applications, it is unlikely that a consensus definition will either be universally desired or practical for future research. Therefore, we recommend against the routine use of MACE as a composite end point at large. However, if a broad heterogeneous composite end point such as MACE is ultimately desired, minimally, it must be clearly defined, and the individual as well as composite end points need to be analyzed, presented, and discussed. If different definitions are used for even 1 component of the composite end point, rates of the composite end point may vary widely. To illustrate, in TAXUS-I (12), only Q-wave MI was included in the definition of MACE, and there were no MIs at 30 days or 1-year in either the TAXUS or control groups. In contrast, in TAXUS-V among patients with complex disease (25), the 30-day MACE rates of 5.1% and 3.6% in the TAXUS and control groups, respectively, were dominated by the inclusion of non–Q-wave MI rates of 4.4% and 3.3%, respectively.
Our general recommendation against the use of MACE is consistent with that of the Academic Research Consortium (ARC) (59), which has aimed to establish consensus definitions for both individual and composite DES study end points. The ARC has suggested 2 composite end points for DES trials: a device-oriented and overall patient-oriented end point, whereas for cardiology studies at large (i.e., not restricted to DES trials), we recommend focusing separately on safety and effectiveness outcomes, and constructing separate composite (and sample size calculations, among others) end points that contain well-defined internally coherent components to match these different clinical entities.
Footnotes
Supported, in part, by Cordis Corporation (a Johnson & Johnson Company). Drs. Kip and Williams have received research support from Cordis Corporation, and Drs. Williams and Marroquin have served as consultants for Cordis Corporation.
- Abbreviations and Acronyms
- ARC
- Academic Research Consortium
- BMS
- bare-metal stent(s)
- CI
- confidence interval
- DES
- drug-eluting stent(s)
- MACE
- major adverse cardiac events
- MI
- myocardial infarction
- PCI
- percutaneous coronary intervention
- RCT
- randomized clinical trial
- ST
- stent thrombosis
- TLR
- target lesion revascularization
- TVR
- target vessel revascularization
- Received May 2, 2007.
- Revision received September 4, 2007.
- Accepted October 29, 2007.
- American College of Cardiology Foundation
