Author + information
- Received March 3, 2013
- Revision received April 2, 2013
- Accepted May 1, 2013
- Published online August 27, 2013.
- ∗Division of Cardiology, University of Colorado School of Medicine, Aurora, Colorado
- †Denver Health Medical Center, Denver, Colorado
- ‡Colorado Prevention Center, Aurora, Colorado
- §Division of Cardiology, Cedars-Sinai Medical Center, Los Angeles, California
- ‖David Geffen School of Medicine at UCLA, Los Angeles, California
- ↵∗Reprint requests and correspondence
: Dr. Mori J. Krantz, Denver Health Medical Center, 777 Bannock Street, MC 0960, Denver, Colorado 80204.
Rivaroxaban is a factor Xa inhibitor that was recently reviewed by the Food and Drug Administration as a potential therapy to reduce the risk of recurrent atherothrombotic events in patients with acute coronary syndromes. Approval of this drug would represent a paradigm shift away from dual antiplatelet therapy toward long-term triple antithrombotic therapy. However, to date, no other experimental anticoagulant agent has demonstrated a favorable risk-benefit profile in this population, in part because of the expected increased risk in major bleeding by combining aspirin, a P2Y12 receptor inhibitor, and an anticoagulant. Approvability of rivaroxaban was considered largely on the basis of the ATLAS ACS 2–TIMI 51 (Anti-Xa Therapy to Lower Cardiovascular Events in Addition to Standard Therapy in Subjects With Acute Coronary Syndrome ACS 2–Thrombolysis In Myocardial Infarction 51) trial, which demonstrated a significant reduction in a composite of cardiovascular death, myocardial infarction, and stroke. Although the primary efficacy endpoint was met, a substantial amount of missing data was observed. We discuss the impact of missing data in this trial, its implications for informative censoring of safety events (major bleeding), and implications for future cardiovascular outcomes trials.
The acute coronary syndromes (ACS) are the most common emergent manifestation of cardiovascular disease, occurring in 1,190,000 patients annually (1). Previously, aspirin monotherapy was the cornerstone of antithrombotic treatment for ACS (2). Since that time, dual antiplatelet therapy (aspirin plus a P2Y12 receptor inhibitor) has emerged as the current therapeutic standard, providing significant incremental benefit above aspirin alone (3). Despite optimal medical therapy, including newer, more potent P2Y12 receptor inhibitors, patients with ACS remain at risk for recurrent vascular events. Upon hospital admission for patients with ACS, thrombin levels are elevated, and this pattern persists for at least 6 months after the initial event (4). This suggests a potential opportunity to lower the risk of recurrent ischemic cardiovascular (CV) events through either direct thrombin or factor Xa inhibition.
The Cardiovascular and Renal Drugs Advisory Committee (CRDAC) of the U.S. Food and Drug Administration (FDA) convened a public meeting on May 23, 2012, to review a supplemental new drug application for rivaroxaban (Janssen Pharmaceuticals, Inc., New Brunswick, New Jersey), a novel factor Xa inhibitor to reduce the risk of thrombotic CV events in patients with ACS in combination with aspirin alone or with aspirin plus a thienopyridine, so-called triple antithrombotic therapy (5). Rivaroxaban has been previously approved for 4 clinical indications in the United States: thromboprophylaxis following total knee replacement surgery; thromboprophylaxis following total hip replacement surgery; to reduce the risk of stroke and systemic embolism in patients with nonvalvular atrial fibrillation; and for treatment of deep vein thrombosis and pulmonary embolism. The CRDAC voted 6-4 (with 1 abstention) against recommending approval of rivaroxaban for an expanded ACS indication. Subsequently, on June 21, 2012, the FDA submitted a complete response letter to the sponsor seeking additional information prior to considering market approvability (6). In this commentary, we highlight the key considerations that drove deliberations of the CRDAC, with particular emphasis on missing data from the ATLAS ACS 2–TIMI 51 (Anti-Xa Therapy to Lower Cardiovascular Events in Addition to Standard Therapy in Subjects With Acute Coronary Syndrome ACS 2–Thrombolysis In Myocardial Infarction 51) trial (7).
The supplemental new drug application for rivaroxaban focused on ATLAS ACS 2–TIMI 51, a single pivotal phase III trial of rivaroxaban on a background of standard dual antiplatelet therapy including aspirin and either clopidogrel or ticlopidine among patients with recent ACS. ATLAS ACS 2–TIMI 51 was a randomized, double-blind, placebo-controlled, event-driven study to evaluate the efficacy and safety of rivaroxaban among 15,526 patients. The primary objective was to demonstrate superiority of rivaroxaban compared with placebo in reducing the major adverse cardiac events (MACE) composite of CV death, myocardial infarction (MI), and stroke. Two doses, 2.5 and 5 mg, were evaluated in patients treated with aspirin therapy alone (stratum 1) or dual antiplatelet therapy (stratum 2), which constituted the majority of patients. The primary evaluation strategy was based on a modified intention-to-treat (mITT) analysis of data combined across both strata (i.e., all strata). A second evaluation was based on the FDA-recommended approach of combined analyses across both dose regimens in patients in stratum 2 (dual antiplatelet therapy) only. The principal study results are summarized in Table 1. Overall, treatment with rivaroxaban, combined doses as well as the 2.5-mg dose, significantly reduced MACE in both stratum 2 and all strata; however, the 5-mg dose achieved statistical significance only in all strata.
Despite seemingly robust efficacy data, several key issues were brought up during the CRDAC meeting that challenge the validity of the ATLAS ACS 2–TIMI 51 trial results. First and foremost, an unanticipated high rate of missing data, particularly the vital status of patients, precludes reliable and valid information. Second, there was a lack of an expected dose response—the 5-mg dose did not have greater efficacy compared with the 2.5-mg dose of rivaroxaban (Table 1). Establishment of dose or exposure response is an important consideration in regulatory decision making. Third, it is difficult to reconcile the results with the divergent impact of the 2 doses on the components of the primary composite endpoint of MACE—CV death, but not MI, driving the treatment benefit with 2.5 mg, whereas MI, but not CV death, driving benefit with the 5-mg dose (Table 1). The increase in bleeding with the higher dose did not account for the null effect on CV death. Fourth, there is a lack of supportive external evidence for incremental benefit associated with novel oral anticoagulants in ACS beyond standard dual antiplatelet therapy—for example, treatment with dabigatran (REDEEM [Randomized Dabigatran Etexilate Dose-Finding Study in Patients With Acute Coronary Syndromes]) (8), apixaban (APPRAISE-2 [Apixaban for Prevention of Acute Ischemic Events 2]) (9), and vorapaxar (TRACER [Thrombin Receptor Antagonist for Clinical Event Reduction in Acute Coronary Syndrome]) (10) failed to demonstrate reductions in MACE and, given the excess bleeding risk, suggested an unfavorable benefit-risk balance. These observations were recently confirmed in a meta-analysis that reported that the addition of direct thrombin or factor Xa inhibitor to standard antiplatelet therapy was associated with a modest benefit in reducing MACE (hazard ratio: 0.87; 95% confidence interval [CI]: 0.80 to 0.95) but more than a doubling of risk of major bleeding (hazard ratio: 2.34; 95% CI: 2.06 to 2.66) (11). Finally, there was a lack of a statistically persuasive efficacy benefit—generally defined by the FDA as equivalent to p < 0.001 in a single superiority trial setting (12), which was not achieved for the primary adjudicated efficacy endpoint in the ATLAS ACS 2–TIMI 51 trial.
Although many of these issues are pertinent to the ultimate approvability and clinical utility of rivaroxaban as part of triple antithrombotic therapy, we focus on the critical issue of missing data that dominated the CRDAC panel discussion, given the threat it poses to the validity of the ATLAS ACS 2–TIMI 51 trial as well as all future CV outcomes trials.
Missing Data in ATLAS and Contemporary ACS Trials
In the ATLAS ACS 2–TIMI 51 trial, 2,402 patients (15.5%) prematurely discontinued from the study, with 1,294 patients (8.3%) withdrawing consent. At the end of the trial, vital status was not ascertained in 1,117 of the 1,294 patients who withdrew consent. By contrast, the rates of withdrawal of consent and in particular, missing vital status, in contemporary randomized ACS trials are appreciably lower (Table 2). The degree to which missing data impacted overall interpretability of the trial results was the principal concern of FDA clinical and statistical reviewers (5). Although there is no regulatory guideline that stipulates an acceptable level of missing data, the following “rule-of-thumb” considerations proposed by Schulz and Grimes (13) may facilitate judgments regarding the impact of “missingness” on the interpretability of clinical trial results.
1. If the loss to follow-up rate exceeds the outcome event rate, results might be questionable.
2. If missing data are <5%, the bias will be minimal; however, if >20%, it poses a serious threat to the validity of the study.
3. If missing data are differential by treatment group, results may be biased, especially if losses are related to treatment efficacy or tolerability.
4. If there are missing data that, when subjected to sensitivity analyses, yield different results, study conclusions are less certain.
In the ATLAS ACS 2–TIMI 51 trial, the number of patients with unknown vital status (n = 1,117) exceeded the total number of primary endpoint events (n = 1,002). Although the extent of missing data was <20%, there was differential missingness for MACE assessment (1.4% greater with rivaroxaban: 11% placebo vs. 12.4% combined rivaroxaban). The difference in the missing data nearly matched the difference in the primary outcome (1.2% favoring rivaroxaban: 7.3% placebo vs. 6.1% combined rivaroxaban), providing ample opportunity to amplify or obscure any true difference in endpoints. Numerous sensitivity analyses, including ITT, per protocol, treatment emergent (plus 2, 7, or 30 days), mITT per investigator, were consistent with the results of the primary efficacy analysis, thereby providing some reassurance that rivaroxaban (combined, 2.5 mg twice daily) significantly reduced the rate of the primary endpoint.
An extension of the missing data concern is the potential to lead to “informative censoring” (i.e., patients who drop out [censored] are either more or less likely to experience the primary outcome of interest compared with those remaining in the trial in a nonrandom fashion). That concern can be compounded if the reasons for, or frequency of, dropout differs between the treatment groups. This is particularly relevant in trials of antithrombotic drugs, such as rivaroxaban, in which one expects the study agent to preferentially increase bleeding, thereby leading to greater discontinuation and dropouts relative to placebo. Because bleeding has been linked to both short-term and long-term increased risk of ischemic CV events and mortality (14), increased bleeding-related dropouts are likely to bias the results toward therapeutic benefit of the study drug. Not surprisingly, in the ATLAS ACS 2–TIMI 51 trial, compared to patients with complete follow-up, TIMI minor or greater bleeding rates were higher in patients with incomplete follow-up by 3-fold in the placebo group (3.1 vs. 0.9 per 100 patient-year exposure), 4.5-fold in the 2.5-mg rivaroxaban treatment group (6.3 vs. 1.4 per 100 patient-year exposure), and 5-fold in the 5-mg rivaroxaban treatment group (9.0 vs. 1.8 per 100 patient-year exposure) (15). It is also clear that discontinuation had a differential impact among drug and placebo groups (i.e., placebo withdrawals were much less likely to have adverse bleeding events than the withdrawals in the rivaroxaban group), illustrating the potential of informative censoring to bias results in favor of active treatment. Furthermore, in patients with complete follow-up, MI rates were 2- to 3-fold higher and mortality rates 5-fold higher in patients who experienced bleeding versus those who did not. Because bleeding led to both patient withdrawals and an increase in MACE, this suggests that a true ITT analysis that includes all randomized patients is preferred for clinical trials over on-treatment analyses to reduce informative censoring.
Evaluating the amount of missingness that is considered “tolerable” depends on how robust the efficacy results are. For marginally significant results, even low levels of missingness can become relevant. In this regard, a sensitivity analysis that evaluates the number of excess events required to overturn statistical significance might be instructive. In the worst-case scenario analysis that assumes that excess MACE occur only in the rivaroxaban treatment arm, the number of excess MACE required to overturn statistical significance is 7 with the 2.5-mg dose in stratum 2 (the pivotal analysis for approval), 13 with the 2.5-mg dose in the combined strata, or 40 with the combined doses in all strata (15). These are well within the clinically plausible range of excessive events required to nullify the rivaroxaban treatment benefit in the ATLAS ACS 2–TIMI 51 trial. For instance, the number of TIMI minor or greater bleeding events reported in rivaroxaban-treated patients with incomplete follow-up and no adjudicated MACE was 98 (15). Given that 37% of patients with TIMI minor or greater bleeding experienced MACE in the overall cohort, this potentially translates into an expected 36 additional MACE events (98 × 0.37) had these patients undergone complete follow-up (15). That as few as 7 excess MACE are required to nullify a rivaroxaban efficacy benefit exposes the fragility of the ATLAS data and amplifies the potential impact of missing data on treatment outcomes. Thus, the totality of the evidence suggests the potential for missing data in the ATLAS ACS 2–TIMI 51 trial to have a material impact on trial interpretation.
Minimizing the Missing Data Problem in Clinical Trials
In 2010, the National Resource Council reported their main findings and recommendations on how to address the problem of missing data as part of the FDA's plan to develop guidance for drug and device companies on clinical trial design (16). A summary of these policy recommendations were recently published (17), emphasizing 3 common scenarios for missing data—missing completely at random, missing at random, and missing not at random. In addition, core issues that might help limit missing data focused on rigorous trial design and conduct and the types of adjustment methods for missing data—complete case analysis, imputation approaches such as the last or baseline observation carried forward, estimating-equation methods, and methods based on statistical modeling. Although imputation of the missing data allows the analysis to conform to an ITT analysis, it requires plausible assumptions (e.g., missing at random) that may be difficult to verify. Thus, “prevention” of missingness rather than “treatment” remains the optimal approach to limit the problem and subsequently enhance the credibility of causal inferences from clinical trials.
Missing data is a common problem in clinical research and can complicate interpretation or even invalidate an otherwise important study. The Federal Food, Drug, and Cosmetic Act provides that for FDA to grant approval for a new drug, there must be “substantial evidence” of efficacy derived from “adequate and well-controlled investigations.” Although several issues related to trial design, conduct, and analysis can have a material impact on what constitutes “adequate and well-controlled investigations,” we and others remain concerned that missing data are generally either not recognized as a major issue or considered a nuisance that is best ignored. The ATLAS ACS 2–TIMI 51 trial highlights the potential impact of missing data on precluding definitive causal inferences in pre-marketing registration trials.
On September 6, 2012, the sponsor submitted to the FDA important data related to patients who had withdrawn from the ATLAS ACS 2–TIMI 51 trial as part of its complete response. Despite the availability of vital status in 843, or 63%, of the 1,338 trial participants who previously had unknown vital status, the FDA issued a second complete response letter on March 4, 2013 (18). Although the contents of the complete response letter have not been revealed, we presume that the FDA continues to have lingering concerns with missing data because the residual level of missing vital status remained higher than that in contemporary ACS trials. On March 21, 2013, the European Medicines Agency Committee for Medicinal Products for Human Use granted an ACS indication for rivaroxaban 2.5 mg in patients with ACS (19). Given these contradictory decisions, the FDA will do well to expeditiously endorse the National Resource Council policy recommendations in the form of explicit guidance for the drug-development industry regarding missing data. Until then, it will be increasingly difficult to “shrug off” the burden of missing data in trials like ATLAS and the additional complexity it poses to regulatory decision making. Whether triple antithrombotic therapy with novel anticoagulants, such as rivaroxaban, for ACS will ultimately become standard practice remains uncertain. This is particularly relevant given the availability of more potent P2Y12 receptor antagonists (20,21).
The authors thank William R. Hiatt for his review of an early draft of the manuscript.
Opinions expressed in this article are those of the authors and do not necessarily reflect the opinions of the Food and Drug Administration (FDA) or any other branch of the U.S. Department of Health and Human Services. Drs. Krantz and Kaul participated as voting members at the May 23, 2012, FDA Cardiovascular and Renal Drugs Advisory Committee meeting. Dr. Krantz has served as a consultant for Novo Nordisk, Abbott Vascular, Cameron Pharmaceuticals, and CoLucid Pharmaceuticals. Dr. Kaul owns stock in Johnson & Johnson.
- Abbreviations and Acronyms
- acute coronary syndrome(s)
- Cardiovascular and Renal Drugs Advisory Committee
- Food and Drug Administration
- major adverse cardiac event(s)
- myocardial infarction
- Received March 3, 2013.
- Revision received April 2, 2013.
- Accepted May 1, 2013.
- American College of Cardiology Foundation
- Rooger V.L.,
- Go A.S.,
- Lloyd-Jones D.M.,
- et al.
- Jneid H.,
- Anderson J.L.,
- Wright R.S.,
- et al.
- ↵U.S. Food and Drug Administration. May 23, 2012, Meeting of the Cardiovascular and Renal Drugs Advisory Committee. Available at: http://www.fda.gov/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/CardiovascularandRenalDrugsAdvisoryCommittee/ucm285415.htm. Accessed November 13, 2012.
- ↵U.S. Food and Drug Administration, Center for Drug Evaluation and Research. Approval Package for Application Number 202439Orig1s000. Available at: http://www.accessdata.fda.gov/drugsatfda_docs/nda/2011/202439Orig1s000Approv.pdf. Accessed November 13, 2012.
- Oldgren J.,
- Budaj A.,
- Granger C.,
- et al.,
- for the RE-DEEM Investigators
- Oldgren J.,
- Wallentin L.,
- Alexander J.H.,
- et al.
- ↵U.S. Department of Health and Human Services, Food and Drug Administration. Guidance for Industry-Providing Evidence of Effectiveness for Human Drug and Biological Products. Available at: http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/ucm078749.pdf. Accessed November 13, 2012.
- Rao S.V.,
- O'Grady K.,
- Pieper K.S.,
- et al.
- ↵Marciniak TA, for the U.S. Food and Drug Administration. NDA 202-439: Rivaroxaban for Acute Coronary Syndromes. Available at: http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/CardiovascularandRenalDrugsAdvisoryCommittee/UCM305921.pdf. Accessed November 13, 2012.
- National Research Council: The Panel on Handling Missing Data in Clinical Trials
- ↵FDA again rejects ACS indication for rivaroxaban (Xarelto). Forbes. Available at: http://www.forbes.com/sites/larryhusten/2013/03/04/fda-again-rejects-acs-indication-for-rivaroxaban-xarelto/. Accessed March 21, 2013.
- ↵Rivaroxaban gets ACS indication recommendation from European regulators. theheart.org. Available at: http://www.theheart.org/article/1521155.do?utm_medium=email&utm_source=20130322_breaking&utm_campaign=newsletter. Accessed March 22, 2013.
- ↵U.S. Food and Drug Administration. February 3, 2009, Meeting of the Cardiovascular and Renal Drugs Q 4 Advisory Committee. Available at: http://www.fda.gov/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/CardiovascularandRenalDrugsAdvisoryCommittee/ucm125999.htm. Accessed March 22, 2013
- U.S. Food and Drug Administration. Ticagrelor NDA 22–433 Briefing Document for Cardiovascular and Renal Drugs Advisory Committee Meeting. Available at: http://www.fda.gov/downloads/AdvisoryCommitteesMeetingMaterials/Drugs/CardiovascularandRenalDrugsAdvisoryCommittee/UCM220197.pdf. Accessed March 22, 2013.