Author + information
- Received May 21, 2012
- Revision received July 20, 2012
- Accepted August 7, 2012
- Published online April 16, 2013.
- Joshua P. Loh, MBBS,
- Israel M. Barbash, MD and
- Ron Waksman, MD⁎ ()
- ↵⁎Reprint requests and correspondence:
Dr. Ron Waksman, MedStar Washington Hospital Center, 110 Irving Street, Northwest, Suite 4B-1, Washington, DC 20010
The CardioMEMS Champion Heart Failure Monitoring System (CardioMEMS, Atlanta, Georgia) is a permanently implantable pressure measurement system designed to wirelessly measure and monitor pulmonary artery (PA) pressure and heart rate in heart failure (HF) patients to guide ambulatory HF management and to reduce HF hospital stays. On December 8, 2011, the Food and Drug Administration (FDA) Circulatory System Device Panel reviewed the CardioMEMS Champion HF Monitoring System premarket approval (PMA) application. The majority of Panel members agreed that that the discussed monitoring system is safe for use in the indicated patient population. However, new information reported by the FDA with regard to preferential support in management of patients in the treatment group raised concerns among the Panel members with regard to potential bias in analyzing the efficacy of the device itself. Additionally, Panel members raised concerns with regard to the efficacy of the device in certain patient subpopulations. Hence, most Panel members decided that there was not reasonable assurance that the discussed monitoring system is effective. This summary aims to describe the discussions and recommendations made during this meeting.
The CardioMEMS Champion Heart Failure Monitoring System (CardioMEMS, Atlanta, Georgia) is a permanently implantable pressure measurement system designed to provide daily pulmonary artery (PA) pressure measurements in an ambulatory setting and thus help guide heart failure (HF) management in an outpatient setting to reduce HF hospital stays. The system consists of an implantable sensor/monitor, which is a battery-free capacitive pressure sensor permanently implanted in the PA; a transvenous delivery system designed to deploy the implantable sensor in the distal PA; and the Champion Electronics System (CardioMEMS), which acquires and processes signals from the implantable sensor/monitor and transfers PA pressure measurements to a secure database accessible by the treating physician.
On December 8, 2011, the Food and Drug Administration (FDA) Circulatory System Devices Panel reviewed the CardioMEMS Champion HF Monitoring System premarket approval (PMA) application. Meeting materials can be accessed at the FDA website (1).
The PMA application was primarily based on data from the pivotal, prospective, multi-centered, randomized, single-blinded CHAMPION (CardioMEMS Heart Sensor Allows Monitoring of Pressure to Improve Outcomes in NYHA Class III Heart Failure Patients) trial, which was published in the Lancet earlier this year (2), which consisted of patients in New York Heart Association (NYHA) functional class III HF with a previous hospital stay within the last year; 270 patients were randomized to PA pressure-guided treatment of HF compared with 280 in the control arm. Of note, all patients underwent implantation of the device, but no PA pressure measurements were performed in the control group.
The following is a summary of the discussions and recommendations made during the Circulatory System Devices Advisory Panel meeting.
On behalf of the trial sponsor, Dr. William Abraham, the Principal Investigator of the CHAMPION trial, presented the trial design and the major clinical outcomes of the pivotal trial. According to the published trial results in terms of primary efficacy, there was a statistically significant risk reduction from 0.44 to 0.32 in heart failure-related (HFR) hospital stays/patient/6 months, with a relative risk reduction of 28% and an absolute risk reduction of 0.12 HFR hospital stay events, demonstrating the superiority of the treatment arm (p = 0.0002). The supplementary efficacy endpoint demonstrated continuous, significant, risk reduction in HFR hospital stays from 0.73 to 0.46 (p < 0.0001), with a 37% relative risk reduction over the full study duration (Fig. 1). The time to first event analysis, combining death from any cause and HFR hospital stay for the full study duration, significantly favored the treatment group with a hazard ratio of 0.73 (95% confidence interval: 0.57 to 0.94, p = 0.015) (Fig. 2).
Although the pre-specified statistical model used by the Sponsor showed a highly significant treatment effect (p = 0.0002), the analysis did not take into account the greater than expected variability in the CHAMPION dataset with regard to the primary endpoint of number of hospital stays based on that model (i.e., the observed variance was larger than the observed mean). Moreover, many patients did not require hospital stay during the follow-up period. If this over-dispersion was not appropriately accounted for, the estimation of standard errors would be too small, thereby leading to smaller p values. As presented by Dr. Yonghong Gao, FDA statistician, alternative statistical models were used to correct for this over-dispersion and indicate that, under certain circumstances, the significant effect of the CardioMEMS Champion HF monitoring system on the primary endpoint of the trial disappears. Hence, the FDA concluded that the statistical methods used by the Sponsor for analysis of the primary efficacy endpoint were not robust and indicated that the potential benefit of the device is so marginal that as few as 13 more HFR hospital stays (from 84 to 97) in the treatment arm would render the results as no longer statistically significant. For an alternative bootstrap model, even 2 additional hospital stays in the treatment arm converted the p value to exceed 0.1.
Trial Conduct Issues
Auditing performed by the FDA Division of Bioresearch Monitoring after completion of patient enrollment and after most of the follow-up period identified evidence that the Sponsor and national Principal Investigators, who were not blinded to the randomization assignment, routinely contacted investigational sites and made specific therapeutic recommendations for some treatment group study subjects via electronic communication. According to Drs. Randall Brockman and Gregory Campbell of the FDA, such communications included but were not necessarily limited to titration of medication doses, addition or discontinuation of medications, recommendations for outpatient intravenous medication administration, addition of medications that were not in the protocol and sleep study evaluations that were not included in the protocol. These “auxiliary interventions” occurred only in the treatment arm patients.
According to the FDA, this mode of preferential communication introduces a significant bias to the study and ultimately disallows objective assessment of the inherent effect of the CardioMEMS Champion HF monitoring device in the management of HF patients.
Incorporating the data as presented by the FDA and the Sponsor, it seemed that although both sides conjointly designed and prospectively agreed on a specific protocol for the CHAMPION trial, there were differences in the interpretation of certain portions (i.e., Appendix E of the CHAMPION trial protocol; see FDA executive summary—Addendum) (3). The approved protocol allowed the Sponsor to review the PA pressure readings and to alert investigators if the pressure readings were beyond set alert limits. The Investigators (or Designees) were to manage the enrolled subjects and make medication changes according to the recommendations in Appendix E. However, the level of interaction between the Sponsor and Clinical Investigators on specific treatment group subjects was inconsistent with the expectations of the FDA and was unbalanced with respect to the control group.
In response to the FDA claims, Dr. William Abraham, Principal Investigator of the CHAMPION Trial, presented data to show that the absolute number of those emails was low (Fig. 3). He showed that these communications did not significantly affect rates of medication changes/patient/month or affect the outcome of HFR hospital stays when the treatment group was stratified into those who received emails and those who did not (Table 1). The Sponsor also presented data showing that even after the un-blinded period with no further follow-up emails, there was no significant change in patient outcome in the treatment group (Table 2). Finally, the Sponsor claimed that the proportion of patients receiving such emails was very small and even in a worst case scenario it would have affected very few patient outcomes.
The FDA was not reassured by the presented data, because there was reasonable doubt whether the FDA had access to all of the questionable email communications and possible telephone conversations. Furthermore, even if the outcomes of only a few patients in the treatment group were affected by these extra-curricular conversations, the results of the entire study might have depicted an insignificant difference between the 2 study groups.
The Advisory Panel members had an extensive discussion with regard to the conduct of the CHAMPION trial and to the validity of the efficacy results. The consensus of the Advisory Panel was that a substantial bias was introduced into the study results, due to the preferential assistance in the management of the treatment arm patients. Moreover, it was difficult to distinguish the inherent benefits of the device versus the effects of the device combined with HF expert management guidance. Unfortunately, these auxiliary activities were done in a manner that precluded any attempts to exclude the involved patients and any attempts to perform post hoc analysis to evaluate a subgroup of patients who did not have such interventions. An additional concern to the Panel was that these communications were identified at a very late stage in the trial, at which point it was too late to reconcile them to salvage the study.
An additional area of disagreement between the FDA and the Sponsor was the clinical efficacy of the Champion HF Monitoring System for certain subpopulations. Dr. Yonghong Gao of the FDA presented a post hoc analysis of HFR hospital stay rates according to sex (Table 3). The data indicated significant interaction between sex and treatment, suggesting that the device has differential treatment effects on men and women. The intervention in the treatment arm reduced the hospital stay rate for men, but there was a nonstatistically significant increase for women at the 6-month follow-up period. When the data were analyzed over the entire study duration, treatment effect was again shown in men but not in women (Fig. 4).
In response, the Sponsor highlighted that the small number of events (14 HFR hospital stays among 75 patients) in the female control group might result in statistical artifacts and prevents drawing meaningful conclusions. The Sponsor elaborated further that, because there was an excess in deaths in the female control group (7 control deaths vs. 3 treatment deaths, hazard ratio = 0.41, p = 0.187), the women in the control group most likely to have HF hospital stays had a reduced duration in the study and reduced contribution to the primary endpoint of HFR hospital stays. When both mortality and HFR hospital stays were considered together in the endpoint, the apparent disparity in the results for women disappeared. In the univariable analysis presented by the Sponsor, the apparent interaction between treatment and sex was explained by other risk factors present at baseline; this interaction was no longer statistically significant (p = 0.940) after the multivariable adjustment.
During the Advisory Panel deliberations, most Panel members agreed that the small number of events in women were not powered to look at differences in sex, but there was some evidence of sex and treatment interaction. The Panel concluded that the data were not sufficient for assessing the impact of the system on women.
Another subgroup analysis discussed during the Panel meeting was HF patients with preserved left ventricular ejection fraction (LVEF) (Fig. 5). The Sponsor presented data that indicated that patients with preserved LVEF had lower hospital stay rates in the treatment group as compared with the control group (0.18 vs. 0.33 HFR hospital stay/patient/6 months; p <0.001).
The Panel agreed that these results were significant across the subgroups of reduced and preserved LVEF, and this trial represented 1 of the first successful management strategies to reduce hospital stay risks for HF patients with preserved LVEF.
In the CHAMPION trial, primary safety endpoints were based on 550 randomized and 25 consented nonrandomized, not implanted patients. Because all randomized patients had implantation of the pressure sensor and there was no real control group for safety, the comparison of safety endpoints with pre-specified Objective Performance Criteria (OPC) was performed. These OPC were based upon the complication and failure rates for other HF monitoring devices and similar to OPC accepted by the FDA. The Panel agreed with the FDA that the primary safety endpoint seemed to have been met for freedom from device/system-related complications at 6 months (98.6%, lower 95% confidence limit = 97.3%, compared with pre-specified OPC of >80%). Freedom from sensor failure at 6 months (100%, lower 95% confidence limit = 99.3%, compared with pre-specified OPC of >90%) was also met.
The Panel was asked to vote on 3 questions with respect to the approvability of the CardioMEMS Champion HF Monitoring System. The Panel voted 9 to 1 that there is reasonable assurance that the CardioMEMS Champion HF Monitoring System is safe for use in patients with NYHA functional class III HF who meet the criteria specified in the proposed indication. One Panel member felt that, although the implantation of the device was safe, there had been conflicting data on women, which represented part of the intended population, and he could not be reasonably assured that it was safe for use in that group. Furthermore, the application of the device in making decisions on PA pressures without the entire support system might potentially cause harm.
The Panel voted 7 to 3 against the statement that there is reasonable assurance that the CardioMEMS Champion HF Monitoring System is effective for use in patients with Class III HF who meet the criteria specified in the proposed indication. Most Panel members who voted negatively on this question reiterated the following points: the treatment effect seen in the trial was a combination of device and support mechanism; there was no way to tease out the treatment effect of each; and major elements in the support system will not be applicable in the general population. One Panel member who voted against also mentioned the conundrum of the treatment effect on women. The Panel members who voted “for” considered the entirety of the treatment effect and voiced hope that this support system could be replicated by the Sponsor outside of the trial setting.
The Panel voted 6 to 4 against the statement that the benefits of the CardioMEMS Champion HF Monitoring System outweigh the risks for use in patients who meet the criteria specified in the proposed indication.
Expedited Publication Versus Lingering Approval
In this FDA advisory panel meeting, most of the discussion focused on the study design and conduct and how these impacted the results of the CHAMPION pivotal trial. Interestingly, the results of the CHAMPION trial were published earlier this year in the Lancet and were accompanied by a positive editorial piece suggesting that, on the basis of the trial results, strong consideration should be given to device implantation in Class III HF patients (2,4). However, auditing performed by the FDA uncovered certain conduct issues that challenged the validity of the published results. This type of auditing and the availability of the FDA advisory panel are obviously beyond the scope and resources of any specific journal. Although medical journals are focusing on publishing late-breaking trials expeditiously, the regulatory agencies are charged to conduct full investigations into the accuracy of the results and the integrity of its conduct. These are usually lingering, and the FDA is often criticized for slowing the approval process. In the case of the CHAMPION trial, due diligence by the FDA uncovered a vastly different trial conduct and hence a different trial interpretation from that contained in the Lancet. This has led to a disparity between praised study results published in a leading medical journal and contradictory interpretation by a respected panel of experts who advised the FDA that there is lack of reasonable assurance this technology is effective in reduction for the need of repeat hospital stay in patients with HF.
Although the CardioMEMS technology might be valuable for monitoring patients with HF, it might now take a new study to prove beyond a reasonable doubt the effectiveness of the technology and, if required, as pre-market rather than post-market. This might further delay the approval of a potentially valuable device. This case underscores the need for close and early monitoring of pivotal clinical trials. Had the FDA identified its findings in the early phase of the study, the deficiencies could have been corrected and the efficacy controversy could have been eliminated. Thus it is reasonable to consider a new FDA policy of performing a preliminary audit early on after initiation of Investigational Device Exemption clinical trials, so that such issues as the 1 encountered in this PMA submission would be identified and corrected in a timely fashion without jeopardizing the entire trial.
This FDA Circulatory System Devices Panel meeting on the CardioMEMS Champion Heart failure monitoring system was unusual. The Panel commended the Sponsor on this innovative technology and its potential for adding value to HF management. The FDA took a very unusual step in devoting a substantial portion of the discussion to challenge the trial conduct and, hence, the validity of the data presented. Finally, the Panel decided that the inherent efficacy of the device could not be determined by the results presented to the Panel. It will be interesting to follow what further steps the FDA and the Sponsor will take to bring this promising technology to a resolution with regard to its future clinical use.
The authors have reported that they have no relationships relevant to the contents of this paper to disclose.
- Abbreviations and Acronyms
- Food and Drug Administration
- heart failure
- heart failure-related
- left ventricular ejection fraction
- New York Heart Association
- objective performance criteria
- pulmonary artery
- premarket approval
- Received May 21, 2012.
- Revision received July 20, 2012.
- Accepted August 7, 2012.
- American College of Cardiology Foundation
- US Food and Drug Administration
- U.S. Food and Drug Administration