Author + information
- ↵∗Address for correspondence:
Dr. James K. Kirklin, University of Alabama at Birmingham, Data Science and Cardiac Surgery, 1900 University Boulevard, Birmingham, Alabama 35294.
The paper by Blackstone et al. (1) in this issue of the Journal provides an insightful exploration into predictors of mortality on the heart transplant waitlist at a single institution. The unique and prescient aspect is the “dynamic” analytic display that follows a patient through his or her experience on the waitlist, continuously updating the predicted mortality based on adverse events and resampling of various laboratory indicators of subsystem function. Each recalculation generates a new solution to the hazard function equation, which translates the cumulative hazard into a mortality risk calculation. Theoretically, this method could provide a new paradigm for allocating organs based on a continuous risk model.
Blackstone et al. (1) are on the cutting edge of data science as it applies to medical decision making. As such, this paper speaks to much broader issues than just heart transplantation. The authors have accepted the challenge not only to develop unique modeling but also to manage the limitations of such data regarding accuracy, completeness, and inherent imprecision. In so doing, they have provided a platform to enhance our ability to leverage large datasets with a goal of improving medical decision making. Of course, this is nothing new for Blackstone; he has been a pathfinder ever since teaming with Dr. John Kirklin at the University of Alabama at Birmingham during the nascent period of cardiac surgery multivariable analyses, when they first began promoting parametric models of multiphase hazard function (2) to generate patient-specific predictions of outcomes. The evolution of these methods and their application is of interest, and noteworthy is the complex presentation of data analytics in this initial epoch (3), followed over the years by progressive simplification of the presentation that coincided with wider application and acceptance of the methods (4).
What barriers, therefore, remain to embracing the methods used in this analysis (1)? The mathematical and statistical framework is obviously sound, given the reputation of Blackstone’s group and the expert statistical reviews. The key is bridging the gap between innovations in statistical modeling and their widespread acceptance as being truly useful in generating inferences that will actually be applied in the clinical arena. Similar challenges exist in the application of “machine-learning” algorithms. The mystery that enshrouds this “black box” concept is, for many, a disincentive to actually embrace and use these methods in making decisions related to interventions with patient lives at stake. This brilliant paper faces some of the same challenges.
The main challenges lie in the communication of the model through the many depictions: how does one demonstrate the usefulness of the method in a way that shows its power yet maintains sufficient simplicity to clearly communicate the logic behind the model and its relevance to clinical decisions? A useful adjunct to complex multivariable modeling is to supplement the model with more easily understood, often risk-unadjusted, examples showing the effect of the identified risk factor. Blackstone et al. (1) have done an excellent job of this by providing numerous supplemental Kaplan-Meier depictions as well as nomograms of solutions to the multivariable equations. They also provide an intuitively useful diagram of the risk “trajectory” over time.
Another important opportunity to focus the reader on clinically relevant interpretations is to actively decide the range of values of continuous variables displayed along the x-axis of nomogram solutions to the multivariable equations. An example in Blackstone et al. (1) is the display of ranges of creatinine and bilirubin values in the nomogram depictions. The displays of creatinine out to 7 and bilirubin out to 45 (Online Figures 5 and 6 ) are considerably outside the range in which any patient would be maintained on the waitlist. In reality, very few patients would be kept “active” on the waitlist for isolated heart transplantation with a bilirubin level >10 mg/dl or creatinine level >2.5 to 3 mg/dl. Even though the cohort contained delisted patients, the focus was on listed patients, which in reality would likely only include a small proportion of the values presented in these figures (levels of creatinine >3 mg/dl and bilirubin >10 mg/dl would exclude most patients from heart transplantation because of excessive risk for organ failure post-transplant).
Similarly, care should be taken when retaining additional cohorts outside the area of interest. For example, with the authors’ decision to retain delisted patients in the model (1), they risk including numerous patients who were delisted because they were at extremely high risk for dying of noncardiac organ failure post-transplant. These patients are not really relevant to the underlying reason for the analysis, which was the introduction of a concept for dynamic calculation of patient risk on the waitlist that would allow intermittent recalculation of risk to potentially adjust patient priority.
Thus, the authors are to be congratulated on an innovative statistical modeling approach to dynamic risk predictions that would allow us to provide a more real-time assessment of mortality risk (1). The decisions about how best to display the data and analytics will be an evolving process as expert clinicians digest the concepts, explore the details of the statistical methods, and assess their relevance to clinical decisions. Blackstone et al. have taken us on an exciting foray into a dynamic risk model, which will certainly assume its place among the most respected and useful methods for predicting patient outcomes.
↵∗ Editorials published in the Journal of the American College of Cardiology reflect the views of the authors and do not necessarily represent the views of JACC or the American College of Cardiology.
Dr. Kirklin is the Chair of the Data and Safety Monitoring Board for a pediatric clinical trial of a pediatric valved conduit, for which he received a one-time stipend of $2,500.