Author + information
- Markus C. Elze, PhD,
- John Gregson, PhD,
- Usman Baber, MD, PhD,
- Elizabeth Williamson, PhD,
- Samantha Sartori, PhD,
- Roxana Mehran, MD, PhD,
- Gregg W. Stone, MD, PhD and
- Stuart J. Pocock, PhD∗ ()
- ↵∗London School of Hygiene and Tropical Medicine, Department of Medical Statistics, Keppel Street, London WC1E 7HT, United Kingdom
We thank Dr. Biondi-Zoccai and colleagues for their constructive comments on our paper, and agree with their view that using propensity score methods should not be precluded on the basis of the results from our study. Although we agree that many of our findings have been previously reported, our aim was to demonstrate these findings in a nontechnical manner accessible to clinicians, epidemiologists, and applied statisticians. In their letter, Dr. Biondi-Zoccai and colleagues focus on 3 limitations of our study.
First, with the exception of the THIN (The Health Improvement Network) study in which results from large clinical trials could be compared with our observational estimates, the true treatment effects in our observational studies were inevitably unknown. Although we acknowledge this limitation, we intentionally chose to avoid using simulated data to compare the relative merits of propensity scores and conventional covariate adjustment under selected hypothetical scenarios. We believed it was more useful to focus on real-world observational studies to demonstrate how the methods perform in practice in commonly encountered scenarios in which complex confounding structures and unmeasured confounders are present.
Second, we focused on larger studies in which there might be less benefit to using propensity score methods compared with conventional covariate adjustment, a limitation that was acknowledged in our paper. However, it should be noted that in the ADAPT-DES (Assessment of Dual AntiPlatelet Therapy with Drug-Eluting Stents) study, we adjusted for 39 covariates despite only observing 56 events and still estimated a sensible hazard ratio and SE for the treatment effect. This demonstrated that conventional covariate adjustment might provide valid conclusions even when there were few events per covariate, and the standard “rule of 10” might be overly conservative (1).
Third, we did not compare disease risk scores to propensity scores or conventional covariate adjustment as an additional approach to dealing with confounders. Although we agree that this would have been interesting, we deliberately restricted our focus to conventional covariate adjustment and propensity score methods because these are the methods most widely used in practice. In addition, disease risk scores tend to perform best relative to other methods when the outcome is common, but the exposure is rare. Because the 4 studies used in our paper all had common exposures, we are skeptical that disease risk scores would have outperformed either propensity score methods or conventional covariate adjustment in these studies.
Finally, delivering the “last nail in the coffin for propensity scores” was absolutely not our intent, and we agree that propensity score methodology could have benefits in selected situations. However, we have at least challenged their fashionable and indiscriminate use, suggesting that old-fashioned covariate adjustment will often do a perfectly good job, and may even be preferred in many scenarios.
Please note: The authors have reported that they have no relationships relevant to the contents of this paper to disclose. Deepak L. Bhatt, MD, MPH, served as Guest Editor-in-Chief for this paper.
- 2017 American College of Cardiology Foundation