Author + information
- Giuseppe Biondi-Zoccai, MD, MStat∗ (, )
- Antonino G.M. Marullo, MD, PhD,
- Mariangela Peruzzi, MD, PhD,
- Tullio Palmerini, MD,
- Leonardo Schirone, PhD,
- Silvia Palmerio, PhD,
- Arturo Giordano, MD, PhD and
- Giacomo Frati, MD, MSc
- ↵∗Department of Medical Surgical Sciences and Biotechnologies, “La Sapienza” University of Rome, Department of AngioCardioNeurology, IRCCS Neuromed, Pozzilli, Italy
We have read with interest the thorough comparative analysis and review on propensity score analysis and inverse probability of treatment weighting analysis reported in the Journal by Elze et al. (1). They compared the performance of different approaches to implement propensity score analysis in 4 large cardiovascular studies, concluding that “PS methods are not necessarily superior to conventional covariate adjustment, and care should be taken to select the most suitable method” (1). This conclusion is not necessarily novel (2), and, most importantly, we identified 3 major issues in the paper that should be borne in mind by the careful reader because they imply some caution in interpreting its implications: 1) lack of a gold standard; 2) focus on large studies; and 3) a missed opportunity to explore disease risk scores.
First, the 4 studies used for the comparative analysis were all observational in scope. This means that Elze et al. eventually missed the key opportunity of appraising accuracy on top of precision using a real gold standard. The best approach would have been instead to begin with randomized trials, and use sampling methods (e.g., Monte Carlo simulation) to create hypothetical observational subsamples of such randomized trials (3). Only in using this could have the gold standard been available for comparison, thus yielding real accuracy (distance from the true value) estimates.
Second, it has been established that standard multivariable methods (Table 1) are robust, valid, and precise when large samples with an event per variable ratio >10 are used (4). Using large studies with several events per variable as done by Elze et al. means that the investigators could have missed the opportunity to truly highlight the incremental benefits of propensity score methods to minimize confounding while minimizing overfitting. This could have been easily accomplished by also including small to moderate size studies or trials with few events.
Third, the disease risk score (DRS) is another appealing approach to deal simultaneously with multiple confounders (5). Despite some conceptual drawbacks, it appears that analysis based on DRS may be superior to traditional adjustment methods, especially when “covariates are not highly correlated with exposure.”
In conclusion, we caution against concluding that propensity scores and inverse probability of treatment weighting have limited incremental usefulness in comparison to standard multivariable adjustment methods in observational cardiovascular studies. Further theoretical and applied research is required in the field before putting the final nail in the coffin of propensity score methods.
Please note: Dr. Biondi-Zoccai has consulted for Abbott Vascular. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose. Deepak L. Bhatt, MD, MPH, served as Guest Editor-in-Chief for this paper.
- 2017 American College of Cardiology Foundation
- Elze M.C.,
- Gregson J.,
- Baber U.,
- et al.
- Austin P.C.,
- Schuster T.,
- Platt R.
- Cepeda M.S.,
- Boston R.,
- Farrar J.T.,
- Strom B.L.
- Arbogast P.G.,
- Ray W.A.