Author + information
- Robert W. Yeh, MD, MSc∗ ( and )
- Neil J. Wimmer, MD, MSc
- ↵∗Cardiology Division, Massachusetts General Hospital, 55 Fruit Street, GRB800, Boston, Massachusetts 02114
We appreciate the interest and perspective of Dr. Valencia-Serrano and colleagues on our exploration of the use of falsification endpoints as a method of exploring the potential for residual confounding in observational comparative effectiveness studies (1). We believe that such endpoints, akin to negative controls in experimental methods, are an underutilized tool in the cardiovascular data. We have used the examination of nonaccess site bleeding in comparing transradial and transfemoral access in percutaneous coronary intervention (PCI) as a simple illustrative example of how such a method might be used in practice.
The utility of any falsification endpoint rests on the assumption that the treatments being compared should not differ with respect to their influence on these endpoints. In this case, the assumption is that the route of access for PCI, whether transradial or transfemoral, should not influence the rate of nonaccess site bleeding in a causal manner. Dr. Valencia-Serrano and colleagues propose that the differences observed in the falsification endpoint in our analysis may, in fact, be due to a causal relationship between the site of arterial access and nonaccess site bleeding, mediated through the “hemorrhagic transformation of clinically unapparent embolic infarctions” related to catheter contact with the aorta during transfemoral procedures. Judging the plausibility of potential biological explanations such as these, versus the perhaps more banal explanation that the study suffers from residual confounding, will most often require both clinical subject matter expertise and an understanding of the limitations of the methods used for statistical adjustment. In this particular example, however, we benefit from the existence of a large randomized trial of transradial and transfemoral PCI in the ST-segment elevation myocardial infarction population showing no difference in nonaccess site bleeding (2), which serves as validation of this falsification endpoint. More often, however, such a randomized study will not exist to validate other potentially useful endpoints. In the absence of such a study, we find the suggestion that investigators “positively demonstrate from properly designed studies that such an observed effect [on the falsification endpoint] is unlikely to be true” leads to an inescapable tautology: a falsification endpoint cannot be validated by the very study it seeks to validate.
Finally, falsification hypotheses and endpoints need not be limited to studies examining rare events. Although the use of multiple such endpoints may provide stronger evidence for or against the likelihood of residual confounding, we do not believe that it is simply a case of “more being better.” As these methods are increasingly utilized by the research community, we will undoubtedly learn more about their merits and limitations. We are hopeful that our brief investigation will promote further examination and innovation in observational research methods through the use of these and other techniques.
Note: Dr. Wimmer is supported by the National Institutes of Health (grant T32-HL00760). Dr. Yeh receives research support from the Harvard Clinical Research Institute.
- American College of Cardiology Foundation
- Wimmer N.J.,
- Resnic F.S.,
- Mauri L.,
- Matheny M.E.,
- Yeh R.W.
- Romagnoli E.,
- Biondi-Zoccai G.,
- Sciahbasi A.,
- et al.