Author + information
- Joseph A. Ladapo, MD, PhD∗ ( and )
- Keith S. Goldfeld, DrPH
- ↵∗New York University School of Medicine, Department of Population Health, 550 First Avenue, VZ30 6th Floor, 614, New York, New York 10016
In a recent study, Ford (1) presents an important analysis, with implications for public health prioritization. However, we believe some of the findings should be interpreted with caution. The Framingham Heart Study has contributed immeasurably to our understanding of cardiovascular disease in the United States and internationally, but the published regression equations for 10-year risk of coronary heart disease (CHD) and cardiovascular disease (CVD) were developed for clinical use, and variance-covariance matrices were not reported (2,3). Thus, it is impossible to quantify uncertainty or estimate confidence intervals for any patient's 10-year risk of CHD or CVD. In other words, while the mean of the risk is known, its variance is unknown.
For this reason, the standard errors for the population-level 10-year risk of CHD and CVD that Ford (1) presents in Table 1 in his article are misleading. The same method is used to estimate these standard errors as used for measures such as age, blood pressure, and cholesterol level. The difference between them is that, unlike Framingham risk scores, these characteristics can be measured with certainty (or are assumed to be measured with negligible error and thus are treated as “certain”); thus, their standard errors are appropriate and accurate. On the other hand, the standard errors reported for population-level 10-year risk of CHD and CVD are inappropriate because they capture only between-person variability in predicted risk but do not account for the fact that each person's risk was estimated using a statistical model (within-person variability). Said differently, Ford (1) treats each person's risk as if it were observed without substantial error, which is true for age, blood pressure, and cholesterol but not the case for Framingham risk functions. Thus, the standard errors Ford reports for 10-year risks of CHD and CVD are systematically underestimated. Ford (1) does not discuss this. Moreover, the method used for evaluating trends does not seem to incorporate uncertainty in risk estimates.
What are the implications of this statistical issue for how clinicians, researchers, and policymakers should interpret Ford's study (1)? The implications may be negligible for readers interested strictly in average population risk and uninterested in trends. However, if the reader is interested in trends, Ford's results (1) are more difficult to interpret, especially in African Americans, Mexican Americans, women, and individuals whose age falls between 30 and 39 years old or 40 and 49 years old. Each of these groups have p values at the borderline of significance, at the 5% or 10% level in some of Ford's (1) analyses. Would incorporation of the uncertainty in 10-year risks of CHD and CVD have affected whether these or other comparisons demonstrated a trend? With the information we have, we cannot tell.
Methods such as bootstrap analysis (4) and the Taylor series-based delta method (5) have been used to capture uncertainty and approximate variance when a closed form estimate is intractable, particularly in decision analysis and cost-effectiveness analysis (6). With adequate information, these methods could be applied to analyses like Ford's (1). In their absence, we believe it is important for readers to recognize that analytics incorporating Framingham CHD or CVD risk scores do not reflect the within-person uncertainty in risk.
- American College of Cardiology Foundation
- Ford E.S.
- D'Agostino R.B. Sr.,
- Vasan R.S.,
- Pencina M.J.,
- et al.
- Wilson P.W.,
- D'Agostino R.B.,
- Levy D.,
- Belanger A.M.,
- Silbershatz H.,
- Kannel W.B.