Author + information
- Thomas J. Wang, MD* ()
- ↵*Reprint requests and correspondence:
Dr. Thomas J. Wang, Massachusetts General Hospital, Cardiology Division, GRB-800, 55 Fruit Street, Boston, Massachusetts 02114
Few topics in cardiology have generated as much recent debate as the use of biomarkers in primary prevention. It is widely acknowledged that conventional Framingham risk factors, such as hyperlipidemia, hypertension, and smoking, do not identify all patients who will develop cardiovascular events. What is uncertain is whether newer tests involving circulating biomarkers add substantially to our ability to identify such patients.
Although biomarkers such as high-sensitivity C-reactive protein (CRP) have shown promising associations with cardiovascular risk, many studies have shown minimal improvements in the performance of risk algorithms that incorporate CRP or other biomarkers (1,2). This has stimulated interest in combining multiple biomarkers into larger panels (a “multimarker” approach), under the assumption that a group of biomarkers may outperform individual ones. In recent years, this multimarker strategy has been evaluated in a number of epidemiologic cohort studies (2–6). The study by Kim et al. (7) in this issue of the Journalrepresents another test of the multimarker approach, this time using data from a large sample of women enrolled in the WHI (Women's Health Initiative) trial. The investigators evaluated the predictive performance of 18 inflammatory and hemostatic biomarkers in a nested case-control study involving 321 patients with coronary heart disease and 743 controls. Five biomarkers were significantly associated with coronary heart disease after adjustment for conventional risk factors: interleukin-6, d-dimer, factor VII, von Willebrand factor, and homocysteine. CRP achieved only borderline significance (p = 0.08). Incorporation of the 5 biomarkers yielded modest improvements in risk discrimination, as measured by the area under the receiver-operating characteristic curve, or c-statistic (0.73 vs. 0.71 in the conventional risk factor model, in those free of cardiovascular disease at baseline). There was also moderate improvement in risk classification (net reclassification improvement [NRI]: 6.45%).
Notable strengths of the WHI report include the large number of cases, the large panel of biomarkers, and the carefully performed statistical analyses. Several limitations should be noted, however. First, the case-control design does not allow an accurate assessment of model calibration (e.g., a comparison of predicted and observed risks). This is because the distribution of predicted risks in the control group is probably skewed because of the matching process. Second, although the investigators provide estimates of NRI, the validity of such estimates from a case-control sample is uncertain, again because of the potentially skewed risk distribution in the control sample. Third, several important biomarkers, such as B-type natriuretic peptide (BNP) and urinary albumin excretion, were not included in the multimarker set. Last, it would have been preferable to exclude patients with prevalent cardiovascular disease from all of the analyses. Conventional risk algorithms are designed to be applied to those without prior cardiovascular disease, in whom decisions regarding preventive therapies need to be made. Fortunately, the WHI investigators do report separate results from this important subgroup.
Despite these limitations, the study by Kim et al. (7) is valuable and timely. The basic findings are reassuringly consistent with those of prior studies. Indeed, now that a number of multimarker studies from large populations have been published, it is useful to take stock of the answers to several core questions in risk prediction.
What Are the Appropriate Statistical Metrics to Use in Evaluating New Risk Markers?
Studies continue to reinforce the concept that statistically significant associations between biomarkers and outcomes do not ensure that the biomarkers will be useful for individual risk prediction. An important reason is that distributions of biomarker values between those who develop and do not develop events overlap substantially, even when the group means are significantly different (8). This point is highlighted nicely in the figures in the WHI substudy. One consequence is a relatively high proportion of false positives and false negatives around any given cut point, which reduces the c-statistic. Because the c-statistic has been criticized as being too stringent a criterion for predictive models, other metrics have been suggested. The newest statistical measures are based on how well a new marker “reclassifies” patients from one risk category to another (9,10). While clinically relevant, reclassification relies on the existence of well-accepted risk thresholds that are linked to management decisions.
A general consensus has emerged that studies of new markers should report a variety of statistical metrics, including both older ones (such as the c-statistic) and newer ones (such as proportion reclassified and NRI) (11). This approach appears well justified. That said, in most multimarker studies, results for the various statistical measures are more similar than they are distinct, particularly after differences in scale are taken into account. For instance, in the Framingham Heart Study (2,12) and the Malmo Diet and Cancer Study (4), biomarker panels led to minimal improvements across the board in c-statistic, calibration, and NRI. In the present study from the WHI, modest changes in both the c-statistic and NRI were observed.
What Are the Best Biomarkers for Predicting Cardiovascular Risk?
Multimarker studies provide an opportunity for head-to-head comparisons of biomarker performance. Surprisingly, the best-known cardiovascular biomarker, CRP, often fares worse than other biomarkers in these comparisons. In several large multimarker studies, including the present one (7), CRP was not a significant predictor of cardiovascular events (2,5) or coronary events (4) after adjustment for other biomarkers, such as BNP. Why might BNP, a marker of hemodynamic stress, be a better predictor of vascularevents than CRP? Several possibilities have been raised, including the presence of elevated ventricular wall stress secondary to ischemia and/or direct effects of ischemia on BNP release. Intuitively, the fact that BNP is secreted almost exclusively from the heart (providing a “window” on the myocardium) (13), could explain its higher specificity compared with markers such as CRP that come from the liver or other tissues.
Why Do Biomarkers Perform Well in Some Studies and Not in Others?
Although most multimarker studies suggest that biomarkers add relatively little on top of conventional risk factors, there are notable exceptions. In the Uppsala Longitudinal Study of Adult Men, a panel of 4 biomarkers (BNP, cystatin C, troponin, and CRP) raised the c-statistic from 0.688 to 0.748 in those without prior cardiovascular disease, with a correspondingly large NRI (26%) (3). In 2 secondary prevention studies, a single biomarker, N-terminal pro-BNP, led to sizable increases in the c-statistic (14,15). In contrast, most primary prevention studies find only modest increases in the c-statistic (about 0.01).
These divergent findings are largely attributable to differences in study populations. The Uppsala cohort was composed entirely of elderly men, at the same age, with a relatively high baseline risk. Conventional risk factors (including age) predict relatively poorly in such a setting, as evidenced by the baseline c-statistics below 0.70 in the Uppsala study. Conventional risk factors also perform poorly in secondary prevention populations, as features of the existing disease are more strongly related to the risk for recurrent events than antecedent risk factors. The worse the prediction with conventional risk factors, the greater the scope to improve risk prediction metrics with newer markers (e.g., it is easier to increase the c-statistic if the starting point is 0.68 rather than 0.75). Thus, biomarkers might be expected to perform best in higher risk populations, and the epidemiologic evidence largely bears this out. This creates a dilemma, however, as the need for improved risk stratification is greatest in low-risk to intermediate-risk patients, who as a group are the least likely to be receiving preventive therapies.
How Many Biomarkers Does It Take?
If 10 or more biomarkers can fail to improve the accuracy of risk models (2), how many biomarkers would be considered sufficient? This question is particularly relevant because contemporary technologies could identify scores of new genetic or circulating biomarkers in coming years (16). Of course, the answer depends on which biomarkers are being added, not how many. As shown in the WHI study and others, significance in single-marker studies does not ensure that a biomarker will enter multimarker risk models. This is because the new biomarkers may provide overlapping information. For instance, once a marker such as interleukin-6 or CRP is in a risk model, additional inflammatory markers would probably not provide new predictive information. A formalization of this concept is described by Pepe and Thompson (17), who performed simulations with 2 hypothetical cancer biomarkers. If the c-statistic is 0.80 with 1 biomarker, the addition of a second predictive biomarker raises the c-statistic to 0.88 if the 2 biomarkers are weakly correlated but only to 0.83 if they are moderately correlated.
Accordingly, a small number of uncorrelated biomarkers will improve risk prediction more than a large number of correlated biomarkers. In the Uppsala cohort, the panel that significantly improved discrimination and calibration consisted of 4 biomarkers that each represented different pathways (BNP [wall stress], troponin [injury], cystatin C [renal function], and CRP [inflammation]). In general, biomarkers from the same pathway are much more likely to be correlated with each other than those coming from different pathways. The challenge of adding biomarkers from different pathways, however, is that one can quickly run out of pathways known to contribute cardiovascular risk. The most promising method of finding uncorrelated biomarkers, in newpathways, may lie in the use of technologies such as proteomics and metabolomics, which are capable of systemically screening large numbers of molecules in specimens in an “unbiased” (or less biased) manner.
What Is the Relevance of Biomarker Studies in the Post-JUPITER Era?
The JUPITER (Justification for the Use of Statins in Prevention: An Intervention Trial Evaluating Rosuvastatin) trial evaluated the effect of rosuvastatin in middle-aged to elderly patients with CRP >2 mg/dl and baseline low-density lipoprotein <130 mg/dl (18). The risk for cardiovascular events was reduced by 44% by treatment with rosuvastatin compared with placebo. If treatment of patients identified on the basis of elevated CRP levels can be so effective, is it relevant that CRP and other biomarkers have only modest predictive utility in observational studies?
It is important to recognize that the JUPITER trial did not address the hypothesis that CRP screening was necessary to identify the subjects who benefited from statin therapy. There was no “control group” for the biomarker comparison composed of subjects with CRP levels below 2 mg/dl. Thus, it is possible that subjects with low CRP levels (<2 mg/dl), but otherwise similar risk factor profiles, would have derived comparable benefit in terms of relative risk reduction with rosuvastatin. Indeed, a secondary analysis examining the interaction between baseline CRP levels and treatment outcomes among enrolled participants yielded a surprising finding. Patients with high CRP levels (above the median of 4.2 mg/dl) actually had lessbenefit from rosuvastatin than those with lower CRP levels (below the median) (p value for interaction = 0.015) (19). Furthermore, there was no evidence of benefit in those who met the age and CRP criteria for entry into the trial but had no other conventional cardiovascular risk factors (19).
Thus, what can one take away from the JUPITER trial with respect to the implementation of screening on the basis of biomarkers? One of the important findings of the JUPITER trial is that the relative risk reduction with rosuvastatin was remarkably consistent across clinical subgroups, including those at low to intermediate risk (18). This finding is consistent with the results of statin trials in higher risk populations (20) and underscores the importance of absolute risk estimation, because the projected risk reduction is driven largely by the baseline absolute risk. Those at very low baseline risk may not derive enough absolute benefit (even if the relative risk reduction is significant) to offset the costs and risks of therapy.
Thus, biomarkers (not limited to CRP) that lead to the refinement of risk estimates might aid treatment decisions. This still does not imply that routine biomarker screening is warranted, because existing biomarkers promote relatively small movements in predicted absolute risk (4,10). Risk classification is unlikely to change for most patients. In contrast, some patients have a predicted risk near a threshold for deciding therapy, and in these patients, the additional information from a biomarker or multimarker panel may be helpful. In practice, the treatment thresholds themselves may be lowered after guideline committees incorporate data from trials such as JUPITER. Additional observational studies, perhaps using pooled data, would be useful for defining the performance of biomarker panels in specific ranges of predicted risk.
Studies such as the one by Kim et al. (7) serve as an ongoing reminder of the importance of conventional risk factors in cardiovascular risk assessment. It remains possible that some combination of existing biomarkers could serve as an adjunct to conventional risk assessment, at least in a subset of the population. Nonetheless, large improvements will likely require the identification of new and more specific biomarkers of cardiovascular risk.
Dr. Wang is named as a co-inventor on patent applications for the use of metabolite predictors for diabetes and the use of pro-adrenomedullin in risk stratification. He has been a coauthor in prior studies that received support for the measurement of biomarkers from Siemens Healthcare Diagnostics and BRAHMS. Dr. Wang participates on the scientific advisory board of DiaSorin, Inc. and is supported by grants R01-HL-083197, R01-HL-086875, R01-DK-81572, and R01-HL-098283from the National Institutes of Healthand a grant from the American Heart Association.
↵* Editorials published in the Journal of the American College of Cardiologyreflect the views of the authors and do not necessarily represent the views of JACCor the American College of Cardiology.
- American College of Cardiology Foundation
- Kim H.C.,
- Greenland P.,
- Rossouw J.E.,
- et al.
- Hlatky M.A.,
- Greenland P.,
- Arnett D.K.,
- et al.,
- American Heart Association Expert Panel on Subclinical Atherosclerotic Diseases and Emerging Risk Factors and the Stroke Council
- Wang T.J.
- Blankenberg S.,
- McQueen M.J.,
- Smieja M.,
- et al.
- Pepe M.S.,
- Thompson M.L.
- What Are the Appropriate Statistical Metrics to Use in Evaluating New Risk Markers?
- What Are the Best Biomarkers for Predicting Cardiovascular Risk?
- Why Do Biomarkers Perform Well in Some Studies and Not in Others?
- How Many Biomarkers Does It Take?
- What Is the Relevance of Biomarker Studies in the Post-JUPITER Era?