Author + information
- Received December 7, 2015
- Revision received February 18, 2016
- Accepted February 23, 2016
- Published online May 10, 2016.
- Jamal S. Rana, MD, PhDa,b,c,
- Grace H. Tabada, MPHb,
- Matthew D. Solomon, MD, PhDa,b,d,
- Joan C. Lo, MDb,c,e,
- Marc G. Jaffe, MDc,f,
- Sue Hee Sung, MPHb,
- Christie M. Ballantyne, MDg and
- Alan S. Go, MDb,c,h,i,∗ ()
- aDivision of Cardiology, Kaiser Permanente Northern California, Oakland, California
- bDivision of Research, Kaiser Permanente Northern California, Oakland, California
- cDepartment of Medicine, University of California, San Francisco, San Francisco, California
- dDepartment of Medicine, Stanford University, Stanford, California
- eDivision of Endocrinology, Kaiser Permanente Oakland Medical Center, Oakland, California
- fDivision of Endocrinology, Kaiser Permanente South San Francisco Medical Center, South San Francisco, California
- gSection of Cardiovascular Research, Department of Medicine, Baylor College of Medicine, and Center for Cardiovascular Disease Prevention, Houston Methodist DeBakey Heart and Vascular Center, Houston, Texas
- hDepartments of Epidemiology and Biostatistics, University of California, San Francisco, San Francisco, California
- iDepartment of Health Research and Policy, Stanford University School of Medicine, Stanford, California
- ↵∗Reprint requests and correspondence:
Dr. Alan S. Go, Division of Research, Kaiser Permanente Northern California, 2000 Broadway, Oakland, California 94612-2304.
Background The accuracy of the 2013 American College of Cardiology/American Heart Association (ACC/AHA) Pooled Cohort Risk Equation for atherosclerotic cardiovascular disease (ASCVD) events in contemporary and ethnically diverse populations is not well understood.
Objectives The goal of this study was to evaluate the accuracy of the 2013 ACC/AHA Pooled Cohort Risk Equation within a large, multiethnic population in clinical care.
Methods The target population for consideration of cholesterol-lowering therapy in a large, integrated health care delivery system population was identified in 2008 and followed up through 2013. The main analyses excluded those with known ASCVD, diabetes mellitus, low-density lipoprotein cholesterol levels <70 or ≥190 mg/dl, prior lipid-lowering therapy use, or incomplete 5-year follow-up. Patient characteristics were obtained from electronic medical records, and ASCVD events were ascertained by using validated algorithms for hospitalization databases and death certificates. We compared predicted versus observed 5-year ASCVD risk, overall and according to sex and race/ethnicity. We additionally examined predicted versus observed risk in patients with diabetes mellitus.
Results Among 307,591 eligible adults without diabetes between 40 and 75 years of age, 22,283 were black, 52,917 were Asian/Pacific Islander, and 18,745 were Hispanic. We observed 2,061 ASCVD events during 1,515,142 person-years. In each 5-year predicted ASCVD risk category, observed 5-year ASCVD risk was substantially lower: 0.20% for predicted risk <2.50%; 0.65% for predicted risk 2.50% to <3.75%; 0.90% for predicted risk 3.75% to <5.00%; and 1.85% for predicted risk ≥5.00% (C statistic: 0.74). Similar ASCVD risk overestimation and poor calibration with moderate discrimination (C statistic: 0.68 to 0.74) were observed in sex, racial/ethnic, and socioeconomic status subgroups, and in sensitivity analyses among patients receiving statins for primary prevention. Calibration among 4,242 eligible adults with diabetes was improved, but discrimination was worse (C statistic: 0.64).
Conclusions In a large, contemporary “real-world” population, the ACC/AHA Pooled Cohort Risk Equation substantially overestimated actual 5-year risk in adults without diabetes, overall and across sociodemographic subgroups.
Publication of the 2013 American College of Cardiology (ACC)/American Heart Association (AHA) Pooled Cohort Risk Equation for estimating atherosclerotic cardiovascular disease (ASCVD) risk is considered an important step forward, as it estimates risk for both heart disease and stroke, and provides estimates applicable to black/African-American subjects (1,2). This equation was developed from several prospective U.S.-based cohorts of enrolled volunteers, primarily conducted in the 1990s, with limited ethnic diversity and age range. Predicted ASCVD risk using the equation was reported to be systematically higher than the observed risk in some highly selected cohorts (3,4) but not in others (5). Importantly, however, these studies included persons screened for participation or enrolled in clinical trials, or in much earlier time periods, with limited generalizability to contemporary and diverse populations that are more representative of eligible patients treated in typical clinical practice (6,7). Validation efforts in more contemporary cohorts have further been limited due to inclusion of analyzed participants treated with statin therapy or having a high likelihood of initiating statins during follow-up (4,6,8). Lack of comprehensive surveillance in some studies is another possible reason for overestimation by the Pooled Cohort Risk Equation due to under-ascertainment of ASCVD events (3,9).
Another major knowledge gap is the lack of accurate ASCVD risk estimation specific to persons of Asian/Pacific Islander and Hispanic ethnicities, who are currently combined with the white population in the Pooled Cohort Risk Equation (10). Furthermore, patterns of risk in more contemporary, community-based populations may be significantly different from the older cohorts used to derive the Pooled Cohort Risk Equation, which does not reflect the recent treatment era and risk factor levels (11,12). Estimates suggest that nearly one-half of U.S. adults and up to 65% of European adults would be eligible for statins on the basis of new ACC/AHA cholesterol guidelines using the Pooled Cohort Risk Equation (13,14). Thus, evaluating the accuracy of ASCVD risk prediction is essential if it is being used as a decision-making tool to determine which persons should receive statin therapy for primary prevention (13,15), and it could have a far-reaching impact if applied at a population level.
To address these knowledge gaps, we examined a very large, contemporary, multiethnic, community-based “real-world” population whose clinical characteristics would typically trigger a discussion of initiation of cholesterol-lowering therapy per risk assessment according to the Pooled Cohort Risk Equation (i.e., patients 40 to 75 years of age, without known ASCVD or diabetes, who have low-density lipoprotein cholesterol [LDL-C] levels between 70 and 189 mg/dl). The study compared predicted versus observed 5-year risks of ASCVD events, overall and within sex and multiple ethnic subgroups, along with measures of calibration and discrimination. The analyses were repeated among a cohort of eligible adults with diabetes mellitus.
Kaiser Permanente Northern California is a large integrated health care delivery system providing comprehensive inpatient and outpatient care to approximately 4 million ethnically diverse persons who are highly representative of the local and statewide population (16).
Using outpatient laboratory databases, we initially identified all members of the Kaiser Permanente Northern California system between January 1, 2008, and December 31, 2008, who were ≥21 years of age and had LDL-C levels between 70 and 189 mg/dl within 5 years before study entry. The earliest LDL-C measurement in the year 2008 was used as the index date or, if the patient’s last available LDL-C measurement was before 2008, the index date used was January 1, 2008. We excluded those who met any of the following criteria: sex or race/ethnicity was unknown; prescribed statins or other lipid-lowering therapies within 5 years before the index date; previous hospitalization for acute myocardial infarction, ischemic stroke, or receipt of coronary artery bypass surgery or percutaneous coronary intervention, identified from health plan hospital discharge or billing claims databases using previously validated methods (17,18); <12 months of continuous membership and pharmacy benefit before the index date (to ensure more complete information on clinical characteristics); and <5 years of complete follow-up, except if due to death. On the basis of the requirements of the ASCVD risk calculator, patients with missing systolic blood pressure, total cholesterol, or high-density lipoprotein cholesterol (HDL-C) information were also excluded. We further excluded patients who received statins during follow-up if used for primary prevention of ASCVD (i.e., statin initiated before a documented ASCVD event) on the basis of filled dispensings found in pharmacy databases. The cohort was then stratified according to diabetes mellitus status at the index date on the basis of data from a validated regional diabetes registry (19,20).
Prediction of ASCVD risk
The 10-year ASCVD predicted risk score was calculated for cohort members between 40 and 75 years of age, as per the guideline-recommended ASCVD prevention age group (1,13). To directly compare observed versus predicted ASCVD risk at 5-year follow-up, we annualized the 10-year predicted ASCVD risk and categorized the population into 4 groups of estimated 5-year risk (<2.50%; from 2.50% to <3.75%; from 3.75% to <5.00%; and ≥5.00%), as was previously used (8).
Ascertainment of ASCVD events
Follow-up occurred through December 31, 2013. ASCVD events were defined as acute myocardial infarction, coronary heart disease (CHD) death, or fatal or nonfatal ischemic stroke. Myocardial infarction and ischemic stroke were identified from comprehensive health plan hospital discharge or billing claims databases on the basis of previously validated algorithms (17,18,21) using International Classification of Diseases-Ninth Revision, primary discharge codes. Coronary death was determined on the basis of International Classification of Diseases-10th Revision, codes (I11, I20–I25) (22) obtained from state death certificate files through 2013, which was the latest available information at the time of analysis.
Comorbidity was characterized during a 5-year period before the index date on the basis of relevant International Classification of Diseases-Ninth Revision, diagnosis and procedure codes (21,23) from health plan electronic medical records and a regional diabetes registry (19,20). Baseline data were collected from electronic medical records on age, sex, self-reported race/ethnicity, cigarette smoking status, and body mass index. Ambulatory measures of baseline total cholesterol, HDL-C, LDL-C, systolic blood pressure, and use of antihypertensive medication within 5 years before the index date were ascertained from comprehensive health plan laboratory, outpatient visit, and pharmacy databases.
Analyses were conducted by using SAS version 9.3 (SAS Institute, Inc., Cary, North Carolina). Baseline characteristics were compared across predicted ASCVD risk categories by using analysis of variance for continuous variables and chi-squared tests for categorical variables. We calculated the 5-year incidence of ASCVD events with 95% confidence limits by using the first hospitalized acute myocardial infarction, first hospitalized ischemic stroke, or coronary death observed during follow-up. The 5-year expected incidence of ASCVD risk was calculated on the basis of the mean ASCVD predicted risk and associated 95% confidence interval (CI) among patients whose predicted risk fell within each risk category. We then examined the observed and expected 5-year incidence of ASCVD events among nondiabetic patients between 40 and 75 years of age, overall and stratified according to sex, race/ethnicity (non-Hispanic white, non-Hispanic black, Asian/Pacific Islander, and Hispanic) and measures of socioeconomic status by using U.S. Census–based residential block–level data (18). The observed and expected 5-year ASCVD incidence rates were also separately examined among eligible patients with diabetes between 40 and 75 years of age.
Accuracy was assessed by evaluating calibration and discrimination. Following the approach used by Cook and Ridker (11), calibration plots were generated comparing observed versus expected risk of ASCVD events within each estimated risk category. Discrimination was assessed by using the C statistic from a logistic regression model evaluating predicted risk categories with observed 5-year ASCVD event risks. Finally, as a sensitivity analysis, the accuracy of the risk equation was examined among eligible adults with and without diabetes who received statin therapy for primary prevention during the follow-up period.
Study cohort and baseline characteristics
A total of 307,591 eligible adults aged 40 to 75 years were identified who did not have diabetes, prior ASCVD, or earlier receipt of lipid-lowering therapy and who had complete 5-year follow-up (Figure 1). In addition, 4,242 patients with diabetes were separately identified who did not have known ASCVD, prior lipid-lowering therapy, or any other exclusion criteria.
Among eligible adults without diabetes, their mean age was 54.8 years, and 61.6% were women; 22,283 were black/African American, 52,917 were Asian/Pacific Islander, and 18,745 were Hispanic. Nearly one-third were former or current smokers, 28% were obese, and 33% were taking medication for hypertension (Table 1).
Fifty-seven percent of patients were in the lowest (<5%) category of 10-year predicted ASCVD risk, with 12% having predicted risk 5.0% to <7.5%, 8% having predicted risk 7.5% to <10%, and the remaining 23% having predicted risk ≥10% (Table 1). Patients with higher predicted ASCVD risk were more likely to be older and male, be former or current smokers, and have higher systolic blood pressure and body mass index, lower HDL-C, and higher LDL-C.
Accuracy of ASCVD risk equation according to diabetes status
Among eligible patients 40 to 75 years of age without diabetes, 2,061 ASCVD events were observed during 1,515,142 person-years, with 1,464 (0.5%) acute myocardial infarctions, 525 (0.2%) CHD-related deaths, and 71 (0.02%) ischemic strokes. The cumulative risk of ASCVD events was linear during the entire 5-year follow-up period (Figure 2). Observed 5-year ASCVD incidence was substantially lower than the predicted risk in each category: 0.20% (95% CI: 0.20 to 0.25) for predicted risk <2.50%; 0.65% (95% CI: 0.55 to 0.70) for predicted risk 2.50% to <3.75%; 0.90% (95% CI: 0.75 to 1.00) for predicted risk 3.75% to <5.00%; and 1.85% (95% CI: 1.75 to 1.95) for predicted risk ≥5.00% (Central Illustration). Calibration between the predicted versus observed 5-year ASCVD incidence was poor (Figure 3) and the discrimination was moderate (C statistic: 0.74).
Among eligible 40- to 75-year-olds with diabetes, 148 events were observed during 19,196 person-years, with 84 (2%) acute myocardial infarctions, 54 (1.3%) CHD-related deaths, and 10 (0.2%) ischemic strokes. Observed 5-year ASCVD incidence in each predicted risk category was as follows: 0.10% (95% CI: 0% to 0.85%) for predicted risk <2.50%; 2.55% (95% CI: 1.35% to 4.95%) for predicted risk 2.50% to <3.75%; 2.65% (95% CI: 1.30% to 5.30%) for predicted risk 3.75% to <5.00%; and 5.50% (95% CI: 4.60% to 6.50%) for predicted risk ≥5.00% (Central Illustration). Calibration was fair between predicted versus observed 5-year risk of ASCVD events, and discrimination was only modest (C statistic: 0.64) (Figure 3).
Accuracy of ASCVD risk equation according to sex, race/ethnicity, and socioeconomic status
Among both nondiabetic men and women, there was systematic overestimation of observed 5-year ASCVD incidence in each predicted risk category, with similarly poor calibration in both sexes (Figures 3 and 4). Discrimination in nondiabetic women (C statistic: 0.72) was better than in men (C statistic: 0.68).
Given that Asian/Pacific Islander and Hispanic subjects are considered under “white or other” in the ASCVD risk equation, we compared the observed versus predicted 5-year ASCVD incidence separately among non-Hispanic white, non-Hispanic black, Asian/Pacific Islander, and Hispanic subjects. We found systematic overestimation of actual ASCVD risk in each of these ethnic groups (Figure 4) and similarly poor calibration, but discrimination varied in each group (C statistic: 0.70 to 0.74) (Figure 3). Results were also similar across measures of socioeconomic status.
Assessment of ASCVD risk equation in persons treated with statins for primary prevention
In sensitivity analyses, eligible adults who did or did not have diabetes but received statin therapy for primary prevention during follow-up were separately examined. In eligible adults without diabetes, there was overestimation of actual 5-year ASCVD incidence in each predicted risk category (Online Figure A). Observed rates for this population were approximately 6 times lower than predicted. In contrast to findings in untreated eligible patients with diabetes, there was a systematic overestimation of the observed 5-year ASCVD incidence in each predicted risk category among eligible adults with diabetes who received statin therapy for primary prevention. Calibration was poor and discrimination was modest in both those with (C statistic: 0.61) and without (C statistic: 0.68) diabetes who received statin therapy for primary prevention (Online Figure B).
In a large, contemporary, “real-world” cohort with broad racial/ethnic diversity, we showed that the ACC/AHA Pooled Cohort Risk Equation substantially overestimated actual 5-year ASCVD risk in each predicted risk category within eligible adults without diabetes (40 to 75 years of age with no known ASCVD and with LDL-C levels 70 to 189 mg/dl) who are a recommended target population for consideration of cholesterol-lowering therapy. Importantly, this overestimation was similar in both men and women, as well as in 4 major ethnic groups, with poor calibration in each subgroup. Discrimination, as measured according to the C statistic, was fair to moderate, depending on the patient group. Of interest, we found better calibration among eligible adults age 40 to 75 years with diabetes but worse discrimination.
Since release of the 2013 ACC/AHA cholesterol treatment guidelines, there has been an ongoing debate, especially regarding application of the new Pooled Cohort Risk Equation for assessing ASCVD risk within and outside the United States. In selected U.S.-based prospective cohorts not used to derive the Pooled Cohort Risk Equation, such as the MESA (Multi-Ethnic Study of Atherosclerosis) and the REGARDS (Reasons for Geographic and Racial Differences in Stroke) studies, the estimated ASCVD risk was higher than observed risk (3,4). Similarly, in a European population ≥55 years of age, the risk equation had poor calibration (15). It has been proposed that results of these and related evaluation efforts are limited due to approximately 25% of participants taking statins or having a high likelihood of initiating statin therapy during follow-up (6,8). Under-ascertainment of ASCVD events in these studies has also been suggested as a possible reason for observed overestimation of risk (3,9). There is also concern as to how well the risk equation works in more contemporary patients compared with the older cohorts used to derive the equation (3,11,12). Analyses from the Women’s Health Study estimated hypothetical rates if no women were prescribed statins or underwent revascularization and suggested that after accounting for intervention effects of statins and revascularization, as well as hypothetical confounding by indication or under-ascertainment of events, discrepancy would still occur in the observed versus predicted ASCVD risk (11). However, because the Women’s Health Study enrolled primarily white female nurses between 1992 and 1995, the results are not necessarily generalizable to more diverse, contemporary populations. Similarly, recent analyses from the MESA study share the important limitation of lack of generalizability, as research participants are not likely to accurately represent the general population in clinical practice and during a recent treatment era (4).
Our study addresses many of these challenges. First, we evaluated the accuracy of the Pooled Cohort Risk Equation in the most contemporary cohort studied to date, with study entry in 2008 and complete follow-up through 2013. Second, we excluded patients who had received any statin therapy during the 5 years before or anytime during follow-up, thus minimizing any confounding from statin use when evaluating performance of the risk calculator. Third, because we studied patients receiving care within an integrated health care delivery system, we had complete ascertainment of ASCVD events. Fourth, even after restricting the analysis to the target population of nondiabetic adults, 40 to 75 years of age, who had LDL-C levels between 70 and 189 mg/dl and no known ASCVD, we studied a very large population of 307,591 men and women. This population is notably larger and more representative than any previous studies (e.g., 3,433 adults ≥55 years of age, including patients with diabetes, in the Rotterdam Study  and 10,997 from the REGARDS study with a selected subset of 3,333 participants ≥65 years of age linked to Medicare claims data ). Fifth, we examined 4 major racial/ethnic groups, including Asian/Pacific Islander and Hispanic subjects, which constitute a major proportion of the U.S. population (24). Present guidelines (1) caution against applying this algorithm to groups that are neither white nor black/African American, and we further demonstrated that the Pooled Cohort Risk Equation overestimated actual ASCVD risk across all 4 ethnic groups in our study population with variable discrimination. Finally, in contrast to previous cholesterol treatment guidelines (25), the Pooled Cohort Risk Equation includes diabetes as part of the scoring criteria, rather than considering it a CHD-equivalent, and a recent study found important differences in predicted risk among those with diabetes (26). We found that calibration was improved, but discrimination was poorer, for untreated eligible adults with diabetes; importantly, there was a wide range of ASCVD risk in these patients, which could inform future studies evaluating different thresholds for initiating statin therapy in patients with diabetes.
First, we used 5-year versus 10-year follow-up, and it is possible that incidence rates in the first 5 years of follow-up could be less than in the subsequent 5 years. However, we observed a cumulative risk of ASCVD that was linear during the 5-year follow-up (Figure 2), which directly supports extrapolation of our results to 10-year risk. Studying 5-year risk also allowed for a contemporary estimate of ASCVD risk. Risk score overestimation may have been due to exclusion of those who were started on statins; however, in a sensitivity analysis, there was poor calibration and suboptimal discrimination of the Pooled Cohort Risk Equation in eligible patients, with or without diabetes, treated with statins for primary prevention during follow-up (Online Figures A and B). Inclusion of death certificate data to help define fatal CHD may have led to some misclassification. Our study population may not be fully generalizable to all parts of the United States or to patients treated in different health care delivery systems; however, approximately one-third of the sample was obese, had known hypertension, and was a current or former smoker. In addition, unlike previous studies, we had a very ethnically diverse sample of patients with uniform access to care, as well as comprehensive ascertainment of clinically recognized ASCVD events, which argues for greater generalizability and validity. Given that most adults with diabetes within our health system were receiving statins for primary prevention, and were therefore not eligible for analysis, the sample we examined is not necessarily fully representative of all patients with diabetes. However, our study sample allowed comprehensive assessment of the observed 5-year ASCVD risk among adults with clinically recognized diabetes across the range of predicted risk, and we required all patients to have the full 5-year follow-up, which is critical to allow enough time to identify all fatal and nonfatal ASCVD events. We are unaware of any evidence that patients with symptoms or signs of possible ASCVD are more or less likely to remain insured and stay or leave our health system. There is a possibility that including those who died, but not those censored from disenrollment, could artificially raise the observed event rates, but given that the absolute risk of death was very low in this population, it is highly unlikely to have had a substantive impact on the results.
Insured patients may have better health habits than patients who are not able to afford insurance or choose not to obtain it. However, if one considers the rationale for risk assessment in the ACC/AHA guideline, this real-world population is more likely to represent typical patients in U.S. communities who are seeing a health care provider and who may be considered for preventive therapies, compared with volunteers enrolled into prospective cohort studies or interventional clinical trials. Furthermore, if significant variation exists in regional population characteristics or outcomes, and if there are systematic differences in cardiovascular risk reduction efforts across health care settings, it is critical that health care providers use a risk assessment tool that is calibrated to the patient population being seen by those providers.
As recently recommended (27), we conducted a critical evaluation of the ASCVD Pooled Cohort Risk Equation from a relevant large, diverse, real-world target population, overall and in key understudied subgroups. We found that this risk equation substantially overestimated actual ASCVD risk in adults not treated with statin therapy for primary prevention without diabetes (overall and across all sociodemographic subgroups), and demonstrated suboptimal accuracy in those with or without diabetes. Our study provides evidence to support recalibration of the ASCVD Pooled Cohort Risk Equation in adults without diabetes, especially given the individual and public health implications of widespread application of this risk calculator. Ongoing research and dialogue in this area remains crucial and should be encouraged to provide more rigorous, valid evidence in contemporary, diverse populations (26–28).
COMPETENCY IN SYSTEMS-BASED PRACTICE: The 2013 ACC/AHA Pooled Cohort Risk Equation estimates risk for both coronary events and stroke. However, the risk equation substantially overestimates ASCVD risk in adults without diabetes mellitus and has suboptimal accuracy across sociodemographic subgroups as well as in adults with diabetes who do not receive statin therapy for primary prevention.
TRANSLATIONAL OUTLOOK: More research is needed to develop risk assessment tools that are calibrated to diverse, contemporary populations, especially given the implications of recommending lifelong statin therapy.
The authors thank Michael Pencina, PhD, and Benjamin Neely from the Duke University Department of Biostatistics and Bioinformatics for assistance with ASCVD risk calculation. They also thank Alda I. Inveiss for her expert technical assistance.
For supplemental figures, please see the online version of this article.
This project was supported in part by funding from the National Heart, Lung, and Blood Institute (U19 HL091179) and the Kaiser Permanente Northern California Community Benefit Fund. The funding sources had no role in study design, data collection, data analysis, data interpretation, or writing of the report. Drs. Go and Lo have received a research grant through their institution from Sanofi. Dr. Go has also received a research grant through his institution from AstraZeneca. Dr. Solomon has received research grants through his institution from Abbott Vascular, Inc., and Genentech. Dr. Lo has received a research grant through her institution from Amgen. Dr. Ballantyne has received research support through his institution from Abbott Diagnostic, Amarin, Amgen, Bristol-Myers Squibb, Eli Lilly, Esperion, Merck, Novartis, Pfizer, Regeneron, Roche Diagnostic, Sanofi-Synthélabo, Takeda Development Centers of America, the National Institutes of Health, the American Heart Association, and the American Diabetes Association; and has been a consultant for Abbott Diagnostics, Amarin, Amgen, AstraZeneca, Eli Lilly, Esperion, Genzyme, Matinas BioPharma, Inc., Merck, Novartis, Pfizer, Regeneron, and Sanofi-Synthélabo. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.
- Abbreviations and Acronyms
- American College of Cardiology
- American Heart Association
- atherosclerotic cardiovascular disease
- coronary heart disease
- high-density lipoprotein cholesterol
- low-density lipoprotein cholesterol
- Received December 7, 2015.
- Revision received February 18, 2016.
- Accepted February 23, 2016.
- American College of Cardiology Foundation
- Goff D.C. Jr..,
- Lloyd-Jones D.M.,
- Bennett G.,
- et al.
- Stone N.J.,
- Robinson J.G.,
- Lichtenstein A.H.,
- et al.
- Andersson C.,
- Enserro D.,
- Larson M.G.,
- et al.
- Lloyd-Jones D.M.,
- Goff D.,
- Stone N.J.
- Hlatky M.A.,
- Ray R.M.,
- Burwen D.R.,
- et al.
- Smith S.C. Jr..,
- Grundy S.M.
- Ray K.K.,
- Kastelein J.J.,
- Boekholdt S.M.,
- et al.
- ↵Gordon NP. Characteristics of adult health plan members in the Northern California region membership, as estimated from the 2011 member health survey. Division of Research, Kaiser Permanente Medical Care Program: Oakland, CA. 2013. Available at: https://www.dor.kaiser.org/external/mhs11reg/. Accessed March 1, 2016.
- Selby J.V.,
- Ray G.T.,
- Zhang D.,
- et al.
- Karter A.J.,
- Schillinger D.,
- Adams A.S.,
- et al.
- ↵Miniño AM, Klein RJ. Health mortality from major cardiovascular diseases: United States, 2007. Health E-Stats. National Center for Health Statistics, 2010. Available at: http://www.cdc.gov/nchs/data/hestat/cardio2007/cardio2007.pdf. Accessed March 1, 2016.
- Smith D.H.,
- Thorp M.L.,
- Gurwitz J.H.,
- et al.
- ↵U.S. Census Bureau. 2010 Census Redistricting Data (Public Law 94-171) Summary File. 2011. Available at: https://www.census.gov/prod/cen2010/doc/pl94-171.pdf. Accessed March 24, 2016.
- Expert Panel on Detection,
- Evaluation, and Treatment of High Blood Cholesterol in Adults
- Karmali K.N.,
- Goff D.C. Jr..,
- Ning H.,
- et al.
- Amin N.P.,
- Martin S.S.,
- Blaha M.J.,
- et al.