Author + information
- Received December 16, 2009
- Revision received March 4, 2010
- Accepted March 23, 2010
- Published online June 29, 2010.
- Paul S. Chan, MD, MS⁎,†,⁎ (, )
- William J. Oetgen, MD, MBA‡,
- Donna Buchanan, PhD⁎,
- Kristi Mitchell, MPH§,
- Fran F. Fiocchi, MPH§,
- Fengming Tang, MS⁎,
- Philip G. Jones, MS⁎,
- Tracie Breeding, RN, BSN⁎,
- Duane Thrutchley, RN⁎,
- John S. Rumsfeld, MD, PhD∥ and
- John A. Spertus, MD, MPH⁎,†
- ↵⁎Reprint requests and correspondence:
Dr. Paul S. Chan, Mid America Heart Institute, 5th Floor, 4401 Wornall Road, Kansas City, Missouri 64111
Objectives We examined compliance with performance measures for 14,464 patients enrolled from July 2008 through June 2009 into the American College of Cardiology's PINNACLE (Practice Innovation And Clinical Excellence) program to provide initial insights into the quality of outpatient cardiac care.
Background Little is known about the quality of care of outpatients with coronary artery disease (CAD), heart failure, and atrial fibrillation, and whether sex and racial disparities exist in the treatment of outpatients.
Methods The PINNACLE program is the first, national, prospective office-based quality improvement program of cardiac patients designed, in part, to capture, report, and improve outpatient performance measure compliance. We examined the proportion of patients whose care was compliant with established American College of Cardiology, American Heart Association, and American Medical Association-Physician Consortium for Performance Improvement (ACC/AHA/PCPI) performance measures for CAD, heart failure, and atrial fibrillation.
Results There were 14,464 unique patients enrolled from 27 U.S. practices, accounting for 18,021 clinical visits. Of these, 8,132 (56.4%) had CAD, 5,012 (34.7%) had heart failure, and 2,786 (19.3%) had nonvalvular atrial fibrillation. Data from the PINNACLE program were feasibly collected for 24 of 25 ACC/AHA/PCPI performance measures. Compliance with performance measures ranged from being very low (e.g., 13.3% of CAD patients screened for diabetes mellitus) to very high (e.g., 96.7% of heart failure patients with blood pressure assessments), with moderate (70% to 90%) compliance observed for most performance measures. For 3 performance measures, there were small differences in compliance rates by race or sex.
Conclusions For more than 14,000 patients enrolled from 27 practices in the outpatient PINNACLE program, we found that compliance with performance measures was variable, even after accounting for exclusion criteria, suggesting an important opportunity to improve the quality of outpatient care.
Performance measures identify aspects of care from clinical guidelines that improve patient outcomes and for which data can be feasibly collected and acted upon (1,2). While performance measures have been created for both the inpatient management of acute cardiac conditions and for outpatient care (3–5), only compliance with the inpatient performance measures has been rigorously studied. Inpatient registries, such as the National Cardiovascular Data Registries (NCDR) and Get With The Guidelines, have demonstrated improvements over time in performance measure compliance at hospital discharge (6–9). To date, however, there have been no systematic efforts to prospectively assess compliance with American College of Cardiology (ACC), American Heart Association (AHA), and American Medical Association-Physician Consortium for Performance Improvement (PCPI) performance measures among outpatients. Given that the majority of cardiac patients are treated as outpatients, there is a compelling need to systematically measure the quality of care, as quantified by established performance measures, among outpatients so that potential gaps in the quality of outpatient care can be identified and addressed as targets for quality improvement.
Accordingly, we examined baseline compliance rates to performance measures among outpatients with coronary artery disease (CAD), heart failure, and atrial fibrillation in the ACC's PINNACLE (Practice Innovation And Clinical Excellence) program. The PINNACLE program, a prospective registry of outpatient care, represents an ideal data source to examine this question because it systematically captures outpatient compliance to each performance measure for these 3 cardiac conditions.
The PINNACLE program and study population
In 2008, the ACC launched the PINNACLE program (formerly known as the Improving Continuous Cardiac Care, or IC3)—the first, national, prospective, office-based, cardiac quality improvement registry in the U.S. Academic and private practices were invited to participate in the PINNACLE program through the ACC's website, e-mails, brochures, and information webinars. Physicians or practice representatives (e.g., administrators) in interested practices underwent a series of educational training sessions before data submission. Within participating practices, a variety of longitudinal patient data were collected at the point of care, including patients' symptoms, vital signs, medications, and recent hospitalizations (Online Appendix). In addition, data for each ACC/AHA/PCPI performance measure for CAD (3), heart failure (4), and nonvalvular atrial fibrillation (5) were collected. Data collection was achieved through 1 of 2 mechanisms at the practice level: 1) paper forms; or 2) modification of a practice's electronic medical record data collection system to comprehensively capture the additional requisite data elements for PINNACLE program participation. Data from practices were routinely submitted to the ACC's NCDR. Data quality checks and analysis were subsequently performed at the St. Luke's Mid America Heart Institute (Kansas City, Missouri), the primary analytical center for the PINNACLE program.
For the purposes of this study, we assessed clinical data from patients enrolled into the PINNACLE program from July 1, 2008, through June 30, 2009. For many of the cardiac performance measures, rates are reported at the patient level; therefore, we included data from only the baseline enrollment visit to minimize over-representation by patients with multiple visits. For several performance measures for which data are reported at the visit level, we included data from all visits during the study period. The final study sample comprised 14,464 patients enrolled from 27 practices encompassing 18,021 clinical visits.
The primary outcome was compliance with each of the ACC/AHA/PCPI performance measures for CAD (n = 11), heart failure (n = 11), and atrial fibrillation (n = 3). Compliance was defined as the number of patients (or visits) that met the performance measure divided by the number of eligible patients (or visits) for that performance measure. Patients were considered eligible if they met the established inclusion criteria for that performance measure and did not have a medical (e.g., beta-blocker use after myocardial infarction in a patient with hypotension or bradycardia) or personal (e.g., patient refusal to take warfarin for atrial fibrillation) contraindication for that measure. Because eligibility requirements differed across measures, a patient could be excluded from analyses for some performance measures but included in others. Thus, within these ideal subsets of patients, optimal performance was a rate of 100%. As secondary outcomes, we also examined whether compliance rates for select performance measures differed by race, sex, or age (<75 years vs. ≥75 years).
Summary statistics for patients' demographic and clinical characteristics were reported as means with standard deviations for continuous variables and as proportions for categorical variables. Compliance rates for each of the ACC/AHA/PCPI performance measures were then determined using aggregate proportions. The numerator for these rates was the total number of patients (or visits) that met the performance measure and the denominator was the total number of eligible patients (or visits) for that performance measure.
A priori secondary analyses examined whether compliance rates for 7 select performance measures differed by patients' race or sex. For the race analysis, we limited the study sample to those patients who identified themselves as of either black or white race (n = 12,261). The select performance measures examined included use of beta-blockers after myocardial infarction, lipid-lowering drug and antiplatelet therapy in patients with CAD, angiotensin-converting enzyme inhibitor (ACE-I) or angiotensin-receptor blocker (ARB) therapy for patients with CAD and either left ventricular systolic dysfunction (ejection fraction ≤40%) or diabetes mellitus (DM), beta-blocker and ACE-I or ARB therapy for patients with heart failure and left ventricular systolic dysfunction, and warfarin in patients with atrial fibrillation and a CHADS2score of ≥2 (10). The CHADS2score is a clinical prediction rule for estimating the risk of stroke in patients with atrial fibrillation, with 1 point each assigned for coexisting congestive heart failure, hypertension, age ≥75 years, and diabetes mellitus, and 2 points assigned for prior transient ischemic attack or stroke.
To accomplish these secondary analyses, separate 2-level hierarchical models were constructed for the race and sex analyses (11). In these models, the clinic site and the proportion of patients who were black or female at each clinic were evaluated as random effects, and their estimates represent within-clinic differences by race or sex, fully adjusting for all observable and unobservable between-clinic differences. Because compliance rates exceeded 10%, we utilized log-binomial or modified Poisson regression models at all steps, which estimate a relative rate (RR) directly (as opposed to an odds ratio obtained from logistic regression, which may overestimate racial or sex differences) (12,13). Finally, we also examined whether warfarin use in patients with atrial fibrillation and a CHADS2score of ≥2 differed by age (<75 years vs. ≥75 years), using similarly constructed hierarchical models.
For each analysis, the null hypothesis was evaluated at a 2-sided significance level of 0.05 with 95% confidence intervals (CIs) calculated. All analyses were performed with SAS version 9.2 (SAS Institute, Cary, North Carolina) and R version 2.7.0 (Foundation for Statistical Computing, Vienna, Austria) (14).
Of the 27 practices, all were cardiology subspecialty practices, 25 (92.3%) were private practices, and 11 (40.7%) were university affiliated. Seventeen (63.0%) practices were located in urban centers, one-half (14 [51.9%]) had 10 or more physician providers, and two-thirds (18 [66.7%]) employed nurse-practitioners or physician-assistants.
Baseline characteristics of the study population of 14,464 patients are presented in Table 1.The median number of patients enrolled from each practice was 354 (interquartile range 89 to 549 patients). The mean age of the study population was 67.2 ± 13.9 years, 2,234 (17.7%) patients with data on race were black, and 6,743 (46.6%) were women. More than one-half of enrolled patients had CAD, 1 in 3 patients had heart failure, and nearly 1 in 5 patients had atrial fibrillation. One-half (7,296 [50.4%]) of the study cohort had at least 2 of these 3 cardiac conditions, and 712 (4.9%) had all 3 conditions.
Within the study sample, the mean body mass index was 31.0 ± 8.0 kg/m2, mean systolic blood pressure was 127.4 ± 18.6 mm Hg, and mean diastolic blood pressure was 74.2 ± 11.4 mm Hg. More than one-half of the study cohort had prevalent hypertension or dyslipidemia, nearly one-half were current or former smokers, 1 in 5 patients had diabetes mellitus, slightly more than 10% had stable angina or a prior myocardial infarction, and a small percentage of patients had peripheral arterial disease, prior stroke, or recent coronary revascularization.
Compliance with CAD performance measures
Table 2summarizes rates of compliance with 11 performance measures among the 8,132 patients identified with CAD. Rates ranged from being very high for blood pressure assessment (94.0% [7,235 of 7,698]) to very poor for cardiac rehabilitation referral after myocardial infarction or coronary artery bypass surgery (18.1% [200 of 1,108]) and screening for DM (13.3% [822 of 6,199]). Adherence to beta-blocker therapy after myocardial infarction (86.4% [1,540 of 1,782]), ACE-I/ARB therapy in patients with concurrent left ventricular systolic dysfunction or DM (72.4% [3,349 of 4,623]), use of antiplatelet therapy (84.9% [6,742 of 7,944]), and annual lipid profile assessment (74.3% [6,044 of 8,132]) fell between these extremes. In addition, we were able to determine that use of thienopyridine therapy within 12 months of a drug-eluting stent—a Class I recommendation—was 81.9% (325 of 397).
Compliance with heart failure performance measures
Table 3summarizes rates of compliance with 11 performance measures among the 5,012 patients identified with any heart failure, of whom 1,076 (21.5%) had left ventricular systolic dysfunction. Similar to the CAD performance measures, compliance rates for heart failure performance measures ranged from being very high for blood pressure (96.7% [6,251 of 6,462 encounters]) and weight (96.0% [6,149 of 6,402 encounters]) assessment and for beta-blocker therapy in patients with left ventricular systolic dysfunction (93.1% [981 of 1,054]), to very poor for heart failure patient education (43.8% [2,194 of 5,012]) and assessment of heart failure clinical signs on examination (23.7% [1,533 of 6,462]). Compliance with ACE-I/ARB in patients with left ventricular systolic dysfunction (85.8% [879 of 1,025]), warfarin for patients with concurrent atrial fibrillation (80.6% [859 of 1,066]), and clinical symptom assessment (88.9% [5,742 of 6,462]) fell between these extremes.
Compliance with atrial fibrillation performance measures
Rates of compliance for 2 of 3 performance measures for atrial fibrillation could be reliably assessed in PINNACLE. Assessment of thromboembolic risk was performed in 1987 (73.6%) patients with nonvalvular atrial fibrillation, and this rate did not differ among those <75 years of age (72.0%) or ≥75 years of age (75.1%) (adjusted RR: 1.00; 95% CI: 0.97 to 1.04; p = 0.84). Nearly 3 in 5 patients with nonvalvular atrial fibrillation had a CHADS2score of ≥2 (n = 1,643 [59.0%]). Warfarin was appropriately used for stroke prophylaxis in nearly 4 in 5 of these patients (1,302 of 1,643 [79.2%]), and this rate did not differ among those <75 years of age (444 of 549 [80.9%]) and ≥75 years of age (858 of 1,094 [78.4%]) (adjusted RR: 0.96; 95% CI: 0.92 to 1.01; p = 0.15). Finally, among patients taking warfarin, we were unable to adequately assess patients' compliance with monthly monitoring of anticoagulation levels.
In secondary analyses for select performance measures, we found that compliance rates were similar between black and white patients for all 4 CAD performance measures and for warfarin use in patients with atrial fibrillation and a CHADS2score of ≥2 (Table 4).For heart failure performance measures among patients with left ventricular systolic dysfunction, compliance rates were similar between black and white patients for beta-blocker use, but blacks were less likely to receive ACE-I/ARB therapy (84.8% vs. 85.3%; adjusted RR: 0.93 [95% CI: 0.86 to 1.00]; p = 0.05), although these differences were numerically small.
We also found that compliance rates were generally similar between men and women for these select performance measures, although small numerical differences by sex were observed for warfarin use among patients with atrial fibrillation and a CHADS2score of ≥2 (80.7% for men vs. 75.7% for women; adjusted RR: 0.94 [0.89 to 0.99]; p = 0.03) and for ACE-I/ARB therapy in eligible CAD patients (72.1% for men vs. 71.7% for women; adjusted RR: 0.96 [95% CI: 0.92 to 1.00]; p = 0.05) (Table 5).
In this study, we examined compliance rates with performance measures among the first 14,000-plus outpatients enrolled into the PINNACLE program—the first, national, outpatient, cardiac quality improvement registry in the U.S. There were several key findings. First, we found that all but 1 of the 25 performance measures for CAD, heart failure, and atrial fibrillation could be feasibly measured in the PINNACLE program, demonstrating the feasibility of prospective performance measurement monitoring. Second, there was wide variation in compliance rates for CAD and heart failure performance measures and only moderate compliance for atrial fibrillation performance measures, suggesting current gaps in the quality of outpatient care. However, while rates were suboptimal for a number of performance measures, these results identify opportunities for future improvement and provide a valuable benchmark with which progress can be measured. Third, although racial and sex differences have been reported for a variety of medical procedures and treatments, there were no substantial racial or sex differences in compliance rates for key performance measures for these 3 cardiac conditions. Collectively, this is the first comprehensive report of cardiac performance measure compliance among outpatients in the U.S., and our findings provide important insights into the quality of care of outpatients with CAD, heart failure, and atrial fibrillation.
Our study findings extend the work of others who have previously examined compliance with cardiac performance measures for hospitalized patients. Studies within quality improvement registries, such as the NCDR and Get With The Guidelines, have previously found baseline gaps in the quality of inpatient cardiac care and improvements in compliance rates over time (6–9). However, determining compliance rates with key quality indicators among outpatients—who represent the large majority of patients with CAD, heart failure, and atrial fibrillation—has until recently remained elusive because of the unique challenges of collecting data on performance measures in outpatient clinics. This is because data on performance measures during an outpatient clinical encounter need to be prospectively captured at the “point of care” to ensure the accuracy and completeness of the requisite clinical data from which performance measures can be quantified. In contrast, data for inpatient registries can often be retrospectively abstracted. In this study, we were able to demonstrate that the PINNACLE program was successful in measuring compliance for 24 of 25 performance measures for 3 common cardiac conditions, thereby laying the foundation for future studies of outpatient compliance with cardiac performance measures and the potential use of the PINNACLE data for public reporting.
We found substantial variation in compliance rates for cardiac performance (from 13% to 97%), even after allowing practices to exclude patients for medical (e.g., contraindications, allergies) or personal (e.g., patient refusal, cost of medication) reasons for each measure. This suggests significant opportunities to elevate the quality of outpatient care, an area of growing national concern given the high 30-day readmission rates for both heart failure and coronary artery disease (15). Moving forward, the critical challenge for the PINNACLE program will be to define the mechanisms through which compliance rates for performance measures can be improved. That could occur through its informative quarterly reports with detailed benchmarking to other PINNACLE practices. However, there is a lag time with such reports, and they do not ensure that physicians in busy practices will be able to improve future compliance. Future development of real-time decision support during the clinical encounter may facilitate higher compliance rates and should be investigated as a means for improving care (16). Future PINNACLE studies will need to examine trends in performance measure compliance, the association of increased compliance rates with disease-specific outcomes (e.g., readmission rates), and the characteristics of practices with the greatest level of performance and improvement.
In general, busy clinical practices have been unwilling or unable to devote significant time and personnel resources for prospective outpatient data collection. The PINNACLE program has addressed these resource concerns in its design and implementation by: 1) allowing for data typically collected by a practice's electronic medical record system to autopopulate the PINNACLE data collection system so as to minimize redundant data entry; and 2) by providing an economic incentive (additional Medicare payments) for practices through improved reporting of performance for the Physician Quality Reporting Initiative (PQRI). The economic incentive arises because clinical practices would no longer have to expend time and resources to retrospectively collect the same data for performance measure and PQRI reporting at the end of the year, when the data would have little utility for quality improvement. If participation in the PINNACLE program further improves performance measure and PQRI compliance, this would provide an additional economic incentive for practices to participate.
Our study findings should be interpreted in light of the following limitations. This first report of the PINNACLE program involved more than 14,000 patients from 27 highly motivated practices. Therefore, compliance rates for cardiac performance measures in other U.S. practices may actually be lower than that reported in this study. Second, while practices were asked to submit data on all their cardiac patients, the PINNACLE program has no way of determining whether data on some cardiac patients were excluded from the program. However, to the extent that participating practices depend upon the PINNACLE program to report for their pay-for-performance and PQRI measures, it is in the practices' economic interests to submit complete data on all their cardiac patients. Third, because the PINNACLE program allows clinicians to designate medical and patient exclusions for each performance measure, it is possible that practices may “game” the system by assigning exclusions for patients who are otherwise not compliant with a particular performance measure. Despite this possibility, we still found substantial gaps in compliance for a number of performance measures. Fourth, it is possible that low compliance rates with certain performance measures reflected underdocumentation rather than underperformance. And finally, while compliance with performance measures is viewed as a metric of quality care, this current study was not designed to examine specific clinical outcomes. Adequately powered studies are therefore needed to examine the association of outpatient performance measure compliance and improvements in compliance over time with outcomes such as readmission and mortality.
Compliance rates for cardiac performance measures among outpatients with CAD, heart failure, and atrial fibrillation vary substantially, ranging from 13% to 97%. These results highlight important gaps in the quality of outpatient cardiac care and provide a valuable benchmark for future improvement.
The efforts and cooperation of the cardiology practices currently enrolled in the PINNACLE study are greatly appreciated by the authors and by the American College of Cardiology PINNACLE Work Group.
For the NCDR PINNACLE Registry Data Collection Form v1.0, please see the online version of this article.
Bristol-Myers Squibb/Sanofi Pharmaceuticals and the American College of Cardiology provide operational funding for the PINNACLE program. Ms. Fiocchi and Mitchell are employees of the American College of Cardiology. Drs. Chan and Spertus and Mr. Jones are affiliated with the Mid America Heart Institute, which is the major analytic center for the PINNACLE program and receives funding from the American College of Cardiology for this role. Dr. Rumsfeld is the Chief Science Officer of the NCDR. Dr. Spertus is supported by a contract from the American College of Cardiology Foundation; receives grant support from AMA, NHLBI, Anger, Johnson & Johnson, Eli Lilly, BMS/Sanofi, and Evoheart; and is a consultant for UnitedHealthcare, Anger, and St. Jude Medical.
- Abbreviations and Acronyms
- American College of Cardiology
- angiotensin-converting enzyme inhibitor
- American Heart Association
- angiotensin-receptor blocker
- coronary artery disease
- confidence interval
- diabetes mellitus
- American Medical Asssociation-Physician Consortium for Performance Improvement
- relative rate
- Received December 16, 2009.
- Revision received March 4, 2010.
- Accepted March 23, 2010.
- American College of Cardiology Foundation
- Spertus J.A.,
- Eagle K.A.,
- Krumholz H.M.,
- Mitchell K.R.,
- Normand S.L.
- Bonow R.O.,
- Masoudi F.A.,
- Rumsfeld J.S.,
- et al.
- Clinical Performance Measures: Chronic Stable Coronary Artery Disease
- Bonow R.O.,
- Bennett S.,
- Casey D.E. Jr..,
- et al.
- Estes N.A. III.,
- Halperin J.L.,
- Calkins H.,
- et al.
- Nallamothu B.K.,
- Wang Y.,
- Bradley E.H.,
- et al.
- Schwamm L.H.,
- Fonarow G.C.,
- Reeves M.J.,
- et al.
- Goldstein H.
- Zou G.
- Greenland S.
- R Development Core Team (2008)
- Krumholz H.M.,
- Merrill A.R.,
- Schone E.M.,
- et al.
- Kawamoto K.,
- Houlihan C.A.,
- Balas E.A.,
- Lobach D.F.