Author + information
- Received March 20, 2012
- Revision received May 16, 2012
- Accepted June 5, 2012
- Published online September 25, 2012.
- Kavitha M. Chinnaiyan, MD⁎,⁎ (, )
- Patricia Peyser, PhD†,
- Tauqir Goraya, MD, PhD‡,
- Karthikeyan Ananthasubramaniam, MD§,
- Michael Gallagher, MD⁎,
- Ann DePetris, RN, MSA⁎,
- Judith A. Boura, MS⁎,
- Ella Kazerooni, MD†,
- Chad Poopat, MD§,
- Mouaz Al-Mallah, MD∥,
- Souheil Saba, MD¶,
- Smita Patel, MD†,
- Steven Girard, MD, PhD‡,
- Thomas Song, MD§,
- David Share, MD# and
- Gilbert Raff, MD⁎
- ↵⁎Reprint requests and correspondence:
Dr. Kavitha M. Chinnaiyan, William Beaumont Hospital, 3601 West 13 Mile Road, Royal Oak, Michigan 48073
Objectives The purpose of the study was to determine the effectiveness of a collaborative educational, continuous quality improvement (CQI) initiative to increase appropriate use of coronary computed tomography angiography (CCTA).
Background Potential overuse of CCTA has prompted multisociety appropriate use criteria (AUC) publications.
Methods This prospective, observational study was conducted with pre-intervention (July 2007 to June 2008), intervention (July 2008 to June 2010), and follow-up (July 2010 to December 2010) periods during which patients were enrolled in the Advanced Cardiovascular Imaging Consortium (ACIC) at 47 Michigan hospitals. Continuous education was provided to referring physicians. The possibility of losing third-party payer coverage in the absence of a measurable change in AUC was emphasized. AUC was compared between the 3 periods.
Results The study group included 25,387 patients. Compared with the pre-intervention period, there was a 23.4% increase in appropriate (61.3% to 80%, p < 0.0001), 60.3% decrease in inappropriate (14.6% to 5.8%, p < 0.0001), 40.8% decrease in uncertain (10.3% to 6.1%, p < 0.0001), and 41.7% decrease in unclassifiable (13.9% to 8.1%, p < 0.0001) scans during follow-up. Between pre-intervention and follow-up, change in CCTA referrals by provider specialty were cardiology (appropriate: 60.4% to 79.5%; inappropriate: 13% to 5.2%; p < 0.0001), internal medicine/family practice (appropriate: 51.1% to 70.4%; inappropriate: 20.2% to 12.5%; p < 0.0001), emergency medicine (appropriate: 83.6% to 91.6%; inappropriate: 9.1% to 0.6%; p < 0.0001), and other (appropriate: 61.1% to 83.2%; inappropriate: 18.6% to 5.9%; p < 0.0001).
Conclusions Application of a systematic CQI and emphasis on possible loss of coverage were associated with a significant improvement in the proportion of CCTA examinations meeting AUC across referring physician specialties.
Coronary artery disease (CAD) results in more than half of cardiovascular deaths in the United States and remains the leading cause of morbidity worldwide (1). Potential consequences of incorrect diagnosis, including adverse events or continued therapy, have led to the overuse of cardiovascular imaging and antecedent healthcare costs during the last decade. This, in turn, has spurred research into the overall effectiveness of imaging modalities; multiple professional organizations have issued appropriate use criteria (AUC).
Rapid advances in coronary computed tomography angiography (CCTA) make it possible to image the coronary vasculature with excellent accuracy for defining the presence and severity of luminal stenoses, especially in low- to intermediate-risk individuals (2). Widespread introduction of 64-detector computed tomography scanners has advanced noninvasive coronary imaging and led to great enthusiasm for the clinical adoption of CCTA. This is now the fastest growing application of computed tomography scanners in the United States; approximately 500,000 Americans undergo CCTA each year, stimulating professional and public concern about appropriateness of its widespread use (3).
The purpose of this study was to evaluate the effectiveness of a collaborative continuous quality improvement (CQI) initiative on improving compliance with multisociety AUC for CCTA and whether such improvement could be demonstrated across all referring physician specialties.
The Advanced Cardiovascular Imaging Consortium (ACIC) is a statewide CQI initiative sponsored by Blue Cross Blue Shield/Blue Care Network of Michigan (BCBS/BCNM) and includes 47 centers performing clinical CCTA (4). It is approved by institutional review boards at participating centers, includes a waiver of consent, and is led by a physician “Clinical Champion” (Online Appendix). Patients undergoing CCTA at all participating centers during the study period were included in this analysis. ACIC data include demographics, risk factors, symptoms, results of prior testing, and medical history, as previously described (4,5). To ensure data accuracy, patients are interviewed at point-of-service for symptoms and history. Scheduled site audits are performed by the coordinating center for proper data collection on 2% to 10% of scans (depending on quarterly volume per site) and all major adverse events (death, acute coronary syndromes, and coronary revascularization). There is no set acceptable error rate for a center to remain in the ACIC.
Classification of appropriateness
This prospective study was conducted over 3.5 years (July 2007 to December 2010). Existing 2006 AUC (6) were used to define appropriateness. After data collection, the 2010 AUC (7) became available and used for data analysis. Additional analysis was performed using the 2006 criteria.
CCTA scans were classified into the 3 AUC categories using an automated computer-generated algorithm (7) (Fig. 1). CCTA scans not thus classified because of insufficient data or specific clinical scenarios not included in the AUC were adjudicated by a pre-identified team consisting of the Program Director (K.C.) and biostatistician (J.B.). When review of all available data did not result in grouping under the 3 categories, these scans were considered “unclassifiable.” This process was repeated using the 2006 AUC (6).
The CQI Program
Pre-intervention period: July 2007 to June 2008
During pre-intervention, sites received quarterly reports on their enrollment data with no emphasis on AUC.
Intervention period: July 2008 to June 2010
1. Study initiation: CQI was announced at the June 2008 quarterly meeting.
2. Site-specific plan: The primary CQI emphasis consisted of site-specific interventions led by Clinical Champions.
3. Educational plan: A secondary emphasis was education; conferences offering 3.75 continuing medical education credits were held in May 2010 at 4 locations around the state.
4. Site-specific activities: From February to June 2010, Clinical Champions led site-specific education (e.g., hospital grand rounds) and sent letters to their referring physician base about the AUC. Wording in the letters included imaging overuse and the possibility of losing coverage entirely by BCBSM/BCN in the absence of a definite, measurable change in appropriate use. All sites were mandated to participate in the CQI. Of all sites, 19 (40.4%) initiated prospective evaluation of indications with real-time feedback to referring physicians. Reporting of these activities was mandated by the CQI and by BCBSM/BCN for continued participation. No financial incentives were provided.
5. Data monitoring: Sites received feedback on their AUC adherence, site-specific trends, and comparison with ACIC as a whole. Clinical Champions were contacted when no changes were noted over two quarters; when deficient, meetings were set up among BCBSM/BCN representatives, Clinical Champions, hospital administrators, and ACIC staff to evaluate continued participation. No site was dropped for lack of CQI participation.
Follow-up period: July to December 2010
Quarterly comparative data were provided to sites; no active interventions were undertaken.
Categoric variables were examined using Pearson's chi-square or Fisher exact tests and are reported as counts and % frequencies. Continuous variables were examined using nonparametric Kruskal-Wallis tests or analysis of variance depending on the distribution of the data and are reported as mean ± SD or medians where appropriate. When considering the changes between pre-intervention and follow-up periods, specific 1 degree of freedom comparisons were made. A p < 0.05 was considered significant. All tests were 2-sided.
Clinical characteristics of study group
The study group consisted of 25,387 patients (Table 1). Compared with pre-intervention, patients in follow-up had higher coronary risk factors but normal coronary arteries.
Indications for CCTA and AUC adjudication
Among appropriate scans (n = 18,266), the most common indications were detection of CAD in symptomatic low- to intermediate-risk patients (10,925 cases, 30.4%) and in the setting of prior tests (5,342 cases, 29.2%) (Table 2). Among inappropriate scans (n = 2,198), the most common indication was the detection of CAD in asymptomatic individuals (1,510 cases, 68.7%), whereas the most common uncertain indication was detection of CAD in symptomatic low-risk patients (1,654 of 1,978 cases, 83.6%).
Primary outcome: appropriate CCTA
Pre-Intervention Versus Follow-Up
Appropriate use increased (3,672 of 5,993 [61.3%] to 2,642 of 3,302 [80.0%], p < 0.0001), inappropriate use decreased (873 of 5,993 [14.6%] to 192 of 3,302 [5.8%], p < 0.0001), uncertain scans decreased (618 of 5,993 [10.3%] to 202 of 3,302 [6.1%], p < 0.0001), and unclassifiable scans decreased (830 of 5,993 [13.9%] to 266 of 3,302 [8.1%], p < 0.0001) (Table 3).
Among inappropriate indications, there was a decrease of CCTA use in low-risk asymptomatic individuals (596 of 820 [72.7%] to 99 of 173 [57.5%], p < 0.0001) and high-risk patients with acute symptoms (137 of 820 [16.7%] to 22 of 173 [13%], p < 0.0001). There was an increase in asymptomatic patients for evaluation of graft patency (68 of 820 [8.3%] to 42 of 173 [24%], p < 0.0001) and with prior positive calcium scores (19 of 820 [2.3%] to 10 of 173 [5.5%], p < 0.0001).
Intervention Versus Follow-Up
Appropriate use increased (1,1601 of 15,396 [75.4%] to 2,642 of 3,302 [80%], p < 0.0001), inappropriate use decreased (1,305 of 15,396 [8.5%] to 192 of 3,302 [5.8%], p < 0.0001), uncertain scans decreased (1,225 of 15,396 [8.0%] to 202 of 3,302 [6.1%], p < 0.0001), and unclassifiable scans decreased (1,265 of 15,396 [8.2%] to 266 of 3,302 [8.1%], p < 0.0001).
Secondary outcome: Appropriate referrals by specialty
Referring physician specialties were available for 24,437 patients (96.3%). Significant changes were noted among all physician specialties between pre-intervention and follow-up (all p < 0.0001) (Table 4).
Comparison of 2006 and 2010 AUC
According to 2006 AUC, 5,053 (19.9%) of scans during the study period were appropriate, 5,255 (20.7%) were inappropriate, and 2,545 (10%) were uncertain. Approximately one half (N = 12,534, 49.4%) were unclassifiable because the 2006 AUC: 1) required more data for classification (e.g., electrocardiogram results and/or patient's ability to exercise); or 2) did not include specific indications (e.g., low-risk patients with acute chest pain and normal electrocardiogram and/or biomarkers).
When comparing 2006 (6) with 2010 (7) AUC, 9,953 (39.2%) CCTA scans remained in the same categories and 15,434 (60.8%) were reclassified (Fig. 2A). The greatest change was noted among the 9,240 “unclassifiable” scans: The 2010 AUC reclassified 8,916 (96.5%) as “appropriate,” 40 (0.4%) as “inappropriate,” and 284 (3.1%) as “uncertain.”
Of note, analysis using 2006 AUC demonstrated an increase in appropriate (18% to 23%, p < 0.0001) and a decrease in inappropriate scans (28.4% to 13.8%, p < 0.0001) during the follow-up period compared with pre-intervention (Fig. 2B).
Concordance between physician- and patient-reported indications
Concordance between physician- and patient-reported indications was studied; data were available for 24,691 patients (97.3%). During pre-intervention, there was 79% concordance, increasing to 82.8% during intervention and 85.3% during follow-up (p < 0.0001).
This study demonstrates that a CQI incorporating education and systematic peer-group feedback in a “real-world” setting across a variety of institutions and physician specialties is associated with more appropriate CCTA use.
In addition to defining the presence or absence of CAD, CCTA is cost-effective, provides diagnostic rapidity, and has excellent prognostic value in selected patients (8). Such characteristics combined with improved ease of interpretation have spurred widespread use and increased the potential for inappropriate use. This concern was confirmed by the fact that during the pre-intervention period, 14.6% of scans were inappropriate. However, reduction of inappropriate use to only 5.8% during follow-up is encouraging. There was a slight (5.9%) increase in the percentage of patients with normal coronary arteries during follow-up. The strength of CCTA lies in its high sensitivity and negative predictive value, that is, to rule out CAD. Thus, although slight, the increase in normal coronary arteries is indirect evidence of a corresponding increase in the appropriate selection of patients. Changes in CCTA use were demonstrated with application of both 2006 and 2010 AUC. Substantial changes to the AUC were evident by reclassification of scans from 2006 to 2010; the 2006 AUC contained 37 clinical situations, whereas the 2010 AUC contained 93, demonstrating their inevitable evolution. The significant increase in overall agreement between the physician- and patient-reported indications by the end of the study period also demonstrates that the change in appropriateness was not a function of physicians “gaming the system” by changing indications to “fit” the AUC.
These results differ from a physician education study by Gibbons et al. (9) that demonstrated no significant change in inappropriate single-photon emission computed tomography at a single academic center. The absence of feedback to referring physicians may have contributed to those results, whereas rigorous feedback to Clinical Champions and mandatory CQI participation were the key elements of the present study. Moreover, single-photon emission computed tomography has been in clinical use longer than CCTA with more “entrenched” referral patterns.
The ability of ACIC to affect quality metrics in “real-world” CCTA use has been demonstrated by a substantial radiation dose reduction by a similar CQI (5). These CQI programs are unique in that they rely on peer-to-peer feedback to encourage compliance with mutually agreed on quality metrics, as opposed to third-party audits and negative reinforcement in achieving similar outcomes.
One approach to moderating imaging use is by mandatory “pre-authorization.” However, pre-authorization processes are not often AUC derived. The present study suggests that voluntary, mutually established quality metrics reinforced by provider education and feedback can succeed in effectuating major changes, with implications extending beyond CCTA. Although the ACIC is a complex undertaking requiring resource organization at every level of the process (referral, scheduling, point-of-care, and follow-up), such collaborative partnerships between payers and physicians could result in similar successes on larger scales.
The most important limitation of this study is the lack of a control group, that is, centers not participating in a CQI. Although it is speculated that a CQI approach would be less burdensome than a pre-authorization model, these approaches were not compared.
During the study period, numerous external events affected CCTA use, including the deficit reduction act, substantial concern regarding test overuse, evolution in CCTA indications, and national recession. Although changes observed in this study could reflect concerted effort, these factors may have contributed to this trend. Moreover, payment for CCTA by BCBSM is contingent on ACIC participation. Although no threshold for appropriate testing was announced, motivation for CQI participation might have been at least partly driven by the appeal to continue to remain in ACIC. This CQI began in 2008, before the 2010 AUC were released; the 2006 AUC were the basis of the intervention. Because the 2010 AUC include more clinical scenarios, all analyses were retrospectively based on these criteria. Nevertheless, appropriate use of CCTA increased and inappropriate use decreased using both the 2006 and 2010 criteria.
Modified Framingham risk score was calculated on the basis of available clinical data. Data sampling for validation of this score was beyond the scope of this study. Although this method has been used in studies arising from large databases, the validity of these estimates has not been tested.
Limitations of the AUC also must be considered. Indications may fit into multiple tables; for example, symptomatic, intermediate-risk patients without known CAD may fit into both Tables 1 and 4 or symptomatic patients with prior revascularization and stress testing may fit into both Tables 4 and 6 (7).
Application of a systematic CQI program resulted in marked improvement in the proportion of CCTA examinations considered appropriate. This study presents an alternative, nontraditional approach to use management wherein physicians and payers collaborate to address the growing problem of inappropriate imaging.
For a list of the ACIC participating sites and clinical champions, please see the online version of this article.
This study was funded by Blue Cross/Blue Shield/Blue Care Network of Michigan. Dr. Ananthasubramaniam has received grant support, Astellas Pharma Global Development, Inc., GE Healthcare, Trovis Pharmaceuticals, and GlaxoSmithKline; speakers bureau/honoraria from Astellas Pharma, Inc. and Lantheus Medical Imaging; and is a consultant/advisory board member for Lantheus Medical Imaging and Astellas Pharma. Dr. Raff has received grant support from Siemens Healthcare. Dr. Share receives salary support from Blue Cross/Blue Shield/Blue Care Network of Michigan. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.
- Abbreviations and Acronyms
- Advanced Cardiovascular Imaging Consortium
- appropriate use criteria
- Blue Cross Blue Shield/Blue Care Network of Michigan
- coronary artery disease
- coronary computed tomography angiography
- continuous quality improvement
- Received March 20, 2012.
- Revision received May 16, 2012.
- Accepted June 5, 2012.
- American College of Cardiology Foundation
- Roger V.L.,
- Go A.S.,
- Lloyd-Jones D.M.,
- et al.
- Mowatt G.,
- Cook J.A.,
- Hillis G.S.,
- et al.
- Berenson A.,
- Abelson R.
- Hendel R.C.,
- Patel M.R.,
- Kramer C.M.,
- et al.
- Taylor A.J.,
- Cerqueira M.,
- Hodgson J.M.,
- et al.
- Gibbons R.J.,
- Askew J.W.,
- Hodge D.,
- Kaping B.,
- Carryer D.J.,
- Miller T.