Author + information
- Received August 19, 2009
- Revision received November 2, 2009
- Accepted November 2, 2009
- Published online January 12, 2010.
- Robert C. Hendel, MD*,* (, )
- Manuel Cerqueira, MD†,
- Pamela S. Douglas, MD‡,
- Karen C. Caruth§,
- Joseph M. Allen, MA§,
- Neil C. Jensen, MHA, MBA∥,
- Wenqin Pan, PhD‡,
- Ralph Brindis, MD¶ and
- Michael Wolk, MD#
- ↵*Reprint requests and correspondence:
Dr. Robert C. Hendel, 1 Kingswood Court, Riverwoods, Illinois 60015
Objectives The aim of this study was to assess the feasibility of evaluation for appropriate use of radionuclide myocardial perfusion imaging (MPI) in multiple clinical sites and to determine use patterns as well as identify areas of apparent inappropriate use.
Background Although cardiac imaging is highly valued for decision-making, the growth and expense related to these procedures has raised questions regarding overuse. The publication of appropriate use criteria (AUC), including those for MPI, were designed to provide guidance in the rational use of testing. However, limited data regarding the implementation and evaluation of AUC are available.
Methods Six diverse clinical sites enrolled consecutive patients undergoing MPI, collecting point-of-service data entered into an online form. An automated algorithm assigned a specific indication from the AUC that was classified as appropriate, uncertain, or inappropriate. Site-specific feedback was later provided to each practice on ordering patterns.
Results Of the 6,351 patients enrolled, 93% were successfully assigned an appropriateness level. Inappropriate use of MPI was found in 14.4% of patients, with a range of 4% to 22% among practices. Women and younger patients were more likely to undergo inappropriate MPI. Asymptomatic, low-risk patients accounted for 44.5% of inappropriate testing. Elimination of the 5 most common inappropriate use indications would reduce overall imaging volume by 13.2%. Inappropriate use by physicians from within the practice performing imaging was not greater than physicians outside of the practice. Educational feedback might have resulted in reduced inappropriate test ordering in 1 site.
Conclusions The tracking of appropriate use is feasible in clinical practice, with an automated system that can readily identify practice patterns and targets for educational and quality improvement initiatives. This approach might provide an alternative to utilization management.
Cardiac imaging is a mainstay in medical decision-making for patients with known or suspected heart disease. However, expenditures related to imaging comprise a significant portion of the health care budget (1–4). Much scrutiny has been focused on cardiovascular imaging with regard to the potential for overuse, especially in view of substantial geographic variation in ordering patterns and the limited amount of evidence-based data supporting the use of imaging as it relates to patient outcomes (3,4). Furthermore, financial incentives and issues related to self-referral have been suggested as explanations for the growth of cardiovascular imaging (3–5).
To address these use issues, appropriate use criteria (AUC) have been developed by several medical specialty societies, including the American College of Cardiology. This group began its focus on imaging use with the publication of the AUC for single-photon emission computed tomography (SPECT) myocardial perfusion imaging (MPI) in 2005, done in collaboration with the American Society of Nuclear Cardiology (6). Since then, AUC have been developed for other cardiac imaging modalities (7–9) as well as coronary revascularization (10). An updated version of the AUC for radionuclide imaging was recently published (11).
Clinical practice guidelines and AUC provide advice regarding the use of technology and therapeutic interventions to improve test selection and overall clinical care. Several single-site evaluations of AUC for cardiovascular imaging have been performed but might not be readily applicable to community clinical practices (12–16). Therefore, a collaborative prospective effort was undertaken by the American College of Cardiology Foundation (ACCF) and UnitedHealthcare (UHC), featuring an automated, objective method with the ACCF Appropriateness Criteria for SPECT MPI. The goals of this pilot study were to: 1) develop a process and data collection tool to evaluate the appropriateness of SPECT MPI testing; 2) implement the process and tool at the point-of-imaging-service in a variety of outpatient settings; 3) determine the rate of appropriate, uncertain, and inappropriate SPECT MPI studies; 4) develop feedback mechanisms to improve adherence to AUC; and 5) assess change in practice patterns over time.
Sites were selected from a list of locations performing at least 200 SPECT MPI studies for UHC members annually in community-based group practices. Hospital-based groups and practices with more than 50 physicians were excluded. Sites were selected by the investigators on the basis of diverse practice sizes, geographic location, and settings (urban, suburban, rural) as well as performance of an average total volume of >60 SPECT studies/month for all payers. Practices signed a letter of agreement outlining the terms for data collection. In exchange for participation, each practice was granted an exemption from prior authorization for cardiac imaging for UHC patients and received an unrestricted grant for participation in the pilot to cover administrative expenses. Local institutional review board approval for the study was obtained at each site.
Data were collected for all patients receiving a SPECT study in a consecutive fashion, irrespective of health plan coverage. Sites had staggered enrollment and terminated their participation at varying times throughout the course of the pilot. Information was obtained at the point-of-service (i.e., where the SPECT MPI test was being performed). Exercise physiologists, nurses, advanced practitioners, and physicians recorded clinical information on a brief, 2-page data collection form, which was then transferred to a web-based instrument that mirrored the data collection form. The information entered online was centralized into a relational database. Alternatively, practitioners or administrative personnel could enter data directly onto the online data entry tool via the pilot website, which was securely housed at ACCF headquarters. Collected data elements included patient demographic information, medical history, risk assessment, and information regarding prior cardiovascular procedures, such as revascularization and imaging procedures. UHC and its employees were not involved in the data collection, analysis, reporting of the results, or drafting of this report.
Appropriate use determination
Computer-based logic was used to assign each patient a specific indication on the basis of the previously published SPECT MPI appropriateness criteria (6), thereby permitting categorization of each scan as appropriate, uncertain, inappropriate, or unclassifiable. This was done in a hierarchical fashion, similar to that previously reported (13). Dyspnea was considered as an anginal equivalent and was categorized as “atypical chest pain.” To estimate Framingham risk, assumptions were made related to the use of medications, with the following conclusions: patients receiving statins were categorized as having hyperlipidemia; those in antihypertensive medications were classified as hypertensive, and so forth.
Approximately 6 months after study initiation, reports of appropriate, uncertain, and inappropriate testing rates were made available online in a real-time fashion at each site (“on-demand reporting”) should sites choose to access it. A summary report was also prepared and distributed to each site and to UHC. This report provided summary data for the sites on their performance, the most frequent appropriate and inappropriate indications for SPECT MPI, and also provided a blinded comparison with the performance of the other sites.
Feedback from the sites on data collection and use of the data was obtained via telephone calls throughout the study.
In addition to the summary reports, each practice was informed of detailed indications for inappropriate ordering of SPECT occurring at their site. Sites were also sent a sample letter that could be used as an educational tool for referring physicians about common reasons for ordering inappropriate tests. Pocket cards providing risk screening criteria and a list of the most common inappropriate indications were also provided.
Contingency tables were generated to explore the relationships between appropriateness classification and patient demographic information, symptoms, risk factors, practice locale, and whether the patient referrals came from within the practice or from physicians not belonging to the practice performing the SPECT study. Pearson chi-square test for nominal data and Kruskal-Wallis tests for ordinal and continuous data were used as tests for statistical associations. Inappropriateness rates were tested and compared across sites or over time with Pearson chi-square test or Cochran-Mantel-Haenszel test. Multivariable logistic regression models were performed to determine the independent covariates associated with patients receiving inappropriate scans. The generalized estimating equation approach was used to account for within-practice clustering. Due to the high rate of missing data for determination of CHD risk (17%), a multiple imputation technique was applied to the observed data to avoid bias in the regression analysis. This method is widely accepted for datasets with missing values (17). All statistical analyses were performed with SAS version 9.1 software (SAS Institute, Cary, North Carolina).
Study sites and enrollment
Enrollment of subjects lasted 1 year, on a rolling basis, from March 1, 2008, to until February 28, 2009. Three time periods were identified during the study: 1) before any report availability (before August 16, 2008), when no feedback or tracking information regarding practice patterns was available; 2) after availability of on-demand reporting (August 16, 2008, to October 14, 2008); and 3) after the distribution of formal, benchmarked site-specific, and overall study performance reports (after October 15, 2008).
Six sites participated in this pilot study; 3 urban, 2 suburban, and 1 rural location. Practices were located in Florida, Wisconsin, Oregon, and Arizona, and the number of cardiologists at each site ranged from 7 to 20 physicians. The number of SPECT MPI patients submitted from each site varied from 328 to 1,597 patients. Sites reported that data entry required 5 to 10 min/patient.
A total of 6,351 subjects with complete data were entered into the pilot database (Table 1).Risk factors were common—hypertension (76.7%), hyperlipidemia (72.9%), diabetes (22.8%). A history of coronary revascularization (coronary artery bypass graft surgery or PCI) was present in 47.5% of patients. Table 1presents the risk categories for patients without documented coronary artery disease (CAD). Among such patients, the pre-test likelihood of disease (18) was very low (<5%) in 6.0%, low (5% to 10%) in 34.7%, moderate (>10% to 90%) in 54.0%, and high (>90%) in 5.3%. The CHD risk data are also presented for patients without CAD; these data are similar to the cohort of asymptomatic patients without known CAD (n = 953), whose risk was assessed with the Framingham risk criteria (19), demonstrating that 68.3% of patients were at low risk, with moderate and high risk noted in 9.5% and 22.1% subjects, respectively.
Exercise SPECT imaging was performed in 54.0% of the subjects, with pharmacologic stress used in 43.9%; the remaining 2.1% had a combination of pharmacologic and exercise stress.
Appropriate use determination
The level of appropriateness was not able to be determined in 423 (6.7%) patients (Fig. 1A).The primary reason, accounting for 75.6% of these unclassified patients (n = 320), was incomplete data that precluded the calculation of risk or a missing date of revascularization, because the time since revascularization is an important determinant of appropriateness in asymptomatic individuals. An additional 85 patients qualified for more than 1 indication, such as patients that had both prior coronary artery bypass graft surgery and prior PCI. Finally, 18 patients had incorrect test dates or incomplete records. When the unclassifiable studies were eliminated from the analysis, evaluable subjects (n = 5,928) demonstrated appropriate use in 70.7% of cases, uncertain level of appropriateness in 14.9%, and inappropriate use in 14.4% of cases (Fig. 1B). Practice variation in inappropriate use ranged from 4.0% to 22.2%. However, significant differences in most patient-related factors, including age, sex, and risk factors, were present among the 6 study sites.
Factors affecting appropriateness category
Inappropriate SPECT MPI testing was performed more often in patients younger than 65 years of age (18.2% vs. 9.0%; p < 0.0001). Inappropriate testing was also more common in women (14.5%) than in men (12.6%; p = 0.039). Among those symptomatic patients without a prior diagnosis of CAD, very-low- or low-risk pre-test probability of CAD was associated with inappropriateness rates of 49.0% and 35.3%, respectively. Similarly, asymptomatic patients without known CAD who were at low CHD risk had a 63.0% incidence of inappropriate SPECT MPI, compared with rates of 1.1% and 6.8% for intermediate- and high-risk patients, respectively (Fig. 2).
Multivariate analysis revealed that asymptomatic status was the best predictor of an inappropriate classification, increasing the odds by 22-fold (Table 2).Sex was also an independent factor, increasing the likelihood of being considered inappropriate by 2.5 times for women compared with men, even when controlling for risk factors and symptoms. Among patients without known CAD, low CHD risk or a low pre-test likelihood for CAD were most predictive of an inappropriate use of SPECT MPI.
A comparison of appropriateness between tests ordered by cardiologists (n = 4,792) and noncardiologists (n = 1,136) noted a higher rate of inappropriate studies ordered by noncardiologists (19.5% vs. 13.2%; p < 0.0001). A similar analysis comparing the source of referral from within the practice (n = 4,881) compared with an external referral (n = 1,047) revealed a higher rate of inappropriate test ordering from outside of the practice (20.1% vs. 13.2%; p < 0.0001). Of note, individual physicians within the practice ordered approximately 5 times the number of SPECT MPI studies as the “outside” referring doctors.
The 3 most common causes for testing overall were all considered appropriate, including: 1) evaluation of chest pain syndrome after revascularization (19.5% of all cases); 2) evaluation of chest pain syndrome in a patient with intermediate pre-test probability of CAD and either unable to exercise or with an uninterpretable electrocardiogram (19.0%); and 3) evaluation of chest pain syndrome in a patient with intermediate probability of CAD who was able to exercise and had an interpretable electrocardiogram (9.6%). Table 3shows the 5 most common inappropriate indications, which alone accounted for 92.0% of all inappropriate testing. These 5 inappropriate indications also account for 13.2% of all SPECT MPI testing. Across all 6 sites, the most common inappropriate indication was for the detection of CAD in asymptomatic patients who were at low risk of CHD. At 5 of the 6 sites, the second most frequent inappropriate indication was the performance of SPECT imaging <2 years after PCI in an asymptomatic patient. This indication accounted for 23.8% of all inappropriate tests.
Temporal patterns of appropriateness
Due to the staggered initiation and termination of the study, only 4 of the 6 practices had sufficient data to examine rates of inappropriate testing during all 3 time periods: 1) before receiving any reports, representing baseline patterns of care; 2) voluntary access to on-demand reports; and 3) after receiving written report of practice performance and other educational materials. Only 1 of these sites had decreased inappropriate testing, whereas the others did not. Please see Figure 3for rates of inappropriate testing rates across time. Most sites chose not to actively review their testing appropriateness and did not attempt to hold educational sessions for ordering physicians. The single site that had a substantial change in the rate of inappropriate test use initiated an internal analysis of appropriateness data and held group meetings and discussions to educate physicians on compliance with the AUC. The management team at this practice was highly motivated to improve performance and made education of physicians a priority; their inappropriate testing rate was the highest of all the sites at baseline and decreased from 22.0% (40 of 182) to 13.3% (34 of 256) at the end of study (p = 0.04).
This pilot study is unique in that the evaluation of the appropriateness of SPECT MPI was done prospectively in “real-world” settings. Furthermore, the assignment of appropriateness category was done with an automated tool, which required a limited amount of data entry. The results of this investigation demonstrate that it is feasible to track patterns of SPECT MPI use in the standard workflow of contemporary clinical practice. The majority of tests were assigned to a designated level of appropriateness with an objective automated computer algorithm.
The appropriateness of SPECT MPI use was high, because only 14.4% of studies were deemed inappropriate, similar to single site reports in academic settings of 13% to 14% (12,13). However, there was significant variation across our community-based sites, with inappropriate studies ranging from 4% to 22% of SPECT examinations, which might be related to varying use by physicians of evidence-based guidelines and AUC. However, patient factors are likely part of the explanation, given that differences in patient characteristics and overall risk were present among the sites.
The most common indications for inappropriate testing with SPECT MPI were well-defined and similar to those noted in other publications (12,13). The identification of these high-impact areas for potential improvement is critical, because these are specific foci where educational efforts might be directed. Because 5 indications accounted for more than 90% of inappropriate testing, reduction or elimination of these procedures could result in an overall decrease in SPECT MPI volume of up to 13%. In contrast to broad-based reimbursement cuts or use management, this selective approach to reducing procedural volume is less likely to restrict access to testing among patients who might benefit from SPECT MPI.
Age and sex-related differences in appropriate use of SPECT were noted even after adjustment for comorbidities—similar to the study by Mehta et al. (12)—because women more often underwent inappropriate testing, as did patients younger than 65 years of age. Not surprisingly, patients with a very low or low likelihood of CAD or who had a low CHD risk were far more likely to have SPECT MPI designated as inappropriate, because the determination of appropriateness category is largely risk-based.
No clear pattern of temporal factors regarding appropriate use of SPECT was noted. However, this is likely due to many factors, including the staggered initiation of recruitment at each site and the limited educational interventions provided at the sites, with poor dissemination of reports at several practices. The reduction in appropriateness rate at one of the sites that did embrace a practice-changing initiative around appropriateness is anecdotal and not a definitive demonstration of the effectiveness of their quality assurance program. However, it does demonstrate that a high level of commitment to quality improvement can result in a reduction of the rate of inappropriate studies. This experience suggests that improving imaging practice patterns with AUC might be feasible.
Self-referral has been cited as a primary reason for the explosive growth rate in cardiac imaging (3–5), but minimal data are available in support of this notion. Although not independent predictors of inappropriate testing, data from this pilot study demonstrate more inappropriate SPECT MPI ordering by physicians outside of the practice that performs the imaging study. The contribution to inappropriate use might be greater in those who lack financial self-incentive as compared with those who might have a conflict, and that overall volume of tests ordered was not related to higher levels of inappropriate use. Noncardiologists order at least as many inappropriate SPECT MPI, suggesting a need for education to referring physicians outside of cardiology as well as use of a point-of-order decision tool to guide appropriate testing. The overall low volume of tests ordered by each external referring physician might partially account for the high inappropriateness rates, perhaps due to a lack of familiarity with current guidelines and appropriateness criteria.
Although much thought and effort went into the development of the SPECT AUC (6), this work was the first effort by the American College of Cardiology and other collaborating organizations to define the appropriate use of cardiac imaging procedures and as such might contain methodologic and/or interpretative inadequacies. Hence, the assessment of appropriateness can only be as good as the current AUC standards. Although the specific clinical indications were rated on the basis of expert opinion to develop the document, both indication writing and rating is intentionally based upon relevant medical published reports and existing practice guidelines. In the absence of a high level of evidence for many indications, AUC do rely on expert opinion, as do the American College of Cardiology/American Heart Association guidelines (20).
Nonevaluable data due to missing information or the presence of multiple potential indications for the same case led to a designation of “unclassified” in some patients. The rolling recruitment periods prevented a consistent time course for feedback and education. The educational efforts provided to practices were nonstandardized, leading to potential differing degrees of penetration into clinical practice. The assignment of appropriateness score might be problematic; however, the computer algorithm developed for this pilot study is automated and therefore objective and should avoid problems with interobserver variability, as noted in a prior study (13). Furthermore, the potential risk for misassignment of indications has been specifically addressed in the recent publication of the revised AUC for radionuclide imaging, which has hierarchical features and is algorithm-based (11). Finally, our findings are specific for the population and sites during the period of the study, and no data are currently available regarding whether the assignment of “appropriate” or “inappropriate” designations per the AUC document itself (6) is correct. Studies capturing the appropriateness of test ordering in the real world, such as the present report, are a necessary first step in the process of validating AUC, and future studies examining outcome data will help to resolve this issue.
Tracking practice performance in the use of medical imaging, such as with SPECT MPI, is feasible and might enhance efficiency with well-defined and carefully constructed AUC and a transparent, automated implementation approach. The patterns of inappropriate use documented in this study provide clear pathways for quality improvement and educational strategies and a foundation for a new type of quality metric in imaging, one that might be used in regulatory decision-making, including laboratory accreditation. Furthermore, these data suggest that clinicians should evaluate the pre-test likelihood for ischemic heart disease or the 10-year coronary heart disease risk when SPECT MPI is being considered.
The current health care environment continues to focus on resource use. Physicians, medical societies, policymakers, and payers can collaborate on potential solutions for medical imaging, optimizing quality and preserving patient access, but in a cost-conscious fashion. The implementation of AUC permits a targeted approach and is both feasible and has the potential to reduce over-use, with an emphasis on feedback and education. This pilot study suggests an alternative approach to traditional utilization management and has potential for widespread application.
The authors greatly appreciate the support of Reed Tuckson, MD, of UnitedHealthcare. The authors also wish to recognize American College of Cardiology Foundation staff Nichole Kallas and Kathleen Hewitt from the National Cardiovascular Data Registry as well as Eileen Hagan and Kathy Flood from Payer Advocacy.
For a supplementary table on the frequency of indications, please see the online version of this article.
Supported by grants from the American College of Cardiology Foundationand UnitedHealthcare.
Dr. Cerqueira is a consultant and on the Speakers' Bureau of GE Healthcare and Astellas Pharma, and is a consultant for Nordion. Joseph Allen is an employee of the American College of Cardiology. Neil Jensen is an employee of United HealthCare. Of note, UnitedHealthcare and its employees had involvement in this trial limited to partial financial support and in identifying candidate trial sites.
Presented as a Late Breaking Clinical Trial at the American College of Cardiology Scientific Sessions in Orlando, Florida, on March 29, 2009.
- Abbreviations and Acronyms
- American College of Cardiology Foundation
- appropriate use criteria
- coronary artery disease
- coronary heart disease
- myocardial perfusion imaging
- percutaneous coronary intervention
- single-photon emission computed tomography
- Received August 19, 2009.
- Revision received November 2, 2009.
- Accepted November 2, 2009.
- American College of Cardiology Foundation
- Hendel R.C.
- Medicare Payment Advisory Commission (MedPAC) Report to Congress: Improving Incentives in the Medicare Program. June 2009 http://www.medpac.gov/documents/Jun09_EntireReport.pdf. Accessed November 14, 2009.
- Brindis R.G.,
- Douglas P.S.,
- Hendel R.C.,
- et al.
- Hendel R.C.,
- Patel M.R.,
- Kramer C.M.,
- et al.
- Douglas P.S.,
- Khandheria B.,
- Stainback R.F.,
- Weissman N.J.
- Douglas P.S.,
- Khandheria B.,
- Stainback R.F.,
- Weissman N.J.
- Patel M.R.,
- Dehmer G.J.,
- Hirshfeld J.W.,
- et al.
- Hendel R.C.,
- Berman D.S.,
- Di Carli M.F.,
- et al.
- Gibbons R.J.,
- Miller T.D.,
- Hodge D.,
- et al.
- Ward R.P.,
- Mansour I.N.,
- Lemieux N.,
- et al.
- Rubin D.B.
- Gibbons R.J.,
- Balady G.J.,
- Bricker J.T.,
- et al.
- Grundy S.M.,
- Pasternak R.,
- Greenland P.,
- et al.