Author + information
- Robert C. Hendel, MD, FACC, FAHA, FASNC,
- Manesh R. Patel, MD, FACC,
- Joseph M. Allen, MA,
- James K. Min, MD, FACC,
- Leslee J. Shaw, PhD, FACC, FASNC, FAHA,
- Michael J. Wolk, MD, MACC,
- Pamela S. Douglas, MD, MACC, FAHA, FASE,
- Christopher M. Kramer, MD, FACC, FAHA,
- Raymond F. Stainback, MD, FACC, FASE,
- Steven R. Bailey, MD, FACC, FSCAI, FAHA,
- John U. Doherty, MD, FACC, FAHA and
- Ralph G. Brindis, MD, MPH, MACC, FSCAI, ex officio
The past several decades have seen rapid and extensive changes in the practice of cardiology, especially in the innovation and utilization practices of imaging, interventional, and electrophysiology procedures. Enhanced radionuclide imaging techniques, evolution of echocardiography, development of cardiac magnetic resonance (MR), and coronary computed tomography (CT) angiography techniques, as a well as drug-eluting stents and cardiovascular implantable electronic devices, have revolutionized how patients are diagnosed and treated. Although these developments have resulted in direct patient benefits including improved survival and enhanced quality of life, there has been an accompanying increase in resource utilization and healthcare costs. Although declines in utilization of many cardiovascular procedures have been observed as of late, during the years preceding 2005, the growth rates were at times substantial as these technologies were adopted. The perceived high rate of growth of expenditures related to cardiovascular procedures has precipitated payers to initiate utilization constraints to markedly reduce spending and reimbursement. Various payer initiatives have created an onerous burden leading to costly administrative requirements, including physician profiling and prior authorization (1). These general programs are also, in part, driven by marked geographic variability in utilization, which underscore the need for further guidance regarding optimal patient selection for procedures (2,3). Professional efforts to better define quality have identified the importance of matching procedures and patients (4).
In response to the imperative for improving the utilization of cardiovascular procedures in an efficient and contemporary fashion, the American College of Cardiology Foundation (ACCF), along with imaging subspecialty societies and other organizations, developed the first set of Appropriate Use Criteria (AUC) in 2005, focusing on indications for radionuclide imaging (5). A concurrent publication defined in some detail the methods involved in the construction of these criteria (6). During the ensuing 7 years, there have been numerous other AUC publications (Figure 1)—including revisions to several of the original criteria—that reflect expansion of the AUC concept and advances within the specific disciplines, as well as methodological changes. The aim of the current paper is to review recent modifications in the methods for developing AUC, most notably substantial alterations in the nomenclature employed for appropriateness categorization (detailed later in the text).
Although the methods for AUC construction have evolved, the core process remains rooted in the application of the validated, prospectively based modified RAND Appropriateness Method (7,8) as was described in the initial methods article (6). Readers interested to learn more about the RAND method and adaptations developed by ACCF are encouraged to review these papers, as well as the current paper. Because the RAND method offered a general approach to constructing criteria, most of the approaches taken by ACCF in the application of the method are interpretations required for implementation rather than changes to the basic framework of the RAND approach. As such, to address the feedback of the clinical cardiologists, other patient-care professionals, payers, and regulators, the ACCF AUC Task Force (formerly the Appropriateness Criteria Working Group) has refined its methods over time. The basic methodology is shown in Figure 2. This paper provides an update describing the various revisions that have improved the rigor and consistency of the AUC development process. A comparison of the major changes between the original and updated methods is summarized in Table 1. Additionally, this paper describes the modifications in methods designed to facilitate the creation of multimodality AUC, an initiative aimed to summarize, in 1 document, current recommendations regarding multiple procedures. A brief summary of studies examining the AUC process and their evaluation and implementation to date is also provided.
Defining Appropriate Use
In the first series of AUC documents, there was strict adherence to the terms of appropriateness established by RAND methodology, including the title, “appropriateness criteria” (6–8). More recent documents have adopted the term “Appropriate Use Criteria.” This change reflects the more active function of the AUC in promoting evidence-based, effective use of cardiovascular technologies. Another change since the first publication of AUC has been the expansion of this initiative from imaging to devices and procedures including coronary revascularization, diagnostic catheterization, implanted cardiac defibrillators, and cardiac resynchronization devices. Although imaging remains a critical focus of the AUC, the ACCF recognizes that there is a need to address the clinical applications of the appropriate utilization of all cardiovascular procedures. In response, the AUC Task Force has expanded the list of potential topic areas.
The definition of appropriate use is largely consistent across technologies and procedures, and includes consideration of benefits and risks; although specific definitions for terms and surrounding assumptions are modified based on the most clinically relevant aspects. The basic definition states:
An appropriate diagnostic or therapeutic procedure is one in which the expected clinical benefit exceeds the risks of the procedure by a sufficiently wide margin such that the procedure is generally considered acceptable or reasonable care.
For diagnostic imaging procedures, benefits include incremental information, which when combined with clinical judgment augment efficient patient care, weighed against the expected negative consequences (risks include the potential hazard of missed diagnoses, radiation, contrast, and/or unnecessary downstream procedures).
For therapeutic procedures such as revascularization or implanted cardiac defibrillator/cardiac resynchronization therapy, the benefits include survival or health outcomes (such as improved symptoms, functional status, and/or quality of life) weighed against the risks of the procedure and subsequent related care.
These definitions may vary slightly but always emphasize whether a test or procedure is a reasonable care option for the population of patients described by the indication after consideration of benefits and risks.
The specific terminology used to define appropriate procedural utilization has been discussed extensively since the publication of the first AUC document by ACCF. The intent of the AUC effort was to focus on patient populations, case mix over time, and quality improvement while informing, but not dictating, care for individual patients. The 3 categories were meant to reflect a continuum of benefits and risks to various patient populations. However, the categorical nature of AUC inherently placed individual patient cases into 3 distinct groups that were often viewed as absolute. Unfortunately, AUC efforts to describe the relative proportion of patients who might or might not benefit from a procedure over time were obscured by concern over how individual cases would be adjudicated. Such interpretation often led to misperceptions by all stakeholders about when a procedure may be considered for a patient. For instance, should procedures classified as Uncertain but still used be considered questionable care choices? Should procedures categorized as Inappropriate always be avoided or could there be mitigating circumstances in an individual patient making the procedure a reasonable choice? Should all procedures that are classified as Appropriate be performed in patients or may some patients go without the procedure? In order to address these issues, the AUC Task Force surveyed numerous stakeholders to consider clarifications to the terminology during the summer of 2012. The feedback suggested a change to the terms was necessary, as the current AUC terminology was unable to properly communicate the goals of each stakeholder in a consistent fashion. The result was a recommendation by the AUC Task Force to modify the terms and clarify the definitions. The updated terms and definitions more closely reflect their application in practice, including an expected distribution between each AUC category for every population, methods for documenting exceptions, and proper application to individual patients. These changes reflect the continued commitment by the AUC Task Force and the College to measuring, benchmarking, and improving the appropriateness of procedures for patients while reducing use in populations who will rarely benefit. As such, the contemporary, revised definitions for the 3 categories described by Appropriate Use documents herein will be the following:
Median Score 7 to 9: Appropriate Care
An appropriate option for management of patients in this population due to benefits generally outweighing risks; effective option for individual care plans although not always necessary, depending on physician judgment and patient specific preferences (i.e., procedure is generally acceptable and is generally reasonable for the indication).
Median Score 4 to 6: May be Appropriate Care
At times, an appropriate option for management of patients in this population due to variable evidence or agreement regarding the benefits/risks ratio, potential benefit based on practice experience in the absence of evidence, and/or variability in the population; effectiveness for individual care must be determined by a patient's physician in consultation with the patient based on additional clinical variables and judgment along with patient preferences (i.e., procedure may be acceptable and may be reasonable for the indication).
Median Score 1 to 3: Rarely Appropriate Care
Rarely an appropriate option for management of patients in this population due to the lack of a clear benefit/risk advantage; rarely an effective option for individual care plans; exceptions should have documentation of the clinical reasons for proceeding with this care option (i.e., procedure is not generally acceptable and is not generally reasonable for the indication).
Although the definition of Appropriate generally has been an effective and widely accepted term, concerns exist that use of the term may stimulate overuse of a procedure in whom alternative management strategies may exist. The task force continues to emphasize that an Appropriate procedure is a reasonable option but may not uniformly be necessary for such patients. AUC do not supplant the need to formulate decisions with individual patients. Necessity reviews have been undertaken by RAND to determine when procedures rated as appropriate are required. Although important, ACCF AUC has not undertaken such reviews to help define areas of potential underuse, but ACCF and the American Heart Association (AHA) clinical practice guidelines and especially performance measures are well suited for addressing such questions.
All of the AUC publications have emphasized that patients in the Uncertain category may be reasonable candidates for the procedure. The task force continues to emphasize that this category represents a patient population for whom individual care must be determined by the physician working directly with the patient. Although additional research may at times help contribute to future understanding of procedure use in this population, the current state of science and clinical experience indicates that physicians and patients may consider the procedure among care options for this population. As such, this rating should be not be used as the primary basis for denying coverage and reimbursement. This understanding has been emphasized and included in the deliberations of each rating panel. To address these issues, as well as to obviate any ambiguity associated with the term, the task force has determined that future published AUC will replace the term Uncertain with May Be Appropriate.
As mentioned in the preceding text, concerns have been raised indicating substantial misperception related to the term Inappropriate, even though this term was extracted directly from the UCLA/RAND methodology. The intent of the term was related to practice patterns, which allows for a small percentage of Inappropriate use (6). Unfortunately, misunderstanding has been present regarding this category as it applies to individual cases and AUC patterns of organizations and physicians over time. The inherent limitation of the AUC are the attempt to categorize individual patients with 3 to 4 simple characteristics, which serve well for a population but may not capture specific individual patients. For these patients, the AUC have consistently emphasized the need to document these differences while avoiding overuse of the procedures in these patient populations. Therefore, the AUC Task Force has substituted the term Rarely Appropriate for the previous term Inappropriate. However, physicians should be aware that procedures in this category should have unique individual patient circumstances that are documented to justify their use, and be especially cautious to avoid procedures that, in clinical guidelines, indicate potential patient harm if the procedure is performed.
Appropriate Use Criteria Development
Step 1: Topic Selection, Nomination Process, and Writing Group Composition
Topics are selected by the AUC Task Force after careful review of the current variation in utilization, stakeholder needs, procedure volume and cost, available evidence, and feasibility. Input is uniformly sought from ACCF councils and committees, cardiovascular and imaging specialty societies, noncardiologists, and relevant stakeholders such as health payers, policymakers, and patients. Although in some instances, AUC development can provide a logistical framework for the implementation of evidence synthesized by clinical practice guidelines, AUC are often of particular importance when gaps in the medical literature, clinical evidence, or guidance publications are present. In particular, the AUC are aimed at guiding both diagnostic and therapeutic procedures based upon specific, commonly encountered clinical scenarios, many of which cannot be studied in randomized trials. The AUC Task Force fosters up-to-date guidance that encompasses the collective expert opinion of a multidisciplinary group of clinical, payer, policy, and other stakeholders. In other words, AUC are both evidence-based and utilize expert consensus. Nominations for the writing group, reviewers, and the rating panel are solicited from a broad set of partnering organizations and societies and selected by the task force in consultation with partnering societies and the writing group. Relationships with industry and intellectual bias based on clinical and professional expertise is considered during the nomination and selection process. Further information about relationships with industry can be found on page 14, and a discussion of the composition of the groups can be found within each of the group descriptions on the following pages.
The writing group composition has evolved over time but continues to include members with significant professional expertise and to broadly represent multiple stakeholders. In contrast to the initial AUC documents' writing groups that were predominantly composed of members from the AUC Working Group, writing groups are now composed of 6 to 10 members from multiple societies and diverse organizations, allowing for broader representation across disciplines. A substantial portion of writing group members remain experts in the technique under consideration to ensure that indications are constructed to capture the clinical applicability and limitations of the technologies or therapy under consideration.
Further, an AUC Task Force member is now appointed to each writing group to serve as a liaison to provide methodological and operational oversight. The full AUC Task Force also serves to review and approve relevant clinical indications, review literature summaries, provide guidance on methodological issues, ensure harmonization of indications, definitions, and assumptions across AUC documents, whenever possible, and to foster finalization of the AUC documents in a time-efficient fashion.
Step 2: Indication Development and Literature Review
Substantial effort has been made to standardize the wording for the clinical indications, assumptions and definitions as they span different documents for the various modalities/procedures.
As clinical indications or scenarios are developed, consistency, clarity, and utility are emphasized to support a foundation for meaningful evaluation. Whenever possible, indications are grouped under common headings. This structured approach, when applied to cardiac imaging, commonly includes construction of tables for patient diagnosis and risk assessment, current symptomatology, prior testing, previous revascularization, evaluation for a change in clinical status, and consideration of special clinical circumstances. At times, additional groups of scenarios may be included, such as the use of routine surveillance testing either early or late after a procedure or test. The AUC increasingly attempt to address such scenarios, recognizing that the clinical trial evidence base is often quite limited and, given the cost, may never be achievable in the current healthcare environment. AUC seek to establish widespread consensus around the timeframes during which such testing is unlikely to yield important clinical information. Although initially it was felt that the AUC should not attempt to be all-inclusive but rather focus on common, real-world situations; external feedback regarding the early AUC documents highlighted gaps in indications or unclear clinical scenarios. The AUC Task Force responded to this feedback, by significant expansion and revision of the indications in subsequent documents (9). The ensuing applications of revised AUC are reflective of this expansion, noting that the added indications have markedly improved the utility of the documents (10).
When possible, AUC now provide a hierarchy of indications to guide use of the AUC in a systematic fashion, tailored for each modality, and assist in applying a particular clinical situation to 1 of the indications, an example of which is depicted in Figure 3. This approach also greatly facilitates AUC evaluation and implementation, as it permits an ordered and reproducible way to apply AUC.
The assumptions present within the AUC have been broadened over time to assist in implementation and are aimed at issues such as the competency to perform procedures, inclusion criteria, exclusion criteria including consideration of common contraindications, standard protocols, and understanding of common procedure risks. For example, the details regarding the structure and performance of the imaging laboratory, catheterization laboratory, or operating suite are now more explicit, as these are likely to be included in evaluation for accreditation. These assumptions ensure that the rating process proceeds with a common baseline so that the rating is based upon clinical issues rather than variability in competence, test quality, or other issues. Although important, these issues are beyond the scope of AUC.
Commonly used terms and scenarios, such as “ischemic equivalent,” are used consistently across documents and have been redefined for improved clarity (10–12). For example, the incorporation of electrocardiographic abnormalities as a clinical finding in the definition of an ischemic equivalent now permit extension of this indication beyond symptoms. In addition, assessment of risk in asymptomatic individuals has been modified to reflect the concept of global risk, which may be based on any of the major literature-based risk scores, including the Adult Treatment Panel III, modified Framingham risk, or the Reynolds's score, to incorporate the most recent data and include items such as family history in the evaluation (10–12). In fact, the AUC has helped to highlight the need for better definitions and methods around risk stratification, pre-test probability, and findings from noninvasive tests.
To ensure consistency across AUC as well as other guidance documents, the task force relies heavily on ACCF/AHA Clinical Practice Guidelines for many of its definitions and assumptions. Additionally, the task force, has had a continued dialog with the Clinical Practice Guidelines Task Force to ensure harmonization.
Guideline Mapping and Evidence Review
Once the clinical indications are developed, a table mapping the indications to recommendations of other guidance documents, in particular ACCF/AHA practice guidelines, is constructed to preserve and encourage agreement across various clinical policy documents as related to the specific AUC technology being evaluated. Practice guidelines often contain a comprehensive summary of the available literature and serve as the primary source of evidence review for the AUC process. As these guidelines are updated, the writing groups are instructed to update the assumptions, indications, and definitions accordingly. New guidelines, such as the pre-operative guidelines (13), are incorporated into the AUC as they are written and revised, often resulting in changes in the wording of clinical indications. For evidence not covered by the Guidelines, the writing group assembles the most up-to-date scientific evidence as it may relate to the topic and specific clinical indications.
Rating Panel Materials
After these various components have been drafted, a standardized literature review and ratings package is prepared by each writing group following a pre-set template. The template includes a preamble, instructions on the rating process, definitions, assumptions, abbreviations, indication tables, and evidence/guideline mapping tables. These items are then reviewed and approved by the parent AUC Task Force. The task force and writing committees have continually expanded and clarified these materials, providing supportive documentation sufficient for reviewers to provide feedback and the members of the rating panel to render a score on each clinical scenario that is reflective of the current evidence base.
Step 3: External Review of Clinical Indications
Subsequent review of the indications by the AUC Task Force and external peer reviewers is aimed at further improvement of the definitions of the clinical scenarios. This was previously done by an ad hoc group of experts, but this process has now been formalized to include a detailed external review by representatives from key participating medical specialty societies, as well as those representing health services research and those with familiarity with clinical practice guidelines and other key stakeholder's policies. This often involves more than 40 individuals in the review of the draft set of indications. This process is crucial to indication development, because revisions are not allowed following final rating panel voting, as this would violate the basic tenets of the modified Delphi methodology.
Step 4: Panel Rating
Rating Panel Composition
The AUC Task Force has always attempted to maintain a balance on the rating panels (previously referred to as the technical panel) of specialists using the technology and other professionals who are referrers, general cardiologists, outcome specialists, and/or generalists who care for germane patient populations, as well as the payer community. Overall, a diverse panel composition ensures the production of balanced, equitable, and reproducible ratings (6–8). Specialists whose key area of interest is the primary focus of the AUC are a minority of rating panel members. A survey of practice experience is sent to all rating panel members before formalizing their appointment to the panel. This information is used to ensure that the task force and writing group, who determine the composition of the rating panel, are able to consider an accurate description of the individuals' expertise, interests, and relationships before selection to serve on the panel. This approach has been recently challenged by some who believe that the rating panel should be composed either exclusively or by a majority of specialists within the field under evaluation, stating the view that such subspecialists have a unique and greater understanding of the field (14). However, the AUC Task Force believes that AUC should be as evidence-based as possible and that all members of the rating panel should be able to determine the balance of clinical benefits and risks of a particular procedure or technology based on practice experience and relevant literature, especially when provided with evidence tables and practice guidelines. In addition, such nonexpert individuals often represent the community of practitioners ordering the test or procedure and provide an important perspective. As such, the AUC Task Force believes that the use of a majority of “nonspecialists” within a rating panel permits a diversity of perspectives and enhances external credibility.
After using an even number of panel members on the first AUC, which required rounding to determine the final median score and resulting AUC category, the task force decided that all rating panels must be composed of an odd number of individuals so that the final median score reflects the whole number score of an actual rating panel member.
Initial Panel Ratings and Meeting
The first round of indication ratings are performed independently by each rater. Following this, participation in a face-to-face meeting is required. In addition to the panel members, the AUC Task Force has standardized specific roles for several other individuals during this meeting. A moderator who has not been involved in indication construction establishes the goals, procedural rules, and facilitates the meeting. This individual typically does not participate in rating the specific technology or procedure under review and does not practice the procedure being reviewed. The moderator may serve as an alternate panel member if unforeseen circumstances prevent a panelist from completing the process. During the face-to-face meeting, the writing group liaison member is also present to answer questions specific to the indications and to assist in any modification or clarification of indications recommended by the rating panel. Lastly, a member of the AUC Task Force is present specifically to address methodological issues as they pertain to the AUC.
During the rating panel meeting, a standardized presentation is given providing an overview of the AUC development process and a review of the assumptions and definitions along with an outline of the indication tables and key clinical parameters used in the document. The panelists are provided with their own votes and a tabulation of all votes presented in an anonymous fashion. All indications are discussed using a “round robin” discussion style to ensure all panel members have the opportunity to lead and participate in the discussion. Particular attention is paid to indications with widely divergent ratings to ensure that there is a uniform understanding of the clinical scenario. Following the face-to-face meeting, the panelists then independently rescore all indications. In a very few cases in which there is a wide dispersion of scores and where further indication rewording would better align the indication with evidence and/or practice experience, a third rating of the specific indication(s) in question may be undertaken. The median score for each clinical scenario becomes the final indication rating.
Final Rating Tabulation
The final scores are reported in discrete categories (Appropriate, May Be Appropriate, and Rarely Appropriate—as well as with the numerical median rating, with anonymized individual scores available in an online appendix).
In addition to the final indication score, all indications are assessed to measure the level of agreement among the panel. When the number of rating panelists is less than or equal to 15 members, the definitions for agreement from BIOMED Concerted Action on Appropriateness are used, as previously described (6,8). For panels with more than 16 members, this definition is applied first, but a second statistical method described in the RAND Appropriateness Method (7) is also applied. Both measures examine dispersion in the ratings and identify whether most ratings are near the final median (agreement) or clustered at opposite sides of the rating scale (disagreement). By definition, indications labeled as having disagreement would automatically become May Be Appropriate. However, to date, no indication has been qualified under this designation that was not already uncertain.
Multimodality Appropriate Use Criteria
The original methods paper (6) discussed the expectation that an evaluation of the efficient use of technology among alternative technologies or procedures would have to be considered in the future across similar clinical indications. It was recognized early on that this would likely be necessary for noninvasive cardiovascular testing and other procedures, such as coronary angiography and revascularization, and other interventional therapeutic procedures.
The AUC Task Force made a first attempt at such an evaluation in the Appropriateness Criteria for Coronary Revascularization (15). The writing group first discussed the variables essential for the consideration of any form of coronary revascularization. This led to the framework for the clinical indications categorized by several domains: acuity, prior bypass surgery status, presenting symptoms, ischemia severity on noninvasive testing or fractional flow reserve, and coronary anatomy, as well as adequacy of medical therapy. The rating panel was then asked to rate each indication for the appropriateness of coronary revascularization, independent of the mode (either percutaneous coronary intervention or coronary artery bypass graft surgery). In a select set of clinical indications with a high burden of ischemia and/or coronary artery disease, each procedure (percutaneous coronary intervention and coronary artery bypass graft surgery) was then rated separately. A separate table presented the relative appropriateness of each procedure for this limited set of indications.
Diagnostic Testing and Cardiovascular Imaging
A document in progress will focus on multiple tests used to evaluate patients presenting with signs and symptoms suggestive of coronary artery disease or follow-up of known coronary disease. This AUC will continue the ongoing process of establishing similar assumptions, definitions, evidence synthesis, and clinical indications for several tests that could be used for patient diagnosis and management of a given clinical indication/scenario.
This multimodality AUC will evaluate clinical scenarios that are similar and often identical to those published in prior AUC documents. However, it will also include testing techniques not previously covered, such as: exercise treadmill testing, so as to include the entire spectrum of cardiac testing for ischemic heart disease. One key aspect of this work is that only the category of appropriate use will be reported rather than a numerical value. The 1 to 9 values used in the AUC development process are used as a guide to help rating panels put procedures into the 3 categories. The rating panel is asked to provide a numerical value corresponding with their final desired category for the procedure. The wider range of numerical values provides individual raters with the ability to grade the procedure on a continuum, thus allowing raters with slightly different viewpoints to dialog but also allow the final ratings to converge into a single message. Given the potential minor variability across a rating panel in their final scores within a category, the numerical scores are less reliable. All raters understand it is the final categories that describe more generally how often the procedure will be considered. Therefore, numerical values should not be used to apply gradations and comparisons between procedures scored in the same category. Central to the rating in this document will be an evaluation of whether any test is needed and if true differences in appropriate use of different tests are present for specific patient populations. Although 1 goal of this document could have been to determine the ranking of one modality versus the others, there is limited comparative evidence for specific imaging modalities. As such, comparative ranking of tests within an appropriateness category is not the focus of this document beyond known limitations of specific tests for certain populations that may result in different appropriate use categorizations. In most cases, it is anticipated that the clinician may have his/her choice of several Appropriate procedures or that several will be rated as May Be Appropriate or Rarely Appropriate. Beyond the ratings in this document, clinicians may selectively identify procedures/scenarios where other secondary considerations may apply for specific patients, including safety, cost, local expertise, availability, patient preference, and so on.
Relationship With Clinical Practice Guidelines and Other Clinical Policy
Consistency of the AUC with clinical practice guidelines is critical to avoid confusion on the part of healthcare practitioners and payers. As mentioned previously, before the rating of the indications, the AUC Task Force and each writing group emphasize the need for harmonization among recommendations for tests and procedures, which is enabled through the construction of guideline/indication mapping tables, references, and careful review during the development process.
Within the ACCF, the Science and Clinical Policy Committee is charged with ensuring communication and collaboration among key documents and standards development groups, such as the task forces and committees for expert consensus documents, clinical practice guidelines, AUC, data standards, and performance measures. An open dialog among members of these groups is necessary to ensure that the ACCF maintains the strengths of each document type while promoting collaboration to ensure consistency and complementary work products. A paradigm of how these documents interrelate has been proposed (16).
Relationships With Industry and Other Entities
Although AUC are not prescriptive, they do define the appropriateness of technology. The ACCF and the AUC Task Force have focused considerable attention on avoiding real or perceived relationships with industry and other entities (RWI) that might impact the rating of a test/procedure for a specific application. In addition to full disclosure of all RWI by all individuals involved with the document, AUC documents are governed by a policy mandating that the writing group chair and fewer than 50% of the rating panel have relevant relationships with industry. The writing group itself is not required to have fewer than 50% without RWI, as it does not participate in the rating of the indications (17).
The task force determined a modification of what constitutes relevant RWI was necessary for the writing group because of the unique role of AUC and the multiple groups of individuals that are involved in the document development process. In addition, the rating panel is balanced for many factors, including clinical and professional bias, as has already been described in the preceding text, further strengthening the range of viewpoints reflected in the AUC.
Implementation and Evaluation
Publication of the AUC is not the final step towards optimal utilization of diagnostic and therapeutic techniques. As demonstrated in Figure 2, the process of AUC construction itself is iterative, building on published evidence, expert consensus, past experience, feedback, and new medical literature and guidelines. Additionally, information garnered from the measurement of appropriateness of actual use in the community provides guidance in the construction of new or revised AUC (9,18).
To ensure optimal utilization of diagnostic and therapeutic techniques, measurement of appropriate use in “real-world” clinical practice based on AUC is necessary to assess practice performance, to provide direction for educational and continuous quality initiatives efforts and to provide valuable real-world data to inform future AUC efforts. A growing number of publications have focused on the evaluation of appropriateness of use of technology, highlighting potential targets for education and utilization improvement (19,20).
Although most of these studies have involved retrospective chart review, it is clear that the evaluation of appropriate use is possible and more actionable when AUC are incorporated as a decision support tool in the evolution of electronic health records and web-based programs. Several studies also are now available regarding prospective application of appropriate use as well whether their utilization promotes improved patient-centered outcomes and/or efficiency within the diagnostic work-up (21–24).
In this light, a hierarchical structure is generally needed so as to clearly and reproducibly assign each case to a specific AUC scenario. To this end, AUC writing groups are now being encouraged to develop standardized ordering sheets and design patient care algorithms to summarize the indications (10–12). The uniform and consistent documentation of AUC in clinical practice will greatly assist in the validation of AUC as well as in the potential development of quality metrics for laboratory or physician standards of appropriate use. To further support this important direction, the ACCF/AHA recently developed a methodology for the creation of a new form of quality metric, termed “appropriate use metrics” that is based heavily on the development of performance measures (25).
Utilization management companies and private health plans have often cited the AUC as justification for their coverage and review policies. However, many of the programs are applied to individual cases without consideration of patterns of care over time for a physician/practice or feedback regarding performance. As such, these programs violate the spirit of AUC that are designed to inform care decisions but are best measured over a patient population rather than rigidly adjudicating care for individual patients.
Clinicians have at times suggested that AUC do not sufficiently reflect the realities of clinical practice and suggested that, as a result, the AUC are not valid. The task force encourages informal feedback and more formal studies to help improve the criteria while supporting clinical judgment needed for individual patient care. Decision making should be guided, but not dictated, by AUC. Some procedures rated as Rarely Appropriate should still occur. Other procedures rated as Appropriate may be reasonable to forgo. Importantly, AUC should be used to assess the overall patient mix undergoing procedures and help physicians tabulate the proportion of patients under their care who fall into each AUC category and clinical indication, and benchmark it to others' data. By using the AUC to better understand practice patterns, physicians can engage in educational discussions with peers and patients about the important clinical variables and patient-specific characteristics that underpin current procedure use. This information can be used to adapt to changes in patient mix over time that can help to optimize the selection of patients for a given procedure based on the benefit and risk thresholds of each physician and patient.
The refinements to AUC methodology, presented in this paper, reflect the responsiveness of the ACCF and its AUC Task Force to queries from the clinician and payer community for a fair, evidence-based, and practical means to guide procedural utilization. Current revisions to the AUC methodology have strengthened the clinical relevance of these documents that are carefully constructed yet continue to undergo evolution to meet the needs of contemporary practice patterns and the developing evidence base within cardiovascular medicine. The focus of AUC is to encourage optimal patient care by the professional stewardship of technology utilization within cardiovascular medicine. The effort aims to join with all cardiovascular practitioners and stakeholders in providing the optimal clinical decision making to foster high quality of cardiovascular care for their patients, and to work toward patterns of care that promote appropriate utilization and minimize use that lacks sufficient value whenever possible. These documents have been welcomed by many in the cardiovascular community, including physicians, patients, and policymakers, and have been incorporated into processes of clinical care, including education, accreditation, and quality improvement programs. Additionally, the AUC now seem to be having an impact on performance of tests in certain patient populations, with the goal of substantial reduction in waste due to unnecessary tests and procedures. The result is that patients may be more likely to benefit from cardiovascular technology using procedures selected for having the potential to favorably impact patient outcomes; however, there are very limited data to support such a claim at this time. Just as pharmacological agents are increasingly targeted toward specific biological actions and patient genetic markers that are meaningful to treatment to produce greater benefit at lower risk, tests and procedures are more effective when matched to patients who have specific clinical markers suggesting higher benefit to risk. Nevertheless, the AUC intent is as a guiding document; the final decision to proceed with testing or a procedure remains at the bedside where patient–physician interaction simply cannot be universally policy-based and must be done in the context of a discussion about treatment and patient goals, which is never a black or white decision.
The refinements to AUC methodology, presented in this paper, reflect the AUC Task Force's commitment to adapt and respond to the ever-evolving needs of cardiovascular practice. Over time, the task force continues to focus on reflecting the evolution of contemporary practice patterns and accruing scientific evidence. The AUC Task Force remains steadfast in its aim to ensure a patient-centered, professional stewardship of technology application within cardiovascular medicine. To this end, the process joins with all potential stakeholders—including patients, cardiovascular specialists, primary care physicians, and other specialists, and regulatory and payer bodies—in support of optimal clinical decision making and ensuring high-quality, cardiovascular care.
American College of Cardiology Foundation
William A. Zoghbi, MD, FACC, President
Thomas E. Arend, Jr, Esq, CAE, Interim Chief Staff Officer
William J. Oetgen, MD, MBA, FACC, Senior Vice President, Science and Quality
Joseph M. Allen, MA, Director, TRIP (Translating Research Into Practice)
Z. Jenissa Haidari, MPH, Senior Research Specialist, Appropriate Use Criteria
Erin A. Barrett, MPS, Senior Specialist, Science and Clinical Policy
Appendix A. Appropriate Use of Cardiovascular Technology: 2013 ACCF Appropriate Use Criteria Methodology Update Writing Group
Robert C. Hendel, MD, FACC, FAHA, FASNC—Chair, 2013 ACCF Appropriate USe Criteria Methodology Update; Director of Cardiac Imaging and Outpatient Services, Division of Cardiology, Miami University School of Medicine, Miami, FL
Manesh R. Patel, MD, FACC—Assistant Professor of Medicine, Division of Cardiology, Duke University Medical Center, Durham, NC
Joseph M. Allen, MA—Director, TRIP (Translating Research Into Practice), American College of Cardiology Foundation, Washington, DC
James K. Min, MD, FACC—Director of Cardiac Imaging Research and Co-Director of Cardiac Imaging, Cedars-Sinai Heart Institute, Los Angeles, CA
Leslee Shaw, PhD, FACC, FASNC—Professor of Medicine, Emory University School of Medicine, Atlanta, GA
Michael J. Wolk, MD, MACC—Chair, Task Force, Past President, American College of Cardiology Foundation and Clinical Professor of Medicine, Weill-Cornell Medical School, New York, NY
Pamela S. Douglas, MD, MACC, FAHA, FASE—Past President, American College of Cardiology Foundation; Past President American Society of Echocardiography; and Ursula Geller Professor of Research in Cardiovascular Diseases, Duke University Medical Center, Durham, NC
Christopher M. Kramer, MD, FACC, FAHA—Professor of Medicine and Radiology, Director, Cardiovascular Imaging Center, University of Virginia Health System, Charlottesville, VA
Raymond F. Stainback, MD, FACC, FASE—Medical Director of Noninvasive Cardiac Imaging, Texas Heart Institute at St. Luke's Episcopal Hospital; Clinical Associate Professor of Medicine, Baylor College of Medicine, Houston, TX
Steven R. Bailey, MD, FACC, FSCAI, FAHA—Chair, Division of Cardiology, Professor of Medicine and Radiology, Janey Briscoe Distinguished Chair, University of Texas Health Sciences Center, San Antonio, TX
John U. Doherty, MD, FACC—Professor of Medicine, Jefferson Medical College of Thomas Jefferson University, Philadelphia, PA
Ralph G. Brindis, MD, MPH, MACC, FSCAI—Clinical Professor of Medicine, Department of Medicine and the Philip R. Lee Institute for Health Policy Studies, University of California, San Francisco, CA
This document was approved by the American College of Cardiology Foundation Board of Trustees in January 2013. The American College of Cardiology Foundation requests this document be cited as follows: Hendel RC, Patel MR, Allen JM, Min JK, Shaw LJ, Wolk MJ, Douglas PS, Kramer CM, Stainback RF, Bailey SR, Doherty JU, Brindis RG. Appropriate Use of Cardiovascular Technology: 2013 ACCF Appropriate Use Criteria Methodology Update. J Am Coll Cardiol 2013;61:1305–17. http://dx.doi.org/10.1016/j.jacc.2013.01.025.
Copies: This document is available on the World Wide Web site of the American College of Cardiology (http://www.cardiosource.org). For copies of this document, please contact Elsevier Inc. Reprint Department, fax (212) 633-3820, e-mail .
Permissions: Multiple copies, modification, alteration, enhancement, and/or distribution of this document are not permitted without the express permission of the American College of Cardiology Foundation. Please contact Elsevier's permission department at.
- American College of Cardiology Foundation
- Hendel R.C.
- Dartmouth Medical School Center for the Evaluation of Clinical Sciences, Maine Medical Center
- Douglas P.,
- Iskandrian A.E.,
- Krumholz H.M.,
- et al.
- Brindis R.G.,
- Douglas P.S.,
- Hendel R.C.,
- et al.
- Patel M.R.,
- Spertus J.A.,
- Brindis R.G.,
- et al.
- Fitch K.
- Hendel R.C.,
- Berman D.S.,
- Di Carli M.F.,
- et al.
- Douglas P.S.,
- Garcia M.J.,
- Haines D.E.,
- et al.
- Taylor A.J.,
- Cerqueira M.,
- Hodgson J.M.,
- et al.
- Fleisher L.A.,
- Beckman J.A.,
- Brown K.A.,
- et al.
- Patel M.R.,
- Dehmer G.J.,
- Hirshfeld J.W.,
- et al.
- Antman E.M.,
- Peterson E.D.
- ↵ACCF Relationship With Industry and Other Entities: Policies and Procedures for the Development of Guidelines and Other Clinical Documents. 2010. http://www.cardiosource.org/Science-And-Quality/Practice-Guidelines-and-Quality-Standards/Relationships-With-Industry-Policy.aspx. Accessed December 17, 2012.
- Hendel R.C.,
- Cerqueira M.,
- Douglas P.S.,
- et al.
- Ward R.P.,
- Krauss D.,
- Mansour I.N.,
- et al.
- Allen J.M.
- Chinnaiyan K.M.,
- Peyser P.,
- Goraya T.,
- et al.
- Barnes E.
- Bonow R.O.,
- Douglas P.S.,
- Buxton A.E.,
- et al.
- Defining Appropriate Use
- Appropriate Use Criteria Development
- Multimodality Appropriate Use Criteria
- Relationship With Clinical Practice Guidelines and Other Clinical Policy
- Relationships With Industry and Other Entities
- Implementation and Evaluation
- Appendix A. Appropriate Use of Cardiovascular Technology: 2013 ACCF Appropriate Use Criteria Methodology Update Writing Group
- Appendix B. Appropriate Use of Cardiovascular Technology: 2013 ACCF Appropriate Use Criteria Methodology Update—Relationships With Industry and Other Entities (Relevant)