ACCF/ASE/AHA/ASNC/HFSA/HRS/SCAI/SCCM/SCCT/SCMR 2011 Appropriate Use Criteria for EchocardiographyA Report of the American College of Cardiology Foundation Appropriate Use Criteria Task Force, American Society of Echocardiography, American Heart Association, American Society of Nuclear Cardiology, Heart Failure Society of America, Heart Rhythm Society, Society for Cardiovascular Angiography and Interventions, Society of Critical Care Medicine, Society of Cardiovascular Computed Tomography, and Society for Cardiovascular Magnetic Resonance Endorsed by the American College of Chest Physicians
This article has a correction. Please see:
Author + information
- Published online March 1, 2011.
Author Information
- Pamela S. Douglas, MD, MACC, FAHA, FASE, Chair, Echocardiography Writing Group⁎,
- Mario J. Garcia, MD, FACC, FACP, Echocardiography Writing Group†,
- David E. Haines, MD, FACC, FHRS, Echocardiography Writing Group‡,
- Wyman W. Lai, MD, MPH, FACC, FASE, Echocardiography Writing Group§,
- Warren J. Manning, MD, FACC, Echocardiography Writing Group∥,¶,
- Ayan R. Patel, MD, FACC, Echocardiography Writing Group#,
- Michael H. Picard, MD, FACC, FASE, FAHA, Echocardiography Writing Group§,
- Donna M. Polk, MD, MPH, FACC, FASE, FASNC, Echocardiography Writing Group⁎⁎,
- Michael Ragosta, MD, FACC, FSCAI, Echocardiography Writing Group††,
- R. Parker Ward, MD, FACC, FASE, FASNC, Echocardiography Writing Group§ and
- Rory B. Weiner, MD, Echocardiography Writing Group⁎
- ACCF Appropriate Use Criteria
- cardiac imaging
- coronary artery disease
- diagnostic testing
- echocardiography
Technical Panel
Steven R. Bailey, MD, FACC, FSCAI, FAHA, Moderator
Rory B. Weiner, MD, Writing Group Liaison
Peter Alagona, Jr, MD, FACC⁎
Jeffrey L. Anderson, MD, FACC, FAHA, MACP⁎∥
Jeanne M. DeCara, MD, FACC, FASE§
Rowena J. Dolor, MD, MHS
Reza Fazel, MD, FACC⁎⁎
John A. Gillespie, MD, FACC‡‡
Paul A. Heidenreich, MD, FACC#
Luci K. Leykum, MD, MBA, MSc
Joseph E. Marine, MD, FACC, FHRS‡
Gregory J. Mishkel, MD, FACC, FSCAI, FRCPC††
Patricia A. Pellikka, MD, FACC, FAHA, FACP, FASE§
Gilbert L. Raff, MD, FACC, FSCCT†
Krishnaswami Vijayaraghavan, MD, FACC, FCCP§§
Neil J. Weissman, MD, FACC, FAHA⁎
Katherine C. Wu, MD¶
Appropriate Use Criteria Task Force
Michael J. Wolk, MD, MACC, Chair
Steven R. Bailey, MD, FACC, FSCAI, FAHA
Pamela S. Douglas, MD, MACC, FAHA, FASE
Robert C. Hendel, MD, FACC, FAHA, FASNC
Christopher M. Kramer, MD, FACC, FAHA
James K. Min, MD, FACC
Manesh R. Patel, MD, FACC
Leslee Shaw, PhD, FACC, FASNC
Raymond F. Stainback, MD, FACC, FASE
Joseph M. Allen, MA
Table of Contents
Abstract…...1128
Preface…...1128
1. Introduction…...1129
2. Methods…...1129
3. General Assumptions…...1130
4. Definitions…...1131
5. Abbreviations…...1132
6. Results of Ratings…...1132
7. Echocardiography Appropriate Use Criteria (by Indication)…...1133
Table 1. TTE for General Evaluation of Cardiac Structure and Function…...1133
Table 2: TTE for Cardiovascular Evaluation in an Acute Setting…...1133
Table 3: TTE for Evaluation of Valvular Function…...1134
Table 4: TTE for Evaluation of Intracardiac and Extracardiac Structures and Chambers…...1135
Table 5: TTE for Evaluation of Aortic Disease…...1135
Table 6: TTE for Evaluation of Hypertension, HF, or Cardiomyopathy…...1135
Table 7: TTE for Adult Congenital Heart Disease…...1136
Table 8: TEE…...1137
Table 9: Stress Echocardiography for Detection of CAD/Risk Assessment: Symptomatic or Ischemic Equivalent…...1137
Table 10: Stress Echocardiography for Detection of CAD/Risk Assessment: Asymptomatic (Without Ischemic Equivalent)…...1138
Table 11: Stress Echocardiography for Detection of CAD/Risk Assessment: Asymptomatic (Without Ischemic Equivalent) in Patient Populations With Defined Comorbidities…...1138
Table 12: Stress Echocardiography Following Prior Test Results…...1139
Table 13: Stress Echocardiography for Risk Assessment: Perioperative Evaluation for Noncardiac Surgery Without Active Cardiac Conditions…...1140
Table 14: Stress Echocardiography for Risk Assessment: Within 3 Months of an ACS…...1140
Table 15: Stress Echocardiography for Risk Assessment: Postrevascularization (PCI or CABG)…...1140
Table 16: Stress Echocardiography for Assessment of Viability/Ischemia…...1141
Table 17: Stress Echocardiography for Hemodynamics (Includes Doppler During Stress)…...1141
Table 18: Contrast Use in TTE/TEE or Stress Echocardiography…...1141
8. Echocardiography Appropriate Use Criteria (by Appropriate Use Rating)…...1142
9. Discussion…...1154
Appendix A: Additional Echocardiography Definitions…...1158
Appendix B: Additional Methods…...1160
Relationships With Industry and Other Entities…...1160
Literature Review…...1160
Appendix C: ACCF/ASE/AHA/ASNC/HFSA/HRS/SCAI/SCCM/SCCT/SCMR 2011 Appropriate Use Criteria for Echocardiography Participants…...1160
Appendix D: ACCF/ASE/AHA/ASNC/HFSA/HRS/SCAI/SCCM/SCCT/SCMR 2011 Appropriate Use Criteria for Echocardiography Writing Group, Technical Panel, Indication Reviewers, and Task Force—Relationships With Industry and Other Entities (in Alphabetical Order Within Each Group)…...1163
References…...1164
TTE for General Evaluation of Cardiac Structure and Function
TTE for Cardiovascular Evaluation in an Acute Setting
TTE for Evaluation of Valvular Function
TTE for Evaluation of Intracardiac and Extracardiac Structures and Chambers
TTE for Evaluation of Aortic Disease
TTE for Evaluation of Hypertension, HF, or Cardiomyopathy
TTE for Adult Congenital Heart Disease
TEE
Stress Echocardiography for Detection of CAD/Risk Assessment: Symptomatic or Ischemic Equivalent
Stress Echocardiography for Detection of CAD/Risk Assessment: Asymptomatic (Without Ischemic Equivalent)
Stress Echocardiography for Detection of CAD/Risk Assessment: Asymptomatic (Without Ischemic Equivalent) in Patient Populations With Defined Comorbidities
Stress Echocardiography Following Prior Test Results
Stress Echocardiography for Risk Assessment: Perioperative Evaluation for Noncardiac Surgery Without Active Cardiac Conditions
Stress Echocardiography for Risk Assessment: Within 3 Months of an ACS
Stress Echocardiography for Risk Assessment: Postrevascularization (PCI or CABG)
Stress Echocardiography for Assessment of Viability/Ischemia
Stress Echocardiography for Hemodynamics (Includes Doppler During Stress)
Contrast Use in TTE/TEE or Stress Echocardiography
Appropriate Indications (Median Score 7–9)
Uncertain Indications (Median Score 4–6)
Inappropriate Indications (Median Score 1–3)
Abstract
The American College of Cardiology Foundation (ACCF), in partnership with the American Society of Echocardiography (ASE) and along with key specialty and subspecialty societies, conducted a review of common clinical scenarios where echocardiography is frequently considered. This document combines and updates the original transthoracic and transesophageal echocardiography appropriateness criteria published in 2007 (1) and the original stress echocardiography appropriateness criteria published in 2008 (2). This revision reflects new clinical data, reflects changes in test utilization patterns, and clarifies echocardiography use where omissions or lack of clarity existed in the original criteria.
The indications (clinical scenarios) were derived from common applications or anticipated uses, as well as from current clinical practice guidelines and results of studies examining the implementation of the original appropriate use criteria (AUC). The 202 indications in this document were developed by a diverse writing group and scored by a separate independent technical panel on a scale of 1 to 9, to designate appropriate use (median 7 to 9), uncertain use (median 4 to 6), and inappropriate use (median 1 to 3).
Ninety-seven indications were rated as appropriate, 34 were rated as uncertain, and 71 were rated as inappropriate. In general, the use of echocardiography for initial diagnosis when there is a change in clinical status or when the results of the echocardiogram are anticipated to change patient management were rated appropriate. Routine testing when there was no change in clinical status or when results of testing were unlikely to modify management were more likely to be inappropriate than appropriate/uncertain.
The AUC for echocardiography have the potential to impact physician decision making, healthcare delivery, and reimbursement policy. Furthermore, recognition of uncertain clinical scenarios facilitates identification of areas that would benefit from future research.
Preface
In an effort to respond to the need for the rational use of imaging services in the delivery of high-quality care, the ACCF has undertaken a process to determine the appropriate use of cardiovascular imaging for selected patient indications.
AUC publications reflect an ongoing effort by the ACCF to critically and systematically create, review, and categorize clinical situations where diagnostic tests and procedures are utilized by physicians caring for patients with cardiovascular diseases. The process is based on current understanding of the technical capabilities of the imaging modalities examined. Although impossible to be entirely comprehensive given the wide diversity of clinical disease, the indications are meant to identify common scenarios encompassing the majority of situations encountered in contemporary practice. Given the breadth of information they convey, the indications do not directly correspond to the Ninth Revision of the International Classification of Diseases system as these codes do not include clinical information, such as symptom status.
The ACCF believes that careful blending of a broad range of clinical experiences and available evidence-based information will help guide a more efficient and equitable allocation of healthcare resources in cardiovascular imaging. The ultimate objective of AUC is to improve patient care and health outcomes in a cost-effective manner, but it is not intended to ignore ambiguity and nuance intrinsic to clinical decision making. AUC thus should not be considered substitutes for sound clinical judgment and practice experience.
The ACCF AUC process itself is also evolving. In the current iteration, technical panel members were asked to rate indications for echocardiography in a manner independent and irrespective of the prior published ACCF ratings for transthoracic echocardiography (TTE) and transesophageal echocardiography (TEE) (1) and stress echocardiography (2) as well as the prior ACCF ratings for diagnostic imaging modalities such as cardiac radionuclide imaging (RNI) (3) and cardiac computed tomography (CT) (4). Given the iterative and evolving nature of the process, readers are counseled that comparison of individual appropriate use ratings among modalities rated at different times over the past several years may not reflect the comparative utility of the different modalities for an indication, as the ratings may vary over time. A comparative evaluation of the appropriate use of multiple imaging techniques is currently being undertaken to assess the relative strengths of each modality for various clinical scenarios.
We are grateful to the technical panel and its chair, Steven Bailey, MD, FACC, FSCAI, FAHA, a professional group with a wide range of skills and insights, for their thoughtful and thorough deliberation of the merits of echocardiography for various indications. We would also like to thank the 27 individuals who provided a careful review of the draft of indications, the parent AUC Task Force ably led by Michael Wolk, MD, MACC, Rory Weiner, MD, and the ACC staff, John C. Lewin, MD, Joseph Allen, Starr Webb, Jenissa Haidari, and Lea Binder for their exceptionally skilled support in the generation of this document.
Pamela S. Douglas, MD, MACC, FAHA, FASE Chair, Echocardiography Writing Group
Michael J. Wolk, MD, MACC Chair, Appropriate Use Criteria Task Force
1 Introduction
This report addresses the appropriate use of TTE, TEE, and stress echocardiography. Improvements in cardiovascular imaging technology and an expanding armamentarium of noninvasive diagnostic tools and therapeutic options for cardiovascular disease have led to an increase in cardiovascular imaging. As the field of echocardiography continues to advance along with other imaging modalities and treatment options, the healthcare community needs to understand how to best incorporate this technology into daily clinical care.
All prior AUC publications from the ACCF and collaborating organizations reflect an ongoing effort to critically and systematically create, review, and categorize the appropriate use of cardiovascular procedures and diagnostic tests. The ACCF recognizes the importance of revising these criteria in a timely manner in order to provide the cardiovascular community with the most accurate indications. Understanding the background and scope of this document are important before interpreting the rating tables.
This document presents a combination and revision of the 2007 ACCF AUC for Transthoracic and Transesophageal Echocardiography (1) and the 2008 ACCF AUC for Stress Echocardiography (2). Appropriate echocardiograms are those that are likely to contribute to improving patients' clinical outcomes, and importantly, inappropriate use of echocardiography may be potentially harmful to patients and generate unwarranted costs to the healthcare system.
2 Methods
The indications included in this publication cover a wide array of cardiovascular signs and symptoms as well as clinical judgments as to the likelihood of cardiovascular findings. Within each main disease category, a standardized approach was used to capture the majority of clinical scenarios without making the list of indications excessive. The approach was to create 5 broad clinical scenarios regarding the possible use of echocardiography: 1) for initial diagnosis; 2) to guide therapy or management, regardless of symptom status; 3) to evaluate a change in clinical status or cardiac exam; 4) for early follow-up without change in clinical status; and 5) for late follow-up without change in clinical status. Certain specific clinical scenarios were addressed with additional focused indications.
The indications were constructed by experts in echocardiography and in other fields and were modified on the basis of discussions among the task force and feedback from independent reviewers and the technical panel. Wherever possible, indications were mapped to relevant clinical guidelines and key publications/references (Online Appendix).
An important focus during the indication revision process was to harmonize the indications across noninvasive modalities, such that the wording of the indications are similar with other AUC (3) whenever it was feasible to do so. New indications as well as indication tables were created, although it remains likely that several clinical scenarios are not covered by these revised AUC for echocardiography. Once the revised indications were written, they were reviewed and critiqued by the parent AUC Task Force and by 27 external reviewers representing all cardiovascular specialties and primary care before being finalized.
A detailed description of the methods used for ranking the selected clinical indications is found in a previous publication, “ACCF Proposed Method for Evaluating the Appropriateness of Cardiovascular Imaging” (5). Briefly, this process combines evidence-based medicine and practice experience by engaging a technical panel in a modified Delphi exercise. Since the original TTE/TEE (1) and stress echocardiography (2) documents and methods paper (5) were published, several important processes have been put in place to further enhance the rigor of this process. They include convening a formal writing group with diverse expertise in imaging and clinical care, circulating the indications for external review prior to rating by the technical panel, ensuring appropriate balance of expertise and practice area of the technical panel, development of a standardized rating package, and establishment of formal roles for facilitating panel interaction at the face-to-face meeting.
The technical panel first rated indications independently. Then, the panel was convened for a face-to-face meeting for discussion of each indication. At this meeting, panel members were provided with their scores and a blinded summary of their peers' scores. After the meeting, panel members were then asked to independently provide their final scores for each indication.
Although panel members were not provided explicit cost information to help determine their appropriate use ratings, they were asked to implicitly consider cost as an additional factor in their evaluation of appropriate use. In rating these criteria, the technical panel was asked to assess whether the use of the test for each indication is appropriate, uncertain, or inappropriate, and was provided the following definition of appropriate use:
An appropriate imaging study is one in which the expected incremental information, combined with clinical judgment, exceeds the expected negative consequences⁎by a sufficiently wide margin for a specific indication that the procedure is generally considered acceptable care and a reasonable approach for the indication.
The technical panel scored each indication as follows:
Median Score 7 to 9
Appropriate test for specific indication (test is generally acceptable and is a reasonable approach for the indication).
Median Score 4 to 6
Uncertain for specific indication (test may be generally acceptable and may be a reasonable approach for the indication). Uncertainty also implies that more research and/or patient information is needed to classify the indication definitively.
Median Score 1 to 3
Inappropriate test for that indication (test is not generally acceptable and is not a reasonable approach for the indication).
The division of these scores into 3 levels of appropriateness is somewhat arbitrary, and the numeric designations should be viewed as a continuum. Further, there is diversity in clinical opinion for particular clinical scenarios, such that scores in the intermediate level of appropriate use should be labeled “uncertain,” as critical patient or research data may be lacking or discordant. This designation should be a prompt to the field to carry out definitive research investigations whenever possible. It is anticipated that the AUC reports will continue to be revised as further data are generated and information from the implementation of the criteria is accumulated.
To prevent bias in the scoring process, the technical panel was deliberately comprised of a minority of specialists in echocardiography. Specialists, although offering important clinical and technical insights, might have a natural tendency to rate the indications within their specialty as more appropriate than nonspecialists. In addition, care was taken in providing objective, nonbiased information, including guidelines and key references, to the technical panel.
The level of agreement among panelists as defined by RAND (6) was analyzed based on the BIOMED rule for a panel of 14 to 16 members. As such, agreement was defined as an indication where 4 or fewer panelists' ratings fell outside the 3-point region containing the median score.
Disagreement was defined as where at least 5 panelists' ratings fell in both the appropriate and the inappropriate categories. Any indication having disagreement was categorized as uncertain regardless of the final median score. Indications that met neither definition for agreement or disagreement are in a third, unlabeled category.
3 General Assumptions
To prevent any inconsistencies in interpretation, specific assumptions were considered by the writing group in developing the indications and by the technical panel when rating the clinical indications for the appropriate use of inpatient and outpatient adult TTE/TEE and stress echocardiography.
1. A TTE and a TEE examination and report will include the use and interpretation of 2-dimensional/M-mode imaging, color flow Doppler, and spectral Doppler as important elements of a comprehensive TTE/TEE (7–9) evaluating relevant cardiac structures and hemodynamics. Stress echocardiography will include rest and stress 2-dimensional imaging at a minimum unless performed for hemodynamics, when Doppler must be included (10).
2. All standard echocardiographic techniques for image acquisition, including standard rest imaging and stress protocols (10), are available for each indication and have a sensitivity and specificity similar to those found in the published literature. Selection for and monitoring of contrast use is assumed to be in accord with practice guidelines (11).
3. The test is performed and interpreted by qualified individual(s) in a facility that is proficient in the echocardiographic technique (12,13).
4. The range of potential indications for echocardiography is quite large, particularly in comparison with other cardiovascular imaging tests. Thus, the indications are, at times, purposefully broad to cover an array of cardiovascular signs and symptoms as well as the ordering physician's best judgment as to the presence of cardiovascular abnormalities. Additionally, there are likely clinical scenarios that are not covered in this document.
5. A complete clinical history and physical exam has been completed by a qualified clinician such that the clinical status of the patient can be assumed to be valid as stated in the indication (e.g., an asymptomatic patient is truly asymptomatic for the condition in question and that sufficient questioning of the patient has been undertaken).
6. If the reason for a test can be assigned to more than 1 indication, it should be classified under the most appropriate indication.
7. Cost should be considered implicitly in the appropriate use determination.
8. For each indication, the rating should reflect whether the echocardiogram is reasonable for the patient according to the appropriate use definition, not whether the test is preferred over another modality. It should not be assumed that for each indication the decision to perform a diagnostic test has already been made. It also should not consider issues of local availability or skill for any modality or attempt in any way to compare 2 tests with each other.
9. The category of “uncertain” should be used when insufficient clinical data are available for a definitive categorization or there is substantial disagreement regarding the appropriateness of that indication. The designation of “uncertain” should not be used as grounds for denial of reimbursement.
10. Indications that describe routine or surveillance echocardiograms imply that the test is being considered for a “periodic” evaluation since a certain period of time has elapsed. The test is not being ordered due to the anticipation of changing clinical decision making or guiding therapy.
11. Prosthetic valves and native valves are to be considered together, except where specifically mentioned otherwise in this document. The severity of valve stenosis or regurgitation is defined in clinical guidelines (14,15).
12. In general, it is assumed that TEE is most appropriately used as an adjunct or subsequent test to TTE when indicated, such as when suboptimal TTE images preclude obtaining a diagnostic study. The indications for which TEE may reasonably be the test of first choice include, but are not limited to, the indications presented in Table 8 of this document.
13. Intraoperative TEE is an important use of cardiovascular ultrasound. However, this application is outside the scope of this document and thus is not addressed here.
14. For all stress imaging, the mode of stress testing is assumed to be exercise (e.g., treadmill, bicycle) for patients able to exercise. For patients unable to exercise, it is assumed that dobutamine is used for echocardiographic stress testing. Any indications requiring a specific mode of stress (e.g., when hemodynamic information is required) are labeled as such.
15. Doppler hemodynamic assessment during stress echocardiography includes both right and left heart hemodynamics (e.g., valvular gradients, pulmonary artery pressure, mitral regurgitation severity).
16. The indications for the perioperative evaluation for noncardiac surgery were modeled after the ACCF/AHA guidelines on perioperative cardiovascular evaluation and care for noncardiac surgery (16). If a patient has signs/symptoms of suspected cardiac etiology, the clinical scenario should be considered in the symptomatic category (e.g., Indication 1) and not in the perioperative section.
17. As with other surgeries, the need for coronary artery disease (CAD) assessment prior to solid organ transplantation is related to patient and surgical risk. In general, solid organ transplantation should be considered in the vascular surgery category given that CAD is common in patients with diabetes mellitus who have end-stage renal disease.
4 Definitions
Definitions of terms used throughout the indication set are listed here. Additional definitions are listed in Appendix A. These definitions were provided to and discussed with the technical panel prior to ratings of indications.
1. Ischemic Equivalent: Chest Pain Syndrome, Anginal Equivalent, or Ischemic Electrocardiographic Abnormalities: Any constellation of clinical findings that the physician feels is consistent with CAD. Examples of such findings include, but are not limited to, chest pain, chest tightness, chest burning, shoulder pain, palpitations, jaw pain, new electrocardiographic abnormalities, or other symptoms/findings suggestive of CAD. Nonchest pain symptoms (e.g., dyspnea or reduced/worsening effort tolerance) that are thought to be consistent with CAD may also be considered to be an ischemic equivalent.
2. Global CAD Risk: It is assumed that clinicians will use current standard methods of global risk assessment such as those presented in the National Heart, Lung, and Blood Institute report on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III [ATP III]) (18) or similar national guidelines.
Absolute risk is defined as the probability of developing CAD over a given time period. The ATP III report specifies absolute risk for CAD over the next 10 years. CAD risk refers to 10-year risk for any hard cardiac event (e.g., myocardial infarction or CAD death). However, acknowledging that global absolute risk scores may be miscalibrated in certain populations (e.g., women, younger men), clinical judgment must be applied in assigning categorical risk thresholds in such subpopulations.
• Low global CAD risk
Defined by the age-specific risk level that is below average. In general, low risk will correlate with a 10-year absolute CAD risk <10%. However, in women and younger men, low risk may correlate with 10-year absolute CAD risk <6%.
• Intermediate global CAD risk
Defined by the age-specific risk level that is average. In general, moderate risk will correlate with a 10-year absolute CAD risk range of 10% to 20%. Among women and younger age men, an expanded intermediate risk range of 6% to 20% may be appropriate.
• High global CAD risk
Defined by the age-specific risk level that is above average. In general, high risk will correlate with a 10-year absolute CAD risk of >20%. CAD equivalents (e.g., diabetes mellitus, peripheral arterial disease) can also define high risk.
3. Pretest Probability of CAD: Symptomatic (Ischemic Equivalent) Patients: Once the physician determines that symptoms are present that may represent CAD, the pretest probability of CAD should be assessed. There are a number of risk algorithms (19,20) available that can be used to calculate this probability. Clinicians should be familiar with those algorithms that pertain to the populations they encounter most often. In scoring the indications, the following probabilities, as calculated from any of the various available validated algorithms, should be applied.
• Very low pretest probability: <5% pretest probability of CAD
• Low pretest probability: Between 5% and 10% pretest probability of CAD
• Intermediate pretest probability: Between 10% and 90% pretest probability of CAD
• High pretest probability: >90% pretest probability of CAD
The method recommended by the ACC/AHA guidelines for chronic stable angina (21) is provided as one example of a method used to calculate pretest probability and is a modification of a previously published literature review (22). Please refer to Table A and the definition of angina in Appendix A. It is important to note that other historical factors or electrocardiographic findings (e.g., prior infarction) can affect pretest probability, although these factors are not accounted for in Table A. Similarly, although not incorporated into the algorithm, other CAD risk factors may also affect pretest likelihood of CAD. Detailed nomograms are available that incorporate the effects of a history of prior infarction, electrocardiographic Q waves and ST- and T-wave changes, diabetes, smoking, and hypercholesterolemia (23).
Pretest Probability of CAD by Age, Gender, and Symptoms⁎
5 Abbreviations
ACS = acute coronary syndrome
APC = atrial premature contraction
CABG = coronary artery bypass grafting surgery
CAD = coronary artery disease
CMR = cardiovascular magnetic resonance
CRT = cardiac resynchronization therapy
CT = computed tomography
ECG = electrocardiogram
HF = heart failure
ICD = implantable cardioverter-defibrillator
LBBB = left bundle-branch block
LV = left ventricular
MET = estimated metabolic equivalents of exercise
MI = myocardial infarction
PCI = percutaneous coronary intervention
RNI = radionuclide imaging
SPECT MPI = single-photon emission computed tomography myocardial perfusion imaging
STEMI = ST-segment elevation myocardial infarction
SVT = supraventricular tachycardia
TEE = transesophageal echocardiogram
TIA = transient ischemic attack
TIMI = Thrombolysis In Myocardial Infarction
TTE = transthoracic echocardiogram
UA/NSTEMI = unstable angina/non–ST-segment elevation myocardial infarction
VPC = ventricular premature contraction
VT = ventricular tachycardia
6 Results of Ratings
The final ratings for echocardiography are listed by indication in Tables 1 to 18. The final score reflects the median score of the 15 technical panel members and has been labeled according to the 3 appropriate use categories of appropriate (median 7 to 9), uncertain (median 4 to 6), and inappropriate (median 1 to 3). Tables 19 to 21 present the indications by the appropriate use categories.
There was less variation in ratings for the indications labeled as either appropriate or inappropriate, with 92% and 90%, respectively, showing agreement as defined in Methods Section 2. There was greater variability (less agreement) in the rating scores for indications defined as uncertain, with 21% showing agreement as defined previously. Two indications, 182 and 189, were distributed into each extreme such that the panel was classified as being in disagreement. However, the median scores for these indications were already placed in the uncertain category, so no changes were required to reflect disagreement. Across all categories, 40 indications did not meet the definition of agreement; however, the scores were not so divergent (as defined by disagreement) as to necessitate a change in the final score.
7 Echocardiography Appropriate Use Criteria (by Indication)
8 Echocardiography Appropriate Use Criteria (by Appropriate Use Rating)
Visual representations (flow diagrams) for all indications are included in the Online Appendix.
Selected flow diagrams for several categories of indications are included here (Figs. 1 to 6).⇓⇓⇓⇓⇓⇓
Stress Echocardiography for Detection of CAD/Risk Assessment: Symptomatic or Ischemic Equivalent
Stress Echocardiography for Detection of CAD/Risk Assessment: Asymptomatic (Without Ischemic Equivalent)
Stress Echocardiography Following Prior Treadmill ECG, Coronary Calcium Scoring, or Carotid Intimal Medial Thickness Test Results
Stress Echocardiography Following Prior Stress Imaging or Coronary Angiogram Test Results
Stress Echocardiography for Risk Assessment—Perioperative Evaluation for Noncardiac Surgery Without Active Cardiac Conditions
Stress Echocardiography for Risk Assessment—Postrevascularization (PCI or CABG)
9 Discussion
Appropriate use criteria define patient subgroups where the available medical evidence supplemented by expert opinion are combined to assess whether the net benefit or risks of a test or procedure make it reasonable to perform testing (in this document, echocardiography) in a particular clinical situation. The intent of these criteria is to guide the rational use of a procedure, namely avoidance of either under- or over-utilization, and thereby lead to improved outcomes, more optimal healthcare delivery, and justifiable healthcare expenditures.
This document is a revision and combination of the original AUC for transthoracic and transesophageal echocardiography (1) and stress echocardiography (2). The revision adds insight provided by interim clinical data and standards documents recently published in the literature and clarifies areas in which omissions or lack of clarity existed in the original criteria. Additionally, since publication of the original AUC, several studies have assessed the application of these criteria in clinical practice; results from these studies were incorporated into this revision and will be briefly summarized here.
Implementation Studies
Application of the 2007 AUC for TTE has been evaluated at academic medical centers (22,24–26), in Veterans Affairs (VA) hospitals (27), and in community settings (28,29). Several common themes deserve emphasis. First, the majority of clinical scenarios for which TTEs were ordered were captured by AUC indications (11% to 16% of TTEs were unclassified) (24,27). Second, across the implementation studies, there are remarkably similar rates of appropriate and inappropriate use of TTE. Among those TTEs with an indication addressed by the AUC (thus removing unclassifiable patients), the majority were rated as appropriate (87% to 91%) and the rate of inappropriate TTEs was consistently low (9% to 13%) (24–27). In 1 study of outpatient TTEs (29), the rate of appropriate TTEs was lower (74%), although this may be attributable to a higher proportion of unclassified studies in the outpatient setting, a pattern that has been observed by others (24,26). The presence of a greater proportion of unclassified TTEs in the outpatient setting might be expected given that many of the indications in the original AUC (1) specifically address symptoms or a “change in clinical status.”
The most common appropriate indications for TTE included initial evaluation of symptoms potentially caused by suspected cardiac etiology, prior testing concerning for heart disease, evaluation of valvular disease, and evaluation of a heart failure indication (24) and are repeated in this revision as Indications 1, 2, 34, and 70. Recommendations for expanding the AUC related to addressing 1) perioperative evaluation (Indications 13 and 14); 2) timing of follow up for valvular heart disease (Indications 38 to 41 and 43 to 49); 3) assessment for device therapy (Indications 76 to 83); and 4) use in some specialized care or “niche” programs (e.g., solid organ transplantation) (Indications 14, 84, and 85), and these scenarios were included in the current document. Finally, more indications reflecting outpatient clinical scenarios (e.g., no change in clinical status) were added.
Studies evaluating the application of AUC for TEE had similar results, with the vast majority of classifiable TEEs being ordered for appropriate indications (94% to 97%) and a smaller number not being classified by the AUC (6% to 9%) (30–32). The fact that the operator is more intimately involved in the decision to perform TEE may help to explain the higher appropriate use rate of TEE compared with TTE. The most common indication for an initial TEE was to guide anticoagulation decisions in patients with atrial fibrillation or flutter (Indications 112 and 113) (30,31). Recommendations for revision focused on refinement of the indications for evaluation of cardiovascular source of embolus (Indications 109 to 111).
Fewer studies have focused on the clinical application of AUC for stress echocardiography (33,34). In 1 study, 19% of stress echocardiograms could not be classified by the AUC (33). Of the echocardiograms that were classified, 66% were for appropriate indications. The majority of unclassified studies were centered in 2 areas: perioperative risk assessment and risk assessment with prior test results. In another study, 88% (n=253) of stress echocardiograms were ordered for indications outlined in the AUC, whereas 12% (n=36) were ordered for indications not addressed by the AUC (34). Of the 253 studies for which the AUC document could be applied, 71% (n=180) studies were appropriate, 9% (n=23) were uncertain, and 20% (n=50) were inappropriate studies.
The results of the implementation studies demonstrate that the rate of inappropriate use of echocardiography is similar in various regions of the United States. In contrast, other studies of resource utilization have documented regional differences in utilization patterns (35). A recent study (36) suggests that a substantial amount of the observed geographic variability in use is attributable to corresponding regional differences in patient health, a conclusion supported by the AUC implementation data which, unlike claims data, inherently address clinical status. Further application of AUC may help to dissect the true variations in care delivery by supplementing claims data with clinical data; however, this warrants further study.
In summary, studies evaluating clinical application of AUC for echocardiography suggest that the majority of clinical scenarios could be classified by the criteria and that the majority of studies were ordered for appropriate indications. Further, the studies identified gaps in the AUC, likely due to both omissions in the initial criteria and subsequent advances in specialized care, which were of substantial utility in guiding the revision process. Although improved, we do not expect this AUC document to be all-inclusive of the wide breadth of all possible clinical scenarios. Although the results from the implementation studies indicate that the original AUC for echocardiography were successful, they also support the need for the current update and revision of the criteria.
Other Features of the Revision
In addition to incorporating the results from implementation studies, several other aspects of the revision deserve emphasis. First, the revised document combines TTE, TEE, and stress echocardiography, whereas the initial TTE and TEE AUC (1) were published separately from the stress echocardiography AUC (2). The indication tables still focus on each modality separately, for example, TTE (or TEE as an adjunct if TTE nondiagnostic), TEE as an initial test, and stress echocardiography. The exception is the final table (Table 18, Indications 201 and 202), which covers contrast use and is applicable to all of the echocardiographic modalities. Second, a new table was created to cover indications related to patients with adult congenital heart disease, as this patient population is being encountered with greater frequency by adult cardiologists (Table 7, Indications 92 to 98) (37). It should be noted that, with the exception of some adults with ligated or occluded patent ductus arteriosus (covered in Indications 95 and 96), most congenital heart conditions have the potential for residual anatomic or physiologic abnormalities, so that, even for many asymptomatic and stable patients, an echocardiogram will be considered to guide therapeutic decision making rather than for routine surveillance. Third, existing tables were expanded to be more comprehensive in covering various clinical situations. Fourth, efforts were made to address clinical scenarios that have recently been addressed in revised or new practice guidelines, such as valvular heart disease (14), perioperative evaluation (16), and evaluation of thoracic aortic disease (38). The goal of relating indications to the available evidence base was a consistent feature during the revision process (see Online Appendix). If randomized trials or practice guidelines relevant to indications were not available, clinical scenarios addressed in expert consensus documents were identified whenever possible. Finally, indications were added to better address evolving therapeutic options such as CRT (Indications 76 to 78) or treatment/follow-up of pulmonary hypertension (Indications 15 to 18).
An important focus during the revision process was to harmonize the indications across noninvasive modalities, such that the wording of the indications is identical with other AUC criteria (3) whenever feasible. For echocardiography, harmonization with other documents was most relevant for the stress echocardiography portion. For instance, Table 13, which addresses the perioperative assessment for noncardiac surgery, mirrors Table 4 in the RNI document (3). This should facilitate clinical application of the criteria and assist the process of future revisions and possibly the development of a multimodality imaging AUC document.
Stress echocardiography tests, like many imaging tests, may provide additional useful information beyond the primary purpose outlined by the indication. In addition, stress echocardiography does not use ionizing radiation. However, the AUC for stress echocardiography were not developed to quantify the incremental information or other test characteristics beyond addressing the diagnostic need inherent in an individual indication.
In ranking indications, panelists were asked to not consider comparisons to other imaging procedures while completing their rankings. Nevertheless, stress echocardiography and SPECT MPI have similar bodies of evidence to support their use. Therefore, it is not surprising that the overwhelming majority of final ratings of stress echocardiography and stress RNI were concordant for similar clinical indications. However, a small number of the final scores and rating categories reported in this document differ from those previously published for stress RNI (3). Specifically, 4 indications (Indications 127, 157, 171, and 172) were rated differently. It is noteworthy that of these 4 indications, 3 also appeared in the first stress echocardiography AUC (2), and all 3 indications were rated similarly in this revision, requiring consistency in ratings across the 2 technical panels composed of different individuals. The difference in the rating for Indication 127 may have been directly affected by publication of the DIAD study (39), which was not available at the time of the RNI ratings. Additionally, although the final rankings were different from the RNI ratings, Indications 127 and 171 demonstrated agreement within the current echocardiography technical panel. Therefore, the several indications with ratings that differed from RNI may reflect new literature that has become available since publication of the SPECT appropriateness criteria and differences in the composition of the 2 panels.
Readers should also note that the categorical summaries tend to accentuate differences that sometimes are slight. For example, small fluctuations in a median rating (e.g., 4 versus 3) will cause an indication to switch appropriateness categories (e.g., from uncertain to inappropriate). This phenomenon was relevant for Indication 157, which was rated as uncertain (median score 6) in this document, while the same indication in the RNI document (corresponding Indication 43) was rated appropriate (median score 7). The most likely reason for this is a simple variation in rating by the different panel members, whether because of composition, different levels of clinical experience, publication of additional literature, or different interpretations of data. The AUC Task Force has carefully examined the issue of panel membership and made every effort to ensure similar composition for each panel. The RAND process has documented that the interpretation of the literature by different sets of experts can yield slightly different final ratings (6).
As described in the Methods section, within each main disease category, a standardized approach was used in order to capture the majority of clinical scenarios without making the list of indications excessive. The approach was to create 5 broad clinical scenarios: 1) for initial diagnosis; 2) to guide therapy or management, regardless of symptom status; 3) to evaluate a change in clinical status or cardiac exam; 4) for early follow-up without change in clinical status; and 5) for late follow-up without change in clinical status. It should be noted that many cardiovascular conditions have the potential for residual anatomic or physiologic abnormalities, so that the timing and follow-up use of echocardiographic imaging depends on the patient's clinical status and the magnitude of or risk for residual abnormalities. Thus, routine surveillance indications for echocardiograms should not apply in those situations in which there has been a change in status or where an echocardiogram is being considered to guide therapeutic decision making. For asymptomatic or stable patients with known or suspected residual anatomic or physiologic abnormalities, the timing of the follow-up for considering changes in therapy in patients should be determined by individual patient factors, and not by the suggested intervals for routine surveillance studies.
Overall, indications focusing on initial diagnosis, guidance of therapy, or evaluation of a change in clinical status were viewed favorably by the rating panel. Uncertain or inappropriate ratings were more likely given to early rather than late follow-up, especially for those indications when the optimal interval of follow-up for asymptomatic patients is uncertain. Whenever possible, indications for timing of follow-up attempted to follow practice guidelines (14), although for many indications, the most appropriate follow-up interval for asymptomatic patients is not well established. For this reason, as well as for clinical expediency, the follow-up interval selected is not meant to be rigid but rather to represent an approximate time interval.
Although the overall approach was broad and inclusive, certain specific clinical scenarios warranted focused indications based on results from the previously mentioned implementation studies. Examples include Indications 71 and 72, which differentiate the re-evaluation of decompensated heart failure when there is no clear precipitating change in medication or diet versus when there is a clear precipitating factor. In the setting of an obvious change in diet or medication, a trial of appropriate medical therapy and monitoring for clinical improvement may be justified prior to ordering a repeat imaging test for assessment of cardiac function (25). As such, Indication 72 (clear precipitating change in medication or diet) was rated as uncertain, and Indication 71 was rated as appropriate. Another focused clinical situation is reflected in Indication 76, “Initial evaluation or re-evaluation after revascularization and/or optimal medical therapy to determine candidacy for device therapy and/or to determine optimal choice of device.” As per the results of an implementation study (24), this clinical scenario was not well captured in the initial AUC document. However, re-evaluation of LV ejection fraction after revascularization or after a period of medical therapy to determine device candidacy represents a standard of care (40) and is a common indication for a TTE. This is now represented by Indication 76, which was rated as appropriate.
Other specific areas identified by implementation studies as common scenarios and now included are bradycardia (Indication 6) and a new subcategory within TTE for the evaluation of syncope (Indications 7 to 9). Additionally, the sections on valvular heart disease (both resting TTE/TEE and stress echocardiography for hemodynamics) have been expanded in an effort to address a greater number of clinical scenarios, and closely follow recent guideline recommendations (14).
Despite these extensive revisions and additions, all potential clinical scenarios were not covered by the revised AUC for echocardiography. Additionally, certain recommendations from implementation studies were considered to represent rare conditions or specialized practices and were therefore not included in the revised document. If certain clinical situations that are not currently covered are found to be more frequent than anticipated, they will be incorporated into future revisions. This emphasizes the iterative nature of this process.
Furthermore, there are several general categories that were purposefully not addressed. For example, intraoperative use of TEE for cardiac surgery was felt to be beyond the scope of this document. More highly specialized echocardiographic techniques, such as 3-dimensional echocardiography or epicardial imaging, are not addressed in this document. Additionally, as stated in the first paragraph of the Assumptions section, the AUC for TTE, TEE, and stress echocardiography are for adult patients. Indications for pediatric echocardiograms were not covered.
New Assumptions and Definition
In addition to adding new clinical indications and clarifying existing indications from the original TTE/TEE AUC (1) and stress echocardiography AUC (2), the writing group also revised and added specific assumptions and definitions. Several general assumptions were added. First, the assumption that cost should be implicitly considered in determining appropriate use of an echocardiogram was added. Second, a new assumption addresses the category of uncertain indications and clarifies that such a rating should not be considered grounds for withholding reimbursement. Third, a new assumption indicates that appropriateness ratings reflect whether a specific test is appropriate for a given patient, not whether it is preferred over another modality (e.g., RNI, CT). Thus, the AUC should not be used to provide clinical support for administrative policies regarding test preferences. Finally, an assumption clarifies that routine or surveillance echocardiograms represent a “periodic” evaluation after a certain period of time has elapsed, and are not being ordered because of any other clinical factors. Other more specific assumptions were also added. These include consideration of prosthetic and native valves together (unless otherwise specified) and that use of Doppler for hemodynamics includes assessment of both right and left heart hemodynamics. Furthermore, it is assumed that if a perioperative patient has symptoms or signs of cardiovascular disease, the study should be classified under a symptomatic indication (e.g., Indication 1), as opposed to an indication in the perioperative category.
Similar to the RNI AUC (3), the writing group revised the definition of “chest pain syndrome” and adopted the term “ischemic equivalent,” which encompasses chest pain syndromes as well as other symptoms and signs that the clinician believes may be attributable to CAD. The writing group also adopted the use of global risk assessment when assessing risk in asymptomatic patients (41). This revision was supported by the writing group, technical panel, and external reviewers and is in harmony with the most recent AUC for Cardiac CT (4).
Limitations
The ratings of the indications as appropriate, uncertain, or inappropriate are reflective of the body of knowledge at the time the rating process occurred. It is likely and expected that as science progresses and new evidence-based guidelines are published, certain indications that are given 1 rating may subsequently be determined to have a different appropriateness rating in the future. Although this necessarily reflects the evolving nature of medical science, it may also introduce apparent discrepancies between appropriateness of similar indications for different modalities evaluated at different time points. The current evidence base and practice guidelines were used to develop the indications whenever available, although for certain indications the literature was limited and clinical expertise played a larger role. This is consistent with the standard methodology and principles of evidence-based medicine as endorsed by the Physician Consortium for Performance Improvement (42). Additionally, as mentioned in the previous text, certain clinical scenarios were intentionally not covered by the indications. When future implementation studies evaluating this revised AUC for echocardiography are conducted, it may become apparent that frequent situations were not covered. As was the case for this current revision, results and recommendations from implementation studies will help shape future modifications to the AUC.
Use of AUC to Improve Care
The AUC in this report provide an estimate of whether it is reasonable to use echocardiography for a particular clinical scenario, specifically for 1 of the 202 indications listed in this document. These criteria are expected to be useful for clinicians, healthcare facilities, and third-party payers engaged in the delivery of cardiovascular imaging. The AUC is expected to be valuable across a broad range of situations, including guiding care of individual patients, educating caregivers, and informing policy decisions regarding cardiovascular imaging.
AUC represent the first component of the chain of quality domains for cardiovascular imaging (43). After ensuring proper test selection, the achievement of quality in imaging includes adherence to best practices in image acquisition, image interpretation and results communication, as well as incorporation of findings into clinical care. All components are important for optimal patient care, although the development of AUC and their ranking by the technical panel is intended to address only the first quality domain, and assumes no barriers to other quality standards are being met.
Although these criteria are intended to provide guidance for care decisions, they cannot serve as substitutes for sound clinical judgment and practice experience. The writing group recognizes that patients encountered in clinical practice may not be represented in these AUC or may have extenuating features when compared with the clinical scenarios presented. Additionally, uncertain indications often require individual physician judgment and an in-depth understanding of the patient to better determine the usefulness of a test for a particular scenario. As such, the ranking of an indication as uncertain (4 to 6) should not be viewed as limiting the use of echocardiography for such patients. It should be emphasized that the technical panel was instructed that the “uncertain” designation was still designed to be considered as a “reimbursable” category.
These ratings reflect the critical medical literature as well as expert consensus and are intended to evaluate the appropriate use of specific patient scenarios to determine overall patterns of care regarding echocardiography. In situations where there is substantial variation between the appropriate use rating and what the clinician believes is the best recommendation for the patient, further considerations or actions, such as a second opinion, may be appropriate. Moreover, it is neither anticipated nor desirable that all physicians or facilities will have 100% of their echocardiograms deemed appropriate. However, it is desirable, though not realistic, that 0% be inappropriate. Related to the overall patterns of care, if the national average of appropriate and uncertain ratings is 80%, for example, and a physician or facility has a 40% rate of inappropriate procedures, further examination of the patterns of care may be warranted and helpful. The use of AUC to guide clinical decision making and its impact on patient outcomes and healthcare quality/efficiency needs to be studied rigorously. AUC are also useful as educational tools for both echocardiography providers and referring physicians. The recently announced and soon to be implemented incorporation of AUC into echocardiography laboratory accreditation requirements will encourage their use (44). However, the greatest opportunity to optimize the use of echocardiography is in improving individual patient decision making. The successful application of AUC into clinical practice represents an important area of ongoing quality improvement.
Staff
American College of Cardiology Foundation
John C. Lewin, MD, Chief Executive Officer
Joseph M. Allen, MA, Director, TRIP (Translating Research Into Practice)
Starr Webb, MPH, Senior Specialist, Appropriate Use Criteria
Jenissa Haidari, MPH, Senior Specialist, Appropriate Use Criteria
Lea Binder, MA, Specialist, Appropriate Use Criteria
Erin A. Barrett, MPS, Senior Specialist, Science and Clinical Policy
Appendix
For supplementary data, please see the online version of this article.
Appendix A Additional Echocardiography Definitions
1. Angina
• Typical Angina (Definite): Defined as 1) substernal chest pain or discomfort that is 2) provoked by exertion or emotional stress and 3) relieved by rest and/or nitroglycerin (45).
• Atypical Angina (Probable): Chest pain or discomfort that lacks 1 of the characteristics of definite or typical angina.
• Nonanginal Chest Pain: Chest pain or discomfort that meets 1 or none of the typical angina characteristics.
2. Acute Coronary Syndrome (ACS)
As defined by the ACC/AHA Guidelines for the Management of Patients with ST-Elevation Myocardial Infarction: patients with an ACS include those whose clinical presentations cover the following range of diagnoses: unstable angina, myocardial infarction without ST-segment elevation (NSTEMI), and myocardial infarction with ST-segment elevation (STEMI) (46).
3. Evaluating Perioperative Risk for Noncardiac Surgery
Method for Determining Perioperative Risk
See Figure A1, “Stepwise Approach to Perioperative Cardiac Assessment,” from the ACCF/AHA guidelines on perioperative cardiovascular evaluation and care for noncardiac surgery (16). Based on the algorithm, once it is determined that the patient does not require urgent surgery, the clinician should determine the patient's active cardiac conditions (see Table A1) and/or perioperative risk predictors (see Table A2). If any active cardiac conditions and/or major risk predictors are present, Figure A1 suggests consideration of coronary angiography and postponing or canceling noncardiac surgery. Once perioperative risk predictors are assessed based on the algorithm, then the surgical risk and patient's functional status should be used to establish the need for noninvasive testing.
Stepwise Approach to Perioperative Cardiac Assessment
Cardiac evaluation and care algorithm for noncardiac surgery based on active clinical conditions, known cardiovascular disease, or cardiac risk factors for patients ≥50 years of age. HR indicates heart rate; LOE, level of evidence; and MET, metabolic equivalent.
Modified from (16).
Active Cardiac Conditions for Which the Patient Should Undergo Evaluation and Treatment Before Noncardiac Surgery (Class I, Level of Evidence: B)
Perioperative Clinical Risk Factors⁎
4. Thrombolysis In Myocardial Infarction (TIMI) Risk Scores
The TIMI risk score (48) is a simple tool composed of 7 (1-point) risk indicators rated on presentation. The composite end points (all-cause mortality, new or recurrent MI, or severe recurrent ischemia prompting urgent revascularization within 14 days) increase as the TIMI risk score increases. The model remained a significant predictor of events and test sensitivity and was relatively unaffected/uncompromised by missing information, such as knowledge of previously documented coronary stenosis of ≥50%. The model's predictive ability remained intact with a cutoff of 65 years of age.
The TIMI risk score is determined by the sum of the presence of 7 variables at admission; 1 point is given for each of the following variables: age ≥65 years, at least 3 risk factors for CAD, prior coronary stenosis of ≥50%, ST-segment deviation on ECG presentation, at least 2 anginal events in prior 24 hours, use of aspirin in prior 7 days, and elevated serum cardiac biomarkers.
Low-Risk TIMI Score: TIMI score <2
High-Risk TIMI Score: TIMI score ≥2
5. ECG–Uninterpretable
Refers to ECGs with resting ST-segment depression (≥0.10 mV), complete LBBB, pre-excitation (Wolff-Parkinson-White Syndrome), or paced rhythm.
6. Coronary Angiography
The term coronary angiography refers to invasive cardiac catheterization or to established noninvasive methods of imaging the coronary arteries, such as coronary CT angiography.
Appendix B Additional Methods
See the Methods section of the report for a description of panel selection, indication development, scope of indications, and rating process.
Relationships With Industry and Other Entities
A list of all individuals participating in the development and review of this document and their institutional and/or organizational affiliations is presented in Appendix C. The American College of Cardiology Foundation and its partnering organizations rigorously avoid any actual, perceived, or potential conflicts of interest that might arise as a result of an outside relationship or personal interest of a member of the technical panel. Specifically, all panelists are asked to provide disclosure statements of all relationships that might be perceived as real or potential conflicts of interest. These statements were reviewed by the Appropriate Use Criteria Task Force, discussed with all members of the technical panel at the face-to-face meeting, and updated and reviewed as necessary. A table of disclosures by the technical panel and oversight working group member can be found in Appendix D. In addition, to ensure complete transparency, complete disclosure information—including relationships not pertinent to this document—is available online as a document supplement.
Literature Review
The technical panel members were asked to refer to the relevant literature provided for each indication table when completing their ratings (see Online Appendix).
Appendix C ACCF/ASE/AHA/ASNC/HFSA/HRS/SCAI/SCCM/SCCT/SCMR 2011 Appropriate Use Criteria for Echocardiography Participants
Echocardiography Writing Group
Pamela S. Douglas, MD, MACC, FAHA, FASE–Chair, Appropriate Use Criteria for Echocardiography, Past President, American College of Cardiology Foundation; Past President American Society of Echocardiography; and Ursula Geller Professor of Research in Cardiovascular Diseases, Duke University Medical Center, Durham, NC
Mario J. Garcia, MD, FACC, FACP–Professor of Medicine, Montefiore Medical Center and Albert Einstein College of Medicine, Bronx, NY
David E. Haines, MD, FACC, FHRS–Professor of Cardiovascular Medicine, Oakland University, William Beaumont School of Medicine, Chairman, Department of Cardiovascular Medicine, Beaumont Hospitals, Royal Oak, MI
Wyman W. Lai, MD, MPH, FACC, FASE–Morgan Stanley Children's Hospital of NY, Pediatric Cardiology, Director, Noninvasive Cardiac Imaging Laboratory, New York, NY
Warren J. Manning, MD, FACC–Professor of Medicine and Radiology, Harvard Medical School, Beth Israel Deaconess Medical Center, Cardiovascular Division, Boston, MA
Ayan R. Patel, MD, FACC–Director, Cardiovascular Imaging & Hemodynamic Laboratory and Physician, Heart Failure & Cardiac Transplant Center, Tufts Medical Center; Associate Professor of Medicine, Tufts University School of Medicine, Boston, MA
Michael H. Picard, MD, FACC, FASE, FAHA–Director, Echocardiography, Massachusetts General Hospital, Associate Professor of Medicine, Harvard Medical School, Boston, MA
Donna M. Polk, MD, MPH, FACC, FASE, FASNC–Director, Preventative Cardiology, Hartford Hospital, Hartford, CT
Michael Ragosta, MD, FACC, FSCAI–Professor of Medicine/Cardiology, Director, Cardiac Catheterization Laboratories, Department of Medicine, Division of Cardiovascular Medicine, University of Virginia Health System, Charlottesville, VA
R. Parker Ward, MD, FACC, FASE, FASNC–Associate Professor of Medicine, Director of Cardiovascular Fellowship Program, University of Chicago Medical Center, Chicago, IL
Rory B. Weiner, MD–Clinical and Research Fellow, Harvard Medical School, Massachusetts General Hospital, Cardiology Division, Boston, MA
Echocardiography Technical Panel
Steven R. Bailey, MD, FACC, FSCAI, FAHA–Moderator of the Technical Panel, Chair, Division of Cardiology, Professor of Medicine and Radiology, Janey Briscoe Distinguished Chair, University of Texas Health Sciences Center, San Antonio, TX
Rory B. Weiner, MD–Writing Group Liaison Appropriate Use Criteria for Echocardiography Technical Panel, Clinical and Research Fellow, Harvard Medical School, Massachusetts General Hospital, Cardiology Division, Boston, MA
Peter Alagona, Jr, MD, FACC–Program Director General Cardiology, Penn State Heart and Vascular Institute, Hershey, PA
Jeffrey L. Anderson, MD, FACC, FAHA, MACP–Professor of Medicine, University of Utah, Associate Chief of Cardiology, Intermountain Medical Center, Salt Lake City, UT
Jeanne M. DeCara, MD, FACC, FASE–Associate Professor of Medicine, University of Chicago Medical Center, Chicago, IL
Rowena J. Dolor, MD, MHS–Assistant Professor, General Internal Medicine, Duke University Medical Center, Durham, NC
Reza Fazel, MD, FACC–Assistant Professor of Medicine, Emory University, Atlanta, GA
John A. Gillespie, MD, FACC–Clinical Assistant Professor of Family Medicine, University of Buffalo, Buffalo, NY
Paul A. Heidenreich, MD, FACC–Assistant Professor of Medicine, Stanford, VA Palo Alto Health Care System, Palo Alto, CA
Luci K. Leykum, MD, MBA, MSc–Associate Professor of Medicine, South Texas Veterans Health Care System/UT Health Science Center San Antonio, San Antonio, TX
Joseph E. Marine, MD, FACC, FHRS–Director of Electrophysiology, Johns Hopkins Bayview Medical Center; Associate Professor of Medicine, Johns Hopkins University School of Medicine, Baltimore, MD
Gregory J. Mishkel, MD, FACC, FSCAI, FRCPC–Co-Director, Cardiac Cath Lab, St. John's Hospital, Director Interventional Cardiology, St. Mary's Hospital, Prairie Cardiovascular Consultants, Springfield and Decatur, IL
Patricia A. Pellikka, MD, FACC, FAHA, FACP, FASE–Professor of Medicine, Mayo Clinic College of Medicine, Co-Director, Echocardiography Laboratory, Mayo Clinic, Rochester, MN
Gilbert L. Raff, MD, FACC, FSCCT–Director, Cardiac CT/MRI, William Beaumont Hospitals, Royal Oak, MI
Krishnaswami Vijayaraghavan, MD, FACC, FCCP–Medical Director, CV Research and Education, Scottsdale Healthcare, Scottsdale, AZ
Neil J. Weissman, MD, FACC, FAHA–President, MedStar Health Research Institute, Professor of Medicine, Georgetown University School of Medicine, Washington, DC
Katherine C. Wu, MD–Associate Professor of Medicine, Johns Hopkins Medical Institutions, Baltimore, MD
External Reviewers of the Appropriate Use Criteria Indications
Drew Baldwin, MD–Assistant Professor of Clinical Medicine, Tulane University Heart and Vascular Institute, New Orleans, LA
Thomas Behrenbeck, MD, PhD, FCCP–Associate Professor of Medicine, Mayo Clinic, Rochester, MN
Michael Blaivas, MD, FACEP–Professor of Emergency Medicine, Northside Hospital Forsyth, Atlanta, GA
Robert O. Bonow, MD, MACC–Goldberg Distinguished Professor, Chief, Division of Cardiology, Northwestern University Feinberg School of Medicine, Chicago, IL
Jeanne M. DeCara, MD, FACC, FASE–Associate Professor of Medicine, University of Chicago Medical Center, Chicago, IL
Peter L. Duffy, MD, FACC–Cardiologist, Partner, Pinehurst Cardiology Consultants, Pinehurst, NC
Kirsten E. Fleischmann, MD, MPH, FACC–Associate Professor of Medicine, University of California San Francisco School of Medicine, Division of Cardiology, San Francisco, CA
Shawn A. Gregory, MD–Cardiologist, Massachusetts General Hospital, Boston, MA
Scott D. Jerome, DO, FACC, FASNC, FSCCT–Assistant Professor of Medicine, University of Maryland School of Medicine, Director of Ambulatory Outreach Cardiology, Baltimore, MD
Frederick G. Kushner, MD, FACC–Clinical Professor, Tulane University Medical Center, Medical Director, Heart Clinic of Louisiana, Marrero, LA
John Lesser, MD, FACC–Director of Cardiovascular CT and MRI, Minneapolis Heart Institute, Minneapolis, MN
Alexander B. Levitov, MD–Professor, Eastern Virginia Medical School, Department of Medicine, Division of Pulmonary and Critical Care Medicine, Norfolk, VA
Kapildeo Lotun, MD–Assistant Professor of Medicine, Chief, Section of Vascular Medicine, Interventional Cardiology, Virginia Commonwealth University, Division of Cardiology, Richmond, VA
Jennifer H. Mieres, MD, FACC–Associate Professor of Medicine, Hofstra NorthShore-LIJ School of Medicine, Department of Cardiology, Manhasset, NY
Todd D. Miller, MD, FACC–Professor of Medicine, Cardiovascular Diseases, Mayo Graduate School of Medicine, Mayo Clinic, Rochester, MN
John V. Nixon, MD, FACC, FAHA–Professor of Medicine, Medical College of Virginia, Virginia Commonwealth University, Director, Echocardiography Laboratories and Heart Station, Richmond, VA
David T. Porembka, MD–Director of Perioperative Echocardiography, Professor of Anesthesia, Surgery, and Internal Medicine, University of Cincinnati Academic Center, Department of Anesthesiology, Cincinnati, OH
Brian D. Powell, MD, FACC–Assistant Professor of Medicine, Mayo Clinic, Rochester, MN
Subha Raman, MD–Associate Professor of Medicine, Ohio State University, Columbus, OH
Gregory S. Thomas, MD, MPH, FACC–Director, Nuclear Cardiology, Co-Director, Lipid Clinic, Mission Internal Medical Group, Clinical Associate Professor of Medicine, Director of Nuclear Cardiology Training, University of California, Irvine School of Medicine, Dana Point, CA
James D. Thomas, MD, FACC–Director of Cardiovascular Imaging, Cleveland Clinic Foundation, Cleveland, OH
Aseem Vashist, MD–Cardiologist, Hartford Cardiology Group, Hartford, CT
Mary N. Walsh, MD, FACC–Medical Director, Heart Failure and Cardiac Transplantation, St. Vincent Hospital, Clinical Associate Professor, Indiana University School of Medicine, Indianapolis, IN
Carole A. Warnes, MD, FACC–Professor of Medicine, Mayo Clinic, Rochester, MN
Joseph N. Wight, Jr, MD, FACC–Clinical Assistant Professor of Medicine, University of Vermont School of Medicine, Director of Heart Failure, Maine Medical Center, Director of Echocardiography, Maine Cardiology Associates, South Portland, ME
Kim A. Williams, MD, FACC, FAHA, FASNC–Dorothy Susan Timmis Professor and Chair, Division of Cardiology, Professor of Medicine, Wayne State University School of Medicine, Harper University Hospital, Detroit, MI
Katherine C. Wu, MD–Associate Professor of Medicine, Johns Hopkins Medical Institutions, Baltimore, MD
Appropriate Use Criteria Task Force
Michael J. Wolk, MD, MACC–Chair, Task Force, Past President, American College of Cardiology Foundation and Clinical Professor of Medicine, Weill-Cornell Medical School, New York, NY
Steven R. Bailey, MD, FACC, FSCAI, FAHA–Chair, Division of Cardiology, Professor of Medicine and Radiology, Janey Briscoe Distinguished Chair, University of Texas Health Sciences Center, San Antonio, TX
Pamela S. Douglas, MD, MACC, FAHA, FASE–Past President, American College of Cardiology Foundation; Past President American Society of Echocardiography; and Ursula Geller Professor of Research in Cardiovascular Diseases, Duke University Medical Center, Durham, NC
Robert C. Hendel, MD, FACC, FAHA, FASNC–Chair, Appropriate Use Criteria for Radionuclide Imaging Writing Group–Director of Cardiac Imaging and Outpatient Services, Division of Cardiology, Miami University School of Medicine, Miami, FL
Christopher M. Kramer, MD, FACC, FAHA–Professor of Medicine and Radiology and Director, Cardiovascular Imaging Center, University of Virginia Health System, Charlottesville, VA
James K. Min, MD, FACC–Assistant Professor of Medicine, Division of Cardiology, Assistant Professor of Radiology, Weill Cornell Medical College, New York Presbyterian Hospital, New York, NY
Manesh R. Patel, MD, FACC–Assistant Professor of Medicine, Division of Cardiology, Duke University Medical Center, Durham, NC
Leslee Shaw, PhD, FACC, FASNC–Professor of Medicine, Emory University School of Medicine, Atlanta, GA
Raymond F. Stainback, MD, FACC, FASE–Medical Director of Noninvasive Cardiac Imaging, Texas Heart Institute at St. Luke's Episcopal Hospital, Houston, Texas; Clinical Associate Professor of Medicine, Baylor College of Medicine; President-Elect, Intersocietal Commission for the Accreditation of Echocardiography Laboratories (ICAEL); Hall-Garcia Cardiology Associates, Houston, TX
Joseph M. Allen, MA–Director, TRIP (Translating Research into Practice), American College of Cardiology Foundation, Washington, DC
Appendix D ACCF/ASE/AHA/ASNC/HFSA/HRS/SCAI/SCCM/SCCT/SCMR 2011 Appropriate Use Criteria for Echocardiography Writing Group, Technical Panel, Indication Reviewers, and Task Force—Relationships With Industry and Other Entities (In Alphabetical Order Within Each Group)
Participant | Consultant | Speaker | Ownership/ Partnership/ Principal | Research | Institutional, Organizational, or Other Financial Benefit | Expert Witness |
---|---|---|---|---|---|---|
Echocardiography Appropriate Use Criteria Writing Group | ||||||
Pamela S. Douglas | None | None | None | None | None | None |
Mario J. Garcia | None | None | None | None | None | None |
David E. Haines | None | None | None | None | None | None |
Wyman W. Lai | None | None | None | None | None | None |
Warren J. Manning |
| None | None |
| None | None |
Ayan R. Patel | None | None | None | None | None | None |
Michael H. Picard | None | None | None |
| None | None |
Donna M. Polk | None | None | None | None | None | None |
Michael Ragosta | None | None | None | None | None | None |
R. Parker Ward | None | None | None | None | None | None |
Rory B. Weiner | None | None | None | None | None | None |
Echocardiography Appropriate Use Criteria Technical Panel | ||||||
Steven R. Bailey | None | None | None | None | None | None |
Rory B. Weiner | None | None | None | None | None | None |
Peter Alagona, Jr | None | None | None | None | None | None |
Jeffrey L. Anderson | None | None | None |
| None | None |
Jeanne M. DeCara | None | None | None | None | None | None |
Rowena J. Dolor | None | None | None | None | None | None |
Reza Fazel | None | None | None | None | None | None |
John A. Gillespie | None | None | None | None | None | None |
Paul A. Heidenreich | None | None | None | None | None | None |
Luci K. Leykum | None | None | None | None | None | None |
Joseph E. Marine | None | None | None | None | None | None |
Gregory J. Mishkel | None | None | None | None | None | None |
Patricia A. Pellikka | None | None | None | None | None | None |
Gilbert L. Raff | None | None | None | None | None | None |
Krishnaswami Vijayaraghavan | None | None | None | None | None | None |
Neil J. Weissman | None | None | None | None | None | None |
Katherine C. Wu | None | None | None | None | None | None |
Echocardiography Appropriate Use Criteria Indication Reviewers | ||||||
Drew Baldwin | None | None | None | None |
| None |
Thomas Behrenbeck | None | None | None | None | None | None |
Michael Blaivas | None | None | None | None | None | None |
Robert O. Bonow | None | None | None | None | None | None |
Jeanne M. DeCara | None | None | None | None | None | None |
Peter L. Duffy | None | None | None | None | None | None |
Kirsten E. Fleischmann | None | None | None | None | None | None |
Shawn A. Gregory | None | None | None | None | None | None |
Scott D. Jerome | None | None | None | None | None | None |
Frederick G. Kushner | None | None | None | None | None | None |
John Lesser |
|
| None | None | None | None |
Alexander B. Levitov |
| None | None | None | None | None |
Kapildeo Lotun | None | None | None | None | None | None |
Jennifer H. Mieres | None | None | None | None | None | None |
Todd D. Miller | None | None | None | None | None | None |
John V. Nixon | None | None | None | None | None | None |
David T. Porembka | None | None | None | None | None | None |
Brian D. Powell | None | None | None | None | None | None |
Subha Raman | None | None | None | None | None | None |
Gregory S. Thomas | None | None | None | None | None | None |
James D. Thomas | None | None | None | None | None | None |
Aseem Vashist | None | None | None | None | None | None |
Mary N. Walsh | None | None | None | None | None | None |
Carole A. Warnes | None | None | None | None | None | None |
Joseph N. Wight, Jr | None | None | None | None | None | None |
Kim A. Williams | None | None | None | None | None | None |
Katherine C. Wu | None | None | None | None | None | None |
Echocardiography Appropriate Use Criteria Task Force | ||||||
Michael J. Wolk | None | None | None | None | None | None |
Steven R. Bailey | None | None | None | None | None | None |
Pamela S. Douglas | None | None | None | None | None | None |
Robert C. Hendel | None | None | None | None | None | None |
Christopher M. Kramer | None | None | None | None | None | None |
James K. Min | None |
| None | None | None | None |
Manesh R. Patel |
| None | None | None | None | None |
Leslee Shaw | None | None | None | None | None | None |
Raymond F. Stainback | None | None | None | None | None | None |
Joseph M. Allen | None | None | None | None | None | None |
This table represents the relevant relationships with industry and other entities that were disclosed by participants at the time of participation. It does not necessarily reflect relationships with industry at the time of publication. A person is deemed to have a significant interest in a business if the interest represents ownership of 5% or more of the voting stock or share of the business entity, or ownership of $10,000 or more of the fair market value of the business entity; or if funds received by the person from the business entity exceed 5% of the person's gross income for the previous year. A relationship is considered to be modest if it is less than significant under the preceding definition. Relationships in this table are modest unless otherwise noted. Names are listed in alphabetical order within each category of review. Participation does not imply endorsement of this document.
↵⁎ Significant relationship.
Footnotes
↵⁎ Official American College of Cardiology Foundation Representative
↵† Official Society of Cardiovascular Computed Tomography Representative
↵‡ Official Heart Rhythm Society Representative
↵§ Official American Society of Echocardiography Representative
↵∥ Official American Heart Association Representative
↵¶ Official Society for Cardiovascular Magnetic Resonance Representative
↵# Official Heart Failure Society of America Representative
↵⁎⁎ Official American Society of Nuclear Cardiology Representative
↵†† Official Society for Cardiovascular Angiography and Interventions Representative
↵‡‡ Official Health Plan Representative
↵§§ Official American College of Chest Physicians Representative
↵⁎ Negative consequences include the risks of the procedure (i.e., radiation or contrast exposure) and the downstream impact of poor test performance such as delay in diagnosis (false-negatives) or inappropriate diagnosis (false-positives).
This document was approved by the American College of Cardiology Foundation Board of Trustees in November 2010.
The American College of Cardiology Foundation requests that this document be cited as follows: Douglas PS, Garcia MJ, Haines DE, Lai WW, Manning WJ, Patel AR, Picard MH, Polk DM, Ragosta M, Ward RP, Weiner RB. ACCF/ASE/AHA/ASNC/HFSA/HRS/SCAI/SCCM/SCCT/SCMR 2011 appropriate use criteria for echocardiography: a report of the American College of Cardiology Foundation Appropriate Use Criteria Task Force, American Society of Echocardiography, American Heart Association, American Society of Nuclear Cardiology, Heart Failure Society of America, Heart Rhythm Society, Society for Cardiovascular Angiography and Interventions, Society of Critical Care Medicine, Society of Cardiovascular Computed Tomography, and Society for Cardiovascular Magnetic Resonance. J Am Coll Cardiol 2011:57:1126–66.
This article is copublished in the Journal of Cardiovascular Computed Tomography and the Journal of the American Society of Echocardiography.
Copies: This document is available on the World Wide Web site of the American College of Cardiology (www.cardiosource.org). For copies of this document, please contact Elsevier Inc. Reprint Department, fax (212) 633-3820, e-mail reprints{at}elsevier.com.
Permissions: Modification, alteration, enhancement, and/or distribution of this document are not permitted without the express permission of the American College of Cardiology Foundation. Please contact Elsevier's permission department healthpermissions{at}elsevier.com.
- American College of Cardiology Foundation
References
- 1.↵
- Douglas P.S.,
- Khandheria B.,
- Stainback R.F.,
- et al.
- 2.↵
- Douglas P.S.,
- Khandheria B.,
- Stainback R.F.,
- et al.
- 3.↵
- Hendel R.C.,
- Berman D.S.,
- Di Carli M.F.,
- et al.
- 4.↵
- Taylor A.J.,
- Cequeira M.,
- Hodgson J.,
- et al.
- 5.↵
- Patel M.R.,
- Spertus J.A.,
- Brindis R.G.,
- et al.
- 6.↵
- Fitch K.,
- Bernstein S.J.,
- Aguilar M.D.
- 7.↵
- Quinones M.A.,
- Otto C.M.,
- Stoddard M.,
- et al.
- 8.
- 9.
- Lang R.M.,
- Bierig M.,
- Devereux R.B.,
- et al.
- 10.↵
- 11.↵
- 12.↵
- Ryan T.,
- Armstrong W.F.,
- Khandheria B.K.
- 13.
- Quinones M.A.,
- Douglas P.S.,
- Foster E.,
- et al.
- 14.↵
- Bonow R.O.,
- Carabello B.A.,
- Chatterjee K.,
- et al.
- 15.
- Zoghbi W.A.,
- Chambers J.B.,
- Dumesnil J.G.,
- et al.
- 16.↵
- Fleisher L.A.,
- Beckman J.A.,
- Brown K.A.,
- et al.
- 17.
- 18.↵
- 19.↵
- 20.
- 20a.
- Gibbons R.J.,
- Antman E.M.,
- Alpert J.S.,
- et al.
- 21.↵
- Gibbons R.J.,
- Abrams J.,
- Chatterjee K.,
- et al.
- 22.↵
- 23.↵
- 24.↵
- Ward R.P.,
- Mansour I.N.,
- Lemieux N.,
- et al.
- 25.↵
- 26.
- 27.↵
- Willens H.J.,
- Gomez-Marin O.,
- Heldman A.,
- et al.
- 28.↵
- Ward R.P.,
- Krauss D.,
- Mansour I.N.,
- et al.
- 29.↵
- 30.↵
- 31.
- 32.
- 33.↵
- McCully R.B.,
- Pellikka P.A.,
- Hodge D.O.,
- et al.
- 34.↵
- 35.↵
- Sirovich B.,
- Gallagher P.M.,
- Wennberg D.E.,
- Fisher E.S.
- 36.↵
- 37.↵
- Warnes C.A.,
- Williams R.G.,
- Bashore T.M.,
- et al.
- 38.↵
- Hiratzka L.F.,
- Bakris G.L.,
- Beckman J.A.,
- et al.
- 39.↵
- 40.↵
- Hunt S.A.,
- Abraham W.T.,
- Chin M.H.,
- et al.
- 41.↵
- Lloyd-Jones D.M.
- 42.↵(2009) Physician Consortium for Performance Improvement (PCPI) Position Statement: The Evidence Base Required for Measures Development (American Medical Association, Chicago, Il).
- 43.↵
- Douglas P.,
- Iskandrian A.E.,
- Krumholz H.M.,
- et al.
- 44.↵The Intersocietal Accreditation Commission. http://www.intersocietal.org/icael/apply/standards.htm. Accessed June 10, 2010.
- 45.↵
- Diamond G.A.
- 46.↵
- Antman E.M.,
- Anbe D.T.,
- Armstrong P.W.,
- et al.
- 47.
- 48.↵
Toolbox
Citation Manager Formats
Article Outline
- Top
- Technical Panel
- Appropriate Use Criteria Task Force
- Table of Contents
- Abstract
- Preface
- 1 Introduction
- 2 Methods
- 3 General Assumptions
- 4 Definitions
- 5 Abbreviations
- 6 Results of Ratings
- 7 Echocardiography Appropriate Use Criteria (by Indication)
- 8 Echocardiography Appropriate Use Criteria (by Appropriate Use Rating)
- 9 Discussion
- Staff
- Appendix
- Appendix
- Appendix A Additional Echocardiography Definitions
- Appendix B Additional Methods
- Appendix C ACCF/ASE/AHA/ASNC/HFSA/HRS/SCAI/SCCM/SCCT/SCMR 2011 Appropriate Use Criteria for Echocardiography Participants
- Appendix D ACCF/ASE/AHA/ASNC/HFSA/HRS/SCAI/SCCM/SCCT/SCMR 2011 Appropriate Use Criteria for Echocardiography Writing Group, Technical Panel, Indication Reviewers, and Task Force—Relationships With Industry and Other Entities (In Alphabetical Order Within Each Group)
- Footnotes
- References
Figures in Article
Tables
Supplementary Materials
- Online Appendix[S0735109710044694_mmc1.pdf]
- Online Appendix[S0735109710044694_mmc2.pdf]