Journal of the American College of Cardiology

Volume 61, Issue 2, January 2013 DOI: 10.1016/j.jacc.2012.09.025
PDF Article

ACCF/AHA Clinical Practice Guideline Methodology Summit Report
A Report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines

Alice K. Jacobs, Frederick G. Kushner, Steven M. Ettinger, Robert A. Guyton, Jeffrey L. Anderson, E. Magnus Ohman, Nancy M. Albert, Elliott M. Antman, Donna K. Arnett, Marnie Bertolet, Deepak L. Bhatt, Ralph G. Brindis, Mark A. Creager, David L. DeMets, Kay Dickersin, Gregg C. Fonarow, Raymond J. Gibbons, Jonathan L. Halperin, Judith S. Hochman, Marguerite A. Koster, Sharon-Lise T. Normand, Eduardo Ortiz, Eric D. Peterson, William H. Roach Jr, Ralph L. Sacco, Sidney C. Smith Jr, William G. Stevenson, Gordon F. Tomaselli, Clyde W. Yancy, William A. Zoghbi, John G. Harold, Yulei He, Pamela B. Mangu, Amir Qaseem, Michael R. Sayre and Mark R. Somerfield

Table5

4. Standards for Synthesizing the Body of Evidence
IOM Standards and ElementsACCF/AHA Current MethodologyWorkgroup 4 Proposal
4.1 Use a prespecified method to evaluate the body of evidence

  • 4.1.1 For each outcome, systematically assess the following characteristics of the body of evidence:

  •   • Risk of bias

  •   • Consistency

  •   • Precision

  •   • Directness

  •   • Reporting bias

Currently, the ACCF/AHA does not systematically look at these characteristics. However, the ACCF/AHA does implicitly judge consistency, precision, and directness when using the COR/LOE to write recommendations.The Workgroup proposes that these IOM-recommended basic characteristics of quality and elements be accepted for assessing and describing quality across studies.

  • 4.1.2 For bodies of evidence that include observational research, also systematically assess the following characteristics for each outcome:

  •   • Dose-response association

  •   • Plausible confounding that would change the observed effect

  •   • Strength of association

A systematic assessment of these characteristics is performed on a case basis, depending on the study analyzed.

  • The Workgroup recommends the following:

  •   • Standardize the method of assessment and description of the quality of the body of evidence across studies.

  •   • Depict qualitative assessment across studies addressing the key elements for each PICO(TS) question in a summary table (e.g., similar to GRADE or other formats [Table 4.1]).

  •   • Where appropriate, provide a risk-of-bias assessment table across studies (e.g., similar to the Cochrane format [10]).

  •   • Generate standardized summary and evidence templates specific to ACCF/AHA requirements. These elements will be covered.

4.1.3 For each outcome specified in the protocol, use consistent language to characterize the level of confidence in the estimates of the effect of an interventionBy using the COR/LOE, the ACCF/AHA uses consistent language to characterize the level of confidence in the estimates of the effect of an intervention.See above. The IOM standard should be followed.
4.2 Conduct a qualitative synthesis
4.2.1 Describe the clinical and methodological characteristics of the included studies, including their size, inclusion or exclusion of important subgroups, timeliness, and other relevant factorsThis is currently presented in the form of evidence tables but may be presented in the text as well.

  • The Workgroup proposes the following:

  •  • Carry out a qualitative assessment of individual studies and aggregated studies where appropriate.

  •  • These elements will be described (see also 4.1 above).

  •  • This will require allocation of resources for additional methodological expertise, because this exercise is judged to be beyond the expertise and time commitment expected of GWC members alone.

  •  • When available, high-quality SRs from reputable organizations (e.g., the NHLBI, Cochrane, Kaiser, or EPC) should be used.

  •  • If a new SR is needed, resources will need to be secured to pursue de novo analysis.

  •  • Notwithstanding these efforts to standardize and improve the qualitative analysis of bodies of evidence, every effort should be made to minimize delays in time to CPG development/revision.

4.2.2 Describe the strengths and limitations of individual studies and patterns across studiesThis is currently presented in the form of evidence tables but may be presented in the text as well.The IOM standard should be considered.
4.2.3 Describe, in plain terms, how flaws in the design or execution of the study (or groups of studies) could bias the results, explaining the reasoning behind these judgmentsThis is currently presented in the form of evidence tables but may be presented in the text as well.The IOM standard should be considered.
4.2.4 Describe the relationships between the characteristics of the individual studies and their reported findings and patterns across studiesThis is currently presented in the form of evidence tables but may be presented in the text as well.The IOM standard should be considered.
4.2.5 Discuss the relevance of individual studies to the populations, comparisons, cointerventions, settings, and outcomes or measures of interestThis is currently presented in the form of evidence tables but may be presented in the text as well.The IOM standard should be considered.
4.3 Decide if, in addition to a qualitative analysis, the SR will include a quantitative analysis (meta-analysis)

  • The GWC, in conjunction with the Task Force, will determine when a specific meta-analysis is needed.

  • When available, high-quality meta-analyses from reputable organizations (e.g., the NHLBI, Cochrane, Kaiser, or EPC) should be used.

  • If a new meta-analysis is needed, resources will need to be secured to pursue de novo analyses.

  • Notwithstanding efforts to standardize and improve the approach to quantitative analyses across studies, every effort should be made to minimize delays in time to CPG development/revision.

4.3.1 Explain why a pooled estimate might be useful to decision makersReporting meta-analyses for the subject is encouraged. Limitations regarding meta-analyses are explained in the text.This may be described in text. See above.
4.4 If conducting a meta-analysis, then do the following:
4.4.1 Use expert methodologists to develop, execute, and peer review the meta-analysesMeta-analyses are not performed by GWC members, but published meta-analyses are reported. Meta-analyses used must be published in peer-reviewed journals, and any limitations of the meta-analysis are explained.

  • With statistical consultation, preferred, acceptable methods for meta-analysis (including a Bayesian analysis when appropriate) should be defined in each instance.

  • See 4.3 above.

4.4.2 Address the heterogeneity among study effectsThe ACCF/AHA does not explicitly do this but implicitly judges heterogeneity among study effects, which determines the need to downgrade recommendations.The IOM standard should be considered.
4.4.3 Accompany all estimates with measures of statistical uncertaintyThis is not currently performed.The IOM standard should be considered.
4.4.4 Assess the sensitivity of conclusions to changes in the protocol, assumptions, and study selection (sensitivity analysis)This is not currently performed.The IOM standard should be considered.

  • Added by Workgroup:

  • Review ACCF/AHA COR/LOE versus GRADE, NHLBI methodology (hybrid of GRADE and COR/LOE), others, and make a recommendation to Summit: keep our method without change, change to GRADE or to NHLBI (or another), or propose a hybrid system.

The ACCF/AHA currently uses COR (I, IIa, IIb, III)/LOE (A, B, C), whereas other popular systems differ.

  • The Workgroup recommends the following:

  •  • Retain the current basic ACCF/AHA COR and LOE structure/nomenclature.

  •  • Standardize how LOE: A, B, C are determined using a validated ACCF/AHA Evidence Grading Tool (which is under development).

  •  • Change the wording of LOE to QOE.

  •  • Add a separate category for QOE: E (expert opinion) and generate specific definitions and examples for QOE: E.

  •  • Add adjectives that “map” COR/LOE to the NHLBI (i.e., I [strong]; IIa [moderate]; IIb [weak]; and III [against]).

  •  • Take every opportunity to align the COR as closely as possible with the LOE/QOE.

  •  • Provide a chart to map the ACCF/AHA system to other major grading systems (e.g., Table 4.2).

  •  • In addition:

  •  • Plan for regular review (e.g., annually) of CPG recommendations and modify as appropriate.

  •  • Develop a list of metrics that should be measured and assessed.

ACCF indicates American College of Cardiology Foundation; AHA, American Heart Association; COR, class of recommendation; CPG, clinical practice guideline; EPC, evidence-based practice center; GRADE, Grading of Recommendations Assessment, Development and Evaluation; GWC, guideline writing committee; IOM, Institute of Medicine; LOE, level of evidence; NHLBI, National Heart, Lung, and Blood Institute; PICO(TS), mnemonic: population, intervention, comparator, outcomes, timing and setting; QOE, quality of evidence; SR, systematic review; and Task Force, ACCF/AHA Task Force on Practice Guidelines.

      View article

    Advertisement