Author + information
- Received August 24, 2010
- Revision received September 28, 2010
- Accepted October 1, 2010
- Published online April 12, 2011.
- Bart S. Ferket, MD⁎,†,
- Tessa S.S. Genders, MSc⁎,†,
- Ersen B. Colkesen, MD‡,
- Jacob J. Visser, MD, PhD⁎,
- Sandra Spronk, PhD⁎,†,
- Ewout W. Steyerberg, PhD§ and
- M.G. Myriam Hunink, MD, PhD⁎,†∥,⁎ ()
- ↵⁎Reprint requests and correspondence:
Dr. M. G. Myriam Hunink, Department of Radiology and Department of Epidemiology, Room Ee 21-40a, Erasmus University Medical Center, Dr. Molewaterplein 40, 3015 GD, Rotterdam, the Netherlands
Objectives The purpose of this study was to critically appraise guidelines on imaging of asymptomatic coronary artery disease (CAD).
Background Various imaging tests exist to detect CAD in asymptomatic persons. Because randomized controlled trials are lacking, guidelines that address the use of CAD imaging tests may disagree.
Methods Guidelines in English published between January 1, 2003, and February 26, 2010, were retrieved using MEDLINE, Cumulative Index to Nursing and Allied Health Literature, the National Guideline Clearinghouse, the National Library for Health, the Canadian Medication Association Infobase, and the Guidelines International Network International Guideline Library. Guidelines developed by national and international medical societies from Western countries, containing recommendations on imaging of asymptomatic CAD were included. Rigor of development was scored by 2 independent reviewers using the Appraisal of Guidelines Research and Evaluation (AGREE) instrument. One reviewer performed full extraction of recommendations, which was checked by a second reviewer.
Results Of 2,415 titles identified, 14 guidelines met our inclusion criteria. Eleven of 14 guidelines reported relationship with industry. The AGREE scores varied across guidelines from 21% to 93%. Two guidelines considered cost effectiveness. Eight guidelines recommended against or found insufficient evidence for testing of asymptomatic CAD. The other 6 guidelines recommended imaging patients at intermediate or high CAD risk based on the Framingham risk score, and 5 considered computed tomography calcium scoring useful for this purpose.
Conclusions Guidelines on risk assessment by imaging of asymptomatic CAD contain conflicting recommendations. More research, including randomized controlled trials, evaluating the impact of imaging on clinical outcomes and costs is needed.
As many as 50% of myocardial infarctions occur in persons without a known history of symptomatic coronary artery disease (CAD) (1). To diminish disease burden, primary prevention on the individual level is currently rendered by targeting high-risk subjects, who are identified by office-based risk assessment using multiple traditional cardiovascular risk predictors: age, sex, smoking, lipid levels, and blood pressure. Screening using these traditional predictors, however, misses a considerable proportion of persons who will suffer from coronary events (2). Because symptomatic CAD has a pre-clinical detectable phase (i.e., coronary atherosclerosis), early detection of CAD in apparently healthy persons may be an important substitute for or supplement to risk assessment based on the traditional risk factors.
Because technical developments have created various imaging techniques to assess a patient's coronary condition, clinicians are faced with multiple options to choose from. Before a doctor decides to test for asymptomatic disease, the intervention should meet a set of specific screening criteria (3–5). Hence, clinicians and decision makers usually rely on clinical practice guidelines in which recommendations are made on the basis of these criteria. As opposed to cancer screening, few large randomized controlled trials (RCTs) studying the effect of early detection of CAD on event rates within an asymptomatic population have been performed. In absence of RCTs demonstrating a net health benefit of imaging, the weighing of harms and benefits is more likely to result in different judgments, and therefore, conflicting recommendations. Therefore, a critical appraisal of guidelines and review of the agreements and the differences among recommendations can serve as a guide for deciding which imaging tests to use in clinical practice.
For this purpose, we systematically reviewed guidelines containing recommendations on imaging of asymptomatic CAD within the general population.
Data sources and searches
To identify appropriate guidelines, the literature search used for a previous article on cardiovascular risk assessment (6) was updated and covered a period from January 1, 2003, to February 26, 2010. Briefly, MEDLINE, Cumulative Index to Nursing and Allied Health Literature (CINAHL), and 4 guideline databases—the National Guideline Clearinghouse (United States), the National Library for Health (United Kingdom) on Guideline Finder, Canadian Medical Association Infobase (Canada), and the Guidelines International Network (G-I-N) International Guideline Library—were searched. Searches were limited to guidelines from the United States, Canada, United Kingdom, Australia, and New Zealand, and international guidelines in the English language. A search on websites of guideline development organizations was performed for additional guidelines.
Articles were considered if they met the Institute of Medicine definition for clinical practice guidelines. The Institute of Medicine defines clinical practice guidelines as “systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.” If doubt existed whether a report met this definition or not, we verified eligibility by checking the inclusion of similar reports in the National Guideline Clearinghouse. This database also uses the Institute of Medicine definition. For this reason, we also considered American Heart Association (AHA) expert consensus documents and scientific statements, and American College of Cardiology Foundation (ACCF) appropriateness criteria reports. We included guidelines if they: 1) contained recommendations on imaging of asymptomatic CAD specifically aimed to prevent a first coronary event; 2) involved apparently healthy persons, that is adults without, for example, diabetes mellitus; and 3) were produced on behalf of a national or international medical specialty society. For completeness, we also included guidelines on electrocardiography and exercise tolerance tests, because these tests are traditionally used in the diagnosis of CAD.
The SRS 4.0 (Mobius Analytics, Ottowa, Ontario, Canada), a web-based software package developed for systematic review data management, was used. Review of titles and abstracts was performed independently by 2 reviewers (B.S.F. and E.B.C.). For a paper to be excluded, both reviewers had to agree that the article was ineligible. For abstracts, disagreements between the reviewers were discussed and resolved by consensus. The final selection based on the full text was performed by the first author.
Data extraction and quality assessment
One reviewer (B.S.F.) extracted all relevant recommendations from each included guideline. A second reviewer (T.S.S.G.) checked the results obtained for accuracy and completeness. Discrepancies were resolved by consensus. Each guideline could provide 1 or more relevant recommendations. Data extracted on a guideline level included the reported methodology for evidence synthesis, and formulating of recommendations. On the recommendation level, we extracted data on consideration of cost effectiveness, the target population, the strategy for delivery of the test, coronary atherosclerosis tests, intervention, and follow-up. In addition, the strength of the recommendation was classified as “for,” “consider,” “not for, not against,” “insufficient evidence,” or “against.” We assessed the quality of development for each included guideline using the 7-item Rigor of Development domain of the Appraisal of Guidelines Research and Evaluation (AGREE) instrument (7). This domain considers the reporting of: 1) methods to search for evidence; 2) criteria for selecting the evidence; 3) methods for formulating the recommendations; 4) consideration of health benefits, side effects, and risks; 5) supporting evidence; 6) procedures for external peer review; and 7) the update process. Each item was independently rated on a 4-point Likert scale by 2 reviewers (B.S.F. and T.S.S.G.). Websites of guideline developers were examined by both reviewers for additional information on the development processes. For each reviewer, AGREE scores were calculated as a percentage using the sum of the 7 items and the maximum possible score. If the total AGREE scores of the 2 reviewers differed >20%, a third independent reviewer (J.J.V.) also assessed the guideline. Final rigor scores were calculated by averaging the AGREE scores from all reviewers. Three guidelines (8–10) were rated by 3 reviewers. We ranked included guidelines according to their score. Editorial independence from funding body, external funding, proportion of guideline panel member-industry relationships, and disclosure of identities and relationships with industry of peer reviewers were assessed by 1 reviewer (B.S.F.) and checked by a second reviewer (T.S.S.G.). Discrepancies were resolved by consensus.
Data synthesis and analysis
A table for comparison of the recommendations from the selected guidelines was constructed. The table was divided into 1) methodology of guideline development; 2) consideration of cost effectiveness regarding the recommendation; 3) target group and delivery of early detection; 4) tests considered; and 5) thresholds for intervention and follow-up. Agreement between reviewers on AGREE scores was assessed using the intraclass correlation coefficient. Given the limited number of guidelines, only explorative quantitative analyses were possible. We examined the correlation between the proportion of guideline panel members who reported relationships with industry and the AGREE score with guidelines as units of analysis. Furthermore, we examined whether the proportion of panel members with industry relationships and the AGREE score were associated with a positive recommendation (“consider” or “for”) by logistic regression. Two guidelines that had no explicit statement on conflicts of interest of panel members were excluded from the analyses. An alpha level of 0.05 was used to indicate statistical significance. All analyses were performed using SPSS, version 15.0 (SPSS Inc., Chicago, Illinois).
Fourteen guidelines (11–22) relevant to testing of asymptomatic CAD were eligible for full data extraction (Fig. 1).Table 1 summarizes the selected guidelines, together with AGREE score and conflict of interest results. Most guidelines (10 of 14) were developed in the United States. The AGREE scores varied from 21% to 93%, with a median AGREE score of 57%. Reproducibility of the 2 reviewers' average AGREE scores was good, with an intraclass correlation coefficient of 0.76. Examples of low scoring guidelines are the ACCF appropriateness criteria reports (ACCF2-4) (20–22). These guidelines provided excellent information on the methods followed for achieving consensus and formulating recommendations, but did not contain detailed information on the search strategy used to identify the evidence. Although “a standardized literature review” was performed for these reports, key words used in the search strategy, and inclusion and exclusion criteria for selecting articles were not reported. In addition, these guidelines did not explicitly discuss benefits and harms of recommendations and methodology for guideline updating.
Twelve of the 14 guidelines contained disclosure of relationships with industry, and in 11, at least 1 panel member declared having a relevant financial relationship. In this limited set of 12 guidelines, no relationship between the AGREE score and the proportion of panel members with an industry relationship was observed (Pearson's correlation r = −0.205; p = 0.523).
General findings among the recommendations
The 14 included guidelines contained 26 recommendations on testing of asymptomatic CAD (Table 2). The following tests were considered: computed tomography (CT) calcium scoring, CT angiography, magnetic resonance (MR) angiography, single-photon emission computed tomography (SPECT), positron emission tomography (PET), stress echography, resting electrocardiography, and exercise tolerance testing. The majority of guidelines, except for the Canadian Cardiovascular Society (CCS) 2 guideline (19), were based on a comprehensive review including study quality assessment. Apart from the Canadian Association of Radiologists (CAR) (18) and CCS2 (19) guidelines, a grading system for assigning the level of evidence was used. Evaluation of cost effectiveness of recommended tests was explicitly done in only 2 guidelines, the U.S. Preventive Services Task Force (USPSTF) 1 (11) and ACCF1 (15) guidelines, by reviewing decision modeling studies on exercise tolerance testing (23–25) and CT calcium scoring (26,27), respectively. However, both guideline groups were unable to find a sufficient number of high-quality cost-effectiveness analyses on which to base their recommendations. In other guidelines (ACCF2  and ACCF4 ), group members were requested to consider costs in their decision making as well, but this was not based on a review of cost-effectiveness studies or decision analyses.
Eight of the 14 guidelines recommended against or concluded that there is insufficient evidence for testing of asymptomatic CAD. In the remaining 6 guidelines (ACCF1 , AHA2 , National Cholesterol Education Program [NCEP] [16,17], CAR , CCS2 , and ACCF2 ), testing was only advocated for patients with an a priori elevated risk level based on absolute CAD risk or multiple risk factors. Generally, risk was determined by Framingham risk equations for estimation of a 10-year risk for coronary events (fatal and nonfatal) using the categories <10%, 10% to 20%, and >20% for, respectively, low, intermediate, and high risk. However, 2 guidelines (CAR  and CCS2 ) did not specify any criteria for low, intermediate, and high risk. None advocated a universal screening approach or screening based on an age criterion alone. Whether a guideline contained a recommendation that supports testing or not did not statistically significantly depend on AGREE score or proportion of panel members with industry relationships. Adjusted odds ratios per 10% increase were 0.73 (95% confidence interval: 0.41 to 1.33) and 0.68 (95% confidence interval: 0.35 to 1.32), respectively.
The indications for further testing and primary preventive measures were not described in much detail. Overall, in guidelines that recommended for consideration of testing of asymptomatic CAD within an intermediate-risk population (ACCF1 , AHA2 , NCEP [16,17], CAR , and CCS2 ), all (previous intermediate risk) subjects were marked as high risk after a positive test. None of these guidelines contained recommendations in which traditional prediction models were updated by including test results as covariate. In addition, none of the guidelines reported whether the tests should be performed once or periodically in case of a negative test result.
CT calcium scoring
Most guidelines (10 of 14) considered the CT calcium score as a test for improvement of total coronary risk assessment based on traditional risk factors. Among these 10 guidelines (USPSTF1 , USPSTF2 , New Zealand Guidelines Group [NZGG] , ACCF1 , CCS1 , AHA2 , NCEP [16,17], CAR , CCS2 , and ACCF4 ), 4 guidelines (ACCF1 , AHA2 , NCEP [16,17], and CCS2 ) concluded that there was sufficient evidence for consideration of its use, and 1 guideline (CAR) (18) recommended for its use. These guidelines recommended CT calcium scoring solely in an intermediate CAD risk population. In contrast, the USPSTF2 (12), NZGG (13), and ACCF4 (22) guidelines concluded that there is insufficient evidence for the intermediate-risk population. For low CAD risk persons and persons already known to be at high CAD risk, guidelines were unanimous in not advocating CT calcium scoring.
Electrocardiography and exercise tolerance testing
The USPSTF1 guideline (11) recommended against performing electrocardiography testing in a low-risk population and found insufficient evidence for subjects at elevated risk. No other guidelines provided recommendations for this test. Exercise tolerance testing was considered in 4 guidelines (USPSTF1 , NCEP [16,17], CCS1 , and AHA3 ): 1 (NCEP [16,17]) recommended considering testing, and 3 (CCS1 , USPSTF1 , and AHA3 ) were inconclusive.
Myocardial perfusion imaging
Single-photon emission computed tomography was considered in 3 guidelines (AHA2 , NCEP [16,17], and ACCF2 ), of which 2 (AHA2  and NCEP [16,17]) also considered PET. The AHA2 guideline (8) recommended against any use of myocardial perfusion imaging in asymptomatic subjects, whereas the NCEP (16,17) and ACCF2 (20) guidelines recommended its use for different target populations: either for intermediate-risk subjects (NCEP [16,17]) or solely for those at high risk (ACCF2 ).
CT angiography and MR angiography
The AHA1 (14), CAR (18), CCS2 (19), and ACCF4 (22) guidelines considered these tests for the asymptomatic population. None of these guidelines advocated their use. For subjects at high risk, insufficient evidence was found by the ACCF4 guideline (22).
Only 1 guideline (ACCF3 ) provided recommendations for stress echocardiography. For adults at high risk, insufficient evidence was found for its use; for the remaining asymptomatic population, stress echocardiography is not justified according to the ACCF3 guidelines (21).
In summary, we identified 14 guidelines on testing of asymptomatic CAD. In the development of most guidelines, relationships with the industry were present. A considerable number of guidelines achieved a low AGREE score. Various inconsistencies were observed among the guidelines regarding interpretation of the value of early detection of CAD. Many guideline groups recommended against testing of asymptomatic CAD or concluded that there is insufficient evidence. The guidelines that contained recommendations to consider testing of asymptomatic CAD only reported benefit for those at elevated risk, that is, those who were either at intermediate or high absolute risk for having a CAD event. The majority of these guidelines supported consideration of CT calcium scoring in case of intermediate CAD risk.
Some possible limitations of this review deserve attention. First, only guidelines developed by national or international medical specialty organizations were reviewed. Hence, guidelines developed by local organizations, private organizations, and individual experts were not considered. An example of an often-cited guideline, therefore, not included is the Society for Heart Attack Prevention and Eradication (SHAPE) guideline (28). The SHAPE guideline recommends periodic measurement of coronary calcium or carotid intima-media thickness in all asymptomatic men ages 45 to 75 years and women ages 55 to 75 years except those defined at very low risk. Such a universal screening approach is, however, not advocated by any of the guidelines included in this systematic review. Second, we used the AGREE instrument, which provides an overall score of the construction process of guidelines, not components. Although we expect that the quality of development across the whole guideline influences the quality of individual recommendations, in theory, a solid recommendation could be created within a poorly developed guideline and vice versa. Third, the AGREE instrument only considers the reported information related to the development of the guideline. The actual quality of the guideline development can, therefore, not be fully captured. For example, guideline groups that performed a full search for evidence and that did not report detailed information on the search strategy followed, received a low AGREE score for this item. In reality, the search followed may be adequate for identifying solid evidence. Fourth, it was difficult to quantify the true degree of influence by industry relationships, also because guidelines did not report payment amounts. Fifth, the ability to detect statistically significant relationships in the quantitative analyses, such as an association between industry relationships and the likelihood of a positive recommendation, was limited owing to the small set of included guidelines.
The disagreements on the value of early detection of CAD across the guidelines could partly be explained by the paucity of experimental research. A search on ClinicalTrials.gov (29) up to March 22, 2010, using search terms “coronary artery disease” and “prevention” or “screening,” provided 97 interventional studies. We found 5 RCTs on the effect of early detection of CAD versus current practice of risk assessment using traditional risk factors. Only 1 RCT (ClinicalTrials.gov identifier: NCT00927693) was conducted in an apparently healthy population, with CT calcium scoring as the intervention. The study's results on hard endpoints are, however, not yet published (30). One RCT (ClinicalTrials.gov identifier: NCT00769275) was performed in a population with diabetes and revealed no effect of screening by myocardial perfusion scans on cardiac event rates, although event rates in the screened and not-screened groups were low, and no standardized preventive treatment strategy was used (31). Other RCTs (ClinicalTrials.gov identifiers: NCT00431977, NCT00488033, and NCT00547872), on CT angiography and exercise tolerance testing, were also conducted in diabetic patients, and are still ongoing.
Patients with subclinical atherosclerosis identified by accurate imaging tests can be expected to benefit from preventive treatment because they are at elevated risk for an event. Ideally, decision making as to whether imaging individual patients is beneficial should be based on RCTs comparing preventive measures guided by imaging versus not imaging and evaluating CAD event rates as outcome. Such RCTs are, however, expensive and time-consuming and not always feasible. In the absence of these RCTs, one would want to combine data from trials evaluating the effect of preventive measures with data from cohort studies reporting the association between imaging test results and CAD event rates. Qualitatively weighing and combining the relevant harms and benefits, as was done in the development of the reviewed guidelines, is difficult and may lead to different judgments about net health gains. Disagreements across guidelines can occur for other reasons, including different judgments about which research is relevant; risk of biases in selected research; the applicability of the research findings to the key questions; the relative importance of the anticipated costs; and also poor guideline development processes and conflicts of interests (32). We explored whether the latter 2 influenced the variation in recommendations, but found no evidence for this in the limited set of guidelines reviewed. Quantitatively, as opposed to qualitatively, weighing harms and benefits can be done using decision models that integrate the best-available evidence from multiple sources. Beneficial effects, adverse effects, and incurred costs of preventive treatment and follow-up can be summarized in an incremental cost-effectiveness ratio. In a few of the included guidelines, decision modeling studies were discussed; however, their quality was considered too low for policy making.
The recommended methods of refining CAD risk stratification using imaging test results can be improved by updating existing prediction models (33). None of the guidelines contained recommendations for the use of prediction models combining traditional risk factors and test results to calculate a new risk estimate. Instead, the Framingham-based intermediate risk (10% to 20% 10-year CAD risk) is reclassified to high risk (a 10-year CAD risk >20%), if the test result is positive, rather than updating the risk estimate. This approach has limitations. First, it requires consensus on these risk categories, which is not the case. Second, validity of the reclassified risk might become an issue. A positive test result may not elevate the predicted absolute CAD risk to the level of high risk if the subject was at the lower end of the intermediate risk distribution, for example, if the 10-year CAD risk was between 10% and 15% (34). Reported risk ratios of asymptomatic CAD adjusted for traditional risk factors, which might reclassify individuals, are usually derived from a comparison with a reference group without or with low indication for asymptomatic CAD (34–37). However, converting a risk ratio to absolute risk also depends on the distribution of the risk marker within the general population, which consists of subjects with and without this risk marker (38). Finally, a communication of a refined numerical risk theoretically offers a benefit in informing patients. Thus, we believe that future research should also focus on the value of updating traditional prediction models.
Guidelines on risk assessment by imaging of asymptomatic CAD contain conflicting recommendations. More research, including RCTs, evaluating the impact of imaging on clinical outcomes and costs is needed.
The authors have reported that they have no relationships to disclose.
- Abbreviations and Acronyms
- American College of Cardiology Foundation
- Appraisal of Guidelines Research and Evaluation
- American Heart Association
- coronary artery disease
- Canadian Association of Radiologists
- Canadian Cardiovascular Society
- computed tomography
- magnetic resonance
- National Cholesterol Education Program
- New Zealand Guidelines Group
- positron emission tomography
- randomized controlled trial
- single-photon emission computed tomography
- U.S. Preventive Services Task Force
- Received August 24, 2010.
- Revision received September 28, 2010.
- Accepted October 1, 2010.
- American College of Cardiology Foundation
- Tunstall-Pedoe H.,
- Morrison C.,
- Woodward M.,
- Fitzpatrick B.,
- Watt G.
- UK National Screening Committee
- Wilson J.,
- Jungner G.
- AGREE Collaboration
- Budoff M.J.,
- Achenbach S.,
- Blumenthal R.S.,
- et al.
- Lauer M.,
- Froelicher E.S.,
- Williams M.,
- Kligfield P.
- New Zealand Guidelines Group (NZGG)
- Bluemke D.A.,
- Achenbach S.,
- Budoff M.,
- et al.
- Greenland P.,
- Bonow R.O.,
- Brundage B.H.,
- et al.
- ↵(2002) Third Report of the National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III) final report. Circulation 106:3143–3421.
- Grundy S.M.,
- Cleeman J.I.,
- Merz C.N.,
- et al.
- Hendel R.C.,
- Berman D.S.,
- Di Carli M.F.,
- et al.
- Douglas P.S.,
- Khandheria B.,
- Stainback R.F.,
- et al.
- Hendel R.C.,
- Patel M.R.,
- Kramer C.M.,
- Poon M.
- ↵(2010) U.S. National Institutes of Health clinicaltrials.gov web site. Accessed March 22, 2010, http://clinicaltrials.gov/.
- Shaw L.J.,
- Min J.K.,
- Budoff M.,
- et al.
- Oxman A.D.,
- Glasziou P.,
- Williams J.W. Jr.
- Steyerberg E.W.
- Vliegenthart R.,
- Oudkerk M.,
- Hofman A.,
- et al.
- Balady G.J.,
- Larson M.G.,
- Vasan R.S.,
- Leip E.P.,
- O'Donnell C.J.,
- Levy D.