Diagnostic Performance of Multislice Spiral Computed Tomography of Coronary Arteries as Compared With Conventional Invasive Coronary AngiographyA Meta-Analysis
Author + information
- Michèle Hamon, MD⁎,
- Giuseppe G.L. Biondi-Zoccai, MD§,
- Patrizia Malagutti, MD∥,
- Pierfrancesco Agostoni, MD¶,
- Rémy Morello, MD‡,
- Marco Valgimigli, MD† and
- Martial Hamon, MD†,⁎ (hamon-m{at}chu-caen.fr)
- ↵⁎Reprint requests and correspondence:
Prof. Martial Hamon, Service des Maladies du Coeur et des Vaisseaux, Centre Hospitalier Universitaire de Caen, Avenue Côte de Nacre 14033 Caen, Normandy, France.
Abstract
Objectives This study was designed to define the current role of multislice spiral computed tomography (MSCT) for the diagnosis of coronary artery disease (CAD) using a meta-analytic process.
Background Multislice spiral computed tomography has recently been proposed as an alternative to conventional coronary angiography (CA) for the diagnosis of CAD.
Methods Using Medline, we identified 29 studies (2,024 patients) evaluating CAD by means of both MSCT (≥16 slices) and conventional CA before July 2006. After data extraction the analysis was performed according to a random-effects model.
Results The per-segment analysis pooled the results from 27 studies corresponding to a cumulative number of 22,798 segments. Among unassessable segments, 4.2% were excluded from the analysis and 6.4% were classified at the discretion of the investigators, underscoring the shortcomings of MSCT. With this major limitation, the per-segment sensitivity and specificity were 81% (95% confidence interval [CI] 72% to 89%) and 93% (95% CI 90% to 97%), respectively, with positive and negative likelihood ratios of 21.5 (95% CI 13.1 to 35.5) and 0.11 (95% CI 0.06 to 0.21), respectively, and positive and negative predictive values of 67.8% (95% CI 57.6% to 78.0%) and 96.5% (95% CI 94.7% to 98.3%), respectively. As expected, the per-patient analysis has shown an increased sensitivity of 96% (95% CI 94% to 98%) but a decreased specificity of 74% (95% CI 65% to 84%).
Conclusions Multislice spiral computed tomography has shortcomings difficult to overcome in daily practice and, at the more clinically relevant per-patient analysis, continues to have moderate specificity in patients with high prevalence of CAD. Studies evaluating the diagnostic performance of the newest generation of MSCT, including patients with low to moderate CAD prevalence, will be critical in establishing the clinical role of this emerging technology as an alternative to CA.
Coronary artery disease (CAD) is the leading cause of death and disability in the U.S. and other Western countries. Conventional coronary angiography (CA) is currently the reference test for coronary artery lumen assessment, and its use has been steadily increasing over the last decade (1). The CA test comes at a considerable cost and, although complications may be infrequent, cardiac catheterizations account for well known procedure-related morbidity (2). Recent advances in multislice computed tomography (MSCT) seem to respond adequately to the need for a noninvasive and reliable assessment of the coronary artery lumen. Several studies have compared CA and MSCT; however, each of these studies was based on a particularly limited sample size, meaning that a reliable and unbiased estimate of the performance of MSCT compared with CA in a reasonably large data set is lacking. To overcome this issue and to provide an evidence-based evaluation of the clinical utility of MSCT, we performed a comprehensive meta-analysis of all currently available studies comparing MSCT and CA for the detection of CAD in native coronary arteries.
Methods
Search strategy
Database searches for English-language articles published from January 2002 to July 2006 were performed in Medline. We combined the medical subject headings for computed tomography, multislice computed tomography, and coronary angiography with the exploded term coronary artery disease and scanned references in retrieved articles and reviews. The retrieved studies were carefully examined to exclude potentially duplicate or overlapping data. Meetings abstracts were excluded because they could not provide adequately detailed data and their results may not have been final. Only papers evaluating the presence of significant obstructive CAD in native coronary arteries by both conventional invasive CA and MSCT in the same subjects were included. Studies were eligible regardless of whether they referred to subjects with suspected or proven CAD.
Study eligibility
We included a study if: 1) it used MSCT as a diagnostic test for obstructive CAD, with >50% diameter stenosis selected as the cut-off criterion for significant CAD, using conventional invasive angiography as the reference standard; 2) it used the newest generation of MSCT (≥16 slices); and 3) it reported cases in absolute numbers of true positive (TP), false positive (FP), true negative (TN), and false negative (FN) results or presented sufficiently detailed data for deriving these figures. Studies were excluded if they were performed: 1) only in patients after coronary artery bypass graft surgery; 2) after percutaneous coronary intervention for long-term stent patency assessment; 3) in a subset of patients with prior heart transplant; or 4) with fewer than 30 enrolled patients.
Data extraction
The following information was extracted from each study: first author, year of publication, and journal; study population characteristics including sample size (number of subjects evaluated with both tests, number of patients excluded); number of patients with documented CAD; gender; mean age (and standard deviation); mean heart rate (and standard deviation); relative timing of the 2 imaging procedures and whether or not evaluation of one test was blind to the result of the other and to the clinical condition of the tested subject; technical characteristics of the MSCT, including type and brand of machine used; and rate of beta-blocker usage. Data were recorded separately, whenever available, at the level of segments, vessels, and subjects. Two investigators performed the data extraction independently. Discrepancies were solved by a third investigator and global consensus. The study quality conformed to the Quadas guidelines (3).
Data synthesis and statistical analysis
Categoric variables from individual studies are presented as n/N (%) and continuous variables are presented as median values. Measures of diagnostic accuracy are reported as point estimates (with 95% confidence intervals [CI]). The main analysis was performed at the coronary artery segment level, because most studies focused on this level of information. Secondary analyses combined the available vessel-level data, considering 4 coronary arteries per patient (left main coronary artery, left anterior descending artery, circumflex artery, and right coronary artery) and patient-level data.
By means of TP, TN, FP, and FN rates we computed sensitivity, specificity, positive and negative likelihood ratios, and diagnostic odds ratios (4). Although sensitivity and specificity are well known as measures of diagnostic accuracy, their results may be influenced by the prevalence of disease in tested subjects. The positive likelihood ratio (the ratio between sensitivity and 1 − specificity) provides an estimate of the probability of a positive test in a patient with disease, and the negative likelihood ratio (the ratio between 1 − sensitivity and specificity) gives an estimate of the probability of a negative test among diseased subjects. Both likelihood ratios are roughly independent from prevalence rates, and there is consensus that a positive likelihood ratio of >10 and a negative likelihood ratio of <0.1 provide reliable evidence of satisfactory diagnostic performance (5). Finally, the information from both positive and negative likelihood ratios can be combined in a single parameter, the diagnostic odds ratio, which is computed as the ratio of positive to negative likelihood ratios and provides an estimate of how much greater the odds of having the disease are for the people with a positive test result than for the people with a negative test result. Although likelihood ratios are the recommended summary statistics for systematic reviews of diagnostic studies, predictive values may also be of interest for clinicians, even if these values vary widely in their dependence on disease prevalence. Such limitations of predictive values notwithstanding, these figures were also computed and reported as exploratory data in this review.
We computed all statistics for individual studies, and then combined them using a random-effects model, weighting each point estimate by the inverse of the sum of its variance and the between-study variance. Between-study statistical heterogeneity was also assessed using the Cochran Q chi-square tests. Because diagnostic parameters are by definition interdependent, independent weighting may sometimes give spurious results and provide biased estimates. Weighted symmetric summary receiver-operating characteristic plots, with pertinent areas under the curve, were computed using the Moses-Shapiro-Littenberg method to overcome this problem of interdependence (6,7).
Sources of clinical and statistical heterogeneity were explored by means of subgroup analyses and meta-regression (7,8). Although the findings of such analyses should be regarded mainly as hypothesis generating, statistical significance may suggest substantial changes in the diagnostic performance of the test under study as the covariate increases. Specifically, we performed stratified per-segment analyses according to publication year, sample size, number of interpretable segments, and 16 versus 64 slices.
Statistical computations were performed with SPSS 11.0 (SPSS, Chicago, Illinois) and Meta-DiSc (9), and significance testing was at the 2-tailed 0.05 level.
Results
The reviewing process is described in Figure 1.Database searches identified 278 potentially relevant citations. After title/abstract assessment, we retrieved 121 studies as complete reports, from which 92 were excluded because: 1) they did not use MSCT or ≥16-slice MSCT; 2) they looked only at grafts or stent patency or at atherosclerotic plaque assessment; 3) they had overlapping data; 4) they were in a language other than English; 5) it was impossible to find or calculate absolute figures from presented data; or 6) no systematic angiographic control was performed. Thus, we included 29 of these studies in the systematic review (10–38).
Flow diagram of the reviewing process.
All studies were published between January 2002 and July 2006. Table 1presents demographic data and details on included studies.
Characteristics of Included Studies
Per-segment meta-analysis
As shown in Figures 2 to 6,⇓⇓⇓⇓per-segment analysis pooled results from 27 studies (Table 2)(1,865 patients after exclusion of 32 patients because of unsuccessful MSCT, corresponding to 22,798 segments after exclusion of 4.2% of segments) and showed that, compared with invasive CA, MSCT had a sensitivity of 81% (95% CI 72% to 89%), a specificity of 93% (95% CI 90% to 97%), a positive likelihood ratio of 21.5 (95% CI 13.1 to 35.5), a negative likelihood ratio of 0.11 (95% CI 0.06 to 0.21), and a diagnostic odds ratio of 189.3 (95% CI 93.5 to 383.4). Overall, these summary estimates confirm the superior (93%) specificity of MSCT in patients undergoing work-up for suspected CAD, even if this occurs at the price of a moderate (81%) value for sensitivity. Indeed, the high specificity of MSCT translates into a positive predictive value (PPV) of 67.8% (95% CI 57.6% to 78.0%) and a negative predictive value (NPV) of 96.5% (95% CI 94.7% to 98.3%), assuming CAD prevalence similar to that found in the studies included in this review (median value of 63.5% and range of 8% to 100%).
Plot and table of per-segment sensitivity of multislice computed tomography-coronary angiography (MSCT-CA) compared with coronary angiography (CA). CI = confidence interval; df = degrees of freedom.
Plot and table of per-segment specificity of MSCT-CA compared with CA. Abbreviations as in Figure 2.
Plot and table of per-segment positive likelihood ratio (LR) of MSCT-CA compared with CA. Other abbreviations as in Figure 2.
Plot and table of per-segment negative likelihood ratio (LR) of MSCT-CA compared with CA. Other abbreviations as in Figure 2.
Plot and table of per-segment diagnostic odds ratio (OR) of MSCT-CA compared with CA. Other abbreviations as in Figure 2.
Per-Segment, Per-Vessel, and Per-Patient Analysis
Statistical heterogeneity was evident for sensitivity (p < 0.001), specificity (p < 0.001), positive likelihood ratio (p < 0.001), negative likelihood ratio (p < 0.001), diagnostic odds ratios (p < 0.001), and predictive values (p < 0.001), which cast a shadow of caution on the previously discussed summary estimates, although the use of random-effects models throughout should have provided relatively robust results even in the presence of statistical inconsistency.
Per-vessel meta-analysis
Per-vessel results were reported in, and thus pooled from, only 8 studies, shown in Table 2(2,726 coronary arteries); therefore, meta-analysis should be viewed in light of the risk of small-study and publication bias. Per-vessel analysis provided the following results: 82% (95% CI 80% to 85%) sensitivity, 91% (95% CI 90% to 92%) specificity, 11.8 (95% CI 6.7 to 20.6) positive likelihood ratios, 0.08 (95% CI 0.02 to 0.32) negative likelihood ratio, and 146.5 (95% CI 31.9 to 671.2) diagnostic odds ratio, with 0.97 symmetric area under the curve. Pooled PPV for per-vessel analysis was 81% (95% CI 78% to 83%), whereas NPV was 92% (95% CI 91% to 93%). Heterogeneity was significant (p < 0.001) for all of the aforementioned diagnostic performance measures.
Per-patient meta-analysis
Table 2shows that somewhat incomplete reporting was available for per-patient results from 22 studies (1,616 patients), so the risk of small-study and publication bias should not be dismissed. The study of Probst et al. (22) was excluded from the pooled analysis because a 0 value for FP, FN, and TN meant that the variance is infinite and confidence intervals could not be computed. As shown in Figures 7 to 11,⇓⇓⇓⇓⇓pooled per-patient analysis—including 21 studies, corresponding to 1,570 patients—provided the following results: 96% (95% CI 94% to 98%) sensitivity, 74% (95% CI 65% to 84%) specificity, 5.4 (95% CI 3.4 to 8.3) positive likelihood ratio, 0.05 (95% CI 0.03 to 0.09) negative likelihood ratio, and 133.05 (95% CI 57.3 to 308.9) diagnostic odds ratio. Pooled PPV for per-patient analysis was 83% (95% CI 76% to 90%) and pooled NPV was 94% (95% CI 89% to 99%). Heterogeneity was again significant (p < 0.001) for all of the aforementioned diagnostic performance measures.
Plot and table of per-patient sensitivity of MSCT-CA compared with CA. Abbreviations as in Figure 2.
Plot and table of per-patient specificity of MSCT-CA compared with CA. Abbreviations as in Figure 2.
Plot and table of per-patient positive likelihood ratio (LR) of MSCT-CA compared with CA. Other abbreviations as in Figure 2.
Plot and table of per-patient negative likelihood ratio (LR) of MSCT-CA compared with CA. Other abbreviations as in Figure 2.
Plot and table of per-patient diagnostic odds ratio (OR) of MSCT-CA compared with CA. Other abbreviations as in Figure 2.
Cumulative results are summarized in Table 3,and quality assessment for all included studies is shown in Table 4.
Pooled Summary Results
Quality Assessment (QUADAS)
Summary receiver-operating characteristics
Findings for individual diagnostic statistics were confirmed with summary receiver-operating characteristic curves providing a symmetric area under the curve of 0.98 for the per-segment analysis (Fig. 12A)and a symmetric area under the curve of 0.97 for the per-patient analysis (Fig. 12B).
Plot of symmetric summary per-segment (A)and per-patient (B)receiver-operating characteristic of MSCT-CA compared with CA. The receiver-operating characteristic curve provides a graphic display of diagnostic accuracy by plotting 1 − specificity in the horizontal axis and sensitivity in the vertical axis. The pertinent area under the curve (AUC) and Q* statistic (the point where sensitivity and specificity are maximal), both with standard errors (SE), are also included. SROC = summary receiver-operating characteristic; other abbreviations as in Figure 2.
Additional analyses
We explored sources of clinical and statistical heterogeneity by performing per-segment subgroup analysis for the number of slices in each CT scan. Although a 64-slice CT scan should be more accurate than a 16-slice one, we did not find significant results by interaction testing at segment level, even if a trend of 3.35 (95% CI 0.70 to 15.92; p = 0.12) increase in relative diagnostic odds ratio was noted. Specifically, for 16-slice CT scans, we found a sensitivity of 76% (95% CI 63% to 89%), a specificity of 95% (95% CI 94% to 97%), a 22.7 (95% CI 13.7 to 37.8) positive likelihood ratio, a 0.13 (95% CI 0.05 to 0.34) negative likelihood ratio, and a 170.8 (95% CI 57.1 to 510.5) diagnostic odds ratio. For 64-slice CT, we found a sensitivity of 87% (95% CI 80% to 94%), a specificity of 96% (95% CI 95% to 97%), a 22.5 (95% CI 17.8 to 28.4) positive likelihood ratio, a 0.10 (95% CI 0.06 to 0.20) negative likelihood ratio, and a 217.6 (95% CI 117.6 to 402.7) diagnostic odds ratio. The same analysis at patient level showed that 64-slice CT increases significantly the per-patient diagnostic yield of the test. Specifically, the increase in relative diagnostic odds ratio was 6.17 (95% CI 1.27 to 29.97; p = 0.026).
Finally, we performed meta-regression analyses exploring the impact of sample size and publication year on the diagnostic performance of MSCT. Whereas the latter did not disclose significant results, we found a significant interaction between changes in sample size and diagnostic odds ratios in the individual studies (p < 0.05), suggesting that smaller studies were more likely to provide higher diagnostic odds ratios. Although it is possible that these results were due to differences in design, setting, and technique, they may also be indirect evidence of small study bias.
Discussion
In the present study, we focused on the diagnostic performance of the newest generation of MSCT (≥16 slices) to assess the extent and severity of coronary stenoses compared with conventional CA. The main analysis was performed at the segment level, because most studies focused on that level of information. Our meta-analysis confirms the relatively high (93%) specificity of MSCT, which occurs at the price of a moderate (81%) value for sensitivity. This high specificity of MSCT-CA may be translated in the future into a clinically useful tool in assessable coronary segments; a high NPV of 96.5% would suggest exclusion of CAD in selected subjects considered for conventional CA. However, these optimistic figures were provided in highly selected patients and after exclusion from the analysis of 4.2% of scanned segments, with an additional 6.4% of unassessable segments classified at the discretion of the investigators, and favoring the positive perception of MSCT as a valuable diagnostic tool. In addition to post hoc exclusion of some segments showing poor image quality, such as calcium-blooming artifacts, the great majority of studies pre-specified post-acquisition exclusion criteria based on reference vessel size. Keeping in mind this major restriction, the present meta-analysis demonstrates that MSCT-CA provides, for the per-segment analysis, across a wide range of diverse clinical centers, consistent diagnostic performance in a highly selected study population.
In our secondary analyses, the results are largely consistent and show the anticipated increase in sensitivity and decrease in specificity as one moves from segments to vessels to subjects. This consistent trend in the diagnostic value is also observed with the NPV of 96.5% at segment level dropping to 94% at patient level and the diagnostic odds ratio of 189.3 (95% CI 93.5 to 383.4) at segment level dropping to 133.05 (95% CI 57.3 to 308.9) at patient level. This finding is likely to reflect the fact that studies on MSCT have primarily focused on patients with a high prevalence of CAD, in whom an increase in false-positive observations (related mainly to heavy calcification) can be anticipated, whereas true negative findings are expected to remain low and essentially unaffected as the analysis moves from per-segment to per-patient. However, this major drawback in reporting data in manuscripts and reporting some unassessable segments as negative findings favors the positive perception of MSCT when reading reports and in the present meta-analysis. In real life and in daily practice, the accuracy of MSCT is certainly decreased by the fact that uninterpretable segments must be considered as positive to avoid misdiagnosis for individuals, especially when these unassessable segments are proximal and masked by heavy calcifications. It is noteworthy that the only multicenter study following this rule in the present meta-analysis found the lowest per-segment specificity, 65%, leading to a specificity of only 55% at patient level (27).
The trend toward increasing sensitivity and specificity in moving from 16-slice to 64-slice CT supports active research in the new generation of CT scanners. From this perspective, it is noteworthy that some studies (32,33), based on the newest generation of 64-slice CT, reported results from almost all evaluated segments, irrespective of vessel size and image quality.
Indeed, cardiac imaging using CT is a technically demanding task. Not only is high spatial resolution required for imaging small structures such as the coronary arteries, but high temporal resolution must also be achieved for motion-free imaging of the heart, given heart rates that may range from 50 to more than 100 beats/min (39). To overcome and minimize the effects of this issue, most investigators use beta-blockers, as found in our systematic review and as previously suggested by some reports, which demonstrated a variation in diagnostic performance according to the patient’s heart rate (40). Respiratory motion must also be eliminated for cardiac imaging, so scanning must be performed in a single breath hold. All of these technical issues are critical for optimal acquisition, and even with the remarkable progress of successive generations of scanners providing faster imaging at progressively higher spatial and temporal resolutions, only highly selected patients can be explored by MSCT for CAD assessment.
Clinical implications and cautionary notes
It should be emphasized that the observed per-segment high specificity has been obtained in patients selected to undergo CA and who are, therefore, presenting with reasonably high probability of CAD. The median prevalence of CAD was as high as 63.5% (95% CI 8% to 100%) among the included studies. Whether the performance of MSCT in terms of negative predictive value can be reproduced in patients at lower prevalence of CAD remains to be assessed. Moreover, patients with a high probability of CAD were selected based on several parameters, including regular and controlled heart rate, renal function, breath-hold capacity, hemodynamic status, and, often, previous coronary instrumentation. This highlights the limited external validity of the present findings and calls for studies evaluating the diagnostic performance of MSCT in a less highly selected patient population before its application in the clinical setting can be suggested as an alternative to CA.
The relatively high radiation dose received with MSCT, compared with CA, should also be acknowledged. The effective radiation dose varies based on the presence of the tube current modulation, ranging from 5.4 to 16.3 mSv for 16-slice CT and from 10 to 21.4 mSv for 64-slice CT in the papers where this information was provided. The effective radiation dose for invasive CA is known to be in the range of 2 to 5 mSv (41,42).
It is also important to acknowledge that if the MSCT detection of coronary calcification results frequently in a false positive classification when CA fails to identify significant luminal narrowing in that segment, from a clinical point of view this finding adds valuable prognostic information. Indeed the presence of coronary calcification has been associated with future coronary events and can justify aggressive risk factor reduction therapy (43–45).
Study limitations
As mentioned in the preceding, substantial statistical heterogeneity has been documented, casting a shadow of caution on the results and interpretation of these estimates of comprehensive, pooled effects. The well known tendency toward publication bias favoring studies with positive and encouraging results also complicates comprehensive evaluation. In the present meta-analysis, data abstraction and quality assessment were done by independent reviewers and, in the case of any divergences, resolution was made by consensus. Therefore, the interoperator agreement could not be quantitatively assessed. We should also acknowledge that not all reports provided complete data concerning subject and vessel levels. More rigorous reporting of future clinical research on coronary artery imaging technologies should be encouraged.
Conclusions
Multislice spiral computed tomography has shortcomings difficult to overcome in daily practice and, at the more clinically relevant per-patient analysis, remains with moderate specificity in patients with high prevalence of CAD. Indeed, an increase in sensitivity and a decrease in specificity as one moves from segments to vessels to patients have been observed in existing studies, with a subset of patients with high prevalence of CAD selected to validate the MSCT diagnostic performance. Studies evaluating the diagnostic performance of the newest-generation MSCT, including patients with low to moderate CAD prevalence undergoing a comprehensive coronary artery tree evaluation, will be critical in establishing the clinical role of this emerging technology as an alternative to CA.
Acknowledgments
Drs. Brunner-La Rocca and C. Kaiser are kindly acknowledged for providing additional data.
- Abbreviations and Acronyms
- AUC
- area under the curve
- CA
- coronary angiography
- FN
- false negative
- FP
- false positive
- MSCT
- multislice computed tomography
- NPV
- negative predictive value
- PPV
- positive predictive value
- TN
- true negative
- TP
- true positive
- Received July 5, 2006.
- Revision received August 16, 2006.
- Accepted August 26, 2006.
- American College of Cardiology Foundation
