Author + information
- Received February 1, 2005
- Revision received May 21, 2005
- Accepted May 31, 2005
- Published online January 3, 2006.
- Rainer Hoffmann, MD⁎,⁎ (, )
- Stephan von Bardeleben, MD†,
- Jaroslaw D. Kasprzak, MD‡,
- Adrian C. Borges, MD§,
- Folkert ten Cate, MD∥,
- Christian Firschke, MD, FACC¶,
- Stephane Lafitte, MD#,
- Nidal Al-Saadi, MD⁎⁎,
- Stefanie Kuntz-Hehner, MD††,
- Georg Horstick, MD†,
- Christian Greis‡‡,
- Marc Engelhardt, MD§§,
- Jean Louis Vanoverschelde, MD, FACC∥∥ and
- Harald Becher, MD¶¶
- ↵⁎Reprint requests and correspondence:
Dr. Rainer Hoffmann, Medical Clinic I, University RWTH Aachen, Pauwelsstraße 30, 52074 Aachen, Germany.
Objectives To define the use of cineventriculography, cardiac magnetic resonance imaging (cMRI), and unenhanced and contrast-enhanced echocardiography for detection of left ventricular (LV) regional wall motion abnormalities (RWMA).
Background Detection of RWMA is integral to the evaluation of LV function.
Methods In 100 patients, cineventriculography and unenhanced and contrast-enhanced echocardiography were performed. Fifty-six of the patients underwent additional cMRI. RWMA were assessed referring to a 16-segment model for cMRI, unenhanced and contrast echocardiography. Cineventriculography was evaluated on a 7-segment model. Hypokinesia in one or more segments defined presence of RWMA. Interobserver agreement among three readers was determined within each imaging modality. Intermethod agreement between imaging modalities was analyzed. A standard of truth for the presence of RWMA was obtained by an independent expert panel decision (EPD) based on clinical data, electrocardiogram, coronary angiography, and blinded information from the imaging modalities.
Results Sixty-seven patients were found to have an RWMA by EPD. Interobserver agreement expressed as kappa coefficient was 0.41 (range 0.37 to 0.44) for unenhanced echocardiography, 0.43 (range 0.29 to 0.79) for cMRT, 0.56 (range 0.44 to 0.70) for cineventriculography, and 0.77 (range 0.71 to 0.88) for contrast echocardiography. Contrast enhancement compared to unenhanced echocardiography improved agreement of echocardiography related to cMRI (kappa 0.46 vs. 0.29) and related to cineventriculography (kappa 0.59 vs. 0.28). Accuracy to detect EPD-defined RWMA was highest for contrast echocardiography, followed by cMRI, unenhanced echocardiography, and cineventriculography.
Conclusions Analysis of RWMA is characterized by considerable interobserver variability even using high-quality imaging modalities. Interobserver agreement on RWMA and accuracy to detect panel-defined RWMA is good using contrast echocardiography.
Although the potential of different imaging modalities to accurately determine global LV function has been extensively evaluated, the accuracy of commonly used imaging methods to define regional left ventricular (LV) function is much less known. However, the analysis of regional systolic function is a similarly important part in the evaluation of LV function, because regional wall motion abnormalities (RWMA) defined at rest as well as under stress conditions carry significant diagnostic, therapeutic, and prognostic implications (1–3). Echocardiography, cineventriculography, and cardiac magnetic resonance imaging (cMRI) are used to detect RWMA (4–7). Each of these imaging methods depends on a subjective visual analysis of regional systolic LV function incorporating assessment of endocardial inward motion and additionally systolic wall thickening with echocardiography and cMRI. Poor imaging conditions are major reasons compromising the determination of RWMA (8–10). Echocardiographic studies are particularly vulnerable by possible impairment of the image quality (11). In single-center studies, contrast enhancement was reported to improve interobserver agreement (IOA) and intermethod agreement between echocardiography and cMRI regarding the detection of RWMA (11–13). Although endocardial border delineation tends to be good for cMRI studies, the subjective nature in the definition of RWMA is not overcome and recent cMRI studies have indicated considerable interobserver variability using this imaging technique (14,15).
The objective of this multicenter study was: 1) to determine the IOA in the definition of RWMA in a comparative manner for each of the applied imaging modalities (unenhanced echocardiography, contrast echocardiography, cineventriculography, and cMRI); 2) to determine the agreement between different imaging modalities in the definition of RWMA; and 3) to assess the adequacy of determined RWMA related to a standard of truth on regional LV function, as defined by an expert panel based on clinical, electrocardiographic, angiographic, and imaging data. The design of this study allowed a direct comparison of the techniques on the same patients. Blinded on-site and off-site readings using experienced independent core laboratories were performed for each imaging technique according to prospectively defined standards.
This multicenter open-label study utilizing intrasubject comparison assessed unenhanced and contrast-enhanced echocardiography as well as biplane cineventriculography and cMRI for determination of RWMA. Coronary angiography for suspected coronary artery stenosis was performed in all patients. All imaging modalities had to be performed within 72 h with the patient being in stable hemodynamic conditions.
Recommendations on the performance of image acquisition were prospectively defined for each imaging modality to secure uniform and interpretable image datasets from all participating institutions. Adherence to the predefined imaging protocols was monitored during the enrollment period of this multicenter trial.
Analysis of image datasets for RWMA was performed for each imaging technique by one on-site reader (OnR) as well as two independent off-site readers (OffR) not affiliated with the participating centers and who were unaware of the clinical data and the results of the other imaging techniques. Recommendations on the evaluation of regional LV function for each imaging technique were prospectively defined and provided as guidelines both to the OnR at the study sites and to the unaffiliated OffR using independent core laboratories (Appendix). For each analyzed segment, regional wall motion was defined as normokinetic, hypokinetic, akinetic, or dyskinetic. Whenever the regional function could not be defined owing to insufficient image quality, the function was assumed to be normal.
The study was conducted according to good clinical practice and in compliance with local regulatory requirements. The research protocol was approved by the applicable central and local institutional ethics committees. All patients gave written informed consent to participate in the study.
Eight European centers experienced in the applied imaging techniques enrolled 100 patients planned for routine coronary angiography. Patients with acute myocardial infarction were excluded. At each center, consideration was taken in patient enrollment for an even distribution within three predefined ejection fraction groups (>55%, 35% to 55%, and <35%) based on results from cineventriculography. Inclusion into the study required sinus rhythm and an interpretable cineventriculography.
Two-dimensional echocardiography using tissue harmonic imaging for unenhanced and contrast-specific imaging for contrast-enhanced echocardiography was performed with a commercially available ultrasound scanner (Sonos 5500; Philips, Andover, Massachusetts). Written recommendations were provided for the uniform use of equipment presets, imaging conventions, imaging sequence, and annotations. Apical four-, two-, and three-chamber views were acquired without and with contrast enhancement. For unenhanced imaging, harmonic imaging (mechanical index [MI] 1.6, gain 50%, compression 70%) was used, whereas for contrast-specific imaging a low MI of 0.3 was preselected (gain 60%, compression 15%). Optimization of imaging conditions for endocardial border definition was performed by modulation of transmit power, gain, focus, and dynamic range as required.
For contrast-enhanced echocardiography, a 20-gauge intravenous catheter was introduced into the right antecubital vein. Sulphur hexafluoride microbubbles (SonoVue, Bracco Imaging, Milan, Italy) were administered with a starting infusion rate of 1 ml/min followed by subsequent rate adjustments and additional bolus injections if required to reach homogeneous LV cavity opacification without attenuation.
Analysis of unenhanced and contrast-enhanced echocardiograms by the OnR and OffR was performed in sequence. After finalization of unenhanced image evaluation, subsequent separate evaluation of contrast-enhanced images was performed. For each of the 16 LV segments defined by the American Society of Echocardiography, regional systolic LV function was determined (5).
Standard biplane cineventriculography was performed using a 30° right anterior oblique projection and a 60° left anterior oblique projection with injection of at least 30 ml contrast medium at a flow rate of 12 to 14 ml/s. Frame rate was set at 30 Hz. Regional systolic LV function was determined for each of seven segments (anterolateral, anterobasal, apical, posterobasal, posterolateral, diaphragmal, and septal) (4).
Electrocardiogram (ECG)-triggered cMRI investigations using a steady-state free-precession (SSFP) sequence at a field strength of 1.5 T during breathhold were performed for cardiac function assessment at five of the participating centers with on-site cMRI facilities using either Siemens Sonata (Siemens, Erlangen, Germany) or Philips Gyroscan (Philips, Eindhoven, the Netherlands) systems. Four-chamber, two-chamber, and three-chamber views were acquired with a temporal resolution of ≤50 ms using a special volume-adapted surface coil. Regional systolic function was determined for each of 16 segments considering wall thickening during systole as well as endocardial inward motion (6).
Definition of true regional LV function
To define a standard of truth for the presence of RWMA, a consensus decision was made for each patient between two independent panelists based on clinical data (known cardiomyopathy, history of myocardial infarction, and prior revascularization procedure), ECG, coronary angiography, and results of all image readings (from OnR and OffR). To define the standard of truth, the two panelists adhered to a predefined decision algorithm (Fig. 1).They had to consider primarily the clinical information and then the results of the given reads. Known cardiomyopathy, history of myocardial infarction in combination with ECG abnormalities, angiographically proven significant coronary artery disease, or previous coronary revascularization and RWMA in at least one imaging method by at least two readers indicated evidence for RWMA. No history of myocardial infarction in combination with normal ECG, no previous coronary revascularization, angiographic exclusion of coronary artery disease, and in all imaging modalities no RWMA by at least two readers was indicative for no RWMA. In case no decision could be made based on these data, the results of all imaging reads were considered. In cases with cMRI imaging, at least 10 points (one point per reader and imaging modality) on a consensus scale of 12 points had to be reached in order to obtain a result on the presence of a RWMA. This was obtained by consensus of at least 10 of the 12 readings (four imaging modalities and three readers per imaging modality). In cases without cMRI, at least 7 points on a consensus scale of 9 points (three imaging modalities and three readers per imaging modality) had to be reached. Furthermore, clinical data had to be compatible with this result.
In 15 cases the achieved consensus score was inconclusive to determine either the presence or the absence of a RWMA. In these cases the two off-site panelists were provided with all imaging cine loops for reassessment. Subsequently, the panelists reached a consensus agreement in all cases.
Statistical analysis was performed using the SPSS software package (SPSS, Chicago, Illinois). Continuous variables are presented as mean ± SD. The Cohen kappa coefficient was calculated first to evaluate IOA for each pair of observers (16). To evaluate IOA among three readers within each imaging modality, the mean kappa and its 95% confidence interval was calculated. A generalization of the Cohen reliability kappa for multiple readers and two categories outcome was estimated to support the results of mean kappa (17). Kappa was also obtained to evaluate intermethod agreement on detection of RWMA evaluated by one OffR of each imaging modality. The same analyses were performed for the agreement between OffR/OnR and the panel decision in terms of RWMA within each individual imaging modality. The kappa coefficient of agreement was graded as follows: 0 to 0.2 = poor to slight; 0.21 to 0.4 = fair; 0.41 to 0.6 = moderate; 0.61 to 0.8 = substantial; and 0.81 to 1.0 = nearly perfect. To evaluate the diagnostic performance of each imaging modality in terms of detection of RWMA, sensitivity, specificity, and accuracy were estimated using the panel decision as gold standard. Sensitivities, specificities and accuracies were compared using McNemar test for dependent samples. A value of p < 0.05 was considered statistically significant.
One hundred patients (mean age 62.5 ± 11.5 years; 77 male) were included in this study. Patient characteristics are given in Table 1.Cineventriculography and unenhanced and contrast-enhanced echocardiography were performed in all patients. Patient characteristics of the 56 patients with cMRI were similar to the total patient population with regard to gender, age, prior myocardial infarction, and frequency of subjects in each of the ejection fraction groups, as defined by cineventriculography.
The SonoVue infusion rate needed for optimal image quality (Fig. 2)was 1.35 ± 0.44 ml/min. After receiving the contrast agent, two nonserious adverse events of mild intensity were reported in two subjects: Single ventricular extrasystoles were observed in one patient during contrast imaging, and another patient reported malaise approximately two hours after echocardiography with transient decrease in blood pressure. The latter event was attributed to initiation of beta-blocker treatment after echocardiography. Both resolved spontaneously without any sequel.
Regional wall motion
Table 2displays the frequency of RWMA detected with the four different imaging techniques for all patients as well as the subgroup of patients with cMRI. Frequency of detected RWMA was similar between the three readers for unenhanced and contrast-enhanced echocardiography and cineventriculography. For cMRI, OffR1 detected the lowest rate of RWMA among all readers, and OffR2 detected a remarkably high rate of detected RWMA, indicating an “overread” (Table 2).
Interobserver agreement on RWMA for each imaging technique
Table 3displays the kappa values between the one OnR and the two OffR for each imaging modality on the presence of RWMA. The kappa value was lowest for unenhanced echocardiography. It was significantly higher for contrast-enhanced echocardiography compared to unenhanced echocardiography. IOA for cineventriculography was in an intermediate range. The IOA for cMRI ranged from a high 0.79 between OnR and OffR1 to low values if OffR2 was considered, probably owing to the considerable overread of OffR2. There were similar findings in the 56 patients in whom all four imaging techniques were performed (Table 3).
Intermethod agreement on wall motion abnormalities
Table 4displays the kappa values between different imaging modalities calculated for the presence of RWMA. For each imaging modality, results of OffR1 were used. Agreement on the presence of RWMA between cineventriculography and unenhanced echocardiography was only fair. Contrast echocardiography compared to unenhanced echocardiography was associated with improved agreement to cMRI and cineventriculography.
Agreement with consensus definition of regional LV function
Considering clinical data, ECG, coronary angiogram, and the results of all imaging modalities, 67 patients were determined by panel decision to have an RWMA. The kappa value considering the findings of all three readers of a method and agreement with the panel decision on presence of RWMA was highest for contrast echocardiography, at 0.71 (Table 5).Agreement between panel decision and unenhanced echocardiography as well as cineventriculography tended to be low for all three readers of each method. For cMRI the agreement was high with OnR and OffR1, whereas it was low for OffR2 owing to the considerable overread. In the 56 cMRI patients, agreement between the panel decision and contrast echocardiography was higher than the agreement between panel decision and unenhanced echocardiography or cineventriculography (Table 5).
Considering the panel decision on the presence of RWMA as standard, a mean sensitivity, specificity, and accuracy of all three readers in detecting an RWMA was calculated for each imaging method. Sensitivity, specificity, and accuracy for assessment of RWMA was on a high level for all imaging modalities, with a trend towards better diagnostic performance for cMRI and contrast echocardiography (Table 6).For cMRI the mean specificity of only 74.4% was due to a specificity of only 33% for OffR2, whereas the other two readers reached a specificity of 95%. Considering only those 56 patients with all four imaging modalities the mean accuracy in the detection of an RWMA was higher using contrast-enhanced echocardiography compared to unenhanced echocardiography (88.2% vs. 79.5%; p = 0.018) and cineventriculography (88.2% vs. 78.7%; p = 0.018).
The present study demonstrates that: 1) IOA on the presence of RWMA is only moderate using unenhanced echocardiography and cineventriculography, whereas it is considerably higher with contrast-enhanced echocardiography; 2) intermethod agreement on the presence of RWMA is only moderate if unenhanced echocardiography is compared with cineventriculography or cMRI and is higher if contrast enhancement is applied; and 3) contrast-enhanced echocardiography reaches a close agreement in RWMA assessment when compared to an expert panel decision considered as the “standard of truth.”
Regional LV function
Analysis of regional systolic function at rest as well as under stress conditions has considerable implications for patient management and prognosis. In the analysis of regional LV function, different parameters come into play, in particular, endocardial border inward motion and myocardial wall thickening. Interpretation thresholds for rating regional LV function as abnormal are subjective. Interobserver variability is a well known problem in the interpretation of cardiac imaging tests (8). Accuracy and reliability of a diagnostic test are, however, a prerequisite for adequate and consistent patient management and the minimization of operator dependence in the test interpretation is therefore of clinical importance.
This study allows for the first time a direct within-patient comparison of the different methods to evaluate the presence of RWMA using up-to-date technology for all four imaging modalities. For each method, similar analysis conditions were established using one OnR and two OffR at independent core laboratories. The first aspect of this study was the IOA between the different readers within each imaging modality. For cineventriculography the average kappa was 0.56 and thus better than previously indicated in an analysis between 11 observers which reported an average kappa of 0.34 (18). The better IOA in the current study may be due to the fact that all cineventriculograms were performed using current biplane technology and that technical adequacy of cineventriculography was a prerequisite for patient inclusion. For unenhanced echocardiography the average kappa over three readers was 0.41. In stress echocardiographic studies kappa values have been reported to be 0.55 using current unenhanced echocardiographic technology and only 0.37 if image quality is impaired (8,10).
The mean kappa on presence of RWMA for the three cMRI readers was only 0.43 in the current study, which was mainly related to a substantial overread by OffR2 resulting in high interobserver variability with OffR1 and OnR. The IOA between OnR and OffR1 was on a level expected for cMRI, with a kappa of 0.79. Cardiac magnetic resonance imaging is known for its high accuracy and reproducibility in the assessment of LV volumes and ejection fraction owing to the full-volumetric data acquisition and the usually excellent image quality (19,20). However, regarding regional function analysis recent publications have indicated IOA for RWMA to be far from perfect also for cMRI. Paetsch et al. (14) reported from a study on 150 patients undergoing dobutamine stress cMRI a mean kappa of 0.55 on test interpretation among three expert readers at different centers. In another study, which involved only readers from one center, the reported kappa values on interpretation of stress cMRI studies were 0.70 and 0.71 (15). For contrast-enhanced echocardiography, an overall high level of IOA on the definition of RWMA was found. Previous studies had already indicated that contrast enhancement improves analysis of global LV function and has a positive impact on regional function analysis (11–13,21). In a comparative single-center study on 40 subjects between cMRI and echocardiography the agreement in the detection of wall motion abnormalities was 82% using unenhanced imaging and increased to 100% after administration of contrast (11).
The current study results indicate that IOA on regional wall motion assessment is—within a range of 0.41 to 0.77 for averaged kappa values—method and thus image-quality dependent. However, the findings also show that nearly perfect agreement was not achieved with any of the imaging modalities and that reader differences continue to exist even with high image quality owing to the different subjective reader thresholds in the evaluation of RWMA.
The second aspect evaluated in the study was the agreement between different methods in the assessment of RWMA. In the pairwise intermethod comparison, kappa values ranged between 0.28 and 0.60. Intermethod agreement was poor between unenhanced echocardiography and both cineventriculography and cMRI, whereas fair agreement was found for cineventriculography versus cMRI and for contrast-enhanced echocardiography versus all other modalities. The findings of the present study underline the importance to report the applied method used to evaluate a RWMA. This extends previous observations on global LV function analysis (22), that results on LV function assessments are not easily interchangeable between different imaging modalities.
To define a standard of truth on RWMA two experienced cardiologists had to find a consensus for each patient. This approach was selected in order to allow the assessment of accuracy in the definition of RWMA for each of the applied imaging techniques. Accuracy of cMRI was affected by a high false positive rate from a single reader, whereas diagnostic performance was extremely high for the remaining two readers.
For cMRI, limitations in frame rate and the potential migration of myocardial segments through imaging planes may cause difficulties in the reading of regional wall motion. Suboptimal baseline image quality is a potential cause for incorrect wall motion ratings using echocardiography. Administration of contrast seems to overcome the image quality-related limitations and results in more accurate and reliable regional wall motion assessment.
cMRI was performed at only five centers, allowing only 56 patients to be included. This reflects the limited number of centers able to perform all applied imaging modalities. However, there were no differences in patient characteristics and frequency of RWMA defined by echocardiography between all patients and the subgroup with cMRI.
The number of segments used to evaluate regional LV function by cineventriculography was only 7 as compared to 16 for the other imaging modalities. This is in part related to the biplane display of LV function instead of the triplane display with echocardiography and cMRI.
There is no objective gold standard for the definition of RWMA with which each imaging modality could be easily compared. We tried to circumvent this problem by defining a “standard of truth” based on a panel decision between two blinded expert cardiologists considering all available information in a well-defined decision algorithm. One might argue that cMRI should be used as the gold standard. The considerable variability between the three cMRI readers indicates that cMRI has limitations in this respect. However, in all of the 56 cMRI patients the panel decision was in agreement with at least two of the three cMRI readers.
All readers in this study were trained experts. The reported reader agreement and accuracy to detect RWMA is likely to reflect the best possible level whereas it may not reflect such a setting with less trained readers.
Analysis of RWMA is characterized by a considerable interobserver variability even using high-quality imaging modalities, owing to different reader thresholds in the evaluation of RWMA. Unenhanced echocardiography and cineventriculography have the lowest IOA in the evaluation of RWMA. These methods are also associated with a low intermethod agreement with other imaging modalities on RWMA. Contrast-enhanced echocardiography as compared to unenhanced echocardiography significantly improves IOA on the presence of RWMA and results in a high accuracy to detect consensus-defined regional function abnormalities.
We thank Bracco-Altana (Konstanz, Germany) for sponsoring the study, Ningyan Shen, MD, PhD (Biometrics, Bracco Diagnostics, Princeton, NJ), for the expert advice in the statistical analyses, Medidata (Konstanz, Germany) for the data management and analyses, Guus Kroes (Pie-Medical Imaging, Maastricht, The Netherlands) for supporting CAAS-LVA software, and Heinrich Beckermann and Sybille Pajain (Philips Medical Systems, Boeblingen, Germany) for their valuable technical support during the conduct of this study.
For a list of the participating institutions and investigators for the SonoVue Study Group, please see the online version of this article.
Christian Greis and Dr. Engelhardt are employed by Bracco. Bracco-Altana (Konstanz, Germany) sponsored this study.
- Abbreviations and Acronyms
- cardiac magnetic resonance imaging
- expert panel decision
- interobserver agreement
- left ventricle/ventricular
- regional wall motion abnormality
- off-site reader
- on-site reader
- Received February 1, 2005.
- Revision received May 21, 2005.
- Accepted May 31, 2005.
- American College of Cardiology Foundation
- Wang T.J.,
- Evans J.C.,
- Benjamin E.J.,
- Levy D.,
- LeRoy E.C.,
- Vasan R.S.
- Marwick T.H.,
- Mehta R.,
- Arheart K.,
- Lauer M.S.
- Underwood S.R.,
- Rees R.S.O.,
- Savage P.E.,
- et al.
- Peshock P.M.,
- Rokey R.,
- Malloy G.M.,
- et al.
- Hoffmann R.,
- Lethen H.,
- Marwick T.,
- et al.
- Nagel E.,
- Lehmkuhl H.-B.,
- Bocksch W.,
- et al.
- Hoffmann R.,
- Marwick T.H.,
- Poldermanns D.,
- et al.
- Hundley W.G.,
- Kizilbash A.M.,
- Afridi I.,
- et al.
- Reilly J.,
- Tunick P.,
- Timmermans R.,
- Stein B.,
- Rosenzweig B.,
- Kronzon I.
- Thomson H.,
- Basmadjian A.J.,
- Rainbird A.J.,
- et al.
- Paetsch I.,
- Jahnke C.,
- Ferrari V.A.,
- et al.
- Janssen C.,
- Kuijpers D.,
- van Dijkman P.,
- Tineke W.,
- Oudkerk M.
- Shoukri M.M.
- De Rouen T.A.,
- Murray J.A.,
- Owen W.
- Mogelvang J.,
- Lindvig K.,
- Sondergaard L.,
- et al.
- Hoffmann R.,
- von Bardeleben S.,
- ten Cate F.,
- et al.
- Bellenger N.G.,
- Burgess M.I.,
- Ray S.G.,
- et al.