Author + information
- Received September 10, 1996
- Revision received February 10, 1997
- Accepted March 12, 1997
- Published online July 1, 1997.
- David A Danford, MD, FACCAB,*,
- Ameeta B Martin, MD, FACCAB,
- Scott E Fletcher, MD, FACCAB,
- Carl H Gumbiner, MD, FACCAB,
- John P Cheatham, MD, FACCAB,
- Philip J Hofschire, MD, FACCAB and
- John D Kugler, MD, FACCAB
- ↵*Dr. David Danford, 8301 Dodge Street, Omaha, Nebraska 68114-4114.
Objectives. This study was undertaken to determine the accuracy of expert examination for ventricular septal defect (VSD) among children with a heart murmur.
Background. Because the frequency and nature of errors that might be made by reliance solely on expert examination for diagnosis of VSD are speculative, the role of echocardiography in such diagnosis is controversial.
Methods. Two hundred eighty-seven consecutive previously unevaluated pediatric subjects were enrolled in the study. For each child, the pediatric cardiologists prospectively recorded a working diagnosis and their level of confidence in the diagnosis, categorizing any VSD diagnosed as small or moderate to large. After echocardiography, VSDs were subcategorized by location and requirement for treatment as minor, intermediate or major. Receiver-operating characteristic (ROC) curves described the accuracy of the clinical examination.
Results. Seventy-three subjects had a VSD (minor in 52, intermediate in 10 and major in 11). ROC areas (1.0 = perfect discrimination, 0.5 = indiscriminate) were minor VSD 0.92 ± 0.02 and major/intermediate VSD 0.69 ± 0.07 (p = 0.0016). Four of 52 minor VSDs were not identified at any level of suspicion; the clinical diagnoses were moderate to large VSD in two patients and atrial septal defect and unlimited differential diagnosis in one patient each. Fourteen of 235 patients without a minor VSD were believed with confidence to have a small VSD, but the final diagnosis was intermediate VSD in 4, innocent murmur in 3, major VSD in 2, pulmonary stenosis in 2 and subaortic membrane, atrial septal defect and mitral regurgitation in 1 patient each.
Conclusions. Almost all minor VSDs are recognized without echocardiography; however, errors can occur even when an expert examiner is confident. Clinical recognition of an intermediate or major VSD is less accurate than clinical recognition of a minor VSD. Failure to distinguish VSDs of major or intermediate importance from minor VSDs is a weakness of the expert clinical examination.
(J Am Coll Cardiol 1997;30:243–6)
Echocardiography is commonly performed to evaluate heart murmurs in children (). However, the high cost of the procedure invites comparisons of diagnostic accuracy of echocardiography with that of less costly methods of diagnosis (). Ventricular septal defect (VSD) produces a characteristic murmur () often recognized with confidence by the experienced examiner (). Whereas it is well documented ([5–7]) that the pediatric cardiologist’s clinical examination is highly sensitive and specific for distinguishing innocent from pathologic murmurs, such careful investigation has not previously analyzed the accuracy of the clinical examination for VSD. Some investigators () propose that the clinical examination is sufficient for management of VSD without echocardiography in some settings; however, the accuracy of the clinical examination for congenital heart disease in general has been disturbingly low in selected patient groups with a high prevalence of disease ([8, 9]). To assess the validity of a nonechocardiographic initial diagnostic approach to VSD, this investigation attempted to determine the sensitivity and specificity of the pediatric cardiologist’s clinical examination for identification of VSD.
Entry into the study was limited to outpatients under the age of 21 years, without prior echocardiography or pediatric cardiology consultation, who appeared for evaluation of a heart murmur in the pediatric cardiology clinics at University of Nebraska Medical Center, Childrens’ Memorial Hospital (Omaha), St. Elizabeth Hospital (Lincoln) or any of the nine outreach clinics staffed by this pediatric cardiology group between December 1, 1994 and August 1, 1996. One of the authors—all certified by the sub-Board of Cardiology, American Board of Pediatric Examiners, with 1 to 21 years of experience after pediatric cardiology fellowship—examined the patients with or without use of chest X-ray films, electrocardiograms or pulse oximetry. An echocardiogram was arranged for 287 of these patients for further evaluation of the murmur, and these patients were enrolled consecutively and prospectively without further selection or exclusion. No echocardiograms were performed that were not, in the opinion of the evaluating cardiologist, clinically indicated for patient care. Complete two-dimensional, Doppler and color Doppler echocardiographic studies were performed and subsequently interpreted by one of the authors. The echocardiographic identification of VSD was made by using well established imaging and Doppler diagnostic criteria (). VSDs were subcategorized as 1) minor(muscular VSDs that required no pharmacotherapy for pulmonary overcirculation and no surgical repair), 2) intermediate(not muscular in location, but not requiring surgical closure or medication for heart failure), or 3) major(requiring surgical closure or medication for heart failure, regardless of location).
1.2 Data Collection.
The cardiologist, without knowledge of the echocardiographic results, prospectively categorized the preechocardiographic diagnosis as follows. If the cardiac lesion could be identified with confidence (and the echocardiogram was to be performed to obtain more detailed anatomic and physiologic data), the cardiologist recorded a single exclusive preechocardiographic diagnosis. Alternatively, if the preechocardiographic diagnosis was uncertain, but could be reasonably limited to three or fewer possibilities, the cardiologist listed the possibilities in descending order of likelihood. When the clinical diagnosis was so uncertain that it could not be limited to three or fewer possibilities, this circumstance was duly noted without a listed differential diagnosis. When the clinical diagnosis was VSD, the cardiologist indicated whether or not the defect was thought to be small.
1.3 Data Analysis.
The relation between the pediatric cardiologist’s clinical diagnosis of small VSD and the final diagnosis of minor VSD was determined at six levels of diagnostic certainty: Small VSD was 1) a single exclusive preechocardiographic diagnosis, 2) listed first on a limited differential, 3) listed second on a limited differential, 4) listed third on a limited differential, 5) implied because no limited differential diagnosis could be generated, and 6) not listed at all on a limited differential diagnosis. A receiver-operating characteristic (ROC) curve (plotting true positive ratio against false positive ratio) was constructed to measure how well the cardiologist’s diagnosis of small VSD distinguished minor VSD from other conditions over the spectrum of clinical diagnostic certainty. A similar set of calculations using the cardiologist’s clinical diagnosis of moderate to large VSD yielded an ROC curve for discrimination of intermediate and major VSDs from other conditions. Areas under the curves were calculated and compared by using nonparametric methods. Instances in which the clinical impression misdiagnosed VSD as a different condition, and those in which the clinical impression falsely diagnosed another condition as VSD, were identified and tallied. Specific discordances between clinical impression of VSD size and final category of VSD significance were identified and tallied.
The mean age of the study group of 287 subjects was 2.72 ± 4.23 years; however, the median of this skewed age distribution was only 0.71 year. There was a slight male predominance (55.4%). Seventy-three (25%) of the 287 had VSD. Fifty-two (71%) of the 73 with VSD had minor VSD. The mean age of patients with minor VSD was 1.32 ± 3.05 years (median 0.15). Ten patients (mean age 0.42 ± 0.68 year, median 0.19) had VSD of intermediate significance. All 10 had perimembranous VSD managed with observation only. Eleven patients (mean age 0.14 ± 0.13 year, median 0.12) had major VSD. This group included 10 patients with perimembranous VSD, of whom 7 had surgical closure of the defect and 3 were treated with medicines for heart failure symptoms; 1 patient with muscular VSD received medical management. There were no subarterial (supracristal) defects in this series.
2.2 Minor significance VSD.
2.2.1 1) Small VSD as the sole diagnostic consideration.
Of the 52 cases of minor VSD, 41 were identified with small VSD as the sole diagnostic consideration (by this most stringent standard, sensitivity = 79%). Only 14 of the 235 subjects without minor VSD were identified by the cardiologist as having small VSD as the sole diagnostic consideration (specificity = 94% by this most lenient standard). In 3 of the 14 cases the echocardiographic findings were normal, leaving open the possibility that the echocardiogram simply failed to detect a tiny muscular VSD. If so, the specificity of the cardiologist’s confident assertion that a small VSD was present could be as high as 95%. The 11 patients with a confident clinical diagnosis of small VSD who did not have minor VSD were found to have intermediate VSD (n = 4), major VSD (n = 2), pulmonary stenosis (n = 2), subaortic membrane (n = 1), mitral regurgitation (n = 1) and atrial septal defect (n = 1).
2.2.2 2) Small VSD Appearing on a Differential Diagnostic List.
Forty-eight of the 52 minor VSDs were listed somewhere on the differential diagnosis before echocardiography as small VSD (by this most lenient standard, sensitivity = 92%). The four minor VSDs not so recognized were present when the clinical diagnosis was large VSD (n = 2), atrial septal defect (n = 1) and unlimited differential (n = 1). In 235 patients with some condition other than minor VSD, small VSD correctly did not appear anywhere on the differential diagnosis in 193 (specificity by this most stringent standard = 82%).
2.2.3 3) ROC Curve.
Sensitivity and specificity of the clinical examination for the detection of minor VSD over the entire range of examiner confidence is summarized in the ROC curve shown in Fig. 1. The area under the ROC curve (1.0 = perfect discrimination; 0.5 = indiscriminate) for recognition of minor VSD is 0.92 ± 0.02.
2.3 Major or intermediate significance VSD.
2.3.1 1) Large VSD as the sole diagnostic consideration.
Of the 21 cases of intermediate or major VSD, 4 were identified confidently with large VSD as the cardiologist’s only realistic diagnostic consideration (by this most stringent standard, sensitivity = 19%). Only 1 of the 266 subjects with a condition other than intermediate or major VSD was identified by the cardiologist as having large VSD as the sole diagnostic consideration (specificity = 99.6% by this most lenient standard). The one patient with an unappreciated cardiac condition falsely diagnosed with confidence as a large VSD on clinical grounds had minor VSD.
2.3.2 2) Large VSD Appearing on a Differential Diagnostic List.
Nine of the 21 patients with intermediate or major VSD were listed as having large VSD somewhere on the differential diagnosis before echocardiography (by this less stringent standard, sensitivity = 43%). The five patients with intermediate or major VSD for whom large VSD was not considered a possibility before echocardiography were thought to have small VSD (n = 7), coarctation of the aorta (n = 2), pulmonary stenosis (n = 1) and aortic stenosis (n = 1). In 273 patients with a condition other than intermediate or major VSD, large VSD correctly did not appear anywhere on the differential diagnosis in 259 (specificity by this most stringent standard = 95%).
2.3.3 3) ROC Curve.
Sensitivity and specificity of clinical diagnosis of intermediate or major significance VSD over the entire range of examiner confidence are summarized in the ROC curve in Fig. 1. The area under the ROC curve for recognition of intermediate or major VSD is 0.69 ± 0.07. When this area is compared with the area under the ROC curve for clinical diagnosis of minor VSD, the difference is significant (p = 0.0016).
2.4 Original clinical impression of VSD size and final importance of VSD.
2.4.1 1) Clinical Impression of Small VSD.
Of the 56 cases of VSD in which the cardiologist’s preechocardiographic impression was small VSD, 48 (86%) proved to be minor, 5 (9%) to be intermediate and 3 (5%) to be major. Therefore, the cardiologist’s impression of small VSD has a positive predictive value of 86% to 95% for minor VSD, depending on whether intermediate VSDs are classified with minor or major VSDs.
2.4.2 2) Clinical Impression of Large VSD.
Of the 10 cases of VSD in which the cardiologist’s preechocardiographic impression was large VSD, 2 (20%) proved to be minor, 2 (20%) to be intermediate and 6 (60%) to be major. Therefore, the cardiologist’s impression of large VSD has a positive predictive value of 60% to 80% for major VSD, depending on whether intermediate VSDs are categorized with minor or major VSDs.
2.4.3 3) Clinical Impression of Some Diagnosis Other Than VSD.
Of the seven cases of VSD in which the cardiologist failed to include VSD of any size in the differential diagnosis, two proved to be minor (representing 3.8% of all minor VSDs), three to be intermediate (30% of all intermediate VSDs) and two to be major (18% of all major VSDs).
3.1 Accuracy and Error in Nonechocardiographic Diagnosis of VSD.
The practice of obtaining echocardiographic confirmation at the time of clinical diagnosis of small VSD in children is controversial (); therefore, further evaluation of the accuracy of the nonechocardiographic diagnosis of VSD is pertinent. This audit of the diagnostic accuracy of pediatric cardiologists’ clinical examination for VSD reveals the following: 1) Almost all minor VSDs are recognized by the pediatric cardiologist without echocardiographic support as small defects; however, errors can occur even when the examiner expresses diagnostic confidence. Minor VSDs are occasionally confused with conditions that produce high pitched or harsh systolic ejection murmurs like those associated with pulmonary stenosis or subaortic membrane or holosystolic murmurs like those of mitral regurgitation. 2) Clinical recognition of intermediate or major VSDs is less accurate than the clinical recognition of minor VSDs. 3) Failure to distinguish VSDs of major or intermediate importance from minor VSDs is a weakness of the expert clinical examination.
3.2 Clinical Impression of VSD Size as a Marker for VSD Importance.
A substantial number of the clinical diagnostic shortcomings uncovered in this study are instances in which the clinical impression of small VSD is associated with VSD of intermediate or major importance. It is well recognized that size is not the only determinant of VSD importance. Small perimembranous defects carry risks for hemodynamic deterioration with growth of subaortic membrane, development of left ventricular to right atrial shunting, development of aortic valve insufficiency or growth of obstructive right ventricular muscle bundle (). In this series, however, many intermediate significance VSDs had auscultatory and other clinical features more in common with minor muscular VSDs than with major defects. The data presented here suggest that the cardiologist cannot reliably distinguish these VSD subgroups without an echocardiogram. Although this series did not include any cases of small subarterial VSD, it is reasonable to speculate that these VSDs with very important late complications are also difficult to distinguish from minor defects on clinical grounds.
3.3 Confusion Between Minor VSDs and Non-VSD Conditions.
Although the ROC curve area serves to emphasize that the expert clinical examination has a very high level of diagnostic accuracy for minor VSDs, close review of the data turns up the occasional patient in whom minor VSD is confused with non-VSD conditions. That there are any errors at all in the clinical diagnosis of minor VSD may surprise some cardiologists who realize that the auscultatory findings of minor VSD are among the most easily recognized of any in the realm of congenital heart disease. However, perceptions that the clinical diagnosis of minor VSD is seldom, if ever, wrong have never been supported by prospective study, and they may well be mistaken. The results reported here invite speculation that they reflect erosion of auscultatory skills among contemporary examiners (). Perhaps cardiologists in decades past, who did not come to depend on echocardiography for diagnostic support, honed their clinical diagnostic skills to levels unmatched in this current investigation. Far more likely, however, is that cardiologists years ago did not have the timely opportunity to be proved wrong in their diagnosis of minor VSDs. The current experience confirms there would be few errors, virtually none of which would be identified promptly. Mistakes could come to light only in the subsequent clinical course, which the original examiner might not have the opportunity to observe over the years. Therefore, as the reference standard for diagnosis of minor VSD, expert clinical auscultation remained unchallenged as definitive. The present study suggests that auscultation for diagnosis of minor VSD is not perfect now and may not have been in the past.
The ability of pediatric cardiologists to recognize innocent murmurs with a high level of sensitivity and specificity is well established ([5–7]). This ability and the nearly 70% prevalence of innocent murmur among children () form the foundation for recommendations that routine echocardiography to support the diagnosis of innocent murmur is not a wise use of resources (). Because minor VSD occurs with a >5% prevalence at birth (), and because echocardiography entails considerable expense, deferral or elimination of the echocardiogram from the diagnostic evaluation could also be economically important. Clearly, however, we would be ill advised to delay or forgo echocardiography if diagnostic accuracy were sacrificed in any way that subsequently threatened the patient’s well-being. The data presented in this study highlight the potential inaccuracies of the clinical examination for diagnosis of small VSD. It is not known, but it is a reasonable conjecture, that the improved diagnostic accuracy would ultimately result in sufficient improvement in outcome to justify the routine use of echocardiography for this purpose.
- receiver-operating characteristic
- ventricular septal defect
- Received September 10, 1996.
- Revision received February 10, 1997.
- Accepted March 12, 1997.
- The American College of Cardiology
- Leatham A,
- Segal B
- Sapin SO,
- Junkel PA,
- Wong AL
- Smythe JF,
- Texeira OHP,
- Vlad P,
- et al.
- Tubman TR,
- Shields MD,
- Craig BG,
- Mulholland HC,
- Nevin NC
- Tulloh RMR,
- Tansey SP,
- Parashar K,
- De Giovanni JV,
- Wright JGC,
- Silove ED
- Kapusta L,
- Hopman JCW,
- Daniels O
- Driscoll D,
- Allen HD,
- Atkins DL,
- et al.
- Tavel ME
- MacLaren MJ,
- Lachman AS,
- Pocack WA,
- et al.
- Danford DA,
- Nasir A,
- Gumbiner C
- Roguin N,
- Du Z-D,
- Barak M,
- Nasser N,
- Hershkowitz S,
- Milgram E