Validation and Re-Evaluation of a Discriminant Model Predicting Anatomic Suitability for Biventricular Repair in Neonates With Aortic Stenosis

Patients

The new study cohort included patients who met the criteria specified in our original report (1): 1) surgical or balloon aortic valvuloplasty for valvar AS performed at our center between 1989 and 2002 at ≤60 days of age, 2) echocardiographic evidence of LV dysfunction and symptoms of congestive heart failure, 3) normally related great arteries, 4) absence of subvalvar/supravalvar AS, 5) intact ventricular septum, 6) no prior procedures on the aortic valve, and 7) adequate pre-intervention echocardiographic images to make all pertinent measurements. The presence of a patent ductus arteriosus was not considered in determining study eligibility. Mitral valve size was not considered for study inclusion, but few patients with significant MV hypoplasia were included in this study, because most neonates with AS and an MV area z-score <−2 were referred for univentricular palliation without attempted biventricular management. Patients who underwent SV palliation without a prior attempt at valvuloplasty were excluded. The only change in entrance criteria from our original report was that patients diagnosed at ≤60 days of age who underwent valvuloplasty beyond this age were not included.

For the repeat discriminant analysis, new patients were pooled with 37 patients from our original report who underwent valvuloplasty at ≤60 days of age. The revised discriminant analysis did not include eight patients from the original series who were diagnosed at ≤60 days of age but underwent valvuloplasty beyond this age or patients who were thought to be “borderline” candidates for biventricular repair but ultimately underwent stage I palliation without previous valvuloplasty.

Echocardiographic measurements

Echocardiographic measurements were made as described in our original report (1). Structures measured included aortic valve annulus diameter, aortic root diameter, transverse arch diameter, aortic isthmus diameter, MV area, LV mass, LV end-diastolic dimension and volume, LV long-axis length, and LAR. Aortic annulus and root diameters were measured from parasternal long-axis images, with the aortic root diameter taken as the maximum dimension at the level of the sinuses. The internal dimensions of the distal transverse arch (between the left common carotid and subclavian arteries) and the aortic isthmus (immediately distal to the left subclavian artery) were measured from suprasternal notch images. Vascular diameters were measured at the maximum systolic dimensions, from inner edge to inner edge. The MV area was calculated as an ellipse from the orthogonal diameters of the MV annulus measured in early diastole from parasternal long-axis and four-chamber images. The morphology of the MV apparatus was not systematically evaluated for this study. The long axis of the heart was measured in apical four-chamber images from the plane of the MV annulus to the apical endocardium of the ventricle (left or right) that formed the apex of the heart. The LV end-diastolic and end-systolic epicardial and endocardial borders were digitized manually from subxyphoid long- and short-axis views. The LV volumes were calculated with the biplane bullet model, according to which LV volume equals five-sixths multiplied by short-axis cross-sectional area of the LV at midventricular level multiplied by the long-axis LV length. Mass was calculated as the difference between endocardial and epicardial LV volumes multiplied by the specific gravity of myocardium (1.04 g/ml). Aside from LAR, measured values were indexed to BSA, which was calculated with the method of Haycock et al. (15). For assessment of the predictive accuracy of the original discriminant and threshold scoring systems in the new patient cohort, indexed measurements were used, as they were in our original report; however, for the revised discriminant analysis, z-scores (number of standard deviations above or below the average indexed size of the structure in question, on the basis of normative data obtained at our institution from children with structurally and functionally normal hearts [16]) were used instead. On the basis of the CHSS analysis (5), LV endocardial fibroelastosis (EFE) was included in the analysis. Severity of EFE was graded separately by two echocardiographers according to the scale used in the CHSS study of neonatal AS (5), with 0 = none, 1 = involvement of MV papillary muscles only, 2 = involvement of papillary muscles and some endocardial surface, and 3 = extensive endocardial surface involvement.

Original threshold and discriminant scoring systems

According to our original threshold scoring system (1), threshold values were determined for the four anatomic variables independently associated with survival or death. For each variable below the threshold value, a patient was assigned one point, such that the range of threshold scores was zero to four. The variables and threshold values included in the score were indexed aortic root diameter (≤3.5 cm/m²), indexed MV area (≤4.75 cm²/m²), LAR (≤0.8), and indexed LV mass (≤35 g/m²). A threshold score <2 predicted survival with a biventricular circulation. Our original discriminant analysis yielded the equation 14.0 (BSA) + 0.943 (indexed aortic root diameter) + 4.78 (LAR) + 0.157 (indexed MV area) − 12.03, with a score of −0.35 discriminating between survival and death (≤−0.35 predicting death or conversion to SV circulation, >−0.35 predicting survival with a biventricular circulation) (1).

CHSS scoring system

In 2001, the CHSS reported the results of a multicenter study of outcomes in 320 neonates with critical AS, over one-half of whom underwent SV palliation (5). From multivariable analyses of factors associated with outcome after stage I palliation or biventricular management, the authors derived a regression equation that can be solved for individual patients to predict the survival benefit of stage I palliation versus a biventricular repair strategy (5):Survivalbenefit=30.55(inverseofageatstudyentry[d]+1)−6.20(aorticrootz-score)+12.14(echocardiographicgradeofEFE)+23.33(logarithmofascendingaortic[mm])−28.30(presenceofmoderateorseveretricuspidregurgitation)−0.70(LVlong-axislengthz-score)−86.47The magnitude of the result indicates the predicted difference in percent survival, with a positive value indicating a survival benefit of stage I palliation and a negative value indicating a survival benefit of biventricular repair. The CHSS equation was solved for each patient using the CHSS online critical AS calculator (17).

Data analysis

The primary outcome was survival with a biventricular circulation. Patients who died or underwent conversion to an SV circulation were considered equally as “events.” Because all events occurred within one year and almost all occurred early (within one month of intervention), data were analyzed in a time-independent manner, as in our original report (1). Descriptive analysis was used to compute the accuracy of the original threshold and discriminant scoring systems when applied to the 89 new patients. For assessment of the effects of measurement variation on predictive accuracy, each of the measures included in the original threshold and discriminant scoring systems was increased and decreased by 5% of its measured value and reinserted into the appropriate scoring equation. For the threshold scoring system, each of the component scores was analyzed independently, and all possible permutations of the overall score were examined. For the revised discriminant analysis, statistical analysis of data from all 126 patients was performed according to the methods described in detail in our original report (1). Although indexed measurements were used in our original analysis, the updated analysis was performed with z-scores. Univariable analysis was conducted to identify variables that differed significantly between survivors and events, with separate analyses performed for patients ≤30 days of age and for patients without coarctation of the aorta (COA). Two-group linear discriminant analysis was performed with forward stepwise selection of variables for inclusion in the model. As in our original analysis, the unstandardized discriminant function was determined to optimize identification of survivors. An updated threshold scoring system was not devised. Unless otherwise specified, data are presented as mean values ± standard deviation or median (range).

Patients

Between 1989 and 2002, 89 patients ≤60 days of age satisfying the inclusion criteria underwent balloon aortic valvuloplasty (n = 87) or surgical aortic valvuloplasty (n = 2). The median age at valvuloplasty was 4 days (0 to 60 days), and the median weight was 3.4 kg (0.9 to 5.8 kg). Seventy-six patients (85%) were ≤30 days of age and 54 patients (61%) were ≤1 week.

Including the 37 patients from our original report whose data are included in the revised discriminant analysis, there were 126 patients. The median age at valvuloplasty was 5 days (0 to 60 days), and the median weight was 3.4 kg (0.9 to 5.8 kg). Age at valvuloplasty was ≤30 days in 104 patients (83%) and ≤1 week in 72 patients (57%). Fourteen patients (11%) underwent repair or angioplasty of COA before or within three months of aortic valvuloplasty. A patent ductus arteriosus was present in 86 patients (66%).

Outcomes (survival, conversion to an SV circulation) in the new and combined cohorts are summarized in Table 1.There was a significant reduction in mortality during the latter one-half of our experience (p = 0.005) as well as a decline in the frequency of events over time (p = 0.07) (Table 1). Moreover, all five patients who converted to an SV circulation from 1994 to 2002 survived, compared with two of nine who converted to an SV circulation before 1994 (p = 0.01). Hospital survivors were followed for a median of 5.7 years (1 month to 18 years). Intermediate outcomes in these patients have been reported separately (18).

Validation of the original threshold and discriminant scoring systems in 89 new patients

The distribution of ordinal threshold scores in the 89 new patients is summarized in Table 2.The threshold score accurately predicted survivors and events in 76% of patients, including 78% of survivors and 76% of events. Accuracy was almost identical among patients undergoing intervention within the first month (77%) or week (75%) of life (p = NS).

The original discriminant score accurately predicted outcome in 68 patients (76%), including 78% of survivors (n = 49) and 76% of events (n = 19) (Fig. 1).Predictive accuracy was similar among patients undergoing valvuloplasty within the first month (75%) and first week (69%) of life (p = NS).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Scatter plot showing discriminant scores (for the overall cohort of 126 patients) calculated with the originally published (1) discriminant equation (solid symbols)and our updated discriminant equation (open symbols). Reference lines show the optimal discriminant cutoffs of −0.35 (original model, solid line) and −0.65 (updated model, dashed line).

There was predictive agreement between the threshold and discriminant scores in 71 patients: in 59 patients (66%), both scores predicted outcome accurately; and in 12 patients (14%), both scores failed to predict outcome. These proportions were similar for survivors and events.

Effect of measurement variation on predictive accuracy of scoring systems

Of the four variables included in the threshold scoring equation, one or more was within 5% of the threshold value in 30 (34%) patients. In other words, with 5% variation in aortic root diameter, MV area, LAR, or LV mass measurements, one or more components of the threshold score (total 36 component scores) would change in 30 patients, increasing (less favorable component score) in 21 cases and decreasing (more favorable component score) in 15 cases. In two patients, changes in multiple components of the threshold score would cancel each other out, such that the overall threshold score would change in 28 patients (32%). In 13 of these patients (15% of 89), all of whom survived with a biventricular repair, the resulting change in the threshold score would result in a change in the outcome predicted: in 2 cases, an incorrect prediction (i.e., death) would be made correct (i.e., survival with biventricular circulation); and in 8 cases, a correct prediction of survival would be made incorrect.

For the discriminant scoring system, 5% variation in measurement of aortic root diameter, MV area, or LAR would change the predicted outcome in eight patients (9%), with a correct prediction made incorrect in seven (five survivors, one event), and an incorrect prediction made correct in two (both survivors).

Predictive accuracy of the CHSS model

For the 108 patients in our cohort who underwent pre-intervention echocardiography at ≤31 days of age (required for entry into the CHSS study), the CHSS regression equation was solved (5). The CHSS model predicted a survival advantage with SV palliation in 63 of these 108 patients (58%), over one-half of whom (n = 32), including 16 of 22 patients (73%) managed between 1994 and 2002, survived with a biventricular circulation. The CHSS model predicted a survival advantage with a biventricular repair strategy in the other 45 patients, 39 of whom (87%) survived with a biventricular circulation.

Updated discriminant analysis

Data from the 89 new patients and 37 patients in our original report were pooled and analyzed by discriminant analysis (1). An updated threshold scoring system was not derived. Differences in independent variables between survivors and events are summarized in Table 3,and selected baseline echocardiographic measurements for the entire cohort are shown in Figure 2.There was agreement about EFE grade by the two reviewing echocardiographers in 90 patients (71%), a discrepancy of one grade in 35 patients (28%), and a discrepancy of two grades in 1 patient (1%) (kappa = 0.58). In cases with discrepant scoring, a consensus score was determined upon further review of the echocardiogram. The most accurate discriminant model for predicting survival with a biventricular circulation among the entire cohort of 126 patients is: 10.98 (BSA) + 0.56 (aortic valve annulus z-score) + 5.89 (LAR) − 0.79 (presence of grade 2 or 3 EFE) − 6.78. Taking a discriminant cutoff of −0.65, this model accurately predicted outcome in 95% of survivors and 80% of events (90% overall) (Fig. 2). Higher cutoff values allowed slightly improved specificity at the expense of sensitivity. The greatest specificity while maintaining >85% sensitivity for predicting survival with a biventricular repair was at a discriminant cutoff of <−0.46, which predicted outcome accurately in 86% of survivors and 82% of events. If EFE is omitted from the analysis (owing to high interobserver variability in grading), the most accurate model for predicting survival with a biventricular circulation is: 12.16 (BSA) + 0.59 (aortic valve annulus z-score) + 5.73 (LAR) − 7.02, with a discriminant cutoff of −0.46 accurately predicting 91% of survivors and 80% of events (87% overall).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Scatter plots showing pre-intervention z-scores for (A)aortic annulus diameter, (B)aortic root diameter, (C)left ventricular (LV) end-diastolic dimension (EDD), and (D)the LV long-axis to heart long-axis ratio (LAR). Survivors are depicted as solid circlesand events are depicted as open symbols, with patients who died shown as open trianglesand patients who survived after conversion to a functional SV circulation shown as open circles.