Author + information
- Received March 16, 1999
- Revision received June 15, 2000
- Accepted July 31, 2000
- Published online November 15, 2000.
- James E Calvin, MD, FRCPC, FACCa,* (, )
- Lloyd W Klein, MD, FACPa,
- Elizabeth J VandenBerg, MSa,
- Peter Meyer, PhDa and
- Joseph E Parrillo, MD, FACPa
- ↵*Reprint requests and correspondence: Dr. James E. Calvin, Rush-Presbyterian-St. Luke’s Medical Center, Chicago, Illinois 60612
In the mid 1990s, two unstable angina risk prediction models were proposed but neither has been validated on separate population or compared.
The purpose of this study was to compare patient outcome among high, medium and low risk unstable angina patients defined by the Agency for Health Care Policy and Research (AHCPR) guideline to similar risk groups defined by a validated model from our institution (RUSH).
Four hundred sixteen patients consecutively admitted to the hospital with unstable angina between January 1, 1995, and December 31, 1997, were prospectively evaluated for risk factors. The presence of major adverse events such as myocardial infarction (MI), death and heart failure was assessed for each patient by chart review.
The composite end point of heart failure, MI or death occurred in 3% and 5% of the RUSH and AHCPR low risk categories, respectively, and in 8% and 10% of AHCPR and RUSH high risk categories, respectively. Recurrent ischemic events were best predicted by the RUSH model (high: 24% vs medium: 12% and low: 10%, p = 0.029), but not by the AHCPR model (high: 14% vs medium: 13% and low: 9%, p = 0.876). The RUSH model identified five times more low risk patients than the AHCPR model.
Both models identify patients with low and high event rates of MI, death or heart failure. However, the RUSH model allowed for five times more patients to be candidates for outpatient evaluation (low risk) with a similar observed event rate to the AHCPR model; also, the RUSH model more successfully predicted ischemic complications. We conclude that the RUSH model can be used clinically to identify patients for early noninvasive evaluation, thereby improving cost effectiveness of care.
In 1994, the Agency for Health Care Policy and Research (AHCPR) published evaluation and management guidelines for unstable angina (1). The management algorithms were driven by the assessment of acuity into low, medium and high risk categories. This risk prediction model based on pain syndrome, ECG changes, age and degree of clinical evidence of heart failure has not been validated to date (and to our knowledge).
Assessing risk of subsequent event in patients who present with chest pain has been an established focus of the investigation over the last 20 years (2–13). Earlier studies focused on the accuracy of predicting an acute myocardial infarction (MI) with the goal of reducing inappropriate admissions to the Coronary Care Unit while at the same time achieving a high sensitivity for diagnosing MI (2,3,14–18). Braunwald (19) called attention to the need of having a risk stratification scheme in unstable angina with the purposes of better matching of resources to need, better designing of clinical research trial designs and improving quality assurance methods. His model has been studied subsequently by others (20) who have demonstrated its correlation with lesion morphology and outcome in unstable angina. Our group (8) also has validated a number of the Braunwald predictors along with age and the presence of diabetes as a way to assess risk of major cardiac complications in unstable angina.
The purpose of this study was to prospectively compare both the AHCPR risk model with the model developed by our group (RUSH model) from the standpoint of clinical outcome and resource utilization.
All patients with unstable angina (total group, n = 416) admitted to the Coronary Care or Coronary Stepdown Units at Rush-Presbyterian-St. Luke’s Medical Center between January 1, 1996, and December 31, 1997, were prospectively evaluated in this study. Patients were identified daily by a nurse coordinator by both admission logs and chart review. Patients were diagnosed as having unstable angina pectoris if they had either of the following:
1. Ischemic type of chest pain, either responsive to nitrates or associated with ST depression, or T-wave inversion >2mm occurring at rest and lasting ≥20 min; or
2. Progressive angina characterized by exertional ischemia pain increasing in frequency, duration, or at decreasing levels of exertion.
Anginal equivalents such as shortness of breath were excluded. An admission diagnosis of unstable angina was excluded if the total creatine kinase (CK) level was greater than twice the upper limit of normal for our laboratory within the first 12 h of presentation to the Emergency Department. These patients were classified as non Q MI. All patients were treated by their attending physicians. Guideline reminders summarizing the AHCPR recommendation for evaluation and treatment of unstable angina were posted on each patients’ chart.
The estimated risk of sustaining a major cardiac complication using the RUSH model (8) was based on the results of a previously validated multiple logistic regression model. The model is calculated as the sum of terms on the log odds scale as follows: Where POSTMI indicates admission followed an MI within 14 days, IVNTG indicates IV nitroglycerin was required on admission, NO_BETA indicates the person was receiving neither a beta-blocker nor on a rate-lowering calcium channel blocker at admission, STDEPR indicates that the person had ST segment depression on admission ECG, DIAB indicates a history of diabetes and AGE is the age in years. The probability (p) of a major event was calculated as p = 1/(1 + e−log odds).
The entire population was then categorized into groups of estimated probability of risk of major complications as follows: estimated probability of major cardiac complication <5%, low risk; 5.1% to 25%, medium risk; >25%, high risk. These three strata were chosen to provide large enough groups for adequate statistical comparison and to allow a direct comparison with the AHCPR model, which is also categorized as low, medium and high risk. Also, the intervals were believed clinically appropriate and approximate the probabilities of risk achieved in other studies (2,3,17).
The AHCPR risk prediction model was applied to all patients (Table 1). Patients were identified as medium risk if at least one medium risk factor was present and high risk if at least one high risk factor was present.
Definition of outcome events
Major cardiac complications were defined by the occurrence of any of the following:
1. MI after the first 24 h defined by a CK >twice the upper limit of normal and peaking after the first 24 h or the development of new Q waves and CKMB index >10%;
2. congestive heart failure defined by documentation of a new S3 or crackles after first 24 h or chest radiographic evidence of pulmonary edema; and
This composite of outcomes differed slightly from our original report by excluding ventricular tachycardia >30 s requiring drug therapy or ventricular fibrillation, which we found to be uncommon.
Secondary end points of recurrent angina (steady chest pain lasting at least 5 min and for which a charge in patient therapy occurred including but not limited to administration of one sublingual nitroglycerin) and recurrent angina with ST depression ≥ 1 mm beyond baseline ECG were also tabulated. Interventional procedures were not treated as end points of clinical outcome in either this study or in our original description.
Two nurse coordinators prospectively collected all clinical predictors and demographic information from the patient’s chart within 24 h of hospital admission. Treatments and major in-hospital outcomes were obtained by both reviewing the patient’s chart and interviewing the patient’s physician on a daily basis. The data were subsequently entered in a relational database (Paradox; Ansa Software; Scott’s Valley, California). Neither of these coordinators was involved with the initial model development and was not trained on it. Assignments of outcomes were verified in two ways: 1) chart audits (J.E.C.) in a 10% random sample and 2) checking discharge ICD-9 codes obtained from the hospital information system.
Univariate comparisons were made using chi-square tests for categorical variables. Analysis of variance was used for normally distributed continuous variable data.
Figure 1 demonstrates the frequency of patients in low, medium and high risk groups for both models. The RUSH model identified five times more low risk patients than the AHCPR model while the AHCPR model identified more high risk patients. Most patients were classified as intermediate risk using either model.
The baseline characteristics of each of the AHCPR risk groups (low, medium or high [Table 2]) revealed that low risk patients were younger, had no evidence of ST depression and were less likely to have had previous coronary bypass surgery. The high risk patients more frequently had more coronary angioplasty.
The baseline characteristics of each of the RUSH risk groups (Table 2) revealed that high risk patients were the eldest, and more frequently required IV nitroglycerin to control ongoing chest pain. They also had more ST depression and were more likely to have diabetes or to have had a recent MI than the other two groups. Low risk patients were more likely to have hypercholesterolemia than the other two groups and more frequently had prior treatment with beta-blockers.
Treatments in hospital
The treatments received in hospital are summarized in Table 3. In general, neither model predicted medical or revascularization therapy, suggesting that interventional revascularization procedures are not guided by risk assessment. The medical therapies were also not influenced by either model with the exception of IV nitroglycerin in the RUSH categorization, which is confounded by being one of the variables used in the probability estimate. There was a trend for more low risk AHCPR patients than high risk patients to receive percutaneous transluminal coronary angioplasty (PTCA) but this was not significant.
Outcomes in hospital
The hospital length of stay for each risk group of both models is summarized in Table 4. The overall length of stay was 5.7 ± 4.0 days and the median was 5 days. The low risk groups’ length of stay was only one day shorter than medium or high risk groups of both models (not significant). The incidence of the composite end point of heart failure, MI or death is shown in Table 5. The RUSH model demonstrates a progressive but not statistically significant increase in the composite end point while the AHCPR model shows little gradation in the composite end point. The RUSH model does show a progressive increase in risk of any complication (including recurrent angina) whereas the AHCPR model does not (Table 5). Furthermore, the RUSH model significantly predicts ischemic end points of recurrent angina (low, 10%; medium, 12%; and high, 24% [p = 0.029] vs. AHCPR low, 9%; medium, 13%; and high, 14% [p = 0.876]) and recurrent angina with ST depression than the AHCPR model (low, 1%; medium, 2%; and high, 9% [p = 0.004] vs. AHCPR low, 0%; medium, 3%; and high, 3% [p = 1.00]) (Table 5). The RUSH model also discriminated the end point of MI, deaths, heart failure or recurrent angina among low, medium and high risk groups compared to the AHCPR model (Table 5). To further explore the accuracy of the RUSH model, four risk strata were arbitrarily established for the RUSH model (0 to <5%, n = 105; 5 to <15%, n = 171; 15 to <25%, n = 80; and ≥25%, n = 59). The rates of a major complication (MI or death or heart failure were 3%, 9%, 6% and 11%, respectively. Thus, the RUSH model predicts very well in the lowest risk groups but overestimates risk in the higher risk groups.
There are several important conclusions of this study. First, risk stratification models do have value in the early evaluation of unstable angina, especially in the low risk category. Both models predicted patients who subsequently experienced less than a 5% incidence of cardiac complications. Second, the RUSH model identified five times more low risk patients than the AHCPR model. This allows more patients to be evaluated by noninvasive testing either within the Emergency Department or within 72 h as an outpatient (as recommended by the AHCPR). The observation of the low yield of low risk patients by the AHCPR model concurs with study of Katz et al. (10) who found that the AHCPR model identified only 6% of patients. Third, the risk gradient for MI, death or heart failure between medium and high-risk patients was small for both models making the distinction between medium and high risk of this composite end point of less clinical value. However, the RUSH model did demonstrate an ability to discriminate a composite end point of death, MI, heart failure or recurrent angina with ST depression compared to the AHCPR model. The inclusion of ST depression with recurrent angina makes this secondary end point reasonably objective and clinically meaningful. The RUSH model also better discriminates recurrent angina which is arguably a soft end point. Indeed, the AHCPR guidelines are weighted more toward complicating heart failure, which was a rare complication in this study and may have influenced the performance of that model.
Admittedly, some bias may exist for nurse data collectors to underestimate complications in low risk patients by virtue of early hospital discharge and incomplete follow-up. This risk appears to be small by virtue of similar hospital length of stay among risk groups of either model. Patients discharged from the Emergency Department because they were low risk may also have affected these results since they were not included in the analysis. Our experience with these patients, which has been recently reported, suggests lower event rate than reported in the low risk categories of this study (21).
Potential impact of changes in concurrent therapy
Aggressive medical therapy directed by clinical practice guidelines was promoted by the design of this study and may have reduced ischemic complications, especially in higher risk groups of both models. This may also have obscured the distinction between medium and high risk patients, by lowering the complication rate in the high risk categories as we have previously demonstrated (22).
The RUSH model appears to demonstrate a trend toward predicting the use of coronary bypass surgery and medical therapy such as IV heparin. Neither model predicted the use of coronary angioplasty. This latter finding was also apparent in an earlier study by our group (23) and by others (24,25), leading one editorialist to suggest another model to assess the need for angioplasty in patients with unstable angina (26). This is disconcerting in light of the recently reported results of the VANQWISH (27) and MATE (28) trials and the OASIS Registry (23,29) of acute ischemic syndromes, which have not demonstrated dramatic benefit from early angioplasty and may have demonstrated potential harm. Further studies with IIb/IIIa platelet receptor inhibitors and early coronary intervention are necessary to answer this question.
Novel methods of risk stratification
Neither risk prediction model studied includes laboratory markers for risk such as troponin I or T, early stress testing or multi-lead ST segment monitoring. Troponin was not available in our institution at the time of the study. Several authors (30–35) have suggested the potential value of either troponin I or T to predict risk of complication in acute ischemic syndromes. In fact, a recent study by Hamm et al. (6) suggests an excellent prognosis if either of these markers is negative. However, none of these studies was designed to demonstrate any incremental value of a troponin marker over clinical predictors. Therefore, the National Heart Attack Alert Program (36) still suggests further evaluation from carefully controlled studies. Because the low risk groups studied herein had very low event rates, a prospective head-to-head comparison of troponin is necessary in this risk group. It is possible that these markers may have greater value in refining the risk estimates in intermediate and high risk groups.
Some clinicians may be uncomfortable with defining “low risk” groups in unstable angina using event rates of 2% to 5%. While Selker and colleagues (13) have demonstrated an extremely low mortality rate in low risk chest pain patients identified by a multivariate predictive instrument, only 25% of patients had an acute ischemic syndrome. Currently, we are testing the effectiveness of combining clinical prediction models with early stress echocardiography. The RUSH model appears to be attractive for this purpose because it predicts a larger proportion of patients who appear to be clinically at low risk. Current AHCPR guidelines recommend outpatient noninvasive testing within 48 h for this group. While rest echocardiography (37), rest sestamibi (38), stress nuclear testing (39) and stress echocardiography (40–42) have been used to assess risk of chest pain patients in the Emergency Department, pretest probabilities are seldom reported. The RUSH model provides an objective pretest probability, thereby enhancing patient selection for early noninvasive testing (12 to 18 h) and improving posttest estimates or risk.
In this study, patients identified by the RUSH model to be at high risk for major cardiac complications had a complication rate for MI, death or heart failure that was lower than predicted by the model. More aggressive therapy may have been responsible, although explanations such as unidentified changes in comorbidity may also be responsible. This finding highlights the need to interpret our findings in the light of current treatment practices and patient selection. Potentially, the use of new serum markers or multi-lead ST segment monitoring could further refine our predictions of high risk patients.
In summary, this study confirms the value of clinical risk prediction to identify the low, medium and high risk patients with unstable angina. It not only validates the use of the AHCPR model for distinguishing low risk from medium and high risk patients but also shows that another easily applied model may identify more low risk patients and better predicts a composite of ischemic complications. These risk prediction models may have a potential impact on the use of glycoprotein IIb/IIIa platelet receptor inhibitors and enoxaparin and earlier use of PTCA and stents.
☆ This study was funded by institutional Cardiology Divisional funds only.
- Agency for Health Care Policy and Research
- creatinine kinase
- myocardial infarction
- percutaneous transluminal angioplasty
- Received March 16, 1999.
- Revision received June 15, 2000.
- Accepted July 31, 2000.
- American College of Cardiology
- ↵US Department of Health and Human Services. National Heart, Lung and Blood Institute. Unstable angina: diagnosis and management. AHCPR publication No. 94-0602; 1994.
- Murphy J,
- Connell P,
- Hampton J
- Gaspoz J.M,
- Lee T.H,
- Weinstein M.C,
- et al.
- Braunwald E
- Van Miltenburg-Van Zijl A.J.M,
- Simoons M,
- Veerhoek R,
- Bossuyt P
- Calvin J.E,
- Stamos T.D,
- Bartlett L.C,
- et al.
- Iliadis E.A,
- Klein L.W,
- Vandenberg B.J,
- et al.
- ↵Mark D. Implications of cost in treatment selection for patients with coronary heart disease. Ann Thorac Surg 1996:S12–5.
- McCullough P,
- O’Neill W,
- Graham M,
- Stomel R
- Katus H,
- Remppis A,
- Neumann F,
- et al.
- Katus H.A,
- Looser S,
- Hallermayer K,
- et al.
- Ravkilde J,
- Nissen H,
- Horder M,
- Thygesen K
- Wu A,
- Lane P
- Lindahl B,
- Venge P,
- Wallentin L,
- et al.
- Fleischmann K,
- Goldman L,
- Robiolio P,
- et al.
- Bilodeau L,
- Theroux P,
- Grégoire J