Author + information
- Received October 27, 2014
- Revision received February 9, 2015
- Accepted February 25, 2015
- Published online May 12, 2015.
- Karen E. Joynt, MD, MPH∗,†∗ (, )
- Deepak L. Bhatt, MD, MPH∗,
- Lee H. Schwamm, MD‡,
- Ying Xian, MD, PhD§,
- Paul A. Heidenreich, MD, MS‖,
- Gregg C. Fonarow, MD¶,
- Eric E. Smith, MD, MPH#,
- Megan L. Neely, PhD§,
- Maria V. Grau-Sepulveda, MD, MPH§ and
- Adrian F. Hernandez, MD, MHS§
- ∗Brigham and Women’s Hospital and Harvard Medical School, Boston, Massachusetts
- †Harvard School of Public Health, Boston, Massachusetts
- ‡Massachusetts General Hospital, Boston, Massachusetts
- §Duke Clinical Research Institute, Durham, North Carolina
- ‖Palo Alto Veterans Affairs Hospital, Palo Alto, California
- ¶University of California Los Angeles, Los Angeles, California
- #Hotchkiss Brain Institute, University of Calgary, Calgary, Alberta, Canada
- ↵∗Reprint requests and correspondence:
Dr. Karen E. Joynt, Brigham and Women’s Hospital, Cardiovascular Medicine, 75 Francis Street, Boston, Massachusetts 02115.
Background Electronic health records (EHRs) may be key tools for improving the quality of health care, particularly for conditions for which guidelines are rapidly evolving and timely care is critical, such as ischemic stroke.
Objectives The goal of this study was to determine whether hospitals with EHRs differed on quality or outcome measures for ischemic stroke from those without EHRs.
Methods We studied 626,473 patients from 1,236 U.S. hospitals in Get With the Guidelines-Stroke (GWTG-Stroke) from 2007 through 2010, linked with the American Hospital Association annual survey to determine the presence of EHRs. We conducted patient-level logistic regression analyses for each of the outcomes of interest.
Results A total of 511 hospitals had EHRs by the end of the study period. Hospitals with EHRs were larger and were more often teaching hospitals and stroke centers. After controlling for patient and hospital characteristics, patients admitted to hospitals with EHRs had similar odds of receiving “all-or-none” care (odds ratio [OR]: 1.03; 95% CI: 0.99 to 1.06; p = 0.12), of discharge home (OR: 1.02; 95% CI: 0.99 to 1.04; p = 0.15), and of in-hospital mortality (OR: 1.01; 95% CI: 0.96 to 1.05; p = 0.82). The odds of having a length of stay >4 days was slightly lower at hospitals with EHRs (OR: 0.97; 95% CI: 0.95 to 0.99; p = 0.01).
Conclusions In our sample of GWTG-Stroke hospitals, EHRs were not associated with higher-quality care or better clinical outcomes for stroke care. Although EHRs may be necessary for an increasingly high-tech, transparent healthcare system, as currently implemented, they do not appear to be sufficient to improve outcomes for this important disease.
- hospital mortality
- length of stay
- medical order entry systems
- outcome assessment (health care)
Electronic health records (EHRs) are increasingly seen as an essential element of improving the quality of health care delivered in the United States and worldwide. The federal government has the ambitious goal of having all U.S. health care delivered in the setting of EHRs by 2015, supported by the Health Information Technology for Economic and Clinical Health (HITECH) Act (1). The HITECH Act provides financial incentives for EHR adoption and use in its first stages but applies penalties for hospitals not adopting EHRs by 2015.
However, there has been considerable debate about whether these tools, in and of themselves, are associated with better-quality care. Although smaller-scale projects have demonstrated that the use of EHRs was associated with improvements in adherence to guideline-based care (2–4) and reduction in medication errors (5), national data have been less convincing (6). Stroke is a condition in which this is a particularly salient issue. First, the evidence base and published guidelines for stroke have evolved rapidly over the past decade, and thus, electronic systems might be particularly useful in ensuring high-quality care. Second, there is increasing interest and attention by policy makers in collecting and publicly reporting performance on quality and outcomes of care for stroke, and stroke was added to federal reporting and quality programs in 2014 (7). Thus, understanding what role EHRs might play in improving quality of care and outcomes for stroke is timely for clinicians, hospitals, and policy makers.
We therefore set out to answer 2 main questions using the Get With the Guidelines (GWTG)-Stroke registry, which allows us to examine a large number and varied types of hospitals representing the vast majority of stroke care in the United States, as well as to use detailed clinical information to assess processes and outcomes of care in a highly rigorous manner. First, is the use of EHRs associated with higher-quality stroke care? Second, is the use of EHRs associated with better stroke outcomes?
Patients and hospitals
Details of the design and conduct of the GWTG-Stroke program were previously described (8). Data from participating hospitals between 2007 and 2010 were included in this analysis. All participating institutions were required to comply with local regulatory and privacy guidelines and, if required, to secure institutional review board approval. Because data were used primarily at the local site for quality improvement, sites were granted a waiver of informed consent under the common rule. Quintiles (Cambridge, Massachusetts) served as the registry coordinating center. The Duke Clinical Research Institute (Durham, North Carolina) served as the data analysis center, and institutional review board approval was granted to analyze aggregate deidentified data for research purposes.
Trained personnel identified patients at each participating hospital. Cases were discovered through prospective clinical identification, retrospective identification with International Classification of Diseases, 9th revision, discharge codes (433.xx, 434.xx, and 436), followed by chart review to confirm eligibility, or both. Data were collected with an Internet-based patient management tool (Quintiles). Data were coded, deidentified, and transmitted securely to maintain patient confidentiality compliant with federal privacy standards. Data were collected for each hospitalization, including demographics, medical history, in-hospital treatment and events, discharge treatment and counseling, and discharge destination. The data collection tool supports concurrent data collection and retrospective data entry; concurrent collection was encouraged. The data abstraction tool included pre-defined logic features and user alerts to identify potentially invalid format or value entry. Required fields were structured such that valid data must be entered before being saved as a complete record and entered into the database. Range checks were used for inconsistent or out-of-range data and prompted the user to correct or review data entries outside of a pre-defined range. All hospital personnel using the tool received individual passwords to create an audit trail for entered data. Training in the use of the tool was provided for all users. A retrospective central chart audit showed good reliability for site abstraction of key database variables (9).
Information on adoption and use of EHRs was obtained from the American Hospital Association (AHA) Health Information Technology Survey. The details of this survey have been described previously (10); it has been distributed to U.S. acute-care hospitals that are AHA members (roughly 97% of U.S. hospitals) annually since 2008 and has a response rate of 60% to 70%. The survey assesses the adoption of specific EHR functionalities by asking respondents to report the degree of adoption for each of 24 individual electronic functionalities. These functionalities include computerized physician order entry (CPOE), clinical decision support (CDS), and electronic viewing of laboratory and radiology reports or images. Responses can range from full adoption of the function across all units to no adoption of the function in any hospital units. We considered hospitals to have EHRs in place if they met the criteria for at least a basic EHR on the basis of 8 key functionalities, as defined in prior studies (10,11). To deal with missing data (20% to 25% of hospitals do not respond to the AHA survey in a given year), we considered nonresponders to the survey in 2008 to have no EHRs; in subsequent years, we imputed missing EHR status from the prior year’s response. We assigned each patient the EHR status of his or her hospital in the admission year.
We excluded hospitals without AHA ID numbers (n = 49) and hospitals that did not respond to the AHA Health Information Technology Survey at any time during the study period (n = 169). We also excluded hospitals that entered fewer than 25 patients with stroke (n = 167), leaving us with 1,236 hospitals.
For this study, we had 1 main quality outcome and 3 main clinical outcomes: 1) a composite “all-or-none” quality performance measure, which required that a patient received each of the achievement measures for which he or she was eligible; 2) length of stay >4 days (the median length of stay in prior studies); 3) discharge home; and 4) in-hospital mortality. We also considered each of the 8 currently tracked achievement and quality measures as secondary endpoints, including intravenous (IV) tissue plasminogen activator (tPA) within 3 h in patients who arrive within 2 h of symptom onset; door-to-needle time within 60 min for those who receive tPA; antithrombotic medication within 48 h of admission; deep venous thrombosis prophylaxis within 48 h of admission; antithrombotic medication at discharge; anticoagulation for atrial fibrillation at discharge; statin therapy at discharge if low-density lipoprotein was >100 mg/dl or if low-density lipoprotein was not documented; and counseling or medication for smoking cessation.
We first summarized patient and hospital characteristics using chi-square/Fisher exact tests and Kruskal-Wallis tests to assess for differences between the EHR versus no-EHR groups. We then created graphic representations of the basic unadjusted rates of our 4 primary outcomes across the 4 study years, stratified by EHR status. We used patient-level data to examine unadjusted outcomes by EHR status at the time of admission and compared these outcomes between the groups using chi-square/Fisher exact tests and Wilcoxon rank-sum tests as appropriate.
We then created patient-level generalized mixed-regression models, with hospital random effects. Because outcomes have improved for ischemic stroke over time, we included year of admission as a covariate in our models; to determine if the association between EHR and outcome differed across the study period, an interaction between EHR status and time was tested in the regression analysis. We created 3 main models. First, we adjusted for admission year only. Second, we added patient characteristics (age, sex, race/ethnicity, insurance status, medical history, arrival mode [emergency medical services vs. self], and arrival off-hours); these variables were chosen based on prior work in the GWTG-Stroke registry in which risk models were developed and validated (12). Third, we added hospital characteristics (size, region, rurality, teaching status, primary stroke center status, ownership, and average annual volume of ischemic stroke cases); these variables were chosen because of prior literature showing their relationship to either stroke outcomes (13,14) or EHR adoption (10). Because the interaction term between EHR status and year of admission was nonsignificant in these models, we computed odds ratios from models lacking the interaction term. We then created 2 additional models by adding the National Institutes of Health Stroke Scale (NIHSS) to the patient characteristics model and then to the fully adjusted model; doing so did not appreciably change our findings. However, due to high levels of missing data, doing so reduced the sample significantly; thus, we present the models without NIHSS as our primary models. The NIHSS-adjusted models are shown in the Online Appendix.
In sensitivity analyses, to explore the possibility that a hospital’s adoption of EHRs would not impact patient care immediately, but rather after a lag, we repeated these analyses using EHR status the year before admission as our primary predictor. To explore the possibility that key functionalities of EHRs would drive improvements in outcomes, rather than the overall presence of EHRs, we repeated these analyses using the presence of CPOE and CDS at the time of admission as our primary predictors.
Two-sided p values <0.05 were considered statistically significant. The Duke Clinical Research Institute performed all analyses using SAS software version 9.1.3 (SAS Institute, Cary, North Carolina).
Patient and hospital characteristics
Our sample consisted of 626,473 patients with acute ischemic stroke admitted to 1,236 hospitals in the GWTG-Stroke program. Patients with stroke treated at hospitals with EHRs tended to be slightly younger (72 vs. 73 years), were more often male (48.6% vs. 47.3%), were less often white (69.9% vs. 72.9%), and were more likely to be uninsured or on Medicaid (Table 1). Patients with stroke at hospitals with EHRs also had a lower burden of medical comorbidities than their counterparts at non-EHR hospitals.
In 2007, 8.7% of study hospitals met criteria for at least basic EHRs; this increased to 18.5% in 2008, 26.5% in 2009, and 38.4% in 2010. Hospitals that acquired EHRs by the end of the study period were larger (median 310 vs. 267 beds; p < 0.001), more likely to be located in the Midwest (26.8% vs. 20.4%; p = 0.040), and more often academic institutions (62.8% vs. 44.5%; p < 0.001) than hospitals without EHRs (Table 2). Hospitals with EHRs were also more likely to be Joint Commission–Certified Primary Stroke Centers (44.8% vs. 35.4%; p < 0.001).
Quality of care and clinical outcomes
We first plotted unadjusted outcomes, including receipt of “all-or-none” care, length of stay over 4 days, discharge home, and in-hospital mortality, and noted no obvious differences between patients treated at hospitals with versus without EHRs (Central Illustration).
When we examined the relationship between EHR status and quality of care, we found that patients at hospitals with EHRs were more likely to receive a number of guideline-driven components of care. For example, they were more likely to receive IV tPA if time eligible without contraindications (77.8% vs. 67.7%; p < 0.001) and to have a door-to-needle time of 60 min or less (28.9% vs. 27.3%; p = 0.003), as well as to receive a number of other treatments indicative of high-quality care (Table 3). Patients at hospitals with EHRs were also more likely to receive “all-or-none” care (87.9% vs. 82.6%; p < 0.001).
When we examined clinical outcomes, we found that patients at hospitals with EHRs were less likely to have a length of stay over 4 days (42.4% vs. 43.9%; p < 0.001). There were no differences in discharge home (50.9% vs. 51.1%; p = 0.116) or in-hospital mortality (5.3% vs. 5.2%; p = 0.397) (Table 3).
In multivariable analyses, after controlling for patient and hospital characteristics, the presence of EHRs was not associated with better quality care and continued to have no association with most clinical outcomes. For example, EHRs were not associated with performance on the all-or-none measure (odds ratio [OR]: 1.01; 95% CI: 0.98 to 1.05; p = 0.371) (Table 4). Patterns for in-hospital mortality and discharge home were similar. For length of stay >4 days, however, even after adjustment, a beneficial association was evident (OR: 0.96; 95% CI: 0.94 to 0.99; p = 0.002). These relationships were unchanged after further adjustment for NIHSS (Online Table 1).
When we repeated these analyses with the primary predictor of the presence of EHRs the year before admission, we found similar patterns for all-or-none care (OR: 0.97; p = 0.123 after adjustment) but no relationship with length of stay (OR: 1.02; p = 0.144 after adjustment) (Online Table 2). When we repeated these analyses assessing the relationship of individual functionalities, such as CDS or CPOE, with our quality and outcome metrics, we found similar patterns. For CDS, fully adjusted models showed no relationship with quality or in-hospital mortality; patients admitted to hospitals with CDS were slightly less likely to have a hospital stay longer than 4 days (OR: 0.98; p = 0.049) but were less likely to be discharged home (OR: 0.97; p = 0.012) (Online Table 3). CPOE was associated with lower odds of a hospital stay longer than 4 days (OR: 0.97; p = 0.001), but there were no differences in the other outcomes after adjustment (Online Table 4).
In a large registry of national stroke care, we found no consistent relationships between the presence of EHRs and better quality of care or clinical outcomes for patients with ischemic stroke; however, we did find that patients admitted to hospitals with EHRs had slightly lower odds of having a prolonged length of stay. Although EHRs may be necessary for an increasingly high-tech, transparent healthcare system, they do not appear to be sufficient, at least as currently implemented, to improve overall quality of care or outcomes for this important disease state.
There are a number of possible explanations for why we did not find a relationship between EHRs and the provision of higher-quality care. The first is that EHRs may have been ineffective in changing care because of their design or how they were being deployed to facilitate high-quality stroke care. The definition of EHRs that we chose to use required a specific set of functionalities across a number of work domains and hospital units (10), and we examined not only overall implementation but also implementation of key functionalities, such as CPOE and CDS. However, although we used a rigorous definition of EHRs, implementation alone fails to capture the quality of EHR use. Even though electronic tools were being used, their integration with clinical care and their ability to support quality in real time may have been suboptimal; addressing this is a highly active area of current research. There is also a small risk that EHRs could actually increase medical errors through faulty documentation and/or alert fatigue, as demonstrated in a small number of prior reports (15–17); it is possible that these problems offset any clinical gains that were attained through better decision support tools. This seems less likely, given that in unadjusted analyses, our EHR hospitals performed significantly better than non-EHR hospitals on quality.
It is also possible that added CDS through EHRs for the hospitals in this sample did not further improve outcomes because the hospitals already had a set of nonelectronic mechanisms in place to ensure the delivery of high-quality care for patients with ischemic stroke. Hospitals participating in GWTG-Stroke may have been investing in efforts to improve the quality of care for ischemic stroke regardless of whether or not they have EHRs, and thus, our ability to detect a difference between the 2 groups may have been reduced. EHRs may provide more value to less-experienced clinicians or to clinicians at hospitals with fewer resources or with less experience caring for patients with ischemic stroke.
Although we failed to find a consistent relationship between EHRs and inpatient quality or outcomes, EHRs may improve care in ways that we did not address in this study, such as post-discharge outcomes. For example, EHRs can theoretically improve a hospital’s ability to communicate with patients’ primary care physicians or nursing home staff (18), which may improve long-term outcomes. Given that hospitals are the entities responsible for adopting EHRs to avoid federal penalties, these gains may be more compelling for hospital administrators in the face of the upcoming 30-day mortality and readmission metrics for ischemic stroke. EHRs can also help hospitals with population management because they enable tracking of patients longitudinally in a more formal manner than with paper records; these may be important functionalities as hospitals increasingly move toward programs such as accountable care organizations and bundled payment models.
Finally, it is worth noting that there are clearly downsides to EHRs, and our findings may be disappointing for clinicians and leaders who have justified the added time and frustration that many clinicians note when using EHRs (19) by touting their promise in improving quality and outcomes. Many physicians have positive attitudes toward EHRs as tools having a positive impact on care (20), and few wish to return to paper records (19). However, EHRs are not without controversy; for example, multiple prior studies have documented the workflow disruption that can be associated with components of EHRs (19), such as reminder documentation burden (21) and CPOE (22,23). Much work is ongoing to address these concerns through better design and implementation of electronic tools. Improving implementation could reduce the opportunity costs of using them for busy clinicians.
To our knowledge, this is the first study specifically examining the relationship between EHRs and ischemic stroke, and as such, extends prior work examining the relationship between EHRs and quality for other clinical conditions (5). Many small studies have demonstrated that EHRs are associated with improved quality of care but have often been limited to single-center or otherwise unique electronic record setups (2–4). As we did in this study, the largest prior study examining this issue nationally used the AHA national survey to quantify EHR adoption and use; although it used claims data to assess outcomes, it similarly demonstrated no relationship between the presence of EHRs and quality or outcomes for acute myocardial infarction, congestive heart failure, or pneumonia (6).
The GWTG-Stroke hospitals are a self-selected group of hospitals that has made a commitment to quality improvement, and thus, our findings may not be generalizable to hospitals in this country that differ from GWTG-Stroke participating hospitals. All participating hospitals have access to and may use the GWTG-Stroke patient management tool to facilitate delivery of evidence-based treatment, which may dilute the effect of EHR-based tools. We only evaluated patients with ischemic stroke; our findings may not be generalizable to hemorrhagic stroke or other time-sensitive clinical conditions. Our identification of hospitals with EHRs was based on self-report and thus may be imperfect, and our cross-sectional approach may mask changes in the impact within centers of EHRs over time. The functionalities required to qualify a hospital for basic EHRs are relatively stringent; thus, some hospitals with electronic systems may be classified as non-EHR hospitals in our study. We addressed this by examining functionalities, such as CDS and CPOE, which we thought were the most likely to impact outcomes, but hospitals in our study may still suffer from a degree of misclassification. Because we were not able to assess how the EHRs were used during the study period, we cannot rule out heterogeneity in the impact of EHRs on the basis of the quality of their design and use. Finally, as with all observational analyses, we cannot infer causality, and residual unmeasured confounding may be present.
In our sample of GWTG-Stroke hospitals, EHRs were not associated with overall higher-quality care or better clinical outcomes, although they were associated with slightly lower odds of a prolonged length of stay. Because EHR systems often create significant added burden for clinicians, further work to ensure that they are better integrated with care is critical. Given the ongoing federal push to have EHRs in all hospitals nationwide, our focus should turn to leveraging these tools to their fullest capabilities to improve quality of care and patient outcomes for stroke and other common, important medical conditions.
COMPETENCY IN MEDICAL KNOWLEDGE: Electronic health records can be used to measure the performance of clinicians managing patients with ischemic stroke and compare this performance with benchmarks established by guideline-directed care.
TRANSLATIONAL OUTLOOK: Additional research is needed to develop better ways to leverage electronic health records to improve clinical outcomes for patients with stroke.
For supplemental tables, please see the online version of this article.
The Get With the Guidelines-Stroke (GWTG-Stroke) program is provided by the American Heart Association/American Stroke Association. GWTG-Stroke was previously funded through support from Boehringer Ingelheim, Merck, Bristol-Myers Squibb/Sanofi Pharmaceutical Partnership, Janssen Pharmaceuticals, and the American Heart Association Pharmaceutical Roundtable. The funders and former funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; or preparation, review, or approval of the manuscript. Dr. Bhatt has served on advisory boards for Cardax and Regado Biosciences; and has received research funding from Amarin, AstraZeneca, Bristol-Myers Squibb, Eisai, Ethicon, Forest Laboratories, Ischemix, Medtronic, Pfizer, Roche, Sanofi, and The Medicines Company. Dr. Hernandez has received grant funding from Bristol-Myers Squibb, Janssen Pharmaceuticals, Portola Pharmaceuticals, Medtronic, Novartis, and The Medicines Company; and has received honoraria from AstraZeneca, Bristol-Myers Squibb, Janssen Pharmaceuticals, Novartis, and Gilead. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose. William Weintraub, MD, served as Guest Editor for this paper.
- Abbreviations and Acronyms
- American Hospital Association
- clinical decision support
- computerized physician order entry
- electronic health record
- Get With the Guidelines
- National Institutes of Health Stroke Scale
- tissue plasminogen activator
- Received October 27, 2014.
- Revision received February 9, 2015.
- Accepted February 25, 2015.
- American College of Cardiology Foundation
- ↵Health Information Technology for Economic and Clinical Health (HITECH) Act, 42 USC §§300jj et seq; §§17901 et seq (2009).
- DesRoches C.M.,
- Campbell E.G.,
- Vogeli C.,
- et al.
- ↵Stroke Measures. QualityNet. 2013. Available at: https://www.qualitynet.org/dcs/ContentServer?c=Page&pagename=QnetPublic%2FPage%2FQnetTier3&cid=1228773321281. Accessed March 6, 2015.
- Fonarow G.C.,
- Reeves M.J.,
- Smith E.E.,
- et al.
- Jha A.K.,
- DesRoches C.M.,
- Kralovec P.D.,
- et al.
- Smith E.E.,
- Shobha N.,
- Dai D.,
- et al.
- Fonarow G.C.,
- Smith E.E.,
- Saver J.L.,
- et al.
- Han Y.Y.,
- Carcillo J.A.,
- Venkataraman S.T.,
- et al.
- Friedberg M.W.,
- Chen P.G.,
- Van Busum K.R.,
- et al.
- Ash J.S.,
- Sittig D.F.,
- Poon E.G.,
- et al.
- Niazkhani Z.,
- Pirnejad H.,
- Berg M.,
- et al.