Author + information
- Received November 14, 2013
- Accepted November 26, 2013
- Published online April 8, 2014.
- Gregory J. Dehmer, MD∗∗ (, )
- Joseph P. Drozda Jr., MD†,
- Ralph G. Brindis, MD, MPH‡,
- Frederick A. Masoudi, MD, MSPH§,
- John S. Rumsfeld, MD, PhD‖,
- Lara E. Slattery, MHS¶ and
- William J. Oetgen, MD, MBA¶,#
- ∗Texas A&M Health Science Center, Division of Cardiology, Baylor Scott & White Health, Temple, Texas
- †Mercy Health, Chesterfield, Missouri
- ‡Department of Medicine and the Philip R. Lee Institute for Health Policy Studies, University of California, San Francisco, San Francisco, California
- §Division of Cardiology, University of Colorado Anschutz Medical Campus, Aurora, Colorado
- ‖Denver VA Medical Center, Denver, Colorado
- ¶American College of Cardiology, Washington, DC
- #Department of Medicine, Georgetown University, Washington, DC
- ↵∗Reprint requests and correspondence:
Dr. Gregory J. Dehmer, Cardiology Division (MS-33-ST156), Baylor Scott & White Health, 2401 South 31st Street, Temple, Texas 76508.
Public reporting of hospital and individual provider quality of care measures is not a new concept. In the United States, the first national public reports of hospital mortality data occurred in 1986, and detailed physician-level data for cardiac surgery are now reported in 4 states. The development of the “Hospital Compare,” and more recently, the “Physician Compare” websites has further expanded public reporting for hospitals and providers. Several professional organizations, including the American Medical Association, Society of Thoracic Surgeons, and the American College of Cardiology, have published policy statements articulating key principles to guide the public reporting process. Despite the rapid proliferation of public reporting efforts, more research is needed to better define meaningful measures to report and fully understand the impact of public reporting on healthcare delivery.
“The very first requirement in a hospital is that it should do the sick no harm.”
— Florence Nightingale, 1859 (1)
In the mid-19th century, Florence Nightingale published mortality rates at British military hospitals caring for war casualties in what is believed to be the earliest attempt at public reporting (1). About 50 years later, Dr. Ernest Codman, an advocate of hospital reform, endured the criticism of his colleagues after calling for the public release of surgical outcomes (2). Although his peers rejected Codman’s vision, his efforts were central to the founding of the American College of Surgeons and The Joint Commission. Since that time, release of information into the public domain about the performance of healthcare systems and individual providers has grown. With the current national emphasis on quality improvement, accountability, and cost effectiveness in health care, stakeholders, such as government, purchaser, provider organizations, and consumers, are seeking information to inform decisions about healthcare facilities and providers (Table 1). Hospital-level public reporting exists in several formats and is now familiar to most clinicians. Although the methods are less developed, public reporting of individual provider data is rapidly progressing. The most compelling justification for public reporting is the public’s right to know about the care that they are likely to receive from hospitals and physicians. Public reporting is fundamentally based on the belief that accessible, transparent quality information will affect decisions and behaviors of the various stakeholders, ultimately resulting in an improvement in healthcare delivery and outcomes.
After Codman, little occurred to nurture transparency and accountability in health care until the late 1980s when the Health Care Financing Administration (HCFA), the predecessor of the Centers for Medicare and Medicaid Services (CMS), published risk-adjusted death rates in U.S. hospitals. Although originally not intended for public release, these reports became public and were widely criticized (3–6). Although the HCFA experience was fraught with challenges, it stimulated development of other quality improvement registries and statewide reporting systems, such as the Northern New England Cardiovascular Study Group, the Society of Thoracic Surgeons (STS) National Adult Cardiac Surgery Database, and the reporting of cardiac surgery and percutaneous coronary intervention (PCI) outcomes in several states (7–13). With implementation of the “Hospital Compare” website in 2005, CMS re-established public reporting for Medicare beneficiaries aggregated at the hospital level, initially with process measures for common conditions (14). Public reporting initiatives include: 1) state initiatives, with some including price transparency; 2) reports from payers; 3) reports from business consumer groups; and 4) reports from independent organizations that display data in a simple format and provide proprietary analysis and ratings using methodology that is nontransparent (15–18). Some payer organizations have a greater focus on cost-profiling physicians; the accuracy of these methods has been questioned (15). There are also multiple internet-based forums where patients report their anecdotal experiences with physicians (19).
Beginning in September 2010, STS, in partnership with Consumers Union, started voluntary publishing information on the performance of coronary artery bypass graft procedures in Consumer Reports (20). Previously, STS data were used as benchmarks to stimulate local quality improvement, but were not available publically. This collaboration represented the first national public reporting effort led by a professional organization and was well received (21). Within 3 years of its inception, 50% of cardiovascular groups voluntarily reported their STS ratings.
Passage of the Patient Protection and Affordable Care Act of 2010 (ACA) created a new framework by mandating a national strategy for quality improvement, including public reporting of healthcare quality information. Two federal agencies, the Agency for Healthcare Research and Quality and CMS, share responsibility for these activities. By law, these agencies are required to engage all relevant stakeholders and develop detailed performance, quality, and cost measures to meet the needs of patients. The ACA also called for a multi-stakeholder group to: 1) identify the best available performance measures for use in specific applications; 2) provide input to the Department of Health and Human Services on measures for use in public reporting, performance-based payment, and other programs; and 3) encourage alignment of public and private sector efforts. The Secretary of the Department of Health and Human Services selected the National Quality Forum (NQF) to perform these functions, and subsequently, the NQF convened a public–private partnership to assist in the selection of performance measures (22,23). Public reporting will be used for insurance plans offered through new state-level health insurance exchanges and for participants in Medicare’s “value-based purchasing” program. Some of these initiatives have already started, with implementation of the “Physician Compare” website by CMS (24). Beginning in 2014, Physician Compare will include quality of care ratings for group practices, with individual ratings added in the future.
Potential Benefits and Unintended Consequences of Public Reporting
Public reporting is intended to improve healthcare delivery and patient outcomes by making quality measures transparent and easily available. Literature on the impact of public reporting is limited, but positive examples are emerging. For example, a national survey from 2008 showed that patients with vascular disease were prescribed prophylactic aspirin by only 35% to 47% of physicians among different specialties (25). However, in Minnesota, which publically reports the use of aspirin prophylaxis, the rate is 95% (26). Likewise, recent data from the Wisconsin Collaborative for Healthcare Quality showed that large group practices will engage in quality improvement efforts and show improvement in response to public reporting, especially when comparative performance is displayed on a website (27,28). Survey data from administrators, physicians, and nurses indicate that public reporting: 1) leads to greater involvement of leadership in performance improvement; 2) creates a sense of accountability to internal and external customers; 3) contributes to a heightened awareness of performance measure data throughout the facility; 4) re-focuses organizational priorities; 5) raises concerns about data quality; and 6) leads to questions about consumer understanding of performance reports (29). However, reviews of public reporting confirm that a rigorous evaluation of many major public reporting systems is lacking, and there are minimal data about public reports of individual provider data and practices (30,31). These reviews cite evidence that publicly releasing performance data stimulates quality improvement activity at the hospital level, but conclude that the overall effect of public reporting on effectiveness, safety, and patient-centeredness remains uncertain.
Studies have also reported unintended consequences of public reporting. The majority of reports highlight the development of risk adverse behavior among physicians and facilities subject to public reporting. This was shown for coronary artery bypass graft surgery in both New York and Pennsylvania, and similar risk adverse behavior was reported for PCI (32–35). In several studies, patients with acute myocardial infarction and cardiogenic shock were less likely to receive PCI in states with public reporting (35–37). In Massachusetts, the risk profile of PCI patients at hospitals identified as having higher than expected mortality was significantly lower after public identification when compared with nonoutlier institutions (38,39). This was partially negated by the inclusion of a “compassionate use” variable into the mortality risk calculation (40). Nevertheless, there is concern that mortality alone is not a good metric to judge the quality of a PCI program (41–43).
In addition to concerns about unintended consequences, there are questions about the accuracy of some reported data. When HCFA data were used to generate hospital mortality reports, there was considerable concern about the potential inaccuracies of administrative (claims) data for this purpose, and these concerns still exist (5,6,44). For example, in comparative studies of cardiac surgery performance using administrative versus clinical data sources, considerable disparities were found, leading to the conclusion that report cards using administrative data were problematic (45–47). Clinical registry data have several advantages over administrative data that are currently the substance of many public reports (48,49) (Table 2).
How Should Public Reports Be Used by Patients and Purchasers?
Although the public has adopted the use of easily available product evaluations to guide decisions about major purchases, consumers have been slow to use comparative information to make healthcare choices. However, this is changing; data from the Pew Internet Research Project indicate that among those with Internet access, 55% have sought medical information from the Internet (50,51). In theory, the use of public reports should facilitate 3 key functions. First, these data should help consumers make informed and better choices about where to obtain health care for themselves and their family. Second, these data should stimulate quality improvement among provider groups as a way to protect or enhance their market share, especially in more competitive markets if they perceive that performance data may affect consumer choice. Finally, access to these data should encourage providers to improve their quality of care and encourage purchasers and health plans to use higher-quality providers in their networks (52,53).
To make public reporting helpful to consumers, it is important to understand that consumers and clinical experts may define quality differently. The top factors consumers identified as being most important in determining the quality of health care were affordability, the physician’s qualifications, and access to care for everyone (54). This is clearly different from the concept of healthcare quality represented in most public performance reports, which often include technical measures of quality and patient experiences. Consumers can also misunderstand reported quality measures. For example, longer length of stay is intended to indicate poor performance, but some consumers may incorrectly believe this a favorable finding. Other measures may be incomprehensible to consumers, such as why certain medications are necessary for some conditions (55).
Research Is Needed to Improve Public Reporting
The effectiveness of public reporting, including both potential benefits and unintended consequences, has not been convincingly proven, and thus, more research is needed (30,31). Presently, it appears that public reporting is more likely to have an impact on healthcare providers than on consumers. Some process measures of quality improve over time, but changes in outcomes like mortality have been more difficult to assess due to many confounders. The most consistent evidence supporting the impact of public reporting comes from the long-term care environment, where there have been more studies of improvements in quality measures due to Nursing Home Compare and Home Health Compare (56,57). Future research should focus on: 1) identifying which types of measures (process, outcome, safety, cost, access, or patient experience) are most meaningful to consumers, and thus, likely to have the greatest impact; 2) better defining the potential for harm to facilities and providers from public reporting; 3) determining the balance of benefits and harms of public reporting of individual physician performance, particularly when process measures are used to assess care and determining attribution of a failure to an individual may be difficult; 4) determining the best formats for presentation of healthcare information to consumers; and 5) assessing whether public reporting results in a change in consumer behavior, healthcare quality, and cost savings.
Why Should Professional Societies Engage in Public Reporting?
Public reporting of healthcare provider performance is here to stay and will continue to grow, including not only process measures, but also outcome measures. Physician engagement is essential and can be facilitated by professional organizations. Together, cardiovascular clinicians and professional societies should take a leadership role to: 1) continue the development of meaningful performance measures; 2) optimize the validity of publically reported information; 3) minimize unintended consequences; 4) promote the use of clinical data to improve public reporting; 5) ensure a link to quality improvement is maintained; and 6) develop ways to use public reporting in ongoing professional development (58).
Public Reporting and Professional Societies
In anticipation of the increase in public reporting, the American College of Cardiology (ACC) developed a health policy statement in 2008 defining 6 core principles of public reporting (59) (Table 3). With great attention to these core principles, the ACC, in partnership with the Society for Cardiovascular Angiography and Interventions and the Heart Rhythm Society, studied the feasibility of public reporting of certain cardiovascular performance measures using data from the National Cardiovascular Database Registry (NCDR). The main advantage is that these data derive from clinical sources rather than administrative data. Independent audits of NCDR data show an accuracy of approximately 90% in several registries compared with source documents (60). All publically-reported NCDR measures must be approved by the NQF, which is a lengthy and rigorous process (61). In addition, the ACC/American Heart Association (AHA) Task Forces on Practice Guidelines and Performance Measures provide guidance on the measures submitted to the NQF for consideration. All measures are evidence-based, predicated on the strongest guideline recommendations generated by the ACC/AHA Task Force on Practice Guidelines, and are developed according to standardized methodology promulgated by the ACC/AHA Task Force on Performance Measures.
The initial measures reported consist of 3 process measures and 4 outcome measures derived from the NCDR’s CathPCI and Implantable Cardioverter Defibrillator Registries (62,63) (Table 4). These measures will be displayed on the Hospital Compare website and will also be posted on other sources, including CardioSource or CardioSmart, when the measures appear in the public domain. A pilot project using 30-day readmission following PCI has already started, with approximately 300 facilities voluntarily reporting their results.
The Future of Public Reporting
The future of public reporting is evident in CMS’s plans for the Physician Compare website (24). The ACA stipulates public reporting of performance measures for physicians, nurse practitioners, physician assistants, and 19 other types of healthcare providers who provide service to Medicare beneficiaries. As this law is phased in, CMS is also charged with developing a plan to provide larger payments to physicians who provide “high-quality care” compared with cost. In addition to measures already collected as part of the Physician Quality Reporting System, metrics reflecting the continuity and coordination of care, patient experience, appropriateness, and timeliness of care are being considered. These categories are based on the Institute of Medicine’s 6 domains of quality (care that is safe, effective, patient-centered, timely, efficient, and equitable) and have been used to measure the quality of health plans and community clinical services (64). Other issues under consideration by CMS are: 1) developing additional measures that are statistically valid, reliable, and risk-adjusted; 2) allowing providers the ability to review results before they are published; and 3) including data reflecting care rendered to all patients, not just Medicare beneficiaries, if that provides a more accurate picture of physician performance.
In the future, U.S. consumers will likely be paying more of their personal healthcare costs, and that will drive patients to seek greater value. Just as consumers search the Internet for the lowest price of an appliance, patients in the future will search for quality and value in health care based upon transparent and reliable data. The challenge moving forward is to ensure public reporting occurs in a fair, accurate, and meaningful way that benefits patients and minimizes the possibilities of unintended negative consequences. Greater involvement of professional societies and employing clinical data in reporting are 2 desirable ways to improve public reporting and guide it to the right place as healthcare delivery evolves. By following clinical practice guidelines, knowing their personal and their institution’s adherence to performance measures, and giving proper attention to appropriate use criteria, cardiovascular specialists will help ensure that they are prepared for the future. Attention to these areas in daily practice can safeguard a clinician from being identified as an outlier.
Health care is changing dramatically in the United States, and cardiovascular professionals face a future likely to be quite different from the past. With valid data, meaningful measures, and a commitment to continuous improvement, cardiovascular specialists and the care teams that assist them should be prepared to succeed in an era of public reporting and to be leaders in the transparent and accountable healthcare system ahead.
Dr. Dehmer is Chair of the National Cardiovascular Data Registry (NCDR) Public Reporting Advisory Group, a volunteer position. Dr. Brindis is the Senior Medical Officer, External Affairs for and receives compensation from the NCDR. Dr. Masoudi is a Senior Medical Officer of and receives compensation from the NCDR; and has served as a consultant to the Oklahoma Foundation for Medical Quality. Dr. Rumsfeld is Chief Science Officer for and receives compensation from the NCDR. Ms. Slattery and Dr. Oetgen are employees of the American College of Cardiology. Dr. Drozda has reported that he has no relationships relevant to the contents of this paper to disclose. The views expressed in this paper are those of the authors and do not necessarily represent those of the American College of Cardiology Foundation.
- Abbreviations and Acronyms
- Patient Protection and Affordable Care Act of 2010
- American College of Cardiology
- American Heart Association
- Centers for Medicare and Medicaid Services
- Health Care Financing Administration
- National Cardiovascular Data Registry
- National Quality Forum
- percutaneous coronary intervention
- Society of Thoracic Surgeons
- Received November 14, 2013.
- Accepted November 26, 2013.
- American College of Cardiology Foundation
- ↵D. Appleton and Company [First American Edition]. New York. 1860.
- Codman E.A.
- Health Care Financing Administration
- Likosky D.S.,
- Nugent W.C.,
- Ross C.S.
- Adult cardiac surgery in New York State, 2005-2007. Available at: http://www.health.state.ny.us/statistics/diseases/cardiovascular/heart_disease/docs/2005-2007_adult_cardiac_surgery.pdf. Accessed July 13, 2013.
- Pennsylvania’s Guide to Coronary Artery Bypass Graft (CABG) Surgery: 2009 Hospital Data. Available at: http://www.phc4.org/cabg. Accessed July 13, 2013.
- Massachusetts Data Analysis Center. Fiscal year 2011 Reports for Risk-Standardized Mortality Rates. Available at: http://www.massdac.org/reports/surgery.html. Accessed July 13, 2013.
- California Healthcare Information Division. Coronary Artery Bypass Graft (CABG) Surgery in California–2009. Available at: http://www.oshpd.ca.gov/HID/Products/Clinical_Data/CABG/index.html. Accessed July 6, 2013.
- New York State Department of Health. Percutaneous Coronary Interventions (Angioplasty) in New York State: 2008-2010. Available at: http://www.health.ny.gov/statistics/diseases/cardiovascular. Accessed July 13, 2013.
- ↵Centers for Medicare & Medicaid Services. Hospital quality initiatives: hospital compare. Available at: http://www.medicare.gov/hospitalcompare/search.html. Accessed July 13, 2013.
- The Leapfrog Group. Available at: http://www.leapfroggroup.org. Accessed July 30, 2013.
- Healthgrades. Available at: http://www.healthgrades.com. Accessed July 30, 2013.
- Catalyst for Payment Reform. Available at: http://www.catalyzepaymentreform.org. Accessed July 12, 2013.
- ↵Aleccia J. Doctors seek to stifle patients’ rants on Web sites. NBCNews.com. January 13, 2010. Available at: http://www.nbcnews.com/id/34794632/ns/health-health_care/t/docs-seek-stifle-patients-rants-web-sites/#.UdgFFxZjA5Q. Accessed July 6, 2013.
- ↵Consumer Reports. Looking for a heart surgeon? These ratings can help you find a good one. Available at: http://www.consumerreports.org/health/doctors-hospitals/surgeon-ratings/heart-surgery-ratings/overview/index.htm. Accessed July 6, 2013.
- ↵National Quality Forum. Measures Application Partnership. Available at: http://www.qualityforum.org/Setting_Priorities/Partnership/Measure_Applications_Partnership.aspx. Accessed January 21, 2013.
- National Quality Forum. Measures Application Partnership Pre-Rulemaking Report: Public Comment Draft. Available at: http://nqf.informz.net/NQF/data/images/map_pre-rulemaking_report_for_comment.pdf. Accessed January 21, 2013.
- ↵Medicare.gov. Physician Compare. Available at: http://www.medicare.gov/physiciancompare/search.html. Accessed August 11, 2013.
- George M.G.,
- Tong X.,
- Sonnenfeld N.,
- Hong Y.
- ↵Minnesota Health Scores. Available at: http://www.mnhealthscores.org/?p=our_reports&sf=clinic&category_section=category_condition&category=2&sub_category=9&name_id=&compare=&search_phrase=&zipcode=&within=5. Accessed July 27, 2013.
- Lamb G.C.,
- Smith M.A.,
- Weeks W.B.,
- Queram C.
- Smith M.A.,
- Wright A.,
- Queram C.,
- Lamb G.C.
- Hafner J.M.,
- Williams S.C.,
- Koss R.G.,
- Tschurtz B.A.,
- Schmaltz S.P.,
- Loeb J.M.
- Ketelaar N.A.B.M.,
- Faber M.J.,
- Flottorp S.,
- Rygh L.H.,
- Deane K.H.O.,
- Eccles M.P.
- Omoigui N.A.,
- Miller D.P.,
- Brown K.J.,
- et al.
- Moscucci M.,
- Eagle K.A.,
- Share D.,
- et al.
- Apolito R.A.,
- Greenberg M.A.,
- Menegus M.A.,
- et al.
- Resnic F.S.,
- Welt F.G.
- McCabe J.M.,
- Joynt K.E.,
- Welt F.G.P.,
- Resnic F.S.
- Resnic F.S.,
- Normand S.-L.T.,
- Piemonte T.C.,
- et al.
- Aggarwal B.,
- Ellis S.G.,
- Lincoff A.M.,
- et al.
- Pinto D.S.,
- Pride Y.B.
- Dehmer G.J.
- Shahian D.M.,
- Silverstein T.,
- Lovett A.F.,
- Wolf R.E.,
- Normand S.L.
- Hammill B.G.,
- Curtis L.H.,
- Fonarow G.C.,
- et al.
- Tang P.C.,
- Ralston M.,
- Arrigotti M.F.,
- Qureshi L.,
- Graham J.
- ↵Pew Internet: The Online Health Care Revolution. Available at: http://www.pewinternet.org/Reports/2000/The-Online-Health-Care-Revolution.aspx. Accessed November 2, 2013.
- Bardach N.S.,
- Hibbard J.H.,
- Dudley R.A.
- Hibbard J.H.,
- Stockard J.,
- Tusler M.
- Hibbard J.H.,
- Stockard J.,
- Tusler M.
- ↵(2004) National Survey on Consumers’ Experiences With Patient Safety and Quality Information–Summary and Chartpack (Kaiser Family Foundation, Menlo Park, CA) Available at: http://www.kff.org/kaiserpolls/7209.cfm. Accessed October 31, 2013.
- Hibbard J.,
- Sofaer S.
- Zoghbi W.A.,
- Gillis A.M.,
- Marshall J.J.
- Drozda J.P. Jr..,
- Hagan E.P.,
- Mirro M.J.,
- Peterson E.D.,
- Wright J.S.
- Messenger J.C.,
- Ho K.K.,
- Young C.H.,
- et al.,
- NCDR Science and Quality Oversight Committee Data Quality Workgroup
- ↵National Quality Forum. Measuring Performance. Available at: http://www.qualityforum.org/Measuring_Performance/Measuring_Performance.aspx. Accessed July 13, 2013.
- ↵National Cardiovascular Data Registry. CathPCI Registry. Available at: https://www.ncdr.com/webncdr/cathpci. Accessed July 13, 2013.
- National Cardiovascular Data Registry. ICD Registry. Available at: http://www.ncdr.com/webncdr/ICD/default.aspx. Accessed July 13, 2013.
- Institute of Medicine
- Historical Perspective
- Potential Benefits and Unintended Consequences of Public Reporting
- How Should Public Reports Be Used by Patients and Purchasers?
- Research Is Needed to Improve Public Reporting
- Why Should Professional Societies Engage in Public Reporting?
- Public Reporting and Professional Societies
- The Future of Public Reporting