Author + information
- Received August 15, 2011
- Revision received November 3, 2011
- Accepted November 25, 2011
- Published online June 12, 2012.
- Michael S. Lauer, MD⁎ ()
- ↵⁎Reprint requests and correspondence:
Dr. Michael S. Lauer, Office of the Director, Division of Cardiovascular Sciences, National Heart, Lung, and Blood Institute, 6701 Rockledge Drive, Room 8128, Bethesda, Maryland 20892
Policy and science often interact. Typically, we think of policymakers looking to scientists for advice on issues informed by science. We may appreciate less the opposite look: where people outside science inform policies that affect the conduct of science. In clinical medicine, we are forced to make decisions about practices for which there is insufficient, inadequate evidence to know whether they improve clinical outcomes, yet the health care system may not be structured to rapidly generate needed evidence. For example, when the Centers for Medicare and Medicaid Services noted insufficient evidence to support routine use of computed tomography angiography and they called for a national commitment to completion of randomized trials, their call ran into substantial opposition. I use the computed tomography angiography story to illustrate how we might consider a “policy for science” in which stakeholders would band together to identify evidence gaps and to use their influence to promote the efficient design, implementation, and completion of high-quality randomized trials. Such a policy for science could create a culture that incentivizes and invigorates the rapid generation of evidence, ultimately engaging all clinicians, all patients, and indeed all stakeholders into the scientific enterprise.
In late 2007, the Centers for Medicare and Medicaid Services (CMS) reviewed the literature on computed tomography coronary angiography (CTCA) and concluded that there was little, if any, evidence showing that it improves patient outcomes (1). CMS proceeded with a “national coverage determination,” proposing that reimbursement for CTCA be limited to patients who were enrolled in clinical trials. Julie Appleby, a health industry reporter for USA Today described what, in her view, happened next: “Specialty societies representing radiologists and cardiologists were outraged—and they launched a letter-writing campaign seeking to block the national coverage determination…[through which CMS effectively] launched the first salvo in what ultimately became a war. Medicare lost—and CTCA remained a covered benefit for a broad range of patients with possible heart disease” (1).
About 8 months later, during the summer of 2008, the National Heart, Lung, and Blood Institute (NHLBI) hosted a workshop on outcomes research in cardiovascular imaging. The workshop participants proposed designs for trials of imaging in asymptomatic people, patients with symptoms suggestive of myocardial ischemia, patients presenting to emergency wards with suspected acute coronary syndromes, and patients with heart failure (2). Shortly thereafter, the NHLBI received 3 investigator-initiated proposals for randomized trials that would test the ability of CTCA to improve clinical outcome. In September 2009, one of the proposals, the PROMISE (PROspective Multicenter Imaging Study for Evaluation of Chest Pain, NCT01174550) trial, was funded; today the trial, which is led by Drs. Pamela Douglas and Kerry Lee, is enrolling patients. Since then, at least 2 other government-funded randomized trials of CTCA were launched.
The ongoing story of computed tomography (CT) angiography illustrates the tensions that often occur when science and policy cross paths (3). We typically think of “science for policy,” in which policymakers seek scientists' expertise to make informed choices about public health recommendations, safety, or environmental regulations and coverage decisions. But there is also “policy for science,” in which a different group of policymakers, often including scientists, develop policies regarding priorities, strategies, and standards for scientific investigation. In the case of CT angiography, policymakers had to make coverage decisions in the setting of inadequate evidence, but at almost the same time, government scientists and leaders in the imaging community engaged in a dialogue that eventually enabled support for 3 major ongoing randomized trials. What lessons can we learn from the CT angiography story?
Even before the CMS issued its CTCA coverage proposal, clinicians debated about what standards should be applied to cardiovascular imaging. Beginning in the mid 1980s, cardiovascular researchers designed and executed dozens of large-scale practical clinical trials that established high-level evidence for the care of patients with acute ST-segment elevation myocardial infarction, acute coronary syndromes, heart failure due to systolic left ventricular dysfunction, hypertension, hypercholesterolemia, life-threatening arrhythmias, and other conditions. Should similar large-scale trials also be required for diagnostic tests? After all, as specialty societies who protested the CMS proposal argued, “It has been well established in medicine that no diagnostic test improves health outcomes by itself; only the resulting therapeutic interventions may do so” (1). Yet, there are ample precedents for performing outcomes-based randomized trials of diagnostic tests: examples include the fecal occult blood test, mammography, ultrasonography for abdominal aortic aneurysm, the prostate-specific antigen test, ultrasonography and biomarkers for ovarian cancer, myocardial perfusion imaging in asymptomatic patients with diabetes, and, most recently, helical CT for lung cancer (4). In some of these cases, such as the fecal occult blood test, mammography, ultrasonography for aortic aneurysm, and helical CT, randomized trials provided the link showing that performance of the diagnostic test improved health outcomes. In the others, randomized trials showed that performance of the test did not improve outcomes. One cannot automatically assume that a test's ability to predict outcomes or diagnose disease translates into its ability to prevent clinical events (5).
Cardiovascular medicine prides itself as an evidence-based specialty, yet the vast majority of guideline recommendations are based on inferior evidence, evidence well short of multiple randomized trials. Thought leaders in our professional societies deserve credit for conducting and reporting a recent review that found that only 11% of active recommendations rely on the highest level evidence (level A) and that nearly half are based only on “expert opinion” (6). Even fewer recommendations for diagnostic tests were supported by high-level evidence. Some critics decry the common, unquestioning use of high-technology imaging and other diagnostic tests (7). Turka and Caplan (8) go further, calling for a “wholesale rethink about how we approach evidence.” Clinicians should think of non–evidence-based practices, practices for which definitive trials have not been done as “a deviation of standard of care…only [to] be undertaken as part of a clinical trial” (8). Admittedly, many thoughtful physicians would question the wisdom (as well as the practicality) of strictly applying this dictum. Furthermore, it is almost certainly not necessary to require randomized trials for extremely effective treatments such as appendectomy and valve replacement for severe symptomatic aortic stenosis (9).
Meanwhile, thought leaders bemoan increasing difficulties performing randomized trials in the United States (10). Most trials encounter problems with enrollment, budget, and timely completion; a recent Institute of Medicine review of cancer trials found that 40% had to be stopped for failure to enroll (11). There are numerous contributing factors, including excessive complexity that interferes with busy clinical workflow and a culture that does not value highly the enrollment of large numbers of patients into trials (10).
We face a paradoxical situation in which clinicians routinely engage in practices that are not based on high-quality evidence, yet we have difficulty getting trials done. The NHLBI is heavily engaged in developing approaches and policies for increasing the success of randomized trials; for example, in the past year, we have held separate workshops on trial recruitment, best practices of data-coordinating centers, statistical considerations in randomized trials, and priorities in cardiovascular comparative effectiveness research (12). We have also adopted a strict policy on trial accrual, enabling the Institute to more easily stop funding failed efforts (13). We recognize that it is unrealistic to apply randomized trials to all unanswered clinical questions, and therefore a critical challenge is to identify those questions that carry the most overriding medical and societal importance. Carefully performed prospective registries play important roles in assessing the impact and feasibility of candidate trials and in interpreting completed trials; these roles are especially clear in our current difficult financial climate.
Still, we can go further. Suppose clinicians, patients, professional societies, payers, research funding agencies, and academics were to band together and adopt the “wholesale rethink” approach of Turka and Caplan (8), insisting that high-priority questions about unproven practices only occur within the context of rigorous trials. There are partial precedents. American maternal–fetal surgeons agreed that all prenatal surgery for myelomeningocele be performed at 3 centers participating in a National Institutes of Health–funded randomized trial (14). The NHLBI and the CMS agreed to a coverage-for-evidence development arrangement enabling performance of a definitive randomized trial of lung reduction surgery for emphysema (15). For decades, pediatric oncologists worked together to enable 80% to 90% of their patients to be enrolled in randomized trials that were funded by the National Cancer Institute (16). The National Marfan Foundation, a patient advocacy group, has called on its members to avoid using losartan until an NHLBI trial is completed (17). In each case, stakeholders came together, recognized that important clinical questions were unanswered and chose to focus their efforts on enabling rapid design, implementation, and completion of definitive randomized trials. In effect, the stakeholders adopted a policy in which science was viewed as the tool to reach an answer. Science and policy were in synergy, not in tension.
There are tradeoffs of the cost of trials versus the time it takes to sort out what is effective over time without a trial. The story of bone marrow transplantation for metastatic breast cancer is a useful illustration. As pointed out by Brownlee (18), >20 years elapsed between the early adoption of the practice and the first reports from definitive clinical trials. Hundreds of oncologists transplanted tens of thousands of women, “only a fraction of them as part of a clinical trial” at a cost of $3 billion, a sum that exceeds by an order magnitude the costs of the trials themselves (18). As stated by Mark Helfand, “It is never rational to say we must do something now because it will take too long to get the evidence. Such actions time and again have been wrong” (19).
Working in collaboration with professional societies such as the American College of Cardiology, we could explicitly articulate a “policy for science.” Guidelines writers often encounter questions about practices that are unsupported by high-level evidence. Instead of settling for classifying a practice as evidence level B or C, they could adopt Class 1 recommendations to “enroll eligible patients into trial X.” For example, guidelines writers could recommend that if logistically possible, eligible patients with suspected coronary disease be randomized into PROMISE (NCT01174550) or that eligible patients with atrial fibrillation be randomized into CABANA (Catheter Ablation Versus Anti-arrhythmic Drug Therapy in for Atrial Fibrillation Trial, NCT00911508). I recognize that although this activity might support efforts to increase the number and quality of trials, some would think that it exceeds the mandate of guidelines committees and could result in their independence being questioned and the impact of their recommendations thereby reduced. Hence, another option might be to highlight uncertainty and ongoing trials in the text of guidelines documents without explicitly endorsing specific trials.
Widespread adoption of a policy for science could create a culture that incentivizes and invigorates the rapid generation of evidence, ultimately engaging all clinicians, all patients, and indeed all stakeholders into the scientific enterprise. Admittedly, it will be difficult to accomplish in a fragmented health care system, one that is still burdened by many forces that promote overutilization (20). Yet, had such a culture existed in 2007, we would have faced pressures to get many CT angiography trials launched and completed. Soon, we would find ourselves occupied with interpreting findings and disseminating evidence-based practices. Most important, instead of being seen as fighting wars, we would instead be perceived correctly as working together to ensure that our patients benefit from the incredible power of the scientific method (21).
Dr. Lauer has reported that he has no relationships relevant to the contents of this paper to disclose. The views expressed herein are those of the author and do not necessarily reflect the official views of NHLBI.
- Abbreviations and Acronyms
- Centers for Medicare and Medicaid Services
- computed tomography
- computed tomography coronary angiography
- National Heart, Lung, and Blood Institute
- Received August 15, 2011.
- Revision received November 3, 2011.
- Accepted November 25, 2011.
- American College of Cardiology Foundation
- Appleby J.
- Douglas P.S.,
- Taylor A.,
- Bild D.,
- et al.
- Jasanoff S.
- Glasziou P.,
- Chalmers I.,
- Rawlins M.,
- McCulloch P.
- Califf R.M.,
- Harrington R.A.
- Institute of Medicine
- ↵NHLBI Meeting Summaries and Scientific Reports. http://www.nhlbi.nih.gov/resources/docs/. Accessed August 7, 2011.
- ↵Guidance and Implementation for Monitoring Adequacy of Accrual of Participants to NHLBI Supported Human Subjects Research. http://www.nhlbi.nih.gov/funding/policies/accrual_guidelines.htm. Accessed August 7, 2011.
- ↵National Marfan Foundation: Losartan vs. Atenolol Clinical Trial. http://www.marfan.org/marfan/2408/Atenolol-vs.-Losartan-Clinical-Trial. Accessed August 7, 2011.
- Brownlee S.
- Cutler D.M.,
- McClellan M.