Author + information
- Anthony N. DeMaria, MD, MACC, Editor-in-Chief, Journal of the American College of Cardiology∗ ()
- ↵∗Address correspondence to:
Dr. Anthony N. DeMaria, Editor-in-Chief, Journal of the American College of Cardiology, 3655 Nobel Drive, Suite 630, San Diego, California 92112
An issue has recently arisen at the Journal that has implications well beyond a few specific papers referenced in Letters to the Editor. The core of the issue relates to differences in data for a given project between the published manuscript and prior versions in abstract form, or even oral presentations. Inquiries concerning several specific papers sought clarification of apparent discrepancies between such various presentations in order to fully understand the findings. However, the questions that were posed invariably raised the unspoken possibility of bias in the handling of the data published. Although seemingly at first blush a simple matter to address, the complexity and implications of this issue were sufficient to provoke considerable discussion and varied opinions at conferences of journal editors at the European Society of Cardiology meeting. It seems clear that it will be important to reach a general agreement on how to handle such matters in the future.
When I first entered the arena of academic medicine over 35 years ago, it was not at all uncommon to submit partially completed studies for abstract presentation, or to include such data in invited lectures. In terms of abstracts, it was assumed that more complete data would be obtained in the interval before the meeting, and accepted that the findings might change considerably. I distinctly recall submitting one abstract based on 8 experiments that was essentially entitled “A Is Greater Than B.” By the time of presentation we had doubled our sample and now found that B was greater than A. Our first slide, therefore, contained the initial title of our abstract with a line through it, and a second title below describing the actual findings. No one seemed to find this the least bit objectionable. Not only was it accepted that abstract findings might change, but in fact, one of the assumed purposes of abstract talks was to obtain constructive criticism so as to improve the research project going forward. A similar approach was taken to preliminary data included in invited lectures.
Although considerable latitude has always been given to preliminary presentations of data, there have also been bounds beyond which questions would arise. Concerns were nearly never expressed when the results of new experiments were added to preliminary data, or when a change in protocol yielded different findings. However, when the number of experiments or patients decreased markedly between abstract and paper, or especially when the number of deaths or adverse events were substantially less, or when measurements presented in the preliminary version were omitted in the final manuscript, questions were often raised as to the criteria upon which such changes were based. Nevertheless, even here it was assumed that final adjudication by endpoint committees or enhanced ability to carry out a protocol was the likely explanation for the variations in data in different versions of the results.
Much has changed in the world of academic medicine over the last 35 years. A new focus upon potential conflicts of interest and relations with industry has become part of our daily life. Competition for scarce research funds or limited academic promotions has blunted some of the basic trust that existed in the scientific community. In fact, concerns regarding potential research misconduct have been borne out by a number of high-profile documented instances of fraudulent data within the medical investigative field. The net result has been a less benign view of differences between preliminary and published versions of research studies, and a greater suspicion of bias.
Investigators whose work exhibits discrepancies between abstract presentations and the published paper point to the fact that abstracts are preliminary and that raw data included in such formats are subjected to additional careful and precise evaluation, often by core laboratories, before final acceptance and classification. They indicate that data from several different protocols may be combined for presentations in order to provide a picture of the overall status of the work. They contend that preliminary presentations were never intended to be definitive and that data from such versions should be disregarded when the final completed paper is published.
While the above assertions of investigators may well be true, and might provide a rational explanation for discrepancies in presented data, they have significant implications regarding the status of abstracts and invited presentations. Data from abstracts continue to be cited in published articles and are often incorporated into meta-analyses. As was recently evidenced in Amsterdam, medical meetings continue to be enthusiastically attended by large numbers of physicians, scientists, and healthcare professionals. The attendees at abstract presentations assume that the data presented have been carefully gathered and analyzed, and are accurate. If we were to accept that such data are preliminary, not fully analyzed and subject to change, and that all bets were off as to what the final published paper might report, attendance at such venues would lose much of its attraction. Therefore, recognizing the justifiable divergence that may occur in data between (especially the writing of) abstracts and publications, it seems reasonable to hold investigators responsible for some degree of accuracy and reproducibility.
So, the issue of discrepancies between abstracts/presentations and published papers has recently received considerable attention and provoked some spirited debate. At one extreme, proponents argue that investigators should be held accountable for data contained in abstracts and presentations, and that the existence of differences between those versions and published data raise the specter of biased analysis. At the other extreme, proponents maintain that data contained in abstracts and presentations should be recognized as preliminary and susceptible to change and that inconsistency with published manuscripts should be generally ignored. Obviously, the optimal resolution lies squarely in between these 2 extremes.
It seems to me that it is reasonable to expect differences between preliminary reports such as abstracts and the final published paper. However, it seems equally reasonable that investigators should be held to account for such inconsistencies. I, therefore, believe that authors should proactively indicate in their manuscripts when such discrepancies exist and provide an explanation. Some may argue that an explanation from the authors is inadequate, and that the editors should conduct an investigation to determine the veracity of the account, including examination of raw data if necessary. However, there continues to be the assumption of honesty for investigators, and the clarification of divergence of data in a manuscript is really no different than the presentation of any data in any article. In addition, editors rarely have the resources to conduct such investigations, and must rely on the participation of the home institution. So, not to excuse editors of taking responsibility for the integrity of the material they publish, I think investigators should, with infrequent exception, be assumed innocent and have their explanations accepted. If the foregoing conditions are agreed to, proactive clarification of differences in data between various versions of study results should allay concerns of readers, strengthen the confidence in the conclusions, and limit the need for extensive post-publication explanations.
Parenthetically, similar concern has been directed to divergence between the data proposed to be included in research protocols filed in open databases (e.g., clinicaltrials.gov), and those actually reported in the finished manuscript. As with abstracts, it is fully understandable that protocols may change as experience is gathered and data acquired while performing the experiments. However, investigators must take the responsibility to update the protocol description, and more important, describe the changes that were made and the rationale for these changes in the final manuscript. Such maneuvers would not only allay concern and strengthen confidence, but also alleviate the need for extensive post-publication explanation.
We have been blessed in medicine in that over many years there have been few major instances of scientific misconduct in the medical literature. I may be a Pollyanna, but I continue to believe that the faith placed in the integrity of medical investigators has been well justified. However, as is apparent from the appearance of blogs such as Retraction Watch, there is more attention to and suspicion of our research publications, and more evidence of inaccurate data. In view of our changing environment, it seems prudent to explain in detail any divergence between preliminary presentations and published articles, and between proposed protocol and reported results, even if the details seem excessive. As the cliché goes: an ounce of prevention …
- American College of Cardiology Foundation