Author + information
- Sudhakar V. Nuti, BA,
- Isuru Ranasinghe, MB ChB, MMed, PhD,
- Karthik Murugiah, MD,
- Abbas Shojaee, MD,
- Shu-Xia Li, PhD and
- Harlan M. Krumholz, MD, SM∗ ()
- ↵∗Center for Outcomes Research and Evaluation, Yale-New Haven Hospital, 1 Church Street, Suite 200, New Haven, Connecticut 06510
It is unknown whether some cardiovascular journals achieve high impact factors (IFs) due to citations from the entire spectrum of their papers or from the publication of a few highly-cited papers, such as clinical practice guidelines. To address this question, we quantified the variation in cardiovascular journals’ citation distributions and whether the variation was associated with the IF. Uneven citation distributions can be studied with techniques that assess inequality. One such measure is the Gini coefficient, which is utilized to characterize inequality in income distributions. It has a range of 0 to 1, where a Gini coefficient of 0 represents perfect equality, meaning, in this case, that all papers have an equal number of citations (1).
We identified all journals from 2010 that primarily published cardiovascular content (n = 241) using SCImago (Granada, Spain), a third-party provider that categorizes journals into fields on the basis of journal data obtained from the Scopus citation database. We used Scopus because it is a larger citation database than Thompson Reuters’ Web of Science, which provides the journal IF (2). We excluded journals with <20 papers (n = 54) and <20 citations (n = 5) to avoid the downward bias associated with small samples sizes with the Gini calculation (3). We then excluded journals not indexed in Web of Science (without a 2012 IF) (n = 53) (4). Among included journals, we assessed the total number of citable documents (original articles and reviews [including guidelines] in 2010; heretofore, we will refer to all of these documents as “papers”) and the cumulative citations from 2010 to 2012 for each paper.
For each journal, we ranked the journal’s papers by the number of citations (from lowest to highest) and calculated the cumulative paper count as a proportion of the total number of the journal’s papers (Xk) and the cumulative citation count as a proportion of the total cumulative citations of all papers within the journal (Yk). We then estimated the Gini coefficient using the equation for approximating Gini using discrete data points (1,5):
We used linear regression to quantitatively assess the relationship between the Gini coefficient and the logarithmically-transformed IF. The log transformation was done to satisfy the homoscedasticity assumption for the linear regression. To determine the effect of guidelines, statements, and statistical reports on citation distributions, we excluded them from the top 3 IF journals and recalculated their Gini coefficients. All analyses were conducted using Stata version 12.0 (Stata Corp Inc., College Station, Texas).
There were 129 journals included in our analysis. IFs of journals ranged from 0.2 to 15.2, with a median of 2.4 (interquartile range: 1.2 to 3.7). The Gini coefficients for cardiovascular journals ranged from 0.36 to 0.82, with a median of 0.54 (interquartile range: 0.50 to 0.61).
There was an inverse relationship between the Gini coefficient and the log of the IF (coefficient: −0.13, p < 0.001; adjusted R2 = 0.26), with journals with higher IFs having lower Gini coefficients (Figure 1), signifying that higher-impact journals have a more equal distribution of citations compared with lower-impact journals. After removing the highly-cited documents, the Gini coefficients of the top 3 IF journals decreased by 0.05, 0.01, and 0.03, respectively.
There is a relationship between more equal citation distributions and higher IF cardiovascular journals. Our findings reassure readers that IFs are generally representative of the quality and contribution of the entire spectrum of a journal’s papers and not just a select few. Nevertheless, there was high variation overall in citations.
Among the 3 highest-impact journals, there is evidence of inequality. However, these journals’ Gini coefficients became more equal, to different degrees, after removing high-citation documents, suggesting that these papers have a marked impact on a given journal’s citation distribution. A limitation of the study is that factors other than citations may influence the IF of a journal, particularly the definition of citable documents. The Gini coefficient is a useful measure of citation inequalities.
Please note: This work was supported by grant U01 HL105270-05 (Center for Cardiovascular Outcomes Research at Yale University) from the National Heart, Lung, and Blood Institute in Bethesda, Maryland. Dr. Krumholz works under contract with the Centers for Medicare & Medicaid Services to develop and maintain performance measures; is chair of a cardiac scientific advisory board for UnitedHealth; is the recipient of research grants, through Yale University, from Medtronic and from Johnson & Johnson to develop methods of clinical trial data sharing; and is editor of Circulation Cardiovascular Quality and Outcomes and Journal Watch and CardioExchange, Massachusetts Medical Society. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.
- American College of Cardiology Foundation
- ↵Gini CW. Variability and mutability, contribution to the study of statistical distribution and relations. In: Studi Economico-Giuricici della R. Universita de Cagliari, 1912.
- ↵Thomson Reuters. Journal Citation Reports. 2014. Available at: http://thomsonreuters.com/journal-citation-reports/. Accessed September 9, 2014.
- Catalano GD. Engineering, poverty, and the earth. Synthesis Lectures on Engineering, Technology, and Society 2007;2:34.