This article analyses the difference in timing between the online availability of articles and their corresponding print publication and how it affects two bibliometric indicators: Journal Impact Factor (JIF) and Immediacy Index. This research examined 18,526 articles, the complete collection of articles and reviews published by a set of 61 journals on Urology and Nephrology in 2013 and 2014. The findings suggest that Advance Online Publication (AOP) accelerates the citation of articles and affects the JIF and Immediacy Index values. Regarding the JIF values, the comparison between journals with or without AOP showed statistically significant differences (P=0.001, Mann-Whitney U test). The Spearman’s correlation between the JIF and the median online-to-print publication delay was not statistically significant. As to the Immediacy Index, a significant Spearman’s correlation (rs=0.280, P=0.029) was found regarding the median online-to-print publication delays for journals published in 2014, although no statistically significant correlation was found for those published in 2013. Most journals examined (n=52 out of 61) published their articles in AOP. The analysis also showed different publisher practices: eight journals did not include the online posting dates in the full-text and nine journals published articles showing two different online posting dates--the date provided on the journal website and another provided by Elsevier’s Science Direct. These practices suggest the need for transparency and standardization of the AOP dates of scientific articles for calculating bibliometric indicators for journals.
RESUMEN
Este artículo analiza la distancia temporal entre la disponibilidad online de los artículos y su publicación en papel y cómo este lapso de tiempo afecta a los indicadores bibliométricos Factor de Impacto (JIF) e Índice de Inmediatez. Esta investigación examinó 18.526 artículos, la colección completa de artículos y revisiones publicadas por un conjunto de 61 revistas de Urología y Nefrología en 2013 y 2014. Los resultados hallados sugieren que la Publicación Electrónica Anticipada (AOP) acelera la citación de artículos, afectando a los valores de JIF e Índice de Inmediatez. Respecto a los valores de JIF, la comparación entre revistas con AOP y sin esta característica mostró diferencias estadísticamente significativas (P=0,001, U test Mann-Whitney). La correlación de Spearman entre JIF y la mediana del intervalo de tiempo publicación online-papel no resultó ser estadísticamente significativa. En cuanto al Índice de Inmediatez, se halló una correlación de Spearman significativa respecto a la mediana del intervalo de publicación online-papel (rs=0,280, P=0,029) para las revistas publicadas en 2014, pero sin correlación significativa para las publicadas en 2013. La mayoría de las revistas examinadas (n= 52 de 61) publicaron sus artículos en AOP. El análisis mostró también diferentes prácticas de los editores, ocho revistas no indicaron en el texto completo las fechas de publicación online, y nueve revistas publicaron los mismos artículos con dos fechas diferentes de publicación online, la fecha proporcionada por la web de la revista y la fecha suministrada por Science Direct de Elsevier. Estas prácticas sugieren la necesidad de transparencia y estandarización de las fechas de publicación online de artículos científicos para el cálculo de los indicadores bibliométricos de las revistas.
KEYWORDSAdvance online publicationimmediacy indexjournal impact factoronline posting datepublication delaysurology and nephrology journalsPALABRAS CLAVEPublicación electrónica anticipadaíndice de inmediatezfactor de impactofecha de publicación onlinetiempos de demora en la publicaciónrevistas de urología y nefrología1. INTRODUCTION
The current system for measuring the impact of scholarly journals is based on the quantitative analysis of citations received by scientific publications. Time is an important factor in the calculation of such indicators, and the most popular indicators take into consideration the time a journal has had to accumulate citations, measuring both the citations a journal has received and the time it has taken to accumulate them. However, online publishing is changing the nature of the publication date, with inevitable consequences for the calculating of bibliometric indicators and the ranking of journals.
The most influential indicator to evaluate journals is the Journal Impact Factor (JIF), devised by Garfield and Sher in the early 1960s (Garfield, 2006). A journal’s impact factor provides information on the relative importance of a journal in terms of influence, prestige and visibility within its subject category. The JIF is calculated from three elements, citations received, number of substantial items published and the measurement of a two-year window after publication. Despite its apparent accuracy, reproducibility and widespread use, the calculation of these variables as proxy for measuring a journal’s influence has been widely discussed (Bordons et al., 2002; Bornmann and Marx, 2016; Glänzel and Moed, 2002; Lozano et al., 2012). The citation window has been criticised for a number of reasons: the shortness of the two-year period for measuring disciplines with different citation latencies, it discriminates against papers whose citation patterns present delayed rises or delayed recognitions (Glänzel and Moed, 2002; Wang, 2013), and the time aggregation for online journal publication escapes from traditional metrics of print journal publication (Haustein et al., 2015; Stuart, 2015; Tort et al., 2012).
The Immediacy Index is another citation metric used for journal evaluation, designed to indicate how quickly articles in a journal are cited. This indicator is calculated as the average number of citations received by articles of a journal in the publication year divided by the number of articles published in that year (Thomson Reuters, 2012). In comparison to JIF, wich is based on citations received by articles published in the two years before the year in question, Immediacy Index is more restrictive in its calculation, considering only citations made in the actual publication year.
This work reports the findings of an investigation into the effect of online-to-print delays on the JIF and Immediacy Index of journals categorised as Urology and Nephrology in the Journal Citation Reports (JCR). The delay is calculated, from the initial date of online posting provided by the publishers in the full-text of the articles to the final official print date, indexed by Thomson Reuters’ Web of Science (WoS). Subsequently, because JIFs form the basis of JCR rankings, we compare the online-to-print publication delays across the four quartiles of journals in JCR. Furthermore, the correlation between online-to-print delays and the number of articles published by a journal per year is studied, as well as the differences in publication delays between journals published by commercial publishers and scientific societies.
2. BACKGROUND
The traditional publication of print journals usually involved long delays from a manuscript’s reception to publication. This was due to the bundling of articles into issues, as well as the backlog of manuscripts waiting for placement in a bound print volume with a limited space per year. In the late 1990s the proposal of many publishers to reduce these delays and give visibility to scientific output was to host electronic versions of their paper-based journals on their websites (Dong et al., 2006). Currently, most of major scientific publishers include posting dates in the full-text of peer-reviewed articles, once they are completely edited and formatted, and before the release print issue. For immediate online publication editors use services so-called: ’Advance online publication’ by Nature Group Publishing, ‘Advance access’ by Oxford Journals, ‘Articles in press’ by Elsevier, ‘Early view’ by John Wiley & Sons and ‘Online first’ by Springer.
AOP enables the dissemination of knowledge more quickly, reduces the publication print delays and increases the visibility and readership of articles. It might also have influence on bibliometric indicators by enlarging citation time before the print publication date, which is the official start of the citation window used by WoS for calculating the JIF and Immediacy Index.
The increasing online-to-print publication delays in recent years, and its influence on bibliometric indicators have been pointed out in a number of recent studies. Tort et al. (2012) showed that the delays between online availability of articles and print publication lead to earlier citations, and thus can artificially raise a journal’s impact factor, calculated by WoS on 2-year citation window. Chen et al. (2013) analysed the publication times in 51 Ophthalmology journals finding that journals with AOP had a higher JIF compared to those without this feature, although this difference was not statistically significant. Heneberg (2013) reported that online first publication severely affects the calculation of the JIF and Immediacy Index and disadvantages online-only and print-only journals when evaluating them according to these bibliometric indicators.
Others studies have considered the issue of publication delays, Alves-Silva et al. (2016) found that ‘online posting’, defined as the period spent between the submission of a manuscript and online publication, was significantly and negatively related to JIF in a set of Ecology journals. Amat (2008) defined ‘publication delay’ as the chronological distance between the stated date of reception of a manuscript by a given journal and its appearance in any print issue of that journal, and determined that the effect of online posting on the publisher’s site of the whole publication process reduced the publication delay by 29% in a group of Food Research journals. In contrast to the assumption that OA journals have shorter publication delays, Dong et al. (2006) showed that the ‘publication speed’ of journals, defined as the interval of time from the reception of the manuscripts to online publication, was faster for manuscripts published in AOP by Nature Group than those published in OA by Biomed Central. Although OA articles had a greater impact than articles that were not accessible freely online, they estimated that the impact of AOP was still uncertain.
3. RESEARCH QUESTIONS
There is a limited amount of research so far into online-to-print delays, carried out on different publication times, research practices and different fields, and thus there is a clear need for further studies in more areas. This study contributes to current understanding of the influence of online-to-print delays on the JIF and Immediacy Index, through an analysis of a set of 61 journals of Urology and Nephrology, indexed in JCR Thomson Reuters. More specifically, the research questions in this paper are:
What is the correlation between online-to-print delays with JIF and Immediacy Index in Urology and Nephrology journals?
Are there any differences between the distribution of journals in quartiles (JCR) and online-to-print delays?
Is there a relationship between online-to-print publication delays and the number of articles published per year by a journal?
Is there any difference in publication delays between journals published by large commercial publishers and scientific societies?
4. METHODOLOGY
To gather research data (see online additional material 2013 [https://figshare.com/s/032dc17ddd0441a4e8bb] and 2014 [https://figshare.com/s/b3b163a87fed2f5a9132] for details) a set of 61 journals out of 77 indexed by JCR (2015), whose subject category was Urology and Nephrology were analysed. In order to construct the dataset of journals (see Appendix Tabla I, and online additional material 2013 [https://figshare.com/s/032dc17ddd0441a4e8bb] and 2014 [https://figshare.com/s/b3b163a87fed2f5a9132], the following methodological criteria were established:
4.1. Criteria for inclusion
All journals indexed by JCR in 2015 under the category Urology and Nephrology.
All original articles and reviews, whose print publication date was included between January 2013 and December 2014.
4.2. Criteria for exclusion
Electronic journals without print publication version.
Journals whose articles do not provide information about the online posting dates, JIF or Immediacy Index. Furthermore, those journals that could not be accessed full-text and journals with a low number of articles published were also excluded.
Book reviews, critiques, comments, and letters from or to editors.
Journal supplements were not included (following Björk and Solomon, 2013; Chen et al., 2013), as well as monographic series.
All papers were examined in electronic form. When an article was posted with different online dates, the date provided on the journal’s website and the date provided by Elsevier’s Science Direct, the date selected was the first date in which the article was available online. For each article the online posting date, provided by publishers in the full-text online, and the official print date provided by WoS, were recorded (see online additional material 2013 [https://figshare.com/s/032dc17ddd0441a4e8bb] and 2014 [https://figshare.com/s/b3b163a87fed2f5a9132]). When a journal did not specify information about the online posting date it was necessary to contact publishers. This only applied to two journals (Archivos Españoles de Urología and Nephrology Nursing Journal) and they confirmed that electronic and print publication dates were the same (see Appendix, Tabla I).
4.3. Journals and papers examined
A total of 18,526 articles and reviews published in 2013 and 2014 were examined. This number of articles and reviews corresponded faithfully to 61 out of 77 journals used by JCR to calculate the JIF score of 2015 and Immediacy Index of 2013 and 2014 of the subject category Urology and Nephrology Journals. The analysis excluded 16 journals, due to criteria detailed in the list of journals (see Appendix, Tabla II). This fact slightly altered the number of journals included in the list of 2013 (59 journals) (see Appendix, Tabla I).
Since only a minority of articles provided the day and month of print publication, the journal issue date was converted to days, based on average unit length. In this way, the date of publication was estimated: 1) as the 15th day of each month for monthly journals, and 2) the last day of the first month for bimonthly journals. In cases of irregular journals (i.e. with 8 or 10 issues per year) the date of publication was distributed among the yearly period (following Amat, 2008). When an article was published in print and online during the same period of time, the number of days was not considered (see online additional material 2013 [https://figshare.com/s/032dc17ddd0441a4e8bb] and 2014 [https://figshare.com/s/b3b163a87fed2f5a9132]).
4.4. Statistical analyses
Median and Interquartile Ranges (IQRs, 25% - 75%) were calculated with the variable online-to-print publication dates, obtained from the time distance between AOP and print publication for all articles and reviews of each journal. All data were tested for normality distribution and because the measures of the variable online-to-print publication dates were not normally distributed non-parametric tests were used.
The Spearman correlation test was used to examine the correlation between the median online-to-print publication delays and JIF, Immediacy Index and the size of journals, based on the number of articles published per year. The Mann-Whitney U test was used to compare the JIFs of journals with AOP or without AOP, and also to determine whether there were differences between journals published by commercial editors and scientific societies regarding the variable online-to-print publication delays. The Kruskal-Wallis test was used for intra-group comparison of distribution of journals in quartiles with the variable online-to-print publication.
In our tests the statistical significance was set at 5% and two-sided. The analyses were performed using the SPSS Statistics software (IBM), version 22.
5. RESULTS
The analysis revealed that a high proportion of Urology and Nephrology journals published their articles ahead of print 85.2% (n=52 out of 61), with a great dispersion in the lapse of time online-to-print publication.
The median online-to-print publications was 86 days (IQR, 48-171) for journals published in 2013, and 95 days (IQR, 36-171) for journals published in 2014. The chronological distance between online-to-print publication dates showed a wide variability. In 2013, the range varied from journals with no-delays 16.9% (n=10) to journals whose median was more than 300 days 5.1% (n=3), with the most frequent interval being 43-84 days 23.7% (n=14) journals (see Fig. 1).
Frequency distribution of the median of days online-to-print publication delays in 2013
The chronological distance between online-to-print publications in 2014 varied from journals with no delays 14.8% (n=9) to 2 (3.2%) journals whose median was more than 300 days, with the most frequent interval being 169-210 days 18% (n=11) (see Fig. 2).
Frequency distribution of the median of days online-to-print publication delays in 2014
Regarding the study of the JIF (2015), the comparison between the JIF of journals with AOP and journals without AOP showed statistically significant differences (Mann-Whitney z=-3.255, P=0.001). The mean of the JIF for journals with AOP was X=2.953, and for journals without AOP X=1.003.
With respect to the correlation between JIF and the median days online-to-print publication delays, whose values for 2013 and 2014 were grouped, there was no statistically significant correlation (rs=0.158, P=0.233).
In regard to Immediacy Index, the correlation between median days online-to-print delays and Immediacy Index was statistically significant (rs=0.280, P=0.029) for journals published in 2014 (n=61) (see Fig. 3). However, for journals published in 2013 (n=56 with Immediacy Index), there was no significant Spearman correlation (rs=0.117, P=0.391) (see Fig. 4).
Online-to-print publication delays (median of days) and Immediacy Index (2013)
Online-to-print publication delays (median of days) and Immediacy Index (2014)
In answer to the second research question, about whether there were differences of rank position of journals in quartiles (JCR) regarding online-to-print delays, the Kruskal-Wallis test did not identify statistically significant differences in 2013 (X2=1.814, DF=3, P=0.612) and 2014 (X2=2.360, DF=3, P=0.501). However, it is important to observe that journals without AOP were classified in the lower quartiles: 5 titles (Q4) and 4 titles (Q3), whereas 16 journals with 267 days of AOP, on average, were classified in the first quartile (Q1).
Regarding the relationship between online-to-print publication delays and the size of journals, in terms of the number of articles published per year by a journal, a significant Spearman correlation was observed, both for journals published in 2013 (rs= 0.335, P=0.010) and 2014 (rs=0.375, P=0.003).
Another issue emerging from this research was whether there were differences regarding publication delays between journals published by commercial publishers (53/61) and by scientific societies (8/61) in relation to the variable of online-to-print publication delays. The Mann-Whitney U test identified statistically significant differences for journals published by commercial publishers and scientific societies in 2013 (z=-2.109, P=0.035, median online-to-print delays of publishers 101 days [IQR, 51-173] vs societies 31 days [IQR, 0-81]). For journals published in 2014 (z= -2.119, P=0.034, median online-to-print delays of publishers 97 days [IQR, 45-171] vs societies 34 days [IQR, 0-74]).
6. DISCUSSION
The analysis revealed that most publishers of Urology and Nephrology journals published articles ahead of print (n= 52 out of 61, 85.2%), with a great dispersion in the lapse time online-to-print publications (see Fig. 1 and Fig. 2). The period spent between online to print publication was almost 3 months (86 days), on average, for journals published in 2013 and more than 3 months (95 days), on average, for journals published in 2014.
Although most publishers included the online posting date in the full-text of articles, some exceptions were observed: 8 journals did not include the date of online posting, and 9 journals published the same articles with 2 different online posting dates, the date provided on the journal’s website and the date provided by Elsevier’ Science Direct. These practices reflected a lack of standardization in terms of how publishers report online dates.
Despite the role of time on the accumulation of citations, the analysis did not show significant correlation between JIF and online-to-print publication delays. This is probably due to the value of the JIF being determined by different parameters such as the journal size, the subject area, prestige, approach, availability or influential authors, and cannot objectively be reduced to a few variables (Alves-Silva et al., 2016). Althought the extent to which online-to-print delays can affect a journal's JIF is difficult to determine, journals without AOP would have less opportunity to receive citations and consequently it may influence their JIF.
In order to contrast this result to other studies (Chen et al., 2013; Heneberg, 2013; Tort et al. 2012), these authors did not register statistically significant correllation between journals with AOP and JIF, although they mentioned the influence of AOP in terms of higher impact factors in comparison to journals without this feature, number of citations utilized for the calculation of JIF, and potential distortions concerning the evaluation of journals. Moed (2007) provided evidence that the inclusion of a paper in arXiv accelerated its citation probably due to this repository making papers available earlier accelerating communication, the process of reading and processing of information. This evidence was observed also by Craig et al. (2007) [mentioned by Heneberg (2013)] “preprint availability causes the citation counting process to start earlier although this earlier citation did not affect the final magnitude of citations accrued to a journal article”. In this regard, Vanclay (2012) pointed out that “[AOP] offers little advantage for a journal that peaks early (e.g. Nature), but may substantially alter the JIF of slow-to-peak journals, such as Ecology”.
Regarding the Immediacy Index, indicator that provides a useful perspective of the cutting-edge fields of research, the analysis showed a statistically significant correlation with online-to-print delays for the journals of 2014 (n=61), although without significant differences for the journals of 2013 (n=57 journals with Immediacy Index). Heneberg (2013), using the database Scopus, studied the number of citations received by articles with AOP (“in press” [Elsevier]) within the year when the article was assigned to a journal issue. According to Heneberg (2013), AOP (“in press” [Elsevier]) severely affects the calculation of Immediacy Index, although without significant differences.
As Immediacy Index is calculated on the number of citations that an article receives in the same year it is published, an article published early or with AOP has a better chance of being cited than one published later in the year. In fact, as Thomson Reuters (2012) states “Many publications that publish infrequently or late in the year have low Immediacy Indexes”. On this point, the potential of technical conditions, such as frequency of publication and publication delays impact the Immediacy Index and have long been recognized and have limited the importance of this Index (Glänzel and Moed, 2002).
In answer to the second research question, about wether there were differences in the distribution of journals in quartiles (JCR) in relation to online-to-print delays, the Kruskal-Wallis test did not show any statistically significant differences, although it is observed that a higher median of AOP corresponds with the higher quartiles, and journals without AOP are located in the lower quartiles. However, this statement would need further analysis.
Regarding the correlation between online-to-print delays and the number of articles published per year by a journal, the results indicated a statistically significant association. The steady growth of the number of articles, the large backlog of articles awaiting publication in bound print volumes and the need to achieve a rapid and efficient scientific communication are in the origin of Advance Online Publication. In this context, the data reflected statistically significant differences between journals published by major commercial editors (Elsevier, John Wiley, Nature Publishing Group, Oxford, Springer, Taylor & Francis) with a large number of articles per year and long delays versus journals published by scientific societies, with a smaller number of articles published and shorter delays.
It is important to note that commercial editors published 87% of Urology and Nephrology journals analysed. As Larivière et al. (2015) pointed out “The proportion of scientific output published in journals under their ownership [large commercial publishing houses] has risen steadily over the past 40 years, and even more so since the advent of the digital era”.
7. CONCLUSION
The practice of making papers available online ahead of print publication has facilitated the rapid communication of scientific research, but at the same time it has resulted in bibliometric consequences, such as enlarging the citation time of JIF and Immediacy Index.
The research revealed that the longest periods of online-to-print publication are mainly produced by large commercial editors that publish journals with a large number of articles per year. This suggests that, to some extent, delays are justified or at least are convenient. However, it would be necessary to study the effects long terms that this gradual increase in the number of days of AOP is having on the number of citations and their contribution to the JIF values, on the differences created between journals with or without AOP, and possible consequences on other bibliometric indicators.
Another feature of some articles with AOP was the lack of standardization of online posting dates. The publishers reported journals with different posting dates for the same article and journals without any mention of the date of online posting. Thus, there is a need for transparency and standardization of posting dates in order to assure the calculation of bibliometric indicators for the evaluation and comparability of journals (Haustein et al., 2015).
Until such standards emerge it is important to have a better understanding of how AOP affects different fields and metrics, and whether there are areas where the online-to-print publication delay is sufficient to not only have an impact on lesser-used indicators such as the Immediacy Index but also the widely-adopted JIF.
8. REFERENCESAlves-SilvaE.PortoA.C.F.FirminoC.SilvaH.V.BeckerI.ResendeL.BorgesL.PfefferL.SilvanoM.GaldianoM.S.SilvestriniR.MouraR.2016Are the impact factor and other variables related to publishing time in ecology journals?108314451453https://doi.org/10.1007/s11192-016-2040-0AmatC.B.2008Editorial and publication delay of papers submitted to 14 selected Food Research journals. Influence of online posting743379389https://doi.org/10.1007/s11192-007-1823-8BjörkB.C.SolomonD.2013The publishing delay in scholarly peer-reviewed journals74914923https://doi.org/10.1016/j.joi.2013.09.001BordonsM.FernándezM.T.GómezI.2002Advantages and limitations in the use of impact factor measures for the assessment of research performance in a peripheral country532195206https://doi.org/10.1023/A:1014800407876BornmannL.MarxW.2016The journal Impact Factor and alternative metrics17810941097https://doi.org/10.15252/embr.201642823ChenH.ChenH.C.JhanjiV.2013Publication times, impact factors, and advance online publication in Ophthalmology journals120816971701https://doi.org/10.1016/j.ophtha.2013.01.044CraigI.D.PlumeA.M.McVeighM.E.PringleJ.AminM.2007Do Open Access Articles Have Greater Citation Impact? A critical review of the literature13239248https://doi.org/10.1016/j.joi.2007.04.001DongP.LohM.MondryA.2006Publication lag in biomedical journals varies due to the periodical’s publishing models692271286https://doi.org/10.1007/s11192-006-0148-3GarfieldE.2006The history and meaning of the journal impact factor29519093https://doi.org/10.1001/jama.295.1.90GlänzelW.MoedH.F.2002Journal impact measures in bibliometric research5320171193https://doi.org/10.1023/A:1014848323806HausteinS.BowmanT.D.CostasR.2015When is an article actually published? An analysis of online availability, publication, and indexation dates15th International Conference on scientometrics and informetrics (ISSI)29 June –3 September 2015Istanbul, Turkey11701179https://arxiv.org/abs/1505.00796HenebergP.2013Effects of print publication lag in dual format journals on scientometric indicators84e59877https://doi.org/10.1371/journal.pone.0059877LarivièreV.HausteinS.MongeonP.2015The oligopoly of academic publishers in the digital era106e0127502https://doi.org/10.1371/journal.pone.0127502LozanoG. A.LarivièreV.GingrasY.2012The weakening relationship between the impact factor and papers' citations in the digital age631121402145https://doi.org/10.1002/asi.22731MoedH.F.2007The effect of “open access” on citation impact: An analysis of ArXiv's condensed matter section581320472054https://doi.org/10.1002/asi.20663StuartD.2015Metrics for an increasingly complicated information ecosystem396848854https://doi.org/10.1108/OIR-06-2015-0174Thomson Reuters2012Available at: http://admin-apps.webofknowledge.com/JCR/help/h_jrnlinfo.htm(Accessed 4 September 2016)TortA.B.L.TarginoZ.H.AmaralO.B.2012Rising publication delays inflate journal impact factors712e53374https://doi.org/10.1371/journal.pone.0053374VanclayJ. K.2012Impact factor: outdated artefact or stepping-stone to journal certification?922211238https://doi.org/10.1007/s11192-011-0561-0WangJ.2013Citation time window choice for research impact evaluation943851872https://doi.org/10.1007/s11192-012-0775-9APPENDIX