Percentage of highly cited articles: A comparable measure of journals’ impact between disciplines
DOI:
https://doi.org/10.3989/redc.2015.3.1230Keywords:
Journal evaluation, bibliometric indicator, citation analysis, impact factor, h-index, citation percentileAbstract
Currently the two most used bibliometric indicators for evaluating scholarly journals are the impact factor and the h-index. However, both indicators vary heavily depending on the scientific field, restricting comparability. In addition, the impact factor is not robust in dealing with articles with a large number of citations, while the h-index is dependent on the size of the journal. These limitations, when comparing journals of different sizes and fields, make it necessary to look to other journal impact factors that can be comparable between scientific fields, that are independent of the journal size, and that are also robust in the presence of items with a high number of citations. An alternative index is based on the percentage of highly cited articles in a journal. This paper empirically compares such an index with the impact factor and the h-index by using different time windows and levels of citation that can determine when a document can be considered as highly cited compared to others of the same year and discipline. The main outcome of this comparison suggests that the best index for obtaining data distributions that are comparable between scientific fields is by taking the 20% citation percentile over a three-year time frame for considering citations.
Downloads
References
Althouse, B. M.; West, J. D.; Bergstrom, C. T.; Bergstrom, T. (2009). Differences in impact factor across fields and over time. Journal of the American Society for Information Science and Technology, vol. 60 (1), 27–34. http://dx.doi.org/10.1002/asi.20936
>Bensman, S. J. (2007). Garfield and the impact factor. Annual Review of Information Science and Technology, vol. 41 (1), 93–155. http://dx.doi.org/10.1002/aris.2007.1440410110
Bergstrom, C. (2007). Eigenfactor: Measuring the value and prestige of scholarly journals. College and Research Libraries News, vol. 68 (5), 314.
Bordons, M.; Barrigón, S. (1992) Bibliometric analysis of publications of Spanish pharmacologists in the SCI (1984-89). Part II. Contribution to subfields other than «Pharmacology & Pharmacy» (ISI). Scientometrics, vol. 25 (3), 425–446. http://dx.doi.org/10.1007/bf02026479
Bornmann, L.; Daniel, H. D. (2008). What do citation counts measure? A review of studies on citing behavior. Journal of Documentation, vol. 64 (1), 45–80. http://dx.doi.org/10.1108/00220410810844150
Costas, R.; Bordons, M. (2007a). Una visión crítica del índice h: algunas consideraciones derivadas de su aplicación práctica. El profesional de la información, vol. 16 (5), 427–432. http://dx.doi.org/10.3145/epi.2007.sep.04
Costas, R.; Bordons, M. (2007b) The h-index: advantages, limitations and its relation with other bibliometric indicators at the micro-level. Journal of Informetrics, vol. 1 (3), 193–203. http://dx.doi.org/10.1016/j.joi.2007.02.001
Dorta-González, P.; Dorta-González, M. I. (2011). Central indexes to the citation distribution: A complement to the h-index. Scientometrics, vol. 88 (3), 729–745. http://dx.doi.org/10.1007/s11192-011-0453-3
Dorta-González, P.; Dorta-González, M. I. (2013a). Comparing journals from different fields of science and social science through a JCR subject categories normalized impact factor. Scientometrics, vol. 95 (2), 645–672. http://dx.doi.org/10.1007/s11192-012-0929-9
Dorta-González, P.; Dorta-González, M. I. (2013b). Hábitos de publicación y citación según campos científicos: Principales diferencias a partir de las revistas JCR. Revista Espa-ola de Documentación Científica, vol. 36 (4): en012. http://dx.doi.org/10.3989/redc.2013.4.1003
Dorta-González, P.; Dorta-González, M. I. (2013c). Impact maturity times and citation time windows: The 2-year maximum journal impact factor. Journal of Informetrics, vol. 7 (3), 593–602. http://dx.doi.org/10.1016/j.joi.2013.03.005
Dorta-González, P.; Dorta-González, M. I.; Santos-Pe-ate, D. R.; Suárez-Vega, R. (2014). Journal topic citation potential and between-field comparisons: The topic normalized impact factor. Journal of Informetrics, vol. 8 (2), 406–418. http://dx.doi.org/10.1016/j.joi.2014.01.013
Egghe, L. (2013). Theoretical justification of the central area indices and the central interval indices. Scientometrics, vol. 95 (1), 25–34. http://dx.doi.org/10.1007/s11192-012-0803-9
Egghe, L.; Rousseau, R. (2002). A general framework for relative impact indicators. Canadian Journal of Information and Library Science, vol. 27 (1), 29–48.
Garfield, E. (1972). Citation analysis as a tool in journal evaluation. Science, vol. 178 (4060), 471–479. http://dx.doi.org/10.1126/science.178.4060.471 PMid:5079701
González-Pereira, B.; Guerrero-Bote, V. P.; Moya-Anegón, F. (2009). The SJR indicator: A new indicator of journals' scientific prestige. Journal of Informetrics, vol. 4 (3), 379–391. http://dx.doi.org/10.1016/j.joi.2010.03.002
Hirsch, J. E. (2005). An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences, vol. 102 (46), 16569–16572. http://dx.doi.org/10.1073/pnas.0507655102 PMid:16275915 PMCid:PMC1283832
Leydesdorff, L. (2012). Alternatives to the journal impact factor: I3 and the top-10% (or top-25%?) of the most-highly cited papers. Scientometrics, vol. 92 (2), 355–365. http://dx.doi.org/10.1007/s11192-012-0660-6 PMid:22844165 PMCid:PMC3399071
Leydesdorff, L.; Bornmann (2011). How fractional counting of citations affects the Impact Factor: Normalization in terms of differences in citation potentials among fields of science. Journal of the American Society for Information Science and Technology, vol. 62 (2), 217–229. http://dx.doi.org/10.1002/asi.21450
Moed, H. F. (2010). Measuring contextual citation impact of scientific journals. Journal of Informetrics, vol. 4 (3), 265–277. http://dx.doi.org/10.1016/j.joi.2010.01.002
Moed, H. F.; Colledge, L.; Reedijk, J.; Moya-Anegon, F.; Guerrero-Bote, V.; Plume, A.; Amin, M. (2012). Citation-based metrics are appropriate tools in journal assessment provided that they are accurate and used in an informed way. Scientometrics, vol. 92 (2), 367–376. http://dx.doi.org/10.1007/s11192-012-0679-8
Van Raan, A. F. J.; Van Leeuwen, T. N.; Visser, M. S.; Van Eck, N. J.; Waltman, L. (2010). Rivals for the crown: Reply to Opthof and Leydesdorff. Journal of Informetrics, vol. 4 (3), 431–435. http://dx.doi.org/10.1016/j.joi.2010.03.008
Wagner, C.; Roessner, J. D.; Bobb, K.; Klein, J.; Boyack, K.; Keyton, J.; Rafols, I.; Börner, K. (2011). Approaches to understanding and measuring interdisciplinary scientific research (IDR): A review of the literature. Journal of Informetrics, vol. 5 (1), 14–26. http://dx.doi.org/10.1016/j.joi.2010.06.004
Waltman, L.; Van Eck, N. J. (2013). Source normalized indicators of citation impact: an overview of different approaches and an empirical comparison. Scientometrics, vol. 96 (3), 699–716. http://dx.doi.org/10.1007/s11192-012-0913-4
Zitt, M.; Small, H. (2008). Modifying the journal impact factor by fractional citation weighting: The audience factor. Journal of the American Society for Information Science and Technology, vol. 59 (11), 1856–1860. http://dx.doi.org/10.1002/asi.20880
Published
How to Cite
Issue
Section
License
Copyright (c) 2015 Consejo Superior de Investigaciones Científicas (CSIC)
This work is licensed under a Creative Commons Attribution 4.0 International License.
© CSIC. Manuscripts published in both the print and online versions of this journal are the property of the Consejo Superior de Investigaciones Científicas, and quoting this source is a requirement for any partial or full reproduction.
All contents of this electronic edition, except where otherwise noted, are distributed under a Creative Commons Attribution 4.0 International (CC BY 4.0) licence. You may read the basic information and the legal text of the licence. The indication of the CC BY 4.0 licence must be expressly stated in this way when necessary.
Self-archiving in repositories, personal webpages or similar, of any version other than the final version of the work produced by the publisher, is not allowed.
