Model for estimating Academic Ranking of World Universities (Shanghai Ranking) scores
DOI:
https://doi.org/10.3989/redc.2018.2.1462Keywords:
international university rankings, research evaluation, Shanghai rankingAbstract
The interest in global university rankings has grown significantly in the last 10 years. The use of just a handful of indicators, the ease of interpretation of the information they contain and the furtherance of inter-university competition and comparability are some of the factors that have popularised their implementation. However, at the same time their critics have identified some conceptual, technical and methodological issues. This paper addresses three such issues that have prompted intense methodological debate around university rankings: replicability of results, relevance of indicators and data retrieval. It also proposes a tool for estimating the scores for the two indicators of the greatest interest for most universities (Papers published in Nature or Science and Papers listed in the WoS). It reports on an alternative method developed to calculate any university’s score in the two most significant Shanghai ranking indicators. One of the foremost features of the proposed method is that the inputs needed are readily available to policymakers, academic authorities, students and other stakeholders and can be applied directly. Furthermore, with this model, scores can also be estimated for universities not listed among the first 500 in the Shanghai ranking.
Downloads
References
Andersson, B.; Mayer, T. (2017). University Rankings: Was Academic Life Perfect before Rankings? In: Downing, K. and Ganotice, F. Jr. (eds.), World University Rankings and the Future of Higher Education, pp. 70-86. Hershey, PA: IGI Global. https://doi.org/10.4018/978-1-5225-0819-9.ch004
ARWU. (2015). Ranking Methodology of Academic Ranking of World Universities – 2015. [Online] Available at: http://www.shanghairanking.com/ARWU-Methodology-2015.html [Accessed 1 Nov. 2015].
Billaut, J. C.; Bouyssou, D.; Vincke, P. (2010). Should you believe in the Shanghai ranking? Scientometrics, 84 (1), 237-263. https://doi.org/10.1007/s11192-009-0115-x
Bornmann, L.; Bauer, J. (2015). Which of the world's institutions employ the most highly cited researchers? An analysis of the data from highlycited.com. Journal of the Association for Information Science and Technology, 66 (10), 2146-2148. https://doi.org/10.1002/asi.23396
Dobrota, M.; Dobrota, M. (2016). ARWU ranking uncertainty and sensitivity: What if the award factor was excluded? Journal of the Association for Information Science and Technology, 67 (2), 480-482. https://doi.org/10.1002/asi.23527
Docampo, D. (2013). Reproducibility of the Shanghai academic ranking of world universities results. Scientometrics, 94 (2), 567-587. https://doi.org/10.1007/s11192-012-0801-y
Docampo, D.; Cram, L. (2015). On the effects of institutional size in university classifications: the case of the Shanghai ranking. Scientometrics, 102 (2), 1325-1346. https://doi.org/10.1007/s11192-014-1488-z
Docampo, D.; Egret, D.; Cram, L. (2015). The effect of university mergers on the Shanghai ranking. Scientometrics, 104 (1), 175-191. https://doi.org/10.1007/s11192-015-1587-5
Fiala, D. (2012). Bibliometric analysis of CiteSeer data for countries. Information Processing & Management, 48 (2), 242-253. https://doi.org/10.1016/j.ipm.2011.10.001
Florian, R. V. (2007). Irreproducibility of the results of the Shanghai academic ranking of world universities. Scientometrics, 72 (1), 25-32. https://doi.org/10.1007/s11192-007-1712-1
Fowles, J.; Frederickson, H. G.; Koppell, J. G. S. (2016). University Rankings: Evidence and a Conceptual Framework. Public Administration Review, 76 (5), 790- 803. https://doi.org/10.1111/puar.12610
Garrett, R.R. (2015). rgr: Applied Geochemistry EDA. R package version 1.1.11. http://CRAN.R-project.org/ package=rgr
Gonzalez-Riano, M. G.; Repiso, R.; Lopez-Cozar, E. D. (2014). The impact of university rankings in the Spanish press. Revista Espa-ola de Documentación Científica, 37 (3): e055.
Hazelkorn, E. (2014). Reflections on a Decade of Global Rankings: what we've learned and outstanding issues. European Journal of Education, 49 (1), 12-28. https://doi.org/10.1111/ejed.12059
Jovanovic, M.; Jeremic, V.; Savic, G.; Bulajic, M.; Martic, M. (2012). How does the normalization of data affect the ARWU ranking? Scientometrics, 93 (2), 319-327. https://doi.org/10.1007/s11192-012-0674-0
Lascurain, M.L.; Pandiella-Dominique, A.; Sanz-Casado, E. (2015). The Weight of Databases in World University Rankings: Spanish Universities in ARWU. In: Luque Martínez, T. (ed.), The University of Granada in light of its V Centenary. "Reflections on the Future of the University". Granada: Universidad de Granada.
Liu, N. C.; Cheng, Y.; Liu, L. (2005). Academic ranking of world universities using scientometrics - A comment to the "Fatal Attraction". Scientometrics, 64 (1), 101-109. https://doi.org/10.1007/s11192-005-0241-z
Marginson, S.; Van der Wende, M. (2007). Globalisation and Higher Education. OECD Education Working Papers, No. 8. OECD Publishing (NJ1). https://doi.org/10.1787/173831738240
Moed, H. F. (2002). The impact-factors debate: the ISI's uses and limits. Nature, 415 (6873), 731-732. https://doi.org/10.1038/415731a PMid:11845176
Olcay, G. A.; Bulu, M. (2016). Is measuring the knowledge creation of universities possible?: A review of university rankings. Technological Forecasting and Social Change, 123, 153-160. https://doi.org/10.1016/j.techfore.2016.03.029
O'Leary, J. (2017). The Origins of International Rankings. In: Downing, K. and Ganotice, F. Jr. (eds.), World University Rankings and the Future of Higher Education, pp. 61-69. Hershey, PA: IGI Global. https://doi.org/10.4018/978-1-5225-0819-9.ch003
Pusser, B.; Marginson, S. (2013). University Rankings in Critical Perspective. The Journal of Higher Education 84 (4), 544-568. https://doi.org/10.1353/jhe.2013.0022
Rauhvargers, A. (2011). Global University Rankings and Their Impact (vol. I). Brussels, Belgium: EUA.
Rauhvargers, A. (2013). Global University Rankings and Their Impact II (vol. II). Brussels, Belgium: EUA.
Robinson-Garcia, N.; Jiménez-Contreras, E. (2017). Analyzing the Disciplinary Focus of Universities: Can Rankings Be a One-Size-Fits-All? In: Downing, K. and Ganotice, F. Jr. (eds.), World University Rankings and the Future of Higher Education, pp. 161-185. Hershey, PA: IGI Global. https://doi.org/10.4018/978-1-5225-0819-9.ch009
Saisana, M.; d'Hombres, B.; Saltelli, A. (2011). Rickety numbers: Volatility of university rankings and policy implications. Research Policy, 40 (1), 165-177. https://doi.org/10.1016/j.respol.2010.09.003
Sanz-Casado, E (Ed.) (2015). Guía de buenas prácticas para la participación de las universidades espa-olas en los rankings internacionales. Madrid: Ministerio de Educación, Cultura y Deporte.
Safón, V. (2013). What do global university rankings really measure? The search for the X factor and the X entity. Scientometrics, 97 (2), 223-244. https://doi.org/10.1007/s11192-013-0986-8
Stewart, T. J. (2014). Value measurement theory and league tables. South African Journal of Science, 110 (3-4), 5-6. https://doi.org/10.1590/sajs.2014/a0057
Turner, D. A. (2017). World Class Universities and the Rest. In: Downing, K. and Ganotice, F. Jr. (eds.), World University Rankings and the Future of Higher Education (pp. 109-128). Hershey, PA: IGI Global. https://doi.org/10.4018/978-1-5225-0819-9.ch006
van Raan, A. F. J. (2005a). Academic ranking of world universities using scientometrics - A comment to the "Fatal Attraction" - Reply. Scientometrics, 64 (1), 111- 112. https://doi.org/10.1007/s11192-005-0242-y
van Raan, A. F. J. (2005b). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62 (1), 133- 143. https://doi.org/10.1007/s11192-005-0008-6
van Raan, A. F. J., van Leeuwen T.N.; Visser M.S. (2011). Severe language effect in university rankings: particularly Germany and France are wronged in citation-based rankings. Scientometrics, 88 (2): 495- 498. https://doi.org/10.1007/s11192-011-0382-1 PMid:21836762 PMCid:PMC3125505
Zornic, N.; Markovic, A.; Jeremic, V. (2014). How the top 500 ARWU can provide a misleading rank. Journal of the Association for Information Science and Technology, 65 (6), 1303-1304. https://doi.org/10.1002/asi.23207
Published
How to Cite
Issue
Section
License
Copyright (c) 2018 Consejo Superior de Investigaciones Científicas (CSIC)
This work is licensed under a Creative Commons Attribution 4.0 International License.
© CSIC. Manuscripts published in both the print and online versions of this journal are the property of the Consejo Superior de Investigaciones Científicas, and quoting this source is a requirement for any partial or full reproduction.
All contents of this electronic edition, except where otherwise noted, are distributed under a Creative Commons Attribution 4.0 International (CC BY 4.0) licence. You may read the basic information and the legal text of the licence. The indication of the CC BY 4.0 licence must be expressly stated in this way when necessary.
Self-archiving in repositories, personal webpages or similar, of any version other than the final version of the work produced by the publisher, is not allowed.
