Scientific output in Chile: limitations in the use of performance indicators to evaluate public universities
DOI:
https://doi.org/10.3989/redc.2018.1.1447Keywords:
Public universities, higher education finance, productivity indicatorsAbstract
In 2015, the 25 universities that are members of the association of public universities (CRUCH) received $15.3 million as a reward for performance. The money that each university received depended on its position in the ranking of performance indicators. The performance indicator with the highest weight is the index of productivity (35%), which represents the ratio between the number of indexed publications divided by the number of academics at each university. This index does not distinguish publications by areas of knowledge, by quality or by intellectual author. Because of this, the index has been criticized as an unfair way to rank universities. To study this, we analyze10.377 papers indexed in the Web of Science by CRUCH universities in 2015. Our objective is to study the unintended consequences of the use of indexed publications as an indicator of productivity and estimate its impact on finance equity among universities and topics.
Downloads
References
Arocena, R.; Sutz, J. (2001). Changing knowledge production and Latin American universities, Research Policy, 30 (8), 1221–1234. https://doi.org/10.1016/S0048-7333(00)00143-8
Bernasconi, A.; Rojas, R. C. (2004). Informe sobre la educación superior en Chile, 1980-2003. Santiago de Chile: Editorial Universitaria, 205 p.
Buela-Casal, G.; Gutiérrez-Martínez, O.; Bermúdez- Sánchez, M. P.; Vadillo-Mu-oz, O. (2007). Comparative study of international academic rankings of universities. Scientometrics, 71 (3), 349–365. https://doi.org/10.1007/s11192-007-1653-8
Burgess, S.; Ratto, M. (2003). The Role of Incentives in the Public Sector: Issues and Evidence, Oxford Review Economic Policy, 19 (2), 285-300. https://doi.org/10.1093/oxrep/19.2.285
Campbell, D. (1979). Assessing the impact of planned social change. Evaluation and Program Planning, 2 (1), 67–90. https://doi.org/10.1016/0149-7189(79)90048-X
Conicyt. (2016). Publicaciones consideradas en AFD (Universidades CRUCH) WOS y SCIELO 200-2015. http://www.conicyt.cl/blog/2015/01/articulos-acusados-para-afd-2000-2015/ [Fecha de Consulta 23 de octubre de 2016].
Contraloría General de la República. (2015). Financiamiento Fiscal a la Educación Superior. Santiago de Chile: Gobierno de Chile, 93 p.
Davies, A.; Thomas, R. (2002). Managerialism and accountability in higher education: The gendered nature of restructuring and the costs to academic service. Critical Perspectives on Accounting, 13 (2), 179–193. https://doi.org/10.1006/cpac.2001.0497
Dixit, A. (1997). Power of Incentives in Private versus Public Organizations. The American Economic Review, 87 (2), 378–382.
Dixit, A. (2002). Incentives and Organizations in the Public Sector An Interpretative. Journal of Human Resources, 37 (4), 696–727. https://doi.org/10.2307/3069614
Edwards, S. (1995). Crisis and reform in Latin America: From despair to hope. New York: Oxford University Press, 380 p.
Gauri, V. (1998). School Choice in Chile: Two Decades of Educational Reform. Pittsburgh: University of Pittsburgh Press, 147 p.
González, L.; Espinoza, Ó. (2008). Calidad en la educación superior: concepto y modelos. Calidad en la Educación, (28), 244–276.
Haertel, E.; Herman, J. (2005). A Historical Perspective on Validity Arguments for Accountability Testing. Los Angeles, CA: National Center for Research on Evaluation, 38 p. https://doi.org/10.1037/e645192011-001
Harvey, L.; Newton, J. (2004). Transforming quality evaluation. Quality in Higher Education, 10 (2), 149– 165. https://doi.org/10.1080/1353832042000230635
Hoecht, A. (2006). Quality assurance in UK higher education: Issues of trust, control, professional autonomy and accountability. Higher Education, 51 (4), 541–563. https://doi.org/10.1007/s10734-004-2533-2
Hood, C. (1995). The "New Public Management" in the 1980s: variations on a theme. Accounting, Organizations and Society, 20 (2-3), 93–109. https://doi.org/10.1016/0361-3682(93)E0001-W
Jacob, B. (2005). Accountability, incentives and behavior: the impact of high-stakes testing in the Chicago Public Schools. Journal of Public Economics, 89 (5–6), 761– 796. https://doi.org/10.1016/j.jpubeco.2004.08.004
Kai, J. (2009). A Critical Analysis of Accountability in Higher Education. Chinese Education & Society, 42 (2), 39–51. https://doi.org/10.2753/CED1061-1932420204
Manríquez, P., Mendoza, D.; Ramírez, K. (2015). Relación entre el aporte fiscal directo, la calidad del cuerpo docente y la producción cientíca de las instituciones pertenecientes al Consejo de Rectores de las universidades chilenas. Ran, 1 (1), 39–52.
Marginson, S. (1998). Harvards of the antipodes?: nation-building universities in a global environment. Leading and Managing, 4 (3), 156.
McLendon, M. K.; Hearn, J. C.; Deaton, R. (2006). Called to Account: Analyzing the Origins and Spread of State Performance-Accountability Policies for Higher Education. Educational Evaluation and Policy Analysis, 28 (1), 1–24. https://doi.org/10.3102/01623737028001001
Mecesup. (2016). Aporte Fiscal Directo. http://www.mecesup.mineduc.cl/index2.php?id_contenido=28082&id_ seccion=4963&id_portal=59 [Fecha de Consulta 11 de Agosto de 2016].
Mineduc. (2015). Decreto 121. División Jurídica. http:// www.mecesup.cl/usuarios/MECESUP/File/2015/AFD/ DEC121_5AFD_cruch.pdf [Fecha de Consulta 29 de Agosto de 2016]
Ministerio de Educación Pública. (1991). Decreto 128. https://www.leychile.cl/Navegar?idNorma=9705 [Fecha de Consulta 25 de Agosto de 2016]
Mohrman, K.; Ma, W.; Baker, D. (2008). The research university in transition: The emerging global model. Higher Education Policy, 21 (1), 5–27. https://doi.org/10.1057/palgrave.hep.8300175
Molas-Gallart, J.; Salter, A.; Patel, P.; Scott, A.; Duran, X. (2002). Measuring third stream activities. Final report to the Russell Group of Universities. Brighton: Sciencie And Technology Policy Research (SPRU), University of Sussex, 85 p.
Neave, G. (2001). The changing frontiers of autonomy and accountability. Higher Education Policy, 14 (1), 1–5. https://doi.org/10.1016/S0952-8733(01)00002-2
OCDE. (2009). La Educación Superior en Chile. Santiago. http://www7.uc.cl/webpuc/piloto/pdf/informe_OECD. pdf [Fecha de Consulta 7 de noviembre de 2016]
Olssen, M.; Peters, M. A. (2005). Neoliberalism, higher education and the knowledge economy: from the free market to knowledge capitalism. Journal of Education Policy, 20 (3), 313–345. https://doi.org/10.1080/02680930500108718
Ordorika, I.; Lloyd, M. (2013). Rankings and accountability in higher education: uses and misuses. En: Marope, P. T. M.; Wells, P. J.; Hazelkorn, E. (eds.), Rankings and accountability in higher education: uses and misuses (pp. 209–231); Unesco.
Palomares-Montero, D.; García-Aracil, A.; Castro-Martínez, E. (2008). Evaluación de las instituciones de educación superior: revisión bibliográfica de sistema de indicadores. Revista Española de Documentación Científica, 31 (2), 205–229. https://doi.org/10.3989/redc.2008.v31.i2.425
Power, M. (1997). The audit society: Rituals of verification. Oxford: Oxford University Press, 183 p.
Rabovsky, T. M. (2012). Accountability in higher education: Exploring impacts on state budgets and institutional spending patterns. Journal of Public Administration Research and Theory, 22 (4), 675–700. https://doi.org/10.1093/jopart/mur069
Ranson, S. (2003). Public accountability in the age of neolliberal governance. Journal Education Policy, 18 (5), 459–480. https://doi.org/10.1080/0268093032000124848
Saravanamuthu, K.; Tinker, T. (2002). The university in the new corporate world. Critical Perspectives on Accounting, 13 (5-6), 545–554. https://doi.org/10.1006/cpac.2002.0551
Saunders, D. (2010). Neoliberal ideology and public higher education in the United States. Journal for Critical Education Policy Studies, 8 (1), 41–77.
Schiefelbein, E. (1990). Chile: Economic incentives in higher education. Higher Education Policy, 3 (3), 21– 26. https://doi.org/10.1057/hep.1990.41
Seshan, R.; Jain, K. (2011). Accountability in Higher Education. Entire Research, 3 (1), 1–6. http://www. ghrws.in/Entire%20Research/E.%20R.%202011/ Volume-3,%20Issue-I,%20January%202011.pdf
Shore, C. (2010). Beyond the multiversity: neoliberalism and the rise of the schizophrenic university. Social Anthropology, 18 (1), 15–29. https://doi.org/10.1111/j.1469-8676.2009.00094.x
Silva, P. (1991). Technocrats and Politics in Chile: from the Chicago Boys to the CIEPLAN Monks. Journal of Latin American Studies, 23 (2), 385–410. https://doi.org/10.1017/S0022216X00014048
Suspitsyna, T. (2010). Accountability in American education as a rhetoric and a technology of governmentality. Journal of Education Policy, 25 (5), 567–586. https://doi.org/10.1080/02680930903548411
Sztompka, P. (1993). Civilizational incompetence: The trap of post-communist societies. Zeitschrift Für Soziologie, 22 (2), 85–95. https://doi.org/10.1515/zfsoz-1993-0201
Testa, J. (2001). La base de datos del ISI y su proceso de selección de revistas. Acimed, 9, 138–140.
Thelwall, M.; Delgado, M. (2015). Arts and humanities research evaluation: no metrics please, just data. Journal of Documentation, 71 (4), 817–833. https://doi.org/10.1108/JD-02-2015-0028
Torche, F. (2005). Privatization Reform and Inequality of Educational Opportunity. Sociology of Education, 78 (4), 316–343. https://doi.org/10.1177/003804070507800403
Torres, C. A.; Schugurensky, D. (2002). The political economy of higher education in the era of neoliberal globalization: Latin America in comparative perspective. Higher Education, 43 (4), 429–455. https://doi.org/10.1023/A:1015292413037
Wallace, M.; Rafols, I. (2015). Research portfolio analysis in science policy: moving from financial returns to societal benefits. Minerva, 53 (2), 89–115. https://doi.org/10.1007/s11024-015-9271-8
Weyland, K. (1999). Economic Policy in Chile's New Democracy. Journal of Interamerican Studies and World Affairs, 41 (3), 67-93. https://doi.org/10.2307/166159
World Bank. (1992). Governance and development. Washington, D.C: Banco Mundial, 69 p. https://doi.org/10.1596/0-8213-2094-7
Published
How to Cite
Issue
Section
License
Copyright (c) 2018 Consejo Superior de Investigaciones Científicas (CSIC)
This work is licensed under a Creative Commons Attribution 4.0 International License.
© CSIC. Manuscripts published in both the print and online versions of this journal are the property of the Consejo Superior de Investigaciones Científicas, and quoting this source is a requirement for any partial or full reproduction.
All contents of this electronic edition, except where otherwise noted, are distributed under a Creative Commons Attribution 4.0 International (CC BY 4.0) licence. You may read the basic information and the legal text of the licence. The indication of the CC BY 4.0 licence must be expressly stated in this way when necessary.
Self-archiving in repositories, personal webpages or similar, of any version other than the final version of the work produced by the publisher, is not allowed.
