Assessing the teaching value of non-English academic books: The case of Spain
DOI:
https://doi.org/10.3989/redc.2018.4.1568Keywords:
books, monographs, course syllabus, evaluation, teaching impactAbstract
This study examines the educational value of 15,117 Spanish-language books published by Spanish publishers in social sciences and humanities fields in the period 2002-2011, based on mentions of them extracted automatically from online course syllabi. A method was developed to collect syllabus mentions and filter out false matches. Manual checks of the 52,716 syllabus mentions found estimated an accuracy of 99.5% for filtering out false mentions and 74.7% for identifying correct mentions. A fifth of the sampled books (2,849; 19%) were mentioned at least once in online syllabi and almost all (95%) were from a third of the publishers included in the study. An in-depth analysis of the 23 books recommended most often in online syllabi showed that they are mostly single-authored humanities monographs that were originally written in Spanish. The syllabus mentions originated from 379 domains, but mostly from Spanish university websites. In conclusion, it is possible to make indicators from online syllabus mentions to assess the teaching value of Spanish-language books, although manual checks are needed if the values are to be used for assessing individual books.
Downloads
References
ANECA (2017). Méritos evaluables para la acreditación nacional para el acceso a los cuerpos docentes universitarios de Profesor Titular de Universidad y Catedrático de Universidad. Criterios de Arte y Humanidades. Available at: http://www.aneca.es/Programas-de-evaluacion/ACADEMIA/Criterios-de-evaluacion-noviembre-2017
Archambault, E.; Vignola-Gagné; É.; Côté, G.; Larivière, V.; Gingras, Y. (2006). Benchmarking scientific output in the social sciences and humanities: The limits of existing databases. Scientometrics, 68 (3), 329-342. https://doi.org/10.1007/s11192-006-0115-z
Barron, P. (2011). The library of the future: Google's vision for books. Learned Publishing, 24 (3), 197-201. https://doi.org/10.1087/20110307
Book Citation Index (2017). Putting books back into the library. Completing the research picture. The Book Citation Index. Available at: http://wokinfo.com/ products_tools/multidisciplinary/bookcitationindex/
Bosch, A. van den; Bogers, T.; Kunder, M. de (2016). Estimating search engine index size variability: A 9-year longitudinal study. Scientometrics, 107 (2), 839-856. https://doi.org/10.1007/s11192-016-1863-z PMid:27122648 PMCid:PMC4833824
Caerols Mateo, R.; Verdú Ruiz, S.; Vi-arás Abad, M. (2017). Las artes en la educación superior: la evaluación en España de la actividad investigadora en Bellas Artes a través de los indicadores de calidad. Revista Española de Documentación Científica, 40 (4), e191. https://doi.org/10.3989/redc.2017.4.1394
Cordón-García, J.A.; Gómez-Díaz, R.; Rodríguez-García, A.; Merchán Sánchez-Jara, J.; Mangas-Vega, A. et al., (2017). Visibility of scientific monographs in the academic field: the institutional assessment of research. In: 5th International Conference on Technological Ecosystems for Enhancing Multiculturality, TEEM. Spain, Cádiz. https://doi.org/10.1145/3144826.3145380
Cronin, B.; Barre, K. La (2004). Mickey Mouse and Milton: Book publishing in the humanities. Learned Publishing, 17 (2), 85-98. https://doi.org/10.1087/095315104322958463
Cummings, J.A.; Bonk, C.J.; Jacobs, F.R. (2002). Twenty-first century college syllabi: Options for online communication and interactivity. The Internet and Higher Education, 5(1), 1-19. https://doi.org/10.1016/S1096-7516(01)00077-X
Dimitrov, S.; Zamal, F.; Piper, A.; Ruths, D. (2015). Goodreads vs Amazon: The Effect of Decoupling Book Reviewing and Book Selling. In: Ninth International AAAI Conference on Web and Social Media, pp. 602- 605. England: Oxford University. Available at: https://www.aaai.org/ocs/index.php/ICWSM/ICWSM15/paper/view/10557/10452
Elsevier (2018). Scopus Content Coverage Guide. Available at: https://www.elsevier.com/solutions/ scopus/content
Engels, T.C.E.; Ossenblok, T.L.B.; Spruyt, E.H.J. (2012). Changing publication patterns in the Social Sciences and Humanities, 2000-2009. Scientometrics, 93 (2), 373-390. https://doi.org/10.1007/s11192-012-0680-2
Garand, J.C.; Giles, M.W. (2011). Ranking Scholarly Publishers in Political Science: An Alternative Approach. PS: Political Science and Politics, 44 (2), 375-383. https://doi.org/10.1017/S1049096511000229
Giménez-Toledo, E. (ed.). (2017). La edición académica española. Indicadores y características. Madrid: Federación de Gremios de Editores de España. Available at: http://ilia.cchs.csic.es/SPI/spi-fgee/docs/EAEV2.pdf
Giménez-Toledo, E.; Tejada-Artigas, C.; Ma-ana- Rodríguez, J. (2013). Evaluation of scientific books' publishers in social sciences and humanities: Results of a survey. Research Evaluation, 22 (1), 64-77. https://doi.org/10.1093/reseval/rvs036
Giménez-Toledo, E.; Ma-ana-Rodríguez, J.; Engels, T.C.E.; Ingwersen, P.; Siverten, G.; Verleysen, F.T.; Zuccala, A. (2016). Taking scholarly books into account: current developments in five European countries. Scientometrics, 107 (2), 685-699. https://doi.org/10.1007/s11192-016-1886-5
Gorraiz, J.; Purnell, P.J.; Glänzel, W. (2013). Opportunities for and limitations of the book citation index. Journal of the American Society for Information Science and Technology, 64 (7), 1388–1398. https://doi.org/10.1002/asi.22875
Gorraiz, J.; Gumpenberger, C.; Purnell, P.J. (2014). The power of book reviews: a simple and transparent enhancement approach for book citation indexes. Scientometrics, 98 (2), 841-852. https://doi.org/10.1007/s11192-013-1176-4
Gutiérrez García, C.; Vega-Álvarez, M.P.; Gómez-Alonso, M.T.; Pérez-Pueyo, A. (2016). Análisis de las bibliografías de las asignaturas de pedagogía y didáctica de los grados en ciencias de la actividad física y el deporte en España. Retos: nuevas tendencias en Educación Física, Deporte y Recreación, (30), 138-143. Available at: https://recyt.fecyt.es/index.php/retos/article/view/45045
Halevi, G.; Nicolas, B.; Bar-Ilan, J. (2016). The Complexity of Measuring the Impact of Books. Publishing Research Quarterly, 32 (3), 187-200. https://doi.org/10.1007/s12109-016-9464-5
Hartley, J. (2006). Reading and writing book reviews across the disciplines. Journal of the American Society for Information Science and Technology, 57 (9), 1194- 1207. https://doi.org/10.1002/asi.20399
Hicks, D. (1999). The difficulty of achieving full coverage of international social science literature and the bibliometric consequences. Scientometrics, 44 (2), 193-215. https://doi.org/10.1007/BF02457380
Huang, M.-h.; Chang, Y.-w. (2008). Characteristics of research output in social sciences and humanities: From a research evaluation perspective. Journal of the American Society for Information Science and Technology, 59 (11), 1819-1828. https://doi.org/10.1002/asi.20885
Kousha, K.; Thelwall, M. (2008). Assessing the impact of disciplinary research on teaching: An automatic analysis of online syllabuses. Journal of the American Society for Information Science and Technology, 59 (13), 2060-2069. https://doi.org/10.1002/asi.20920
Kousha, K.; Thelwall, M. (2015). An automatic method for extracting citations from Google Books. Journal of the Association for Information Science and Technology, 66 (2), 309-320. https://doi.org/10.1002/asi.23170
Kousha, K.; Thelwall, M. (2016a). An automatic method for assessing the teaching impact of books from online academic syllabi. Journal of the Association for Information Science and Technology, 67 (12), 2993-3007. https://doi.org/10.1002/asi.23542
Kousha, K.; Thelwall, M. (2016b). Can Amazon.com reviews help to assess the wider impacts of books? Journal of the Association for Information Science and Technology, 67 (3), 566-581. https://doi.org/10.1002/asi.23404
Kousha, K.; Thelwall, M.; Abdoli, M. (2017). Goodreads reviews to assess the wider impacts of books. Journal of the Association for Information Science and Technology, 68 (8), 2004-2016. https://doi.org/10.1002/asi.23805
Larivière, V.; Macaluso, B. (2011). Improving the coverage of social science and humanities researchers' output: The case of the Érudit journal platform. Journal of the American Society for Information Science and Technology, 62 (12), 2437-2442. https://doi.org/10.1002/asi.21632
Leydesdorff, L.; Felt, U. (2012). Edited Volumes, Monographs and Book Chapters in the Book Citation Index (BKCI) and Science Citation Index (SCI, Sosci, A&HCI). Journal of Scientometric Research, 1 (1), 28- 34. https://doi.org/10.5530/jscires.2012.1.7
López-Navarro, I.; Moreno, A.I.; Quintanilla, M.Á.; Rey- Rocha, J. (2015). Why do I publish research articles in English instead of my own language? Differences in Spanish researchers' motivations across scientific domains. Scientometrics, 103 (3), 939-976. https://doi.org/10.1007/s11192-015-1570-1
Marta-Lazo, C.; Grandío Pérez, M.M.; Gabelas Barroso, J.A. (2014). La educación mediática en las titulaciones de Educación y Comunicación de las universidades españolas. Análisis de los recursos recomendados en las guías docentes. Revista de Comunicación Vivat Academia, XVII (126), 63-78. https://doi.org/10.15178/va.2014.126.63-78
Mas-Bleda, A.; Thelwall, M. (2016). Can alternative indicators overcome language biases in citation counts? A comparison of Spanish and UK research. Scientometrics, 109 (3), 2007-2030. https://doi.org/10.1007/s11192-016-2118-8
MECD (2017). Panorámica de la edición española de libros 2016. Ministerio de Educación, Cultura y Deporte: Secretaría General Técnica. Available at: https:// sede.educacion.gob.es/publiventa/panoramica-de-la-edicion-de-libros-2016/edicion/21061C
Moed, H.F.; Luwel, M.; Nederhof, A.J. (2002). Towards Research Performance in the Humanities. Library Trends, 50 (3), 498-520. https://www.ideals.illinois.edu/handle/2142/8415
Nederhof, A.J. (2006). Bibliometric monitoring of research performance in the Social Sciences and the Humanities: A Review. Scientometrics, 66 (1), 81-100. https://doi.org/10.1007/s11192-006-0007-2
Osca-Lluch, J.; Veyrat, A.; Morales, J. (2013). El consumo de información en Humanidades. Arbor, 189 (760), a026. https://doi.org/10.3989/arbor.2013.760n2012
Prieto-Paíno, C. (2013). Tipología, características y género de las bibliografías recomendadas. El caso de Humanidades, ciencias sociales y especialidades científico-técnicas en la Universidad de Salamanca [Dissertation]. Salamanca: Universidad de Salamanca. Available at: https://gredos.usal.es/jspui/handle/10366/123403?mode=full
Romanos de Tiratel, S. (2000). Conducta informativa de los investigadores argentinos en Humanidades y Ciencias Sociales. Revista Española de Documentación Científica, 23 (3), 267-285. https://doi.org/10.3989/redc.2000.v23.i3.324
San-Martín Gutiérrez, S.; Jiménez Torres, N.; Jerónimo Sánchez-Beato, E. (2016). La evaluación del alumnado universitario en el Espacio Europeo de Educación Superior. Aula Abierta, 44 (1), 7-14. https://doi.org/10.1016/j.aula.2015.03.003
Sivertsen, G. (2016). Patterns of internationalization and criteria for research assessment in the social sciences and humanities. Scientometrics, 107 (2), 357-368. https://doi.org/10.1007/s11192-016-1845-1 PMid:27122643 PMCid:PMC4833822
Sivertsen, G.; Larsen, B. (2012). Comprehensive bibliographic coverage of the social sciences and humanities in a citation index: an empirical analysis of the potential. Scientometrics, 91 (2), 567-575. https://doi.org/10.1007/s11192-011-0615-3
Slattery, J.M.; Carlson, J.F. (2005). Preparing an effective syllabus: Current best practices. College Teaching, 53 (4), 159-164. https://doi.org/10.3200/CTCH.53.4.159-164
Sorli Rojo, A.; Mochón Bezares, G.; Martín-Carretero, C. (2011). Rese-as en revistas científicas españolas de Ciencias Sociales y Humanidades: análisis de la producción entre 2007 y 2009. Revista Española de Documentación Científica, 34 (4), 526-544. https://doi.org/10.3989/redc.2011.4.841
Spink, A.; Robins, D.; Schamber, L. (1998). Use of Scholarly Book Reviews: Implications for Electronic Publishing and Scholarly Communication. Journal of the American Society for Information Science and Technology, 49 (4), 364-374. https://doi.org/10.1002/(SICI)1097-4571(19980401)49:4<364::AID-ASI6>3.0.CO;2-3
Thelwall, M. (2017). Three practical field normalised alternative indicator formulae for research evaluation. Journal of Informetrics, 11 (1), 128-151. https://doi.org/10.1016/j.joi.2016.12.002
Thelwall, M.; Sud, P. (2012). Webometric research with the Bing Search API 2.0. Journal of Informetrics, 6 (1), 44-52. https://doi.org/10.1016/j.joi.2011.10.002
Thompson, J.W. (2002). The Death of the Scholarly Monograph in the Humanities? Citation Patterns in Literary Scholarship. Libri, 52 (3), 121-136. https://doi.org/10.1515/LIBR.2002.121
Torres-Salinas, D.; Moed, H.F. (2009). Library Catalog Analysis as a tool in studies of social sciences and humanities: An exploratory study on published book titles in Economics. Journal of Informetrics, 3 (1), 9-26. https://doi.org/10.1016/j.joi.2008.10.002
Torres-Salinas, D.; Robinson-García, N.; Jiménez-Contreras, E.; Delgado López-Cózar, E. (2012). Towards a "Book Publishers Citation Reports". First approach using the "Book Citation Index". Revista Española de Documentación Científica, 35(4), 615–624. https://doi.org/10.3989/redc.2012.4.1010
Torres-Salinas, D.; Robinson-García, N.; Cabezas-Clavijo, Á.; Jiménez-Contreras, E. (2014a). Analyzing the citation characteristics of books: edited books, book series and publisher types in the book citation index. Scientometrics, 98 (3), 2113-2127. https://doi.org/10.1007/s11192-013-1168-4
Torres-Salinas, D.; Robinson-García, N.; Campanario, J.M.; Delgado López-Cózar, E. (2014b). Coverage, field specialisation and the impact of scientific publishers indexed in the Book Citation Index. Online Information Review, 38 (1), 24-42. https://doi.org/10.1108/OIR-10-2012-0169
UNE (2016). Las editoriales universitarias en cifras 2016. Available at: http://www.une.es/media/Ou1/Image/ webdiciembre2016/InformeComercioInteriorUNE2015conlogoMECD.pdf
Vila-Suero, D.; Villazón-Terrazas, B.; Gómez-Pérez, A. (2012). datos.bne.es: A library linked dataset. Semantic Web Journal, 4 (3), 307-313. Available at: http://oa.upm.es/14467/1/4.datosBNEdataset.pdf
White, H.D.; Boell, S.K.; Yu, H.; Davis, M.; Wilson, C.S.; Cole, F.T.H. (2009). Libcitations: A measure for comparative assessment of book publications in the humanities and social sciences. Journal of the American Society for Information Science and Technology, 60 (6), 1083-1096. https://doi.org/10.1002/asi.21045
Williams, L.M.; Cody, S.A.; Parnell, J. (2004). Prospecting for New Collaborations: Mining Syllabi for Library Service Opportunities. The Journal of Academic Librarianship, 30 (4), 270-275. https://doi.org/10.1016/j.acalib.2004.04.009
Zuccala, A.A; Leeuwen, T. van (2011). Book reviews in humanities research evaluations. Journal of the American Society for Information Science and Technology, 62 (10), 1979-1991. https://doi.org/10.1002/asi.21588
Zuccala, A.A.; Guns, R.; Cornacchia, R.; Bod, R. (2015a). Can we rank scholarly book publishers? A bibliometric experiment with the field of history. Journal of the Association for Information Science and Technology, 66 (7), 1333-1347. https://doi.org/10.1002/asi.23267
Zuccala, A.A.; Verleysen, F.T.; Cornacchia, R.; Engels, T.C. (2015b). Altmetrics for the humanities: Comparing Goodreads reader ratings with citations to history books. Aslib Journal of Information Management, 67 (3), 320- 336. https://doi.org/10.1108/AJIM-11-2014-0152
Published
How to Cite
Issue
Section
License
Copyright (c) 2018 Consejo Superior de Investigaciones Científicas (CSIC)
This work is licensed under a Creative Commons Attribution 4.0 International License.
© CSIC. Manuscripts published in both the print and online versions of this journal are the property of the Consejo Superior de Investigaciones Científicas, and quoting this source is a requirement for any partial or full reproduction.
All contents of this electronic edition, except where otherwise noted, are distributed under a Creative Commons Attribution 4.0 International (CC BY 4.0) licence. You may read here the basic information and the legal text of the licence. The indication of the CC BY 4.0 licence must be expressly stated in this way when necessary.
Self-archiving in repositories, personal webpages or similar, of any version other than the final version of the work produced by the publisher, is not allowed.
