Evaluation of the indexing consistency of the repertoire Documentación Médica Española

Abstract

The selection of the documents to be included in the original input and their indexing are two of the most dificult tasks in running one IRS. The adecuacy of the performance of these tasks is dificult to mea sure because there is no correct way to do them due to individual differences. One way to measure these differences is to calculate the degree of consistence with which the same and different indexers carry out the se processes. The aim is to evaluate the inter-indexer consistency in the selection of documents to be included in the third volume of the repertoire Documentación Médica Española (1996) and in the asignation of the main cathegory of the classification and to analize the causes of discrepancies. The new selection and re-indexing of the material has been carried out by an external indexer with expertise in the field. 370 documents have been reselected. The re-indexed documents have been those where both indexers agree that were relevants. To measure consistency the agreement proportion between indexers was calculated. The agreement proportion for the selection of the material was 86 % and for the assignation of the main cathegory of the classification was 78.6 %. The analysis of the discrepancies has showed an high degree of consistency for documents that deal with central topics of the field. Indexing discrepancies are mainly due to errors of both indexers, exemption made of a small group of documents of similar contents.