A model for ontology evaluation. The case of Onto-Satcol
DOI:
https://doi.org/10.3989/redc.2011.3.788Keywords:
Ontologies, Ontology Evaluation, Domain ontologies, Ontological analysis and evaluation, port engineering, coastal engineeringAbstract
This paper analyzes the conceptual and theoretical framework for the evaluation of ontologies, in order to understand the procedures used in the evaluation of these systems and to establish new guidelines for evaluating the system employed by the ontological program, SATCOL, that specializes in Port and Coastal Engineering. This paper describes the characteristics of the Onto-SATCOL ontology and evaluates it by using several indicators (lexical, information retrieval, and syntactic structure). Through an experiment conducted by six experts aided by the tool, Protex, semantic and structural inconsistencies are identified, as are errors in the ontology’s organization of knowledge.
Downloads
References
Boehm, B. W. (1981): Software Engineering Economics. New York; Prentice-Hall.
Brank, J.; Grobelnik, M., y Mladenic, D. (2005): A survey of ontology evaluation techniques. Paper presented at the SIKDD 2005 at multiconference IS 2005, Ljubljana, Slovenia. http://eprints.pascal-network.org/archive/00001198/01/BrankEvaluationSiKDD2005.pdf.
Brewster, C.; Alani, H.; Dasmahapatra, S., y Wilks, Y. (2004): Data Driven Ontology Evaluation. Paper presented at the Proceedings of the Language Resources and Evaluation Conference (LREC 2004), Lisboa, Portugal.
Domíguez, S. (2010): PROTEX (Version beta) [Lematizador]. Santa Clara, Cuba: Universidad Central de las Villas, Departamento de automática.
Evermann, J. E., y Fang, J. (2010): Evaluating ontologies: Towards a cognitive measure of quality. Information Systems, vol. 35, 391-403. http://dx.doi.org/10.1016/j.is.2008.09.001
Fahad, M., y Abdul Qadir, M. (2008): A Framework for Ontology Evaluation. Paper presented at the Supplementary Proceedings of the 16th International Conference on Conceptual Structures (ICCS’08), Toulouse, France. http://ftp.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-354/p59r.pdf.
Fellbaum, C. (1998): WordNet – An Electronic Lexical Database. Massachusetts: MIT-Press.
Fernández-Breis, J. T.; Egaña Aranguren, M., y Stevens, R. (2009): Quality evaluation framework for bio-ontologies. Paper presented at the ICBO: International Conference on Biomedical Ontology, Buffalo: New York.
Fernández-López, M.; Gómez-Pérez, A., y Juristo, N. (1997): Methontology: from ontological art towarads ontological engineering. Paper presented at the Spring Symposium on Ontological. Engineering of AAAI, California.
Gangemi, A.; Catenacci, C.; Ciaramita, M., y Lehmann, J. (2005): A Theoretical Framework for Ontology Evaluation and Validation.
Gómez Pérez, A. (1994): Some Ideas and Examples to Evaluate Ontologies. Conference paper at an IEEE Event.
Grunninger, M., y Fox, M. S. (1995): Methodology for the Design and Evaluation of Ontologies. Paper presented at the Workshop on Basic Ontological Issues in Knowledge Sharing, IJCAI-95, Montreal.
Guarino, N., y Welty, C. (2002): Evaluating Ontological Decisions with OntoClean. Communications of the ACM, vol. 45 (2), 61-65. http://dx.doi.org/10.1145/503124.503150
Hartmann, J.; Spyns, P.; Giboin, A.; Maynard, D.; Cuel, R.; Suárez-Figueroa, M. C., y otros. (2005): D1.2.3 Methods for ontology evaluation, Knowledgeweb.
Kent, A. (1955): Machine literature searching. VIII. Operational Criteria for Designing Information Retrieval Systems American Documentation, vol. 6 (2), 93-101.
Kuhn, T. (1979): The structure of scientifi c revolutions. Chicago: University of Chicago Press.
Leiva-Mederos, A.; Senso, J. A.; Domínguez-Velasco, S., e Hípola, P. (2009): An Automat for the Semantic Processing of Structured Information. Paper presented at the International Conference on Intelligent Systems Design and Applications. http://dx.doi.org/10.1109/ISDA.2009.120
Lozano-Tello, A., y Gómez-Pérez, A. (2004): ONTOMETRIC: A method to choose the appropriate ontology. Journal of Database Management, vol. 15 (2). http://dx.doi.org/10.4018/jdm.2004040101
Maedche, A.; Motik, B.; Stojanovic, L.; Studer, R., y Volz, R. (2003): Ontologies for Enterprise Knowledge Management. EEE Intelligent System, 26-34.
Maedche, A., y Staab, S. (2002): Measuring Similarity between Ontologies. Paper presented at the In: Proc. of the European Conference on Knowledge Acquisition and Management – EKAW-2002, Madrid: Spain.
Maiga, G., y Ddembe, W. (2008): A Systems Approach to User Evaluation of Biomedical Ontologies Paper presented at the First International Workshop on Ontologies in Interactive Systems.
Ning, H., y Shihan, D. (2006): Structure-Based Ontology Evaluation. Paper presented at the International Conference on e-business on Engineering.
Obrst, L.; Werner, C.; Inderjeet, M.; Ray, S., y Smith, B. (2007): The Evaluation of Ontologies: Toward Improved Semantic Interoperability. En C. J. O. Baker & K.-H. Cheung (Eds.), Semantic Web: Revolutionizing Knowledge Discovery in the Life Sciences: Springer.
Paslaru Bontas, E., y Mochol, M. (2005): A Cost Model for Ontology Engineering. En F. U. Berlin (Ed.).
Qadir, M., y Noshairwan, W. (2007): Warnings for Disjoint Knowledge Omission in Ontologies. Paper presented at the Second International Conference on internet and Web Applications and Services (ICIW07).
Ramos, E.; Núñez, H., y Casañas, R. (2009): Esquemas para evaluar ontologías únicas para un dominio de conocimiento. Revista Venezolana de Información, Tecnología y Conocimiento, vol. 6 (1), 57-71.
Sabou, M.; López, V.; Motta, E., y Uren, V. (2006): Ontology Selection:Ontology Evaluation on the Real Semantic Web. Paper presented at the WWW2006, Edimburgo.
Salton, G.; Wong, A., y Yang, C. S. (1975): A Vector Space Model for Automatic Indexing. Communications of the ACM, vol. 18 (11), 613-620. http://dx.doi.org/10.1145/361219.361220
Spyns, P. (2005): EvaLexon: Assessing triples mined from texts. Bruselas.
Staab, S. (2004): Why Evaluate Ontology Technologies? Because It Works! IEEE Intelligent Systems.
Uschold, M., y King, M. (1995): Towards a Methodology for Building Ontologies. Workshop on Basic Ontological Issues in Knowledge Sharing.
Yao, H.; Mark, A.; Orme, M., y Etzkorn, L. (2005): Cohesion Metrics for Ontology Design and Application. Journal of Computer Science, vol. 1 (1), 107-113. http://dx.doi.org/10.3844/jcssp.2005.107.113
Yu, J.; James, A. T., y Tam, A. (2009): Requirements-oriented methodology for evaluating ontologies. Information Systems, vol. 34, 766-791. http://dx.doi.org/10.1016/j.is.2009.04.002
Zhang, T.; Xu, D., y Chen, J. (2008): Application-oriented purely semantic precision and recall for ontology mapping evaluation. Knowledge-Based Systems, vol. 21, 794-799. http://dx.doi.org/10.1016/j.knosys.2008.03.060
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2011 Consejo Superior de Investigaciones Científicas (CSIC)
This work is licensed under a Creative Commons Attribution 4.0 International License.
© CSIC. Manuscripts published in both the print and online versions of this journal are the property of the Consejo Superior de Investigaciones Científicas, and quoting this source is a requirement for any partial or full reproduction.
All contents of this electronic edition, except where otherwise noted, are distributed under a Creative Commons Attribution 4.0 International (CC BY 4.0) licence. You may read the basic information and the legal text of the licence. The indication of the CC BY 4.0 licence must be expressly stated in this way when necessary.
Self-archiving in repositories, personal webpages or similar, of any version other than the final version of the work produced by the publisher, is not allowed.
