TROvE: a Graphical Tool to Evaluate OWL Reasoners Jean-Rémi Bourguet and Luca Pulina POLCOMING, Università di Sassari, Viale Mancini 5 – 07100 Sassari – Italy boremi@uniss.it - lpulina@uniss.it Abstract. In this paper we present TROvE (Tool for Rapid OWL Rea- soner Evaluation), a tool aimed to offer to a non-expert user the possi- bility to evaluate OWL reasoners on several reasoning tasks by means of a simple “push-button” solution. 1 Introduction Reasoning with ontologies is one of the core tasks of research in Description Logics (see, e.g., [1]). This is also witnessed by the large number of reasoners currently available, see, e.g., the results of the last OWL Reasoner Evaluation (ORE) [2]. Given the wide range of potential practical applications in the Se- mantic Web [3], practitioners leveraging semantic reasoners in their applications have to answer the question “Which system should I use?”. To answer such a question, several projects and events have been implemented, e.g., the SEALS project http://www.seals-project.eu and the ORE. Also if in such kind of events reasoners are evaluated using transparent and fair methods, in a practical application context a practitioner could be interested in understanding the cur- rent state of the art related to a particular reasoning task for which data could not be available to the research community. This can lead a non-expert user to deal with several issues, both technical and theoretical. In this paper we present TROvE (Tool for Rapid OWL Reasoner Evalua- tion), a tool aimed to offer to a non-expert user the possibility to evaluate OWL reasoners on several reasoning tasks. In the spirit of tools such as weka [4] for the experimental evaluation of machine learning algorithms, TROvE offers to the user a “push-button” solution aimed to help the user answer the question above, focusing only on input data at the user execution stage, and showing data in order to evaluate both correctness and performance at the user validation stage. 2 Architecture of TROvE The architecture of TROvE1 builds on and extends the one of FRaQuE [5], a command line tool aimed to offer to a non-expert user easy solutions aimed to help her to evaluate query processors for Ontology Based Data Access. Figure 1 presents the architecture of TROvE. It is composed of six modules: 1 TROvE is available for download at http://sites.google.com/site/trove14. Fig. 1: The architecture of TROvE. INTERFACE manages the input received by the user by means of the Graphical User Interface (GUI) depicted in Figure 2. It also dispatches the input data to TASK MANAGER, ONTOLOGY MANAGER, and QUERY MANAGER (if it is the case). ONTOLOGY MANAGER is devoted to manage the OWL input file(s). TASK MANAGER is aimed to manage the reasoning task selected by the user. At the time of writing, TROvE supports classification, consistency checking and query answering. QUERY MANAGER is devoted to process the query input file. It checks the compli- ance of the query with the SPARQL 1.0 syntax, and, considering the selected reasoner(s), it applies syntactic modifications to the input query file or re- turns to INTERFACE an error message if the input query is not supported by the selected reasoner(s). REASONER MANAGER is aimed to manage the interaction of TROvE with the supported reasoners. Selected reasoners are executed in a sequential way with default parameters. RESULT MANAGER takes in charge outputs: (i) as performance metrics managing shell and log files, (ii) as text file(s) containing the query result(s) in case of query answering task. The reasoners currently supported by TROvE are listed below: – ARQ [6] (version 2.9.4) is the built-in query processor of the Jena library. In TROvE it is used with owl-dl semantics. – BaseVISor (version 2.0)2 a reasoner for owl2 rl. – Clipper [7] (version 0.1) is a reasoner for conjunctive query answering via query rewriting. The core of Clipper is a novel query rewriting technique which transforms an input conjunctive query into a union of queries that a Datalog program performs over the Abox completed by some rules. – ELK [8] (version 0.4.1), a reasoner for ontologies in owl2 el. – HermiT [9] (version 1.3.6) is a DL reasoner based on hypertableau calcu- lus [10]. It can be used to answer sparql 1.0 queries by means of the sparql 1.0 wrapper owl-bgp [11]. – jcel [12] (version 0.19.1), a reasoner for owl2 el. – Jfact(version 1.2.1) is a Java implementation of the C++ reasoner Fact++ [13] which supports the owl2 dl fragment with extended datatype support. – Kaon2 [14, 15] (version 2008-06-29) implements reasoning algorithms aimed to reduce a knowledge base to a disjunctive datalog program, allowing the usage of deductive database techniques. So, with respect to other DL reason- ers like HermiT, Pellet and TrOWL, it does not implement a tableau-like calculus. – Konclude [16] (version 0.5.0-275) is a reasoner supporting ontologies in owl2 dl fragment. – MORe [17] (version 0.1.6) is a reasoner using HermiT, with the specificity to perform extraction techniques by identifying ontology subsets that can be completely classified by ELK. – Owlim [18] (version lite 5.3) is a family of semantic repositories, or rdf database management systems, with the original characteristics to be a ro- bust support for the semantics of owl2 rl and owl2 ql profiles. – Pellet [19] (version 2.3.0) is a description logic reasoner supporting owl- dl. – Quest [20] (version 1.9) is a reasoner for OBDA with databases, and it is restricted to owl2 ql. In TROvE it is used in “virtual mode”. – SnoRocket [21] (version 2.4.4) is a reasoner initially developed to classify the SNOMED CT ontology. It is restricted to the owl2 el profile. – TReasoner (version 2014-01-28) [22] is a reasoner which supports the owl2 dl fragment with extended datatype support. – TrOWL [23] (version 1.3) is an infrastructure aimed to reasoning, and querying OWL 2 ontologies by means of several techniques, e.g., quality guaranteed approximations and forgetting (see [23] for details). – WSClassifier [24] (version 2013) is a reasoner using a hybrid of the rea- soners ConDOR [25] and HermiT. Concerning the GUI, looking at Figure 2, we can see that it is composed of several blocks. In Reasoning Tasks, users can select a reasoning task and, accordingly, in Reasoners, users can choose a set of suitable reasoners by checking them (reasoners not available to cope with the selected task are not checkable). 2 See the BaseVISor web site at http://visiology.com/basevisor/basevisor. html Fig. 2: Graphical User Interface of TROvE. In Ontologies Loading, users can load a (set of) OWL file(s) in the following ways: (i) single file selection; (ii) folder selection; and (iii) copy & paste using a text area. In Queries Loading, users can load a (set of) query files with the same options available in Ontologies Loading. In Output directory, users can select a folder to gather the results. Finally, in Results user can launch the experiment pushing the “Start” button. 3 Conclusions In this paper, we presented TROvE, a graphical tool that allows a non-expert user to perform a rapid evaluation of state-of-the-art reasoners on different rea- soning tasks, e.g., classification, consistency checking, and query answering. Currently, we are working to extend TROvE in two main directions. The first one concerns the usability of the GUI, especially considering the visualization of the results related to a given experiment. The second one is related to the modularity of TROvE, in order to simplify the mechanism used to add a new reasoner to the pool supported by TROvE. Acknowledgments The authors wish to thank the anonymous reviewers for their comments and suggestions for improving the paper. This work is supported by Regione Autonoma della Sardegna e Autorità Portuale di Cagliari con L.R. 7/2007, Tender 16 2011, CRP-49656 “Metodi innovativi per il supporto alle decisioni riguardanti lottimizzazione delle attività in un terminal container”, the desctop project (http://visionlab.uniss.it/desctop). References 1. F. Baader. The Description Logic Handbook: Theory, Implementation, and Appli- cations. Cambridge University Press, 2003. 2. Rafael S. Gonçalves, Samantha Bail, Ernesto Jiménez-Ruiz, Nicolas Matentzoglu, Bijan Parsia, Birte Glimm, and Yevgeny Kazakov. Owl reasoner evaluation (ore) workshop 2013 results: Short report. In Informal Proceedings of the 2nd Interna- tional Workshop on OWL Reasoner Evaluation (ORE13), volume 1015 of CEUR Workshop Proceedings, pages 1–18, 2013. 3. Tim Berners-Lee, James Hendler, Ora Lassila, et al. The semantic web. Scientific american, 284(5):28–37, 2001. 4. Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H Witten. The weka data mining software: an update. ACM SIGKDD explorations newsletter, 11(1):10–18, 2009. 5. Jean-Rémi Bourguet and Luca Pulina. Fraque: A framework for rapid query pro- cessing evaluation. In Informal Proceedings of the 2nd International Workshop on OWL Reasoner Evaluation (ORE13), volume 1015 of CEUR Workshop Proceed- ings, pages 53–60, 2013. 6. A. Seaborne. ARQ – A SPARQL Processor for Jena, 2010. http://jena. sourceforge.net/ARQ/. 7. Thomas Eiter, Magdalena Ortiz, Mantas Simkus, Trung-Kien Tran, and Guohui Xiao. Query rewriting for horn-shiq plus rules. In Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence. AAAI Press, 2012. 8. Yevgeny Kazakov, Markus Krötzsch, and Frantisek Simancik. Concurrent classifi- cation of el ontologies. In The Semantic Web - ISWC 2011 - 10th International Se- mantic Web Conference, Bonn, Germany, October 23-27, 2011, Proceedings, Part I, volume 7031 of Lecture Notes in Computer Science, pages 305–320. Springer, 2011. 9. Rob Shearer, Boris Motik, and Ian Horrocks. Hermit: A highly-efficient owl rea- soner. In Proceedings of the 5th International Workshop on OWL: Experiences and Directions (OWLED 2008), pages 26–27, 2008. 10. Boris Motik, Rob Shearer, and Ian Horrocks. Optimized reasoning in description logics using hypertableaux. Automated Deduction–CADE-21, pages 67–83, 2007. 11. Ilianna Kollia and Birte Glimm. Optimizing sparql query answering over owl ontologies. Journal of artificial intelligence research, 48(253-303):3, 2013. 12. Julian Mendez. jcel: A modular rule-based reasoner. In Proceedings of the 1st International Workshop on OWL Reasoner Evaluation (ORE-2012), Manchester, UK, July 1st, 2012, volume 858 of CEUR Workshop Proceedings. CEUR-WS.org, 2012. 13. D. Tsarkov and I. Horrocks. Fact++ description logic reasoner: System description. In Proc. of the Int. Joint Conf. on Automated Reasoning (IJCAR 2006), volume 4130 of LNAI, pages 292–297. Springer, 2006. 14. Ullrich Hustadt, Boris Motik, and Ulrike Sattler. Reducing shiq- description logic to disjunctive datalog programs. Proc. KR, 4:152–162, 2004. 15. Boris Motik, Ulrike Sattler, and Rudi Studer. Query answering for owl-dl with rules. Web Semantics: Science, Services and Agents on the World Wide Web, 3(1):41–60, 2005. 16. derivo GmbH. Reasoning system konclude. 17. Ana Armas Romero, Bernardo Cuenca Grau, Ian Horrocks, and Ernesto Jiménez- Ruiz. More: a modular owl reasoner for ontology classification. In Informal Pro- ceedings of the 2nd International Workshop on OWL Reasoner Evaluation (ORE- 2013), Ulm, Germany, July 22, 2013, volume 1015 of CEUR Workshop Proceedings, pages 61–67. CEUR-WS.org, 2013. 18. Atanas Kiryakov, Damyan Ognyanov, and Dimitar Manov. Owlim - a pragmatic semantic repository for owl. In Web Information Systems Engineering - WISE 2005 Workshops, volume 3807 of LNCS, pages 182–192, 2005. 19. E. Sirin, B. Parsia, B. Cuenca-Grau, A. Kalyanpur, and Y. Katz. Pellet: A practical OWL-DL reasoner. Web Semantics: science, services and agents on the World Wide Web, 5(2):51–53, 2007. Available on-line from http://pellet.owldl.com/. 20. M. Rodrıguez-Muro and D. Calvanese. Quest, an OWL 2 QL Reasoner for Ontology-based Data Access. OWLED 2012, 2012. 21. Alejandro Metke-Jimenez and Michael Lawley. Snorocket 2.0: Concrete domains and concurrent classification. In Informal Proceedings of the 2nd International Workshop on OWL Reasoner Evaluation (ORE-2013), Ulm, Germany, July 22, 2013, volume 1015 of CEUR Workshop Proceedings, pages 32–38. CEUR-WS.org, 2013. 22. Andrey V. Grigorev and Alexander G. Ivashko. Treasoner: System description. In Informal Proceedings of the 2nd International Workshop on OWL Reasoner Evaluation (ORE13), volume 1015 of CEUR Workshop Proceedings, pages 26–31, 2013. 23. Edward Thomas, Jeff Z. Pan, and Yuan Ren. TrOWL: Tractable OWL 2 Rea- soning Infrastructure. In the Proc. of the Extended Semantic Web Conference (ESWC2010), 2010. 24. Weihong Song, Bruce Spencer, and Weichang Du. A transformation approach for classifying alchi(d) ontologies with a consequence-based alch reasoner. In Infor- mal Proceedings of the 2nd International Workshop on OWL Reasoner Evaluation (ORE-2013), Ulm, Germany, July 22, 2013, volume 1015 of CEUR Workshop Pro- ceedings, pages 39–45. CEUR-WS.org, 2013. 25. Frantisek Simancik, Yevgeny Kazakov, and Ian Horrocks. Consequence-based rea- soning beyond horn ontologies. In Toby Walsh, editor, IJCAI 2011, Proceedings of the 22nd International Joint Conference on Artificial Intelligence, Barcelona, Catalonia, Spain, July 16-22, 2011, pages 1093–1098. IJCAI/AAAI, 2011.