=Paper=
{{Paper
|id=Vol-1207/paper_3
|storemode=property
|title=TROvE: a Graphical Tool to Evaluate OWL Reasoners
|pdfUrl=https://ceur-ws.org/Vol-1207/paper_3.pdf
|volume=Vol-1207
|dblpUrl=https://dblp.org/rec/conf/ore/BourguetP14
}}
==TROvE: a Graphical Tool to Evaluate OWL Reasoners==
https://ceur-ws.org/Vol-1207/paper_3.pdf
TROvE: a Graphical Tool to Evaluate
OWL Reasoners
Jean-Rémi Bourguet and Luca Pulina
POLCOMING, Università di Sassari, Viale Mancini 5 – 07100 Sassari – Italy
boremi@uniss.it - lpulina@uniss.it
Abstract. In this paper we present TROvE (Tool for Rapid OWL Rea-
soner Evaluation), a tool aimed to offer to a non-expert user the possi-
bility to evaluate OWL reasoners on several reasoning tasks by means of
a simple “push-button” solution.
1 Introduction
Reasoning with ontologies is one of the core tasks of research in Description
Logics (see, e.g., [1]). This is also witnessed by the large number of reasoners
currently available, see, e.g., the results of the last OWL Reasoner Evaluation
(ORE) [2]. Given the wide range of potential practical applications in the Se-
mantic Web [3], practitioners leveraging semantic reasoners in their applications
have to answer the question “Which system should I use?”. To answer such a
question, several projects and events have been implemented, e.g., the SEALS
project http://www.seals-project.eu and the ORE. Also if in such kind of
events reasoners are evaluated using transparent and fair methods, in a practical
application context a practitioner could be interested in understanding the cur-
rent state of the art related to a particular reasoning task for which data could
not be available to the research community. This can lead a non-expert user to
deal with several issues, both technical and theoretical.
In this paper we present TROvE (Tool for Rapid OWL Reasoner Evalua-
tion), a tool aimed to offer to a non-expert user the possibility to evaluate OWL
reasoners on several reasoning tasks. In the spirit of tools such as weka [4] for the
experimental evaluation of machine learning algorithms, TROvE offers to the
user a “push-button” solution aimed to help the user answer the question above,
focusing only on input data at the user execution stage, and showing data in
order to evaluate both correctness and performance at the user validation stage.
2 Architecture of TROvE
The architecture of TROvE1 builds on and extends the one of FRaQuE [5], a
command line tool aimed to offer to a non-expert user easy solutions aimed to
help her to evaluate query processors for Ontology Based Data Access.
Figure 1 presents the architecture of TROvE. It is composed of six modules:
1
TROvE is available for download at http://sites.google.com/site/trove14.
Fig. 1: The architecture of TROvE.
INTERFACE manages the input received by the user by means of the Graphical
User Interface (GUI) depicted in Figure 2. It also dispatches the input data
to TASK MANAGER, ONTOLOGY MANAGER, and QUERY MANAGER (if it is the case).
ONTOLOGY MANAGER is devoted to manage the OWL input file(s).
TASK MANAGER is aimed to manage the reasoning task selected by the user. At
the time of writing, TROvE supports classification, consistency checking
and query answering.
QUERY MANAGER is devoted to process the query input file. It checks the compli-
ance of the query with the SPARQL 1.0 syntax, and, considering the selected
reasoner(s), it applies syntactic modifications to the input query file or re-
turns to INTERFACE an error message if the input query is not supported by
the selected reasoner(s).
REASONER MANAGER is aimed to manage the interaction of TROvE with the
supported reasoners. Selected reasoners are executed in a sequential way
with default parameters.
RESULT MANAGER takes in charge outputs: (i) as performance metrics managing
shell and log files, (ii) as text file(s) containing the query result(s) in case of
query answering task.
The reasoners currently supported by TROvE are listed below:
– ARQ [6] (version 2.9.4) is the built-in query processor of the Jena library.
In TROvE it is used with owl-dl semantics.
– BaseVISor (version 2.0)2 a reasoner for owl2 rl.
– Clipper [7] (version 0.1) is a reasoner for conjunctive query answering via
query rewriting. The core of Clipper is a novel query rewriting technique
which transforms an input conjunctive query into a union of queries that a
Datalog program performs over the Abox completed by some rules.
– ELK [8] (version 0.4.1), a reasoner for ontologies in owl2 el.
– HermiT [9] (version 1.3.6) is a DL reasoner based on hypertableau calcu-
lus [10]. It can be used to answer sparql 1.0 queries by means of the sparql
1.0 wrapper owl-bgp [11].
– jcel [12] (version 0.19.1), a reasoner for owl2 el.
– Jfact(version 1.2.1) is a Java implementation of the C++ reasoner Fact++
[13] which supports the owl2 dl fragment with extended datatype support.
– Kaon2 [14, 15] (version 2008-06-29) implements reasoning algorithms aimed
to reduce a knowledge base to a disjunctive datalog program, allowing the
usage of deductive database techniques. So, with respect to other DL reason-
ers like HermiT, Pellet and TrOWL, it does not implement a tableau-like
calculus.
– Konclude [16] (version 0.5.0-275) is a reasoner supporting ontologies in
owl2 dl fragment.
– MORe [17] (version 0.1.6) is a reasoner using HermiT, with the specificity
to perform extraction techniques by identifying ontology subsets that can be
completely classified by ELK.
– Owlim [18] (version lite 5.3) is a family of semantic repositories, or rdf
database management systems, with the original characteristics to be a ro-
bust support for the semantics of owl2 rl and owl2 ql profiles.
– Pellet [19] (version 2.3.0) is a description logic reasoner supporting owl-
dl.
– Quest [20] (version 1.9) is a reasoner for OBDA with databases, and it is
restricted to owl2 ql. In TROvE it is used in “virtual mode”.
– SnoRocket [21] (version 2.4.4) is a reasoner initially developed to classify
the SNOMED CT ontology. It is restricted to the owl2 el profile.
– TReasoner (version 2014-01-28) [22] is a reasoner which supports the owl2
dl fragment with extended datatype support.
– TrOWL [23] (version 1.3) is an infrastructure aimed to reasoning, and
querying OWL 2 ontologies by means of several techniques, e.g., quality
guaranteed approximations and forgetting (see [23] for details).
– WSClassifier [24] (version 2013) is a reasoner using a hybrid of the rea-
soners ConDOR [25] and HermiT.
Concerning the GUI, looking at Figure 2, we can see that it is composed
of several blocks. In Reasoning Tasks, users can select a reasoning task and,
accordingly, in Reasoners, users can choose a set of suitable reasoners by checking
them (reasoners not available to cope with the selected task are not checkable).
2
See the BaseVISor web site at http://visiology.com/basevisor/basevisor.
html
Fig. 2: Graphical User Interface of TROvE.
In Ontologies Loading, users can load a (set of) OWL file(s) in the following
ways: (i) single file selection; (ii) folder selection; and (iii) copy & paste using a
text area. In Queries Loading, users can load a (set of) query files with the same
options available in Ontologies Loading. In Output directory, users can select a
folder to gather the results. Finally, in Results user can launch the experiment
pushing the “Start” button.
3 Conclusions
In this paper, we presented TROvE, a graphical tool that allows a non-expert
user to perform a rapid evaluation of state-of-the-art reasoners on different rea-
soning tasks, e.g., classification, consistency checking, and query answering.
Currently, we are working to extend TROvE in two main directions. The first
one concerns the usability of the GUI, especially considering the visualization
of the results related to a given experiment. The second one is related to the
modularity of TROvE, in order to simplify the mechanism used to add a new
reasoner to the pool supported by TROvE.
Acknowledgments The authors wish to thank the anonymous reviewers for their
comments and suggestions for improving the paper. This work is supported
by Regione Autonoma della Sardegna e Autorità Portuale di Cagliari con L.R.
7/2007, Tender 16 2011, CRP-49656 “Metodi innovativi per il supporto alle
decisioni riguardanti lottimizzazione delle attività in un terminal container”, the
desctop project (http://visionlab.uniss.it/desctop).
References
1. F. Baader. The Description Logic Handbook: Theory, Implementation, and Appli-
cations. Cambridge University Press, 2003.
2. Rafael S. Gonçalves, Samantha Bail, Ernesto Jiménez-Ruiz, Nicolas Matentzoglu,
Bijan Parsia, Birte Glimm, and Yevgeny Kazakov. Owl reasoner evaluation (ore)
workshop 2013 results: Short report. In Informal Proceedings of the 2nd Interna-
tional Workshop on OWL Reasoner Evaluation (ORE13), volume 1015 of CEUR
Workshop Proceedings, pages 1–18, 2013.
3. Tim Berners-Lee, James Hendler, Ora Lassila, et al. The semantic web. Scientific
american, 284(5):28–37, 2001.
4. Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann,
and Ian H Witten. The weka data mining software: an update. ACM SIGKDD
explorations newsletter, 11(1):10–18, 2009.
5. Jean-Rémi Bourguet and Luca Pulina. Fraque: A framework for rapid query pro-
cessing evaluation. In Informal Proceedings of the 2nd International Workshop on
OWL Reasoner Evaluation (ORE13), volume 1015 of CEUR Workshop Proceed-
ings, pages 53–60, 2013.
6. A. Seaborne. ARQ – A SPARQL Processor for Jena, 2010. http://jena.
sourceforge.net/ARQ/.
7. Thomas Eiter, Magdalena Ortiz, Mantas Simkus, Trung-Kien Tran, and Guohui
Xiao. Query rewriting for horn-shiq plus rules. In Proceedings of the Twenty-Sixth
AAAI Conference on Artificial Intelligence. AAAI Press, 2012.
8. Yevgeny Kazakov, Markus Krötzsch, and Frantisek Simancik. Concurrent classifi-
cation of el ontologies. In The Semantic Web - ISWC 2011 - 10th International Se-
mantic Web Conference, Bonn, Germany, October 23-27, 2011, Proceedings, Part
I, volume 7031 of Lecture Notes in Computer Science, pages 305–320. Springer,
2011.
9. Rob Shearer, Boris Motik, and Ian Horrocks. Hermit: A highly-efficient owl rea-
soner. In Proceedings of the 5th International Workshop on OWL: Experiences and
Directions (OWLED 2008), pages 26–27, 2008.
10. Boris Motik, Rob Shearer, and Ian Horrocks. Optimized reasoning in description
logics using hypertableaux. Automated Deduction–CADE-21, pages 67–83, 2007.
11. Ilianna Kollia and Birte Glimm. Optimizing sparql query answering over owl
ontologies. Journal of artificial intelligence research, 48(253-303):3, 2013.
12. Julian Mendez. jcel: A modular rule-based reasoner. In Proceedings of the 1st
International Workshop on OWL Reasoner Evaluation (ORE-2012), Manchester,
UK, July 1st, 2012, volume 858 of CEUR Workshop Proceedings. CEUR-WS.org,
2012.
13. D. Tsarkov and I. Horrocks. Fact++ description logic reasoner: System description.
In Proc. of the Int. Joint Conf. on Automated Reasoning (IJCAR 2006), volume
4130 of LNAI, pages 292–297. Springer, 2006.
14. Ullrich Hustadt, Boris Motik, and Ulrike Sattler. Reducing shiq- description logic
to disjunctive datalog programs. Proc. KR, 4:152–162, 2004.
15. Boris Motik, Ulrike Sattler, and Rudi Studer. Query answering for owl-dl with
rules. Web Semantics: Science, Services and Agents on the World Wide Web,
3(1):41–60, 2005.
16. derivo GmbH. Reasoning system konclude.
17. Ana Armas Romero, Bernardo Cuenca Grau, Ian Horrocks, and Ernesto Jiménez-
Ruiz. More: a modular owl reasoner for ontology classification. In Informal Pro-
ceedings of the 2nd International Workshop on OWL Reasoner Evaluation (ORE-
2013), Ulm, Germany, July 22, 2013, volume 1015 of CEUR Workshop Proceedings,
pages 61–67. CEUR-WS.org, 2013.
18. Atanas Kiryakov, Damyan Ognyanov, and Dimitar Manov. Owlim - a pragmatic
semantic repository for owl. In Web Information Systems Engineering - WISE
2005 Workshops, volume 3807 of LNCS, pages 182–192, 2005.
19. E. Sirin, B. Parsia, B. Cuenca-Grau, A. Kalyanpur, and Y. Katz. Pellet: A practical
OWL-DL reasoner. Web Semantics: science, services and agents on the World
Wide Web, 5(2):51–53, 2007. Available on-line from http://pellet.owldl.com/.
20. M. Rodrıguez-Muro and D. Calvanese. Quest, an OWL 2 QL Reasoner for
Ontology-based Data Access. OWLED 2012, 2012.
21. Alejandro Metke-Jimenez and Michael Lawley. Snorocket 2.0: Concrete domains
and concurrent classification. In Informal Proceedings of the 2nd International
Workshop on OWL Reasoner Evaluation (ORE-2013), Ulm, Germany, July 22,
2013, volume 1015 of CEUR Workshop Proceedings, pages 32–38. CEUR-WS.org,
2013.
22. Andrey V. Grigorev and Alexander G. Ivashko. Treasoner: System description.
In Informal Proceedings of the 2nd International Workshop on OWL Reasoner
Evaluation (ORE13), volume 1015 of CEUR Workshop Proceedings, pages 26–31,
2013.
23. Edward Thomas, Jeff Z. Pan, and Yuan Ren. TrOWL: Tractable OWL 2 Rea-
soning Infrastructure. In the Proc. of the Extended Semantic Web Conference
(ESWC2010), 2010.
24. Weihong Song, Bruce Spencer, and Weichang Du. A transformation approach for
classifying alchi(d) ontologies with a consequence-based alch reasoner. In Infor-
mal Proceedings of the 2nd International Workshop on OWL Reasoner Evaluation
(ORE-2013), Ulm, Germany, July 22, 2013, volume 1015 of CEUR Workshop Pro-
ceedings, pages 39–45. CEUR-WS.org, 2013.
25. Frantisek Simancik, Yevgeny Kazakov, and Ian Horrocks. Consequence-based rea-
soning beyond horn ontologies. In Toby Walsh, editor, IJCAI 2011, Proceedings
of the 22nd International Joint Conference on Artificial Intelligence, Barcelona,
Catalonia, Spain, July 16-22, 2011, pages 1093–1098. IJCAI/AAAI, 2011.