=Paper=
{{Paper
|id=Vol-2285/ICBO_2018_paper_9
|storemode=property
|title=A Protégé Plug-In for Test-Driven Ontology Development
|pdfUrl=https://ceur-ws.org/Vol-2285/ICBO_2018_paper_9.pdf
|volume=Vol-2285
|authors=Konstantin Schekotihin,Patrick Rodler,Wolfgang Schmid,Matthew Horridge,Tania Tudorache
|dblpUrl=https://dblp.org/rec/conf/icbo/SchekotihinRSHT18a
}}
==A Protégé Plug-In for Test-Driven Ontology Development==
https://ceur-ws.org/Vol-2285/ICBO_2018_paper_9.pdf
Proceedings of the 9th International Conference on Biological Ontology (ICBO 2018), Corvallis, Oregon, USA 1
A Protégé Plug-In for Test-Driven
Ontology Development
Konstantin Schekotihin, Patrick Rodler, Wolfgang Schmid Matthew Horridge, Tania Tudorache
University of Klagenfurt Stanford University
Universitätsstr. 65-67 1265 Welch Rd
Klagenfurt, Austria Stanford, California, USA
Email: firtstname.lastname@aau.at Email: lastname@stanford.edu
Abstract—Ontology development is a hard and often error- and, thus, find a modeling error: “The dorsalis pedis artery is a
prone process, which requires ontology authors to correctly part of the abdomen and pelvis.” The verification of test cases
express their domain knowledge in a formal language. One way as well as the localization and repair of axioms causing their
to ensure the quality of the resulting ontology is to use test
cases, similarly to the best practices in software development. For violation is very difficult without appropriate tool support.
ontology development, test cases can be specified as statements
describing expected and/or unwanted logical consequences of an II. T HE O NTO D EBUG P ROT ÉG É P LUG -I N
ontology. However, verifying the test cases and identifying the To address this tooling gap, we developed the OntoDebug
ontology parts that cause their violation is a complex task, which
requires appropriate tool support.
Protégé plug-in.1 It supports the test-driven development of
In this demo, we present OntoDebug—a plug-in for the Protégé ontologies, and can be installed from the standard plug-ins
editor—that supports test-driven ontology development. OntoDe- repository directly in the editor. The user interface of the plug-
bug can automatically verify whether the ontology satisfies all in is shown in Fig. 1. The functionality of OntoDebug can be
defined test cases. If any test case is violated, the plug-in assists summarized as follows:
the user in debugging and repairing the ontology in an interactive
way. The plug-in asks a series of questions about the ontology to a) Test-driven Development: Ontology authors define
pin point the faulty axioms. Once a fault is repaired, all answers test cases in the Original Test Cases window (Fig. 1, view
that the author provided in the interactive debugging session, 5). The window has two sections: (1) Entailed Test Cases, in
may be converted into test cases, thus preserving the additional which users define expected inferred axioms of the intended
knowledge, which can be used in future testing of the ontology. ontology; and (2) Non Entailed Test Cases, in which users
I. T EST-D RIVEN O NTOLOGY D EVELOPMENT specify axioms that must never be inferred. For the specifi-
cation of test cases it can be helpful to browse the ontology,
Semantic applications, especially in biomedicine, depend
which can be done in the Possibly Faulty Axioms window
on high-quality ontologies. Prior studies in psychology found
(Fig. 1, view 3).
that it is generally hard for humans to formulate correct
By default, all ontology axioms are viewed as potential
logical descriptions [1]. More recent studies indicate that this
sources of fault. However, the ontology author can move an
observation holds also in the context of ontology development,
axiom from the Possibly Faulty Axioms to the Correct Axioms
and that available ontology editing tools lack appropriate
(Fig. 1, view 6), if the axiom is assumed or known to be
support for quality assurance [2], [3].
correct, and the plug-in should never consider it as faulty.
Test-driven ontology development is a paradigm that has
The Start/Restart button triggers the verification of all test
its roots in the best-practices of software development. The
cases. If any test case is violated, the plug-in starts a debugging
main idea is to enable ontology authors to specify test cases
session automatically.
to ensure the correctness of the ontology with respect to
b) Debugging of Ontologies: OntoDebug implements
the intended meaning. Each test case is represented as an
several ontology debugging algorithms [5], [6], [7], [8], which
ontology axiom, and describes some required or unwanted
enable efficient localization of faulty axioms responsible for
logical consequence of the ontology.
test case failures. The result of running the debugging al-
Rector et al. [4] identified several unintended consequences
gorithm is shown in the Possible Ontology Repairs window
in a study of SNOMED CT, for example, “feet are a part
(Fig. 1, view 4). An ontology repair from the displayed repair
of pelvis”, or “diabetes is a disease of the abdomen”. Such
list is a set of axioms that should be changed in order to
unintended consequences, rooted in faulty modeling, can be
make all test cases hold. The debugging algorithms use the DL
prevented by using a test-driven development approach. For
reasoner extensively—for example, to check the consistency of
the first example (“feet are a part of pelvis”), a developer may
an ontology, or to compute the list of inferred axioms. As the
add test cases ensuring that different body regions are disjoint
performance of reasoners may vary for different ontologies,
This work was partially supported by the Carinthian Science Fund (contract
KWF-3520/26767/38701). 1 See http://isbi.aau.at/ontodebug for the source code and documentation.
ICBO 2018 August 7-10, 2018 1
Proceedings of the 9th International Conference on Biological Ontology (ICBO 2018), Corvallis, Oregon, USA 2
we implemented reasoner-independent debugging techniques
that can use any reasoner available in Protégé, such as Pellet
[9] or Hermit [10].
c) Interactive Debugging: The defined test cases are
often insufficient to localize the faulty axioms in the ontology.
In such a case, the debugger might return several alternative
repairs. OntoDebug will help the ontology author find the
optimal repair by asking the user a series of questions in
an interactive debugging session [11], [12]. The questions are
generated automatically and ask the user whether some axioms
are logical consequences of the intended ontology or not. For
example, in Fig. 1 view 1, the plug-in asks the ontology author
if “KoalaWithPhD is a Koala”, and if “KoalaWithPhD is a
Person”. The user responds affirmatively to the first question,
and negatively to the second. The plug-in will then add the
user’s answers to the list of Acquired Test Cases (Fig. 1, view
2), and it will refine the set of repairs by removing those that
violate the newly acquired answers. The interactive process
continues until the developer finds a satisfactory repair.
Fig. 2. Repair interface of the OntoDebug plug-in
proper parameters are selected, the performance improvements
can be significant.
III. C ONCLUSIONS
In this demo, we will showcase the OntoDebug plug-in
with several use cases. We will demonstrate the different
capabilities of the tool, and walk the audience through a
complete debug and repair session.
R EFERENCES
[1] P. N. Johnson-Laird, “Deductive reasoning,” Annu Rev Psychol, vol. 50,
pp. 109–135, 1999.
[2] M. Vigo, S. Bail, C. Jay, and R. Stevens, “Overcoming the pitfalls of
ontology authoring: Strategies and implications for tool design,” Int. J.
Hum.-Comput. Stud., vol. 72, no. 12, pp. 835–845, 2014.
[3] A. Rector, N. Drummond, M. Horridge, J. Rogers, H. Knublauch,
R. Stevens, H. Wang, and C. Wroe, “OWL Pizzas: Practical Experience
of Teaching OWL-DL: Common Errors & Common Patterns,” in
EKAW, pp. 63–81, 2004.
[4] A. Rector, S. Brandt, and T. Schneider, “Getting the foot out of the
pelvis: modeling problems affecting use of SNOMED CT hierarchies
Fig. 1. Interactive ontology debugging session in OntoDebug in practical applications,” JAMIA, vol. 18, no. 4, pp. 432–440, 2011.
[5] M. Horridge, B. Parsia, and U. Sattler, “Laconic and Precise
Justifications in OWL,” in ISWC, pp. 323–338, 2008.
d) Ontology Repair: After identifying the right repair [6] K. M. Shchekotykhin, G. Friedrich, P. Rodler, and P. Fleiss, “Sequential
(i.e., the faulty axioms), the ontology author can do a dry run diagnosis of high cardinality faults in knowledge-bases by direct
of various modifications of these axioms on a copy of the diagnosis generation,” in ECAI, pp. 813–818, 2014.
[7] K. M. Shchekotykhin, D. Jannach, and T. Schmitz, “Mergexplain:
ontology. OntoDebug automatically tests the altered axiom(s) Fast computation of multiple conflicts for diagnosis,” in IJCAI, pp.
against the specified test cases, and reports if, which and why 3221–3228, 2015.
problems persist (Fig. 2). Once all test cases hold and the [8] P. Rodler, W. Schmid, and K. Schekotihin, “Inexpensive cost-optimized
measurement proposal for sequential model-based diagnosis,” in DX
ontology author is satisfied with the performed amendments, Workshop, pp. 200–218, 2018.
the changes can be applied to the original ontology. [9] E. Sirin, B. Parsia, B. C. Grau, A. Kalyanpur, and Y. Katz, “Pellet: A
e) Versatile Parametrization and Customization: The practical OWL-DL reasoner,” J. Web Sem., vol. 5, pp. 51–53, 2007.
[10] B. Motik, R. Shearer, and I. Horrocks, “Hypertableau Reasoning for
performance of testing and diagnosis algorithms depends on Description Logics,” JAIR, vol. 36, pp. 165–228, 2009.
multiple factors, such as the reasoning complexity of the [11] K. M. Shchekotykhin, G. Friedrich, P. Fleiss, and P. Rodler,
developed ontology, the number of faulty axioms, and so on. “Interactive ontology debugging: Two query strategies for efficient fault
localization,” J. Web Sem., vol. 12, pp. 88–103, 2012.
OntoDebug allows the ontology author to control the working [12] P. Rodler, “Interactive Debugging of Knowledge Bases,” Ph.D.
of the testing and debugging algorithms by customizing a thesis, Alpen-Adria Universität Klagenfurt, 2015. [Online]. Available:
detailed set of parameters in a Preferences window. If the http://arxiv.org/pdf/1605.05950v1.pdf
ICBO 2018 August 7-10, 2018 2