Ontology testing can help guarantee ontology quality, by finding errors and inconsistencies in the ontology. There are few approaches for ontology testing inspired in methods employed by software engineering, and usually they depart from software requirements. Requirements in ontologies are the set of competency questions. A drawback of some ontology testing approaches resides on the fact that testing takes place only at the assertional level, since the query language SPARQL is relied on. In this work, we propose an approach and its implementation to test semi-automatically OWL-DL ontologies with both assertional and terminological queries, by using competency questions and the concept of unit testing. The tests accomplished in small ontologies sounds promising with good results.
Bearing this in mind, in this work we propose an approach and a tool to test
semiautomatically OWL-DL ontologies with both ABox and TBox queries, by using
competency questions and checking whether they are being met or not. We rely on the idea of
unit testing, which aims to testing the individual units of a code in Software Engineering
[
This paper is organized as follows: section 2 describes presents the proposed method; section 3 presents a tool that implements the method, section 4 shows preliminary results, section 5 is devoted to related work; and section 6 presents some conclusions and future work.
To introduce our ontology testing approach with DL, we start by presenting an ontology formalization: Definition 1. Vocabulary of Ontology. Voc(O) is the vocabulary, i.e., defined and used in ontology O, defined over the triple (NC ; NR; NO) as explained before. We denote by O j= if is true in all the possible models of O.
Definition 2. Competency Questions. A competency question is a hQ; i such that Q is a query expressed in a formal language and is an answer to this query expressed as a variable substitution.
Definition 3. Satisfied competency question. A competency question hQ; i is satisfied by an ontology O if O [ Q j= .
Since we are dealing with controlled natural language, we are dealing with some patterns of CQs that we have found, below some of them: Pattern: Which + <class> + <property> + <class> + not + <property> + <class>? Pattern: From which + <property> + <class>? Pattern: Is + <individual>+ a +<class>? Pattern: Is <class> + <individual> + or + <individual>?
In Software Engineering, the phase of testing is very important to guarantee software quality, since it aims at detecting errors, and, to check if the software was implemented in accordance with the requirements and user expectations. The motivation of this work is applying the very same idea in the context of Ontology Engineering.
Analogously, CQs represent user demands for knowledge regarding a domain of discourse. They usually enable developers to define the ontology. Given that domains of discourse and user requirements may change through time, ontologies should evolve in order to represent them accordingly. Thus, it is important that a permanent correspondence among the users goals and the ontology requirements exists. This must be accomplished using a method of systematic testing.
In this paper, we propose description logic ontology testing, relying on competency questions as the units to test single requirements. Following, we formalize the components of an ontology testing approach.
Definition 4. Test case, test plan. A Test case is represented by a CQ that is represented as already defined pair hQ; i. A Test Plan contains a set of test cases, besides other information like author and date. A Test Plan is represented by fhQ; igin=1,where n is the number of CQs.
Definition 5. Test suite and execution result. A Test Suite is a subset of Test cases of a Test Plan, i.e, if ts is a Test Suite, and tp is its Test Plan then ts tp. A test suite is important to reuse a set of test cases between test executions. A Test Execution result is an instance of a test suite that corresponds a set of test cases selected to be tested, the result of the tests, the author the performed the test, and date. The execution result is represented by the tuple < fhQ; iin=1g; fhQ; 0 iin=1gauthor; date >, where is the expected result, and 0 is the result of the test for each CQ of the suite, whereas the suite has n CQs.
Definition 6. Base Ontology and Testing Ontology. Given an ontology O, we assume that exists (defined by the developer or by the test engineer) a small set of O that represents a base set Ob, so that Ob Pa2setofaxiomsfromO a [ Pi2setofinstancesofO i ,on the other hand, we assumed that the complement of Ob is Ot, which is the part the needs to be tested, such that O = Ob [ Ot, e Ob \ Ot = ;.
Definition 7. Representative Testing Ontology. Given an ontology O, a representative testing ontology Ot’ is a subset of axioms and/or a set of representative instances from Ot, defined by the test engineer as Ot0 Pa2setofaxioms a [ Pi2setofaxioms i.
These definitions rely in the initial knowledge contained in the ontology, i.e. the
base ontology, which requires no competency questions. An example of such type of
ontology can be upper-domain ones, like BTL2 [
With the aid of this tool, a user can manage test plans, test suites, test cases and execute test suites through the modules of the layer. First, it is required to create a test plan that contains test cases. As put before, we consider here a test case as a CQ. Figure 1 displays our testing method steps. Considering the Pizza ontology, a test case could be “From which nation is the American Pizza”, “America”, where “From which nation is the American Pizza” is the small piece from the ontology to be tested, while “America” is the expected result from the test. In the test execution, besides the test case there is a field that indicates if the test case passes or not.
In a test execution module, the user chooses which test suite will be executed and the ontology to be tested. The execution is performed by the CQChecker, a tool to support the automation of verifying CQs against OWL ontologies. It provides a mechanism to verify whether the ontology meets its corresponding CQs. CQChecker supports both assertional and terminological queries.
CQChecker module [
Consider the CQ “What is the base of Real Italian Pizza?” about the Pizza ontology. With it and the CQChecker, we want to check if there is any class that would provide an individual for the image of RealItalianPizza, through the relation hasBase, which is a PizzaBase. For this to be achieved, we first search for an object property which can be a verb or a noun, and afterwards we look for the class, which has an image that is a PizzaBase in the relation, in this case ThinAndCrispyBase.
As a validation, we performed tests with three ontologies: Pizza, Travel and Wine ontologies, which are available on the Prote´ge´ website. We created a test plan with 12 test cases and 3 test suites. The preliminaries experiments were preformed by the authors.
The CQs are related to several constructs of OWL-DL like class hierarchies, individuals, disjoint classes, intersection (A \ B) and union of classes (A [ B); equivalent classes(A B), universal (8), existential quantification(9) and “has-value” restrictions; and cardinality restrictions.
We performed a test execution in each of them. Each test suite contains the test result. The tool deploys the CQ set, so that the user can select CQs to test. The user can also compare the answer of the test with the expected answer, thus checking if the test passed.
The first measured used is accuracy that is % of execution test correctly answered by the tool. For the Pizza test we get 90% of accuracy. For the Travel test we get 80% of accuracy, and for the Wine test we get 90% of accuracy.
We show now a demonstration of our approach with the pizza ontology. First we create the test cases to the test plan and two test suites available in https://www.dropbox.com/sh/ec6dwv83anqp64s/ AAB-7VgJytljII9f5jqpgw2sa?dl=0. The test suite number 1 is showed below:
CQ1: What is the spiciness of a chicken Topping? Correct answer: Mild CQ2: What is the base of Real Italian Pizza? Correct answer: ThinAndCrispyBase CQ3: What is the country of Origin of American Pizza? Correct answer: American CQ4: Which are the pizzas disjoint of Vegetarian Pizza? Correct answer: NonVegetarianPizza
Considering an user named joseph that takes this test suite to do a test execution on 08/10/2016. The goal is to verify if the ontology follows this set of CQs.
So, Joseph load the pizza ontology, the test suite number 1, and test each CQ. The tool returned an answer that can be equal or not the correct answer. If the the returned answer is equal to correct answer then, Joseph will check this CQ as correct. In the end, the tool will save the results from the table, moreover who did and date, in this case, Joseph and the date 08/10/2016.
Few engineering methodologies cite how, for what purpose, and by which means the
ontology engineer has to use CQs. However, there are few proposals of test ontologies
approaches. [
[
In relation to these works, ours deals at both assertional and terminological levels.
Moreover, while [
In this paper, we proposed a new method to ontology testing based on competency questions for OWL-DL ontologies, for this, we use the concept of unit testing. Furthermore, we proposed a tool that implements this method. Ontology testing helps to guarantee the quality of ontologies, by detecting errors and inconsistences in the ontology. Competency questions can be used both in requirements specification as in evaluation phase.
There are some limitations of our approach, for instance, only treats simple English sentences by identifying key words. It is necessary to build a mechanism to accept complex sentences like “Does a bouquet or body of a specific wine change with vintage?”.
For future work, solving these deficiencies is certainly the main task. However, in general, we believe that our approach and the tool can make the job of to test description logic ontologies efficiently and rapidly. And, we intend in the future to make experiments with other users outside of the project.