<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>ADVANCES IN ONTOLOGIES</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Editors</string-name>
        </contrib>
      </contrib-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>The Australasian Ontology Workshop series was initiated in 2005, and AOW
2012 is the eighth in the series. In 2012, AOW was held on the 4th of December
2012 in Sydney, Australia. Like most of the previous events AOW 2012 was held
as a workshop of the Australasian Joint Conference on Artificial Intelligence
celebrating its 25th anniversary as AI2012.</p>
      <p>Out of papers submitted, we accepted 5 full papers and 4 short papers on the
basis of three or four reviews submitted by our Program Committee of
international standing. The submissions covered an interesting balance of topics
with papers on fundamental research in ontologies, to ontology applications. We
were pleased to note that we again attracted international authors.
As in previous years, an award of $250 AUD was made available for the best
paper, sponsored this year by CAIR 1 (the Centre for Artificial Intelligence
Research in South Africa). In 2012 the best paper prize was awarded to Giovanni
Casini and Alessandro Mosca for their paper "Defeasible reasoning in ORM2".
AOW 2012 was the last AOW in its current form. From 2013 AOW will be
replaced by the Australasian Semantic Web Conference (ASWC). The 12th
International Semantic Web Conference (ISWC) and the 1st Australasian
Semantic Web Conference (ASWC) will be held 21-25 October 2013 in Sydney,
Australia and from 2014 onwards, ASWC will be a free-standing conference.
Many individuals contributed to this workshop. We thank our contributing authors
and dedicated international Program Committee for their careful reviews in a tight
time frame. We also thank CAIR for sponsoring the memory keys containing the
proceedings. We acknowledge the EasyChair conference management system,
which was used in all stages of the paper submission and review process and
also in the collection of the final camera-ready papers, as well as the Yola web
authoring system for our website available at http://aow2012.yolasite.com. We
hope that you found this eighth Australasian Ontology Workshop to be
informative, thought provoking, and most of all, enjoyable!
Kerry Taylor (CSIRO ICT Centre, Australia) (co-chair)
Aurona J. Gerber (CAIR, South Africa) (co-chair)
Tommie Meyer (CAIR, South Africa) (co-chair)
Mehmet A. Orgun (Macquarie University, Australia) (co-chair)</p>
    </sec>
    <sec id="sec-2">
      <title>Organisers of AOW 2012 December 2012</title>
      <p> 
 
                                                                                                              
1  http://www.cair.za.net/  
Conference Organisation</p>
      <sec id="sec-2-1">
        <title>Programme Chairs</title>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>Kerry Taylor (CSIRO ICT Centre, Australia) Aurona J. Gerber (CAIR – Centre for Artificial Intelligence Research, South Africa) Tommie Meyer (CAIR – Centre for Artificial Intelligence Research, South Africa) Mehmet A. Orgun (Macquarie University, Australia)</title>
      <sec id="sec-3-1">
        <title>Programme Committee</title>
        <p> 
AOW 2012 Accepted Papers</p>
        <p>Defeasible reasoning in ORM2.</p>
        <p>OntoMerge: A System for Merging DL-Lite Ontologies.
Lexichographic Closure for Defeasible Description Logics.
A normal form for hypergraph-based module extraction for
SROIQ.</p>
        <p>Two Case Studies of Ontology Validation.</p>
        <p>Non-Taxonomic Concept Addition to Ontologies.</p>
        <p>Deep Semantics in the Geosciences: semantic building blocks for
a complete geoscience infrastructure.</p>
        <p>Assessing Procedural Knowledge in Open-ended Questions
through Semantic Web Ontologies.</p>
        <p>Using Formal Ontologies in the Development of
Countermeasures for Military Aircraft.
p.4
p.16
p.28
p.40
p.52
p.64
p.74
p.86
p.98</p>
        <sec id="sec-3-1-1">
          <title>Defeasible reasoning in ORM2</title>
          <p>Giovanni Casini1 and Alessandro Mosca2
1 Centre for Artificial Intelligence Research, CSIR Meraka Institute and UKZN, South Africa</p>
          <p>Email: GCasini@csir.co.za
2 Free University of Bozen-Bolzano, Faculty of Computer Science, Italy</p>
          <p>
            Email: mosca@inf.unibz.it
1 Introduction
ORM2 (‘Object Role Modelling 2’) is a graphical fact-oriented approach for modelling,
transforming, and querying business domain information, which allows for a
verbalisation in a language readily understandable by non-technical users [
            <xref ref-type="bibr" rid="ref1">1</xref>
            ]. ORM2 is at
the core of the OGM standard SBVR language (‘Semantics of Business Vocabulary
and Business Rules’), and of conceptual modelling language for database design in
Microsoft Visual Studio (VS). In particular, the Neumont ORM Architect (NORMA) tool
is an open source plug-in to VS providing the most complete support for the ORM2
notation.
          </p>
          <p>
            On the other hand, in the more general field of formal ontologies in the last years a
lot of attention has been dedicated to the implementations of forms of defeasible
reasoning, and various proposals, such as [
            <xref ref-type="bibr" rid="ref16 ref17 ref18 ref2 ref3 ref4 ref5 ref6 ref7 ref8">2,3,4,5,6,7,8</xref>
            ], have been made in order to integrate
nonmonotonic reasoning mechanisms into DLs.
          </p>
          <p>
            In what follows we propose an extension of ORM2 with two new formal constraints,
with the main aim of integrating a form of defeasible reasoning in the ORM2 schemas;
we explain how to translate such enriched ORM2 schemas into ALCQI knowledge
bases and how to use them to check the schema consistency and draw conclusions.
In particular, the paper presents a procedure to implement a particular construction in
nonmonotonic reasoning, i.e. Lehmann and Magidor’s Rational Closure (RC)[
            <xref ref-type="bibr" rid="ref19 ref9">9</xref>
            ], that is
known for being characterized by good logical properties and for giving back intuitive
results.
2
          </p>
          <p>
            Fact-oriented modelling in ORM2
‘Fact-oriented modelling’ began in the early Seventies as a conceptual modelling
approach that views the world in terms of simple facts about individuals and the roles they
play [
            <xref ref-type="bibr" rid="ref1">1</xref>
            ]. Facts are assertions that are taken to be true in the domain of interest about
objects playing certain roles (e.g. ‘Alice is enrolled in the Computer Science program’).
In ORM2 one has entities (e.g. a person or a car) and values (e.g. a character string or
a number). Moreover, entities and values are described in terms of the types they belong
to, where a type (e.g. House, Car) is a set of instances. Each entity in the domain of
interest is, therefore, an instance of a particular type. The roles played by the entities
in a given domain are introduced by means of logical predicates, and each predicate
has a given set of roles according to its arity. Each role is connected to exactly one
object type, indicating that the role is played only by the (possible) instances of that type
((e.g. TYPE(isBy.Student,Student)) - notice that, unlike ER, ORM2 makes no use of
‘attributes’). ORM2 also admits the possibility of making an object type out of a
relationship. Once a relation has been transformed into an object type, this last is called the
objectification of the relation.
          </p>
          <p>According to the ORM2 design procedure, after the specification of the relevant
object types (i.e. entity and value types) and predicates, the static constraints must be
considered. The rest of this section is devoted to an informal introduction of the
constraint graphical representation, together with their intended semantics. Fig. 1 shows an
example of an ORM2 conceptual schema modelling the ‘academic domain’ (where the
soft rectangles are entity types, the dashed soft rectangles are value types, and the
sequences of one or more role-boxes are predicates). The example is not complete w.r.t.
the set of all the ORM2 constraints but it aims at giving the feeling of the expressive
power of the language. The following are among the constraints included in the schema
(the syntax we devised for linearizing them is in square brackets):
1. Subtyping (depicted as thick solid and dashed arrows) representing ‘is-a’
relationships among types. A partition, made of a combination of an
exclusive constraint (a circled ‘X’ saying that ‘Research&amp;TeachingStaff,
Admin, Student are mutually disjoint’), and a total constraint (a circled dot for
‘Research&amp;TeachingStaff, Admin, Student completely cover their common
supertype’). [O-SETTot(fResearch&amp;TeachingStaff, Admin, Studentg,UNI-Personnel)]
2. An internal frequency occurrence saying that if an instance of Research&amp;TeachingStaff
plays the role of being lecturer in the relation isGivenBy, that instance can play the role
at most 4 times [FREQ(isGivenBy.Research&amp;TeachingStaff,(1,4))]. A frequency
occurrence may span over more than one role, and suitable frequency ranges can be specified. At
most one cardinalities (depicted as continuos bars) are special cases of frequency occurrence
called internal uniqueness constraints [e.g. FREQ(hasWeight.Course,(1,1))].
3. An external frequency occurrence applied to the roles played by Student and Course,
meaning that ‘Students are allowed to enrol in the same course at most twice’.
[FREQ(isIn.Course,isBy.Student,(1,2))]
4. An external uniqueness constraint between the role played by Course in isIn and
the role played by Date in wasOn, saying that ‘For each combination of Course
and Date, at most one Enrollment isIn that Course and wasOn that Date’.
[FREQ(isIn.Course,wasOn.Date,(1,1))]
5. A mandatory participation constraints (graphically represented by a dot),
among several other, saying that ‘Each Course isGivenBy at least one
instance of the Research&amp;TeachingStaff type’ (combinations of
mandatory and uniqueness translate into exaclty one cardinality constraints).
[MAND(isGivenBy.Research&amp;TeachingStaff,Research&amp;TeachingStaff)]
6. A disjunctive mandatory participation, called inclusive-or constraint (depicted as
a circled dot), linking the two roles played by the instances of
AreaManager meaning that ‘Each area manager either works in or heads (or both)’.
[MAND(fworksIn.AreaManager,heads.AreaManagerg,AreaManager)]
7. An object cardinality constraint forcing the number of the Admin instances to be less or
equal to 100 (role cardinality constraints, applied to role instances, are also part of ORM2).
[O-CARD(Admin)=(0,100)]
8. An object type value constraint indicating which values are allowed in Credit (role value
constraints can be also expressed to indicate which values are allowed to play a given role).
[V-VAL(Credit)=f4,6,8,12g]
9. An exclusion constraint (depicted as circled ‘X’) between the two roles played by the
instances of Student, expressing the fact that no student can play both these roles.
Exclusion constraint can also span over arbitrary sequences of roles. The combination of
exclusion and inclusive-or constraints gives rise to exclusive-or constraints meaning that
each instance in the attached entity type plays exactly one of the attached roles.
Exclusion constraints, together with subset and equality, are called set-comparison constraints.
[R-SETExc(worksFor.Student,collaborates.Student)]
10. A ring constraint expressing that the relation reportsTo is asymmetric.
[RINGAsym(reportTo.Admin,reportTo.AreaManager)]</p>
          <p>
            A comprehensive list of all the ORM2 constraints, together with their graphical
representation, can be found in [
            <xref ref-type="bibr" rid="ref1">1</xref>
            ].
3
          </p>
          <p>
            The ALCQI encoding of ORM2zero
With the main aim of relying on available tools to reason in an effective way on ORM2
schemas, an encoding in the description logic ALCQI for which tableaux-based
reasoning algorithms with a tractable computational complexity have been developed [
            <xref ref-type="bibr" rid="ref10 ref20">10</xref>
            ].
ALCQI corresponds to the basic DL ALC equipped with qualified cardinality
restrictions and inverse roles, and it is a fragment of the OWL2 web ontology language (a
complete introduction of the syntax and semantics of ALCQI can be found in [
            <xref ref-type="bibr" rid="ref11 ref21">11</xref>
            ]).
We also introduce in the ALCQI language the expression ‘C D’ as an abbreviation
for the expression ‘:C t D’.
          </p>
          <p>
            Now, the discrepancy between ORM2 and ALCQI poses two main obstacles that
need to be faced in order to provide the encoding. The first one, caused by the absence of
n-ary relations in ALCQI , is overcome by means of reification: for each relation R of
3
arity n 2, a new atomic concept AR and n functional roles (R:a1); : : : ; (R:an) are
introduced. The tree-model property of ALCQI guarantees the correctness encoding
w.r.t. the reasoning services over ORM2. Unfortunately, the second obstacle fixes,
once for all, the limits of the encoding: ALCQI does not admit neither arbitrary
set-comparison assertions on relations, nor external uniqueness or uniqueness involving
more than one role, or arbitrary frequency occurrence constraints. In other terms, it can
be proven that ALCQI is strictly contained in ORM2. The analysis of this inclusion
thus led to identification of the fragment called ORM2zero which is maximal with
respect to the expressiveness of ALCQI, and still expressive enough to capture the
most frequent usage patterns of the conceptual modelling community. Let ORM2zero =
fTYPE; FREQ ; MAND; R-SET ; O-SETIsa; O-SETTot; O-SETEx; OBJg be the
fragment of ORM2 where: (i) FREQ can only be applied to single roles, and (ii)
R-SET applies either to entire relations of the same arity or to two single roles. The
encoding of the semantics of ORM2zero shown in table 1 is based on the SALCQI
signature made of: (i) A set E1; E2; : : : ; En of concepts for entity types; (ii) a set
V1; V2; : : : ; Vm of concepts for value types; (iii) a set AR1 ; AR2 ; : : : ; ARk of concepts
for objectified n-ary relations; (iv) a set D1; D2; : : : ; Dl of concepts for domain
symbols; (v) 1; 2; : : : ; nmax + 1 roles. Additional background axioms are needed here
in order to: (i) force the interpretation of the ALCQI knowledge base to be correct
w.r.t. the corresponding ORM2 schema, and (ii) guarantee that that any model of the
resulting ALCQI can be ‘un-reified’ into a model of original ORM2zero schema.
The correctness of the introduced encoding is guaranteed by the following theorem
(whose complete proof is available at [
            <xref ref-type="bibr" rid="ref12 ref22">12</xref>
            ]):
Theorem 1. Let zero be an ORM2zero conceptual schema and ALCQI the ALCQI
KB constructed as described above. Then an object type O is consistent in zero if and
only if the corresponding concept O is satisfiable w.r.t. ALCQI .
          </p>
          <p>
            Let us conclude this section with some observation about the complexity of
reasoning on ORM2 conceptual schemas, and taking into account that all the reasoning tasks
for a conceptual schema can be reduced to object type consistency. Undecidability of the
ORM2 object type consistency problem can be proven by showing that arbitrary
combinations of subset constraints between n-ary relations and uniqueness constraints over
single roles are allowed [
            <xref ref-type="bibr" rid="ref13 ref23">13</xref>
            ]. As for ORM2zero, one can conclude that object type
consistency is EXPTIME-complete: the upper bound is established by reducing the ORM2zero
problem to concept satisfiability w.r.t. ALCQI KBs (which is known to be
EXPTIMEhard) [
            <xref ref-type="bibr" rid="ref14 ref24">14</xref>
            ], the lower bound by reducing concept satisfiability w.r.t. ALC KBs (which is
known to be EXPTIME-complete) to object consistency w.r.t. ORM2zero schemas [
            <xref ref-type="bibr" rid="ref15 ref25">15</xref>
            ].
Therefore, we obtain the following result:
Theorem 2. Reasoning over ORM2zero schemas is EXPTIME-complete.
4
          </p>
        </sec>
        <sec id="sec-3-1-2">
          <title>Rational Closure in ALCQI</title>
          <p>
            Now we briefly present the procedure to define the analogous of RC for the DL
language ALCQI. A more extensive presentation of such a procedure can be found in [
            <xref ref-type="bibr" rid="ref4">4</xref>
            ]:
it is defined for ALC, but it can be applied to ALCQI without any modifications. RC
is one of the main construction in the field of nonmonotonic logics, since it has a solid
4
Background domain axioms:
TYPE(R:a; O)
FREQ (R:a; hmin; maxi)
MAND(fR1:a1; : : : ; R1:an;
          </p>
          <p>: : : ; Rk:a1; : : : ; Rk:amg; O)
(A) R-SETSub(A; B)
(A) R-SETExc(A; B)
(B) R-SETSub(A; B)
(B) R-SETExc(A; B)
O-SETIsa(fO1; : : : ; Ong; O)
O-SETTot(fO1; : : : ; Ong; O)
O-SETEx(fO1; : : : ; Ong; O)
OBJ(R; O)
logical characterization, it generally maintains the same complexity level of the
underlying monotonic logic, and it does not give back counter-intuitive conclusions; its main
drawback is in its inferential weakness, since there could be desirable conclusions that
we won’t be able to draw (see example 2 below).</p>
          <p>As seen above, each ORM2zeroschema can be translated into an ALCQI TBox. A
TBox T for ALCQI consists of a finite set of general inclusion axioms (GCIs) of form
C v D, with C and D concepts. Now we introduce also a new form of information,
the defeaible inclusion axioms C &lt; D, that are read as ‘Typically, an individual falling
under the concept C falls also under the concept D’. We indicate with B the finite set of
such inclusion axioms.</p>
          <p>Example 1. Consider a modification of the classical ‘penguin example’, with the concepts
P; B; F; I; F i; W respectively read as ‘penguin’, ‘bird’, ‘flying’, ‘insect’, ‘fish’, and ‘have wings’,
and a role P rey, where a role instantiation (a; b):P rey read as ‘a preys for b’. We can define a
defeasible KB K = hT ; Bi with T = fP v B; I v :F ig and B = fP &lt; :F , B &lt; F ,
P &lt; 8P rey:F i; B &lt; 8P rey:I; B &lt; W g.</p>
          <p>
            In order to define the rational closure of a knowledge base hT ; Bi, we must first of
all transform the knowledge base hT ; Bi into a new knowledge base h ; i, s.t. while
T and B are sets of inclusion axioms, and are simply sets of concepts. Then, we
shall use the sets h ; i to define a nonmonotonic consequence relation that models the
rational closure. Here we just present the procedure, referring to [
            <xref ref-type="bibr" rid="ref4">4</xref>
            ] for a more in-depth
explanation of the various steps.
          </p>
          <p>Transformation of hT ; Bi into h ;
steps.</p>
          <p>i. Starting with hT ; Bi, we apply the following
Step 1. Define the set representing the strict form of the set B, i.e. the set Bv = fC v D j
C &lt; D 2 Bg, and define a set AB as the set of the antecedents of the conditionals in B,
i.e. AB = fC j C &lt; D 2 Bg.</p>
          <p>Step 2. We determine an exceptionality ranking of the sequents in B using the set of the
antecedents AB and the set Bv.</p>
          <p>Step 2.1. A concept is considered exceptional in a knowledge base hT ; Bi only if it is
classically negated (i.e. we are forced to consider it empty), that is, C is exceptional in hT ; Bi
only if</p>
          <p>T [ Bv j= &gt; v :C
where j= is the classical consequence relation associated to ALCQI. If a concept is
considered exceptional in hT ; Bi, also all the defeasible inclusion axioms in B that have
such a concept as antecedent are considered exceptional. So, given a knowledge base
hT ; Bi we can check which of the concepts in AB are exceptional (we indicate the set
containing them as E(AB)), and consequently which of the axioms in B are exceptional
(the set E(B) = fC &lt; D j C 2 E(AB)g).</p>
          <p>So, given a knowledge base hT ; Bi we can construct iteratively a sequence E0; E1; : : : of
subsets of B in the following way:
– E0 = B
– Ei+1 = E(Ei)
Since B is a finite set, the construction will terminate with an empty set (En = ; for
some n) or a fixed point of E.</p>
          <p>Step 2.2 Using such a sequence, we can define a ranking function r that associates to every
axiom in B a number, representing its level of exceptionality:
r(C &lt; D) = i if C &lt; D 2 Ei and C &lt; D 2= Ei+1</p>
          <p>1 if C &lt; D 2 Ei for every i :
Here we shall assume that every concept has a finite ranking value, and we shall deal with the
possible occurrence of some concept with 1 as ranking value in the following section.
Step 3. Now we build a new formalization of the information contained in the knowledge base
hT ; Bi, translating each of the two sets of axioms into two sets of concepts, and
respectively. The set will simply correspond to the materialization of the inclusion axioms,
i.e. the concepts translating the axioms.</p>
          <p>= fC D j C v D 2 T g
In order to define the set , given the rank value of the sequents in B, we construct a set of
default concepts = f 0; : : : ; ng (with n the highest rank-value in B), with
i = lfC</p>
          <p>D j C &lt; D 2 B and r(C &lt; D)
ig :
Hence we substitute the conceptual system hT ; Bi with the pair h ; i, where and
are sets of concepts, the former containing concepts to be considered valid for every
individual of the domain, the latter containing concepts to be considered defeasibly valid,
i.e. we apply such default concepts to an individual only if they are consistent with the
information in our knowledge base. It is not difficult to see that the concepts in are
linearly ordered by j=, that is, for every i, 0 i &lt; n 1, j= i v i+1.
Rational Closure. Consider now = fC1 D1; : : : ; Cm Dmg and =
f 0; : : : ; ng. We define a nonmonotonic consequence relation between the concepts
j h ; i that determines what presumably follows from a finite set of concepts .
Simply, a concept D is a defeasible consequence of if it classically follows from , the
strict information contained in the knowledge base (i.e. ), and the first default concept
i that in the sequence h 0; : : : ; ni results classically consistent with the rest of the
premises.</p>
          <p>6</p>
          <p>D, where i is the first (
)Definition 1. j h ; iD iff j= d u d u i v
consistent formula 3 of the sequence h 0; : : : ; ni.</p>
          <p>
            You can find in [
            <xref ref-type="bibr" rid="ref4">4</xref>
            ] an explanation of why the above procedure for DL corresponds to
the rational closure defined by Lehmann and Magidor for propositional languages, and
satisfies the DL translation of the basic properties characterizing rational consequence
relations.
          </p>
          <p>
            Proposition 1 ([
            <xref ref-type="bibr" rid="ref4">4</xref>
            ], Proposition 4). j h ; i is a consequence relation containing K =
hT ; Bi and satisfying the properties of the rational consequence relations.
          </p>
          <p>Moreover, as deciding entailment in ALCQI is EXPTIME-complete (see Theorem
2), and since the decidability problem for the rational closure is reducible to a finite
number of decision w.r.t. the classical ALCQI consequence relation, we obtain immediately
that
Proposition 2. Deciding Cj hTe; eiD in ALCQI is an EXPTIME-complete problem.
Example 2. Consider the KB of Example 1. Hence, we start with K = hT ; Bi. The strict form
of B is Bv = fP v :F , B v F , P v 8P rey:F i; B v 8P rey:I; B v W g, with AB =
fP; Bg. Following the procedure at Step 2, we obtain the exceptionality ranking of the sequents:
E0 = fP v :F , B v F , P v 8P rey:F i; B v 8P rey:I; B v W g; E1 = fP v :F , P v
8P rey:F ig; E2 = ;. Automatically, we have the ranking values of every sequent in B: namely,
r(B v F ) = r(B v 8P rey:I) = r(B v W ) = 0; r(P &lt; :F ) = r(P &lt; 8P rey:F i) = 1.
From such a ranking, we obtain a set of default concepts = f 0; 1g, with
0 = (B
1 = (P</p>
          <p>F ) u (B
:F ) u (P
8P rey:I) u (P
8P rey:F i) :
:F ) u (P
8P rey:F i) u (B</p>
          <p>W )
Now, referring to definition 1, we can derive a series of desirable conclusions, as :F j :B, ,
B ^ greenj F , P ^ blackj :F , P j 8P rey::I. Instead, other counterintuitive connections are
not valid, such as B ^ :F j P , B ^ :F j :P , or P j F . Here we can notice the main weakness of
the Rational Closure: even if it would be intuitive to conclude that penguins have wings (P j W ),
we cannot conclude that a class that results atypical (as penguins) cannot inherit any of the typical
properties of its superclasses (as having wings), even if such properties are not logically connected
to the ones that determine the exceptionality (not flying and eating fish).
5</p>
          <p>Defeasible constraints for ORM2
As seen above, in order to introduce defeasible reasoning in DL we introduce the
defeasible inclusion axiom C &lt; D, indicating that the elements of the concept C
typically, but not necessarily, are elements of the concept D. We want to introduce in the
ORM2zeroschemas constraints playing an analogous role, i.e. representing defeasible
constraints in the ontological organization of a particular domain. With this goal in mind,
two constraints aimed at representing forms of defeasible constraints between classes,
and classes and their properties, are introduced.</p>
          <p>– A defeasible subclass relation: we introduce an arrow ‘;’, where ‘C ; D’
indicates that each element of the class C is also an element of the class D, if not
informed of the contrary.
3 That is, 6j= d
u d
v : i.
Example 3 (Defeasible subclass relation). Consider figure 2. The schema on the left
represents in ORM2 the classic penguin example: penguins are birds and do not fly (the class
Penguin is a subclass, respectively, of the classes Birds and Non-FlyingObj), while birds
fly and have wings (the class Birds is a subclass, respectively, of the classes FlyingObj and
WingyObj). The translation procedure into ALCQI gives back the TBox T in table 2. From
T we can derive that the schema is inconsistent, since we have T j= :Penguin, i.e. the
concept Penguin must be empty. We can modify the knowledge base introducing defeasible
information, in particular stating that birds typically fly and typically have wings, and penguins
typically do not fly. In this way we obtain the schema on the right, and in ALCQI we obtain a
set B = fBird &lt; WingyObject; Bird &lt; FlyingObject; Penguin &lt; NonFlyingObjectg,
substituting the corresponding classical inclusion axioms in the TBox.
– A defeasible mandatory participation: we introduce a new mandatory participation
constraint ‘ ’, to use instead of the classically mandatory constraint ‘ ’. If the
connection between a class C and a relation R is constrained by a constraint , we
read it as ‘each element of the class C participates to the relation R, if we are not
informed of the contrary’.</p>
          <p>Example 4 (Defeasible mandatory participation). Consider figure 3. The schema
represents the organization of a firm: the class Manager is a subclass of the class Employee, and
every employee must work for a project. while every project must have at least an employee
working on it. The class Manager is partitioned into AreaManager and TopManager. Each
top manager mandatorily manages a project. The translation procedure into ALCQI of the
left version of the schema gives back the TBox T in table 3. Since managing and working for
a project are not compatible roles, T implies that the class TopManager is empty, since a top
manager would manage and would work for a project at the same time. Instead, if we declare
that typically an employee works for a project, we can consider the top managers as
exceptional kind of employees; hence we substitute the mandatory constraint between Employee
and WorkFor with a defeasible constraint (i.e. the schema on the right in figure 3); from such
8
WorksFor v 9f1 :Employee; WorksFor v 9f2 :Project
Manages v 9f1 :TopManager; Manages v 9f2 :Project
9f1 :Manages v = 1 f1 :Manages
Employee v 9f1 :WorksFor
TopManager v 9f1 :Manages
Project v 9f2 :WorksFor
Project v 9f2 :Manages
9f1 :WorksFor v A&gt;2 u :9f1:Manages
Manager v Employee u (AreaManager t TopeManager)</p>
          <p>AreaManager v :TopeManager
a change we obtain a knowledge base as the one above, but with the defeasible inclusion
axiom Employee &lt; 9f1 :WorksFor instead of the axiom Employee v 9f1 :WorksFor.</p>
          <p>Introducing such constraints, we introduce the forms of defeasible subsumptions
appropriate for modeling nonmonotonic reasoning. In particular:
– A subclass relation, as the ones in example 3, is translated into an inclusion axiom
C v D, and correspondingly we translate the defeasible connection C ; D into a
defeasible inclusion axiom C &lt; D.
– Analogously, consider the strict form of the example 4. The mandatory participation
of the class B to the role AN is translated into the axiom B v 9f1 :AN. If we use
the defeasible mandatory participation constraint, we simply translate the structure
using the defeasible inclusion &lt;, obtaining the axiom B &lt; 9f1 :AN.</p>
          <p>9</p>
          <p>Hence, from a ORM graph with defeasible constraints we obtain an ALCQI
knowledge base K = hT ; Bi, where T is a standard ALCQI Tbox containing concept
inclusion axioms C v D, while the set B contains defeasible axioms of the form C &lt; D.
Once we have our knowledge base K, we apply to it the procedure presented in the
previous section, in order to obtain the rational closure of the knowledge base.</p>
          <p>Consistency. In ORM2, and in conceptual modeling languages in general, the notion
of consistency is slightly different from the classical form of logical consistency. That
is, generally from a logical point of view a knowledge base K is considered inconsistent
only if we can classically derive a contradiction from it; in DLs that corresponds to
saying that K j= &gt; v ?, i.e. every concept in the knowledge base results empty. Instead,
dealing with conceptual modeling schemas we generally desire that our model satisfies a
stronger form of consistency constraint, that is, we want that none of the classes present
in the schema are forced to be empty.</p>
          <p>Definition 2 (Strong consistency). A TBox T is strongly consistent if none of the
atomic concepts present in its axioms are forced to be empty, that is, if T 6j= &gt; v :A
for every atomic concept A appearing in the inclusion axioms in T .</p>
          <p>As seen above, the introduction of defeasible constraints into ORM2zero allows to
build schemas that in the standard notation would be considered inconsistent, but that,
once introducing the defeasible constraints, allow for an instantiation such that all the
classes result non-empty. Hence it is necessary to redefine the notion of consistency
check in order to deal with such situations.</p>
          <p>Such a consistency check is not problematic, since we can rely on the ranking
procedure presented above. Consider a TBox T obtained by an ORM2zero schema, and
indicate with C the set of all the atomic concepts used in T . It is sufficient to check the
exceptionality ranking of all the concepts in C with respect to T : if a concept C has
an exceptionality ranking r(C) = n, with 0 &lt; n &lt; 1, then it represents an atypical
situation, an exception, but that is compatible with the information conveyed by the
defeasible inclusion axioms. For example, in the above examples the penguins and the top
managers would be empty classes in the classical formalization, but using the
defeasible approach they result exceptional classes in our schemas, and we can consider them
as non-empty classes while still considering the schema as consistent. The only case
in which a class has to be considered necessarily empty, is when it has 1 as ranking
value: that means that, despite eliminating all the defeasible connections we can, such
a concept still results empty. Then, the notion of strong consistency for ORM2zero with
defeasible constraints is the following:
Definition 3 (Strong consistency with defeasible constraints). A knowledge base K =
hT ; Bi is strongly consistent if none of the atomic concepts present in its axioms are
forced to be empty, that is, if r(A) 6= 1 for every atomic concept A appearing in the
inclusion axioms in K.</p>
          <p>Given a defeasible ORM2zeroschema , eliminate from it all the defeasible
constraints (call strict the resulting schema). From the procedures defined above, it is
immediate to see that if strict is a strongly inconsistent ORM2zeroschema, in the
‘classical’ sense, then is a strongly inconsistent defeasible schema: simply, if the negation
of a concept is forced by the strict part of a schema, it will be necessarily forced at each
ranking level, resulting in a ranking value of 1.</p>
          <p>10</p>
          <p>On the other hand, there can be also strongly inconsistent defeasible schemas in
which inconsistency depends not only on the strict part of the schema, but also on the
defeasible part. For example, the schema in figure 4 is inconsistent, since the class A
results to have a ranking value of 1 (the schema declares that the class A is directly
connected to two incompatible concepts). Now, we can check the results of the defined
procedure in the examples presented.</p>
          <p>Example 5. Consider example 3. From the translation of the defeasible form of the schema we
conclude that the axiom Penguin v NonFlyingObject has rank 1, while Bird v WingyObject
and Bird v FlyingObject have rank 0, that means that we end up with two default concepts:
–
–
0 := dfPenguin
1 := Penguin</p>
          <p>NonFlyingObject; Bird
NonFlyingObject</p>
          <p>WingyObject; Bird</p>
          <p>FlyingObjectg;</p>
          <p>We can derive the same kind of conclusions as in example 2, and again we can see the limits of
the rational closure, since we cannot derive the desirable conclusion that Penguinj WingyObject.
Example 6. Consider the knowledge base obtained in the example 4. We have only a defeasible
inclusion axiom Employee &lt; 9f1 :WorksFor, and, since Employee does not turn out to be an
exceptional concept, we end up with a single default concept in B:
–
0 := fEmployee</p>
          <p>9f1 :WorksForg;</p>
          <p>Since TopManager is not consistent with all the strict information contained in the schema
plus 0, we cannot associate 0 to TopManager and, despite we have the information that for
non-exceptional cases an employee works for a project, we are not forced to conclude that for the
exceptional class of the top managers.
6</p>
          <p>Conclusions and further work
In this paper we have presented a way to implement a form of defeasible reasoning into
the ORM2 formalism. Exploiting the possibility of encoding ORM2zero, that represents a
big portion of the ORM2 language, into the description logic ALCQI on one hand, and
a procedure appropriate for modeling one of the main forms of nonmonotonic reasoning,
i.e. rational closure, into DLs on the other hand, we have defined two new constraints,
a defeasible subclass relation and a defeasible mandatory participation, that are
appropriate for modeling defeasible information into ORM2, and that, once translated into
ALCQI , allow for the use of the procedures characterizing rational closure to reason
about the information contained into an ORM2zero schema.</p>
          <p>
            The present proposal deals only with reasoning on the information contained in the
TBox obtained from an ORM2 schema, but, once we have done the rational closure of
the TBox, we can think also of introducing an ABox, that is, the information about a
particular domain of individuals. A first proposal in such direction is in [
            <xref ref-type="bibr" rid="ref4">4</xref>
            ]. Actually we
still lack a complete semantic characterization of rational closure in DLs, but hopefully
we shall obtain soon such a result (a first step in such a direction is in [
            <xref ref-type="bibr" rid="ref3">3</xref>
            ]). Another future
step will be the implementation of nonmonotonic forms of reasoning that extend rational
closure, overcoming its inferential limits (see example 2), such as the lexicographic
closure [
            <xref ref-type="bibr" rid="ref26">16</xref>
            ] or the defeasible inheritance based approach [
            <xref ref-type="bibr" rid="ref5">5</xref>
            ].
          </p>
          <p>12
OntoMerge: A System for Merging DL-Lite</p>
          <p>Ontologies
Zhe Wang1, Kewen Wang2, Yifan Jin2, and Guilin Qi3;4
1 University of Oxford, United Kingdom
2 Gri th University, Australia
3 Southeast University, China
4 State Key Laboratory for Novel Software Technology</p>
          <p>Nanjing University, Nanjing, China
Abstract. Merging multi-sourced ontologies in a consistent manner is
an important and challenging research topic. In this paper, we propose
a novel approach for merging DL-LitebNool ontologies by adapting the
classical model-based belief merging approach, where the minimality of
changes is realised via a semantic notion, model distance. Instead of using
classical DL models, which may be in nite structures in general, we
de ne our merging operator based on a new semantic characterisation
for DL-Lite. We show that subclass relation w.r.t. the result of merging
can be checked e ciently via a QBF reduction. We present our system
OntoMerge, which e ectively answers subclass queries on the resulting
ontology of merging, without rst computing the merging results. Our
system can be used for answering subclass queries on multiple ontologies.
1</p>
          <p>
            Introduction
Ontologies are widely used for sharing and reasoning over domain knowledge,
and their underlying formalisms are often description logics (DLs). To e
ectively answer queries, ontologies from heterogeneous sources and contributed
by various authors are often needed. However, ontologies developed by multiple
authors under di erent settings may contain overlapping, con icting and
incoherent domain knowledge. The ultimate goal of ontology merging is to obtain
a single consistent ontology that preserves as much knowledge as possible from
two or more heterogeneous ontologies. This is in contrast to ontology matching
[
            <xref ref-type="bibr" rid="ref5">5</xref>
            ], whose goal is to align entities (with di erent name) between ontologies, and
which is often a pre-stage of ontology merging.
          </p>
          <p>
            Existing merging systems often adopt formula-based approaches to deal with
logical inconsistencies [10; 9; 14]. Most of such approaches can be described as
follows: the system rst combine the ontologies by taking their union; then, if any
inconsistency is detected (through a standard reasoning), it pinpoints the axioms
which (may) cause inconsistency; and nally, remove certain axioms to retain
consistency. However, such an approach is sometimes unsatisfactory because it
is not ne-grained either in the way it measures the minimality of changes, and
thus it is often unclear how close the result of merging is to the source ontologies
semantically; or in the way it resolve inconsistency. In [
            <xref ref-type="bibr" rid="ref12 ref22">12</xref>
            ], an attempt is made
to provide some semantic justi cation for the minimality of changes, however,
the result of merging is still syntax-dependant and is often a set of ontologies.
          </p>
          <p>
            On the other hand, model-based merging operators have been intensively
studied in propositional logic, which are syntax-independent and usually satisfy
more rationality postulates than formula-based ones. However, a major
challenge in adapting model-based merging techniques to DLs is that DL models
are generally in nite structures and the number of models of a DL ontology is
in nite. Several notions of model distance are de ned on classical DL models
for ontology revision [
            <xref ref-type="bibr" rid="ref13 ref23">13</xref>
            ]. Mathematically, it is possible to de ne a distance o
classical DL models. Such a distance is computationally limited as it is unclear
how to develop an algorithm for the resulting merging operator. A desirable
solution is to de ne ontology merging operators based on a suitable nite semantic
characterisation instead of classical DL models.
          </p>
          <p>
            In this paper, we focus on merging ontologies expressed as DL-Lite TBoxes,
which can be also accompanied with the ontology-based data access (OBDA)
framework for data integration [
            <xref ref-type="bibr" rid="ref2">2</xref>
            ]. We propose a novel approach for merging
ontologies by adapting a classical model-based belief merging approach, where
the minimality of changes is realised via a semantic notion, model distance.
Instead of using classical DL models, which may be in nite structures in general,
we de ne our merging operator based on the notion of types. We show that
subclass relation w.r.t. the result of merging can be checked e ciently via a
QBF reduction, which allows us to make use of the o -the-shelf QBF solvers [
            <xref ref-type="bibr" rid="ref18 ref8">8</xref>
            ].
We present our system OntoMerge, which e ectively answers subclass queries
on merging results, without rst computing the merging results. Our system can
be used to answer subclass queries on multiple ontologies.
2
          </p>
          <p>
            A New Semantic Characterisation
[
            <xref ref-type="bibr" rid="ref1">1</xref>
            ]:
In our approach, it is su cient to consider a nite yet large enough signature.
A signature S is a union of four disjoint nite sets SC , SR, SI and SN , where
SC is the set of atomic concepts, SR is the set of atomic roles, SI is the set of
individual names and SN is the set of natural numbers in S. We assume 1 is
always in SN .
          </p>
          <p>Formally, given a signature S, a DL-LitebNool language has the following syntax
R</p>
          <p>P j P</p>
          <p>S</p>
          <p>P j :P</p>
          <p>B &gt; j A j &gt; n R C B j :C j C1 u C2
where n 2 SN , A 2 SC and P 2 SR. B is called a basic concept and C is called
a general concept. BS denotes the set of basic concepts on S. We write ? for
:&gt;, 9R for 1 R, and C1 t C2 for :(:C1 u :C2). Let R+ = P , where P 2 SR,
whenever R = P or R = P . A TBox T is a nite set of concept axioms of the
form C1 v C2, where C1 and C2 are general concepts. An ABox A is a nite set
of membership assertions of the form C(a) or S(a; b), where a; b are individual
names. In this paper, an ontology is represented as a DL TBox.</p>
          <p>The classical DL semantics are given by models. A TBox T is consistent
with an ABox A if T [ A has at least one model. A concept or role is satis able
in T if it has a non-empty interpretation in some model of T . A TBox T is
coherent if all atomic concepts and atomic roles in T are satis able. Note that
a coherent TBox must be consistent. TBox T entails an axiom C v D, written
T j= C v D, if all models of T satisfy C v D. Two TBoxes T1; T2 are equivalent,
written T1 T2, if they have the same models.</p>
          <p>Now, we introduce a semantic characterisation for DL-Lite TBoxes in terms
of types. A type BS is a set of basic concepts over S, such that &gt; 2 , and
&gt; n R 2 implies &gt; m R 2 for each pair m; n 2 SN with m &lt; n and each
(inverse) role R 2 SR [ f P j P 2 SR g. Type satis es basic concept B if
B 2 , :C if does not satisfy C, and C1 u C2 if satis es both C1 and C2.
Given a TBox T , type satis es T if satis es concept :C1 t C2 for each axiom
C1 v C2 in T .</p>
          <p>For a TBox T , de ne TM(T ) to be the maximal set of types satisfying
the following conditions: (1) all the types in TM(T ) satisfy T ; (2) for each
type 2 TM(T ) and each 9R in , there exists a type 0 2 TM(T ) (possibly
0 = ) containing 9R . A type is called a type model (T-model) of T if
2 TM(T ). Note that TM(T ) is uniquely de ned for each TBox T . Note that
for a coherent TBox T , TM(T ) is exactly the set of all types satisfying T . Let
TM( ) = TM(T1) TM(Tn) for = hT1; : : : ; Tni.</p>
          <p>Proposition 1. Given a TBox T , we have the following results:
{ T is consistent i TM(T ) 6= ;.
{ For a general concept C, C is satis able wrt T i there exists a T-model in</p>
          <p>TM(T ) satisfying C.
{ For two general concepts C; D, T j= C v D i either TM(T ) = ; or all</p>
          <p>T-models in TM(T ) satisfy C v D.
{ T T 0 i TM(T ) = TM(T 0), for any TBox T 0.</p>
          <p>Given a type , an individual a and an ABox A, we say is a type of a
w.r.t. A if there is a model I of A such that = fB j aI 2 BI ; B 2 BS g. For
example, given A = fA(a); :B(b); C(c)g, type = fA; Bg is a type of a, but
not a type of either b or c in A. For convenience, we will say a type of a when
the ABox A is clear from the context. Let TMa(A) be the set of all the types
of a in A if a occurs in A; and otherwise, TMa(A) be the set of all the types.
A set M of T-models satis es an ABox A if there is a type of a in M , i.e.,
M \ TMa(A) 6= ;, for each individual a in A.</p>
          <p>Proposition 2. Given a TBox T and an ABox A, T [ A is consistent i
TM(T ) \ TMa(A) 6= ; for each a in A.
3</p>
          <p>Merging Operator
In this section, we introduce an approach to merging DL-Lite ontologies to obtain
a coherent uni ed ontology.</p>
          <p>
            An ontology pro le is of the form = hT1; : : : ; Tni, where Ti is the ontology
from the source n.o. i (1 i n). There are two standard de nitions of integrity
constraints (ICs) in the classical belief change literature [
            <xref ref-type="bibr" rid="ref3">3</xref>
            ], the
consistencyand entailment-based de nitions. We also allow two types of ICs for merging,
namely the consistency constraint (CC), expressed as a set Ac of data, and the
entailment constraint (EC), expressed as a TBox Te. We assume the IC is
selfconsistent, that is, Te [ Ac is always consistent. For an ontology pro le , a
CC Ac and a EC Te, an ontology merging operator is a mapping ( ; Te; Ac) 7!
r( ; Te; Ac), where r( ; Te; Ac) is a TBox, s.t. r( ; Te; Ac)[Ac is consistent,
and r( ; Te; Ac) j= Te.
          </p>
          <p>
            In classical model-based merging approaches, merging operators are often
de ned by certain notions of model distances [11; 6]. We use S4S0 to denote the
symmetric di erence between two sets S and S0, i.e., S4S0 = (S n S0) [ (S0 n S).
Given a set S and a tuple S = hS1; : : : ; Sni of sets, the distance between S and
S is de ned to be a tuple d(S; S) = hS4S1; : : : ; S4Sni: For two n-element
distances d and d0, d d0 if di d0i for each 1 i n, where di is the i-th element
in d. Given two sets S and S0, de ne (S; S0) = S if S0 is empty, and otherwise,
(S; S0) = f e0 2 S j 9e00 2 S0 s.t. 8e 2 S; 8e0 2 S0; d(e; e0) 6 d(e0; e00) g: In [
            <xref ref-type="bibr" rid="ref16 ref6">6</xref>
            ],
given a collection = f'1; : : : ; 'ng of propositional formulas, and some ECs
expressed as a propositional theory , the result of merging '1; : : : ; 'n w.r.t.
is the theory whose models are exactly (mod( ); mod( )), i.e., those models
satisfying and having minimal distance to .
          </p>
          <p>Inspired by classical model-based merging, we introduce a merging
operator in terms of T-models. For an ontology pro le and an EC Te, we could
de ne the T-models of the merging to be a subset of TM(Te) (so that Te is
entailed) consisting of those T-models which have minimal distance to , i.e.,
(TM(Te); TM( )). However, this straightforward adoption does not take the
CC into consideration, and the merging result obtained in this way may not
be coherent. For example, let T1 = fA v :Bg, T2 = f&gt; v Bg, Te = ;, and
Ac = fA(a); B(a)g. Then, (TM(Te); TM(hT1; T2i)) consists of only one type
fBg. Clearly, the corresponding TBox fA v ?; &gt; v Bg does not satisfy the CC,
and it is not coherent.</p>
          <p>Note that in the above example, once the merging result satis es the CC, then
it is also coherent, because both concepts A and B are satis able. In general,
it is also the case that coherency can be achieved by applying certain CC to
merging. We introduce an auxiliary ABox Ay in addition to the initial CC Ac,
in which each concept and each role is explicitly asserted with a member. That
is, Ay = fA(a) j A 2 SC ; a 2 SI is a fresh individual for Ag [ fP (b; c) j P 2
SR; b; c 2 SI are fresh individuals for P g: As assumed, SI is large enough for
us to take these auxiliary individuals. From the de nition of CCs, the merged
TBox T must be consistent with all the assertions in Ay, which assures all the
concepts and roles in T to be satis able. Based on this observation, we have the
following lemma.</p>
          <p>Lemma 1. T is coherent i</p>
          <p>T [ Ay is consistent for any TBox T .</p>
          <p>To ensure the coherence of merging, we only need to include Ay into the CC.
utes and values to their respective concept, and also identifies which concepts
participate in a relation.</p>
          <p>In OeLE, the texts are annotated semiautomatically, meaning that the teacher only
needs to manually annotate the fragments unknown to the system or incorrectly
tagged. In our system, the natural language processing is done manually for the
moment, as GATE does not sufficiently support French (out-of-the-box) for our
purposes. Performing automatic French annotation is planned as a future work.</p>
          <p>
            As an example, we use an actual question from a computer algorithms course given
at our university: “Describe Depth-First Search (DFS)”. Table 1 shows the annotation
set (at the end of the NLP phase) of the partial student’s answer: “Depth-First Search
(DFS) is an exhaustive algorithm that explores a graph...” The ideal answer supplied
by the teacher is similarly annotated; however, for every annotated entity, a numerical
value ought to be supplied specifying the relative importance of that entity within the
question.
The grading stage consists of calculating the semantic distance between the
annotation sets (obtained in Section 3.1) of each student’s answer and that of the teacher’s
ideal answer, with respect to the course ontology. Because of space limitations, we
cannot give detailed calculations for the example. The reader is advised to see the full
explanation in the original publication [
            <xref ref-type="bibr" rid="ref2">2</xref>
            ], or an easy-to-follow example in [
            <xref ref-type="bibr" rid="ref26">16</xref>
            ].
          </p>
          <p>
            The formulas used in [
            <xref ref-type="bibr" rid="ref2">2</xref>
            ] for calculating the semantic distances are given below. In
every function, teacher-provided constants allow for certain elements to be weighted
more or less heavily according to their importance. The best combination of these
constants is problem-dependent and should be discovered empirically. The “linguistic
distance” between the textual representation of the entities in the student and teacher’s
answer is also taken into account. All functions return values in the [
            <xref ref-type="bibr" rid="ref1">0,1</xref>
            ] interval.
Concept similarity. To calculate the concept similarity (CS) between concepts
, the following function is used:
(
)
(
)
(
)
(
)
          </p>
          <p>The constants
elements. Also,
,
,
indicate the relative importance of the corresponding
and .
and
(1)</p>
          <p>The concept proximity (CP) is calculated using the taxonomy formed in the
ontology by the class hierarchy defined in OWL. Note that the &lt;is-a&gt; relation is explicitly
added to the course ontology (with the class as domain and the subclass as image)
where rdfs:subClassOf is used:
&lt;owl:Class rdf:about="DepthFirstSearch"&gt;</p>
          <p>&lt;rdfs:subClassOf rdf:resource="Algorithm"/&gt;
&lt;/owl:Class&gt;</p>
          <p>If the concepts and have no taxonomic parent (other than the root), this value
is zero, otherwise it is defined as such:
(
)
|
|
(
|
)|
where | ( )| is the number of concepts separating and through the
shortest common path through the taxonomic tree, and | | is the total number
of concepts in the ontology. A shorter path thus indicates a stronger similarity
between the two concepts.</p>
          <p>The properties similarity (PS) calculates the similarity between the set of properties
associated with and . The properties of a concept c are the union of the set of
attributes that have c as domain, and the set of relations that have c as domain or
image.</p>
          <p>Lastly, ( ) uses the Levenshtein distance between the string representation
of concepts and , written below, and is defined as follows:
Attribute similarity. The attribute similarity between two attributes
concepts is calculated by a similar function:
and
of two
(2)
(3)
(4)
(5)
(
(
(
(
)
)
)
|
|
)
(</p>
          <p>)
(</p>
          <p>)
{|
|
|}|
Here also, the non-negative constants , , must add up to 1. The function
returns the (most specific) concept which is in the domain of a. The function
( ) is defined as such:
that is, the similarity of their value sets. The function
the attribute a.
returns the image of
Relation similarity. The relation similarity between two relations
lated in a similar manner:
and</p>
          <p>is
calcu(6)
It is required that the sum of the non-negative constants , be 1. The function
returns the most specific concept in the domain of r, while returns
the most specific concept in the image of r. The concept similarity is calculated twice,
to compare the domains of the relations and (obtained by ) and the
images of the relations (obtained by ), respectively.</p>
          <p>
            Global evaluation. In order to accomplish the evaluation of a question, each of the
concepts of the student’s answer is associated with the closest concept of the ideal
answer, given that each concept can only be used once. The similarity between each
pair of concepts is then calculated and is multiplied by the relative numerical value of
the concept in the ideal answer. The similarity is then added to the final grade. The
same process is repeated for relations and attributes.
3.3
Our system uses the same grading algorithm as OeLE [
            <xref ref-type="bibr" rid="ref2">2</xref>
            ]. The students’ answers are
compared to the teacher’s ideal answer. The grades are calculated based on the most
similar entity in the expected answer. In OeLE, the order of the entities is not factored
in the grade and any permutation of the linguistic expressions of the student’s answer
yields the same grade.
          </p>
          <p>However, this is not appropriate for assessing procedural knowledge in our system.
If the above method is applied to evaluate text describing procedural knowledge such
as algorithms-related answers, the grade calculation ought to take into account the
relative order of a subset of concepts expressing procedural knowledge.
Functional concepts. In order to address this issue, we propose to add functional
concepts to the course ontology. A functional concept represents a global procedure, a
sequence of sub-procedures or individual steps to accomplish a given task.</p>
          <p>Let us consider the following example algorithm, DepthFirstSearch, given in
pseudocode:
procedure DepthFirstSearch</p>
          <p>VisitRoot
VisitFirstChildNode</p>
          <p>VisitOtherSiblings
end
procedure VisitRoot [...]
procedure VisitFirstChildNode [...]
procedure VisitOtherSiblings [...]</p>
          <p>For every procedure or sub-procedure, we create a corresponding functional
concept: DepthFirstSearch, VisitRoot, VisitFirstChildNode, and VisitOtherSiblings. The
last three sub-procedures could in turn be further decomposed.</p>
          <p>
            The functional concepts allow for a high-level description of the algorithm and
mask implementation details, which would be difficult to express in the ontology
using relations or attributes. Further decomposition of VisitRoot into individual steps
could be stated in any of the following ways:
DepthFirstSearch &lt;visits&gt; Root [using relation &lt;visits&gt;]
VisitRoot &lt;visits&gt; Root [same relation with a more specific concept]
Root.visited=true [the value of the attribute &lt;visited&gt; becomes true]
Representing functional concepts in OWL. Relationships between functions are
defined as meta-functions in [
            <xref ref-type="bibr" rid="ref27">17</xref>
            ]. These meta-functions are implemented in our
system as relations between two functional concepts. In this example, two instances of
the &lt;is-preceded-by&gt; relation are needed. One instance is needed between
VisitFirstChildNode and VisitRoot, because the root has to be visited first, and another
between VisitOtherSiblings and VisitFirstChildNode, because the first child node
should be visited first. Similarly, three instances of the &lt;is-achieved-by&gt; relation are
used between VisitRoot and each of the remaining functional concepts.
          </p>
          <p>
            The same idea is found in [
            <xref ref-type="bibr" rid="ref28">18</xref>
            ], where the relation preceded_by is defined similarly
to &lt;is-preceded-by&gt; and can be used to order any pair of classes P and P1. In other
words, P preceded_by P1 is defined as “Every P is such that there is some earlier P1”.
This relation is defined as transitive, and is neither symmetric, reflexive nor
antisymmetric.
          </p>
          <p>
            In [
            <xref ref-type="bibr" rid="ref29">19</xref>
            ], an irreflexive and transitive relation precedes is used when “the sequence
of the related events is of utmost importance for the correct interpretation”. This paper
also defines the inverse relation follows.
          </p>
          <p>
            Similarly, the working draft: “Time Ontology in OWL” [
            <xref ref-type="bibr" rid="ref30">20</xref>
            ] of the World Wide
Web Consortium (W3C) states that: “There is a before relation on temporal entities,
which gives directionality to time. If a temporal entity T1 is before another temporal
entity T2, then the end of T1 is before the beginning of T2.” This relation is part of the
time namespace.
          </p>
          <p>In our implementation, the functional concepts and the &lt;is-preceded-by&gt; relation
are defined as such in OWL:
&lt;owl:Class rdf:about="FunctionalConcept"/&gt;
&lt;owl:Class rdf:about="DepthFirstSearch"&gt;</p>
          <p>&lt;rdfs:subClassOf rdf:resource="FunctionalConcept"/&gt;
&lt;/owl:Class&gt;
&lt;owl:Class rdf:about="VisitRoot"&gt;</p>
          <p>&lt;rdfs:subClassOf rdf:resource="DepthFirstSearch"/&gt;
&lt;/owl:Class&gt;
&lt;owl:Class rdf:about="VisitFirstChildNode"&gt;</p>
          <p>&lt;rdfs:subClassOf rdf:resource="DepthFirstSearch"/&gt;
&lt;/owl:Class&gt;</p>
          <p>Note that the &lt;is-achieved-by&gt; relation is implied by the class hierarchy rooted at
the concept FunctionalConcept, just as the &lt;is-a&gt; relation is implied by the class
hierarchy in OeLE.</p>
          <p>For every algorithm, a separate (meta) ontology lists the required orderings specific
to that algorithm. Although there exists many algorithms for graph exploration, we
only need to define the functional concepts once in the course ontology, and their
ordering can then be declared in a separate ontology. For instance, the
BreadthFirstSearch algorithm can be defined with the same functional concepts as above,
only ordered differently.</p>
          <p>For DepthFirstSearch, the meta-ontology is as follows:
VisitFirstChildNode &lt;is-preceded-by&gt; VisitRoot
VisitOtherSiblings &lt;is-preceded-by&gt; VisitFirstChildNode</p>
          <p>Note that the following relation is also inferred by the transitive property:
VisitOtherSiblings &lt;is-preceded-by&gt; VisitRoot
Grading with functional concepts. In our approach, the question evaluation process
remains mostly unchanged. No special treatment is given to the functional concept
class hierarchy rooted at the concept FunctionalConcept, even though its implied
relation is &lt;is-achieved-by&gt;, rather than the &lt;is-a&gt; relation implied for the other
concepts. This takes into account function nesting and composition, while allowing
calculating the proximity of the functional concepts.</p>
          <p>However, the global evaluation of a student answer R takes into account the
algorithm-specific orderings of the meta-ontology. The new evaluation function is given
below:
(
(7)
(8)</p>
          <p>
            The final grade (FG) for the student answer R is proportional to the global
evaluation of the answer, , obtained from Section 3.2. Here, is a constant in the
interval [
            <xref ref-type="bibr" rid="ref1">0,1</xref>
            ] allowing the teacher to adjust the relative importance of the correct
ordering of concepts in the global evaluation. The ordering factor of the answer, ,
is defined as follows:
|
|
where represents the number of functional concepts having the right
ordering in the student answer R, and | | the number of functional concepts
orderings in the meta-ontology.
          </p>
          <p>It should be noted that if functional concepts in the student’s answer are ordered
with the opposite relation (that is, &lt;is-followed-by&gt;), the evaluation algorithm inverts
the relation between the functional concepts.</p>
          <p>Also, the individual student grades are affected by the number of defined
orderings. If there are only a few orderings, as demonstrated below, students are strongly
penalized for every mistake. This is also the case with the concept proximity defined
in Formula 2, where the number of concepts in the ontology affects students' grades.
However, we can assume that the course ontology is fixed during evaluation, and that
the students' grades are therefore affected similarly (in a linear fashion).
4</p>
          <p>Working Example and Results
Using Depth-First Search as an example, we can quantify the effect of the new
evaluation function on a student’s answer. To simplify, we omit the conceptual grading of
the answer and concentrate on the functional grading. Since the same entities are
present in both the student and teacher’s answers, the conceptual grade is 100%. The
ideal functional answer could be as follows: “Depth-First Search first visits the root
[of a graph], then [recursively] visits its first child node before visiting its other
siblings.” Table 2 shows the produced functional concepts.
Any permutation of this ideal answer taken as input by the original approach would
yield a grade of 100%. Now, consider the following student’s answer: “Depth-First
Search visits the root [of a graph], then [recursively] visits its first child node after
visiting its other siblings.” Here, “after” inverts the ordering of the two last concepts
(highlighted in bold below), yielding the following answer:
The student gave here the incorrect ordering:
VisitFirstChildNode &lt;is-preceded-by&gt; VisitOtherSiblings
However, these two student orderings are correct:
VisitFirstChildNode &lt;is-preceded-by&gt; VisitRoot [inferred]
VisitOtherSiblings &lt;is-preceded-by&gt; VisitRoot</p>
          <p>As stated above, the conceptual grading of this answer, as performed by OeLE, is
100%. By using the new evaluation function (Formula 7), the final grade (FG)
becomes:
(
)
(9)
where the global evaluation (GE) is 100%, the ordering factor (DF) is 66.67%, and
the constant is given a value of 1.0. Considering that the ideal answer to this
algorithm contains only three orderings for pairs of functional concepts (one is inferred)
and that a third is out of order, this low grade seems acceptable, or at least a
reasonable improvement over the former grade of 100% that would have been attributed had
we only used the conceptual grading system.
5</p>
          <p>
            Conclusion and Future Work
The work presented in this paper adapts the OeLE system to include procedural
knowledge. The example was taken from an algorithms course given at Université de
Moncton. This approach could be used in other domains where procedural knowledge
is central to processing the text. For example, [
            <xref ref-type="bibr" rid="ref28">18</xref>
            ] and [
            <xref ref-type="bibr" rid="ref29">19</xref>
            ] apply similar methods to
biomedical ontologies.
          </p>
          <p>The approach put forth in this paper introduces functional concepts to represent
procedural knowledge in ontologies. The class hierarchy of functional concepts is
considered as a series of instances of the relation &lt;is-achieved-by&gt; instead of &lt;is-a&gt;.
For every computer algorithm (or procedure, for other domains), a series of instances
of the relation &lt;is-preceded-by&gt; specify an ordering for pairs of functional concepts.</p>
          <p>In this paper, the texts were annotated manually. We are considering annotating the
French texts semiautomatically as future work. The detection of the orderings
(detecting keywords such as “first”, “before”, “after” in the example of Section 4) could also
be performed automatically.</p>
          <p>In the case where the student answer uses the opposite ordering relation
(&lt;isfollowed-by&gt;), the relation between the functional concepts is inverted prior to
evaluation. Some more complex answers could require more inversions, for example if the
student wrote “X and Y should be done after Z”.</p>
          <p>Future work could also consider flow control structures, such as loops or branches,
although the textual representation of those structures without proper indentation or
braces could be ambiguous. For example, the VisitOtherSiblings functional concept
can be decomposed into the following loop: (for every other sibling, VisitNode).</p>
          <p>Another idea that could be explored would be to add the notion of recursive
procedures, such as Depth-First Search. VisitFirstChildNode and (every VisitNode of)
VisitOtherSiblings should include recursive calls. As an ideal answer, the teacher could
give either: DFS.isRecursive=true, or VisitFirstChildNode.isRecursive=true and
VisitOtherSiblings.isRecursive=true. Depending on the ideal answer given and their own
answer, students could be unjustly penalized.
16. Fernández-Breis, J.T., Valencia-García, R., Cañavate- Cañavate, D., Vivancos-Vicente,
P.J., Castellanos-Nieves, D. OeLE: Applying ontologies to support the evaluation of open
questions-based tests. In: Proceedings of the KCAP’05 WORKSHOP. SW-EL’05:
Aplications of Semantic Web Technologies for E-Learning (in conjunction with 3rd
International Conference on Knowledge Capture (KCAP’05)), Banff, Canada (2005)
17. Aroyo, L., Dicheva, D.: Courseware authoring tasks ontology. In: Proceedings of the</p>
          <p>International Conference on Computers in Education, pp. 1319-1320. (2002)
18. Smith, B., Ceusters W., Klagges, B., Köhler, J., et al.: Relations in biomedical ontologies.</p>
          <p>Genome Biology 6(R46) (2005)
19. Schulz, S., Markó, K., Suntisrivaraporn, B. Formal representation of complex SNOMED</p>
          <p>CT expressions. BMC Medical Informatics and Decision Making 8(1) (2008)
20. World Wide Web Consortium (W3C),
http://www.w3.org/TR/2006/WD-owl-time20060927/, last accessed 2012-11-21.</p>
          <p>Short paper: Using Formal Ontologies in the
Development of Countermeasures for Military</p>
          <p>Aircraft
Nelia Lombard1;2, Aurona Gerber2;3, and Alta van der Merwe3</p>
          <p>1 DPSS, CSIR
nlombard@csir.co.za
http://www.csir.co.za
2 CAIR - Centre for AI Research
Meraka CSIR and Univerity of Kwazulu-Natal
http://www.cair.za.net/
3 Department of Informatics
University of Pretoria, Pretoria
http://www.up.ac.za/</p>
          <p>South-Africa
Abstract. This paper describes the development of an ontology for use
in a military simulation system. Within the military, aircraft represent
a signi cant investment and these valuable assets need to be protected
against various threats. An example of such a threat is shoulder-launched
missiles. Such missiles are portable, easy to use and unfortunately,
relatively easy to acquire. In order to counter missile attacks,
countermeasures are deployed on the aircraft. Such countermeasures are developed,
evaluated and deployed with the assistance of modelling and simulation
systems. One such system is the Optronic Scene Simulator, an
engineering tool that is able to model and evaluate countermeasures in such a way
that the results could be used to make recommendations for successful
deployment and use.</p>
          <p>The use of formal ontologies is no longer a foreign concept in the support
of information systems. To assist with the simulations performed in the
Optronic Scene Simulator, an ontology, Simtology, was developed.
Simtology supports the system in various ways such as providing a shared
vocabulary, improving the understanding of the concepts in the
environment and adding value by providing functionality that improves
integration between system components.</p>
          <p>
            Keywords: Ontology, Countermeasure, Simulation, Design Research
1
Military forces consider aircraft as important and expensive assets often
representing huge investments. The protection of these assets is considered to be a
priority by most countries. Protection is needed from various threats and one of
these threats are attacks through enemy missiles such as surface-to-air missiles,
which are relatively cheap and easy to operate, and in addition, widely
available in current and old war-zone areas [
            <xref ref-type="bibr" rid="ref1">1</xref>
            ]. These surface-to-air missiles are often
complex and they are continually being updated to withstand aircraft
countermeasures. In addition, missile systems di er from one another and the need to
understand how each type of missile reacts in an aircraft engagement is crucial in
the development of aircraft protection countermeasures[
            <xref ref-type="bibr" rid="ref1">1</xref>
            ]. The development of
these countermeasures is often not possible in real-life situations, and modelling
and simulation are therefore necessary for the development of aircraft protection
countermeasures. Figure 1 illustrates a military aircraft ejecting a are, which
is a speci c type of countermeasure used to protect against missile attacks.
          </p>
          <p>
            Simulation systems model real-world objects and simulate them in an
articial world [
            <xref ref-type="bibr" rid="ref2">2</xref>
            ]. One such a simulation system is the Optronic Scene Simulator
(OSSIM), which has an application called the Countermeasure Simulation
System (CmSim). CmSim uses models of real world objects such as the aircraft
and the missile, and simulates di erent scenarios to evaluate the behaviour of
these models under di erent circumstances [
            <xref ref-type="bibr" rid="ref2">2</xref>
            ]. Often these evaluations require
substantial computing power and it is not uncommon to wait a few hours for
simulation results.
          </p>
          <p>At present, various problems are experienced when constructing the input
models for CmSim simulations. Because models are used as input into CmSim
simulations, it is necessary to ensure that these models are adequate and accurate
for useful simulations. The input model is adequate when it captures su cient
input variables and context, and a model is accurate when it correctly captures
the input variables and relations. It is for example possible to create input
models that are syntactically correct, but the interaction between the models are not
correctly set up in the simulation and therefore the results have no correlation
with the real world. In addition, di erent users with various roles work with the
system and it is necessary to acquire a common understanding and vocabulary
for the constructs of the models and their characteristics. Furthermore, the
creation of reference models for reuse across the user base would ensure better use
of resources and time.</p>
          <p>
            When investigating possible technologies that support modelling within
information systems, ontologies and ontology technologies feature extensively. One
of the original de nitions for the term ontology is that by Gruber who de ned an
ontology as a formalisation of a shared conceptualisation [
            <xref ref-type="bibr" rid="ref3">3</xref>
            ]. A formal
conceptualisation is a representation in a formal language of the concepts in a speci c
domain representing a part of the world. Formal ontologies are therefore
ontologies constructed using a formal representation language such as Description
Logics (DL) [
            <xref ref-type="bibr" rid="ref4">4</xref>
            ]4. Ontology is also used as a technical term denoting an artefact
that is designed for the speci c purpose of modelling knowledge about some
domain of interest. Typically a domain ontology provides a shared and common
understanding of the knowledge in the chosen domain [
            <xref ref-type="bibr" rid="ref5">5</xref>
            ]. Given the
characteristics and purpose of ontologies, we decided to investigate the use of an ontology
to address the identi ed needs when constructing CmSim Models.
          </p>
          <p>The remainder of this paper is structured as follow: Section 2 provides some
background of the simulation environment and why it was necessary to build
an ontology, as well as some background on ontologies. Section 3 discusses the
development and nature of Simtology. Sections 4 and 5 discuss the contribution
and conclude the paper in addition to discussing future work, as well as possible
extensions to the ontology.
2</p>
          <p>
            Background
One of the largest investments in the military of a country is aircraft. Aircraft
is the target of unfriendly forces in order to weaken the military forces of a
country. These attacks include attacks executed by shoulder-launched missiles,
which are portable, easy to use and relatively easy to acquire. In order to counter
these missile attacks, the military deploy various kinds of countermeasures on
aircraft, and these countermeasures are developed, evaluated and deployed with
the assistance of modelling and simulation systems such as the Optronic Scene
Simulator (OSSIM).
CmSim is a software application that is part of the broader Optronic System
Simulation (OSSIM) system [
            <xref ref-type="bibr" rid="ref2">2</xref>
            ]. OSSIM is an engineering tool used to model and
evaluate the imaging and dynamic performance of electro-optical systems under
diverse environmental conditions. OSSIM are typically used for the following
applications:
{ Development of optronic systems
{ Mission preparation
4 For the remainder of this paper we mean formal ontology when we use the term
ontology
{ Real-time rendering of infra-red and visible scenes
          </p>
          <p>
            CmSim is speci cally designed to do countermeasure evaluation for the
protection of military aircraft. Models of the aircraft, the missile, the
countermeasure and the environment are used to construct a scenario that simulates what
will happen in the real world [
            <xref ref-type="bibr" rid="ref2">2</xref>
            ]. The models are used as input into CmSim, and
it is necessary to carefully construct these models to get accurate simulations
results. The generation of simulation results are complex and time consuming,
and when inaccurate or faulty input models are used, valuable time is lost.
          </p>
          <p>In order to construct better input models, it is necessary to improve the
understanding of the simulation and the meaning of concepts in the simulation
environment. Users of models often does not know what models exist already, to
what level the models were constructed, and the scenarios that might be possible
in the simulation, and knowledge is not shared between di erent role-players.
The simulation practitioner setting up the simulation scenario might not have
specialist knowledge of how the models interact, and can set up scenarios that
are syntactically correct but do not correlate with the real world scenario. There
is therefore a need to capture the specialised knowledge in reference models that
could be used before the scenario is fed to the simulation. Figure 2 depicts the
di erent role-players that could be involved in the simulation environment.</p>
          <p>Fig. 2. Di erent Role-players Involved in a Simulation Environment</p>
          <p>
            In order to address the above mentioned needs, we initiated a project based
following the guidelines of design science research (DSR) [
            <xref ref-type="bibr" rid="ref16 ref6">6</xref>
            ]. DSR provides a
research method for research that is concerned with the design of an artefact
that solves an identi ed problem. The creation of an ontology based application
was identi ed as a possible solution to the needs articulated when constructing
OSSIM simulation models. DSR will be described further in Section 3.1. The
next sections brie y introduce background on ontologies in computing.
2.2
          </p>
          <p>
            Ontologies and Ontology Tools
The origins of the term ontology could be traced to the philosophers of the
ancient world who analysed objects in the world and study their existence [
            <xref ref-type="bibr" rid="ref3">3</xref>
            ].
Modern ontologies use the principles of the ontology of early philosophers [
            <xref ref-type="bibr" rid="ref17 ref7">7</xref>
            ].
Ontologies formally describe concepts, so it is often used to capture knowledge of
a speci c domain. The role of ontologies in a speci c domain are thus generally
de ned by [
            <xref ref-type="bibr" rid="ref5">5</xref>
            ] as to:
{ Provide people and agents in a domain with a shared, common understanding
of the information in the domain;
{ Enable reuse of domain knowledge;
{ Explicitly publish domain assumptions;
{ Provide a way to separate domain knowledge from operational knowledge;
and
{ Setting a platform for analysis of the domain knowledge.
          </p>
          <p>
            From the characteristics listed above it is possible to argue that an ontology
may be a solution to the problems experienced in OSSIM simulations. Formal
ontologies are represented in a speci c formal knowledge representation language
[
            <xref ref-type="bibr" rid="ref4">4</xref>
            ]. For building and maintaining Simtology, we adopted Protege 4 constructing
an OWL ontology. [
            <xref ref-type="bibr" rid="ref18 ref19 ref8 ref9">8, 9</xref>
            ]. Protege is widely used and support for the use of the
editor and the development of ontologies are readily available [10{12]. Protege
bundles reasoners such as Fact++ and Pellet with the environment [
            <xref ref-type="bibr" rid="ref13 ref19 ref23 ref9">9, 13</xref>
            ] and
we used these reasoners to test for consistency or to compute consequences over
the knowledge base during the development of Simtology [
            <xref ref-type="bibr" rid="ref14 ref24">14</xref>
            ].
          </p>
          <p>
            Ontology Use in Modelling, Simulation and Military Systems
Within computing, modelling and simulation are used to build a representation
of the real world and simulate the behaviour of objects presented in the models
[
            <xref ref-type="bibr" rid="ref2">2</xref>
            ]. A simulation system is a speci c application that uses a model as input
and execute a computer program that determines consequences and resulting
scenario information about the system being investigated [
            <xref ref-type="bibr" rid="ref15 ref25">15</xref>
            ].
          </p>
          <p>
            Military systems and the knowledge captured therein are complex and
often consist of layered information from di erent sources. To support this view,
Clausewitz, in his book, On War, wrote about military information as follow
[
            <xref ref-type="bibr" rid="ref26">16</xref>
            ]:
'...three quarters of the information upon which all actions in War are
based on are lying in a fog of uncertainty...'
'...in war more than any other subject we must begin by looking at the
nature of the whole; for here more than elsewhere the part and the whole
must always be thought of together...'
          </p>
          <p>
            Furthermore, Mandrick discussed the use of ontologies to model information
in the military environment. According to Mandrick, ontologies in the military
must adhere to the same requirements as ontologies in other domains, as
described in Section 2.2. Important aspects to highlight is the ability of the ontology
to provide a common vocabulary between planners, operators and commanders
in the di erent military communities [
            <xref ref-type="bibr" rid="ref26">16</xref>
            ].
          </p>
          <p>
            At present the adoption of ontologies in the military domain is primarily for
support of command and control in the battle eld, as well as the management of
assets and the sharing of knowledge[
            <xref ref-type="bibr" rid="ref11 ref21 ref27">11, 17</xref>
            ]. We also nd ontologies where there is
a need to integrate di erent data sources and the communication between these
data sources [
            <xref ref-type="bibr" rid="ref28 ref29">18, 19</xref>
            ]. Although ontologies are used in the military modelling
and simulation domain, examples are sparse and at present do not support the
construction of input models for systems such as OSSIM. It could be argued that
Simtology will therefore present a unique contribution to military information
management.
3
          </p>
          <p>
            Simtology
The development of Simtology was in response to the identi ed needs when
using the Optronic Scene Simulator (OSSIM) [
            <xref ref-type="bibr" rid="ref2">2</xref>
            ] to develop countermeasures for
missile attacks on aircraft as discussed in Section 2.1.
          </p>
          <p>
            The Design and Development Process
The research design adopted for the development of Simtology, was Design
Research (DSR). DSR is a research methodology for the design and construction
of computing artefacts through the use of rigour (the use of fundamental
knowledge) and relevance (basing the development of the artefact on real needs) [
            <xref ref-type="bibr" rid="ref16 ref30 ref6">6, 20</xref>
            ].
In this project, the artefact is Simtology, the fundamental knowledge is obtained
from ontology knowledge, and the need is the construction of models for the
OSSIM simulation environment. A DSR execution method that was proposed by
Vaishnavi et al.[
            <xref ref-type="bibr" rid="ref31">21</xref>
            ] is depicted on the left in Figure 3. This method was adopted
for this research project, and the development of Simtology is discussed further
according to the steps in Figure 3.
          </p>
          <p>Awareness and Suggestion
The rst steps in the design research process is awareness of the problem and
proposing possible suggestions for a solution. The following list summaries the
issues and needs in the simulation system as discussed in Section 2.1 that created
awareness of the problem:
{ Di erent role-players: There are developers, model builders and users
involved in the system. Inconsistencies in the terminology used between
different users often led to frustration and wrong use of concepts. There is lack
of a common vocabulary that is shared by everyone involved in building and
using the system.
{ Model complexity: One of the main characteristics is the ability of the system
to execute models at di erent levels of detail. This poses a problem to users,
when to know at which level of detail a model is implemented at.
{ Reference models: Speci c users that only interact with the system at a
certain level, need more technical insight into model detail to know what is
available in the system. This means that reference models are required that
can de ne domain-speci c concepts to these users.
{ Model Interaction: A simulation consists of a scenario that is built up from
interacting models. The models interact using a set of rules but there is
currently no rules that verify model behaviour when a scenario are constructed.</p>
          <p>Previous research e orts in the simulation environment attempted to address
the the need for a standard notation for documentation of the simulation models.
This proved to be problematic and one of the suggestions for further research was
to investigate the use ontologies in the simulation environment. The suggestion
according to the DSR process is therefore that a formal ontology is created to
address the above mentioned needs for the simulation environment.
Fig. 3. The Design Research Process on the left, and the Adaptive Methodology
Process on the right
3.3</p>
          <p>
            Development
The ontology was build using the approach followed by the researchers who
develop the Adaptive Methodology [
            <xref ref-type="bibr" rid="ref32">22</xref>
            ], and was chosen for its lightweight,
incremental approach. Figure 3 depicts the development process steps as well as
the alignment with the design process.
{ Scope and Purpose: The rst step is to scope the purpose and the
extent of the ontology. In the case of a domain ontology, the concepts of the
domain must be included. It is not necessary to include all the concepts of
the domain. The level of detail will be determined by the purpose of the
ontology.
{ The Use of Existing Structures: There are several documents, structures
and sources available in the OSSIM simulation environment available to use
in order to gather information to develop the ontology and to, for example,
make a list of the concepts in the simulation. Modelling reports, installation
guides, white papers and technical documentation, as well as the source
code of the system and the documented test point con gurations were used
as input into the ontology development process.
{ The Prototype: The prototype structure is the rst version of the ontology
that is operational. The prototype for the simulation environment contains
only a selected set of components from the domain. The concepts are on a
high level and the nested structures of complex concepts were not included
in the prototype. The prototype was developed in Protege and is illustrated
in Figure 4 on the left.
          </p>
          <p>The prototype is a proof-of-concept and in this project it played an
important role to demonstrate the feasibility of the suggested solution. The
prototype ontology supported the role of an ontology in the simulation system
environment, and supported an ontology as a solution to a shared, common
vocabulary. The tools also provided graphical views of the concepts de ned
in the ontology.
{ Development of the Working Ontology: During this phase the
prototype ontology was expanded by adding concepts from the domain not
previously included in the ontology, as well as developing new functionality. The
working ontology contains a full set of domain concepts that describe the
simulation models and model properties and is called Simtology. The next
section describes Simtology in more detail, as well as how Simtology is used
in CmSim.
3.4</p>
          <p>Description of Simtology
To do a simulation in CmSim, a scenario must be set up to act as input to
the simulation. The scenario consists of various con guration les but the main
le is the scenario le itself, which contains links to all other les necessary to
describe a scenario and the components in it. Although the prototype already
contained enough information to set up basic scenarios, Simtology contains all
the concepts in the domain of CmSim.</p>
          <p>The rst task was thus to expand the prototype to present not only the
basic objects, but all the possible objects in the CmSim domain. The classes
and properties were expanded. The following list describes the concepts and
properties de ned in Simtology.</p>
          <p>{ Concepts: In Simtology, an example of a concept representing all the
individual aircrafts modelled in the simulation environment, is Aircraft. Figure
4 depicts an extract of the top-level concepts de ned in Simtology, where the
concepts were selected to present those in a simulation scenario. The main
concepts in Simtology are the Moving, Observer and Scenario concepts. The
choice of concepts relied heavily on the structure of the simulation con
guration les. Therefore objects of type Moving have speci c behaviour in the
simulation and belong together in a concept.
{ Individuals: Individuals are asserted to be instances of speci c concepts.</p>
          <p>Speci c scenarios can be build by choosing individuals from the ontology
and thus creating an individual scenario.</p>
          <p>ScenarioC130 is an individual of the Scenario concept that uses a speci c
type of aircraft.
{ Object Properties: Object properties are used to link individuals to each
other. In Simtology, a scenario must have a clock object, so having a clock
object is an object property of the scenario concept. The name of the
property is \hasClock\ and links an individual of class Clock to an individual
from class Scenario.</p>
          <p>In Simtology the main object properties belong to a scenario. The following
properties are su cient to denote a valid scenario that can run in a
simulation: ScenarioC130 hasClock Clock10ms
ScenarioC130 hasTerrain TerrainBeachFynbos
ScenarioC130 hasMoving C130Flying120knots
ScenarioC130 hasObserver SAMTypeA
ScenarioC130 hasAtmosphere MidLatitudeSummer
{ Data Properties: Data properties were added to Simtology to add data to
individuals. Examples of data properties are geometric locations of moving
objects, or data belonging to the class Clock, as depicted below:
Clock10ms hasInterval 10ms and Clock10ms hasStopTime 10sec
Functionality: A scenario can be complex and rules were built in to ensure that
a valid scenario is constructed, for instance only certain types of ares can be
used as a valid countermeasure on a speci c aircraft. After building the scenario
in the ontology, the scenario can be processed by a reasoner. The reasoners are
used to compute the inferred ontology class hierarchy and to do consistency
checking after a scenario is created.</p>
          <p>Additional functionalities were developed for use with Simtology such as the
integration of the ontology with the graphical user interface (GUI) used to set
up the simulation. The ontology is used to populate the elements in the
interface, resulting in several advantages such as that only one source of simulation
information has to be maintained, as well as that the ontology can be used to
change the language displayed in the GUI .</p>
          <p>Functionalities were also developed to write out scenarios created in the
ontology to les that can act as input to the simulation. This made it possible
that a scenario can rst be checked for logical correctness before it is run in
the simulation. Modelling errors not handled by the simulation software are
handled early in the simulation process by using the reasoning technology in the
ontology. By having a scenario de ned in the ontology, it is possible to export a
high-level description of a scenario and its components to be used for reporting
and documentation of simulation studies.</p>
          <p>Testing of Simtology Testing the ontology was an important step towards
creating a useful Simtology. When an ontology is small with a few concepts, it
is easier to identify modelling problems but when there are large numbers of
concepts with complex relationships, it is important to test the ontology
regularly in order to avoid inconsistencies immediately and eliminate rework. During
ontology veri cation the focus was mainly to ensure that the ontology was built
correctly and that the ontology concepts match the domain it represents. The
test phase of the ontology is part of the adaptive methodology process and this
phase complements the evaluation phase of the design research process.
4</p>
          <p>Evaluation
In Section 3.2, the simulation system environment was discussed. In order to
evaluate the use of Simtology in the simulation system and the contribution it
has for the improvement of the environment, the issues mentioned in Section
3.2 are used as evaluation criteria. The identi ed issues are 1) di erent users; 2)
model complexity; 3) reference models; and 4) model interaction. When
evaluated against the identi ed issues, Simtology provided the necessary solutions.
{ Di erent users: Simtology provided a common, shared 'language' to
assist with eliminating ambiguities and the inconsistent use of terminology by
the di erent users of the system. The feedback by all concerned users was
positive. An example of how Simtology assisted with regards to a common
understanding is in the use of ambiguous terms. Some terms in the simulation
had di erent meanings depending on the user using it and the application
it was used for. An example of such a term is countermeasure, which was
vague and previously many di erent types of countermeasures existed. In
Simtology the concept Countermeasure was de ned in such a way that it
can be used as an explanatory tool to illustrate the di erent countermeasures
available in the simulation as well as the use of each countermeasure.
The Protege editor allows for several ways to communicate the ontology, for
example a graphical display of the concepts and the relations in the ontology
A visual display of the di erent components in the simulation leads to better
communication between all the people involved.
{ Model complexity: Simtology formally de ned the concepts, properties
and individuals necessary for the construction of input models. When a user
uses Simtology to construct her input model, the appropriate level of detail
and complexity of the input models are speci ed.
{ Reference models: Simtology provides a reference model for all the
different users of the system to create their speci c input models from. After
introducing Simtology, very few problems were experienced by users when
constructing simulation models because Simtology acted as a reference model
informed their speci c model design.
{ Model Interaction: A simulation consists of a scenario that is build up
from interacting models. Simtology provides a common shared language to
be used in the simulation environment for both users and when interacting
with other applications. The de nitions of concepts in the system are kept
in Simtology and made available to applications in the environment such as
the Graphical User Interface.</p>
          <p>
            As a nal evaluation, the guidelines proposed by Hevner et al. [
            <xref ref-type="bibr" rid="ref16 ref6">6</xref>
            ] for a design
research artefact were used to evaluate and present the research process
followed to develop Simtology. This discussion is outside the scope of this
paper but it was demonstrated that the construction of Simtology followed
the proposed guidelines.
          </p>
          <p>Conclusion and Future Work
The outcome of the research project was Simtology, a domain ontology for the
simulation environment that contains the model information for simulation
scenarios. An added bene t was that the process to analyse the contents of the
simulation environment to construct the ontology clari ed the knowledge in the
domain.</p>
          <p>During the construction of Simtology, the following observations were made:
{ With regards to modelling, it is important to distinguish part-of from
subclassof. An aircraft body is part of an aircraft, not part of a speci c type of
aircraft.
{ It is important to correctly model roles. Modelling a missile as an observer
in the simulation means that it can never be used in the simulation as an
object of type moving. In Simtology, a missile can therefore never be used in
a di erent role.
{ Another important modelling decision has to do with the modelling of
individuals vs. concepts. This decision has an impact on how the ontology could
ultimately be used. The choice between concept and individual is often
contextual and application-dependent but it needs to be evaluated in one of the
development cycles.
{ The development and use of the ontology should be an iterative process. As
new functionality is added, it must be tested, used and evaluated.
Existing functionality is maintained by making changes where necessary. Proper
version control is therefore also necessary when constructing ontologies.</p>
          <p>Several advantages of having an ontology in the simulation environment
emerged after the ontology was created. The ontology can, for instance, be used
in training exercises to show aircraft personnel the technical detail of the
countermeasures deployed on the aircraft. Furthermore, the simulation environment
is always expanding and improving through the addition of new models, the
addition of new properties to existing entities in the system or through the addition
of new functionality to entities. Future versions of the ontology need to
incorporate these changes and there should therefore always be future expansions to
the Simtology ontology. Furthermore, Simtology should ideally be expanded to
not only include concepts in CmSim, but also in the Optronic Scene Simulator.
One of the planned functions to be developed is to reverse engineer previously
run simulations and add the scenario descriptions of those simulations to the
ontology.</p>
          <p>References
1. Birchenall, R.P., Richardson, M.A., Butters, B., Walmsley, R.: Modelling an
infrared man portable air defence system. Infrared Physics &amp; Technology (July 2010)
2. Willers, C., Willers, M.: Ossim: Optronics scene simulator white paper. Technical
report, Council for Scienti c and Industrial Research (2011)
3. Gruber, T.R.: A translation approach to portable ontology speci cations.
Knowledge Acquisition 5(2) (1993) 199{220
4. Baader, F., Horrocks, I., Sattler, U.: Description logics as ontology languages for
the semantic web. In Hutter, D., Stephan, W., eds.: Mechanizing Mathematical
Reasoning: Essays in Honor of Jorg H. Siekmann on the Occasion of His 60th
Birthday. Volume 2605 of Lecture Notes in Arti cial Intelligence. Springer-Verlag
(2005) 228{248
5. Noy, N.F., McGuiness, D.L.: Ontology development 101: A guide to creating your
rst ontology. Technical report, Stanford Knowledge Systems Laboratory (2001)</p>
        </sec>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Bloom</surname>
            ,
            <given-names>B.S.:</given-names>
          </string-name>
          <article-title>Taxonomy of Educational Objectives, Handbook 1: The Cognitive Domain</article-title>
          .
          <article-title>David McKay Co Inc</article-title>
          ., New York (
          <year>1956</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Castellanos-Nieves</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fernández-Breis</surname>
            ,
            <given-names>J.T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Valencia-García</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Martínez-Béjar</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Iniesta-Moreno</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Semantic web technologies for supporting learning assessment</article-title>
          .
          <source>Information Sciences</source>
          <volume>181</volume>
          (
          <issue>9</issue>
          ),
          <fpage>1517</fpage>
          -
          <lpage>1537</lpage>
          (
          <year>2011</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Pérez-Marín</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pascual-Nieto</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rodríguez</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Computer-assisted assessment of freetext answers</article-title>
          .
          <source>The Knowledge Engineering Review</source>
          <volume>24</volume>
          (
          <issue>4</issue>
          ),
          <fpage>353</fpage>
          -
          <lpage>374</lpage>
          (
          <year>2009</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Callear</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jerrams-Smith</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Soh</surname>
          </string-name>
          , V.:
          <article-title>CAA of short non-MCQ answers</article-title>
          .
          <source>In: Proceedings of the 5th International CAA Conference</source>
          , Loughborough, UK (
          <year>2001</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Jordan</surname>
          </string-name>
          , S., Mitchell, T.:
          <article-title>e-Assessment for learning? The potential of short-answer free-text questions with tailored feedback</article-title>
          .
          <source>British Journal of Educational Technology</source>
          <volume>40</volume>
          (
          <issue>2</issue>
          ),
          <fpage>371</fpage>
          -
          <lpage>385</lpage>
          (
          <year>2009</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Sukkarieh</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pulman</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Raikes</surname>
          </string-name>
          , N.:
          <article-title>Auto-marking: using computational linguistics to score short, free text responses</article-title>
          .
          <source>In: Proceedings of the 29th IAEA Conference</source>
          , Philadelphia, USA (
          <year>2003</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Rudner</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Liang</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Automated essay scoring using Bayes' theorem</article-title>
          .
          <source>In: Proceedings of the Annual Meeting of the National Council on Measurement in Education</source>
          , New Orleans, LA (
          <year>2002</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Rosé</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Roque</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bhembe</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          , VanLehn, K.:
          <article-title>A hybrid text classification approach for analysis of student essays</article-title>
          .
          <source>In: Proceedings of the HLT-NAACL Workshop on Educational Applications of NLP</source>
          , Edmonton, Canada (
          <year>2003</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Mason</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Grove-Stephenson</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          :
          <article-title>Automated free text marking with paperless school</article-title>
          .
          <source>In: Proceedings of the 6th International CAA Conference</source>
          , Loughborough, UK (
          <year>2002</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Burstein</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Leacock</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Swartz</surname>
          </string-name>
          , R.:
          <article-title>Automated evaluation of essays and short answers</article-title>
          .
          <source>In: Proceedings of the 5th International CAA Conference</source>
          , Loughborough, UK (
          <year>2001</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Hou</surname>
          </string-name>
          , W.-J.,
          <string-name>
            <surname>Tsao</surname>
            ,
            <given-names>J.-H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Li</surname>
          </string-name>
          , S.-Y.,
          <string-name>
            <surname>Chen</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          :
          <article-title>Automatic Assessment of Students' Free-Text Answers with Support Vector Machines</article-title>
          .
          <source>LNCS 6096</source>
          ,
          <fpage>235</fpage>
          -
          <lpage>243</lpage>
          (
          <year>2010</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Lutticke</surname>
          </string-name>
          , R.:
          <article-title>Graphic and NLP Based Assessment of Knowledge about Semantic Networks</article-title>
          .
          <source>In: Proceedings of the Artificial Intelligence in Education conference</source>
          , IOS Press (
          <year>2005</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Wiemer-Hastings</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Allbritton</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Arnott</surname>
          </string-name>
          , E.:
          <article-title>RMT: A dialog-based research methods tutor with or without a head</article-title>
          .
          <source>In: Proceedings of the 7th International Conference on Intelligent Tutoring Systems</source>
          , Springer-Verlag, Berlin (
          <year>2004</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Pérez-Marín</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Alfonseca</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rodríguez</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pascual-Nieto</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          :
          <article-title>Willow: Automatic and adaptive assessment of students free-text answers</article-title>
          .
          <source>In: Proceedings of the 22nd International Conference of the Spanish Society for the Natural Language Processing (SEPLN)</source>
          , Zaragoza, Spain (
          <year>2006</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Klein</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kyrilov</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tokman</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Automated Assessment of Short Free-Text Responses in Computer Science using Latent Semantic Analysis</article-title>
          .
          <source>In: ITiCSE '11 Proceedings of the 16th annual joint conference on Innovation and technology in computer science education</source>
          , New York, USA, pp.
          <fpage>158</fpage>
          -
          <lpage>162</lpage>
          (
          <year>2011</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          6.
          <string-name>
            <surname>Hevner</surname>
            ,
            <given-names>A.R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>March</surname>
          </string-name>
          , S.T.,
          <string-name>
            <surname>Park</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ram</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          :
          <source>Design Science in Information Systems Research. MIS Quarterly</source>
          <volume>28</volume>
          (
          <issue>1</issue>
          ) (
          <year>2004</year>
          )
          <volume>75</volume>
          {
          <fpage>105</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          7.
          <string-name>
            <surname>Guarino</surname>
          </string-name>
          , N.:
          <article-title>Formal ontology and information systems</article-title>
          . In Guarino, N., ed.
          <source>: Proceedings of the First International Conference on Formal Ontologies in Information Systems (FOIS-98)</source>
          , June 6-8,
          <year>1998</year>
          , Trento, Italy, IOS Press, Amsterdam, The Netherlands (
          <year>1998</year>
          )
          <volume>3</volume>
          {
          <fpage>15</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          8. OWL:
          <article-title>Owl2 web ontology language</article-title>
          . Available at http://www.w3.
          <source>org/TR/owl2- overview. [1 April</source>
          <year>2011</year>
          ]
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          9. Protege:
          <article-title>The protege ontology editor</article-title>
          . Available at http://protege.stanford.
          <source>edu. [13 April</source>
          <year>2011</year>
          ]
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          10.
          <string-name>
            <surname>Gaevic</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Djuric</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          , Deved c, V.:
          <article-title>Model Driven Engineering and Ontology Development. 2</article-title>
          . edn. Springer, Berlin (
          <year>2009</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          11.
          <string-name>
            <surname>Schleno</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          , Washington, R.,
          <string-name>
            <surname>Barbera</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>An intelligent ground vehicle ontology to enable multi-agent system integration</article-title>
          . In:
          <article-title>Integration of Knowledge Intensive Multi-Agent Systems</article-title>
          . (
          <year>2005</year>
          )
          <volume>169</volume>
          {
          <fpage>174</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          12.
          <string-name>
            <surname>Nagle</surname>
            ,
            <given-names>J.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Richmond</surname>
            ,
            <given-names>P.W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Blais</surname>
            ,
            <given-names>C.L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Goerger</surname>
            ,
            <given-names>N.C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kewley</surname>
            ,
            <given-names>R.H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Burk</surname>
            ,
            <given-names>R.K.</given-names>
          </string-name>
          :
          <article-title>Using an ontology for entity situational awareness in a simple scenario</article-title>
          .
          <source>The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology</source>
          <volume>5</volume>
          (
          <issue>2</issue>
          ) (
          <year>2008</year>
          )
          <volume>139</volume>
          {
          <fpage>158</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          13.
          <string-name>
            <surname>Tsarkov</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Horrocks</surname>
          </string-name>
          , I.:
          <article-title>FaCT++ description logic reasoner: System description</article-title>
          .
          <source>In: Proc. of the Int. Joint Conf. on Automated Reasoning (IJCAR 2006). Volume 4130 of Lecture Notes in Arti cial Intelligence</source>
          ., Springer (
          <year>2006</year>
          )
          <volume>292</volume>
          {
          <fpage>297</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          14.
          <string-name>
            <surname>Bock</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Haase</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ji</surname>
            ,
            <given-names>Q.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Volz</surname>
          </string-name>
          , R.:
          <source>Benchmarking OWL Reasoners. CEUR Workshop Proceedings (June</source>
          <year>2008</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          15.
          <string-name>
            <surname>Benjamin</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Patki</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mayer</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          :
          <article-title>Using ontologies for simulation modeling</article-title>
          .
          <source>In: Proceedings of the 38th conference on Winter simulation. WSC '06</source>
          ,
          <string-name>
            <surname>Winter</surname>
            <given-names>Simulation Conference</given-names>
          </string-name>
          (
          <year>2006</year>
          )
          <volume>1151</volume>
          {
          <fpage>1159</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          16.
          <string-name>
            <surname>Mandrick</surname>
            ,
            <given-names>L.B.</given-names>
          </string-name>
          :
          <article-title>Military ontology</article-title>
          . http://militaryontology.com/
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          17.
          <string-name>
            <surname>Valente</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Holmes</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Alvidrez</surname>
            ,
            <given-names>F.C.</given-names>
          </string-name>
          :
          <article-title>Using a military information ontology to build semantic architecture models for airspace systems</article-title>
          .
          <source>In: Aerospace Conference</source>
          ,
          <source>2005 IEEE. (March</source>
          <year>2005</year>
          )
          <volume>1</volume>
          {
          <fpage>7</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          18.
          <string-name>
            <surname>Winklerova</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          :
          <article-title>Ontological approach to the representation of military knowledge</article-title>
          .
          <source>Technical report</source>
          , Military Academy in Brno, Command and
          <string-name>
            <given-names>Sta</given-names>
            <surname>Faculty</surname>
          </string-name>
          , Czech
          <string-name>
            <surname>Republic</surname>
          </string-name>
          (
          <year>2003</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          19.
          <string-name>
            <surname>Smart</surname>
            ,
            <given-names>P.R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Russell</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shadbolt</surname>
            ,
            <given-names>N.R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shraefel</surname>
            ,
            <given-names>M.C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Carr</surname>
            ,
            <given-names>L.A.</given-names>
          </string-name>
          :
          <source>Aktivesa. Comput. J. 50 (November</source>
          <year>2007</year>
          )
          <volume>703</volume>
          {
          <fpage>716</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          20.
          <string-name>
            <surname>Hevner</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chatterjee</surname>
          </string-name>
          , S. In: Evaluation. Volume
          <volume>22</volume>
          of Integrated Series in Information Systems. Springer US (
          <year>2010</year>
          )
          <volume>109</volume>
          {
          <fpage>120</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          21.
          <string-name>
            <surname>Vaishnavi</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kuechler</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          :
          <article-title>Design research in information systems</article-title>
          . Available at http://desrist.org/design-research-in
          <source>-information-systems/ Last Updated 16 August</source>
          <year>2009</year>
          (
          <year>January 2004</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          22.
          <string-name>
            <surname>Bergman</surname>
            ,
            <given-names>M.:</given-names>
          </string-name>
          <article-title>A new methodology for building lightweight, domain ontologies</article-title>
          . Available at http://www.mkbergman.com/908/a
          <article-title>-new-methodology-for-buildinglightweight-domain-ontologies/ (</article-title>
          <year>2010</year>
          )
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>