<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>The Role of Foundational Ontologies for Preventing Bad Ontology Design</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Stefan SCHULZ</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>AVERBIS GmbH</institution>
          ,
          <addr-line>Freiburg</addr-line>
          ,
          <country country="DE">Germany</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Institute for Medical Informatics, Statistics and Documentation, Medical University of Graz</institution>
          ,
          <country country="AT">Austria</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Ontology engineering is error-prone, and many published ontologies suffer from quality problems. This paper initiates a discussion about how axiomatically rich foundational ontologies can contribute to prevent and to detect bad ontology design. Examples T-boxes are presented, and it is demonstrated how typical design errors can be detected by upper-level axioms, in particular disjoint class axioms, existential and value restrictions. However, debugging large domain ontologies under an expressive top level raises scalability issues. During domain ontology design this can be mitigated by using small random ontology modules for debugging. For reasoning in applications, however, less expressive variants of such foundational ontologies are necessary.</p>
      </abstract>
      <kwd-group>
        <kwd />
        <kwd>Foundational ontologies</kwd>
        <kwd>description logics</kwd>
        <kwd>quality assurance</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>Constructing domain ontologies is a demanding endeavour. The formalization of basic
regularities of a domain requires not only familiarity with the domain but also
understanding of the basic principles of logic and formal ontology. This is especially the
case if ontology engineering is understood not as building (closed-world) models limited
to the support of well-delineated reasoning use cases in restricted domains, but as
providing interoperable and re-usable (open world) representations of the domain itself.</p>
      <p>
        This is a basic principle stressed not only by the defenders of so-called realist
ontologies [
        <xref ref-type="bibr" rid="ref1 ref2">1,2</xref>
        ] but also by some (moderate) critics [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ], which documents an increasing
consensus on how to represent those areas of knowledge where people tend to agree on
an observer-independent reality and benefit from standardised terms, such as in natural
science and technology domains. An important tenet of these ontologies is collaborative
ontology development and interoperability. Principles for this kind of ontology
development have been formulated by the OBO Foundry consortium [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] and within the
Good Ontology Design (GoodOD) guidelines [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. Both propagate a concise foundational
upper-level as a mainstay for interoperability, and there is also some evidence that
foundational ontologies – domain-independent top-level or domain-related upper-level
ontologies – speed up ontology development and improve quality [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ].
      </p>
      <p>1 Corresponding Author: Stefan Schulz, Institute for Medical Informatics, Statistics and Documentation,
Medical University of Graz, Auenbruggerplatz 2/V, 8036 Graz, Austria, E-mail: steschu@gmail.com</p>
      <p>This paper is intended to initiate a discussion on how foundational ontologies can
help prevent typical errors in domain ontologies. This is also the reason why prototypical,
partly made-up but easily understandable examples are used.
2.</p>
    </sec>
    <sec id="sec-2">
      <title>Materials and Methods</title>
      <p>
        The work is based on BioTopLite (BTL2), an upper-level ontology [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], linked to BFO
[
        <xref ref-type="bibr" rid="ref8">8</xref>
        ], using description logics [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] with OWL-DL expressiveness [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. BTL2 has been
designed with the intent to provide a rich set of constraining axioms to enforce the
consistency of ontologies modelled thereunder. Although BTL2 is, in principle,
domainindependent, its content is geared to the domains of health care and biomedical research.
This explains, e.g., the provision of more fine-grained classes for chemical and biological
entities (e.g. ‘mono molecular entity’, organism, cell, population) as well as the
disjunctive class condition, created in order to deal with the ontological heterogeneity of
key medical concepts like diseases, signs, and symptoms. Fig. 1 provides Protégé
screenshots of the class and relation hierarchies, together with sample axioms.
      </p>
      <p>BTL2 Classes</p>
      <p>BTL2 Relations</p>
      <p>BTL Axioms (examples)
In order to test and demonstrate how BTL2 axioms are useful for preventing
ontology design errors, T-boxes with typical examples of bad modelling will be
presented in order to challenge the underlying foundational ontology. Some of these
examples are formulated very abstractly due to their high level of generality; others use
terms from a specific domain but are still understandable for a broader public.</p>
      <p>
        Each T-box is modelled as an extension of BTL2. The HermIT [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] reasoner was
used to detect inconsistencies. The explanation of inconsistencies follows Protégé’s
OWL entailment explanation feature [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ]. The examples are provided together with their
results in the following section. The presentation of the original OWL expressions, the
entailments and their justifications is done in the following order:
 SRC – Axioms from ontology source (known satisfiable).
 CHA – New axioms added that challenge the satisfiability of SRC. Cases
in which an axiom is only transiently added, are marked by CHA-T.
 INF – Inference, in particular the detection of classes that are unsatisfiable
w.r.t. the T-box constituted by SRC and CHA.
 EXP – Explanations of INF, with axioms collected from the explanation
plug-in [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ].
3.
      </p>
    </sec>
    <sec id="sec-3">
      <title>Results</title>
      <p>Reasoning examples under BTL2 are presented, in which deliberately erroneous axioms
lead to an incoherent ontology, i.e., where one or more named classes turn out to be
unsatisfiable, i.e. necessarily empty w.r.t. the T-box. This is expected to be detected by
a DL classifier. In the following, a distinction is made between five error types, viz. (1)
simple category errors, (2) value restrictions and transitive role errors, (3) complex
domain/range constraint errors, (4) physical granularity errors, and (5) errors regarding
unrealized realisables and non-referring information entities.
3.1.</p>
      <sec id="sec-3-1">
        <title>Simple category errors</title>
        <p>
          A category error occurs whenever a class is a taxonomic descendent of upper-level
classes that are modelled as mutually exclusive (Disjoint Classes in OWL). Such errors
typically occur when ontology mapping and alignment is guided by lexical criteria. For
instance, when mapping content of the clinical ontology SNOMED CT [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ] (namespace
sct:) to the foundational ontology BTL2 (namespace btl2:) one might be tempted to
equate sct:Process with btl2:process and place ‘sct: Qualifier Value (qualifier value)’
under ‘btl2:quality’:
SRC
NEW
NEW
INF
EXP
‘sct:Process (qualifier value)’ SubClassOf
        </p>
        <p>‘sct:Qualifier Value (qualifier value)’
‘sct:Qualifier Value (qualifier value)’ SubClassOf btl2:quality
‘sct:Process (qualifier value)’ EquivalentTo btl2:process
‘sct:Process (qualifier value)’ EquivalentTo owl:Nothing
btl2:quality DisjointWith btl2:process
(1a)
(1b)
(1c)
(1d)
(1e)</p>
        <p>
          The origin of such category errors may be manifold. A common cause is misleading
class labelling. Ontology labels should be context-independent and self-explanatory, and
they should avoid ambiguous terms [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ]. A more severe problem – like here – arises
where the ontologies to be aligned fundamentally differ in upper level assumptions. In
SNOMED CT, e.g., the subhierarchy ‘sct:Qualifier Value (qualifier value)’ is currently
a badly organized reservoir for the most diverse terms, and the reason why hierarchies
of pathological and physiological processes are placed therein remains unclear2.
3.2.
        </p>
      </sec>
      <sec id="sec-3-2">
        <title>Value restrictions and transitive roles</title>
        <p>Value restrictions (universal constraints expressed by “only” in OWL Manchester
syntax) restrict the range of allowed role fillers. It is tempting to use value restrictions
together with mereological statements, e.g. stating that all members of a class have only
parts of a certain kind, like in the following example:
SRC
SRC
CHA
INF
EXP
EXP
btl2:cell SubClassOf btl2:compound
‘cell culture’ SubClassOf ‘btl2:material object’
‘cell culture’ SubClassOf and (‘btl2:has part’ some btl2:cell)</p>
        <p>and (‘btl2:has part’ only btl2:cell)
‘cell culture’ EquivalentTo owl:Nothing
‘btl2:material object’ SubClassOf ‘btl2:has part’ some</p>
        <p>‘btl2:subatomic particle’
btl2:compound DisjointWith ‘btl2:subatomic particle’</p>
        <p>Axiom like (2c) may fulfil their purpose in domain ontologies in which cells are the
smallest objects, but as soon as smaller objects are allowed, the expression is inadequate
– assuming 'has part' being transitive. BTL2 obviates such a granularity restriction by
stating that all material objects have subatomic particles as (transitive) parts.
3.3.</p>
        <p>
          Complex domain / range restrictions
(2a)
(2b)
(2c)
(2d)
(2e)
(2f)
(3a)
(3b)
(3c)
(3d)
(3e)
(3f)
(3g)
2 Processes, like all kinds of entities, may play the role of values in information models, e.g., “Infectious
process”. Their confusion with their referents, i.e. domain entities proper (an infectious process in a patient) is
a classic case of use-mention confusion, which still haunts many domain ontologies [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ], especially those
deriving from frames, thesauri and other knowledge organization systems.
        </p>
        <p>Domain / range restrictions are a suitable means to avoid ontology errors. However, in
ontologies that use a small number of object properties like BTL2, this resource may be
not expressive enough. For instance, if the relations ‘btl2:is part of’ and ‘btl2:has part’
are valid for material objects, immaterial objects, as well as for processes and information
objects, formulating just domain / range restrictions at the level of these relations would
still be compatible with an (unintended) model in which an object is part of a process or
vice versa. This is why BTL2 encodes these restrictions using axioms like the one in (3f).
SRC
SRC
CHA
INF
EXP
EXP
EXP</p>
      </sec>
      <sec id="sec-3-3">
        <title>Object_A SubClassOf ‘btl2:material object’</title>
      </sec>
      <sec id="sec-3-4">
        <title>Process_A SubClassOf btl2:process</title>
        <p>Object_A SubClassOf ‘btl2:is part of’ some Process_A
Object_A EquivalentTo owl:Nothing
‘btl2:has part’ InverseOf ‘btl2:is part of’
btl2:process SubClassOf ‘btl2:has part’ only btl2:process
btl2:process DisjointWith ‘btl2:material object’</p>
        <p>A similar example is typical for medical ontologies, where domain experts are
tempted to use “diagnosis” and “disease” interchangeably, and where signs and
symptoms are referred to, in colloquial discourse, as being “parts” of diagnoses:</p>
      </sec>
      <sec id="sec-3-5">
        <title>Diagnosis_A SubClassOf Diagnosis</title>
      </sec>
      <sec id="sec-3-6">
        <title>Diagnosis SubClassOf ‘btl2:information object’</title>
      </sec>
      <sec id="sec-3-7">
        <title>Symptom_A SubClassOf Symptom</title>
      </sec>
      <sec id="sec-3-8">
        <title>Symptom SubClassOf btl2:condition</title>
        <p>Diagnosis_A SubClassOf ‘btl2:has part’ some Symptom_A</p>
      </sec>
      <sec id="sec-3-9">
        <title>Diagnosis_A EquivalentTo owl:Nothing</title>
        <p>‘btl2:information object’ SubClassOf</p>
        <p>‘btl2:has part’ only ‘btl2:information object’
btl2:condition EquivalentTo btl2:function or btl2:disposition or
‘btl2:material object’ or btl2:process
DisjointClasses: ‘btl2:material object’, btl2:process, btl2:function or
btl2:disposition, ‘btl2:information object’, ‘btl2:immaterial object’,
btl2:role, btl2:quality, ‘btl2:temporal region’, ‘btl2:value region’
SRC
SRC
SRC
SRC
CHA
INF
EXP
EXP
EXP
SRC
SRC
CHA
INF
EXP
(3h)
(3i)
(3j)
(3k)
(3l)
(3m)
(3n)
(3o)
(3p)
(4a)
(4b)
(4c)
(4d)
(4e)
3.4.</p>
      </sec>
      <sec id="sec-3-10">
        <title>Physical granularity</title>
        <p>Ontologies for natural sciences and technology deal largely with physical objects of
several degrees of granularity. Levels of material granularity obey certain mereological
constraints, e.g. that biological cells can be parts of organisms but not vice versa, or that
polymolecular entities can never be part of single molecules. BTL2 incorporates such
constraints. They help detect modelling errors like the following one.</p>
      </sec>
      <sec id="sec-3-11">
        <title>Chromosome SubClassOf ‘btl2:poly molecular composite entity’</title>
      </sec>
      <sec id="sec-3-12">
        <title>ProteinMolecule SubClassOf ‘btl2:mono molecular entity’</title>
      </sec>
      <sec id="sec-3-13">
        <title>Chromosome SubClassOf ‘btl2:is part of’ some ProteinMolecule</title>
      </sec>
      <sec id="sec-3-14">
        <title>Chromosome EquivalentTo owl:Nothing</title>
        <p>‘btl2:poly molecular composite entity’ and (‘btl2:is part of’ some
(btl2:atom or ‘btl2:mono molecular entity’ or
‘btl2:subatomic particle’)) SubClassOf owl:Nothing
3.5.</p>
      </sec>
      <sec id="sec-3-15">
        <title>Unrealised realisables and non-referring information entities</title>
        <p>
          Realisable entities like functions and dispositions [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ] depend on material entities and
are realised in processes. However, the existence of realisables does not imply their
realisation: The function of a screwdriver is to drive screws, and the disposition of a glass
is to break under certain circumstances, but as there are screwdrivers that are never used
and glasses that are never thrown to the floor. Because for all types of functions and
dispositions there are instances that have never been realised, ontologies have to deal
with unrealised realisables, which could be, e.g., expressed by (5a).
        </p>
        <p>SRC</p>
      </sec>
      <sec id="sec-3-16">
        <title>Unrealized_Function EquivalentTo btl2:function and not</title>
        <p>(‘btl2:has realization’ some owl:thing)
(5a)</p>
        <p>In order not to preclude the possibility that dispositions and functions happen to be
never realised, ontologies under BTL2 should define them using the value restriction
constructor:
SRC btl2:function SubClassOf ‘btl2:has realization’ only btl2:process (5b)
SRC Function_A EquivalentTo btl2:function and (5c)
(‘btl2:has realization’ only Process_A)
Nevertheless, BTL2 does not reject a definition using an existential quantifier like in the
following definition:
CHA</p>
        <p>
          Function_A EquivalentTo btl2:function and
(‘btl2:has realization’ some Process_A)
(5d)
Function_A would then exclude all unrealised function instances. Such a class (which
might be considered anti-rigid [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ] if assuming that realisable entities are always
unrealised when they come into existence) is most likely not intended by the modeller.
The detection of these errors requires checking consistency after transiently adding
axiom (5e).
        </p>
        <p>CHA-T
INF
btl2:function EquivalentTo Unrealized_Function
btl2:function EquivalentTo owl:Nothing
The explanation is given by the conjunction of axioms (5a), (5d), and (5e).</p>
        <p>
          The axiomatization of non-referring information entities follows the same pattern.
Information entities can be referring and non-referring [
          <xref ref-type="bibr" rid="ref18">18</xref>
          ]. A typical example is medical
diagnosis [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ]. BTL2 here uses the relation represents. This relation connects
information entities with domain entity they correctly characterise. This allows to
distinguish wrong diagnoses from correct diagnoses.
        </p>
        <p>SRC
SRC
SRC
CHA</p>
      </sec>
      <sec id="sec-3-17">
        <title>Diagnosis SubClassOf ‘btl2:information object</title>
      </sec>
      <sec id="sec-3-18">
        <title>False_diagnosis SubClassOf Diagnosis and</title>
        <p>(not btl2:represents some btl2:condition)</p>
      </sec>
      <sec id="sec-3-19">
        <title>Cancer_Diagnosis EquivalentTo Diagnosis and btl2:represents</title>
        <p>only (Cancer or not btl2:condition)</p>
      </sec>
      <sec id="sec-3-20">
        <title>Cancer_diagnosis EquivalentTo Diagnosis and btl2:represents some Cancer</title>
        <p>Challenged by the axiom (5k) the T-box becomes incoherent.
CHA-T
INF</p>
      </sec>
      <sec id="sec-3-21">
        <title>Diagnosis EquivalentTo False_Diagnosis</title>
      </sec>
      <sec id="sec-3-22">
        <title>Cancer_Diagnosis EquivalentTo owl:Nothing</title>
        <p>The explanation is given by the conjunction of axioms (5h), (5i), and (5k).
(5e)
(5f)
(5g)
(5h)
(5i)
(5j)
(5k)</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Discussion and Further Work</title>
      <p>It was shown how a highly axiomatised foundational ontology like BioTopLite (BTL2)
can incorporate constraints that reject bad modelling decisions that lead to unsatisfiable
classes. A distinction was made between those cases in which the upper level axioms
suffice for detecting such inconsistencies and those in which additional “challenges”, i.e.
transiently added axioms are necessary.</p>
      <p>
        Most of the former cases capitalise on disjoint class axioms present in the upper
level ontology. This comes near to the so-called logical anti-patterns introduced by [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ],
all of which require disjoint class axioms in order to detect inconsistencies. In contrast
to the work presented, anti-patterns are very abstract logical expressions and independent
of foundational ontologies. OntoClean [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ] was presented as a methodology for detecting
improper subclass axioms based on philosophically inspired, domain-independent
properties of classes, the metaproperties unity, identity and rigidity. Although DL
reasoners do not support meta-level reasoning, it has to be investigated whether certain
elements from OntoClean could also included in DL-based foundational ontologies, e.g.
by reifying them in terms of additional top-level classes3.
      </p>
      <p>
        Several limitations of this work have to be highlighted:
 The typology presented is certainly non-exhaustive. It is primarily motivated by
the author’s experience and not yet by the relevance of those types of problems
in ontologies employed in real-world applications. It could further be related to
existing work in ontology evaluation, e.g., the OQuaRE framework [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ].





      </p>
      <p>The proposed approach will probably fail if application ontologies bypass the
partition of upper categories of the foundational ontology or introduce new
object properties that are not subproperties of the existing ones. The BTL2
authors claim that their inventory of object properties is close to sufficient and
recommend to introduce predicates required by the domain (e.g., in the
biomedical domain: treats, prevents, diagnoses, interacts, binds) not as object
properties but as subclasses of btl2:process.</p>
      <p>Important causes of bad ontology design cannot be prevented or remedied by a
foundational ontology. This includes erroneous representation of individuals as
classes or vice versa (a typical error would be an A-Box OWL expression like
“SodiumAtom Type btl2:atom), bad naming and insufficient documentation, as
well as constraints on a meta-class level like in OntoClean.</p>
      <p>
        Constraining axioms similar to the proposed ones can be added to domain
ontologies, e.g. to assure mereotopological non-overlapping [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ].
      </p>
      <p>Although BTL2 was used as a testbed, the proposed approach would lend itself
to other ontologies as well. Especially BFO would benefit from a stronger
axiomatization, as the most popular version, which is the umbrella of most OBO
ontologies lacks axioms beyond subclass and disjoint class axioms. This
deficiency has been addressed by the 2.0 version, which is, however, not fully
available in OWL due to its use of ternary relations.</p>
      <p>
        The fact that BTL2 uses the whole range of constructors allowed by OWL-DL
has a negative impact on reasoning performance. This makes debugging of large
ontologies intractable. This was the case when aligning SNOMED CT with
BTL2 [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ]. The solution was to use small modules created from random
3 Which would be orthogonal to the existing ones, e.g. ‘rigid entity’‚ ‘anti-rigid entity’, ‘whole’
signatures as described in [
        <xref ref-type="bibr" rid="ref24">24</xref>
        ]. As a solution a two-step approach was proposed:
(i) at design time using the rich foundational ontology for debugging random
modules of a (large) domain ontology under construction, and (ii) at runtime
placing the final domain ontology under a light version of the same foundational
ontology for enabling efficient reasoning.
      </p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>I. Johansson</surname>
          </string-name>
          , Bioinformatics and biological reality.
          <source>J. Biomed Inform</source>
          <volume>39</volume>
          (
          <year>2006</year>
          ),
          <fpage>274</fpage>
          -
          <lpage>287</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>B.</given-names>
            <surname>Smith</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Ceusters</surname>
          </string-name>
          ,
          <article-title>Ontological realism: A methodology for coordinated evolution of scientific ontologies</article-title>
          .
          <source>Applied Ontology</source>
          <volume>5</volume>
          (
          <year>2010</year>
          )
          <fpage>139</fpage>
          -
          <lpage>188</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>P.</given-names>
            <surname>Lord</surname>
          </string-name>
          and
          <string-name>
            <given-names>R.</given-names>
            <surname>Stevens</surname>
          </string-name>
          .
          <article-title>Adding a little reality to building ontologies for biology</article-title>
          .
          <source>PLoS one 5</source>
          .9 (
          <issue>2010</issue>
          ),
          <year>e12258</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>M.</given-names>
            <surname>Ashburner</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Rosse</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Bard</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Bug</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Ceusters</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.J.</given-names>
            <surname>Goldberg</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Eilbeck</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Ireland</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.J.</given-names>
            <surname>Mungall</surname>
          </string-name>
          , the OBI Consortium,
          <string-name>
            <given-names>N.</given-names>
            <surname>Leontis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Rocca-Serra</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Ruttenberg</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. A.</given-names>
            <surname>Sansone</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.H.</given-names>
            <surname>Scheuermann</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Shah</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.L.</given-names>
            <surname>Whetzel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Lewis</surname>
          </string-name>
          ,
          <article-title>The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration</article-title>
          ,
          <source>Nature Biotechnology</source>
          <volume>25</volume>
          (
          <year>2007</year>
          ),
          <fpage>1251</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>S.</given-names>
            <surname>Schulz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Seddig-Raufie</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Grewe</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Röhl</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Schober</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Boeker</surname>
          </string-name>
          ,
          <string-name>
            <surname>L. Jansen,</surname>
          </string-name>
          <article-title>Guideline on Developing Good Ontologies in the Biomedical Domain with Description Logics</article-title>
          , http://www.purl.org/goodod/guideline ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>C. M.</given-names>
            <surname>Keet</surname>
          </string-name>
          .
          <article-title>The use of foundational ontologies in ontology development: an empirical assessment</article-title>
          .
          <source>Extended Semantic Web Conference</source>
          . Springer, Berlin, Heidelberg (
          <year>2011</year>
          )
          <fpage>321</fpage>
          -
          <lpage>335</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>S.</given-names>
            <surname>Schulz. M. Boeker</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Martinez-Costa</surname>
          </string-name>
          ,
          <article-title>The BioTop family of upper level ontological resources for biomedicine</article-title>
          ,
          <source>Stud Health Technol Inform</source>
          .
          <volume>235</volume>
          (
          <year>2017</year>
          ),
          <fpage>441</fpage>
          -
          <lpage>445</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>R.</given-names>
            <surname>Arp</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Smith</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Spear</surname>
          </string-name>
          ,
          <article-title>Building Ontologies with Basic Formal Ontology</article-title>
          , MIT Press,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>F.</given-names>
            <surname>Baader</surname>
          </string-name>
          (ed.),
          <source>The Description Logic Handbook, Second Edition</source>
          . Cambridge University Press,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <fpage>W3C</fpage>
          , OWL2 Web Ontology Language http://www.w3.org/TR/owl2-overview/ ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>B.</given-names>
            <surname>Glimm</surname>
          </string-name>
          et al.
          <article-title>HermiT: an OWL 2 reasoner</article-title>
          .
          <source>Journal of Automated Reasoning</source>
          ,
          <volume>3</volume>
          (
          <year>2014</year>
          ),
          <fpage>245</fpage>
          -
          <lpage>269</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>M.</given-names>
            <surname>Horridge</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Parsia</surname>
          </string-name>
          and
          <string-name>
            <given-names>U.</given-names>
            <surname>Sattler</surname>
          </string-name>
          .
          <article-title>Explanation of OWL entailments in Protégé 4</article-title>
          .
          <source>Proceedings of the Poster and Demonstration Session at the 7th International Semantic Web Conference. CEUR-WS. org</source>
          , http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>401</volume>
          /iswc2008pd_submission_47.pdf ,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <article-title>SNOMED international</article-title>
          .
          <source>SNOMED CT</source>
          , https://www.snomed.org/snomed-ct ,
          <year>2018</year>
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>D.</given-names>
            <surname>Schober</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Smith</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.E.</given-names>
            <surname>Lewis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Kusnierczyk</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Lomax</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Mungall</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.F.</given-names>
            <surname>Taylor</surname>
          </string-name>
          , P.
          <string-name>
            <surname>Rocca-Serra</surname>
            ,
            <given-names>S.A.</given-names>
          </string-name>
          <string-name>
            <surname>Sansone</surname>
          </string-name>
          ,
          <article-title>Survey-based naming conventions for use in OBO Foundry ontology development</article-title>
          ,
          <source>BMC Bioinformatics 10.1</source>
          (
          <issue>2009</issue>
          ),
          <fpage>125</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>B.</given-names>
            <surname>Smith</surname>
          </string-name>
          ,
          <article-title>Against idiosyncrasy in ontology development</article-title>
          .
          <source>Frontiers in Artificial Intelligence and Applications</source>
          <volume>150</volume>
          (
          <year>2006</year>
          ),
          <fpage>15</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>J.</given-names>
            <surname>Röhl</surname>
          </string-name>
          , L. Jansen,
          <article-title>Why functions are not special dispositions: an improved classification of realizables for top-level ontologies</article-title>
          .
          <source>Journal of biomedical semantics 5</source>
          .1 (
          <issue>2014</issue>
          ),
          <fpage>27</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>N.</given-names>
            <surname>Guarino</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.A.</given-names>
            <surname>Welty</surname>
          </string-name>
          .
          <article-title>An overview of OntoClean</article-title>
          . In Handbook on ontologies,
          <fpage>151</fpage>
          -
          <lpage>171</lpage>
          , Springer, Berlin, Heidelberg,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>S.</given-names>
            <surname>Schulz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Brochhausen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Hoehndorf</surname>
          </string-name>
          ,
          <article-title>Higgs bosons, Mars missions, and unicorn delusions: How to deal with terms of dubious reference in scientific ontologies</article-title>
          .
          <source>Proc. of ICBO 2011. CEUR-WS</source>
          <volume>833</volume>
          (
          <year>2011</year>
          ), http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>833</volume>
          /paper24.pdf .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <given-names>S.</given-names>
            <surname>Schulz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Martínez-Costa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Karlsson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Cornet</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Brochhausen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.L.</given-names>
            <surname>Rector</surname>
          </string-name>
          ,
          <article-title>An ontological analysis of reference in health record statements</article-title>
          .
          <source>Formal Ontology in Inf. Systems</source>
          (
          <year>2014</year>
          ),
          <fpage>289</fpage>
          -
          <lpage>302</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>C.</given-names>
            <surname>Roussey</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.</given-names>
            <surname>Corcho</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L. M.</given-names>
            <surname>Vilches-Blázquez</surname>
          </string-name>
          .
          <article-title>A catalogue of OWL ontology antipatterns</article-title>
          ,
          <source>Proceedings of the fifth international conference on Knowledge capture. ACM</source>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>A.</given-names>
            <surname>Duque-Ramos J.T.</surname>
          </string-name>
          Fernández-Breis,
          <string-name>
            <given-names>R.</given-names>
            <surname>Stevens</surname>
          </string-name>
          ,
          <string-name>
            <surname>N.</surname>
          </string-name>
          <article-title>Aussenac-Gilles, OQuaRE: A SQuaRE-based approach for evaluating the quality of ontologies</article-title>
          .
          <source>Journal of Research and Practice in Information Technology 43.2</source>
          (
          <issue>2011</issue>
          ),
          <fpage>159</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <given-names>M.</given-names>
            <surname>Boeker</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Hastings</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Schober</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Schulz</surname>
          </string-name>
          (
          <year>2011</year>
          ).
          <article-title>A T-Box generator for testing scalability of OWL mereotopological patterns</article-title>
          .
          <source>OWLED</source>
          <year>2011</year>
          -
          <article-title>OWL Experiences and Directions</article-title>
          . Eight International Workshop, San Francisco, California, USA, June 5-6,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <given-names>S.</given-names>
            <surname>Schulz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Martínez-Costa</surname>
          </string-name>
          ,
          <article-title>Harmonizing SNOMED CT with BioTopLite: An Exercise in Principled Ontology Alignment</article-title>
          .
          <source>Stud Health Technol Inform</source>
          .
          <volume>216</volume>
          (
          <year>2015</year>
          ),
          <fpage>832</fpage>
          -
          <lpage>836</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <given-names>P.</given-names>
            <surname>López-García</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Schulz</surname>
          </string-name>
          ,
          <article-title>Can SNOMED CT be squeezed without losing its shape</article-title>
          ?
          <source>J Biomed Semantics</source>
          <volume>7</volume>
          (
          <year>2016</year>
          ),
          <fpage>56</fpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>