<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Visualizing ontologies with AberOWL</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Miguel Angel Rodr guez-Garc a</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Luke Slater</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Keiron O'Shea</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Paul N Scho eld</string-name>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Georgios V Gkoutos</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Robert Hoehndorf</string-name>
          <email>robert.hoehndorfg@kaust.edu.sa</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Aberystwyth University</institution>
          ,
          <addr-line>Aberystwyth, SY23 3DB, Wales</addr-line>
          ,
          <country country="UK">UK</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Computational Bioscience Research Center, King Abdullah University of Science and Technology</institution>
          ,
          <addr-line>Thuwal 23955-6900, KSA</addr-line>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>University of Cambridge</institution>
          ,
          <addr-line>Downing Street, CB2 3EG, England</addr-line>
          ,
          <country country="UK">UK</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Ontologies are formal theories that specify the kinds of entities and relations found in a domain. To quickly gain access to the content and structure of ontologies, ontology visualization techniques are commonly used. Visualization of ontologies often uses representations of hierarchical structures that are extracted from ontologies, most notably representations of the taxonomic relationships between classes. These graph-based representations can also be used to visualize structural changes in ontologies. We have developed a novel visualization environment for ontologies in which automated reasoning is used to generate a graph-based representation of an ontology's deductive closure, and subclass relations as well as description logic axioms that are entailed to hold between two classes are represented visually. The visualization environment can also be used to show di erences between the entailed axioms of di erent ontology versions. The source code of the visualization environment is freely available, and we added our visualization environment to AberOWL (http://aber-owl.net), an ontology repository that contains over 400 ontologies, all of which can now be visually explored using our system.</p>
      </abstract>
      <kwd-group>
        <kwd>biomedical ontology</kwd>
        <kwd>visualization</kwd>
        <kwd>automated reasoning</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        In recent years, a large number of ontologies has been developed across many
scienti c domains. These ontologies are often formalized in languages such as
the Web Ontology Language (OWL) [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] or an OWL-compatible language such
as the OBO Flat le Format [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ]. The major role of ontologies in biology and
biomedicine is in data integration, as they formally describe the kinds of
biological entities found within a domain, and their interrelations, and can therefore
be used to provide semantic annotations that can be shared across databases.
      </p>
      <p>
        Along with the increase in the number of ontologies, the need to develop
tools that enable both ontology experts and domain experts to interact with
ontologies has grown as well. One crucial aspect of interacting with ontologies is
the ability to browse and visualize the content of ontologies. A widely used form
of visualization for ontologies are graphs that represent classes and the axioms
that hold between these classes. This form of representation is used in ontology
editors such as Protege [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ] or the (now abandoned) OBO-Edit [
        <xref ref-type="bibr" rid="ref24">24</xref>
        ], as well as
in ontology repositories such as BioPortal [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ], OntoBee [
        <xref ref-type="bibr" rid="ref31">31</xref>
        ] or the Ontology
Lookup Service (OLS) [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. There are two key features based on which methods
for visualizing ontologies as graph di er: the kind of `relations' that are shown
between classes as part of the graph structure, and whether only the asserted
axioms are used to generate the graph structure or the inferences that can be
drawn from these axioms.
      </p>
      <p>
        Relations between classes [
        <xref ref-type="bibr" rid="ref26">26</xref>
        ] have traditionally been used for biological and
biomedical ontologies, with the intention to represent axiom patterns that hold
between two classes. In its simplest form, a relation is-a between two classes
X and Y expresses a subclass axiom that holds between the two classes.
However, many ontologies employ more complex axiom patterns, such as the Part-of
pattern between two classes: X and Y are said to stand in the relation Part-of
if and only if X is a subclass of part-of some Y [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. In ontology repositories
and ontology editors, these kinds of relations are rarely shown, with OLS and
OBO-Edit as exceptions.
      </p>
      <p>Another key distinguishing feature is whether only asserted axioms in the
ontology are visualized or also inferred statements. In Protege, and to some
degree in OBO-Edit, it is possible to explore inferred relations between classes
visually. In Protege, these relations are limited to subclass axioms, while
OBOEdit is also able to show other kinds of relations.</p>
      <p>Finally, visualization can also aid to structurally identify di erences between
ontologies, or between di erent versions of one ontology. Similarly, a key
component in exploring and visualizing di erences in ontology versions is whether only
syntactic changes are identi ed or whether di erences are also identi ed based
on inferred axioms.</p>
      <p>
        Here, we present an ontology visualization environment that provides a
simple and intuitive way to represent classes in ontologies and their interrelations.
The visualization environment employs an automated reasoner to identify axiom
patterns that hold between two classes, thereby visualizing the inferences that
can be drawn from an ontology, including complex patterns that represent more
than simple subclass relations. The environment can also be used to visualize
multiple ontologies at the same time, thereby enabling the exploration of di
erences between ontologies and ontology versions. The visualization environment
we developed is integrated in the AberOWL ontology repository [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ], available at
http://aber-owl.net, which currently provides access to over 400 ontologies, and
thereby allows exploring these ontologies, and their di erent versions, visually.
      </p>
    </sec>
    <sec id="sec-2">
      <title>A brief overview of AberOWL</title>
      <p>AberOWL is an ontology repository and framework for ontology-based data
access. It allows access to hundreds of ontologies using automated reasoning
through a web interface and a REST API. AberOWL is constituted of three
main modules: the AberOWL server, the AberOWL synchronization service,
and the AberOWL web repository.</p>
      <p>
        The AberOWL server provides the core of the system. It uses the ELK
reasoner [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ], an OWL reasoner supporting the OWL EL pro le [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ], to ensure
polynominal-time reasoning and querying. The ELK reasoner is fast enough for
many practical uses even when applied to large ontologies [
        <xref ref-type="bibr" rid="ref25">25</xref>
        ]. The reasoner
is used to classify each ontology and the server maintains a classi ed version
of each ontology in memory. From there, it provides a JSON-based REST API
for interacting with the ontologies loaded. In particular, the AberOWL server
o ers the possibility to query one or all ontologies by transforming a Manchester
OWL Syntax [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] query string into an OWL class expression using the OWL API
and retrieving its sub-, super- or equivalent classes. Additionally, the AberOWL
server uses Apache Lucene [
        <xref ref-type="bibr" rid="ref29">29</xref>
        ] to create an index of all class and relation labels,
synonyms, descriptions, and all other annotation properties, thereby allowing
fast retrieval of classes and relations through substring-based search.
      </p>
      <p>
        The AberOWL synchronization module integrates a service that monitors
other ontology repositories for new ontologies as well as new versions of existing
ontologies, and incorporates them into the AberOWL server. Currently, only
the BioPortal repository [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ] is monitored, which contains, amonst others, all
the OBO Foundry ontologies [
        <xref ref-type="bibr" rid="ref27">27</xref>
        ].
      </p>
      <p>The AberOWL web repository provides a web-based front end which
constitutes the main user-interface for the repository. Its function is to allow users to
interact with the AberOWL server, providing the possibility to query, browse,
download, and visualize explore ontologies. It also makes a set of services built
on top of AberOWL available to users, such as SPARQL query expansion or
ontology-based PubMed searches.
3</p>
    </sec>
    <sec id="sec-3">
      <title>Visualizing ontologies in AberOWL</title>
      <p>We developed a visualization environment for ontologies in AberOWL that can
visualize inferences drawn from ontologies, visualize both the subclass hierarchy
as well as other types of relations between classes, and which can show the di
erences between the inferences drawn from di erent versions of an ontology. The
aim is to provide an intuitive and easy-to-use method to explore the structure
and inferences of ontologies in AberOWL, and visualization of the ontologies is
done in real time using the AberOWL reasoning infrastructure.</p>
      <p>In AberOWL, ontologies are visualized as directed graphs in which nodes
represent classes and edges represent axioms that are inferred to hold between
two classes. The subclass hierarchy of an ontology is always shown, and
generated by dynamically using the AberOWL reasoning services to query for direct
subclasses. A subclass edge is created between two nodes representing classes C
and D in ontology O if and only if C SubClassOf: D can be inferred from the
ontology O and there exists no other class E such that both C SubClassOf:
E and E SubClassOf: D. The root of the subclass hierarchy is owl:Thing, and
ontologies are initially visualized by querying for direct subclasses of owl:Thing
using AberOWL. Whenever a user expands a node (by clicking on it) that
represents class C, AberOWL is queried for direct subclass of C and the results
of the query are generated dynamically as new nodes and linked to the node
representing C through directed subclass edges.</p>
      <p>
        To visualize axioms that represent more complex patterns, we follow the
relational patterns proposed in the OBO Relation Ontology [
        <xref ref-type="bibr" rid="ref26">26</xref>
        ] and its
corresponding approximation in OWL [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. In particular, we identify the set of object
properties that occur in an ontology O, and for each object property R in O,
we generate a pattern of the type X SubClassOf: R some Y, where X and Y
are variables standing for classes [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. Given a node representing the class C in
the ontology O, we dynamically generate an R-successor D of this node (with
an R-labeled directed edge) if and only if D is a direct subclass of R some C
in O. For example, to show part-of successors of the class Apoptosis in the
Gene Ontology, we generate the class description part-of some Apoptosis,
use AberOWL to query for the direct subclasses of part-of some Apoptosis,
and dynamically generate a new node for each of the resulting classes together
with a part-of-labeled edge from Apoptosis to this new node.
      </p>
      <p>In AberOWL, we can also simultaneously visualize multiple ontologies within
the same visualization environment. This is particularly useful to explore di
erences between multiple versions of the same ontology. AberOWL maintains older
versions of ontologies in its repository; however, these versions are not, by
default, accessible through automated reasoning. Therefore, when a user request is
made to visualize an older version of an ontology, the AberOWL server will rst
classify this version so that queries can be answered using an automated
reasoner. To allow faster subsequent queries to ontology versions, a classi ed model
of these ontologies is kept in memory until it has not been queried for at least
90 minutes, at which time it is removed. We then use our visualization
environment to show the subclass hierarchy as well as complex axiom patterns for two
or more ontology versions simultaneously. If classes are shared between ontology
versions, they are represented by the same node; if axiom patterns between two
classes hold in two versions of an ontology, they are represented by the same
edge. On the other hand, if axiom patterns or subclasses (of class descriptions)
di er between versions, multiple di erent nodes and edges are created and
visually distinguished through colors. This allows to visually explore di erences in
the inferences that can be drawn from di erent ontology versions.</p>
      <p>To visually di erentiate the origin of the each node (i.e., the ontology version
in which it is present), we color-code ontology versions; we further color-code
object properties. To further improve usability of the interface should multiple
versions and object properties be selected, we add tooltips to nodes and edges
that show the ontology version in which they appear and the kind of axiom
pattern that is represented by the edge.
4</p>
    </sec>
    <sec id="sec-4">
      <title>Implementation</title>
      <p>Our visualization environment is implemented in JavaScript and utilizes the
AberOWL reasoning services. Ontologies are visualized through several recursive
functions, allowing accurate control over the growth of the tree. As inputs, the
implementation of the algorithm requires:
{ The root node of the ontology; by default, owl:Thing is used for all
ontologies.
{ The ID (or URI) of the ontology to visualize.
{ A list of versions of the selected ontology that are visualized in parallel.
{ A list of the object properties in the ontology, from which we generate axiom
patterns and visualize them as additional edges.</p>
      <p>Furthermore, the ontology visualization environment can be con gured with
additional parameters:
{ The number of children that are shown for each ontology level; in case a
node has many successors, only a subset of the successors is shown while the
other nodes can be shown on request. This allows us to limit the number of
classes that are shown in order to improve usability.
{ The number of levels that are expanded through a single request; this allows
us to show more than just direct successors of a node in a single request.
{ The number of hierarchical levels that will be pre-loaded from the AberOWL
server during the ontology visualization. The goal of this parameter is to
optimize the load time when users expand additional nodes.</p>
      <p>We use the JavaScript Promise pattern and AJAX together with the AberOWL
REST API to generate new nodes and edges based on user requests, and to
preload nodes that users may want to expand further. We use the D3j_s graph
library to generate the resulting graphs. In particular, we use node-link diagrams,
implemented in D3j_s, to represent the ontologies in AberOWL. Node-link
diagrams can be used to visualize both acyclic and cyclic graphs and therefore
allows us the exibility to visualize multiple types of relational patterns
between classes. Whenever a user changes the choice of which relational patterns
or which ontology versions to display, the visualization environment interacts
with the AberOWL server in order to regenerate the graph based on the user's
selections.
5
5.1</p>
    </sec>
    <sec id="sec-5">
      <title>Discussion</title>
      <sec id="sec-5-1">
        <title>Comparison to related work</title>
        <p>The signi cant increase in number of ontologies available online has stimulated
the need among the research community to develop visualization tools which
support their navigation. However, the visualization of ontologies is not an easy
task, since ontologies are expressed as formal theories (i.e., sets of axioms) from
which inferences can be drawn, and visualizing the kind of inferences is
challenging.</p>
        <p>
          One way for classifying visualization methods is the number of dimensions
used to represent the ontology. Visualization approaches such as OntoSphere [
          <xref ref-type="bibr" rid="ref1">1</xref>
          ]
and Onto3DViz [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ] use three dimensions to visualize ontologies, while methods
employed in most ontology editors like Protege [
          <xref ref-type="bibr" rid="ref22">22</xref>
          ] or OBO-Edit [
          <xref ref-type="bibr" rid="ref24">24</xref>
          ], and
specialized visualization methods such as KC-Viz [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ] , OWLViz [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ] and GrOWL
[
          <xref ref-type="bibr" rid="ref17">17</xref>
          ], utilize a two-dimensional representation.
        </p>
        <p>
          The structure of ontologies can be visualized in two-dimensional space using
several di erent methods [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ]. However, the main aim of visualization of
ontologies is often to e ectively present hierarchical structures to users [
          <xref ref-type="bibr" rid="ref30">30</xref>
          ], and,
consequently, the most widely used visualization forms are hierarchical graphs
or treemaps [
          <xref ref-type="bibr" rid="ref11 ref28 ref3">11, 3, 28</xref>
          ].
        </p>
        <p>
          The graph representation in ontologies can be a taxonomy (induced by
subclass relations between classes in the ontology), or a representation of other types
of relations (i.e., axiom patterns that hold) between classes [
          <xref ref-type="bibr" rid="ref26 ref8">26, 8</xref>
          ]. Strategies for
visualizing ontologies also di er in the types of relations between classes that
can be visualized. In ontology editors, for most parts the subclass relations in
an ontology are shown while other types of axioms that hold between classes are
rarely visualized. A prominent exception has been OBO-Edit [
          <xref ref-type="bibr" rid="ref24">24</xref>
          ], an ontology
editor intended for use by biological domain experts and based on the OBO
Flatle Format, which could show di erent types of relations between classes beyond
subclass relations. However, development on OBO-Edit has recently been
abandoned in favor of ontology development environments that are based more on
OWL, which rarely show relations other than subclass relations. Our approach
can be used to generate graphs that represent any kind of axiom pattern in
which two classes occur as variables.
        </p>
        <p>A further distinction between visualization methods is whether they are able
to visualize the asserted structure of an ontology or if they can also visualize
the ontologies' inferred structure. The Protege ontology editor, for example, is
able to visualize both asserted subclass relations and inferred subclass relations.
In AberOWL, only the inferred (subclass or other types of) relations between
classes are visualized.</p>
        <p>
          Ontologies are not static and will evolve due to extensions in their
application domain or changes in the shared conceptualization, changes in the scienti c
knowledge of the domain, or correction of mistakes [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ]. As a result of this
evolution, di erent versions of the same ontology arise, and it is often useful to
visualize the di erences between di erent versions to understand the changes that may
be necessary in applying the ontology within a use case. Research on ontology
versioning and ontology evolution has focused on providing collaborative tools
for editing ontologies. For instance, PromptDi [
          <xref ref-type="bibr" rid="ref20">20</xref>
          ] is an ontology-versioning
environment which, among others functions, is able to track structural changes
of di erent versions of the ontology; OntoView [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ] provides a methodology for
ontology versioning which allows users specify relations between versions of
ontologies; PROMPT-Viz [
          <xref ref-type="bibr" rid="ref23">23</xref>
          ] which is a Protege plugin which provides advanced
visualization of location, impact, type and extent of changes that have occurred
between versions on an ontology; COntoDi [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ] tracks changes across multiple
versions of ontologies; and the di tool Ecco [
          <xref ref-type="bibr" rid="ref4">4</xref>
          ] that incorporates structural and
semantic techniques that allows to distinguish e ectual and ine ectual changes
between ontologies. In AberOWL, we visualize the changes of di erent versions
of ontologies across multiple versions, and using the AberOWL system for
automated reasoning, we can visualize not only the direct syntactic changes to
an ontology but also their impact of the inferences that can be drawn from
them. Table 1 provides an overview over the main features of di erent ontology
visualization approaches.
5.2
        </p>
      </sec>
      <sec id="sec-5-2">
        <title>Conclusions</title>
        <p>
          We developed a novel visualization environment for biological and biomedical
ontologies, and integrated that environment in the AberOWL ontology
repository. Using this visualization environment, it is possible to visualize the inferred
structure of one ontology, including the structure induced by subclass relations
as well as arbitrary axiom patterns that hold between classes [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ]. Furthermore,
we can visualize multiple versions of a single ontology at once, thereby
allowing users to explore structural changes between ontology version. All structural
relations between classes in the visualization environment are generated using
an OWL reasoner, thereby allowing users to explore the inferences that can be
drawn from the ontologies, or their di erent versions. Our work also
demonstrates that the AberOWL system can be used as a service that enables the
development of novel kinds of semantic applications using automated reasoning
and semantic querying.
        </p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Bosca</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bonino</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pellegrino</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Ontosphere: more than a 3d ontology visualization tool</article-title>
          . In: Swap (
          <year>2005</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Cote</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jones</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Apweiler</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hermjakob</surname>
          </string-name>
          , H.:
          <article-title>The ontology lookup service, a lightweight cross-platform tool for controlled vocabulary queries</article-title>
          .
          <source>BMC Bioinformatics</source>
          <volume>7</volume>
          (
          <issue>1</issue>
          ),
          <volume>97</volume>
          + (
          <year>2006</year>
          ), http://dx.doi.org/10.1186/
          <fpage>1471</fpage>
          -2105-7-97
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Garc</surname>
            a-Pen~alvo,
            <given-names>F.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Colomo-Palacios</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Garc</surname>
            <given-names>a</given-names>
          </string-name>
          , J.,
          <string-name>
            <surname>Theron</surname>
          </string-name>
          , R.:
          <article-title>Towards an ontology modeling tool. a validation in software engineering scenarios</article-title>
          .
          <source>Expert Systems with Applications</source>
          <volume>39</volume>
          (
          <issue>13</issue>
          ),
          <volume>11468</volume>
          {
          <fpage>11478</fpage>
          (
          <year>2012</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Goncalves</surname>
            ,
            <given-names>R.S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Parsia</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sattler</surname>
            ,
            <given-names>U.</given-names>
          </string-name>
          :
          <article-title>Ecco: A hybrid di tool for owl 2 ontologies</article-title>
          . In: OWLED (
          <year>2012</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Grau</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Horrocks</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Motik</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Parsia</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Patelschneider</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sattler</surname>
            ,
            <given-names>U.</given-names>
          </string-name>
          :
          <article-title>OWL 2: The next step for OWL</article-title>
          .
          <source>Web Semantics: Science, Services and Agents on the World Wide Web</source>
          <volume>6</volume>
          (
          <issue>4</issue>
          ),
          <volume>309</volume>
          {322 (November
          <year>2008</year>
          ), http://dx.doi.org/10.1016/j.websem.
          <year>2008</year>
          .
          <volume>05</volume>
          .001
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Guo</surname>
            ,
            <given-names>S.S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chan</surname>
            ,
            <given-names>C.W.:</given-names>
          </string-name>
          <article-title>A tool for ontology visualizaiton in 3d graphics: Onto3dviz</article-title>
          . In: Electrical and Computer Engineering (CCECE),
          <year>2010</year>
          23rd Canadian Conference on. pp.
          <volume>1</volume>
          {
          <issue>4</issue>
          .
          <string-name>
            <surname>IEEE</surname>
          </string-name>
          (
          <year>2010</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Hartung</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gross</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rahm</surname>
          </string-name>
          , E.:
          <string-name>
            <surname>COnto-Di</surname>
          </string-name>
          :
          <article-title>Generation of Complex Evolution Mappings for Life Science Ontologies</article-title>
          .
          <source>Journal of Biomedical Informatics</source>
          <volume>46</volume>
          ,
          <issue>15</issue>
          {
          <fpage>32</fpage>
          (
          <year>2013</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Hoehndorf</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Oellrich</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dumontier</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kelso</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rebholz-Schuhmann</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Herre</surname>
          </string-name>
          , H.:
          <article-title>Relations as patterns: Bridging the gap between OBO and OWL</article-title>
          .
          <source>BMC Bioinformatics</source>
          <volume>11</volume>
          (
          <issue>1</issue>
          ),
          <volume>441</volume>
          + (
          <year>2010</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Hoehndorf</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Slater</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Scho</surname>
            <given-names>eld</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>P.N.</given-names>
            ,
            <surname>Gkoutos</surname>
          </string-name>
          ,
          <string-name>
            <surname>G.V.</surname>
          </string-name>
          :
          <article-title>Aber-OWL: a framework for ontology-based data access in biology</article-title>
          .
          <source>BMC Bioinformatics</source>
          <volume>16</volume>
          ,
          <issue>26</issue>
          (
          <year>2015</year>
          ), http://www.biomedcentral.com/1471-2105/16/26/abstract
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Horridge</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Drummond</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Goodwin</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rector</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stevens</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wang</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          :
          <article-title>The Manchester OWL Syntax</article-title>
          .
          <source>Proc. of the 2006 OWL Experiences and Directions Workshop (OWL-ED2006)</source>
          (
          <year>2006</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Horridge</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <string-name>
            <surname>Owlviz</surname>
          </string-name>
          (
          <year>2004</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Horrocks</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          :
          <article-title>OBO at le format syntax and semantics and mapping to OWL Web Ontology Language</article-title>
          .
          <source>Tech. rep.</source>
          , University of Manchester (March
          <year>2007</year>
          ), http://www.cs.man.ac.uk/ horrocks/obo/
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Katifori</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Halatsis</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lepouras</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vassilakis</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Giannopoulou</surname>
          </string-name>
          , E.:
          <article-title>Ontology visualization methodsa survey</article-title>
          .
          <source>ACM Computing Surveys (CSUR) 39(4)</source>
          ,
          <volume>10</volume>
          (
          <year>2007</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Kazakov</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          , Krotzsch,
          <string-name>
            <given-names>M.</given-names>
            ,
            <surname>Simancik</surname>
          </string-name>
          ,
          <string-name>
            <surname>F.</surname>
          </string-name>
          :
          <article-title>The incredible elk</article-title>
          .
          <source>Journal of Automated Reasoning</source>
          <volume>53</volume>
          (
          <issue>1</issue>
          ),
          <volume>1</volume>
          {
          <fpage>61</fpage>
          (
          <year>2014</year>
          ), http://dx.doi.org/10.1007/s10817-013-9296-3
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Klein</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fensel</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kiryakov</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ognyanov</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>Ontology versioning and change detection on the web</article-title>
          . In:
          <article-title>Knowledge Engineering and Knowledge Management: Ontologies and the Semantic Web</article-title>
          , pp.
          <volume>197</volume>
          {
          <fpage>212</fpage>
          . Springer (
          <year>2002</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Klein</surname>
            ,
            <given-names>M.C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fensel</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>Ontology versioning on the semantic web</article-title>
          .
          <source>In: SWWS</source>
          . pp.
          <volume>75</volume>
          {
          <issue>91</issue>
          (
          <year>2001</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>Krivov</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Williams</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Villa</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          :
          <article-title>Growl: A tool for visualization and editing of owl ontologies</article-title>
          .
          <source>Web Semantics: Science, Services and Agents on the World Wide Web</source>
          <volume>5</volume>
          (
          <issue>2</issue>
          ),
          <volume>54</volume>
          {
          <fpage>57</fpage>
          (
          <year>2007</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Motik</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Grau</surname>
            ,
            <given-names>B.C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Horrocks</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wu</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fokoue</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lutz</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>Owl 2 web ontology language: Pro les</article-title>
          .
          <source>Recommendation, World Wide Web Consortium (W3C)</source>
          (
          <year>2009</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>Motta</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mulholland</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Peroni</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , dAquin,
          <string-name>
            <given-names>M.</given-names>
            ,
            <surname>Gomez-Perez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.M.</given-names>
            ,
            <surname>Mendez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            ,
            <surname>Zablith</surname>
          </string-name>
          ,
          <string-name>
            <surname>F.</surname>
          </string-name>
          :
          <article-title>A novel approach to visualizing and navigating ontologies</article-title>
          .
          <source>In: The Semantic Web{ISWC</source>
          <year>2011</year>
          , pp.
          <volume>470</volume>
          {
          <fpage>486</fpage>
          . Springer (
          <year>2011</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <surname>Noy</surname>
            ,
            <given-names>N.F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kunnatur</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Klein</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Musen</surname>
            ,
            <given-names>M.A.</given-names>
          </string-name>
          :
          <article-title>Tracking changes during ontology evolution</article-title>
          .
          <source>In: The Semantic Web{ISWC</source>
          <year>2004</year>
          , pp.
          <volume>259</volume>
          {
          <fpage>273</fpage>
          . Springer (
          <year>2004</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21.
          <string-name>
            <surname>Noy</surname>
            ,
            <given-names>N.F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shah</surname>
            ,
            <given-names>N.H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Whetzel</surname>
            ,
            <given-names>P.L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dai</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dorf</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gri</surname>
            <given-names>th</given-names>
          </string-name>
          , N.,
          <string-name>
            <surname>Jonquet</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rubin</surname>
            ,
            <given-names>D.L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Storey</surname>
            ,
            <given-names>M.A.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chute</surname>
            ,
            <given-names>C.G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Musen</surname>
            ,
            <given-names>M.A.</given-names>
          </string-name>
          :
          <article-title>Bioportal: ontologies and integrated data resources at the click of a mouse</article-title>
          .
          <source>Nucleic acids research</source>
          37(
          <issue>Web Server issue</issue>
          ),
          <source>W170{173 (July</source>
          <year>2009</year>
          ), http://dx.doi.org/10.1093/nar/gkp440
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22.
          <string-name>
            <surname>Noy</surname>
            ,
            <given-names>N.F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sintek</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Decker</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Crubezy</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fergerson</surname>
            ,
            <given-names>R.W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Musen</surname>
            ,
            <given-names>M.A.</given-names>
          </string-name>
          :
          <article-title>Creating semantic web contents with Protege-2000</article-title>
          .
          <source>IEEE Intelligent Systems</source>
          <volume>16</volume>
          (
          <issue>2</issue>
          ),
          <volume>60</volume>
          {71 (March/April 2001)
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          23.
          <string-name>
            <surname>Perrin</surname>
            ,
            <given-names>D.S.J.</given-names>
          </string-name>
          :
          <article-title>Prompt-viz: Ontology version comparison visualizations with treemaps</article-title>
          .
          <source>Ph.D. thesis</source>
          , Citeseer (
          <year>2004</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          24.
          <string-name>
            <surname>Richter</surname>
            ,
            <given-names>J.D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Harris</surname>
            ,
            <given-names>M.A.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Haendel</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lewis</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          :
          <article-title>Obo-edit - an ontology editor for biologists</article-title>
          .
          <source>Bioinformatics (June</source>
          <year>2007</year>
          ), http://dx.doi.org/10.1093/bioinformatics/btm112
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          25.
          <string-name>
            <surname>Slater</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gkoutos</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <article-title>Scho eld</article-title>
          ,
          <string-name>
            <given-names>P.N.</given-names>
            ,
            <surname>Hoehndorf</surname>
          </string-name>
          ,
          <string-name>
            <surname>R.</surname>
          </string-name>
          :
          <article-title>Using aber-owl for fast and scalable reasoning over bioportal ontologies</article-title>
          .
          <source>In: Proceedings of International Conference on Biomedical Ontologies (ICBO)</source>
          . pp.
          <volume>72</volume>
          {
          <issue>76</issue>
          (
          <year>July 2015</year>
          ), http://icbo2015.fc.ul.pt/ICBO2015Proceedings.pdf
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          26.
          <string-name>
            <surname>Smith</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ceusters</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Klagges</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          , Kohler, J.,
          <string-name>
            <surname>Kumar</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lomax</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mungall</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Neuhaus</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rector</surname>
            ,
            <given-names>A.L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rosse</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>Relations in biomedical ontologies</article-title>
          .
          <source>Genome Biol</source>
          <volume>6</volume>
          (
          <issue>5</issue>
          ),
          <source>R46</source>
          (
          <year>2005</year>
          ), http://dx.doi.org/10.1186/gb-2005
          <source>-6-5-r46</source>
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          27.
          <string-name>
            <surname>Smith</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ashburner</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rosse</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bard</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bug</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ceusters</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Goldberg</surname>
            ,
            <given-names>L.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Eilbeck</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ireland</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mungall</surname>
            ,
            <given-names>C.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Leontis</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Serra</surname>
            ,
            <given-names>P.R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ruttenberg</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sansone</surname>
            ,
            <given-names>S.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Scheuermann</surname>
            ,
            <given-names>R.H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shah</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Whetzel</surname>
            ,
            <given-names>P.L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lewis</surname>
            ,
            <given-names>S.:</given-names>
          </string-name>
          <article-title>The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration</article-title>
          .
          <source>Nat Biotech</source>
          <volume>25</volume>
          (
          <issue>11</issue>
          ),
          <volume>1251</volume>
          {
          <fpage>1255</fpage>
          (
          <year>2007</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          28.
          <string-name>
            <surname>Storey</surname>
            ,
            <given-names>M.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Musen</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Silva</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Best</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ernst</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fergerson</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Noy</surname>
          </string-name>
          , N.:
          <article-title>Jambalaya: Interactive visualization to enhance ontology authoring and knowledge acquisition in protege</article-title>
          . In: Workshop on
          <article-title>Interactive Tools for Knowledge Capture (K-CAP-</article-title>
          <year>2001</year>
          ). p.
          <fpage>93</fpage>
          .
          <string-name>
            <surname>Citeseer</surname>
          </string-name>
          (
          <year>2001</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          29.
          <article-title>The Apache Software Foundation: Apache Lucene</article-title>
          . http://lucene.apache.org
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          30.
          <string-name>
            <surname>Wang</surname>
            ,
            <given-names>T.D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Parsia</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          :
          <article-title>CropCircles: topology sensitive visualization of OWL class hierarchies</article-title>
          . Springer (
          <year>2006</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          31.
          <string-name>
            <surname>Xiang</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mungall</surname>
            ,
            <given-names>C.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ruttenberg</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>He</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          :
          <article-title>Ontobee: A linked data server and browser for ontology terms</article-title>
          .
          <source>In: Proceedings of International Conference on Biomedical Ontology</source>
          . pp.
          <volume>279</volume>
          {
          <issue>281</issue>
          (
          <year>2011</year>
          )
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>