<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>OBOPedia: An Encyclopaedia of Biology Using OBO Ontologies</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Robert Stevens</string-name>
          <email>robert.stevens@manchester.ac.uk</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Adam Nogradi</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>School of Computer Science, University of Manchester</institution>
          ,
          <country country="UK">UK</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Ontologies contain knowledge about a domain for use by tools or humans. A source of knowledge should be usable by a human to 'find out about' or learn about the entities of a domain and their relationship to each other. The corpus of biomedical ontologies now contains 'encyclopaedic' knowledge about biology and should be capable of being used by humans to learn about entities in molecular biology. Yet multiple separate ontologies and the typical style of presentation of the knowledge in the ontologies mean that their use as a learning resource is sub-optimal. To address this issue we have created OBOPedia, a web based encyclopaedia of biology as seen by the Open Biomedical Ontologies (OBO) Consortium. OBOPedia exploits the OBO's use of standard representations and meaningful human readable terms and natural language definitions to create a basic OBOPedia encyclopaedia entry. An entry is supplemented with an ontology's synonyms and uses the ontology's taxonomic links to provide 'see also' cross-references within the alphabetical list of entries. Currently, OBOPedia has access to ten OBO ontologies, including all the OBO Foundry ontologies, which have a total of over 210,000 entries. Our evaluations indicate that an OBOPedia style of presentation has a role as an alternative way of presenting knowledge of a domain collected as an ontology or ontologies. OBOPedia offers another view into a field of interest that is based on a collection of ontologies as a reference resource, but one in which a user need not know it is an ontology. OBOPedia may be used via http://www.obopedia.org.uk. The source code and documentation for OBOPedia are available via https://bitbucket. org/adam944/ontologyencyclopaedia.</p>
      </abstract>
      <kwd-group>
        <kwd>Encyclopaedia</kwd>
        <kwd>Open Biomedical Ontologies</kwd>
        <kwd>Natural Language Definitions</kwd>
        <kwd>User Interface</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        We present OBOPedia, which offers the knowledge of biology captured in a corpus
of ontologies, as an encyclopaedia whose intended audience are those seeking to learn
about biological entities and are not experts in ontologies or wishing to use knowledge
in the form of an ontology. An ontology captures what is universally true about a field of
interest [15]. Ontologies contain representations of the entities in a domain, their logical
definitions, descriptions and how they are related to one another [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. An ontology may
also be decorated with natural language information such as non-logical definitions and
the labels for entities used for display. As such, ontologies should be able to be used
effectively as a source of knowledge about a field of interest by humans and machines,
by ontology experts and people unaware of ontologies; an ontology talks about ‘things’
in a domain and should be usable by a range of users to find out about ‘things’ in that
field of interest without them necessarily being au fait with ontologies.
      </p>
      <p>
        The field of biology has captured a lot of its knowledge in the form of
ontologies [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. BioPortal [14], for example, now has over 400 ontologies covering many
subdomains of biology. The Open Biomedical Ontologies [16] (OBO) has attempted to
co-ordinate the development of ontologies from genotype to phenotype and outwards
to small molecules, scientific investigations, environments and so on; there are 1331
ontologies available via the OBO Foundry [16] website2.
      </p>
      <p>
        This collection of biological knowledge has many uses, mainly for annotation of
data that facilitates search and retrieval via query generalisation and analysis of data
through techniques such as over-expression analysis [
        <xref ref-type="bibr" rid="ref9">17, 9</xref>
        ]. Most of this is facilitated
by ontologies acting as a shared vocabulary for describing data and then exploiting the
structure in which that vocabulary is held.
      </p>
      <p>
        Despite an ontology being a mechanism for capturing and representing knowledge,
ontologies do not appear to be used as a source of knowledge for learning about a
domain by humans. The TAMBIS Ontology [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] offered itself as a tutorial for molecular
biology and related bioinformatics, but did not realise this function. The widespread
development of ontologies across biomedicine means there is a knowledge resource
that spans broad parts of the subject; enabling this valuable resource to be used as
knowledge to learn about biology should be a useful resource.
      </p>
      <p>OBO ontologies are created in a common representation style, down to what
relationships and annotation types are used. All OBO classes should have human readable
labels, synonyms for those labels and natural language definitions.</p>
      <p>
        An example of labels and natural language definition for the class GO:0005794
from the Gene Ontology (GO) [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] are shown in Table 1. The structure of the ontology
gives more general and more specialised concepts (along with definitions and labels).
Here the subsumption and other relationships say what is universally true about this
class of objects; in GO each and every golgi apparatus is also a kind of ‘intracellular
membrane-bounded organelle’ as well as being a ‘cytoplasmic part’ and each and every
golgi apparatus is also part of an ‘endomembrane system’. The natural language
definition fragment in Table 1 contains much more ‘background’ knowledge, sometimes
knowledge that is not universally true about the entities in the ontology. This broader
knowledge may still be useful for someone wishing to explore and understand a field
of interest. In this sense, the universal, logical knowledge in an ontology, together with
the wider ranging knowledge in the natural language definitions can be said to form
‘background’ knowledge [15].
      </p>
      <p>An encyclopaedia is “a type of reference work holding a comprehensive summary
of information from either all branches of knowledge or a particular branch of
knowledge” [19]. An encyclopaedia consists of entries, containing concise, factual
descrip1 11 OBO Foundry ontology and 122 candidate ontologies and others of interest (http://
www.obofoundry.org/ on 23 June 2015).
2 http://www.obofoundry.org/</p>
      <p>A compound membranous cytoplasmic organelle of eukaryotic cells,
consisting of flattened, ribosome-free vesicles arranged in a more or less
regular stack. The Golgi apparatus differs from the endoplasmic reticulum [. . . ]
tions about entities. This description of an encyclopeadia entry looks remarkably like
an OBO class’ descriptive metadata (see Table 1); so a collection of OBO entities and
their metadata could be presented as an encyclopaedia of biology. OBOPedia’s attempt
at such a presentation for the entry in Table 1 is shown in Figure 1.</p>
      <p>The knowledge captured within the OBO is now encyclopaedic. To capitalise on this
resource as a source of background knowledge about biology we present the content of
some OBO in the form of an encyclopaedia as OBOPedia (see Figure 1). OBOPedia’s
value is in the ability to hide away everything that is unnecessary to show in a simple
encyclopaedia. Users can browse a corpus of ontologies’ knowledge about a field of
interest by reading definitions, search for terms and their synonyms, or explore super- and
subclasses of terms using the ‘see-also’ links next to them. OBOPedia is made by using
each class’ labels, its synonyms, its natural language definition as the encyclopaedia
entry text, and the relationships held by the entity as the ‘see also’ links of an entry.</p>
    </sec>
    <sec id="sec-2">
      <title>Method</title>
      <p>The building of OBOPedia’s content is straight forward:
– Select the OBO ontologies that may form OBOPedia’s content.
– Remove all obsolete classes.
– Create an encyclopaedia object aggregating the concept’s label, synonyms, natural
language definition, origin and sub- and super-class relationships (see table 1).
– Index on the labels using Apache Solr3, normalising to the first alphabetic character
of the label.
– Display using OBOPedia’s user interface.</p>
      <p>
        WE took the OBO Foundry ontologies in their OWL representation from http:
//purl.obolibrary.org/obo/xxx.owl (where ‘xxx’ is the ontology’s short
name—GO for the Gene Ontology and ChEBI for the Chemicals of Biological Interest
ontology) in May 2015. Processed using the OWL API version 3.5.0 [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. The list of
ontologies used in this version of OBOPedia is:
1. The Gene Ontology (GO)—the Molecular Function, Biological Process and
Cellular Component ontologies.
2. The Protein Ontology (PrO).
3. Chemical Entities of Biological Interest ontology (ChEBI).
4. The Human Disease Ontology (HDO).
5. The Human Phenotype Ontology (HPO).
6. The Ontology of Biomedical Investigations (OBI).
7. the Plant Ontology (PO).
8. The Phenotypic quality Ontology (PATO).
9. The Xenopus Anatomy and development Ontology (XAO).
10. The Zebrafish Anatomy and Development Ontology (ZFA).
      </p>
      <p>OBOPedia follows the classic structure of a Web application, implemented in a
Model-View-Controller style. It processes data independently from the UI layer and the
two ends are connected via servlets acting as controllers. Figure 2 shows an architecture
for OBOPedia.</p>
      <p>The created encyclopaedia entries can then be read by the front end’s JSP
mechanisms using getter methods, thus abstracting all the logic away from the UI layer.
The application uses Bootstrap4 components to make the menus, buttons and text more
attractive. Most of the design components are based on Bootstrap templates or
adjustments of them. The letter browser on the left hand side of the main entries page uses a
part of the MooTools5 JavaScript framework to make the menu more interactive.</p>
      <p>
        OBOPedia’s evaluation proceeded as follows with evaluator and participant face to
face using a think aloud protocol [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ] followed by a questionnaire:
1. Introduction and demonstration of OBOPedia.
3 http://lucene.apache.org/solr/ accessed on 15 July 2015
4 http://getbootstrap.com (visited on 25 April 2015)
5 http://mootools.net/ (visited on 25 April 2015)
2. Participant then uses OBOPedia, exploring topics of his or her own choosing.
3. Finally, a short questionnaire was filled out, in which the participant rated the
application, stated their background and provided any type of feedback.
      </p>
      <p>The evaluation had ethical approval and the Personal Information sheet and consent
form are available on the project’s repository.
3</p>
    </sec>
    <sec id="sec-3">
      <title>Results</title>
      <p>
        The main page of the OBOPedia user interface has an alphabetic index and a search
box to give access to the content of OBOPedia. Many entries are put on the screen for
context and to promote ‘serendipitous discovery’ of interesting topics, as is sometimes
the case when looking up topics in a traditional paper based encyclopaedia or
dictionary [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ].
      </p>
      <p>The current version of OBOPedia has available the ontologies listed in Section 2,
giving OBOPedia over 210,000 entries from ten ontologies (counting GO as one
ontology). At present OBOPedia’s administration section is rudimentary. An administrator
can choose whether or not to make available to the user any number of ontologies in
OBOPedia’s library of ontologies. The ability to add new ontologies and to update the
versions of the ontologies in OBOPedia’s library remain to be done. The administration
panel also shows some basic metrics for OBOPedia, such as the number of ontologies
available at the front end, the number of entries, synonyms per entry, and so on.
3.1</p>
      <p>OBOPedia evaluation
OBOPedia was evaluated as described in Section 2. The ‘test drive’ of OBOPedia took
approximately 30 minutes. There were 13 participants, 4 biologists and 9 computer
scientists. The quantitative question ‘Rate the application overall 1–10 ’, gave the mean
score of 8.9; median of 9 and the mode of 9.</p>
      <p>Some quotes from evaluators of OBOPedia are:
– ‘The encyclopaedia was very straightforward to use. The search function was
simple and presented the results in a way which made choosing the most appropriate
entry very easy.’
– ‘As someone with a biology background, having all the biology information sources
in one place instead of searching many different databases is very useful.’
– ‘A very useful tool for searching ontological terms. I wanted to have this before;
when I was searching for the terms for my annotation scheme.’
– ‘Very well organised and easy to navigate.’
– ‘The search bar could have an autocomplete feature to ease finding entities.’
– ‘The application could use the inferred class hierarchy rather than the asserted one,
thus making the set of contents broader.’
– ‘I’d prefer the letter browser bar on the right hand side.’
These demonstrate an appreciation of what OBOPedia is attempting and the benefit
of gathering the breadth of knowledge in the OBO into one view. Most of the quotes
here are about technical and usability enhancements. Most of them are relatively minor,
indicating a reasonably well polished, if early, version of the software. The addition of
reasoning support to broaden the querying capabilities, and thus the retrievals of views
across the content, is a more major change.</p>
      <p>Users appeared to appreciate the look and feel of the application, ensuring that no
important changes have to be made on the user interface layout. They found it
straightforward to understand, browse and search, with minimal amount of training provided
prior to the test.</p>
      <p>Users with a biology background particularly liked the idea of having data from
multiple areas of biology present under one site, thus saving them the effort of
searching for a term in Google then ending up on multiple different sites for the different
subdomains.</p>
      <p>A list of enhancements to OBOPedia to come out of the evaluation are:
– To provide some simple graphic of the ‘see also’ context to show what is at the
other end of the cross-references.
– To use the other ontological relationships in the ‘see also’ cross-references.
– Using an automated reasoner behind OBOPedia would afford the ability to
construct query driven views of content beyond that of the alphabetical or search based
views.
– To enable ‘roll over’ from one letter’s entries to another, so when coming to the end
of the ‘A’ entries, the content just scrolls straight to the start of the ‘B’ entries.
– Enable searching over not just the primary label, but also the synonyms and natural
language definition. Searching would also be enhanced by auto-complete within
the search field to avoid having to type in sometimes lengthy and difficult to spell
names.
– Adding hyperlinks to OBOPedia entries from within the natural language
definitions.
– Add a list of the currently available ontologies to the front page so users know what
is available to explore.
– Add an ‘entry of the day’—a pseudo-randomly selected entry is put on the front
page as a ‘taster’ and potential serendipitous start point.</p>
      <p>Overall, the evaluation was positive, both from an exploration of OBOPedia’s
usability and of its aim as a reference point for a broad collection of biology’s entities.
4</p>
    </sec>
    <sec id="sec-4">
      <title>Discussion</title>
      <p>
        Ontologies capture knowledge about a domain or field of interest. This knowledge is
used by humans and computers in a variety of tasks. One of those tasks should be to
‘find out’ about the field of interest of the ontology or group of ontologies without being
an ontologist. Biology, for example, has developed a wide range of ontologies covering
the domain from genotype to phenotype and then beyond [
        <xref ref-type="bibr" rid="ref4">4, 16</xref>
        ].
      </p>
      <p>
        Most presentations and uses of ontologies are for people knowingly using ontologies—
presentation as either some sort of graph [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] or search that helps to find classes or
ontology [
        <xref ref-type="bibr" rid="ref10 ref5">5, 10, 14</xref>
        ] and with the task of finding the appropriate ontology or the
appropriate term for tasks such as annotation of data [
        <xref ref-type="bibr" rid="ref10">10, 17</xref>
        ]. The task of finding out about
domain knowledge for humans to read and learn appears to be neglected.
      </p>
      <p>To address the task of finding out about ‘stuff’ in biology, OBOPedia exploits the
situation that:
– The OBO cover a broad range of biomedical knowledge that will provide enough
content for the encyclopaedia.
– The OBO are available in the same format and importantly having common
annotation properties and types of annotation.
– The OBO are intended to be clearly delineated, so each entry should be unique
(this is not always true; for instance, anatomical entries will be duplicated, but for
different organisms). This means OBOPedia entries should not be repeated.
– The OBO principles demand each class in the ontology has a natural language
definition; these are mostly orientated to domain users rather than ontologists, which
should increase the accessibility of OBOPedia to its intended audience.
– The rich synonyms within OBO ontologies offer the potential for many avenues of
access for users of OBOPedia.</p>
      <p>Many of these features capitalise on some of the OBO Foundry principles6. OBOPedia
is, therefore, restricted to OBO and there is no intention for it to use ontologies outside
the OBO regime.</p>
      <p>What OBOPedia seeks to do is present the collected knowledge of OBO’s
ontologies as a catalogue of biology’s entities in an encyclopaedic form for non-ontology
users. It has been evaluated by two kinds of users. A technical evaluation was conducted
by people in the domain of computer science; this elicited general usability issues.
Biologists were also used to evaluate OBOPedia. As well as usability issues, this group, as
the intended primary users, found the knowledge afforded by OBOPedia a useful way
of finding out about entities in biomedicine. The broader claims of OBOPedia have not
been fully evaluated, but feedback so far suggests the approach is feasible.</p>
      <p>
        There are other ways of accessing the content of a corpus of bio-ontologies [
        <xref ref-type="bibr" rid="ref1 ref5">1, 5,
18</xref>
        ]. They all appear to be aimed at an audience seeking to find the ontology or its
content, rather than learn about the knowledge in the ontology as a human seeking domain
knowledge. Ontobee seems to be closest to what OBOPedia seeks to support; it has a
page for a term from an ontology, highlighting the label, synonyms and the natural
language definition. it also has a large quantity of ontological information which is
something that OBOPedia avoids. BioPortal [14, 18] has a repository of over 400 biomedical
ontologies. It offers a range of facilities to developers of ontologies and to those that
use ontologies as ontologies. The type of presentation of ontology knowledge and role
of OBOPedia is not currently part of BioPortal’s remit. Nevertheless, what
OBOPedia could be implemented using BioPortal’s web services [18] and offered as part of
BioPortal’s web presence–as long as OBOPedia’s ‘non-ontology’ view of knowledge
captured in an ontology is part of BioPortal’s web presence’s role. OBOPedia does in
no way negate the role of resources such as Ontobee and BioPortal; they have different
roles and different audiences.
      </p>
      <p>
        OBOPedia is intended for use by biologists or users wishing to find out about
biology and not to find out about the ontology itself. Some users will wish to find out
about the biology content of an ontology, but not in an ontological presentation: What
an entity is and how it is known to tbe related to another entity. This is what an ontology
does, but just putting an ontology or collection of ontologies ‘as an ontology’ in front of
a biologist or someone wishing to find out about biology is not necessarily sufficient for
them to find out about the knowledge content in an ontology. OBOPedia is an attempt
to look sideways at the problem of giving access to the considerable effort that has been
made to collect what is known about biology into an easily accessible form. We can see
6 http://wiki.obofoundry.org/wiki/index.php/Category:Principles
that an encyclopaedic form can do this job. As a community we should think of how
we can bring the semantic view of biology to as wide an audience as possible.
Applications like kupkb [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] do this—using an ontology without knowing one is doing it, but
offering working biologists a useful thing that is facilitated by ontology.
      </p>
    </sec>
    <sec id="sec-5">
      <title>Acknowledgments</title>
      <p>We acknowledge the input of the participants in OBOPedia’s evaluation.
14. Noy, N.F., Shah, N.H., Whetzel, P.L., Dai, B., Dorf, M., Griffith, N., Jonquet,
C., Rubin, D.L., Storey, M.A., Chute, C.G., Musen, M.A.: Bioportal: ontologies
and integrated data resources at the click of a mouse. Nucleic Acids Research
(2009), http://nar.oxfordjournals.org/content/early/2009/05/29/
nar.gkp440.abstract
15. Rector, A.: Title of the Deliverable: Barriers, approaches and research priorities for
integrating biomedical ontologies (2008), deliverable D6.1 for the Semantic Health project.
16. Smith, B., Ashburner, M., Rosse, C., Bard, J., Bug, W., Ceusters, W., Goldberg, L.J., Eilbeck,
K., Ireland, A., Mungall, C.J., et al.: The obo foundry: coordinated evolution of ontologies
to support biomedical data integration. Nature biotechnology 25(11), 1251–1255 (2007)
17. Stevens, R., Lord, P.: Application of ontologies in bioinformatics. In: Staab, S., Studer, R.
(eds.) Handbook on Ontologies in Information Systems, pp. 347–485. International
Handbook on Information Systems, Springer, second edn. (2008)
18. Whetzel, P.L., Noy, N.F., Shah, N.H., Alexander, P.R., Nyulas, C., Tudorache, T., Musen,
M.A.: Bioportal: enhanced functionality via new web services from the national center for
biomedical ontology to access and use ontologies in software applications. Nucleic acids
research 39(suppl 2), W541–W545 (2011)
19. Wikipedia: Encyclopedia (2015), http://en.wikipedia.org/wiki/
Encyclopedia, (visited on 8 April 2015)</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Xiang</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mungall</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ruttenberg</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>He</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          :
          <article-title>Ontobee: A Linked Data Server and Browser for Ontology Terms</article-title>
          .
          <source>In: Proceedings of the 2nd International Conference on Biomedical Ontology (ICBO)</source>
          . pp.
          <fpage>279</fpage>
          -
          <lpage>281</lpage>
          (
          <year>2011</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Ashburner</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ball</surname>
            ,
            <given-names>C.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Blake</surname>
            ,
            <given-names>J.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Botstein</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Butler</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Cherry</surname>
            ,
            <given-names>J.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Davis</surname>
            ,
            <given-names>A.P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dolinski</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dwight</surname>
            ,
            <given-names>S.S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Eppig</surname>
            ,
            <given-names>J.T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Harris</surname>
            ,
            <given-names>M.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hill</surname>
            ,
            <given-names>D.P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Issel-Tarver</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kasarskis</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lewis</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Matese</surname>
            ,
            <given-names>J.C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Richardson</surname>
            ,
            <given-names>J.E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ringwald</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rubin</surname>
            ,
            <given-names>G.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sherlock</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          :
          <article-title>Gene Ontology: tool for the unification of biology</article-title>
          .
          <source>Nature Genetics</source>
          <volume>25</volume>
          (
          <issue>1</issue>
          ),
          <fpage>25</fpage>
          -
          <lpage>29</lpage>
          (
          <year>2000</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Baker</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Goble</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bechhofer</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Paton</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stevens</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Brass</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>An Ontology for Bioinformatics Applications</article-title>
          .
          <source>Bioinformatics</source>
          <volume>15</volume>
          (
          <issue>6</issue>
          ),
          <fpage>510</fpage>
          -
          <lpage>520</lpage>
          (
          <year>1999</year>
          ), http://www.cs.man. ac.uk/˜stevensr/papers/bioinformatics-ontology99.doc
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Bodenreider</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stevens</surname>
          </string-name>
          , R.: Bio-Ontologies:
          <article-title>Current Trends and Future Directions</article-title>
          .
          <source>Briefings in Bioinformatics</source>
          <volume>7</volume>
          (
          <issue>3</issue>
          ),
          <fpage>256</fpage>
          -
          <lpage>274</lpage>
          (
          <year>2006</year>
          ), http://bib.oxfordjournals.org/ content/7/3/256.full
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5. Coˆte´,
          <string-name>
            <given-names>R.</given-names>
            ,
            <surname>Reisinger</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            ,
            <surname>Martens</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            ,
            <surname>Barsnes</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            ,
            <surname>Vizcaino</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.A.</given-names>
            ,
            <surname>Hermjakob</surname>
          </string-name>
          , H.:
          <article-title>The ontology lookup service: bigger and better</article-title>
          .
          <source>Nucleic Acids Research</source>
          <volume>38</volume>
          (
          <issue>suppl 2</issue>
          ),
          <fpage>W155</fpage>
          -
          <lpage>W160</lpage>
          (
          <year>2010</year>
          ), http://nar.oxfordjournals.org/content/ 38/suppl_2/W155.abstract
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Foster</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ford</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          :
          <article-title>Serendipity and information seeking: an empirical study</article-title>
          .
          <source>Journal of Documentation</source>
          <volume>59</volume>
          (
          <issue>3</issue>
          ),
          <fpage>321</fpage>
          -
          <lpage>340</lpage>
          (
          <year>2003</year>
          ), http://dx.doi.org/10.1108/ 00220410310472518
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Guarino</surname>
          </string-name>
          , N.:
          <article-title>Formal Ontology and Information Systems</article-title>
          .
          <source>In: Proceedings of FOIS98</source>
          ,. pp.
          <fpage>3</fpage>
          -
          <lpage>15</lpage>
          (
          <year>1998</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Horridge</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bechhofer</surname>
            ,
            <given-names>S.:</given-names>
          </string-name>
          <article-title>The OWL API: A Java API for Working with OWL 2 Ontologies</article-title>
          . In
          <source>: OWLED</source>
          <year>2009</year>
          ,
          <article-title>6th OWL Experienced</article-title>
          and Directions Workshop. Chantilly, Virginia (
          <year>October 2009</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Jensen</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bork</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Ontologies in Quantitative Biology: A Basis for Comparison, Integration, and Discovery</article-title>
          .
          <source>PLoS Biology</source>
          <volume>8</volume>
          (
          <issue>5</issue>
          ),
          <year>e1000374</year>
          (
          <year>2010</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Jonquet</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Musen</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>NH</surname>
          </string-name>
          , S.:
          <article-title>Building a biomedical ontology recommender web service</article-title>
          .
          <source>Journal of Biomedical Semantics</source>
          <volume>1</volume>
          (
          <issue>S1</issue>
          ) (
          <year>2010</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Jupp</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Klein</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Schanstra</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stevens</surname>
          </string-name>
          , R.:
          <article-title>Developing a Kidney and Urinary Pathway Knowledge Base</article-title>
          .
          <source>Journal of Biomedical Semantics</source>
          <volume>2</volume>
          (
          <issue>Suppl</issue>
          . 2),
          <source>S7</source>
          (
          <year>2011</year>
          ), http://www. jbiomedsem.com/content/2/S2/S7
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Katifori</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Halatsis</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lepouras</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vassilakis</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Giannopoulou</surname>
          </string-name>
          , E.:
          <article-title>Ontology visualization methodsa survey</article-title>
          .
          <source>ACM Computing Surveys (CSUR) 39(4)</source>
          ,
          <volume>10</volume>
          (
          <year>2007</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Lewis</surname>
            ,
            <given-names>C.H.</given-names>
          </string-name>
          :
          <article-title>Using the “Thinking Aloud” Method In Cognitive Interface Design(Technical report)</article-title>
          .
          <source>Tech. Rep. RC-9265</source>
          ., IBM. (
          <year>1982</year>
          )
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>