<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Challenges for ontology repositories and applications to biomedicine &amp; agronomy</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Clement Jonquet</string-name>
          <email>jonquet@lirmm.fr</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Laboratory of Informatics, Robotics, and Microelectronics of Montpellier (LIRMM), University of Montpellier &amp; CNRS, France &amp; Center for BioMedical Informatics Research (BMIR), Stanford University</institution>
          ,
          <addr-line>USA, ORCID: 0000-0002-2404-1582</addr-line>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2017</year>
      </pub-date>
      <fpage>56</fpage>
      <lpage>60</lpage>
      <abstract>
        <p />
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>The explosion of the number of ontologies
and vocabularies available in the Semantic
Web makes ontology libraries and
repositories mandatory to find and use them.
Their functionalities span from simple
ontology listing with more or less of
metadata description to portals with advanced
ontology-based services: browse, search,
visualization, metrics, annotation, etc.
Ontology libraries and repositories are usually
developed to address certain needs and
communities. BioPortal, the ontology
repository built by the US National Center
for Biomedical Ontologies BioPortal relies
on a domain independent technology
already reused in several projects from
biomedicine to agronomy and earth sciences.
In this position paper, we describe six high
level challenges for ontology repositories:
metadata &amp; selection, multilingualism,
alignment, new generic ontology-based
services, annotations &amp; linked data, and
interoperability &amp; scalability. Then, we
present some propositions to address these
challenges and point to our previously
published work and results obtained within
applications –reusing NCBO technology–
to biomedicine and agronomy in the
context of the NCBO, SIFR and AgroPortal
projects.
Ontologies, ontology libraries &amp;
repositories, ontology metadata, ontology-based
services, ontology selection, semantic
annotation, BioPortal.
1</p>
    </sec>
    <sec id="sec-2">
      <title>Introduction</title>
      <p>
        The Semantic Web produces many vocabularies
and ontologies to represent and annotate any kind
of data. However, those ontologies are spread out,
in different formats, of different size, with
different structures and from overlapping domains. The
scientific community has always been interested
in designing common platforms to list and
sometime host and serve ontologies, align them, and
enable their (re)use
        <xref ref-type="bibr" rid="ref13 ref15">(Ding and Fensel, 2001;
Hartmann et al., 2009; D’Aquin and Noy, 2012; ,
1995)</xref>
        . These platforms range from simple
ontology listings or libraries with structured metadata,
to advanced repositories (or portals) which
feature a variety of services for multiple types of
semantic resources (ontologies, vocabularies,
terminologies, taxonomies, thesaurus) such as
browse/search, visualization, metrics,
recommendation, or annotation. In this paper, we will focus
on ontology repositories, they allow to address
important questions:
      </p>
      <p>If you have built an ontology, how do you let
the world know and share it?
How do you connect your ontology to the rest
of the semantic world?
If you need an ontology, where do you go to
get it?
How do you know whether an ontology is any
good?
If you have data to index, how do you find the
most appropriate ontology for your data?
If you look for data, how may the semantics
of ontologies help you locate them?
More generally, ontology repositories help
“ontology users” to deal with ontologies without
managing them or engaging in the complex and long
process of developing them.
However, with big number of ontologies new
problems have raised such as describing,
selecting, evaluating, trusting, and interconnecting
them. From our experience working first on the
US National Center for Biomedical Ontologies
(NBCO) BioPortal, the most widely adopted
biomedical ontology repository and later on the SIFR
BioPortal, a specific sub-portal to address the
French biomedical community and AgroPortal, an
ontology repository for agronomy, we review and
discuss six challenges in designing such
platforms:
1. Metadata &amp; selection. Ultimately, ontology
repositories are made to share and reuse
ontologies. But which ontology should I reuse?
With too many different and overlapping
ontologies, properly describing them with
metadata and facilitate their identification
and selection becomes and important issue.
We believe, as any other data, ontologies
must be FAIR.
2. Multilingualism. We live in a multilingual
world, so are the concepts and entities from
this world. The Semantic Web offers now
tools and standards to develop multilingual
and lexically rich ontologies. Repositories
must be able to deal with multiple languages
also.
3. Ontology alignment. No conceptualization
is an island. It is now commonly agreed data
interoperability cannot be achieved by means
of a single common ontology for a domain,
and interlinking ontologies is the way
forward. But the more ontologies are being
produced, the more the need for ontology
alignment becomes important.
4. Ontology-based services. On reason to
adopt Semantic Web standards and use
ontology repositories is to benefit from multiple
services for –and based on– ontologies. No
one likes to reimplement something already
existing and that can be generalized to
another ontology just by dropping it in a
repository. The portfolio of services for ontologies
available in repositories should then grows.</p>
    </sec>
    <sec id="sec-3">
      <title>5. Annotations and linked data. Ontologies</title>
      <p>and vocabularies are the backbone of
semantically rich data (Linked Open Data,
knowledge bases, etc.) as they are used to
semantically annotate and interlink datasets.
It is also important to facilitate semantic
indexing, search and data access directly from
the repositories.
6. Scalability &amp; interoperability. The
community of ontology developers and users is
growing both horizontally (i.e., new
domains) and vertically (i.e., new adopters
inside a domain). Ontology repositories shall
therefore scale to high number of ontologies,
while facilitating their alignments, and when
multiple repositories are created, they must
be interoperable.</p>
      <p>In the following, we will detail these challenges
and briefly describe/point to results obtained in
the context of our multiple ontology repository
projects. In some sense, this article is an index of
10-years of published research in the domain of
ontology repositories. We do not report hereafter
all related work for each challenge neither we
claim to have addressed them all. However, we
believe our results illustrate potential solutions to
move forward in that domain of research.
2
2.1</p>
    </sec>
    <sec id="sec-4">
      <title>Background</title>
    </sec>
    <sec id="sec-5">
      <title>Ontology libraries &amp; repositories</title>
      <p>With the growing number of ontologies
developed, ontology libraries and repositories have
always been of interest in the Semantic Web
community. Ding and Fensel (2001) introduced the
notion of ontology library and presented a review
of libraries at that time:</p>
      <p>“A library system that offers various
functions for managing, adapting and
standardizing groups of ontologies. It should fulfill the
needs for re-use of ontologies. In this sense,
an ontology library system should be easily
accessible and offer efficient support for
reusing existing relevant ontologies and
standardizing them based on upper-level
ontologies and ontology representation languages.”
The terms “collection”, “listing” or “registries”
are also used to describe ontology libraries. All
correspond to systems that help reuse or find
ontologies by simply listing them (e.g., DAML or
DERI listings) or by offering structured metadata
to describe them (e.g., FAIRSharing, BARTOC).
But those systems do not support any services
beyond description, especially based on the content
of the ontologies.</p>
      <p>
        Hartmann et al., (2009) introduced the concept
of ontology repository, with advanced features
such as search, browsing, metadata management,
visualization, personalization, and mappings and
an application programming interface to query
their content/services:
“A structured collection of ontologies (…)
by using an Ontology Metadata Vocabulary.
References and relations between ontologies
and their modules build the semantic model of
an ontology repository. Access to resources is
realized through semantically-enabled
interfaces applicable for humans and machines.
Therefore, a repository provides a formal query
language.”
By the end of the 2000’s, the topic was of high
interest as illustrated by the 2010 ORES
workshop (d’Aquin et al., 2010) or the 2008
OntologySummit.1 The Open Ontology Repository
Initiative
        <xref ref-type="bibr" rid="ref6 ref7">(Baclawski and Schneider, 2009)</xref>
        was a
collaborative effort to develop a federated
infrastructure of ontology repositories. At that time, the
effort already reused the NCBO
technology
        <xref ref-type="bibr" rid="ref43">(Whetzel and Team, 2013)</xref>
        that was
the most advanced open source technology for
managing ontologies but not yet packaged in an
“virtual appliance” as it is today. More recently
the effort also studied OntoHub
        <xref ref-type="bibr" rid="ref40">(Till et al., 2014)</xref>
        technology for generalization but the OOR
initiative is now discontinued.
      </p>
      <p>
        In parallel, there have been effort do index any
Semantic Web data online (including ontologies)
and offer search engines such as Swoogle and
Watson
        <xref ref-type="bibr" rid="ref14">(Ding et al., 2004; D’Aquin et al., 2007)</xref>
        .
We cannot talk about ontology library or
repositories for those “Semantic Web indexes”, even if
they support some features of ontology libraries or
repositories (e.g., search).
      </p>
      <p>
        In the biomedical or agronomic domains there
are several standards and/or ontology libraries
such as FAIRSharing (fairsharing.org)
        <xref ref-type="bibr" rid="ref27">(McQuilton
et al., 2016)</xref>
        , the FAO’s VEST Registry
(aims.fao.org/vest-registry), and the agINFRA
linked data vocabularies (vocabularies.aginfra.eu).
They usually register ontologies and provide a few
metadata attributes about them. However, because
they are registries not especially focused on
vocabularies and ontologies, they do not support the
level of features that an ontology repository offers.
In the biomedical domain, the OBO
Foundry
        <xref ref-type="bibr" rid="ref35">(Smith et al., 2007)</xref>
        is a reference
community effort to help the biomedical and
biological communities build their ontologies with an
enforcement of design and reuse principles that have
made the effort very successful. The OBO
Foundry Web application (http://obofoundry.org) is not
an ontology repository per se, but relies on other
1 http://ontolog.cim3.net/wiki/OntologySummit2008.html
applications that pull their data from the foundry,
such as the NCBO BioPortal
        <xref ref-type="bibr" rid="ref18 ref31">(Noy et al., 2009)</xref>
        ,
OntoBee
        <xref ref-type="bibr" rid="ref32">(Ong et al., 2016)</xref>
        , the EBI Ontology
Lookup Service (Côté et al., 2006) and more
recently AberOWL (Hoehndorf et al., 2015). In
addition, there exist other ontology libraries and
repository efforts unrelated to biomedicine, such as
the Linked Open Vocabularies
        <xref ref-type="bibr" rid="ref41">(Vandenbussche et
al., 2014)</xref>
        , OntoHub
        <xref ref-type="bibr" rid="ref40">(Till et al., 2014)</xref>
        , and the
Marine Metadata Initiative’s Ontology Registry
and Repository (Rueda et al., 2009). More
recently, the SIFR BioPortal
        <xref ref-type="bibr" rid="ref19 ref24 ref3">(Jonquet et al., 2016a)</xref>
        prototype was created at University of Montpellier to
build a French Annotator and experiment
multilingual issues in BioPortal
        <xref ref-type="bibr" rid="ref23">(Jonquet et al., 2015)</xref>
        .
The same university is also developing
AgroPortal, an ontology repository for agronomy and
neighboring domains such as food, plant sciences
and biodiversity
        <xref ref-type="bibr" rid="ref20 ref21 ref22">(Jonquet et al., 2017a)</xref>
        .
      </p>
      <p>D’Aquin and Noy, (2012) and Naskar and
Dutta, (2016) provided the latest reviews of
ontology repositories. In Table 1, we provide a
nonexhaustive –but quite rich– list of ontology
libraries, repositories and Web indexes available today.
ESIPportal*
AgroPortal*
OntoHub
Finto
EcoPortal (proposition end 2017)*</p>
    </sec>
    <sec id="sec-6">
      <title>Semantic Web indexes</title>
      <p>
        Swoogle
Watson
Sindice
Falcons
Technology
NCBO Virtual Appliance (Stanford)
OLS technology (EBI)
LexEVS (Mayo Clinic)
Intelligent Topic Manager (Mondeca)
SKOSMOS (Nat. Library of Finland)
Abandoned projects include: Cupboard, Knoodl,
Schemapedia, SchemaWeb, OntoSelect,
OntoSearch, OntoSearch2, TONES, SchemaCache,
Soboleo
In the biomedical domain, BioPortal
(http://bioportal.bioontology.org)
        <xref ref-type="bibr" rid="ref18 ref31">(Noy et al.,
2009)</xref>
        , developed by the National Center for
Biomedical Ontologies (NBCO) at Stanford is a
wellknown open repository for biomedical ontologies
originally spread out over the Web and in different
formats. There are +650 public ontologies in this
collection as of end 2017. By using the portal’s
features, users can browse, search, visualize and
comment on ontologies both interactively through
a Web interface, and programmatically via Web
services. Within BioPortal, ontologies are used to
develop an annotation workflow
        <xref ref-type="bibr" rid="ref18">(Jonquet et al.,
2009)</xref>
        that indexes several biomedical text and
data resources using the knowledge formalized in
ontologies to provide semantic search features that
enhance information retrieval experience
        <xref ref-type="bibr" rid="ref18">(Jonquet
et al., 2011)</xref>
        . The NCBO BioPortal functionalities
have been progressively extended in the last 12
years, and the platform has adopted Semantic Web
technologies (e.g., ontologies, mappings,
metadata, notes, and projects are stored in an RDF2 triple
store)
        <xref ref-type="bibr" rid="ref34">(Salvadores et al., 2013)</xref>
        .
      </p>
      <p>
        An important aspect is that NCBO
technology
        <xref ref-type="bibr" rid="ref43">(Whetzel and Team, 2013)</xref>
        is
domainindependent and open source. A BioPortal virtual
appliance3 is available as a server machine
embedding the complete code and deployment
environment, allowing anyone to set up a local
ontology repository and customize it. The NCBO virtual
appliance is quite regularly reused by
organizations which need to use services like the NCBO
Annotator but have to process sensitive data in
house e.g., hospitals. Via the virtual appliance,
NCBO technology has already been adopted for
different ontology repositories in related domains
and was also originally chosen as foundational
software of the OOR Initiative
        <xref ref-type="bibr" rid="ref6 ref7">(Baclawski and
Schneider, 2009)</xref>
        . The MMI Ontology Registry
and Repository (Rueda et al., 2009) used it as its
backend storage system for over 10 years, and the
Earth Sciences Information Partnership earth and
environmental semantic portal (Pouchard L.
Huhns M., 2012) was deployed several years ago.
We are also currently working on the SIFR
BioPortal
        <xref ref-type="bibr" rid="ref19 ref24 ref3">(Jonquet et al., 2016a)</xref>
        and
AgroPortal
        <xref ref-type="bibr" rid="ref20 ref21 ref22">(Jonquet et al., 2017a)</xref>
        projects
described hereafter.
2.3
      </p>
    </sec>
    <sec id="sec-7">
      <title>Two collaborative ontology repository projects</title>
      <p>
        In the context of our projects, to avoid building
new ontology repositories from scratch, we have
considered which of the previous technologies are
reusable. While most of them are “open source,”
only the NCBO BioPortal4 and OLS5 are really
meant for reuse, both in their construction, and
with their documentation provided. Although we
2 The Resource Description Framework (RDF) is the W3C
language to described data. It is the backbone of the
semantic web. SPARQL is the corresponding query language. By
adopting RDF as the underlying format, an ontology
repository based on NCBO technology can easily make its data
available as linked open data and queryable through a public
SPARQL endpoint. To illustrate this, the reader may consult
the Link Open Data cloud diagram (http://lod-cloud.net) that
since 2017 includes ontologies imported from the NCBO
BioPortal (most of the Life Sciences section).
3 www.bioontology.org/wiki/index.php/Category:NCBO_Virtual_Appliance
4 The technology has always been open source, and the
appliance has been made available since 2011. However, the
product became concretely and easily reusable after
BioPortal v4.0 end of 2013.
5 The technology has always been open source but some
significant changes (e.g., the parsing of OWL) facilitating
the reuse of the technology for other portals were done with
OLS 3.0 released in December 2015.
cannot know all the applications of other
technologies, the visibly frequent reuse of the NCBO
technology definitively confirmed it is a good
candidate for reuse when building a new ontology
repository. Also, of the two candidate
technologies, we believe NCBO technology implements
the highest number of required features in our
projects
        <xref ref-type="bibr" rid="ref20 ref21 ref22">(Jonquet et al., 2017a)</xref>
        .
      </p>
    </sec>
    <sec id="sec-8">
      <title>SIFR BioPortal</title>
      <p>
        In the context of the Semantic Indexing of French
Biomedical Data Resources (SIFR) project, we
have developed the SIFR BioPortal
(http://bioportal.lirmm.fr)
        <xref ref-type="bibr" rid="ref19 ref24 ref3">(Jonquet et al., 2016a)</xref>
        ,
an open platform to host French biomedical
ontologies and terminologies based on the
technology developed by the NCBO. The portal facilitates
use and fostering of terminologies and ontologies
which were only developed in French or translated
from English resources and are not well served in
the English-focused NCBO BioPortal. As of
today, the portal contains 25 public ontologies and
terminologies (+ 6 private ones) that cover
multiple areas of biomedicine, such as the French
versions of standards terminologies (e.g., MeSH,
MedDRA, ATC, ICD-10) but also multilingual
ontologies. In this later cases, we use the NCBO
BioPortal as a source repository –so users do not
have to upload their multilingual ontologies
twice– and only parse and index the French
content on the SIFR BioPortal.
      </p>
      <p>
        The original motivation in building the SIFR
BioPortal was to develop the SIFR Annotator
(http://bioportal.lirmm.fr/annotator) to address the
lack of out-of-the-shelve openly and easily
accessible semantic annotation system for
French
        <xref ref-type="bibr" rid="ref1 ref19 ref22 ref24 ref3 ref38 ref39">(Jonquet et al., 2016a; Tchechmedjiev et
al., 2017a)</xref>
        . The service is originally based on the
NCBO Annotator [8], a Web service allowing
scientists to utilize available biomedical ontologies
for annotating their datasets automatically, but
was significantly enhanced and customized for
French. The annotator service processes raw
textual descriptions, tags them with relevant
biomedical ontology concepts and returns the annotations
to the users in several formats such as JSON-LD,
RDF or BRAT.
      </p>
    </sec>
    <sec id="sec-9">
      <title>AgroPortal: a vocabulary and ontology repository for agronomy</title>
      <p>
        We have been reusing the NCBO BioPortal
technology to design AgroPortal, an ontology
repository for agronomy, food, plant sciences, and
biodiversity (http://agroportal.lirmm.fr)
        <xref ref-type="bibr" rid="ref19 ref20 ref21 ref22 ref24 ref3">(Jonquet et
al., 2016c; Jonquet et al., 2017a)</xref>
        . AgroPortal, is an
advanced prototype featuring all BioPortal
services and new ones implemented to address the
requirements of the agronomy community. The
platform currently hosts 77 ontologies among
which 50 are not present in any comparable
repository. We have identified 93 other candidate
ontologies that will be loaded in the future to
complement this valuable resource.
3
      </p>
    </sec>
    <sec id="sec-10">
      <title>Challenges, propositions and results</title>
      <p>In the following sections, we describe some
challenges we identified by working on ontology
repository and exchanging with our user
communities. In each case, we describe a few results
obtained on the relevant topic.
3.1</p>
    </sec>
    <sec id="sec-11">
      <title>Metadata &amp; selection</title>
      <p>The first questions we ask ourselves when
entering a bookstore are often: “Where is the book I am
looking for?” or “Which book will I discover and
pick up today?” The same questions are true for
ontology libraries. To address them, we need
better description of the ontologies, with precise
and harmonized metadata and we need also
means to facilitate the identification and
selection of the ontologies of interest. Ontologies
serve to make data FAIR (Wilkinson et al., 2016),
ontology repositories shall serve to make
ontologies FAIR.</p>
      <p>As any resources, ontologies, vocabularies and
terminologies need to be described with relevant
metadata to facilitate their identification and
selection. However, none of the existing metadata
vocabularies can completely meet this need if taken
independently. Indeed, some metadata properties
are intrinsic to the ontology (name, license,
description); others, such as community feedbacks,
or relations to other ontologies are typically
information that an ontology library shall capture,
populate and consolidate to facilitate the ontology
landscape comprehension (e.g., selection of an
ontology).</p>
      <p>In Jonquet et al., (2017b), we have reviewed the
most standard and relevant vocabularies (23
totals) currently available to describe metadata for
ontologies (such as Dublin Core, Ontology
Metadata Vocabulary, VoID, etc.) as well as the
different metadata implementation in multiple
ontology libraries or repositories. We have then built
a new metadata model for AgroPortal. The
repository now parses 346 standard properties that could
be used to describe different aspects of ontologies:
intrinsic descriptions, people, date, relations,
content, metrics, community, administration, and
access. We use them to populate a model of 127
properties implemented in the portal and
harmonized for all the ontologies. We have spent a
significant amount of time to edit the metadata of the
ontologies with the goal to facilitate the
comprehension of the agronomical ontology landscape by
displaying diagrams and charts about all the
ontologies on the portal. We have now a specific
page (http://agroportal.lirmm.fr/landscape)
dedicated to visualizing the ontology landscape in
AgroPortal that facilitates analysis of the repository
content. The landscape page helps to figure out
what are some of the main domain of interests as
well as common development practices when
creating an ontology in agronomy.</p>
      <p>
        In Dutta et al., (2017), we have generalized our
work done within AgroPortal to propose a new
Metadata vocabulary for Ontology Description
and publication, called MOD
(https://github.com/sifrproject/MOD-Ontology).
MOD 1.2 is defined in OWL and consists of 19
classes and 88 properties most of them to describe
the mod:Ontology object. MOD 1.2 may serve as
(i) a vocabulary to be used by ontology developers
to annotate and describe their ontologies, or (ii) an
explicit OWL ontology to be used by ontology
libraries to offer semantic descriptions of ontologies
as linked data. MOD 1.2 is an initiative which
attempts to overcome some of the limitations of the
Ontology Metadata Vocabulary
        <xref ref-type="bibr" rid="ref37">(Suarez-Figueroa
et al., 2005)</xref>
        but is still a temporary proposition
that will be discussed in the next months within
the Research Data Alliance recently re-configured
Vocabulary &amp; Semantic Services Interest Group.6
      </p>
      <p>
        Automatic ontology selection or
recommendation has been a subject of interest to facilitate
ontology reuse (Sabou et al., 2006)
        <xref ref-type="bibr" rid="ref8">(Butt et al.,
2016)</xref>
        . The number and variety of ontologies in
certain domains is now so large that choosing one
for an annotation task or for designing a specific
application is quite cumbersome.
      </p>
      <p>
        In Martinez-Romero et al., (2017), we
developed the NCBO Ontology Recommender. This
service suggests relevant ontologies from the
repository for annotating text data. The new
rec6
https://www.rd-alliance.org/groups/vocabulary-servicesinterest-group.html
ommendation approach evaluates the relevance of
an ontology to biomedical text data according to
four different criteria: (1) the extent to which the
ontology covers the input data; (2) the acceptance
of the ontology in the community; (3) the level of
detail of the ontology classes that cover the input
data; and (4) the specialization of the ontology to
the domain of the input data. This new version of
a service ori
        <xref ref-type="bibr" rid="ref10">ginally released in 2010</xref>
        <xref ref-type="bibr" rid="ref18">(Jonquet et
al., 2010)</xref>
        combines the strengths of its
predecessor with a range of adjustments and new features
that improve its reliability and usefulness.
Because it is integrated in the NCBO technology, the
Recommender is already available within the
SIFR BioPortal and AgroPortal. We shall note that
these services do not yet rely on the new metadata
model previously cited.
3.2
      </p>
    </sec>
    <sec id="sec-12">
      <title>Multilingualism</title>
      <p>
        Scientific discoveries that could be made with
help of ontologies to annotate, integrate, mine and
search data, are often limited by the availability of
ontology-based tools and services only for one
natural language, usually English, for which there
exist the most ontologies. Recently, ontology
localization, i.e., “the process of adapting an
ontology to a concrete language and culture
community” (Cimiano et al., 2010), has become very
important in the ontology development lifecycle, but
when efforts are made to properly represent
lexical (e.g., using Lemon
        <xref ref-type="bibr" rid="ref26">(McCrae et al., 2011)</xref>
        ) or
multilingual information (e.g., using
LexOMV
        <xref ref-type="bibr" rid="ref29">(Montiel-Ponsoda et al., 2007)</xref>
        or
Lemon translation module (Gracia et al., 2014)) are
made, it is rarely leveraged by ontology libraries
and repositories. In the future, we need ontology
repositories to entirely support interface and
content internationalization (i.e., both
displaying user interfaces (e.g., menu names, help, etc.)
in different languages and displaying their content
(e.g., ontology labels, mappings, etc.) in different
languages) and be multilingual by enabling a
complete use of their functionalities and
services for multilingual ontologies or
monolingual ontologies linked one another.
      </p>
      <p>In Jonquet et al., (2015), we presented a
roadmap for addressing the issues of dealing with
multilingual or monolingual ontologies in the
NCBO BioPortal, which takes English as primary
language. We proposed a set of representations to
support multilingualism in the portal and to enable
a complete use of the functionalities and services
for any kind of ontologies and data:
(i) Representation of natural language property for
an ontology; (ii) Representation of translation
relations between ontologies; (iii) Representation of
the distinction between ontologies with
multilingual content i.e., multilingual and mono lingual
ontologies; (iv) Representation of multilingual
mappings. Those aspects have been addressed
now within MOD and/or the new AgroPortal
metadata model previously cited. In addition,
in Annane et al., (2016b), we reconciled more
than 228K mappings between ten English
ontologies hosted on NCBO BioPortal and their French
translations hosted on the SIFR BioPortal. The
next big step is now to internationalize the portal.
3.3</p>
    </sec>
    <sec id="sec-13">
      <title>Ontology alignment</title>
      <p>
        Ontologies, or other semantic resources, will
inevitably overlap in coverage. Therefore, the need for
ontology alignment. This need has been explicitly
expressed by almost all our partner organizations
in biomedicine, agronomy or ecology.
Surprisingly, it seems there is a gap between the
state-of-theart results obtained at each edition of the Ontology
Alignment Evaluation Initiative (OAEI
http://oaei.ontologymatching.org) and the
day-today reality of ontology developers. Tools are often
hardly reusable, and results cannot be easily
reproduced outside of the benchmarking effort.
Another key role of ontology repositories is to store
mappings (or alignments) between ontologies.
Ontology repositories shall support the
extraction, generation, validation, evaluation, storage
and retrieval of mappings between the
ontologies they host. Automatic mapping generation
within ontology repositories shall go beyond
simple lexical or ID-based approaches7 and
state-ofthe-art tools shall be incorporated within
repositories. An equivalent effort, such as the one made to
harvest ontologies, must be made to harvest the
mappings between these ontologies and describe
them with metadata and provenance information
to facilitate trust and reuse.
7 To the best of our knowledge, only the NCBO technology
automatically computes ontology alignments when
ontologies are hosted within the portal. The portal automatically
creates some mappings when two classes share the same
identifiers properties, or when they share a common
normalized preferred label or synonym. Although basic lexical
mapping approaches can be inaccurate and should be used
with caution
        <xref ref-type="bibr" rid="ref17 ref6 ref7">(Faria et al., 2014; Pathak and Chute, 2009)</xref>
        ,
they usually work quite well to interconnect
ontologies
        <xref ref-type="bibr" rid="ref18">(Ghazvinian et al., 2009)</xref>
        .
      </p>
      <p>
        In Ghazvinian et al., (2009), we have analyzed
the mappings automatically generated within
BioPortal and what they tell us about the ontologies
themselves, the structure of the ontology
repository, and the ways in which the mappings can help
in the process of ontology design and evaluation.
This study demonstrated the value of having a
mapping repository goes beyond
ontology-toontology alignment, but concretely helps analyze
the structures, dependencies and overlap of
ontologies in the same domain. A similar, more recent
study about ontology terms reuse have been done
by Kamdar et al.
        <xref ref-type="bibr" rid="ref22">(Kamdar et al., 2017)</xref>
        . In Annane
et al., (2016a), we have also demonstrated that
existing mappings between ontologies can also be
used to improve ontology alignment methods
based on background knowledge; in other words,
a centralized mapping repository will also be an
excellent resource to curate and generate new
mappings.
3.4
      </p>
    </sec>
    <sec id="sec-14">
      <title>Generic ontology-based services</title>
      <p>Ontology repositories offer a large span of
services: file hosting, versioning, search and browse
content, visualization, metrics, notes, mapping,
etc. These services are ‘generic’ if they are domain
independent i.e., not specific to a domain, group
of ontologies, specific format or design principles.
It is important that ontology repositories
continue to enhance ontology-based services and offer
new generic ones to enlarge the spectrum of
possible use of ontologies. Using standard
formats such as OWL or SKOS has facilitated the
development of a wide range of tools and services
for semantic resources. The challenge is now to
package them inside ontology repositories and
keep vertical quality (i.e., one ontology) while
enabling quantitative horizontal use.</p>
      <p>
        One important use of ontologies is for
annotating and indexing text data
        <xref ref-type="bibr" rid="ref36">(Spasic et al., 2005;
Handschuh and Staab, 2003)</xref>
        . Therefore, we often
see aside of ontology repositories, ontology-based
annotation services. For instances, BioPortal has
the NCBO Annotator
        <xref ref-type="bibr" rid="ref18">(Jonquet et al., 2009)</xref>
        , OLS
had Whatizit (Rebholz-Schuhmann et al., 2008)
and now moved to ZOOMA, HeTOP had
FMTI
        <xref ref-type="bibr" rid="ref33">(Sakji et al., 2010)</xref>
        and UMLS has
MetaMap
        <xref ref-type="bibr" rid="ref4">(Aronson, 2001)</xref>
        . Hereafter, we focus on
services for text data (annotation &amp; terminology
extraction).
      </p>
      <p>
        In Lossio-Ventura et al., (2014), we presented
BioTex, a Web application that implements
stateof-the-art measures for automatic extraction of
biomedical terms from English and French free text.
The application includes a new methodology for
automatic term extraction mixing linguistic,
statistical, graph and Web-based approaches that have
been demonstrated quite efficient
        <xref ref-type="bibr" rid="ref23">(Lossio-Ventura
et al., 2015)</xref>
        . Among other use of BioTex, we have
shown it can be part of an ontology enrichment
workflow that could be highly valuable for
ontology developers
        <xref ref-type="bibr" rid="ref24">(Lossio-Ventura et al., 2016)</xref>
        .
However, this work has not yet been incorporated
within an ontology repository technology.
      </p>
      <p>
        In Tchechmedjiev et al.,(2017), we present
multiple enhancement to the semantic annotation
workflow that we have developed on top of the
NCBO Annotator and when building a French
version of the service. Some of these new
functionalities are particularly relevant to process
electronic health records. These new features include:
annotation scoring
        <xref ref-type="bibr" rid="ref25 ref28 ref40">(Melzi and Jonquet, 2014)</xref>
        ,
additional output formats (for evaluation and
integration with standard clinical systems), clinical
context detection (negation, experiencer and
temporality through the integration of the
NegEx/ConText algorithm)
        <xref ref-type="bibr" rid="ref1 ref38 ref39">(Abdaoui et al., 2017)</xref>
        ,
coarse-grained entity type annotations (using
UMLS Semantic Groups, e.g., anatomy, disorders,
devices).
3.5
      </p>
    </sec>
    <sec id="sec-15">
      <title>Annotations and Linked Data</title>
      <p>
        Data integration and semantic interoperability
enable new scientific discoveries that could be made
by merging different currently available data.
These is one major reason for adopting ontologies.
They are used to design semantic indexes of data
and linked open datasets that could be used for
various type of cross datasets studies
        <xref ref-type="bibr" rid="ref7">(Handschuh
and Staab, 2003; Bizer et al., 2009)</xref>
        . Ontology
repositories must facilitate indexing/annotation,
search and access to semantically described,
interoperable, actionable, open, rich linked data
directly from the within the repositories.
Working with big data represents a set of challenges for
ontology repositories when designing these
semantic indexes: scalability, consistency,
completeness in a context where both ontologies and
data constantly evolve. In addition, cross
ontologies semantics and indexed data consistency shall
be checked by ontology repositories using OWL
reasoning.
      </p>
      <p>
        In Jonquet et al., (2011), we have built the
NCBO Resource Index, an ontology-based index
of more than twenty heterogeneous biomedical
resources (later extended to 50) included within
BioPortal. Directly when browsing the ontologies or
using a dedicated search engine, users can
discover datasets of interest. The indexing relied on the
NCBO Annotator workflow and used the
semantics that the ontologies encode, such as synonyms,
class hierarchies, and the mappings between
ontologies, to improve the search experience. The
Resource Index, was a tentative developed before
2010 that did not rely neither on big data
technologies and did not followed linked open data
principles. Both were in their infancies at that time.
More recently, in agronomy, we have followed
new efforts such as AgroLD project
        <xref ref-type="bibr" rid="ref42">(Venkatesan
et al., 2015)</xref>
        to build a database of resources
described in RDF, and annotated with ontologies.
We are currently working on the interoperation of
AgroLD and AgroPortal.
3.6
      </p>
    </sec>
    <sec id="sec-16">
      <title>Scalability &amp; interoperability</title>
      <p>
        In 2007, Swoogle claimed to “Search over 10.000
ontologies”. Today, a simple Google Search for
“filetype:owl” returns around 34K results. The
NCBO BioPortal, which is generally considered
has the biggest ontology repository (not library)
contains +650 ontologies as of end of 2017. More
and more vocabularies are being developed and
hosted by the LOV platform. Multiple domain
specific ontology repository efforts have started
often inspired by results in the biomedical domain
and usually by reusing NCBO technology (e.g.,
MMI OOR, AgroPortal, ESIPPortal). The more
ontologies and ontology repositories are being
developed, the more scalability and
interoperability issues become important. Some
ontologies are useful to different communities and shall
then be hosted in multiple repositories e.g.,
domain ontologies such as the Gene
Ontology
        <xref ref-type="bibr" rid="ref5">(Ashburner et al., 2000)</xref>
        , or the
Environment Ontology
        <xref ref-type="bibr" rid="ref9">(Buttigieg et al., 2013)</xref>
        .
Because no repository will host them all, ontology
repositories have to offer a certain level of
interoperability to ensure their users that they will
not have to work with multiple web applications
and programming interfaces if their ontologies of
interest are not all hosted by the same repositories.
As previously explained standard ontology
metadata is a crucial aspect to achieve this.
      </p>
      <p>In Jonquet et al., ; Jonquet et al., (2016b), our
projects described Section 2.3, we have been
particularly careful in not redeveloping features and
functionalities that to our knowledge were already
available. We have designed and implemented two
advanced prototype ontology repositories for the
French biomedical community and for the
agronomy domain. Our choice to reuse the NCBO
technology was justified by the large spectrum of
features and services, but in addition our motivation
was: (i) to avoid re-developing tools that have
already been designed and extensively used and
contribute to long term support of the commonly
used technology; and (ii) to offer the same tools,
services and formats to different but still
interconnected communities, to facilitate the interface and
interaction between their domains (agro, bio,
health (French)). Relying on the same original
technology enhance both technical reuse (for
example, enabling queries to either systems with the
same code), and semantic reuse. Then, we have
developed new functionalities –as previously
described– while keeping our systems backward
compatible with the original technology to
facilitate a convergence of the efforts. We strongly
believe that sharing the technology is the best way to
guaranty long term support and development by
engaging different ontology practitioners and
communities all around the world with their
respective funding and supporting schemes. Also,
sharing the technology is the best way to make
ontology repositories interoperable. As explained
in Tchechmedjiev et al.,(2017), all of the new
features implemented (e.g., NCBO Annotator + or
the new Recommender) are available across any
other NCBO based platform at minimum cost.
4</p>
    </sec>
    <sec id="sec-17">
      <title>Conclusions</title>
      <p>In this paper, we have presented our vision on
challenges and issues in building ontology
repositories. We have illustrated our thoughts with
results obtained over the last 10 years within our
projects in biomedicine and agronomy. By
adopting NCBO technology, we inherit some
advantages and inconvenients but we can now
contribute to this field of research with concrete use
cases, communities and outcomes. NCBO-based
ontology repositories adopted a vision where
multiple semantic resources are made available in a
common place (though not combined and
consistency checked), and cast to a common model.
While doing so, the repositories arguably limits
the full power of ontologies –which has been a
recurrent criticism– constraining their use to
features supported by the common model. We see
two general scenarios of use for these repositories:
The repositories provide basic ontology
library services for users with a “vertical need”
—those who want to do very precise things
(e.g., reasoning, using specific relations)
using only suitable ontologies (developed by the
same communities and in the same format).
Such users may just use the repositories as
libraries to find and download ontologies, and
work in their own environment.</p>
      <p>The repositories provide many
ontologybased services to users with “horizontal
needs” —those who wants to work with a
wide range of ontologies and vocabularies
useful in their domain but developed by
different communities, overlapping and in
different formats. Such users greatly appreciate
the unique endpoints (Web application and
programmatic for REST and SPARQL
queries) offered by the repositories under a
simplified common model.</p>
      <p>In this position paper, we have unfortunately not
covered all related work on the cited challenges
and we have certainly skipped other important
challenges: semantic consistency, ontology
evaluation, visualization, community feedback. But we
offered a short summary of multiple various
contributions on ontology repository and
ontologybased service research. In the future, we will
continue our efforts to address the identified
challenges (and others), while continue to offer to various
scientific communities the means to share and
leverage their ontologies or semantic resources
and enable new science in their fields.</p>
    </sec>
    <sec id="sec-18">
      <title>Acknowledgments</title>
      <p>This work is partly achieved within the Semantic
Indexing of French biomedical Resources (SIFR –
www.lirmm.fr/sifr) project that received funding
from the French National Research Agency (grant
ANR-12-JS02-01001), the European Union’s
Horizon 2020 research and innovation programme
under the Marie Sklodowska-Curie grant
agreement No 701771, the NUMEV Labex (grant
ANR-10-LABX-20), the Computational Biology
Institute of Montpellier (grant
ANR-11-BINF0002), as well as by the University of Montpellier
and the CNRS. I also acknowledge the National
Center for Biomedical Ontologies for their
insights and thanks all my collaborators in
Montpellier or Stanford interested like me on ontology
repositories.</p>
      <p>Philipp Cimiano, Elena</p>
      <p>Buitelaar, Mauricio</p>
      <p>Montiel-Ponsoda, Paul
Espinoza, and Asuncion
Four Million Mappings Can Tell You about Two
Hundred Ontologies. In A Bernstein, D R Karger, T
Heath, L Feigenbaum, D Maynard, E Motta, and K
Thirunarayan, editors, 8th International Semantic
Web Conference, ISWC’09, volume 5823, pages
229–242, Washington DC, USA. Springer.</p>
      <p>Jorge Gracia, Elena Montiel-Ponsoda, Daniel
VilaSuero, and Guadalupe Aguado-de Cea. 2014.
Enabling Language Resources to Expose
Translations as Linked Data on the Web. In N
Calzolari, K Choukri, T Declerck, H Loftsson, B
Maegaard, J Mariani, A Moreno, J Odijk, and S
Piperidis, editors, 9th International Conference on
Language Resources and Evaluation, LREC’14,
pages 409–4013, Reykjavik, Iceland. European
Language Resources Association.</p>
      <p>Siegfried Handschuh and Stephen Staab, editors.
2003. Annotation for the Semantic Web.volume 96
of Frontiers in Artificial Intelligence and
Applications. IOS Press.</p>
      <p>Jens Hartmann, Raúl Palma, and Asunción
GómezPérez. 2009. Ontology Repositories. Handbook on
Ontologies:551–571.</p>
      <p>Robert Hoehndorf, Luke Slater, Paul N Schofield, and
Georgios V Gkoutos. 2015. Aber-OWL: a
framework for ontology-based data access in
biology. BMC Bioinformatics, 16(1):1–9.</p>
      <p>Clement Jonquet, Amina Annane, Khedidja Bouarech,
Vincent Emonet, and Soumia Melzi. 2016a. SIFR
BioPortal : Un portail ouvert et générique
d’ontologies et de terminologies biomédicales
françaises au service de l’annotation sémantique. In
16th Journées Francophones d’Informatique
Médicale JFIM’16.</p>
      <p>Clement Jonquet, Amina Annane, Khedidja Bouarech,
Vincent Emonet, and Soumia Melzi. 2016b. SIFR
BioPortal : Un portail ouvert et générique
d’ontologies et de terminologies biomédicales
françaises au service de l’annotation sémantique. In
16th Journées Francophones d’Informatique
Médicale, JFIM’16.</p>
      <p>Clement Jonquet, Vincent Emonet, and Mark A
Musen. 2015. Roadmap for a multilingual
BioPortal. In J Gracia, J P McCrae, and G Vulcu,
editors, 4th Workshop on the Multilingual Semantic
Web, MSW4’15, volume 1532, pages 15–26,
Portoroz, Slovenia.</p>
      <p>Clement Jonquet, Paea LePendu, Sean Falconer,
Adrien Coulet, Natalya F Noy, Mark A Musen, and
Nigam H Shah. 2011. NCBO Resource Index:
Ontology-Based Search and Mining of Biomedical
Resources. Web Semantics, 9(3):316–324. 1st prize
of Semantic Web Challenge at the 9th International
Semantic Web Conference, ISWC’10, Shanghai,
China.
In M Horridge, M Rospocher, and J Ossenbruggen,
editors, 13th International Semantic Web
Conference, Demonstration, ISWC’14, volume
1272, pages 157–160, Riva del Garda, Italy.
Marcos Martinez-Romero, Clement Jonquet, Martin J
O’Connor, John Graybeal, Alejandro Pazos, and
Mark A Musen. 2017. NCBO Ontology
Recommender 2.0: An Enhanced Approach for
Biomedical Ontology Recommendation.</p>
      <p>Biomedical Semantics, 8(21).</p>
      <p>Depriest A Pouchard L. Huhns M. 2012. Lessons
learned in deploying a cloud-based knowledge
platform for the ESIP Federation. American
Geophysical Union Fall Meeting, poster
session:22725.</p>
      <p>Dietrich Rebholz-Schuhmann, Miguel Arregui,
Sylvain Gaudan, Harald Kirsch, and Antonio
Jimeno. 2008. Text processing through Web
services: Calling Whatizit. Bioinformatics,
24(2):296–298, January.</p>
      <p>Carlos Rueda, Luis Bermudez, and Janet Fredericks.
2009. The MMI Ontology Registry and Repository:
A Portal for Marine Metadata Interoperability. In
MTS/IEEE Biloxi - Marine Technology for Our
Future: Global and Local Challenges,
OCEANS’09, page 6, Biloxi, MS, USA.</p>
      <p>Marta Sabou, Vanessa Lopez, and Enrico Motta. 2006.</p>
      <p>Ontology Selection on the Real Semantic Web:
How to Cover the Queens Birthday Dinner? In S
Staab and V Svátek, editors, 15th International
Conference on Knowledge Engineering and
Knowledge Management Managing Knowledge in
a World of Networks, EKAW’06, volume 4248,
pages 96–111, Podebrady, Czech Republic.</p>
      <p>Springer.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <string-name>
            <given-names>Amine</given-names>
            <surname>Abdaoui</surname>
          </string-name>
          , Andon Tchechmedjiev,
          <string-name>
            <given-names>William</given-names>
            <surname>Digan</surname>
          </string-name>
          , Sandra Bringay, and
          <string-name>
            <given-names>Clement</given-names>
            <surname>Jonquet</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>French ConText: a Publicly Accessible System for Detecting Negation, Temporality and Experiencer in French Clinical Notes. Biomedical Informatics, UNDER REVI</article-title>
          .
          <article-title>Under review</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <string-name>
            <given-names>Amina</given-names>
            <surname>Annane</surname>
          </string-name>
          , Zohra Bellahsene, Faical Azouaou, and
          <string-name>
            <given-names>Clement</given-names>
            <surname>Jonquet</surname>
          </string-name>
          . 2016a.
          <article-title>Selection and Combination of Heterogeneous BK to Enhance Biomedical Ontology Matching</article-title>
          .
          <source>In 20th International Conference on Knowledge Engineering and Knowledge Management</source>
          ,
          <source>EKAW'16</source>
          , volume
          <volume>10024</volume>
          LNAI, Bologna, Italy.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <string-name>
            <given-names>Amina</given-names>
            <surname>Annane</surname>
          </string-name>
          , Vincent Emonet, Clement Jonquet, Faical Azouaou, and
          <string-name>
            <given-names>Clement</given-names>
            <surname>Jonquet</surname>
          </string-name>
          . 2016b.
          <article-title>Multilingual Mapping Reconciliation between English-French Biomedical Ontologies</article-title>
          .
          <source>In 6th International Conference on Web Intelligence</source>
          , Mining and Semantics, WIMS'
          <volume>16</volume>
          , volume
          <volume>13</volume>
          -15- June, page 12,
          <string-name>
            <surname>Nimes</surname>
          </string-name>
          , France. ACM.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <string-name>
            <surname>Alan R Aronson</surname>
          </string-name>
          .
          <year>2001</year>
          .
          <article-title>Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program</article-title>
          . In American Medical Informatics Association Annual Symposium, AMIA'
          <volume>01</volume>
          , pages
          <fpage>17</fpage>
          -
          <lpage>21</lpage>
          , Washington, DC, USA.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <string-name>
            <given-names>Michael</given-names>
            <surname>Ashburner</surname>
          </string-name>
          ,
          <article-title>Catherine A Ball, Judith A Blake, David Botstein</article-title>
          ,
          <string-name>
            <given-names>Heather</given-names>
            <surname>Butler</surname>
          </string-name>
          ,
          <string-name>
            <surname>J Michael Cherry</surname>
          </string-name>
          , Allan P Davis, Kara Dolinski, Selina S Dwight, Janan T Eppig,
          <article-title>and</article-title>
          <string-name>
            <surname>others.</surname>
          </string-name>
          <year>2000</year>
          .
          <article-title>Gene Ontology: tool for the unification of biology</article-title>
          .
          <source>Nature genetics</source>
          ,
          <volume>25</volume>
          (
          <issue>1</issue>
          ):
          <fpage>25</fpage>
          -
          <lpage>29</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <string-name>
            <given-names>Kenneth</given-names>
            <surname>Baclawski</surname>
          </string-name>
          and
          <string-name>
            <given-names>Todd</given-names>
            <surname>Schneider</surname>
          </string-name>
          .
          <year>2009</year>
          .
          <article-title>The open ontology repository initiative: Requirements and research challenges</article-title>
          . In T Tudorache,
          <string-name>
            <given-names>G</given-names>
            <surname>Correndo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N</given-names>
            <surname>Noy</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H</given-names>
            <surname>Alani</surname>
          </string-name>
          , and M Greaves, editors,
          <source>Workshop on Collaborative Construction, Management and Linking of Structured Knowledge, CK'09</source>
          , volume
          <volume>514</volume>
          , page 10, Washington, DC., USA. CEUR-WS.org.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <string-name>
            <given-names>Christian</given-names>
            <surname>Bizer</surname>
          </string-name>
          , Tom Heath, and
          <string-name>
            <surname>Tim</surname>
          </string-name>
          Berners-Lee.
          <year>2009</year>
          .
          <article-title>Linked Data - The Story So Far</article-title>
          .
          <source>Semantic Web and Information Systems</source>
          ,
          <volume>5</volume>
          (
          <issue>3</issue>
          ):
          <fpage>1</fpage>
          -
          <lpage>22</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <string-name>
            <given-names>Anila</given-names>
            <surname>Sahar</surname>
          </string-name>
          <string-name>
            <surname>Butt</surname>
          </string-name>
          , Armin Haller, and
          <string-name>
            <given-names>Lexing</given-names>
            <surname>Xie</surname>
          </string-name>
          .
          <year>2016</year>
          .
          <article-title>RecOn: Ontology recommendation for structureless queries</article-title>
          .
          <source>Applied Ontology</source>
          ,
          <volume>11</volume>
          (
          <issue>4</issue>
          ):
          <fpage>301</fpage>
          -
          <lpage>324</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          <string-name>
            <given-names>Pier</given-names>
            <surname>Luigi</surname>
          </string-name>
          <string-name>
            <surname>Buttigieg</surname>
          </string-name>
          , Norman Morrison,
          <string-name>
            <given-names>Barry</given-names>
            <surname>Smith</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Christopher J Mungall,</given-names>
            and
            <surname>Suzanna</surname>
          </string-name>
          <string-name>
            <given-names>E</given-names>
            and
            <surname>Lewis</surname>
          </string-name>
          .
          <year>2013</year>
          .
          <article-title>The environment ontology: contextualising biological and biomedical entities</article-title>
          .
          <source>Biomedical Semantics</source>
          ,
          <volume>4</volume>
          (
          <issue>1</issue>
          ):
          <fpage>43</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          <string-name>
            <surname>Gàmez-Pérez</surname>
          </string-name>
          .
          <year>2010</year>
          .
          <article-title>A note on ontology localization</article-title>
          .
          <source>Applied Ontology</source>
          ,
          <volume>5</volume>
          (
          <issue>2</issue>
          ):
          <fpage>127</fpage>
          -
          <lpage>137</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          <string-name>
            <surname>Richard G Côté</surname>
          </string-name>
          , Philip Jones, Rolf Apweiler, and
          <string-name>
            <given-names>Henning</given-names>
            <surname>Hermjakob</surname>
          </string-name>
          .
          <year>2006</year>
          .
          <article-title>The Ontology Lookup Service, a lightweight cross-platform tool for controlled vocabulary queries</article-title>
          .
          <source>BMC bioinformatics</source>
          ,
          <volume>7</volume>
          :
          <fpage>97</fpage>
          ,
          <string-name>
            <surname>January</surname>
          </string-name>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          <year>2010</year>
          . 1st Workshop on Ontology Repositories and
          <article-title>Editors for the Semantic Web</article-title>
          , ORES'
          <fpage>10</fpage>
          . In Mathieu d'Aquin, Alexander G Castro,
          <article-title>Christoph Lange</article-title>
          , and Kim Viljanen, editors,
          <source>number 596</source>
          ,
          <string-name>
            <surname>Hersonissos</surname>
          </string-name>
          , Greece.
          <source>CEUR-WS.org.</source>
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          <string-name>
            <surname>Mathieu D'Aquin and Natasha F Noy</surname>
          </string-name>
          .
          <year>2012</year>
          .
          <article-title>Where to Publish and Find Ontologies? A Survey of Ontology Libraries</article-title>
          . Web semantics,
          <volume>11</volume>
          (
          <year>August</year>
          ):
          <fpage>96</fpage>
          -
          <lpage>111</lpage>
          ,
          <year>March</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          <string-name>
            <given-names>Li</given-names>
            <surname>Ding</surname>
          </string-name>
          ,
          <string-name>
            <surname>Tim Finin</surname>
            , Anupam Joshi, Yun Peng,
            <given-names>R Scott</given-names>
          </string-name>
          <string-name>
            <surname>Cost</surname>
            , Joel Sachs, Rong Pan, Pavan Reddivari, and
            <given-names>Vishal</given-names>
          </string-name>
          <string-name>
            <surname>Doshi</surname>
          </string-name>
          .
          <year>2004</year>
          .
          <article-title>Swoogle: A Semantic Web Search and Metadata Engine</article-title>
          . In D A Grossman,
          <string-name>
            <given-names>L</given-names>
            <surname>Gravano</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C</given-names>
            <surname>Zhai</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O</given-names>
            <surname>Herzog</surname>
          </string-name>
          , and
          <string-name>
            <given-names>D</given-names>
            <surname>Evans</surname>
          </string-name>
          , editors,
          <source>13th ACM Conference on Information and Knowledge Management</source>
          ,
          <source>CIKM'04</source>
          ,
          <string-name>
            <surname>Washington</surname>
            <given-names>DC</given-names>
          </string-name>
          , USA. ACM.
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          <string-name>
            <given-names>Ying</given-names>
            <surname>Ding</surname>
          </string-name>
          and
          <string-name>
            <given-names>Dieter</given-names>
            <surname>Fensel</surname>
          </string-name>
          .
          <year>2001</year>
          .
          <article-title>Ontology Library Systems: The key to successful Ontology Re-use</article-title>
          .
          <source>In 1st Semantic Web Working Symposium, SWWS'01</source>
          , pages
          <fpage>93</fpage>
          -
          <lpage>112</lpage>
          , Stanford, CA, USA. CEUR-WS.org.
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <string-name>
            <given-names>Biswanath</given-names>
            <surname>Dutta</surname>
          </string-name>
          , Anne Toulet, Vincent Emonet, and
          <string-name>
            <given-names>Clement</given-names>
            <surname>Jonquet</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>New Generation Metadata vocabulary for Ontology Description and Publication</article-title>
          . In E Garoufallou,
          <string-name>
            <given-names>S</given-names>
            <surname>Virkus</surname>
          </string-name>
          , and G Alemu, editors,
          <source>11th Metadata and Semantics Research Conference, MTSR'17</source>
          , Tallinn, Estonia.
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          <string-name>
            <given-names>Daniel</given-names>
            <surname>Faria</surname>
          </string-name>
          , Ernesto Jiménez-Ruiz, Catia Pesquita, Emanuel Santos, and Francisco M Couto.
          <year>2014</year>
          .
          <article-title>Towards Annotating Potential Incoherences in BioPortal Mappings</article-title>
          . In 13th International Semantic Web Conference,
          <source>ISWC'13</source>
          , volume
          <volume>8797</volume>
          , pages
          <fpage>17</fpage>
          -
          <lpage>32</lpage>
          ,
          <source>Riva del Garda</source>
          ,
          <source>Italy</source>
          . Springer.
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          <string-name>
            <given-names>Amir</given-names>
            <surname>Ghazvinian</surname>
          </string-name>
          , Natasha F Noy,
          <string-name>
            <surname>Clement Jonquet</surname>
          </string-name>
          ,
          <string-name>
            <surname>Nigam H Shah</surname>
          </string-name>
          , and Mark A Musen.
          <year>2009</year>
          .
          <article-title>What Clement Jonquet, Mark A Musen,</article-title>
          and
          <string-name>
            <surname>Nigam H Shah</surname>
          </string-name>
          .
          <year>2010</year>
          .
          <article-title>Building a Biomedical Ontology Recommender Web Service</article-title>
          .
          <source>Biomedical Semantics</source>
          ,
          <volume>1</volume>
          (
          <issue>S1</issue>
          ). Selected in Pr. R.
          <article-title>Altman's 2011 Year in Review at AMIA TBI</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          <string-name>
            <given-names>Clément</given-names>
            <surname>Jonquet</surname>
          </string-name>
          , Anne Toulet, Elizabeth Arnaud, Sophie Aubin, Esther Dzalé-Yeumo, Vincent Emonet, John Graybeal, Mark A.
          <string-name>
            <surname>Musen</surname>
            , Cyril Pommier, and
            <given-names>Pierre</given-names>
          </string-name>
          <string-name>
            <surname>Larmande</surname>
          </string-name>
          . 2016c.
          <article-title>Reusing the NCBO BioPortal technology for agronomy to build AgroPortal</article-title>
          . In P Jaiswal and R Hoehndorf, editors,
          <source>7th International Conference on Biomedical Ontologies, ICBO'16</source>
          ,
          <string-name>
            <surname>Demo</surname>
            <given-names>Session</given-names>
          </string-name>
          , volume
          <volume>1747</volume>
          , pages
          <fpage>4</fpage>
          -
          <lpage>6</lpage>
          , Corvallis, Oregon, USA.
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          <string-name>
            <given-names>Clement</given-names>
            <surname>Jonquet</surname>
          </string-name>
          , Anne Toulet, Elizabeth Arnaud, Sophie Aubin, Esther Dzalé Yeumo, Vincent Emonet, John Graybeal,
          <string-name>
            <surname>Marie-Angélique</surname>
            <given-names>Laporte</given-names>
          </string-name>
          , Mark A Musen,
          <string-name>
            <given-names>Valeria</given-names>
            <surname>Pesce</surname>
          </string-name>
          , and
          <string-name>
            <given-names>Pierre</given-names>
            <surname>Larmande</surname>
          </string-name>
          .
          <year>2017a</year>
          .
          <article-title>AgroPortal: an ontology repository for agronomy. Computers and Electronics in Agriculture</article-title>
          , IN PRESS:30. Under review.
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          <string-name>
            <given-names>Clement</given-names>
            <surname>Jonquet</surname>
          </string-name>
          , Anne Toulet, Biswanath Dutta, and
          <string-name>
            <given-names>Vincent</given-names>
            <surname>Emonet</surname>
          </string-name>
          . 2017b.
          <article-title>Harnessing the power of unified metadata in an ontology repository: the case of AgroPortal. Data Semantics, UNDER REVI</article-title>
          .
          <article-title>Under review</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          <string-name>
            <surname>Maulik R Kamdar</surname>
          </string-name>
          ,
          <source>Tania Tudorache, and Mark A Musen</source>
          .
          <year>2017</year>
          .
          <article-title>A Systematic Analysis of Term Reuse and Term Overlap across Biomedical Ontologies</article-title>
          .
          <source>Semantic web</source>
          ,
          <volume>8</volume>
          (
          <issue>6</issue>
          ):
          <fpage>853</fpage>
          -
          <lpage>871</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          <string-name>
            <surname>Juan-Antonio</surname>
          </string-name>
          Lossio-Ventura, Clement Jonquet, Mathieu Roche, Maguelonne Teisseire, Juan Antonio, Lossio-ventura
          <string-name>
            <surname>Clement</surname>
            <given-names>Jonquet</given-names>
          </string-name>
          , Mathieu Roche, and
          <string-name>
            <given-names>Maguelonne</given-names>
            <surname>Teisseire</surname>
          </string-name>
          .
          <year>2015</year>
          .
          <article-title>Biomedical term extraction: overview and a new methodology</article-title>
          . Information Retrieval,
          <source>Special issue on Medical Information Retrieval</source>
          ,
          <volume>19</volume>
          (
          <issue>1</issue>
          ):
          <fpage>59</fpage>
          -
          <lpage>99</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          <string-name>
            <surname>Juan Antonio</surname>
            Lossio-Ventura, Clement Jonquet, Mathieu Roche, and
            <given-names>Maguelonne</given-names>
          </string-name>
          <string-name>
            <surname>Teisseire</surname>
          </string-name>
          .
          <year>2016</year>
          .
          <article-title>A Way to Automatically Enrich Biomedical Ontologies</article-title>
          .
          <source>In 19th International Conference on Extending Database Technology, EDBT'16</source>
          ,
          <string-name>
            <surname>Poster</surname>
            <given-names>Session</given-names>
          </string-name>
          , number
          <volume>305</volume>
          , page 2,
          <string-name>
            <surname>Bordeaux</surname>
          </string-name>
          , France. OpenProceedings.org.
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          Juan Antonio Juan Antonio Lossio-Ventura, Clement Jonquet, Mathieu Roche, and
          <string-name>
            <given-names>Maguelonne</given-names>
            <surname>Teisseire</surname>
          </string-name>
          .
          <year>2014</year>
          .
          <article-title>BIOTEX: A system for Biomedical Terminology Extraction, Ranking, and</article-title>
          <string-name>
            <surname>Validation.</surname>
          </string-name>
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          <string-name>
            <surname>John McCrae</surname>
            ,
            <given-names>Dennis</given-names>
          </string-name>
          <string-name>
            <surname>Spohr</surname>
            ,
            <given-names>and Philipp</given-names>
          </string-name>
          <string-name>
            <surname>Cimiano</surname>
          </string-name>
          .
          <year>2011</year>
          .
          <article-title>Linking lexical resources and ontologies on the semantic web with lemon</article-title>
          . In G Antoniou,
          <string-name>
            <given-names>M</given-names>
            <surname>Grobelnik</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E</given-names>
            <surname>Simperl</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B</given-names>
            <surname>Parsia</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D</given-names>
            <surname>Plexousakis</surname>
          </string-name>
          ,
          <string-name>
            <surname>P DeLeenheer</surname>
          </string-name>
          , and J Z Pan, editors,
          <source>8th Extended Semantic Web Conference, ESWC'11, number 6643</source>
          , pages
          <fpage>245</fpage>
          -
          <lpage>259</lpage>
          , Heraklion, Crete, Greece. Springer.
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          <string-name>
            <surname>Peter McQuilton</surname>
          </string-name>
          ,
          <string-name>
            <surname>Alejandra</surname>
          </string-name>
          Gonzalez-Beltran, Philippe Rocca-Serra, Milo Thurston, Allyson Lister, Eamonn Maguire, and
          <string-name>
            <surname>Susanna-Assunta Sansone</surname>
          </string-name>
          .
          <year>2016</year>
          .
          <article-title>BioSharing: curated and crowdsourced metadata standards, databases and data policies in the life sciences</article-title>
          .
          <source>Database</source>
          ,
          <year>baw075</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          <string-name>
            <given-names>Soumia</given-names>
            <surname>Melzi</surname>
          </string-name>
          and
          <string-name>
            <given-names>Clement</given-names>
            <surname>Jonquet</surname>
          </string-name>
          .
          <year>2014</year>
          .
          <article-title>Scoring semantic annotations returned by the NCBO Annotator</article-title>
          . In A Paschke,
          <string-name>
            <given-names>A</given-names>
            <surname>Burger</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P</given-names>
            <surname>Romano</surname>
          </string-name>
          ,
          <string-name>
            <surname>M S Marshall</surname>
          </string-name>
          , and A Splendiani, editors,
          <source>7th International Semantic Web Applications and Tools for Life Sciences, SWAT4LS'14</source>
          , volume
          <volume>1320</volume>
          ,
          <string-name>
            <surname>page</surname>
            <given-names>15</given-names>
          </string-name>
          , Berlin, Germany. CEUR-WS.org.
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          <string-name>
            <given-names>Elena</given-names>
            <surname>Montiel-Ponsoda</surname>
          </string-name>
          , Guadalupe Aguado de Cea,
          <string-name>
            <surname>Mari Carmen</surname>
            Suarez-Figueroa, Raul Palma, Asuncion Gomez-Pérez, and
            <given-names>Wim</given-names>
          </string-name>
          <string-name>
            <surname>Peters</surname>
          </string-name>
          .
          <year>2007</year>
          .
          <article-title>LexOMV: an OMV extension to capture multilinguality</article-title>
          . In P Buitelaar,
          <string-name>
            <given-names>K</given-names>
            <surname>Choi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A</given-names>
            <surname>Gangemi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C</given-names>
            <surname>Huang</surname>
          </string-name>
          , and A Oltramari, editors, Lexicon/Ontology Interface Workshop, OntoLex'07,
          <string-name>
            <surname>page</surname>
            <given-names>10</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Busan</surname>
          </string-name>
          , South-Korea.
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          <string-name>
            <given-names>Debashis</given-names>
            <surname>Naskar</surname>
          </string-name>
          and
          <string-name>
            <given-names>Biswanath</given-names>
            <surname>Dutta</surname>
          </string-name>
          .
          <year>2016</year>
          .
          <article-title>Ontology And Ontology Libraries: A Study From An Ontofier And An Ontologist Perspective</article-title>
          .
          <source>In 19th International Symposium on Electronic Theses and Dissertations</source>
          ,
          <source>ETD'16</source>
          , pages
          <fpage>1</fpage>
          -
          <lpage>12</lpage>
          , Lille, France.
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          <string-name>
            <surname>Natalya F Noy</surname>
          </string-name>
          ,
          <string-name>
            <surname>Nigam H Shah</surname>
          </string-name>
          , Patricia L Whetzel,
          <string-name>
            <surname>Benjamin Dai</surname>
          </string-name>
          , Michael Dorf, Nicholas B Griffith, Clement Jonquet, Daniel L Rubin,
          <string-name>
            <surname>Margaret-Anne</surname>
            <given-names>Storey</given-names>
          </string-name>
          , Christopher G Chute,
          <article-title>Mark A Musen, Cherie H Youn</article-title>
          , Adrien Coulet, Chris Callendar, and
          <string-name>
            <given-names>Barry</given-names>
            <surname>Smith</surname>
          </string-name>
          .
          <year>2009</year>
          .
          <article-title>BioPortal: ontologies and integrated data resources at the click of a mouse</article-title>
          . In B Smith, editor,
          <source>Nucleic Acids Research</source>
          , volume
          <volume>37</volume>
          , pages
          <fpage>170</fpage>
          -
          <lpage>173</lpage>
          , Stockholm, Sweden, July.
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          <string-name>
            <given-names>Edison</given-names>
            <surname>Ong</surname>
          </string-name>
          , Zuoshuang Xiang,
          <string-name>
            <given-names>Bin</given-names>
            <surname>Zhao</surname>
          </string-name>
          , Yue Liu,
          <string-name>
            <surname>Yu</surname>
            <given-names>Lin</given-names>
          </string-name>
          , Jie Zheng, Chris Mungall, Mélanie Courtot, Alan Ruttenberg, and
          <string-name>
            <given-names>Yongqun</given-names>
            <surname>He</surname>
          </string-name>
          .
          <year>2016</year>
          .
          <article-title>Ontobee: A linked ontology data server to support ontology term dereferencing, linkage, query and integration</article-title>
          .
          <source>Nucleic acids research</source>
          ,
          <volume>45</volume>
          (
          <issue>D1</issue>
          ):
          <fpage>D347</fpage>
          -
          <lpage>D352</lpage>
          , October.
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          <string-name>
            <given-names>Saoussen</given-names>
            <surname>Sakji</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Q</given-names>
            <surname>Gicquel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S</given-names>
            <surname>Pereira</surname>
          </string-name>
          ,
          <string-name>
            <given-names>I</given-names>
            <surname>Kergoulay</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D</given-names>
            <surname>Proux</surname>
          </string-name>
          ,
          <string-name>
            <surname>Darmoni</surname>
            <given-names>SJ</given-names>
          </string-name>
          , and
          <string-name>
            <given-names>M H</given-names>
            <surname>Metzger</surname>
          </string-name>
          .
          <year>2010</year>
          .
          <article-title>Evaluation of a French Medical Multi-Terminology Indexer for the Manual Annotation of Natural Language Medical Reports of HealthcareAssociated Infections</article-title>
          . In C Safran et al.,
          <source>editor, 13th World Congress on Medical Informatics</source>
          ,
          <source>MedInfo'10</source>
          , volume
          <volume>160</volume>
          , pages
          <fpage>252</fpage>
          -
          <lpage>256</lpage>
          ,
          <string-name>
            <surname>Cape</surname>
            <given-names>Town</given-names>
          </string-name>
          , South Africa. IOS Press.
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          <string-name>
            <given-names>Manuel</given-names>
            <surname>Salvadores</surname>
          </string-name>
          , Paul R Alexander,
          <article-title>Mark A Musen,</article-title>
          and Natalya F Noy.
          <year>2013</year>
          .
          <article-title>BioPortal as a dataset of linked biomedical ontologies and terminologies in RDF</article-title>
          .
          <source>Semantic Web</source>
          ,
          <volume>4</volume>
          (
          <issue>3</issue>
          ):
          <fpage>277</fpage>
          -
          <lpage>284</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          <string-name>
            <given-names>Barry</given-names>
            <surname>Smith</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Michael</given-names>
            <surname>Ashburner</surname>
          </string-name>
          , Cornelius Rosse, Jonathan Bard,
          <string-name>
            <given-names>William</given-names>
            <surname>Bug</surname>
          </string-name>
          , Werner Ceusters, Louis J Goldberg, Karen Eilbeck, Amelia Ireland, Christopher J Mungall,
          <string-name>
            <surname>The O B I Consortium</surname>
          </string-name>
          , Neocles Leontis, Philippe Rocca-Serra, Alan Ruttenberg,
          <string-name>
            <surname>Susanna-Assunta</surname>
            <given-names>Sansone</given-names>
          </string-name>
          , Richard H Scheuermann, Nigam H Shah, Patricia L Whetzel, and
          <string-name>
            <given-names>Suzanna</given-names>
            <surname>Lewis</surname>
          </string-name>
          .
          <year>2007</year>
          .
          <article-title>The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration</article-title>
          .
          <source>Nature Biotechnology</source>
          ,
          <volume>25</volume>
          (
          <issue>11</issue>
          ):
          <fpage>1251</fpage>
          -
          <lpage>1255</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          <string-name>
            <given-names>Irena</given-names>
            <surname>Spasic</surname>
          </string-name>
          , Sophia Ananiadou,
          <string-name>
            <surname>John McNaught</surname>
            , and
            <given-names>Anand</given-names>
          </string-name>
          <string-name>
            <surname>Kumar</surname>
          </string-name>
          .
          <year>2005</year>
          .
          <article-title>Text mining and ontologies in biomedicine: making sense of raw text</article-title>
          .
          <source>Briefings in Bioinformatics</source>
          ,
          <volume>6</volume>
          (
          <issue>3</issue>
          ):
          <fpage>239</fpage>
          -
          <lpage>251</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref37">
        <mixed-citation>
          <string-name>
            <surname>Suarez-Figueroa</surname>
            , Jens Hartmann, York Sure, Peter Haase, and
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Suarez-Figueroa</surname>
          </string-name>
          .
          <year>2005</year>
          .
          <article-title>OMVontology metadata vocabulary</article-title>
          . In C. Welty, editor,
          <source>Workshop on Ontology Patterns for the Semantic Web, WOP'05, page 9</source>
          ,
          <string-name>
            <surname>Galway</surname>
          </string-name>
          , Irland. Springer.
        </mixed-citation>
      </ref>
      <ref id="ref38">
        <mixed-citation>
          <string-name>
            <given-names>Andon</given-names>
            <surname>Tchechmedjiev</surname>
          </string-name>
          , Amine Abdaoui, Vincent Emonet, and
          <string-name>
            <given-names>Clement</given-names>
            <surname>Jonquet</surname>
          </string-name>
          .
          <year>2017a</year>
          .
          <article-title>ICD10 Coding of Death Certificates with the NCBO and SIFR Annotator(s) at CLEF eHealth 2017 Task 1</article-title>
          .
          <source>In Working Notes of CLEF eHealth Evaluation Lab</source>
          , volume
          <year>1866</year>
          , page 16, Dublin, Ireland.
        </mixed-citation>
      </ref>
      <ref id="ref39">
        <mixed-citation>
          <string-name>
            <given-names>Andon</given-names>
            <surname>Tchechmedjiev</surname>
          </string-name>
          , Amine Abdaoui, Vincent Emonet, Soumia Melzi, Jitendra Jonnagaddala, and
          <string-name>
            <given-names>Clement</given-names>
            <surname>Jonquet</surname>
          </string-name>
          . 2017b.
          <article-title>Enhanced Functionalities for Annotating and Indexing Clinical Text with the NCBO Annotator+</article-title>
          . Bioinformatics, UNDER REVI. Under Review.
        </mixed-citation>
      </ref>
      <ref id="ref40">
        <mixed-citation>
          <string-name>
            <given-names>Mossakowski</given-names>
            <surname>Till</surname>
          </string-name>
          , Oliver Kutz, and
          <string-name>
            <given-names>Mihai</given-names>
            <surname>Codescu</surname>
          </string-name>
          .
          <year>2014</year>
          .
          <article-title>Ontohub: A semantic repository for heterogeneous ontologies</article-title>
          .
          <source>In Theory Day in Computer Science, DACS'14, page 2</source>
          , Bucharest, Romania.
        </mixed-citation>
      </ref>
      <ref id="ref41">
        <mixed-citation>
          <string-name>
            <surname>Pierre-Yves</surname>
            <given-names>Vandenbussche</given-names>
          </string-name>
          ,
          <article-title>Ghislain A Atemezing, Maria Poveda-Villalon, and</article-title>
          <string-name>
            <given-names>Bernard</given-names>
            <surname>Vatant</surname>
          </string-name>
          .
          <year>2014</year>
          .
          <article-title>Linked Open Vocabularies (LOV): a gateway to reusable semantic vocabularies on the Web</article-title>
          .
          <source>Semantic Web</source>
          ,
          <volume>1</volume>
          :
          <fpage>1</fpage>
          -
          <lpage>5</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref42">
        <mixed-citation>
          <string-name>
            <given-names>Aravind</given-names>
            <surname>Venkatesan</surname>
          </string-name>
          , Nordine El Hassouni, Florian Phillipe, Cyril Pommier, Hadi Quesneville, Manuel Ruiz, and
          <string-name>
            <given-names>Pierre</given-names>
            <surname>Larmande</surname>
          </string-name>
          .
          <year>2015</year>
          .
          <article-title>Exposing French agronomic resources as Linked Open Data</article-title>
          . In James Malone, Robert Stevens, Kerstin Forsberg, and Andrea Splendiani, editors,
          <source>8th International Conference on Semantic Web Applications and Tools for Life Sciences, SWAT4LS'15</source>
          , pages
          <fpage>205</fpage>
          -
          <lpage>207</lpage>
          , Cambridge, UK. CEUR.
        </mixed-citation>
      </ref>
      <ref id="ref43">
        <mixed-citation>
          <string-name>
            <surname>Patricia L Whetzel and NCBO Team</surname>
          </string-name>
          .
          <year>2013</year>
          . NCBO Technology:
          <article-title>Powering semantically aware applications</article-title>
          .
          <source>Biomedical Semantics</source>
          ,
          <volume>4S1</volume>
          (
          <issue>S8</issue>
          ):
          <fpage>49</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref44">
        <mixed-citation>
          <year>2016</year>
          .
          <article-title>The FAIR Guiding Principles for scientific data management and stewardship</article-title>
          .
          <source>Scientific Data</source>
          ,
          <volume>3</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref45">
        <mixed-citation>
          1995.
          <article-title>A case study in ontology library construction</article-title>
          .
          <source>Artificial Intelligence in Medicine</source>
          ,
          <volume>7</volume>
          (
          <issue>3</issue>
          ):
          <fpage>227</fpage>
          -
          <lpage>255</lpage>
          , June.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>