<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Ontologies Guidelines for Best Practice and a Process to Evaluate Existing Ontologies Mapping Tools and Algorithms</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Ian Harrow</string-name>
          <email>ian.harrow@pistoiaalliance.org</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Martin Romacker</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Andrea Splendiani</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Stefan Negru</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Peter Woollard</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Scott Markel</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Yasmin Alam-Faruque</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Martin Koch</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Erfan Younesi</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>James Malone</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Ernesto Jimenez-Ruiz</string-name>
          <email>ernesto@cs.ox.ac.uk</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Computer Science, University of Oxford</institution>
          ,
          <country country="UK">UK</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Pistoia Alliance Ontologies Mapping Project, Pistoia Alliance Inc.</institution>
          <country country="US">USA</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>This extended abstract presents an ongoing work by the Pistoia Alliance Ontologies Mapping project to develop user requirements for an ontologies mapping service.</p>
      </abstract>
      <kwd-group>
        <kwd>Ontologies</kwd>
        <kwd>Mapping</kwd>
        <kwd>Guidelines</kwd>
        <kwd>Tool</kwd>
        <kwd>Algorithm</kwd>
        <kwd>Evaluation</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>The Pistoia Alliance Ontologies Mapping project1 was set up to nd or create
better tools or services for mapping between ontologies in the same domain
and to establish best practices for ontology management in the Life Sciences. It
was proposed through the Pistoia Alliance Ideas Portfolio Platform2 which was
selected by the Pistoia Alliance Operations Team for development of a formal
business case.</p>
      <p>The project has delivered a set of guidelines for best practice which build on
existing standards. We show how these guidelines can be used as a \checklist"
to support the application and mapping of source ontologies in the disease and
phenotype domain. These guidelines are accessible on a public wiki.3. The project
has contributed a process to evaluate existing ontologies mapping tools and a
new phenotype track at OAEI (OM) 2016 to support evaluation of ontology
matching (OM) algorithms.</p>
      <p>User Requirements and Evaluation of existing</p>
      <p>Ontologies Mapping Tools
Another important output of this project was to specify the requirements for an
Ontologies Mapping Tool. The three major aspects of the user requirements are
illustrated in Figure 1. This shows the speci cation of the 1) User interface; 2)
Framework, including ontology matching algorithms and 3) Import and export
of mappings.</p>
      <p>These requirements were used in a preliminary survey that established that
such tools already exist which substantially meet them. Therefore, we have
developed a formal process to de ne and submit a request for information (RFI)
from existing ontologies mapping tool providers. We will summarise our
ndings from this evaluation of seven ontologies mapping tools from academic and
commercial providers on the poster. The guidelines and RFI materials, including
the speci c details of the requirements are accessible on the same public wiki,
mentioned previously.
3</p>
      <p>Evaluation of Ontology Matching Algorithms for
Disease and Phenotype
A critical component of any Ontologies Mapping tool is the embedded
ontology matching algorithm. Therefore, the Pistoia Alliance Ontologies Mapping
1 http://www.pistoiaalliance.org/projects/ontologies-mapping
2 IP3:https://www.qmarkets.org/live/pistoia/home
3 https://pistoiaalliance.atlassian.net/wiki/display/PUB/Ontologies+</p>
      <p>Mapping+Resources
Project is supporting development and evaluation of ontology matching
algorithms though sponsorship and organisation of the new Disease and Phenotype
track for OAEI (OM) 2016,4 which is also be summarised in this section. This
new track has been organised because currently, mappings between ontologies in
a given data domain are mostly curated by bioinformatics and disease experts in
academia or industry, who would bene t from automation of their procedures.
This could be accomplished through implementation of ontology matching
algorithms into their existing work ow environment or investment in an ontologies
mapping tool for management of the ontologies mapping life cycle.
3.1</p>
    </sec>
    <sec id="sec-2">
      <title>OAEI Disease and Phenotype Dataset</title>
      <p>There will be two tasks for this novel OAEI track which will comprise of pairwise
alignment of:5
{ Human Phenotype Ontology6 (HPO) to Mammalian Phenotype Ontology7
(MP), and
{ Human Disease Ontology8 (DOID) to the Orphanet and Rare Diseases
Ontology9 (ORDO).</p>
      <p>The rst task is important for translational science where human phenotype
includes inherited diseases and mammalian phenotype ontology which is based on
rodents as a model mammalian organism for many laboratory studies, including
gene knock out. The second task includes representation of rare human diseases
in both ontologies which are of fundamental importance for understanding how
genetic variation can cause disease.</p>
      <p>
        We have extracted a \baseline" reference alignments for the track based
on the available BioPortal mappings [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. Most of the BioPortal mappings are
automatically generated by the LOOM system;10 therefore this BioPortal-based
reference alignment will only be considered as a baseline since it is incomplete
or may contain erroneous mappings.
3.2
      </p>
    </sec>
    <sec id="sec-3">
      <title>OAEI Evaluation Process</title>
      <p>
        The evaluation of the Disease and Phenotype Track will be run with support of
the SEALS infrastructure.11 Systems will be evaluated and ranked according to
the following criteria:
4 The OAEI is an annual campaign for the evaluation of ontology matching systems [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]:
http://oaei.ontologymatching.org/
5 http://oaei.ontologymatching.org/2016/phenotype/index.html
6 http://bioportal.bioontology.org/ontologies/HP
7 http://bioportal.bioontology.org/ontologies/MP
8 http://bioportal.bioontology.org/ontologies/DOID
9 http://bioportal.bioontology.org/ontologies/ORDO
10 http://www.bioontology.org/wiki/index.php/BioPortal_Mappings
11 http://oaei.ontologymatching.org/2016/seals-eval.html
{ Precision and Recall with respect to a voted reference alignment that will
be built automatically to generate consensus voting for the outputs of the
participating systems using the standard evaluation process.
{ Recall with respect to manually generated mappings for three areas
(carbohydrate, obesity and breast cancer). This will be a novel aspect of the
evaluation.
{ Manual assessment of a subset of the generated mappings, especially the
ones that are not suggested by other systems. This will be a novel aspect of
the evaluation.
{ Performance in other tracks will also be taken into account (especially the
interactive session at the OAEI 2016 campaign [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]).
4
      </p>
      <sec id="sec-3-1">
        <title>Summary and Plans</title>
        <p>This poster describes access to ontologies guidelines and a process to evaluate
existing tools and algorithms for ontology matching. The Ontologies Mapping
project is currently de ning the user requirements for a sustainable service, which
will use such OM tools or algorithms. We will conduct a survey of Pistoia Alliance
members to understand the need for such a service and whether it should be
implemented in future.</p>
      </sec>
      <sec id="sec-3-2">
        <title>Acknowledgements</title>
        <p>This work was partially funded by the Pistoia Alliance Ontology Mappings
project, the EU project Optique (FP7-ICT-318338), and the EPSRC projects
ED3 and DBOnto.</p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Cheatham</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dragisic</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Euzenat</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Faria</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ferrara</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Flouris</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fundulaki</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Granada</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ivanova</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jimenez-Ruiz</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lambrix</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Montanelli</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pesquita</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Saveta</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shvaiko</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Solimando</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>dos Santos</surname>
          </string-name>
          , C.T.,
          <string-name>
            <surname>Zamazal</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          :
          <article-title>Results of the ontology alignment evaluation initiative 2015</article-title>
          .
          <source>In: Proceedings of the 10th International Workshop on Ontology Matching</source>
          . (
          <year>2015</year>
          )
          <volume>60</volume>
          {
          <fpage>115</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <given-names>Fridman</given-names>
            <surname>Noy</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            ,
            <surname>Shah</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.H.</given-names>
            ,
            <surname>Whetzel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.L.</given-names>
            ,
            <surname>Dai</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            ,
            <surname>Dorf</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            ,
            <surname>Gri</surname>
          </string-name>
          <string-name>
            <given-names>th</given-names>
            , N.,
            <surname>Jonquet</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            ,
            <surname>Rubin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.L.</given-names>
            ,
            <surname>Storey</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.A.D.</given-names>
            ,
            <surname>Chute</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.G.</given-names>
            ,
            <surname>Musen</surname>
          </string-name>
          ,
          <string-name>
            <surname>M.A.</surname>
          </string-name>
          :
          <article-title>BioPortal: ontologies and integrated data resources at the click of a mouse</article-title>
          .
          <source>Nucleic Acids Research</source>
          <volume>37</volume>
          (
          <string-name>
            <surname>Web-Server-Issue)</surname>
          </string-name>
          (
          <year>2009</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Dragisic</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ivanova</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lambrix</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Faria</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jimenez-Ruiz</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pesquita</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>User validation in ontology alignment</article-title>
          . In: International Semantic Web Conference. (
          <year>2016</year>
          )
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>