<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Web-based Ontology Alignment with the GeneTegra Alignment tool</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Nemanja Stojanovic</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Ray M. Bradley</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Sean Wilkinson</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Mansur Kabuka</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>E. Patrick Shironoshita INFOTECH Soft</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Brickell Avenue</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Suite</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Miami</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Florida</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>USA [nemanja</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>rbradley</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>kabuka</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>patrick]@infotechsoft.com</string-name>
        </contrib>
      </contrib-group>
      <fpage>127</fpage>
      <lpage>132</lpage>
      <abstract>
        <p>Ontologies are increasingly gaining practical usage for semantic data in various ways and across multiple domains. From this growing applicability arises an evergreater need to manage large datasets, reduce analytical complexity and efficiently as well as accurately integrate different heterogeneous ontologies into or within existing systems, all while minimizing data corruption and maintaining existing semantics. In this paper, we present the GeneTegra Alignment Tool (GT-Align), a practical implementation of the ASMOV ontology alignment algorithm within a Web-based interface, focusing on biomedical data and using Unified Medical Language System (UMLS) for the background knowledge. GT-Align allows iterative alignment of multiple ontologies as well as active user involvement throughout the process.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        Ontologies have been increasingly acknowledged
as an appropriate abstraction instruments for
representing entities and their relationships within
various domains
        <xref ref-type="bibr" rid="ref1 ref4">(Euzenat and Shvaiko, 2007)</xref>
        .
Due to this abstract expressiveness, ontologies
have been proven to have a highly extensible
applicability spectrum, allowing a greater variety of
systems to incorporate them in their modeling
        <xref ref-type="bibr" rid="ref10 ref12">(Kalfoglou and Schorlemmer, 2003; Noy, 2004;)</xref>
        .
Because of this increasing development, the need
for flexible tools enabling semantic matching of
heterogeneous ontologies is becoming much more
apparent
        <xref ref-type="bibr" rid="ref13 ref2">(Shvaiko and Euzenat, 2013)</xref>
        .
      </p>
      <p>In this paper, we present a demonstration of an
ontology alignment Web interface called
GTAlign, consisting of a server implementation that
wraps an ontology alignment algorithm and
exposes REST API endpoints which the client side
user interface employs to enable iterative
alignment of biomedical ontologies.</p>
      <p>
        Our solution to the problem of ontology
alignment is twofold. First, we use an ontology
alignment algorithm to identify shared relationships
between heterogeneous entities and generate a set of
suggested mappings. Second, we allow the user to
engage with the results on each iteration by
accepting, rejecting or clearing (reverting an
acceptance or a rejection) mappings or mapping
groups. The interface also allows the user to
upload a set of equivalence mappings as an input
partial alignment to bootstrap a new alignment
process. All mappings must be positively
accepted; in other words, no mappings are deemed
accepted until positively indicated as such by the
user. Rejection of a mapping is an indication that
such a mapping should never happen. Clearing of
a mapping, on the other hand, indicates that it is
not accepted but still possible. The GT-Align Web
Interface supports manual evaluation of results
through visual inspection, including inspection of
parents and children of elements as well as
inspection of labels and other textual information. The
tool also provides information on the confidence
of a mapping as calculated by the underlying
ASMOV algorithm
        <xref ref-type="bibr" rid="ref6 ref8 ref9">(Jean-Mary et al., 2009;
JeanMary and Kabuka, 2014)</xref>
        , and where applicable it
also provides reference to codes in the Unified
Medical Language System (UMLS) to which
concepts are tagged. For algorithm details and the
explanation on UMLS usage, see Algorithm section.
      </p>
      <p>The main goal of GT-Align is to enable easier
ontology alignment of biomedical data and allow
domain experts to validate the results and thus
ensure high quality alignments. Put succinctly,
GTAlign enables the production of an alignment
between any two biomedical ontologies, allowing
users to review and revise mappings interactively.</p>
    </sec>
    <sec id="sec-2">
      <title>Algorithm</title>
      <p>
        The underlying alignment algorithm for GT-Align
is called ASMOV, which was developed for use
in the integration of data and ontologies in the
biomedical and life sciences domain within the
GeneTegra Information System
(www.genetegra.com). The algorithm makes
use of an iterative approach with similarity
calculations along multiple dimensions coupled with a
process of semantic verification that seeks to
remove mapping inconsistencies. ASMOV uses a
combination of string-, constraint-,
formalresource-, graph-, model-, and instance-based
matching mechanisms. ASMOV has participated
in several rounds of the evaluations performed by
the Ontology Alignment Evaluation Initiative
(OAEI), placing as one of the top three performers
in the benchmark tests of the contests in which it
has participated
        <xref ref-type="bibr" rid="ref1 ref4 ref5 ref6 ref6 ref8 ref8">(Jean-Mary and Kabuka, 2007;
Jean-Mary and Kabuka, 2008; Jean-Mary et al.,
2009; Jean-Mary et al., 2009)</xref>
        . The ASMOV
algorithm uses UMLS as an underlying vocabulary
aimed at improving lexical matching between
source and target entities. The interface enables
the user to turn off this feature, in which case
lexical matching is based on Levenshtein edit
distance. Prior evaluations of the algorithm showed
that the use of an underlying vocabulary
significantly improves the quality of mapping, while
also reducing the time needed for completion of the
alignment process
        <xref ref-type="bibr" rid="ref1 ref4">(Jean-Mary and Kabuka, 2007)</xref>
        .
3
      </p>
    </sec>
    <sec id="sec-3">
      <title>User Interface</title>
      <p>
        Visualization types that have been shown as
most effective at enabling user involvement in an
alignment process are tree and graph structures,
with both having specific benefits to the user
        <xref ref-type="bibr" rid="ref2">(Bo
Fu, 2013)</xref>
        . Furthermore,
        <xref ref-type="bibr" rid="ref3">Granitzer et al. (2010)</xref>
        have shown that an intelligent combination of
both structures is present in many advanced
alignment visualization tools. By combining list,
tree and graph visualizations to present alignment
data to the user at distinct levels of abstraction,
GT-Align can yield a more productive alignment
through user feedback. The user can explore
detailed information on individual concepts as well
as parameters of mapping candidates such as
estimated confidence and status. Furthermore, the
user can filter this data by ontological sub regions
or by individual mapping features.
      </p>
      <p>
        Additionally, the GT-Align user interface is built
on modern web technologies including
JavaScript/HTML/CSS as well as SVGs for data
visualizations, enabling GT-Align to stay on the cutting
edge of UI tools
        <xref ref-type="bibr" rid="ref11">(Li et al., 2015)</xref>
        . In addition to a
wide platform support (including mobile), web
technologies maintain consistent high quality of
UI capabilities via frequent improvements. This
enables GT-Align to be easily deployed into any
environment as well as quickly updated with latest
technological advances at a minimum expense to
the user.
      </p>
      <p>The following sections focus on the individual
visualizations and capabilities of the different
views present in the GT-Align Web Interface.
3.1</p>
      <sec id="sec-3-1">
        <title>Ontology Import View</title>
        <p>This view allows the user to upload ontologies
into the system, which they can further inspect in
the Hierarchical Tree View. The ontology import
process uses an extensible set of rules to
normalize lexical labels used within it, marking one as
the preferred label and others as alternative labels.
These labels are then annotated to concepts within
the UMLS Metathesaurus. Having normalized
labels provides a consistent visual identification
scheme that is more easily recognized and thus
friendlier to the user.
3.2</p>
      </sec>
      <sec id="sec-3-2">
        <title>Hierarchical Tree View</title>
        <p>
          This part of the system displays ontologies as a
hierarchical tree of concepts. The ontology tree
visualization serves as the fundamental
visualization in GT-Align. The view is shown in Figure 1.
When concepts are asserted as children of
multiple parents, they are displayed within each parent,
as is standard practice. Metadata about the
ontology, including its ID and any annotations such as
textual descriptions, are displayed on an
information pane. The system also utilizes an
autocomplete search for individual concepts within the
ontology. The indented tree visualization presents
information in a commonly used abstraction
allowing users to explore ontologies without a
specialized knowledge of the visualization itself. This
enables anybody familiar with the concept of an
ontology to get started with the software very
quickly. The usefulness of an a tree visualization
in displaying hierarchical relationships has been
demonstrated by its long-term usage in many
areas. From visualizing file systems or HTML
structures to visualizing ontologies in tools such as
WebProtégé
          <xref ref-type="bibr" rid="ref14">(Tudorache et al., 2013)</xref>
          , an indented
tree visualization is familiar to most, enabling
quicker onboarding into the GT-Align system.
3.3
        </p>
      </sec>
      <sec id="sec-3-3">
        <title>Alignment Execution View</title>
        <p>Through this view, the user can execute an
alignment with specific parameters. The user starts by
selecting two ontologies that will be aligned. Each of
the ontologies can either be selected from the set of
ontologies that were previously added or may be
uploaded by the system. Additionally, the user can
upload a partial alignment as an input parameter to
the alignment process.
3.4</p>
      </sec>
      <sec id="sec-3-4">
        <title>Alignment Selection View</title>
        <p>This view provides a tabular summary of all the
alignments executed in the system. It contains the
historical overview the alignment processes ran by
the user along with details about each process. An
example of this view is shown in Figure 2.
Clicking on a specific column heading will sort the
table according to the corresponding parameter. The
displayed parameters include links to the source and
target ontologies used in the alignment, the custom
name supplied by the user along with the date and
time for when the alignment was created or last
modified and how long it took to execute. The view
also displays the current execution status which gets
updated as an alignment progresses through
different stages. Alignments can execute asynchronously
within the GT-Align platform, allowing the user to
perform tasks in parallel. The extensive
computational work is offloaded to the server and doesn’t
hinder the user experience. Once an alignment is
completed, the user receives a notification of its
final status. Selection of an alignment transitions
the user to the Alignment Overview.
3.5</p>
      </sec>
      <sec id="sec-3-5">
        <title>Alignment Overview</title>
        <p>After an alignment is obtained, the mappings are
presented in an overview pane with a circular
graph layout. Due to the structure of ontologies, a
graph-based visualization is a natural fit for
displaying their alignment. Unlike indented trees,
graphs are more suitable to display multiple
inheritance without any visual redundancy. This
prevents the user of potentially needing to make
additional efforts when understanding the data at hand
or being confused by concept repetition. Tree
visualization is particularly less adequate when
displaying large ontologies because the expansion of
nodes to greater depths can quickly become
overwhelming. Large trees also make it difficult to
access the overall structure of an ontology. Using a
graph visualization allows us to handle large
amounts of data in a way that is more
customizable and flexible. The graph visualization is shown
in Figure 3.</p>
        <p>Concepts from both ontologies are distinguished
in the graph by color and positioning. They are
separated based on their originating ontology
where the concepts from the source ontology are
placed on one side and the concepts from the
target ontology on another. The user can rotate the
graph as they please. Further clustering of
concepts is performed based on their hierarchical
position in the ontology. The closer a concept is to
the root, the closer it is to the center of the graph.
Conversely, the outer section of the graph
represents the leaf nodes. This design allows the user to
easily asses the structure of both ontologies while
performing minimal work.</p>
        <p>Two concepts connected with a line represent a
single mapping. Thickness of this line
corresponds to the confidence value, i.e. the level of
confidence in the mapping being correct,
according to the underlying algorithm. The thicker the
line the higher the generated mapping confidence.
This lets the user evaluate a section of the
alignment by the amount of high- or low- confidence
mappings it contains.</p>
        <p>The user can click on a specific mapping or
group of mappings under a common parent.
Selection of a group of mappings transitions to the
Mapping Group View, while selection of any
individual mappings transitions to the Mapping
View. At the top is a toolbar that provides control
to filter the mappings by the confidence value.
Additionally, the mappings can be filtered by the
mapping origin (suggested by algorithm, found by
algorithm, provided by partial alignment),
mapping state (accepted by user, rejected by user,
undefined), and a branch in the ontology. This
filtering is automatically reflected in the graph,
allowing users to quickly see the alignment overview at
different scales. Besides filtering, the toolbar
allows for bulk editing of mappings, enabling the
user to accept, reject or clear mappings for large
sections of the alignment. The toolbar additionally
allows the user to export the mappings through the
Alignment RDF format and the EDOAL format,
as well as a merged OWL ontology.
3.6</p>
      </sec>
      <sec id="sec-3-6">
        <title>Mapping Group View</title>
        <p>This view, shown in Figure 4, displays the
suggested mappings for a concept and its children.
The main purpose of this view is to allow the user
to examine a group of mappings separately from
the alignment as a whole. The mappings are
shown as a vertical list of concept pairs, giving the
user an alternative presentation to the graph that is
familiar and straightforward.</p>
        <p>Each concept pair contains a rectangular
visualization of their mapping confidence on a scale of
0-100. The mapping state is show above each of
the confidence visualizations. Control buttons are
provided allowing the user to alter the mapping
state of the whole group as well as of individual
mappings within it. Selection of an individual
mapping transitions to the Mapping View section.
This multi-pane view, shown in Figure 5, provides
an in-depth visualization of a single mapping,
allowing the user to review or modify the mapping.</p>
        <p>Top left pane: This pane contains information
about the currently focused source concept
selected by the user, including a preferred label,
alternative labels, and annotated UMLS codes if
available. All other panes display information in relation
to this currently focused concept.</p>
        <p>Top right pane: This pane contains a selected
target concept for mapping to the currently
focused concept. In the center between the top right
and left panes are controls allowing the user to
accept or reject the mapping between the focused
and selected concept, or create a new one if none
yet exists. The controls also include an option for
the user to swap the focused and selected
concepts, causing other panes to adjust accordingly.
Three central panes under the controls show one
or more mappings in different states.</p>
        <p>Bottom center panes: The top pane shows the
accepted mapping for the focused source concept,
if one exists. The middle central pane shows a list
of possible mappings according to ASMOV. The
top concept in this list is the mapping suggested
by ASMOV, other mappings are alternative
possibilities. The bottom central pane shows a list of
rejected mappings, if any exist. In all three panes,
mappings show their preferred label and mapping
confidence value. Clicking on a concept in any of
these panes places it on the selected mapping
pane, making it the new focused concept.</p>
        <p>Bottom side panes: Two bottom panes on each
side contain the hierarchical tree view of both
ontologies (see Hierarchical Tree View). On the
bottom left is the ontology tree corresponding to the
focused source concept, and on the bottom right is
the ontology tree corresponding to the selected
target concept. Each ontology tree highlights the
selection of the corresponding concepts. All
ancestors of both concepts are shown in the
respective tree view along with all siblings of each
ancestor, but children of ancestor siblings are
initially hidden although the user can explore them if
desired. Clicking on a concept within the left
ontology tree will set it as the currently focused
concept, updating all other panes accordingly.
Clicking on any concept from the right ontology tree
places it on the selected concept pane, choosing it
as the mapping for the source concept.
4</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Future Work</title>
      <p>Previous sections highlight the current status and
main features of the GT-Align Web Interface.
There are several planned features expected to be
produced in the future. Subsequent releases will
allow users to specify subsumption relationships
in addition to equivalence relationships. GT-Align
will include the capability of performing an
automated evaluation of precision and recall against a
reference alignment, and to display the results of
such evaluation. Additionally, it will incorporate
functionality to dynamically change the set of
fixed weights for the various similarity values
calculated by ASMOV. It will also present the
separate confidence scores for the different measures
of similarity generated by ASMOV. GT-Align
will support pivot systems such as EDOAL to
allow importing of alignments from other systems.
Finally, GT-Align will support interactive
collaboration by multiple users.
5</p>
    </sec>
    <sec id="sec-5">
      <title>Conclusion</title>
      <p>In this paper, we presented GT-Align, a versatile
implementation of an ontology alignment Web
interface, backed by an efficient, robust and
fieldtested algorithm called ASMOV. Besides the
algorithmic alignment, the interface enables
iterative user involvement, allowing domain experts to
improve and validate the results thus contributing
to the quality of the alignment. We showed
features to evaluate relationships between entities in
different biomedical ontologies as well as explore
ontologies on their own through hierarchical trees.
The interface enables users to analyze aligned
mappings from a high-level perspective through
groups refined by optional user filters and
combined with algorithm results. It further provides
capabilities for fine grain inspection of individual
concepts through their mappings as well as
relations to other concepts. An assortment of
visualizations provided by the user-interface enables
multiple perspectives on the data itself along with
the alignment results. Additionally, we presented
our plans for future development. In summary,
GT-Align is a robust and easy to use solution for
ontology alignment of biomedical data.</p>
    </sec>
    <sec id="sec-6">
      <title>Acknowledgements</title>
      <p>This work is supported by grant R44GM097851
from the National Institute of General Medical
Sciences (NIGMS), part of the U.S. National
Institutes of Health (NIH).</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <string-name>
            <surname>Euzenat</surname>
            , Jérôme, and
            <given-names>Pavel</given-names>
          </string-name>
          <string-name>
            <surname>Shvaiko</surname>
          </string-name>
          .
          <article-title>Ontology matching</article-title>
          . Vol.
          <volume>18</volume>
          . Heidelberg: Springer,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <string-name>
            <surname>Fu</surname>
            , Bo,
            <given-names>Natalya F.</given-names>
          </string-name>
          <string-name>
            <surname>Noy</surname>
          </string-name>
          , and
          <string-name>
            <surname>Margaret-Anne Storey</surname>
          </string-name>
          .
          <article-title>"Indented tree or graph? A usability study of ontology visualization techniques in the context of class mapping evaluation."</article-title>
          <source>In International Semantic Web Conference</source>
          , pp.
          <fpage>117</fpage>
          -
          <lpage>134</lpage>
          . Springer, Berlin, Heidelberg,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <string-name>
            <surname>Granitzer</surname>
            , Michael, Vedran Sabol, Kow Weng Onn, Dickson Lukose, and
            <given-names>Klaus</given-names>
          </string-name>
          <string-name>
            <surname>Tochtermann</surname>
          </string-name>
          .
          <article-title>"Ontology alignment-a survey with focus on visually supported semi-automatic techniques." Future Internet 2</article-title>
          , no.
          <issue>3</issue>
          (
          <year>2010</year>
          ):
          <fpage>238</fpage>
          -
          <lpage>258</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <string-name>
            <surname>Jean-Mary</surname>
          </string-name>
          , Yves R., and
          <string-name>
            <surname>Mansur</surname>
            <given-names>R.</given-names>
          </string-name>
          <string-name>
            <surname>Kabuka</surname>
          </string-name>
          .
          <article-title>"ASMOV results for OAEI 2007</article-title>
          .
          <article-title>"</article-title>
          <source>In Proceedings of the 2nd International Conference on Ontology Matching-Volume</source>
          <volume>304</volume>
          , pp.
          <fpage>150</fpage>
          -
          <lpage>159</lpage>
          . CEUR-WS. org,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <string-name>
            <surname>Jean-Mary</surname>
          </string-name>
          , Yves R., and
          <string-name>
            <surname>Mansur</surname>
            <given-names>R.</given-names>
          </string-name>
          <string-name>
            <surname>Kabuka</surname>
          </string-name>
          .
          <article-title>"ASMOV: results for OAEI 2008”</article-title>
          <source>In Proceedings of the 3rd International Workshop on Ontology Matching-Volume</source>
          <volume>431</volume>
          , pp.
          <fpage>132</fpage>
          -
          <lpage>139</lpage>
          . CEUR-WS. org,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <string-name>
            <surname>Jean-Mary</surname>
            , Yves R.,
            <given-names>E. Patrick</given-names>
          </string-name>
          <string-name>
            <surname>Shironoshita</surname>
          </string-name>
          , and
          <string-name>
            <surname>Mansur</surname>
            <given-names>R.</given-names>
          </string-name>
          <string-name>
            <surname>Kabuka</surname>
          </string-name>
          .
          <article-title>"ASMOV: Results for OAEI 2009”</article-title>
          <source>In Proceedings of the 4th International Workshop on Ontology Matching-Volume</source>
          <volume>551</volume>
          , pp.
          <fpage>152</fpage>
          -
          <lpage>159</lpage>
          . CEUR-WS. org,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <string-name>
            <surname>Jean-Mary</surname>
            , Yves R.,
            <given-names>E. Patrick</given-names>
          </string-name>
          <string-name>
            <surname>Shironoshita</surname>
          </string-name>
          , and
          <string-name>
            <surname>Mansur</surname>
            <given-names>R.</given-names>
          </string-name>
          <string-name>
            <surname>Kabuka</surname>
          </string-name>
          .
          <article-title>"ASMOV: Results for OAEI 2010”</article-title>
          <source>In Proceedings of the 5th International Workshop on Ontology Matching-Volume</source>
          <volume>689</volume>
          , pp.
          <fpage>126</fpage>
          -
          <lpage>133</lpage>
          . CEUR-WS. org,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <string-name>
            <surname>Jean-Mary</surname>
            , Yves R.,
            <given-names>E. Patrick</given-names>
          </string-name>
          <string-name>
            <surname>Shironoshita</surname>
          </string-name>
          , and
          <string-name>
            <surname>Mansur</surname>
            <given-names>R.</given-names>
          </string-name>
          <string-name>
            <surname>Kabuka</surname>
          </string-name>
          .
          <article-title>"Ontology matching with semantic verification."</article-title>
          <source>Web Semantics: Science, Services and Agents on the World Wide Web</source>
          <volume>7</volume>
          , no.
          <issue>3</issue>
          (
          <year>2009</year>
          ):
          <fpage>235</fpage>
          -
          <lpage>251</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          <string-name>
            <surname>Jean-Mary</surname>
          </string-name>
          , Yves R., and
          <string-name>
            <surname>Kabuka</surname>
          </string-name>
          , Mansur.
          <article-title>"Ontology alignment with semantic validation."</article-title>
          <source>U.S. Patent</source>
          <volume>8</volume>
          ,
          <issue>738</issue>
          ,636, issued May 27,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          <string-name>
            <surname>Kalfoglou</surname>
            , Yannis, and
            <given-names>Marco</given-names>
          </string-name>
          <string-name>
            <surname>Schorlemmer</surname>
          </string-name>
          .
          <article-title>"Ontology mapping: the state of the art." The knowledge engineering review 18, no</article-title>
          .
          <volume>01</volume>
          (
          <year>2003</year>
          ):
          <fpage>1</fpage>
          -
          <lpage>31</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          <string-name>
            <surname>Li</surname>
            , Yiting,
            <given-names>Cosmin</given-names>
          </string-name>
          <string-name>
            <surname>Stroe</surname>
            ,
            <given-names>and Isabel F.</given-names>
          </string-name>
          <string-name>
            <surname>Cruz</surname>
          </string-name>
          .
          <article-title>"Interactive Visualization of Large Ontology Matching Results."</article-title>
          <source>In VOILA@ ISWC</source>
          , p.
          <fpage>37</fpage>
          .
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          <string-name>
            <surname>Noy</surname>
          </string-name>
          , Natalya F.
          <article-title>"Semantic integration: a survey of ontology-based approaches</article-title>
          .
          <source>" ACM Sigmod Record</source>
          <volume>33</volume>
          , no.
          <issue>4</issue>
          (
          <year>2004</year>
          ):
          <fpage>65</fpage>
          -
          <lpage>70</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          <string-name>
            <surname>Shvaiko</surname>
            , Pavel, and
            <given-names>Jérôme</given-names>
          </string-name>
          <string-name>
            <surname>Euzenat</surname>
          </string-name>
          .
          <article-title>"Ontology matching: state of the art and future challenges." IEEE Transactions on knowledge and data engineering 25</article-title>
          , no.
          <issue>1</issue>
          (
          <year>2013</year>
          ):
          <fpage>158</fpage>
          -
          <lpage>176</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          <string-name>
            <surname>Tudorache</surname>
            , Tania, Csongor Nyulas,
            <given-names>Natalya F.</given-names>
          </string-name>
          <string-name>
            <surname>Noy</surname>
            , and
            <given-names>Mark A.</given-names>
          </string-name>
          <string-name>
            <surname>Musen</surname>
          </string-name>
          .
          <article-title>"WebProtégé: A collaborative ontology editor and knowledge acquisition tool for the web." Semantic web 4</article-title>
          , no.
          <issue>1</issue>
          (
          <year>2013</year>
          ):
          <fpage>89</fpage>
          -
          <lpage>99</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>