=Paper=
{{Paper
|id=Vol-1766/oaei16_paper15
|storemode=property
|title=XMap results for OAEI 2016
|pdfUrl=https://ceur-ws.org/Vol-1766/oaei16_paper15.pdf
|volume=Vol-1766
|authors=Warith Eddine Djeddi,Mohamed Tarek Khadir,Sadok Ben Yahia
|dblpUrl=https://dblp.org/rec/conf/semweb/DjeddiKY16
}}
==XMap results for OAEI 2016==
https://ceur-ws.org/Vol-1766/oaei16_paper15.pdf
XMap : Results for OAEI 2016
Warith Eddine DJEDDIa,b , Mohamed Tarek KHADIRa and Sadok BEN YAHIAb
a
LabGED, Computer Science Department, University Badji Mokhtar, Annaba, Algeria
b
Faculty of Sciences of Tunis, University of Tunis El-Manar, LIPAH-LR 11ES14, 2092, Tunisia
{djeddi,khadir}@labged.net
sadok.benyahia@fst.rnu.tn
Abstract. We describe in this paper the XMap system and the results achieved
during the 2016 edition of the Ontology Alignment Evaluation Initiative. XMap
is an automated ontology matching system based on parallel composition of basic
ontology matchers and on the use of external resources as background knowledge.
1 Presentation of the system
XMap, as for eXtended Mapping, is one of the leading ontology matching systems for
large-scale ontology matching relying on the notion of context in order to deal with lex-
ical ambiguity as well as a divide-and-conquer approach to tackle the issue of matching
large ontologies.
A semantic similarity measure has been defined using UMLS [1] and WordNet [3]
to provide a synonymy degree between two entities from different ontologies, by ex-
ploring both of their lexical and structural contexts. The translation into many languages
is based on the Microsoft ⃝Translator.
R Our system stores locally all translation results
from Microsoft ⃝Translator
R in dictionary files. The translator will also be queried only
when no stored translation are found in order to gain time and avoid overloading the
server.
In this version, the system architecture remained unchanged but the system imple-
mentation was modified as well as the implementation of several basic matchers in order
to prepare the system for the following test sets: ”‘Interactive matching evaluation”’ and
”‘Disease and Phenotype”’ tracks.
2 State, purpose, general statement
As stated before, the architecture of the new version of the system remained unchanged
according to the version from 2015 [2]. We only added an interactive matcher [4] in
XMap using an oracle by modifying the validation process of the candidate mappings
according to the quality of the interactive matching in terms of F-measure and num-
ber of required interactions. This process is performed after each round of candidate
retrieving.
To recapitulate, our approach is based on semantic techniques and on a parallel
execution strategy, to address the challenge of scalability and efficiency of matching
techniques. One of the main trusts of the introduced approach is the increasing scala-
bility and speed of ontology alignment by matching linguistic and structural features. It
is a multi-layer system which uses three different layers to perform the ontology align-
ment process: a terminological layer, a structural layer and an alignment layer. The
output values of each layer serves as input to the upper one and each layer provides an
improvement in the computation of the similarity between concepts.
3 Results
In this section, we present the evaluation results obtained by running XMap under the
SEALS client with Benchmark, Anatomy, Conference, Multifarm, Interactive matching
evaluation, Large Biomedical Ontologies and Disease and Phenotype tracks.
Benchmark XMap performs very well on the biblio and film data set. Table 1 sum-
marises the average results obtained by XMap.
Table 1. Results for Benchmark track.
Test Precision Recall F-Measure
biblio 0.95 0.40 0.56
film 0.78 0.49 0.60
Anatomy The Anatomy track consists of finding an alignment between the Adult
Mouse Anatomy (2744 classes) and a part of the NCI Thesaurus (3304 classes) de-
scribing the human anatomy. XMap achieves a good F-Measure value of ≈89% in
a reasonable amount of time (45 sec.) (see Table 2). In terms of F-Measure/runtime,
XMap is ranked 3nd among the tools participated in this track.
Table 2. Results for Anatomy track.
System Precision F-Measure Recall Time(s)
XMap 0.929 0.896 0.865 45
Conference The Conference track uses a collection of 16 ontologies from the domain
of academic conferences. Most ontologies were equipped with OWL DL axioms of
various types; this opens a useful way to test our semantic matchers. The match quality
was evaluated against the original (ra1) as well as entailed reference alignment (ra2)
and violation free version of reference alignment (ra2). As Table 3 shows, for the three
evaluations, we achieved a good F-Measure values.
For each reference alignment, three evaluation modalities are applied : a) M1 only
contains classes, b) M2 only contains properties, c) M3 contains classes and properties.
XMap achieved the highest improvement between the 2016 and 2014 evaluation.
Table 3. Results for Conference track.
Precision F-Measure 1 Recall
Original reference alignment (ra1)
ra1-M1 0.86 0.73 0.63
ra1-M2 0.75 0.32 0.2
ra1-M3 0.85 0.68 0.57
Entailed reference alignment (ra2)
ra2-M1 0.81 0.68 0.58
ra2-M2 0.83 0.35 0.22
ra2-M3 0.81 0.63 0.52
Violation reference alignment (rar2)
rar2-M1 0.8 0.69 0.6
rar2-M2 0.83 0.35 0.22
rar2-M3 0.8 0.65 0.55
Multifarm This track is based on the translation of the OntoFarm collection of on-
tologies into 9 different languages. XMap have low performance due to many internal
exceptions. The results are showed in Table 4.
Table 4. Results for Multifarm track.
System Different ontologies Same ontologies
P F R P F R
XMap 0.30 0.007 0.003 0.00 0.00 0.00
Interactive matching evaluation For the 2016 edition, participating systems are eval-
uated on the Conference and Anatomy data set using an oracle based on the reference
alignment.
In this evaluation, we look at how interacting with the user improves the matching
results, which methods are most promising and how many interactions are necessary.
XMap uses various similarity measures to generate candidate mappings. It applies
two thresholds to filter the candidate mappings - one for the mappings that are directly
added to the final alignment and another for those that are presented to the user for
validation. The latter threshold is selected to be high in order to minimize the number of
requests and the rejected candidate mappings from the oracle; the requests are mainly
about incorrect mappings. The mappings accepted by the user are moved to the final
alignment. On the opposite side is XMap - it benefits the least from the interaction with
the oracle. All XMap’s measures differ with less than 0.2% from the non-interactive
runs, and performance does not change at all with the increasing error rates.
Large biomedical ontologies This track consists of finding alignments between the
Foundational Model of Anatomy (FMA), SNOMED CT, and the National Cancer Insti-
tute Thesaurus (NCI). The results obtained by XMAP are depicted by Table 5.
Table 5. Results for the Large BioMed track.
Test set Precision Recall F-Measure Time(s)
Small FMA-NCI 0.977 0.901 0.937 17
Whole FMA-NCI 0.902 0.847 0.874 116
Small FMA-SNOMED 0.989 0.846 0.912 54
Whole FMA- Large SNOMED 0.965 0.843 0.900 366
Small SNOMED-NCI 0.911 0.564 0.697 267
In general, we can conclude that XMap achieved a good precision/recall values. The
high recall value can be explained by the fact that UMLS thesaurus contains definitions
of highly technical medical terms.
Disease and Phenotype This track based on a real use case where it is required to find
alignments between disease and phenotype ontologies. Specifically, the selected ontolo-
gies are the Human Phenotype Ontology (HPO), the Mammalian Phenotype Ontology
(MP), the Human Disease Ontology (DOID), and the Orphanet and Rare Diseases On-
tology (ORDO).
XMap achieved fair results according to the three evaluation (Silver standard, Man-
ually generated mappings and Manual assessment of unique mappings).
4 General comments
4.1 Comments on the results
This is the 4th time that we participate in the OAEI campaign. The official results of
OAEI 2016 show that XMap is competitive with other well-known ontology matching
systems in all OAEI tracks. The current version of XMap has shown a significant im-
provement (both in terms of matching quality and runtime) in comparison to the version
from 2015 [2].
4.2 Comments on the OAEI 2016 procedure
As a fourth participation, we found the OAEI procedure very convenient and the orga-
nizers very supportive. The OAEI test cases are various, and this leads to a comparison
on different levels of difficulty, which is very interesting. We found that SEALS plat-
form is a precious tool to compare the performance of our system with the others.
5 Conclusion
In this paper, we presented the results achieved during the 2016 edition of the OAEI
campaign. The system managed to improve its performance significantly compared to
the previous year, which is reflected in the performance on several tracks. XMap par-
ticipated for the first year to the interactive track. The results are promising especially
on large-scale tasks which is a critical challenge in ontology matching.
References
1. Bodenreider, O. : The Unified Medical Language System (UMLS): integrating biomedical
terminology, Nucleic Acids Research, vol. 32, D267-D270 (2004).
2. Djeddi, W., Khadir, M. T. & Ben Yahia, S.: XMap: results for OAEI 2015. In Proceedings
of the 10th International Workshop on Ontology Matching collocated with the 14th Interna-
tional Semantic Web Conference (ISWC 2015), October 12, pp. 216–221. Bethlehem, PA, USA
(2015).
3. Fellbaum, C. : WordNet: An Electronic Lexical Database, MIT Press, Cambridge, MA (1998).
4. Paulheim, H., Hertling, S. & Ritze, D.: Towards evaluating interactive ontology matching
tools. In Proc. 10th Extended Semantic Web Conference (ESWC), Montpellier (FR), pp. 31–
45, (2015).