=Paper=
{{Paper
|id=Vol-1545/oaei15_paper17
|storemode=property
|title=XMap: results for OAEI 2015
|pdfUrl=https://ceur-ws.org/Vol-1545/oaei15_paper17.pdf
|volume=Vol-1545
|dblpUrl=https://dblp.org/rec/conf/semweb/DjeddiKY15
}}
==XMap: results for OAEI 2015==
https://ceur-ws.org/Vol-1545/oaei15_paper17.pdf
XMap : Results for OAEI 2015
Warith Eddine DJEDDIa,b , Mohamed Tarek KHADIRa and Sadok BEN YAHIAb
a
LabGED, Computer Science Department, University Badji Mokhtar, Annaba, Algeria
b
Faculty of Sciences of Tunis, University of Tunis El-Manar, LIPAH-LR 11ES14, 2092, Tunisia
{djeddi,khadir}@labged.net
sadok.benyahia@fst.rnu.tn
Abstract. This paper describes the configuration of XMap for the OAEI 2015
competition and discusses its results. XMap is able to automatically adapt to the
matching task, choosing the best conguration for the given pair of ontologies. This
is our third participation in the OAEI and we can see an overall improvement on
nearly every task.
1 State, purpose, general statement
XMap [1] [2] is a highly scalable ontology matching system, which is able to deal with
hundreds of thousands of entities with an efficient computation time [3]. It is a fast and
effective high precision system able to perform matching large ontologies. A semantic
similarity measure has been defined using UMLS and WordNet [4] to provide a syn-
onymy degree between two entities from different ontologies, by exploring both their
lexical and structural context. XMap relies on the Microsoft Translate API to translate
ontologies into many languages.
1.1 Specific techniques used
A high-level view of mapping process is depicted in Figure 1. It is a multi-layer system
which uses three different layers to perform the ontology alignment process: a termino-
logical layer, a structural layer and an alignment layer. The output values of each layer
serves as input to the upper one and each layer provides an improvement in the compu-
tation of the similarity between concepts. Figure 1 shows the architecture of the XMap
system.
Matchers in XMap are the algorithms that compare two ontologies and return an
alignment between them. The matchers employed various strategies (entity label, struc-
tural description of concepts, range for relations, instantiated attributes or extensional
descriptions) in each layer which are listed below:
a) Terminological Layer
The terminological layer is responsible for carrying out the process of computing
the similarity between the entity names within the ontologies, combining linguistic sim-
ilarity with the semantic elements of the context of the entities. This layer receives as
inputs the values of the string similarity, the linguistic similarity, the semantic similar-
ity and translation-based similarity computed within the lexical-semantic module. The
output variable represents the terminological similarity:
Fig. 1. Architecture for XMAP.
1. The string strategies usually can be applied to the name, the label or the comments
concerning entities to discover those which are similar. In general, it can be used
for comparing class names and/or URIs. The scaled range is [0, 1] for compar-
ing strings. Our system applies many terminological approaches for computing the
similarity measures between two terms: the Levenshtein distance, the Jaro Winkler
distance, the n-grams, the Jaccard distance, the Cosine, etc. Note that XMap does
not currently store or use comments, definitions, nor instances;
2. The linguistic strategies explore the semantic similarity of the concepts and rela-
tions labels. The linguistic based matchers use the external resources WordNet and
UMLS to find the semantic similarity between two entities;
3. The translation-based strategies use an automatic translation for obtaining correct
matching pairs in multilingual ontology matching. The translation is done locally
by querying Microsoft Translator for the full name;
4. The semantic strategies based on auxiliary sources use a domain knowledge avail-
able from external sources, such as WordNet, to find additional information for the
concepts (synonyms) and the relationships between them. The semantic similarity
is incorporated with the aim of adding context information of the concepts during
the mapping process.
b) Structural Layer
The structural layer performs two key tasks related to the structure of ontologies. One
is the computation of the similarity between the concepts taking into account the taxo-
nomic hierarchy, as well as the computation of the similarity using the information of
the internal structure of concepts, i.e., their properties, types and cardinality restrictions:
1. Structural strategies are usually based on the internal structure of an entity or its
relations to other entities as a source of detecting correspondences. The first using
the relational structure of concepts in the ontology, specifically the taxonomic hi-
erarchy, and the second using the information of the internal structure of concepts,
including their properties, types and cardinality restrictions;
2. Constraint strategies consider the concepts and properties data types and cardinal-
ities. They are usually used to provide supplementary information, not as primary
matchers (i.e., sting matcher or linguistic matcher); these techniques consider crite-
ria regarding the internal structure of the entities, such as the domain and range of
the properties or the types of the attributes, to calculate the similarity between them.
c) Alignment Layer
The alignment layer is the final layer and its aim is to provide the final similarity
matrix between the concepts taking into account the influence of the number of
properties and the value of similarity that properties bring to the final similarity be-
tween them. Once the similarity between ontology entities are available based on
different strategies (e.g., string similarity, semantic similarity, structural similarity),
aggregating similarities algorithms are needed to combine matchers. Combining
and filtering the similarity values obtained from the different matchers, comes most
often, to combine similarity values using three types of aggregation operator; these
strategies are aggregation, selection and combination [2]. Furthermore, those pairs
of concepts with similarity values equal to or greater than a particular threshold are
retained in order to obtain the mapping suggestions.
For the requirements of different ontology matching tasks, the selected alignment
in XMap can be one to one, one to many, or many to many alignments. Whereas
in our case, the desired cardinality in ontology matching is typically one-to-one.
The matching rules are created via the Java API Alignment Format, allowing the
generation of outputs in different formats.
2 Results
In this section, we present the evaluation results obtained by running XMap under
the SEALS client with Benchmark, Anatomy, Conference, Multifarm and Large
Biomedical Ontologies tracks. Adding to that, we present the results of the test
Ontology Alignment for Query Answering which not follow the classical ontology
alignment evaluation on the SEALS platform.
Benchmark XMap performs very well in terms of Precision (1.0) while flagging
out a low recall (0.4) in the Benchmark track. Those low values are explained by
the fact that ontological entities with scrambled labels and lexical similarity become
ineffective. Whereas for the others two test suites our algorithm performed worse in
terms of F-Measure because our system does not handle ontology instances. Table
1 summarises the average results obtained by XMap.
Table 1. Results for Benchmark track.
Test Precision Recall F-Measure
biblio 1.0 0.40 0.57
energy 1.0 0.22 0.51
Anatomy The Anatomy track consists of finding an alignment between the Adult
Mouse Anatomy (2744 classes) and a part of the NCI Thesaurus (3304 classes) de-
scribing the human anatomy. XMap achieves a good F-Measure value of ≈89% in
a reasonable amount of time (50 sec.) (see Table 2). In terms of F-Measure/runtime,
XMap is ranked 2nd among the 15 tools participated in this track.
Table 2. Results for Anatomy track.
System Precision F-Measure Recall Time(s)
XMap 0.928 0.896 0.865 50
Conference The Conference track uses a collection of 16 ontologies from the
domain of academic conferences. Most ontologies were equipped with OWL DL
axioms of various types; this opens a useful way to test our semantic matchers. The
match quality was evaluated against the original (ra1) as well as entailed reference
alignment (ra2) and violation free version of reference alignment (ra2). As Table 3
shows, for the three evaluations, we achieved a good F-Measure values.
For each reference alignment, three evaluation modalities are applied : a) M1 only
contains classes, b) M2 only contains properties, c) M3 contains classes and prop-
erties.
Table 3. Results for Conference track.
Precision F-Measure 1 Recall
Original reference alignment (ra1)
ra1-M1 0.86 0.73 0.63
ra1-M2 0.67 0.22 0.13
ra1-M3 0.85 0.68 0.56
Entailed reference alignment (ra2)
ra2-M1 0.81 0.68 0.58
ra2-M2 0.78 0.25 0.15
ra2-M3 0.81 0.63 0.51
Violation reference alignment (rar2)
rar2-M1 0.8 0.69 0.62
rar2-M2 0.78 0.27 0.16
rar2-M3 0.8 0.64 0.54
Multifarm This track is based on the translation of the OntoFarm collection of
ontologies into 9 different languages. XMap’s results are showed in Table 4.
Table 4. Results for Multifarm track.
System Different ontologies Same ontologies
P F R P F R
XMap 0.22 0.24 0.27 0.66 0.37 0.27
Large biomedical ontologies This track consists of finding alignments between
the Foundational Model of Anatomy (FMA), SNOMED CT, and the National Can-
cer Institute Thesaurus (NCI). There are 5 sub-tasks corresponding to different
sizes of input ontologies (small fragments and whole ontology for FMA and NCI
and small and large fragments for SNOMED CT). XMAP has been evaluated with
two variants: XMAP-BK and XMAP. XMAP-BK uses synonyms provided by the
UMLS Metathesaurus, while XMAP has this feature desactivated. The results ob-
tained by XMAP-BK are depicted by Table 6. XMAP-BK provided the best results
Table 5. Results for the Large BioMed track.
Test set Precision Recall F-Measure Time(s)
Small FMA-NCI 0.971 0.902 0.935 31
Whole FMA-NCI 0.872 0.849 0.860 337
Small FMA-SNOMED 0.968 0.847 0.903 49
Whole FMA- Large SNOMED 0.769 0.844 0.805 782
Small SNOMED-NCI 0.928 0.606 0.733 396
Whole NCI- Large SNOMED 0.913 0.536 0.675 925
(ranked 1st) among the 12 participating systems in terms of F-measure in FMA-
NCI and FMA-SNOMED matching sub-tasks. In general, we can conclude that
XMap achieved a good precision/recall values. The high recall value can be ex-
plained by the fact that UMLS thesaurus contains definitions of highly technical
medical terms.
Ontology Alignment for Query Answering The objective of this test is to check
the ability of the generated alignments to answer a set of queries in an ontology-
based data access scenario where several ontologies exist. Table 6 shows the F-
measure results for the whole set of queries. XMap was one of the 5 matchers
whose alignments allowed to answer all the queries of the evaluation.
Table 6. Results for Ontology Alignment for Query Answering.
System RA1 Reference RAR1 Reference
P R F P R F
XMap 0.778 0.675 0.702 0.720 0.654 0.671
3 General comments
3.1 Comments on the results
This is the third time that we participate in the OAEI campaign. We foresee an
improvement in the performance of our system which consists of expanding the
supported domain of matching problems, such that large-scale biomedical or multi-
lingual ontologies can be matched as well. The official results of OAEI 2015 show
that XMap is competitive with other well-known ontology matching systems in
all OAEI tracks. The current version of XMap has shown a significant improve-
ment both in terms of matching quality and runtime. Additionally, to improve
our f-measure for large biomedical ontologies we made use of the UMLS Meta-
thesaurus. Finally, we pre-compiling a local dictionary in order to avoid multiple
accesses to the Microsoft Translator during the matching process.
3.2 Comments on the OAEI 2015 procedure
As a third participation, we found the OAEI procedure very convenient and the
organizers very supportive. The OAEI test cases are various, and this leads to a
comparison on different levels of difficulty, which is very interesting. We found
that SEALS platform is a precious tool to compare the performance of our system
with the others.
4 Conclusion
In this paper, we presented the results achieved during the 2015 edition of the OAEI
campaign. The system managed to improve its performance significantly compared
to the previous year, which is reflected in the performance of the different tracks.
We have used the UMLS resource for better discarding incorrect mappings for life
sciences related ontologies. Moreover, we implemented a cross-lingual ontology
matching approach in order to align ontologies in different languages.
References
1. Djeddi, W., Khadir, M. T.: A Novel Approach Using Context-Based Measure for Match-
ing Large Scale Ontologies. In Proceedings of 16th International Conference on Data
Warehousing and Knowledge Discovery (DAWAK 2014), September 2-4, pp. 320–331.
Springer, Munich, Germany (2014)
2. Djeddi, W., Khadir, M.T.: Ontology alignment using artificial neural network for large-
scale ontologies. In the International Journal of Metadata, Semantics and Ontologies
(IJMSO), Vol.8, No.1, pp.75-92 (2013)
3. Dragisic Z., Eckert K., Euzenat J., Faria D., Ferrara A., Granada R., Ivanova V., & al.:
Results of the Ontology Alignment Evaluation Initiative 2014. In Proceedings of the 9th
International Workshop on Ontology Matching collocated with the 13th International Se-
mantic Web Conference (ISWC 2014), Riva del Garda, Trentino, Italy, October 20, 61–
104, (2014).
4. Fellbaum, C. : WordNet: An Electronic Lexical Database, MIT Press, Cambridge, MA
(1998)