We describe in this paper the XMap system and the results achieved during the 2016 edition of the Ontology Alignment Evaluation Initiative. XMap is an automated ontology matching system based on parallel composition of basic ontology matchers and on the use of external resources as background knowledge. Presentation of the system XMap, as for eXtended Mapping, is one of the leading ontology matching systems for large-scale ontology matching relying on the notion of context in order to deal with lexical ambiguity as well as a divide-and-conquer approach to tackle the issue of matching large ontologies. A semantic similarity measure has been defined using UMLS [1] and WordNet [3] to provide a synonymy degree between two entities from different ontologies, by exploring both of their lexical and structural contexts. The translation into many languages is based on the Microsoft ⃝R Translator. Our system stores locally all translation results from Microsoft ⃝R Translator in dictionary files. The translator will also be queried only when no stored translation are found in order to gain time and avoid overloading the server. In this version, the system architecture remained unchanged but the system implementation was modified as well as the implementation of several basic matchers in order to prepare the system for the following test sets: ”'Interactive matching evaluation”' and ”'Disease and Phenotype”' tracks.
techniques. One of the main trusts of the introduced approach is the increasing scalability and speed of ontology alignment by matching linguistic and structural features. It is a multi-layer system which uses three different layers to perform the ontology alignment process: a terminological layer, a structural layer and an alignment layer. The output values of each layer serves as input to the upper one and each layer provides an improvement in the computation of the similarity between concepts. 3
In this section, we present the evaluation results obtained by running XMap under the SEALS client with Benchmark, Anatomy, Conference, Multifarm, Interactive matching evaluation, Large Biomedical Ontologies and Disease and Phenotype tracks. Benchmark XMap performs very well on the biblio and film data set. Table 1 summarises the average results obtained by XMap. Anatomy The Anatomy track consists of finding an alignment between the Adult Mouse Anatomy (2744 classes) and a part of the NCI Thesaurus (3304 classes) describing the human anatomy. XMap achieves a good F-Measure value of 89% in a reasonable amount of time (45 sec.) (see Table 2). In terms of F-Measure/runtime, XMap is ranked 3nd among the tools participated in this track. Conference The Conference track uses a collection of 16 ontologies from the domain of academic conferences. Most ontologies were equipped with OWL DL axioms of various types; this opens a useful way to test our semantic matchers. The match quality was evaluated against the original (ra1) as well as entailed reference alignment (ra2) and violation free version of reference alignment (ra2). As Table 3 shows, for the three evaluations, we achieved a good F-Measure values.
For each reference alignment, three evaluation modalities are applied : a) M1 only contains classes, b) M2 only contains properties, c) M3 contains classes and properties.
XMap achieved the highest improvement between the 2016 and 2014 evaluation. ra1-M1 ra1-M2 ra1-M3 ra2-M1 ra2-M2 ra2-M3 rar2-M1 rar2-M2 rar2-M3 Multifarm This track is based on the translation of the OntoFarm collection of ontologies into 9 different languages. XMap have low performance due to many internal exceptions. The results are showed in Table 4. Interactive matching evaluation For the 2016 edition, participating systems are evaluated on the Conference and Anatomy data set using an oracle based on the reference alignment.
In this evaluation, we look at how interacting with the user improves the matching results, which methods are most promising and how many interactions are necessary.
XMap uses various similarity measures to generate candidate mappings. It applies two thresholds to filter the candidate mappings - one for the mappings that are directly added to the final alignment and another for those that are presented to the user for validation. The latter threshold is selected to be high in order to minimize the number of requests and the rejected candidate mappings from the oracle; the requests are mainly about incorrect mappings. The mappings accepted by the user are moved to the final alignment. On the opposite side is XMap - it benefits the least from the interaction with the oracle. All XMap’s measures differ with less than 0:2% from the non-interactive runs, and performance does not change at all with the increasing error rates. Large biomedical ontologies This track consists of finding alignments between the Foundational Model of Anatomy (FMA), SNOMED CT, and the National Cancer Institute Thesaurus (NCI). The results obtained by XMAP are depicted by Table 5.
In general, we can conclude that XMap achieved a good precision/recall values. The high recall value can be explained by the fact that UMLS thesaurus contains definitions of highly technical medical terms.
Disease and Phenotype This track based on a real use case where it is required to find alignments between disease and phenotype ontologies. Specifically, the selected ontologies are the Human Phenotype Ontology (HPO), the Mammalian Phenotype Ontology (MP), the Human Disease Ontology (DOID), and the Orphanet and Rare Diseases Ontology (ORDO).
XMap achieved fair results according to the three evaluation (Silver standard, Manually generated mappings and Manual assessment of unique mappings). 4 4.1
This is the 4th time that we participate in the OAEI campaign. The official results of
OAEI 2016 show that XMap is competitive with other well-known ontology matching
systems in all OAEI tracks. The current version of XMap has shown a significant
improvement (both in terms of matching quality and runtime) in comparison to the version
from 2015 [
As a fourth participation, we found the OAEI procedure very convenient and the organizers very supportive. The OAEI test cases are various, and this leads to a comparison on different levels of difficulty, which is very interesting. We found that SEALS platform is a precious tool to compare the performance of our system with the others.
In this paper, we presented the results achieved during the 2016 edition of the OAEI campaign. The system managed to improve its performance significantly compared to the previous year, which is reflected in the performance on several tracks. XMap participated for the first year to the interactive track. The results are promising especially on large-scale tasks which is a critical challenge in ontology matching.