The XMapGen and XMapSig systems are flexible and self-configuring matching tools using different strategies for combining multiple similarity measures into a single aggregated metric with the final aim of improving the ontology alignment quality of large scale ontologies. XMapGen and XMapSig are two variants of XMap++. The results obtained by the two ontology matching tools within the 9th edition of the Ontology Alignment Evaluation Initiative (OAEI 2013) campaign are therefore presented.
1.1
XMapGen and XMapSig are a scalable ontology alignment tools capable of matching English language ontologies described in different OWL languages (i.e., OWL Lite, OWL DL, and OWL Full). The major principle of the matching strategy in XMapGen and XMapSig approaches is combining multiple similarity measures into a single similarity metric using weights determined by intelligent strategies in order to skip over the burden of manual selection. Despite the impressive strategy in adding GA, aligning medium-sized and large-scale ontologies is still very time consuming and computationally expensive. This inspires us to consider the use of a particular parallel matching on multiple cores or machines for dealing with the scalability issue on ontology matching. 1.2
In this section, the workflow of XMap++ and its main components is briefly described and shown in Fig.1. Both systems XMapGen and XMapSig calculate three different basic measures to create three similarity matrixes. String-based, semantic and structural methods are the three different categories of measuring similarities.
In XMap++ approach, a generic workflow for a given ontology matching scenario is as follows: ′ 1. Matching inputs are two ontologies, source O and target O parsed by an Ontology
Parser component;
2. The String Matcher based on linguistic matching compares the textual descriptions
of the concepts associated with the nodes (labels, names) of each ontology;
3. The Linguistic matcher jointly aims at identifying words in the input strings,
relaying on WordNet [
Several technical adaptations were required for integrating the system into the Seals
platform, such as:
– Updating some libraries (e.g., Alignment API) or changing the way some
parameters are communicated.
– To deal with large ontologies, XMapGen and XMapSig conducted specific
experiments to see whether a matching system can exploit a multi-core architecture [
Evaluation results of XMapGen and XMapSig in the OAEI 2013 campaign are here evaluated and discussed. We participated in five tracks: Benchmarks, Conference, Library, Anatomy and Large Biomedical Ontologies. Detailed results and descriptions about the used computation environments are provided on the OAEI 2013 result page. 2.1
In this track, there are multiple match tasks per sub-track where one source ontology is compared with a number of systematically modified target ontologies. According to Table 1, it is shown that approximately 4% is the percentage improvement of XMapSig versus XMapGen. The recall low values are explained by the fact that ontological entities with scrambled labels, lexical similarity becomes ineffective. For two algorithms, structural similarity stems from lexical similarity hence scrambling the labels makes the alignment more challenging. This trend of reduction in precision, recall and f-measure which can be observed throughout the test cases from 248 till 266. The Anatomy track consists of finding an alignment between the Adult Mouse Anatomy (2744 classes) and a part of the NCI Thesaurus (3304 classes) describing the human anatomy. XMapSig achieves a good F-Measure value of ≈75% in an acceptable amount of time (393 sec.) (see Table 2). In a separate configuration using genetic algorithm (XMapGen) we could increase the recall to ≈2% but the precision is decreased of ≈5%, due to running the structural matcher in a lightweight version (restriction of the contextual similarity). XMapGen needs around 403 minutes to compute the alignment. We plan to use bio medical lexical databases like Unified Medical Language System (UMLS) for improving the recall. The Conference track uses a collection of 16 ontologies from the domain of academic conferences. Most ontologies were equipped with OWL DL axioms of various kinds; this opens a useful way to test our semantic matchers. The match quality was evaluated against an original (ra1) as well as entailed reference alignment (ra2). As the Table 3 shows, for both evaluations we achieved F-Measure values better than the Baseline1 results (57% for ra1 and 53% for ra2) when using XMapGen1 4. Also with XMapSig1 3 we achieved F-Measure values better than the Baseline1 results (58% for ra1 and 53% for ra2). The library track involves the matching of the STW thesaurus (6,575 classes) and the Soz thesaurus (8,376 classes). Both of these thesauri provide vocabulary for economic and social sciences. Table 4 summarizes the results obtained by XMapGen and XMapSig. The mapping quality achieved by XMapSig on the library track is not as positive as on the other tracks. XMapSig attains a precision of 0.79 and a recall of 0.31. Possible reasons may be the absence of domain and range definitions (in fact, of properties in general), as for anatomy, and the presence of multi-lingual labels. As XMapSig does not respect languages, this may lead to false positives. XMapSig requires ≈ 48 min and 34 sec. It is mainly due to the fact that our approach uses the notion of context with a value of radius not fixed as an input parameter at the starting of the matching task. So the algorithm looks at all the depth for the compared ontologies which involve the creation of a matrix with M > 1.3 billion pairs.
XMapGen could perform worse in terms of precision (0.031) and obtained higher for recall than XMapSig (0.37). The low of precision is due to a problem in the training of the genetic algorithm. We fixed this problem with an improved version delivered after deadline (precision and recall performance was different). In this paper we decided to present (see Table 4), only the results generated with the official version of our tool (before the deadline of the contest), and not the one generated with an improved version (fixing the training problem of GA) submitted after the deadline. This data set consists of several large scale ontologies, containing up to tens of thousands of concepts. Our two systems were only capable to match the small task for FMA-NCI and FMA-SNOMED. The large ones are not finished in time due to the high computational complexity. We found the NCI thesaurus very time consuming for context based mapping as its concepts have many siblings. Among the varying evaluation methods, XMapGen and XMapSig produced fairly consistent alignments when matching the FMA and NCI ontologies, all resulting in f-measures of approximately 0.60 (See Table 5). However, the results of the completed tasks indicate that our system is already capable of producing alignments of high quality in this domain, thus improving its efficiency, for instance by applying the complete functionalities of XMap++ , should result in an overall satisfying performance during the next evaluation. As not expected from our two systems, they could perform the alignment in less than 3 hours 25 min of Small FMA-SNOMED fragments with high precision and low recall (See Table 6).
As previously stated, the aim of this development experience was not to deliver a tool to compete with others in terms of precision and recall. Instead, we aimed at the development of a new and stable version of XMap++ using new and state-of-the-art technologies and alignment methods. Additionally, to tackle the large ontology matching problem we improved the runtime of the algorithm using a divide-and-conquer approach that can partition the execution of the matchers into small threads was improved and joins their results after each similarity calculation. A direct comparison between the XMapGen and XMapSig shows that the addition of GA does not has a negative effect on the algorithm but, on the contrary, leads to slightly better results, especially in terms of recall. In most track, XMapSig supplies high precision than XMapGen. Whereas using Genetic Algorithm (XMapGen) performs quite high in terms of recall than using sigmoid function (XMapSig). The reason is behind the using of the sigmoid function and the weight for linguistic matcher. Therefore, for a high value of the linguistic weight, some important properties of classes founded by XMapSig may be omitted, as the weight of linguistic matcher is high and the algorithm focuses more on the linguistic level (names of classes) than the structural level (properties and their restrictions). This problem can be resolved by using a sigmoidal function, which increases proportionally the important similarity of the structural matcher, to be considered in the final calculation of two classes similarities. Finally we participated in the OAEI 2013 with two variants with the aim to analyze the strength and the weakness of each strategy (Sigmoid and Genetic) at the goal for combing them in one ontology alignment task. 3.2
Some probable approaches to improving our tools are listed as follows: 1. Adopt more flexible strategies in defining the way for automatic threshold rather than manually tuning. Developing dynamic strategies for setting the correct threshold value for each compared ontologies and not one for all; 3.3 As a first participation, we found the OAEI procedure very convenient and the organizers very supportive. The use of Seals allows objective assessments. The OAEI test cases are various and this leads to comparison on different levels of difficulty, which is very interesting. We found that SEALS platform is a very valuable tool to compare the performance of our system with the others. 4
Our system participated to the campaign with two versions (XMapGen and XMapSig) of our approach, corresponding to different strategies of weights aggregation. Generally, according to our results in OAEI 2013, our two systems delivered fair results comparatively to other participants. The preliminary results were quite good to encourage us to continue seeking better solutions. It seems that both systems XMapGen and XMapSig can efficiently match semantically rich ontologies containing tens (and even hundreds) of thousands of classes. We confirm that the addition of Genetic Algorithm (GA) keeps the performance and, furthermore, eliminates the necessity of tuning the weights manually. Moreover, the learning framework is very flexible: many combinations of matchers and parameters may be used in the future, various types of training models (Resilient propagation, Levenberg marquardt, Backpropagation, Anneal, Radial or Manhatan method, etc.) and new metrics.