Most of solutions provided by current ontology matching tools lack exibility and extensibility namely for adding new matchers and dealing with users' requirements. In this paper, we present a system YAM++, which supports self-conguration, exibility and extensibility in combining individual matchers. Moreover, it is more human-centered approach since it allows users to express their preference between precision and recall. A set of experiments over OAEI benchmark dataset demonstrate its eectiveness and eciency in terms of quality of matching and exibility of the system.
Ontology matching is needed in many application domains. For example, the possibility of content-based query of the semantic Web depends only on the capacity of the system to nd correspondences (mappings) between ontologies of the related information sources. Many diverse solutions of matching have been proposed so far; however, there is no integrated solution that is a clear success, which is robust enough to and exible be the basis for future development, and which is usable by non expert users.
In this paper, we present our system YAM++, which supports self-conguration, exibility and extensibility in combining individual matchers. To demonstrate the important of the exibility in terms of system extensibility and user preference, let’s us introduce two scenarios that frequently arise when people study schema and ontology matching.
In the rst scenario, reseachers and developers of a matching system usually
have to supplement new invented similarity metrics or update existing metrics
with the new ones. According to [
In the second scenario, imagine that users run a matching system to nd all mapping pairs of entities between two ontologies. A matching system, generally, outputs a list of candidate mappings and corresponding condent values. Users then must verify these mappings in order to remove incorrect ones. This process will not take much time because number of the suspect mappings is limited. Next, users need to nd missing mappings which matching system did not discover. This process is very time consuming because it will be done manually on a huge number of candidate mappings. The manual eort of this phase is called postmatch eort. Users may spend many hours or even few days to nish this work. Therefore, users desire to have a way to improve number of correct mappings in order to reduce post-match eort.
Based on these scenarios, the motivation of our system can be described as follows: Giving two ontologies represented in some ontology languages (N3, RDF, OWL, etc.), nd a exible approach to combine individual matchers with the following features: (i) achieving high matching quality result ( precision, recall and f-measure ), (ii) system’s self-conguration, (iii) system’s extensibility, (iv) generating a dedicated matcher according to the user’s preference between precision and recall.
The remainder of this paper is organized as follows: In section 2, we describe our ontology matching system in detail. In Section 3, we present the results of experiments performed to highlight the main interesting features of our ontology matching tool. Section 4 contains the related work. Finally, Section 5 contains concluding remark about our system. 2
Our approach has been implemented in YAM++ - (not) Yet Another Matcher
system for ontology matching. It follows the same approach used in YAM schema
matching system [
New similarity metrics working with dierent features (e.g. name, label, comments, relations) of ontologies’ entities.
New dictionary metrics based on dierent algorithms.
New metrics based on information retrieval technique calculate similarity score between context and descriptive information of entities.
Graphical user interface for setting parameters, displaying and verifying discovered mappings returned from system.
The main components of YAM++ system are depicted in Figure 1. It only requires as input, the set of ontologies to be matched. However, the user can also provide additional inputs, i.e., some preferences between precision and recall.
The Knowledge Base is a system repository, containing library of similarity metrics and library of learning models. It also stores list of gold standard datasets, which is a pair of ontologies with expert mappings between some of their entities built by domain experts.
The Data Generation module transforms gold standard datasets and input ontologies to learning (including training and testing) and unclassied data respectively. The idea is that each pair of entities ( ei; ej ) becomes a machine learning instance, which its features are the similarity score calculated by similarity metrics on (ei; ej ). In training and testing data, the class of instances is determined by the condence value of the corresponding pair ( ei; ej ) in the expert mappings set. For unclassied data, the class of instances is set to unknown value.
The Training module nds the optimal conguration for each learning model according to training data passed from Data Generation module. Besides, it can also take a user preference for either Precision or Recall in training process to generate classication models that favor this preference. The conguration process is automatic and transparent to users. The average performances (Precision, Recall, F-Measure) of all learning models achieved from running 10-fold cross validation and dierent testing data are temporarily saved for comparison purpose.
The Selection module by default will select a classication model, whose the obtained average F-Measure is highest. If user provides a preference between Precision and Recall, a classication model, which obtains the best result corresponding to this preference, is selected for next stage. In this paper, we call it dedicated matcher or dedicated model.
In the Classication module, a dedicated matcher predicts each instance in unclassied data by a predicted value. If the classication model is nominal, the predicted value is TRUE or FALSE, which means two entities corresponding with classied instance are matched or not.
Finally, these mappings are displayed in graphical user interface. Users can judge a mapping whether it is correct or not by their knowledge of ontologies’ domain. Users also can modify, remove incorrect mappings or add new mappings with the help of command operations appeared in system’s menu (see Figure 2). 1. We show the exibility and extensibility of our system in term of integrating new similarity metrics automatically and transparently to users. 2. We show another ability of generating a dedicated matcher based on the user’s preference (promoting recall).
In the rst scenario, we use a set of string metrics as individual matchers to calculate similarity value between entities based on entities’ names and labels. These metrics are taken from open source code library SecondString 1, SimMetric2. These metrics are Levenstein, SmithWaterman, JaroWikler, Stoilos, QgramDistance, MongeEklan and Level2 metrics.
In the second scenario, we add metrics working with dictionary WordNet 3
to exploit semantic features and metrics working with entities’ description.
We have implemented Lin and WuPalmer[
In both scenarios we train dierent learning models such as: tree-based (J48, CART, ADTree, NBTree), probability-based (NaiveBayes, BayesNet), function-based (SMO, LibSVM, Logistic, MultiLayerPerceptron), instancebased (IBk, NNge, VFI). These models are taken from open source Weka 4 library. The gold standard datasets are taken from OAEI 5 and I3CON6 repositories. In both scenarios, we do not set preference between Precision and Recall, so the criterion for selection is maximum F-Measure. The winner model after running selection process in both cases is DecisionTree J48 model. It means that the dedicated models used in Classication module are a trained J48 in both the scenarios.
For comparison purpose, we run matching on set of datasets of OAEI 2009: {#104, #203, #204, #205, #206, #201, #201-2, #201-4, #201-6, #201-8}.
On datasets #104, #203 and #204 , both scenarios have the same FMeasure ( 1.0). This is because the linguistic information of these test ontologies is highly similar to with that of the reference ontology. In fact, entities’ names of these ontologies are identical or are modied by some naming conventions. Therefore, adding the new metrics do not have a signicant impact on the result.
On datasets #205 and #206 entities’ names in test ontologies are replaced by synonym words or are translated in another language. Thank to using metrics based on dictionary and descriptive information, the achieved average F-Measure is increased 57% from the rst ( 0.36) to the second scenario (0.93).
On datasets #201, #201-2, #201-4, #201-6 and #201-8 , entities’ names in test ontologies are replaced by random sequence symbols with 100%, 20%, 40%, 60% and 80% respectively. However, entities are also described by labels and comments, so the achieved average F-Measure is increased 45% from the rst ( 0.52) to the second scenario ( 0.97).
4 http://www.cs.waikato.ac.nz/ml/weka/ 5 http://oaei.ontologymatching.org/2009/ 6 http://www.atl.lmco.com/projects/ontology/i3con.html The most interesting feature to note is that the process of reconguration system with new metrics is totally automatic and transparent to the users. 3.2
Traditionally, the measure used to compute performance quality of matching
tools, is the F-Measure: a combination of Precision (the ratio of correctly found
correspondences (a.k.a true positive) over the total number of returned
correspondences [
Technically, most classication models suer from two errors during
classifying: i) discovering an irrelevant correspondence (a.k.a. false positive) and ii)
missing a relevant correspondence (a.k.a. false negative). The rst error decreases
precision while the second one decreases recall. In order to get better result in
term of recall, we need to set the cost on false negative error higher than that
on false positive error. This is a well-known issue called Cost-Sensitive Learning
in Data Mining [
In order to deal with cost-sensitive learning, we use MetaCost and
CostSensitiveClassication algorithms [
We perform our experiments with dierent proportion values, on the real datasets in OAEI 2009: { #301, #302, #303, #304 }. The base learning models, the list of similarity metrics and training data are the same as described in the second scenario in the rst experiment.
proportion = 1 proportion = 5 proportion = 10 proportion 15 Total True Positive Total False Positive Total Undiscovered
By increasing the proportion value, the total of candidate mappings discovered as True Positive is increased. This advantage helps users to reduce time for discovering missing mappings.
When the proportion is equal to 10, the total number of true positive mappings is maximum. After that, only the total number of false positive increases. This is a disadvantage, because users must to remove more irrelevant mappings.
Notice that whatever the value we set for proportion, it always remains some matches we are not able to discover automatically.
As an example, when proportion is set to 10, the dedicated matcher discovers 11 (133 - 122) additional true positives, but 43 (83 - 40) additional false positives in comparison with the default matcher. In fact, the eort for manually removing an incorrect mapping is much less than the one for discovering a new correct mapping among 9949 pairs (total candidate mappings of 4 datasets). Therefore, by promoting recall, our system reduces user’s post-match eort during the validation phase. 4
There are many studies on Ontology Matching [
GLUE [
Another systems using machine learning approach for ontology mapping such
as APFEL [
In this paper, we present a exible system for ontology matching task that proves the following interesting features:
Flexibility and extensibility in terms of combining individual matchers.
Generating a dedicated matcher according to the user’s preference. We have developed a prototype which has been tested with the datasets of OAEI 2009 benchmark. Through these experiments, we have validated the features listed above.