This paper presents the results obtained by the TOMATO system in the OAEI 2022 evaluation campaign. We describe here the results in the Conference track. We report a general discussion on the results and on the future improvements of the system. TOMATO (TOolkit for MATching Ontologies) is heavily inspired by POMAP++[1]. It is a pairwise ontology matching system; ie. matching couples of ontologies together. We employ mostly Machine Learning approaches based on element similarities[2] (using mainly string-based similarities). The workflow of TOMATO is depicted in Figure 1. The workflow being built upon several steps starting with loading the OWL ontologies and the reference RDF aligned files. Tomato can be used in two different ways : 1. in a learning mode, where it can generate a model given two ontologies and their reference matching : (1, 2, (1, 2)) = (1, 2) 2. in a matching mode, where a previously generated model is used against two ontologies to produce an alignment : ℎ(1, 2, (1, 2)) = (1, 2) In both cases, we can choose to apply a global matching strategy, using all the elements from both ontologies given their types and producing a single model ; or a set of local matching strategies, depending on the data, in where we treat subsets of the elements from both ontologies (given a specific characteristic of the data) to produce one model per subset.
For its base operation, TOMATO parses and loads a given set of two input ontologies. We rely mostly on the owlready2 Python library to populate our internal structure, which contains all the elements indexed by their respective names (classes, data properties and object properties), and their relations.
The name of an element is either using its label (if found) or, for a fallback approach, the last component of its URI.
For the relations, we chose to store all the relationships between classes.
Since string similarity measures are case-sensitive, symbol-sensitive and very dependent on what words are being parsed, we first need to apply multiple filters in order to have data that can be properly analysed.
Here are the steps we chose to keep : • CamelCase words are converted to snake_case, which allows us to separate words ; • snake_case and non-alphanumeric symbols are replaced by spaces ; • we perform an English stemming process as the one proposed by Porter[? ] ; • we then remove all stop words (such as "a", "who", "me", etc.) using WordNet approach to have only the concept words ; • we stem the words for the similarity measures that support them.
Once the preprocessing steps are complete, the input ontologies are quite ready to be processed by the level matcher.
The entities are then paired : we compute all possible combination of class x class, data property x data property, and object property x object property. For each combination, we compute the similarity of the couple through an all-against-all strategy, which stands for measuring the similarity score (using a combination of Levenshtein, JaroWinkler, ISUB, and 3-Gram similarity measures). This step leaves us with a vector of scores for each possible couple, coordinates being the different similarity measures used.
To ensure a better result, we apply a local strategy based on what kind of entity we are matching. We defined three possible situations : • both entities contains only a single word ("mono"-term) ; • both entities contains more than one word ("multi"-term) ; • entities contains a different term count ("dual"-term).
This strategy is somewhat naive, but it would serve as a starting point for our next exploration.
Training a model In order to find the best measures that can indicate a potential match, we use a classifier from the scikit-learn Python module (GridSearchCV). The train/test split ratio is 0.4 : we use 60% of our data to train the model, and 40% to test, doing ten splits of cross-validation for each of the three models.
We feed the similarity matrix with the reference data (normalized as "0" if no match and "1" if there is a match). We pass the similarity matrix to the SVM machine-learning algorithm, and we save the model trained. Given our localized strategy, we end up with three different models, one for each situation (mono, multi, dual).
Computing a match In order to compute a match, we build a similarity matrix for all the couples we are interested in. The matrix is then fed through our classifier using the previously trained model. The classifier then outputs the expected alignment.
For our participation in OAEI 2022, we decided to test against both strategies (local and global) so we could tune our models. As such, we participated using only our local strategies. We included our comparison in the next section.
TOMATO runs on a Ubuntu server enviroment, with 16 GiB of RAM and a Intel Xeon Gold 6244 CPU at 3.6GHz. We can submit jobs to it through the web interface at
http://tomato.irit.fr/match 2.2. Comparison global and local strategies The global strategy consists in applying a single model for all possible couples of conferences. This model is therefore built by finding a global optimum between the similarity measures corresponding to the features, and the model weights
The Local strategy consists in applying a model depending on the entity word count (mono, multi or dual) as per section 1.2.4.
Ontology 2 cmt conference cmt confof cmt edas cmt ekaw cmt iasted cmt sigkdd conference confof conference edas conference ekaw conference iasted conference sigkdd confof edas confof ekaw confof iasted confof sigkdd edas ekaw edas iasted edas sigkdd ekaw iasted ekaw sigkdd iasted sigkdd all ontologies
Applying local models helps to smooth the averaging effects of the averages made by using a single model for all the ontologies. While our strategy is very simple, it helps showing us that selecting a different model based on some feature from the data gives us better results as shown in table 1.
2.3.1. Ranking
LogMap GraphMatcher SEBMatcher ATMatcher ALIN edna LogMapLt AMD LSMatch StringEquiv KGMatcher+ ALIOn TOMATO Matcha
While our system has the highest recall (0.6) of the list, we came second to last in terms of F-Measure as table 2 shows. This is due to our low precision score (9%), and shows the limits of using only a simple string-matching strategy.
Sharp
Discrete Continuous
0.09 0.08 0.08
0.16 0.15 0.15
0.63 0.74 0.73
The OAEI 2022 results for conference cover three situations, as described in the track details : – discrete, which considers that there is a correct match if the reference alignment has a confidence value of 0.5 or greater ; and a fully incorrect match if the confidence value is less than 0.5; – continuous, which penalizes TOMATO more if it misses a match on which most people agree than if it misses a more controversial match : if the reference alignement gives that 1 and 2 match with a 0.85 confidence, and if a matching algorithm gives that match a confidence of 0.40, then that is counted as 0.85 * 0.40 = 0.34 of a true positive and 0.85 – 0.40 = 0.45 of a false negative.
In all three cases, while we have a good recall, our very low precision score is detrimental to our F-measure, and highlights once more the limits of our simplistic approach.
This paper presented the TOMATO matcher, a matcher that produces pairwise ontology alignements using a Machine Learning approach mixed with a strategy selection depending on the data used.
While our strategy is very simple, it is sufficient to show that our hypothesis is correct : using a local strategy depending on the data is better than a global single strategy.
In future versions, we intend to study other Machine Learning algorithms and different similarity measures to improve the quality of the generated alignments. We will also study the structure of the ontologies in depth to increase our performances, and include other measures to strengthen our results.