This paper presents the matching system TOM together with its results in the Ontology Alignment Evaluation Initiative 2021 (OAEI 2021). This is the rst participation of TOM in the OAEI. Very recently, transformers achieved remarkable results in the natural language processing community on a variety of tasks. The TOM matching system exploits a zero-shot transformer-based language model to calculate con dences for each instance. The matcher uses the pre-trained transformer model paraphrase-TinyBERT-L6-v2.3
1.1
Transformers for Ontology Matching (TOM) is a matching system that uses a
transformer-based language model to calculate a con dence for a pair of entities.
The matcher is implemented as a pipeline of subsequent steps using pre-de ned
matching modules of the Matching EvaLuation Toolkit (MELT) [
neural networks that have a speci c architecture [
Commons License Attribution 4.0 International (CC BY 4.0). 4 https://dwslab.github.io/melt/matcher-packaging/web#
web-interface-http-matching-interface
of tasks [
To perform well, transformers need to be trained on large amounts of data. Many researchers and organizations train their transformers on textual data from various domains. After training the model (so called pre-training ), the transformer models can be ne-tuned for speci c tasks or domains. Typically, the ne-tuning process is computationally cheap compared to training the main model. However, training data is required.
The TOM research project explored the capabilities of such pre-trained
models for ontology matching. TOM does not use ne-tuned models, but there is a
ne-tuned version of TOM, which is named Fine-TOM (F-TOM) [
There are several python libraries, like the transformers library by
huggingface5 [
TOM Pipeline TOM consists of ve components that are arranged in a pipeline which is shown in Figure 1. The arrows indicate how the ontologies and alignments are passed between the components. Each component can be conceptually regarded as a matcher. This design pattern is proposed by MELT. For chaining the components, class MatcherYAAAJena was used.
First, the ontologies are aligned in with string matching methods. Since these are typically of high precision, the resulting alignment of this step is directly added to the nal alignment.
The transformer component in MELT is, by default, implemented as a lter.
Therefore, candidates have to be generated which are then ltered by the
transformer. In this case, a string overlap metric is used. The transformer component
adds a con dence to each candidate. After the transformer matcher calculated
the con dence for each candidate pair, the alignment is ltered by a threshold.
Based on an evaluation on di erent OAEI tracks, we decided to set the threshold
to 0.8. Since most OAEI datasets are typically of one-to-one parity, we use an
e cient implementation of the Hungarian method, known as Maximum Weight
Bipartite Matching (MWBM) [
String Matcher The string matcher is used to nd very obvious correspondences. To do that it compares the inputs word by word. It assigns a con dence of 1:0 to the obvious correspondences. Therefore, the resource-consuming transformer matcher does not have to evaluate those pairs of elements again. We 5 https://huggingface.co 6 https://www.sbert.net use the default string matcher implementation of MELT (class SimpleStringMatcher).
Candidate Generator The Candidate Generator is used to generate an input alignment for the transformer matcher. It creates a cross product of both ontologies, so that the alignment consists of all possible pairs of elements. For large ontologies, the alignment would be too large to be proceeded by the transformer matcher within an appropriate time frame. Therefore, the Candidate Generator excludes the correspondences that are found by the string matcher and obvious non-correspondences.
Those are found through broad string operations. The candidate generator class splits the labels and saves single words in a set. Then the sets are compared and if over 50 percent of the words are equal, the correspondence is further processed by the transformer matcher. If less than 50 percent of the words are equal, it is not necessary to calculate a similarity for this pair. Finally, the alignment is passed to the transformer matcher.
Transformer Matcher The transformer matcher iterates over the alignment
and passes each pair of elements to the sentence-bert library [
Threshold Filter The threshold lter uses the alignment and cuts o all correspondences with a low con dence. To do that, the lter uses a threshold between zero an one. All correspondences with a con dence lower than the threshold are excluded from the alignment. The submitted matcher has a threshold t = 0:8 which yielded good results for all evaluated tracks. We use the default string matcher implementation of MELT (class ThresholdFilter).
Max Weight Bipartite Extractor Up to this step in the pipeline, we could have multiple correspondences for an ontological element in the alignment. Therefore, the max weight bipartite extractor converts the current state to a one-to-one alignment. We use the default class MaxWeightBipartiteExtractor of MELT. 2
This section discusses the results of TOM for the tracks of OAEI 2021 on which
the matcher was able to produce results. These include the Anatomy [
TOM could achieve a higher F-measure than the OAEI Baseline (0.866 vs. 0.766). It was noticeable that the recall was improved by TOM (0.808 vs 0.622) but the precision is lower than with only the OAEI Baseline (0.997 vs 0.933). So there are non-obvious matches that cannot be found with string based matching but by TOM. Examples for these would be the matches Parietal Lobe of the Brain & parietal cortex with a con dence of 0:8202 and great vein of heart & Great Cardiac Vein with a con dence of 0:8794. Those are found because of the ability of transformers to detect semantic similarities between two phrases or words such as heart and cardiac, even though they are spelled di erently. 2.2
Also on the Conference track, TOM is able to nd more correspondences than the OAEI Baseline. So the recall is higher (0.48 vs 0.41) and also the F-measure is higher (0.57 vs 0.53). The precision is a bit lower (0.69 vs 0.76). 7 https://github.com/scikit-learn/scikit-learn/blob/844b4be24/sklearn/ metrics/pairwise.py#L1211l
We thank the OAEI organizers for their support and commitment. 4
In this paper, we presented the TOM matching system and its results in the OAEI 2021. We can conclude that transformer-based language models are able to improve performance in the task and process of ontology matching.
The developed matching system achieves an overall better F-measure than the baseline matchers and it improves the recall. It is important to note that the highest possible recall is set by the Candidate Generator's alignment. The Transformer Matcher works only with this alignment and so it is not possible to achieve a higher recall.
The research also showed that the presented architecture and the implementation in Java and Python are appropriate approaches to use transformers for ontology matching. Most of the used matching components are available via the MELT framework to allow other developers to re-use them. The docker packaging allows to submit any implementation without set-up e orts on the organizer side which is also bene cial for matcher developers who do not have to worry about the execution of their system.
This is the rst OAEI participation of TOM and the system can be greatly improved in the future, for example by using ne-tuned models or by improving the candidate generation pipeline. The reported results motivate further research in the area of transformer-based ontology matching.