This paper presents the results from the PropMatch system in the OAEI 2023 campaign. PropMatch is a system dedicate to the generation of alignments between ontology properties. It combines word and sentence embeddings with alignment extension. The system has participated in the Conference track. This is the first participation of PropMatch in the OAEI campaigns.
CEUR
ceur-ws.org CEUR Workshop Proceedings
This matcher extends the work of [
PropMatch1 is based on four main techniques, introduced below. The overview of the system architecture is presented in Figure 1.
The TF-IDF and Soft TF-IDF models are created before the matching process. Each entity from the ontologies is represented by a virtual document that is a piece of text containing entity information. Ontology class virtual documents use the class labels and comments, while properties virtual documents include the property labels, domain, and range. These documents are then tokenized and converted to lowercase. After that, the frequency models, vocabulary, and IDF are generated from this set of documents using Scikit-learn 2. Cosine similarity is then used to compute the final similarity between the embeddings of the property pairs. For property labels, the system employs the Soft TF-IDF approach, using the Jaro-Winkler metric with a threshold of 0.8.
After constructing the TF-IDF models, the system calculates the similarity score for each property pair. The final score is the minimum of three confidence values based on domain, range, and property label similarities. This ensures similarity only when all three exceed the threshold. If the metric yields zero similarity, an embedding similarity is used as an alternative measure.
When TF-IDF models generate pairs of properties with low similarity, the system instead uses
word embeddings for domain similarity and sentence embeddings for property label similarity.
1https://gitlab.irit.fr/melodi/ontology-matching/propmatch
2https://scikit-learn.org/
For domain similarity, we employ pre-trained word embeddings from the Finnish Internet
Parsebank [
In property label similarity, we first remove the last word if it matches the first word in the range label.
Similar to the domain similarity approach, we apply a fallback strategy for property labels. This happens when domain and range similarity exceeds 0.9, but label similarity falls below 0.1. These parameters are found to perform better in the Conference track and are hyperparameters of the system that can be adjusted. We use a sentence embedding similarity model from the HuggingFace 3 repository to generate embeddings for property labels. The property label with the range labels is composed in a sentence that is fed to the model to generate an embedding used for the similarity calculation.
A common practice in matching is to use existing alignments to find new correspondences,
following the ”locality principle” [
Following the locality principle, since the system can use information from previous alignments to find new correspondences, we apply a process of similarity reinforcement. Assuming that the system can find a subset of the final alignment set, repeating the matching process with the previous alignment found by the system can increase its confidence in new similarity measures given the previously discovered correspondences.
In order to keep track of the previous correspondences, a key-value store is used named Confidence-Map, where keys represent pairs of classes, and values represent the similarity between them. When a property correspondence is found, the domains in each property form a pair in the Confidence Map. In the next iterations, the system checks if the domain pair of the evaluated properties is present in the Confidence Map. If found, the domain confidence in the similarity computation is increased by 0.66. Multiple iterations are needed to fully reflect the Confidence Map’s influence, and the number of iterations is a system hyperparameter.
The similarity reinforcement works as the following. Consider the triples: • Ontology 1: (Paper, hasTitle, Title), (Author, writes, Paper) • Ontology 2: (Contribution, hasTitle, Title), (Author, contributes, Contribution) 3https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2 consulted at 07/09/2023.
In the first iteration, if the similarity measure between 2 properties is higher than the threshold , their domains are added to the map. (Paper, Contribution):0.66 is added to the map because the similarity between 1 ∶ ℎ and 2 ∶ ℎ is higher than . In a second iteration, the map entries are taken into account for calculating the property similarity. In the example, the similarity between 1 ∶ and 2 ∶ will have a higher range similarity since the pair (Paper, Contribution) is present in the map.
In the Conference track some property domains are composed of complex constructors. in order to match properties with those domains and to compare their similarity the first step of the matching process consists of converting the complex entities into simple entities that have all labels of the complex entity concatenated. For example, complex constructors such as owl:UnionOf can be handled. For example, the property hasTitle which has a complex domain containing two entities Conference and Paper. After the processing, the labels of the Conference and Paper are concatenated to generate a single label Conference_Paper that the matcher can use to measure the similarity between domains.
PropMatch was evaluated on Conference in the modality M2 which is based only on the alignment of properties. In these results, ALIN, AMD, LSMatch, and SORBETMatch do not produce any property alignments. While using language models, PropMatch is not fine-tuned and does not require any reference alignments as input. ra1 is the original reference alignment. ra2 is an entailed reference alignment created through transitive closure from the original reference alignment (ra1). And rar2 is a violation-free version of reference alignment ra2 as described in the result page of the OAEI 4.
Table 1 presents the results of the systems that participated in the OAEI 2023 in the ra1-M2 (property matching only) modality. PropMatch achieved the best results in all metrics in this modality.
Tables 2 and 3 present the results of the systems in the modalities ra2-m2 and rar2-m2 respectively. In both ra2-m2 and rar2-m2 modalities PropMatch still stays at first winning in all metrics while improving its precision by 0.03 and recall by 0.02 in the ra2-m2 modality compared to the ra1 modality, and also improving its precision by 0.03 and recall by 0.04 in the rar2-m2 modality compared to the ra1 modality.
PropMatch achieves the best results in the M2 modality in all metrics. This corroborates the interest in combining embeddings with classical matching metrics for property matching. It 4https://oaei.ontologymatching.org/2023/results/conference/eval.html F.5-measure 0.77 0.48 0.55 0.50 0.35 0.24 0.18 0.15 0.05 F.5-measure 0.78 0.57 0.56 0.51 0.35 0.24 0.16 0.18 0.05 F1-measure 0.66 0.47 0.40 0.39 0.30 0.23 0.14 0.16 0.03 F1-measure 0.68 0.47 0.41 0.40 0.30 0.23 0.16 0.14 0.03 F2-measure 0.58 0.46 0.32 0.31 0.26 0.22 0.12 0.17 0.02 F2-measure 0.60 0.40 0.33 0.32 0.26 0.22 0.17 0.12 0.02 Recall 0.54 0.46 0.28 0.28 0.24 0.22 0.11 0.17 0.02 Recall 0.56 0.36 0.29 0.29 0.24 0.22 0.18 0.11 0.02 gives the system an increased recall while preserving higher precision.
The embeddings in the system are still not the main source of similarity computation. In order to improve the system performance, moving to a full embedding approach could improve the capacity of finding more correspondences. Along with that, better models that aggregate more context information into embeddings can also improve the system’s performance. Another improvement is extending the approach to class matching since the confidence map built for the property alignment could contribute to finding class correspondences. Lastly, the discovery of complex correspondences between properties could be addressed.
Furthermore, since the representation of properties can change between tracks, the system still has dificulties to be evaluated in all tracks. Improving the system’s capacity to find the properties in the ontology structure and also dealing with complex domain and ranges (composed of multiple entities) can help the system’s generalization and ability to run in other tracks.
This paper presented the PropMatch system and its results in the OAEI 2023 campaign. This year, we have participated only in the Conference track. Next year, we plan to participate in more tracks.