Complex ontology matching is a process to match complex structures in ontologies. While many matching tools tackle simple ontology matching, complex ontology matching is still rare. However, one entity in one ontology can be similar to a complex structure (1-to-n) or even complex structures can be on both sides (m-to-n). Therefore, the application, e.g., data integration, must consider complex correspondences within ontology alignment. Our poster paper presents a pattern-based approach where particular SPARQL queries correspond to a specific pattern, e.g., Class by Attribute Type (CAT), for its detection. SPARQL queries are anchored to entities from simple correspondences on input. Detected complex correspondence candidates are verbalized to be validated by the Large Language Model (LLM). Further, we provide a zero-shot prompting preliminary experiment and evaluation. The poster paper is equipped with the Jupyter notebook for automation of the pipeline and the full report of the experiment at: https://github.com/OndrejZamazal/ComplexOntologyMatching-SEMANTiCS2024
CEUR ceur-ws.org
Sharing domain knowledge is often made via domain ontology [
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
tools and evaluation eforts (e.g., OAEI 2023 1 had ten tracks targetting simple matching), only
one OAEI 2023 track focused on complex matching; even without tool participation in 2023.
There have been two complex alignment benchmarks in OAEI so far: the GeoLink complex
alignment benchmark [
Several approaches have dealt theoretically with complex ontology matching, mostly
patternbased approaches [
LLMs have already been applied for simple ontology matching. Conference ontologies have
been matched using Chat-GPT [
We describe our pattern-based pipeline along with an example (in italics) targeting the alignment
pattern Class by Attribute Type (CAT) [
Often, ontology matching techniques limit the search space. We use simple alignment and limit complex matching to entities in complex structures related to entities from simple
alignment. We assume that potential matching complex structures involves entities from simple correspondences.
Input ontologies are conference ontologies: O1=cmt, O2=ekaw. For input alignment, we used the subset of the alignment from LogMap OAEI 2023 (O1:Paper=O2:Paper; O1:Person=O2:Person) to keep the experiment shorter. Any highly certain correspondences could be used.
Step 1. Detection is based on a structural aspect. We use a pair of SPARQL queries designed according to given alignment pattern for O1 and O2, resp.2 One alignment pattern can lead to diferent structural aspects for detection (i.e., diferent SPARQL queries) in O1 or O2, resp. Some entities in SPARQL queries are anchored based on input alignment.
Figure 1 consists of a visualization of SPARQL queries for O1 and O2 for CAT, and the correspondence from input alignment is depicted as a both-sided arrow. The detection is run for each correspondence from input alignment. In our case, 4 and 7 SPARQL results for O1 per input correspondence and 2 and 14 SPARQL results for O2 per input correspondence resp.; Example for O1 given Person (?ent1) from input correspondence: O1:Class1=Reviewer. Example for O2 given Person (?ent1) from input correspondence: O2:property1=authorOf, O2:class1=Document, O2:class2=Review.
Step 2. Results from detecting both ontologies are joined according to the alignment pattern separately per each input correspondence.
In our example, there are 4 ×2=8 complex correspondence candidates for the Paper entity from input correspondence and 14 ×7=98 complex correspondence candidates for the Person entity from input correspondence. E.g., regarding the first input correspondence, O1:Person=O2:Person, O1:Reviewer is joined with O2:Person, O2:authorOf, and O2:Review.
Step 3. Pattern-based template-driven verbalization to natural language (English) is applied to complex correspondence candidates to enable their validation using LLM. We also apply several natural language preprocessing steps, such as tokenization and lowercasing. Similarly, we use a template for serialization into Manchester OWL syntax.
For CAT, we have the following verbalization pattern into English: ”<O1:class1> is the same as <O2:ent1> which is3 <O2:property1> of <O2:class2>”. Example: Reviewer is the same as person which is author of review. 2The approach is pattern-based in two ways: because it is driven by the alignment pattern and because the SPARQL query can also be considered as a pattern for detection. 3If no ”has” exists in property1, ”is” is added.
Step 4. Finally, LLM is used to validate whether verbalized complex correspondence candidates are (probably) positives/negatives.
We experimented with diferent LLMs. So far, the best results have been achieved using GPT-4o. 4
The experiment is related to step 4. LLM is used to validate whether verbalized complex correspondence candidates are (probably) positives/negatives. Our online supplementary material contains the full report (SPARQL queries, their results and their joints, verbalized complex correspondence candidates, the prompt for GPT-4o,5 its results, evaluation, complex correspondences in EDOAL and Manchester OWL syntax). Of 106 candidates, 14 were labeled as negatives, 74 as probably negatives, ten as probably positives, and eight as positives. The supplementary material contains a detailed evaluation of all eight candidates labeled by LLM as positives. As a result of eight candidates’ evaluations by human, there were one true positive, one partly true positive, and six partly false positives. For the sake of brevity, here we present three evaluation examples: 1. Reviewer is the same as person which is author of review.
Although it is debatable whether being an author of a review is enough to be a real reviewer, it is certainly close enough. It was evaluated as a true positive.
2. Meta-reviewer is the same as person which is author of review.
Meta-reviewer is not only the author of the review but (s)he has a specific role within a reviewing process. A meta-reviewer is instead a subclass of person which is an author of review, i.e., Class ⊑ Class Expression. It was evaluated as a partly true positive example. 3. Author is the same as person which is author of abstract.
Since some conferences call for abstracts, being an author can be merely based on abstract authorship. However, the subsumption relation would be more fitting (i.e., Author subsumes person which is author of abstract). Since it leads to General Concept Inclusion (GCI) subsumption, i.e., Class ⊒ Class Expression, being not always allowed, it was evaluated as partly false positive.
Considering only equivalence, precision equals 18 = 0.125. However, subsumption is also important for interoperability, meaning relaxed precision ( ) could be used. If GCI axioms (partly false positives) are not allowed, = 28 = 0.25. If GCI axioms are allowed,6 = 1.0. Regarding recall, the preliminary evaluation (details in the online supplementary material) shows that all negatives are true. However, it needs further evaluation in terms of relaxed recall.
We reported on our approach and preliminary experiments with pattern-based complex ontology matching using SPARQL for detection and verbalization before validation with LLM.
Similarly to [
5We ran the GPT-4o several times with minor changes in its reply; it is not substantial for the evaluation.
6GCI subsumptions could also help with interoperability within some scenarios.
of the approach for validation. Contrary to [
For now, we consider a 1-to-n relationship. However, it can be generalized to an m-to-n relationship. In our experiments, we use GPT-4o. However, we will experiment more with other LLMs (e.g., Llama 3, Mixtral). Further, while we employ direct question prompting, we will also experiment with contextual question prompting, where context will be gathered automatically. We aim to explore more alignment patterns, involve more ontologies, and conduct in-depth evaluations in future work. While the provided Jupyter notebook covers three steps, the fourth step, dealing with LLM, will be implemented after further experimentation with other LLMs.
This work has been supported by the EU’s Horizon Europe grant no. 101058682 (Onto-DESIDE).