Ontology matching targets identical concepts from diferent ontologies with the final purpose of interoperability and ontologies merging. The matching task is not restricted to ontologies, it is also relevant for knowledge graphs. Ontology matching solutions based on transformer-based embeddings, textual similarity, logical mapping, or Large Language Models (LLMs) are still facing problems, mainly due to the lack of uniform information about the concepts and lack of homogeneous semantic granularity along diferent ontologies. In this work, we present a framework that combines vector-based similarity and string-based similarity through hybrid searches. LLMs are used to generate descriptions for ontology concepts, hence the concepts' representation is enriched and the alignment process can benefit from both the knowledge captured by the initial ontologies and the extended LLM-generated textual descriptions. The proposed system, HybridOM, is an unsupervised approach independent of the ontologies' domain. HybridOM is evaluated within Bio-ML 2024 track for the task of concept matching. It achieves the highest values for F1-score and Recall for most of the ontology pairs while maintaining a balance between precision and recall. The proposed method has been adapted for industrial usage in a human capital management product called msg.ProfileMap.
Despite the long-standing goal of ontologies to provide a structured representation of domain knowledge, matching two ontologies from the same domain is not a straightforward task. The primary challenges in ontology matching arise from dificulties in creating comprehensive, exhaustive, and standardized representations of meaning.
Since ontologies are diverse, either in the naming schema, the granularity of properties and taxonomical structure, or the level of detailed textual descriptions, we combine diferent sources for meaning and diferent meaning representations can help the ontology matching. The considered sources for the meaning are: i) asserted axiom-based descriptions, ii) knowledge derived through symbolic reasoning, iii) textual descriptions from the ontology, and iv) textual descriptions generated with LLMs. The considered meaning representations are: i) axioms in the ontology, ii) plain text used in labels or concept descriptions, iii) text embeddings. The result is a solution for ontology mapping that is domain-agnostic.
In this work, we give a detailed description of our method together with the evaluation results obtained in 2024 BioML task, part of Ontology Alignment Evaluation Initiative. Elements of the current method were also tested in the context of msg.ProfileMap. msg.ProfileMap is a human capital management product that uses an ontology containing competences, job roles, and certificates. New entities can be entered into the ontology automatically by importing existing vocabularies of customers or by extracting them from CVs or job descriptions. Duplicates in the ontology would create problems when searching for employees or during statistical analyses. HybridOM’s method supports these processes by identifying the corresponding ontology entities if existing and thus ensuring the quality of the ontology.
Most of the ontology matching systems, such as OLaLa [
Hybrid search is a special case of search that combines multiple semantic searches and string-based searches and matches the results into a single ranking output. HybridOM uses hybrid search to extract the best candidates for matching an external concept to concepts in the ontology. Using OpenSearch as a vector database, the data about concepts are stored as string-based and vector embedding fields.
HybridOM architecture consists of a pipeline (Figure 1). The first step is data preprocessing which i)
identifies the essential raw data that can be used from ontology and ii) generates descriptions using
LLM for each concept based on its medical domain. The ontology axioms are integrated through their
verbalization based on the algorithm described in [
For preprocessing of ontology data, python package DeepOnto [
In case EquivalentClasses axiom is missing, the verbalization is represented just by the subclass axiom or parent concepts extracted using reasoning. In the case of no SubClassOf axiom, the verbalization is formed by parents found using the Elk reasoner. In case the reasoner does not find any parent, a hardcoded generic parent is used. The verbalizer and the reasoner are provided by DeepOnto package. ⟨⟩ It is a type of ⟨ _1⟩, ⟨ _2⟩ . . .
It is an equivalent of ⟨ _1⟩, ⟨ _2⟩ . . . ⟨⟩ It is a type of ⟨ 1⟩, ⟨ 2⟩, . . .
⟨⟩ It is a type of General Bio Medical Entity. 1.2.1.3. Generating description using LLaMa In most cases, the ontologies do not include a meaningful textual description for each concept. String-based matching can’t provide enough context to distinguish between similar concepts but with diferent names, such as Alcohol and Ethanol. We assume that the existence of a description could help in reducing ambiguity. Inspired by the way LLMs are used in RAG approaches, HybridOM proposes the usage of LLM-generated descriptions as a means to improve the ontology matching by providing useful information that might even extend the ontology’s properties. The possible negative impact of using LLM-generated description is limited since hybrid searches combines the generated meaning with the one existing in the ontologies. In case the concept is sensitive (e.g. substance drug related), LLaMa doesn’t generate a valid description. HybridOM detects this based on the text length and replaces the result with label.
LLaMa3 is an open-source large language model with impressive results that exceed GPT-3 in various
benchmarks [
Because the medical domain of all ontologies is known, HybridOM uses specific prompts that generate
simple definitions, chemical structures, symptoms, anatomy, etc. The medical domains were extracted
from [
The next step in the pipeline is the embedding generation for textual data extracted or generated from
ontology. The fields to be embedded are: verbalization, description, and LLM generated description. While
HybridOM can use any language model for the embeddings, gtr-t5-large2 and BioLord-2023 [
Hybrid Search is the integration of semantic searches using Approximate k-NN search and keyword searches using BM25 algorithm The search uses multiple keyword/vector similarity searches that run in parallel. The concepts from the source ontology () are searched in all the concepts from the destination ontology () based on the existing fields in the database. In the case of NCIT-DOID dataset, the is NCIT and the is DOID. All concepts are stored in OpenSearch indexes, one index per ontology. Based on a search configuration, for each search in hybrid search a score is set and the sum of all scores must be equal to 1.
Figure 2 shows the way we use the hybrid search. For keyword/semantic search, one field from is searched in multiple fields from . Based on the concept’s id and the field names used in the keyword and semantic search, the concept is identified in the database and its fields’ values are retrieved. These values are searched in several fields from .
In the given example (Figure 2) the keyword field is label and the semantic search field is description_llm_emb (embedding of LLM generated description). Two values are retrieved based on the concept’s id by the Data Module: the string Ethanol and the stored embedding of the LLM description. Based on the given search configuration, the string ℎ is searched in the fields label and synonyms, and two k-NN searches are running in parallel for pairs: description_llm_emb (SO) - description_emb (DO) and description_llm_emb (SO) - verbalization_emb (DO). In the end the final score is a weighted score based on the configured weights for each search.
The final ranking is calculated by running multiple hybrid searches. The score for ontology matching is the sum of all scores returned by hybrid searches divided by the number of searches. 1.2.3.1. Keyword search Lucene Engine from OpenSearch is used for the keyword search. HybridOM search configuration defines one field from (keyword_field ) whose value is to be searched in one or more fields ( keyword_fields_search ) (Figure 3). Only one search is executed and if the searched value is found in multiple fields the score increases. In the search configuration, it can also be configured the importance of finding a searched value in a certain keyword field (Figure 3).
For the keyword search, OpenSearch returns a score that is not limit-bounded. Consequently,
HybridOM uses a custom [
HybridOM is evaluated on all datasets from Bio-ML track for unsupervised tasks (Table 4). For a fair
evaluation, each dataset was evaluated using the same configuration. The system can be adapted
by changing its parameters per track to obtain better results. The best HybridOM configuration
uses BioLord-2023 [
According to the values for Hits@K metric from Table 4, the HybridOM system seems to be struggling to discover the true matching concept between the first two candidates in ranking. For the Hits@K and MRR metrics a system should provide for each list of possible candidates, a ranking or a score for each candidate. The HybridOM provides a matching ranking but with a variable length of concepts based on the search result. Some candidates used for evaluation of Hits@K and MRR metrics are not present in the HybridOM final ranking, as a result, the score for them will be 0.0. If the ground truth candidate is not found in the final HybridOM ranking (does not appear on any rank), the concept will have a value of -1.0 such that we are still able to build the evaluation file. This value is assigned with awareness of the ground truth, but with the scope of not increasing the metrics score; the candidate will have the lowest score among the possible candidates from the file. HybridOM returns a ranking of top candidates for ontology matching, and concepts that are not present in the ranking will be considered automated with a score 0.0 for matching.
The benefits of using BioLord-2023 as the main embedding model, compared to a general domain language model, are outlined in Table 6. The only exception is the dataset SNOMED-NCIT Pharm, where the performance is higher using the GTR model.
Normalization has an important role in hybrid search performance in comparison with a simple approach (Table 7). It increases the score distance between candidates in the local search ranking. The combined local rankings facilitate the creation of a more realistic final matching ranking of candidates, achieving better results for all datasets.
Nevertheless, the backward hybrid search for the retrieved candidate provides a double checking for the initial matching and it increases the performance (Table 8). For some cases, the one-direction hybrid search exceeds the dual hybrid search, but the diference is insignificant.
The results obtained of HybridOM in unsupervised Bio-ML track 2024 stand out especially for Recall and F-score metrics. Taking three 1st places and two 2nd places for F-score metric and three 1st places, one 2nd place and one 3rd place for Recall metric according to unsupervised track on all datasets.
In comparison with other models, HybridOM has a weaker precision score among all datasets. For the
SNOMED-FMA, BERTMap [
Using simple semantic and keyword searches, HybridOM achieves consistency among the scores for all metrics (P, R, F-score). The proposed method has a simple yet efective approach of integrating multiple searches such that the limitation of one search is alleviated by the strengths of the others. Integration of LLM for generating description, the usage of the verbalizations from ontology’s axioms, and raw data from ontology are the core data used by HybridOM.
HybridOM can be further developed by adopting new methods for extracting or generating data for ontology concepts. Diferent prompts can be applied along with other LLMs to provide more domainspecific information. The usage of diferent embedding models can increase the retrieval performance. New ways of combining diferent kinds of searches besides keyword or semantic-based should ofer a large perspective for ontology matching. Enriching the verbalization semantics using other axioms and relations might also be beneficial. An automated way of calculating the search weights based on ontology data, searching fields, or statistics, might be a key improvement for developing a highly adaptable and reusable ontology matching system agnostic to ontology data variance and domain.
HybridOM proposes a domain-agnostic solution for ontology matching. It tackles the diversity of ontologies with a diversity of meaning sources, meaning representation, and a combination of hybrid searches. Explicit structured knowledge captured in the ontology is combined with text-based description generated by LLM. The role of LLM is not decisive since no decision is taken based only on LLM, yet, LLM generated descriptions extend the meaning associated to each concept. The impact of wrongly generated descriptions is reduced in HybridOM due to the way hybrid searches are configured.
HybridOM obtained very good and balanced values for precision and recall for all the ontology pairs in the unsupervised 2024 Bio-ML track: for three of them, it obtained the best F1 scores, respectively second best for two of them.