LogMap Family welcomes LogMapLLM in the OAEI 2025

Abstract

We present the participation of LogMap and its variants in the OAEI 2025 campaign. The LogMap project started in January 2011 with the objective of developing a scalable and logic-based ontology matching system. This year we have introduced a new variant called LogMap-LLM that uses a large language model as an Oracle for the cases where LogMap is uncertain. LogMap [1, 2] is an ontology matching system that (i) can eficiently match semantically rich ontologies containing tens (and even hundreds) of thousands of classes, (ii) incorporates sophisticated reasoning and repair techniques to minimise the number of logical inconsistencies [3], and (iii) provides support for user intervention during the matching process [4]. LogMap ISWC 2011 paper [1] was awarded the SWSA Ten-Year Award.1 Figure 1 shows an overview of LogMap's architecture.

1.1. LogMap variants in the 2025 campaign

As in previous campaigns, we have participated in the OAEI 2025 with several variants in addition to LogMap: LogMapLLM (newly introduced in the OAEI 2025), LogMapBio, and LogMapLt. The OAEI results also report on LogMapKG, which is the same as LogMap but with the flag to output instance mappings activated. In some tasks, as expected, LogMap and LogMapKG produce the same results. LogMapLLM We have extended LogMap to perform calls to an LLM-based Oracle [ 5 ] (see overview in Figure 2). The LLM-based Oracle is used to validate a subset of the correspondences where LogMap is uncertain. Thus, the power of the LLM is centred on complex cases where traditional ontology alignment techniques are not suficient. We have experimented with GPT-4o Mini model2 (OpenAI), a range of Google Gemini Flash models3 (v1.5, 2.0, 2.0 Lite, and 2.5 Preview), and the open-weight models Mistral Small-2402, Llama 3-70b, Qwen3-1.7b, and Qwen3-8b. In the OAEI 2025 campaign, we participated with Gemini 2.5 as Oracle. Note that the F-scores reported in [ 5 ] are diferent from those in the OAEI Bio-ML track, as this track does not consider the whole ground truth for the (global matching) evaluation.4 LogMapLt is a “lightweight” variant of LogMap, which essentially only applies (eficient) string matching techniques.

LogMapBio includes an extension to use BioPortal [ 6 ] as a (dynamic) provider of mediating ontologies instead of relying on a few preselected ontologies [ 7 ]. LogMapBio architecture is depicted in Figure 3.

1.2. Link to the system and parameters file

LogMap is open-source and released under the Apache-2.0 License.5 LogMap components and source code are available from the LogMap’s GitHub page: https://github.com/ernestojimenezruiz/ logmap-matcher/. LogMap distributions can be easily customised through a configuration file containing the matching parameters. LogMapLLM codes to call the LLM-based Oracle are available in GitHub: https://github.com/city-artificial-intelligence/rai-ukraine-kga-llm

1.3. Results

The complete results of the LogMap family in the OAEI 2025 [ 8 ] campaign are reported in http: //oaei.ontologymatching.org/2025/results/. It is worth mentioning that LogMapBio and LogMapLLM 2https://openai.com/index/gpt-4o-mini-advancing-cost-eficient-intelligence/ 3https://deepmind.google/technologies/gemini/ 4See details in the Bio-ML track: https://liseda-lab.github.io/OAEI-Bio-ML/2025/index.html 5http://www.apache.org/licenses/ were the top systems in the Bio-ML track [ 9, 10 ]. LogMapBio, using traditional mediating ontology matching, surpasses more sophisticated systems relying on machine learning and large language model techniques. Table 1 shows the results and ranking of LogMap, LogMapBio and LogMapLLM in the OAEI’s Anatomy and Bio-ML tracks. In the Bio-ML track, on average, LogMapBio and LogMapLLM ranked first and second, respectively, while LogMap ranked fifth, right after BertMapLt and BertMap [ 11 ]. Table 2 shows the results of LogMap in other OAEI tracks.

Declaration on Generative AI During the preparation of this work, the authors used Grammarly to check grammar, spelling, and readability. After using the tool, the authors reviewed and edited the content as needed to take full responsibility for the publication’s content. We would like to thank Bernardo Cuenca-Grau, Ian Horrocks, Alessandro Solimando, Jiaoyan Chen, Valerie Cross, Anton Morant, Yujiao Zhou, Weiguo Xia, Xi Chen, Yuan Gong, Shuo Zhang and Rob Upson, who have contributed to the LogMap project in the past.

The LogMapLLM research was partially supported by the NYU Center for Responsible AI and carried out within its RAI for Ukraine program. This initiative was coordinated by the Center for Responsible AI at the Tandon School of Engineering, New York University (NYU), with support from the Faculty of Applied Sciences at the Ukrainian Catholic University (UCU). The Ukrainian students involved in the project gratefully acknowledge the support from the mentors at City St George’s, University of London, and the support of the RAI for Ukraine program organisers.

References

[1] E. Jiménez-Ruiz , B. Cuenca Grau , LogMap: Logic-based and Scalable Ontology Matching , in: Int'l Sem. Web Conf. (ISWC) , 2011 , pp. 273 - 288 .

[2] E. Jiménez-Ruiz , B. Cuenca Grau , Y. Zhou , I. Horrocks , Large-scale interactive ontology matching: Algorithms and implementation , in: Europ. Conf. on Artif. Intell. (ECAI) , 2012 .

[3] A. Solimando , E. Jiménez-Ruiz , G. Guerrini, Minimizing conservativity violations in ontology alignments: Algorithms and evaluation, Knowledge and Information Systems ( 2016 ).

[4] H. Li , Z. Dragisic , D. Faria , V. Ivanova , E. Jiménez-Ruiz , P. Lambrix , C. Pesquita, User validation in ontology alignment: functional assessment and impact , Knowl. Eng. Rev . 34 ( 2019 ) e15 .

[5] S. Lushnei , D. Shumskyi , S. Shykula , E. Jiménez-Ruiz , A. d'Avila Garcez, Large Language Models as Oracles for Ontology Alignment , CoRR abs/2508 .08500 ( 2025 ). URL: https://doi.org/10.48550/ arXiv.2508.08500. doi: 10 .48550/ARXIV.2508.08500. arXiv: 2508 . 08500 .

[6] N. Fridman Noy , N. H. Shah , P. L. Whetzel , B. Dai , et al., BioPortal: ontologies and integrated data resources at the click of a mouse , Nucleic Acids Research 37 ( 2009 ) 170 - 173 .

[7] X. Chen , W. Xia , E. Jiménez-Ruiz , V. Cross , Extending an ontology alignment system with bioportal: a preliminary analysis , in: Poster at Int'l Sem. Web Conf. (ISWC) , 2014 .

[8] M. A. N. Pour , et al., Results of the Ontology Alignment Evaluation Initiative 2025 , in: Proceedings of the 20th International Workshop on Ontology Matching co-located with the 24th International Semantic Web Conference (ISWC) , CEUR Workshop Proceedings, CEUR-WS.org , 2025 .

[9] Y. He , J. Chen , H. Dong , E. Jiménez-Ruiz , A. Hadian , I. Horrocks , Machine Learning-Friendly Biomedical Datasets for Equivalence and Subsumption Ontology Matching , in: 21st International Semantic Web Conference , 2022 .

[10] Y. He , J. Chen , H. Dong , E. Jiménez-Ruiz , A. Hadian , I. Horrocks , Bio-ML: Machine LearningFriendly Biomedical Datasets for Equivalence and Subsumption Ontology Matching (Version OAEI Bio-ML 2023 ), 2023 . URL: https://doi.org/10.5281/zenodo.8193375.

[11] Y. He , J. Chen , D. Antonyrajah , I. Horrocks , BERTMap: A BERT-Based Ontology Alignment System , in: Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI) , AAAI Press, 2022 , pp. 5684 - 5691 . URL: https://doi.org/10.1609/aaai.v36i5.20510. doi: 10 .1609/AAAI.V36I5.20510.

[12] J. da Silva , K. Revoredo , F. A. Baião , C. Lima, ALIN results for OAEI 2025 , in: Proceedings of the 20th International Workshop on Ontology Matching co-located with the 24th International Semantic Web Conference (ISWC) , CEUR Workshop Proceedings, CEUR-WS.org , 2025 .

[13] D. Faria , M. C. Silva , P. Cotovio , L. Ferraz , L. Balbi , C. Pesquita , Results in OAEI 2025 for matcha , in: Proceedings of the 20th International Workshop on Ontology Matching co-located with the 24th International Semantic Web Conference (ISWC) , CEUR Workshop Proceedings, CEUR-WS.org , 2025 .

[14] A. Becker , TIM Results in the OAEI 2025 , in: Proceedings of the 20th International Workshop on Ontology Matching co-located with the 24th International Semantic Web Conference (ISWC) , CEUR Workshop Proceedings, CEUR-WS.org , 2025 .