Despite the development of several ontology reasoning optimizations, the traditional methods either do not scale well or cover only a subset of OWL 2 language constructs. The performance of the SPARQL query engines that support reasoning is also limited. As an alternative, neuro-symbolic approaches are gaining significant attention. However, the existing methods can not deal with very expressive ontology languages. To find and improve the performance bottlenecks of the reasoners, we ideally need several real-world ontologies that span the broad spectrum in terms of their size and expressivity. However, that is often not the case. One of the potential reasons for ontology developers to not build ontologies that vary in terms of size and expressivity is the performance bottleneck of the reasoners. SemREC aims to deal with this chicken and egg problem. The second edition of this challenge included the following tasks.
1. Improving Transitive Embeddings in Neural Reasoning Tasks via Knowledge
Based Policy Networks. Shervin Mehryar and Remzi Celebi. 2. Memory Networks for RDFS reasoning: Experiments. Sulogna Chowdhury,
Monireh Ebrahimi, Aaron Eberhart and Pascal Hitzler. Each submission was reviewed by at least three reviewers and their feedback was forwarded to the authors. Since this is a challenge, all the submissions were accepted.
For fair evaluation, we ran all the submissions on our server with an AMD Ryzen Threadripper 2990WX 32-Core Processor and 128GB of RAM. The server ran on a 5.3.046-generic #38 18.04.1-Ubuntu SMP operating system. We use Java 1.8 and OWL API 5.1.11 in the experiments. We set the heap space to 24GB. We report the average time taken in 2 independent runs for the classification task. Both ontologies submitted under task 1 were evaluated in terms of the time taken (in seconds) Konclude 0.7.01 and Openllet 2.6.462. We set the heap space to 24GB. The time-out was set to 30 minutes. We reported the average time taken in 2 independent runs for the classification task. We could not evaluate the neuro-symbolic reasoners submitted under task 2 as the models were not provided. Also, this category had no winner because both submissions were based on different techniques. One was based on an end-to-end Memory Network (MemN2N) with attention that captures the most relevant information to conduct logical reasoning. The results were reported for a query-based task. The other one, called aTransE, extended TransE and rTransE by leveraging transitive relations and was designed for the link prediction task (subclass and type of relations). The results that were reported in the submissions were considered for our evaluation. For more details, refer to https://semrec.github.io/evaluation2022.html.
Kendall)
1 https://www.derivo.de/products/konclude/ 2 https://github.com/Galigator/openllet In this section, we list the people who organized and contributed to the success of this event.
● Gunjan Singh, KRaCR Lab, IIIT-Delhi, India ● Raghava Mutharaju, KRaCR Lab, IIIT-Delhi, India ● Pavan Kapanipathi, IBM T.J. Watson Research Center, USA
The challenge program committee helped review the four submitted papers. The organizers would like to thank them for their valuable time. ● Monika Jain, IIIT-Delhi, India ● Sudip Mittal, Mississippi State University, USA ● Cogan Shimizu, Kansas State University, USA ● Monireh Ebrahimi, IBM Watson, USA ● Sumit Bhatia, Adobe Inc., India ● Manas Gaur, University of South Carolina, USA ● Md. Kamruzzaman Sarker, University of Hartford, USA ● Nidhi Goyal, IIIT-Delhi, India ● Fariz Darari, University of Indonesia, Indonesia
We thank the ISWC Semantic Web Challenge chairs, Catia Pesquita, and Daniele Dell’Aglio, and the ISWC organizing committee for their invaluable support. We would also like to thank the challenge participants who played a key role in the success of SemREC by submitting their quality work and informative presentations during the event. 4. J. Chen, P. Hu, E. Jimenez-Ruiz, O. M. Holter, D. Antonyrajah, and I. Horrocks,
OWL2Vec*: embedding of OWL ontologies. Machine Learning, 2021. 5. S. Mondal, S. Bhatia, and R. Mutharaju. EmEL++: Embeddings for EL++ Description Logic. Spring Symposium on Combining Machine Learning and Knowledge Engineering (AAAI-MAKE), 2021. 6. R. Mehri, V. Haarslev, and H. R. Chinaei, A machine learning approach for optimizing heuristic decision‐ making in Web Ontology Language reasoners.
Computational Intelligence. 37. 10.1111/coin.12404, 2020. 7. B. Peng, Z. Lu, H. Li, and K.-F.Wong, Towards neural network-based reasoning.
arXiv preprint arXiv:1508.05508, 2015. 8. X. V. Lin, R. Socher, and C. Xiong, Multi-Hop Knowledge Graph Reasoning with Reward Shaping. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3243–3253, 2018.