SemREC 2023 was the third edition of the Semantic Reasoning Evaluation Challenge (SemREC), co-located with the 22nd International Semantic Web Conference (ISWC 2023).

Abstract

Despite the development of several ontology reasoning optimizations, the traditional methods either do not scale well or cover only a subset of OWL 2 language constructs. The performance of the SPARQL query engines that support reasoning is also limited. As an alternative, neuro-symbolic approaches are gaining significant attention. However, the existing methods cannot deal with very expressive ontology languages. To find and improve the performance bottlenecks of the reasoners, we ideally need several real-world ontologies that span the broad spectrum in terms of their size and expressivity. However, that is often not the case. One of the potential reasons for ontology developers to not build ontologies that vary in terms of size and expressivity is the performance bottleneck of the reasoners. SemREC aims to deal with this chicken and egg problem. The third edition of this challenge included the following tasks.

-

Each submission was reviewed by at least two reviewers and their feedback was forwarded to the authors. Since this is a challenge, both the submissions were accepted.

Task 2 Winners

Evaluating Different Methods for Semantic Reasoning Over Ontologies

Fernando Zhapa-Camacho and Robert Hoehndorf The winner was decided based on the performance of submitted systems assessed against the provided datasets. Performance evaluation encompassed factors such as OWL expressivity, dataset sizes, and correctness measures. Additional details can be found at https://semrec.github.io/evaluation2022.html.

Organization

In this section, we list the people who organized and contributed to the success of this event.

Challenge Chairs ● Gunjan Singh, KRaCR Lab, IIIT-Delhi, India ● Raghava Mutharaju, KRaCR Lab, IIIT-Delhi, India ● Pavan Kapanipathi, IBM T.J. Watson Research Center, USA

Challenge Program Committee Members

The challenge program committee helped review the submitted papers. The organizers would like to thank them for their valuable time. ● Monika Jain, IIIT-Delhi, India ● Nidhi Goyal, Mahindra University, Hyderabad, India ● Mayank Kharbanda, IIIT-Delhi, India

Acknowledgements

We thank the ISWC Semantic Web Challenge chairs, Valentina Ivanova and Wen Zhang, and the ISWC organizing committee for their invaluable support. We would also like to thank the challenge participants who played a key role in the success of SemREC by submitting their quality work and informative presentations during the event.

References

1. M. Ebrahimi , M.K. Sarker , F. Bianchi , N. Xie , D. Doran , and P. Hitzler , Reasoning over RDF knowledge bases using deep learning, arXiv preprint , arXiv: 1811 .04132, 2018 .

2. P. Hohenecker and T. Lukasiewicz , Deep learning for ontology reasoning , CoRR, arXiv: 1705 .10342, 2017 .

3. B. Makni and J. Hendler , Deep learning for noise-tolerant RDFS reasoning , SemanticWeb 10 ( 5 ) ( 2019 ), 823 - 862 .

4. J. Chen , P. Hu , E. Jimenez-Ruiz , O. M. Holter , D. Antonyrajah , and I. Horrocks , OWL2Vec*: embedding of OWL ontologies . Machine Learning , 2021 .

5. S. Mondal , S. Bhatia , and R. Mutharaju . EmEL++: Embeddings for EL++ Description Logic . Spring Symposium on Combining Machine Learning and Knowledge Engineering (AAAI-MAKE) , 2021 .

6. R. Mehri , V. Haarslev , and H. R. Chinaei , A machine learning approach for optimizing heuristic decision‐making in Web Ontology Language reasoners . Computational Intelligence . 37 . 10.1111/coin.12404, 2020 .

7. B. Peng , Z. Lu , H. Li , and K.-F.Wong , Towards neural network-based reasoning . arXiv preprint arXiv:1508.05508 , 2015 .

8. X. V. Lin , R. Socher , and C. Xiong, Multi-Hop Knowledge Graph Reasoning with Reward Shaping . Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pages 3243 - 3253 , 2018 .