5th International Workshop on Scientific Knowledge: Representation, Discovery, and Assessment

Abstract

The International Workshop on Scientific Knowledge: Representation, Discovery, and Assessment (Sci-K 2025) is now running its fifth edition and will be co-located with the 24th International Semantic Web Conference, the premier conference in the Semantic Web. The Sci-K workshop brings together researchers and practitioners from diverse fields-including, but not limited to, Digital Libraries, Information Extraction, Machine Learning, Semantic Web, Knowledge Engineering, Natural Language Processing, Scholarly Communication, Science of Science, Scientometrics and Bibliometrics-as well as industry professionals, to explore innovative solutions and ideas for the production and consumption of Scientific Knowledge Graphs (SKGs) and assessing their impact on research. The workshop invited high-quality submissions centred on three main themes in the study of scientific knowledge: representation, discovery, and assessment. In response to the call for papers, the workshop has received 17 submissions from researchers in 10 diferent countries: Germany, USA, Finland, Spain, United Kingdom, Netherlands, Australia, India, Sri Lanka, and Austria. Each paper was reviewed by at least three members of the program committee. Given the quality and the interesting topics covered by the submissions, we accepted a total of 13 papers: 7 long papers, and 6 short ones. The full program can be found on the Sci-K website: https://sci-k.github.io/2025. Accepted papers are listed and briefly introduced in Section 3. Sci-K 2025 builds on three previous successful editions (Sci-K 2021 to 2024) and keeps attracting a broad and diverse pool of attendees. The first edition (Sci-K 2021) was held on 13 April 2021 in conjunction with The Web Conference 2021. Its program consisted of two keynote talks and the presentation of 11 research papers. The second edition (Sci-K 2022) took place on 26 April 2022 at The Web Conference

1. Preface

2. Sci-K Workshop Series

3. Accepted Papers

3.1. Representation

Here, we briefly introduce the accepted papers according to the main themes of the workshop. • MOP: Augmenting and Standardizing Heterogeneous Knowledge Graph Data Sources (Julia Evans, Mirjan Hofmann, Sophie Matter and Axel Klinger ) The authors present MOP (Metadata Optimization Pipeline) [ 6 ], an application to harmonise and enrich heterogeneous metadata from scientific knowledge graphs. Such metadata often varies widely in its quality, completeness, and consistency, particularly in free-text fields like titles and descriptions, which negatively impacts findability. MOP addresses this limitation by leveraging large language models (LLMs) to enrich existing metadata or generate missing fields. In this paper, the authors provide implementation details, analyse LLM output quality, and reflect on challenges encountered and lessons learned, particularly concerning computing resource limitations. • Are Scientific Annotations Consistently Represented across Science Knowledge Graphs? (Jenifer Tabita Ciuciu-Kiss and Daniel Garijo) This paper investigates how consistently four major Scientific Knowledge Graphs (SKGs)–ORKG, OpenAlex, OpenAIRE, and Papers with Code–annotate the same AI-related publications [ 7 ]. The authors find significant discrepancies in annotation coverage, terminology, and semantic alignment. Manual systems like ORKG ofer high precision but limited coverage, while automated systems such as OpenAlex and OpenAIRE provide broader annotations with lower accuracy. PwC, using a hybrid approach, delivers the most annotations but introduces redundancy and noise. A manually curated gold-standard dataset reveals that over 60% of raw annotations are incorrect or irrelevant, highlighting the need for improved annotation practices and interoperability across SKGs. 1Sci-K History – https://sci-k.github.io/history • KONDA: An LLM-based Tool for Semantic Annotation and Knowledge Graph Creation Using Ontologies for Research Data (Soo-Yon Kim, Martin Görz and Sandra Geisler) The increasing demand for FAIR (Findable, Accessible, Interoperable, and Reusable) research data management (RDM) practices has underscored the need for tools that support semantic annotation and structuring of datasets [ 8 ]. The authors of this work investigate how large language models (LLMs) can be leveraged to lower this barrier by guiding researchers through the ontology-aligned annotation and transformation of heterogeneous research data into knowledge graphs. To this end, they design and implement a modular, domain-agnostic tool that is able to process and annotate various types of research data. Results from a usability survey show that the tool is considered practical and useful, and they argue that LLMs, when embedded in an interactive and user-centred system, can significantly enhance the accessibility and efectiveness of semantic research data management. • AI4DiTraRe: Building the BFO-Compliant Chemotion Knowledge Graph (Ebrahim Norouzi, Nicole Jung, Anna M. Jacyszyn, Jörg Waitelonis and Harald Sack) This paper introduces a semantic pipeline for constructing the BFO-Compliant Chemotion Knowledge Graph, providing an integrated, ontology-driven representation of chemical research data [9]. The Chemotion-KG has been developed to adhere to the FAIR (Findable, Accessible, Interoperable, Reusable) principles and to support AI-driven discovery and reasoning in chemistry. Experimental metadata were harvested from the Chemotion API in JSON-LD format, converted into RDF, and subsequently transformed into a Basic Formal Ontology-aligned graph through SPARQL CONSTRUCT queries. Outcomes presented in this work were achieved within the Leibniz Science Campus “Digital Transformation of Research” (DiTraRe) and are part of an ongoing interdisciplinary collaboration. • A Survey on Metadata for Machine Learning Models and Datasets: Standards, Practices, and Harmonization Challenges (Genet Asefa Gesese, Zongxiong Chen, Oussama Zoubia, Fidan Limani, Kanishka Silva, Muhammad Asif Suryani, Benjamin Zapilko, Leyla Jael Castro, Ekaterina Kutafina, Dhwani Solanki, Heike Fliegl, Sonja Schimmler, Zeyd Boukhers and Harald Sack ) In this survey, the authors review and compare a range of general-purpose and ML-specific metadata standards, evaluating their suitability for cross-platform alignment, discoverability, extensibility, and interoperability [10]. The authors assess these standards based on defined criteria and analyse their potential to support unified, FAIR-compliant metadata infrastructures for ML, laying the groundwork for scalable and interoperable tooling in future ML ecosystems. • COPE: Chronic Observation and Progression Events Ontology (Asara Senaratne, Oshani Seneviratne, Hon Zent Lim and Leelanga Seneviratne) This work introduces the COPE ontology [11]. By bridging patient characteristics, temporal health trajectories, intervention strategies, and AI/ML capabilities within a unified semantic framework, the COPE ontology provides a robust foundation for interpretable, reproducible, and patient-centred decision support. The authors demonstrate its utility through exemplar queries, ofering it as a reusable resource for advancing the integration of AI in health trajectory modelling for chronic disease care. • What Are Research Hypotheses? (Jian Wu and Sarah Rajtmajer)

Over the past decades, alongside advancements in natural language processing, significant attention has been paid to training models to automatically extract, understand, test, and generate hypotheses in open and scientific domains [ 12]. However, interpretations of the term hypothesis for various natural language understanding (NLU) tasks have migrated from traditional definitions in the natural, social, and formal sciences. Even within NLU, we observe diferences in defining hypotheses across the literature. This work provides an overview and delineates various definitions of a hypothesis. Especially, the authors discern the nuances of definitions across recently published NLU tasks, and highlight the importance of well-structured and well-defined hypotheses, particularly as we move toward a machine-interpretable scholarly record. • Ontologies in Motion: A BFO-Based Approach to Knowledge Graph Construction for Motor Performance Research Data in Sports Science (Sarah Rebecca Ondraszek, Jörg Waitelonis, Katja Keller, Claudia Niessner, Anna M. Jacyszyn and Harald Sack) The Leibniz Science Campus Digital Transformation of Research (DiTraRe) studies the process of digitalisation of research in an interdisciplinary environment. The following paper [13], being a result of collaboration between computer and sports scientists, presents a base for the ontology construction of the Motor Research (MO|RE) data repository. MO|RE is a platform for collecting, publishing, and sharing motor performance data. The ontology is based on the BFO and reuses existing ontologies to build individual modules with privacy-aware extensions. The presented approach centres on formally representing the interrelation of plan specifications, specific processes, and related measurements. The main goal is to transform how motor performance data are modelled and shared across studies, making it standardised and machine-understandable.

3.2. Discovery

• Interlinking Research Data and Services in the Historical Sciences with MemO and the NFDI4Memory Knowledge Graph (Sarah Rebecca Ondraszek, Tabea Tietz, Jörg Waitelonis and Harald Sack) The German National Research Data Infrastructure (NFDI) aims to harmonise the flow of data from science and research, to realise a FAIR digital research environment in which users can discover content through shared semantic structures on previously unconnected materials [14]. This work introduces the NFDI4Memory Ontology (MemO) and the NFDI4Memory Knowledge Graph (MemO KG). In combination, they build the ground for the NFDI4Memory Data Space, supporting federated searches and semantic interoperability. This infrastructure lays the groundwork for a unified point of access to research data across disciplines and consortia.

Thereby, it fosters new modes of exploration and research data acquisition in historical research. • Knowledge Representation and Discovery for Cultural Heritage Research Data with CTO and SHMARQL (Tabea Tietz, Etienne Posthumus, Linnaea Söhn, Jonatan Jalle Steller, Oleksandra Bruns, Joerg Waitelonis, Torsten Schrade and Harald Sack) This paper presents an approach to representing, discovering, and exploring research (meta-)data in the cultural heritage (CH) domain [15]. One key component is the NFDI4Culture Ontology (CTO), a modular ontology for CH research data. Another key component is the lightweight Linked Data platform SHMARQL, which supports querying and storytelling with RDF data, ofering new possibilities for data discovery, reuse, and cross-domain integration. Both CTO and SHMARQL are integrated within NFDI4Culture, a consortium of the German NFDI programme for the national research data infrastructure that focuses on material and immaterial CH data. Designed with modularity and reuse in mind, both tools demonstrate great generalisability and have been successfully applied beyond their original CH context. • ClimaFactsKG: Towards an Interlinked Knowledge Graph of Scientific Evidence to Fight Climate Misinformation (Grégoire Burel and Harith Alani) The authors of this work introduce ClimaFactsKG, a knowledge graph that links common climate change denial narratives with scientific corrections [ 16]. ClimaFactsKG currently consists of 252 common climate myths and the corresponding scientific counter-arguments. A key feature of ClimaFactsKG is its strategic integration with CimpleKG, one of the largest existing misinformation knowledge graphs. This connection allows the interlinking of scientific corrections with over 611 misinforming climate claims found in CimpleKG and significantly enhances the utility of ClimaFactsKG. By providing a structured and interlinked repository of climate change myths and their scientific rebuttals, ClimaFactsKG ofers a valuable resource for researchers studying climate misinformation, fact-checkers seeking reliable counter-evidence, and educators aiming to improve climate literacy.

3.3. Assessment

• Towards AI-Supported Research: a Vision of the TIB AIssistant (Sören Auer, Allard Oelen, Mohamad Yaser Jaradeh, Mutahira Khalid, Farhana Keya, Sasi Kiran Gaddipati, Jennifer D’Souza, Lorenz Schlüter, Amirreza Alasti, Gollam Rabby, Azanzi Jiomekong and Oliver Karras) The article [17] addresses the topic of efectively integrating AI into research by introducing the vision of a TIB AIssistant which supports tasks across the research life-cycle. The AI-supported, human-centred, and domain-agnostic platform is envisioned as a collaborative research environment where humans and machines co-create knowledge. The TIB AIssistant does not aim for full automation; instead, it focuses on human-machine collaboration, enabling researchers to retain control, orchestrate processes, and critically evaluate AI-generated results. The paper presents the conceptual framework, system architecture, and implementation of an early prototype that demonstrates the feasibility and potential impact. • Deep Research in the Era of Agentic AI: Requirements and Limitations for Scholarly Research (Mohamad Yaser Jaradeh and Sören Auer) The authors [18] define an agentic deep search as long-horizon research workflows in which an autonomous agent formulates queries, harvests evidence from heterogeneous APIs, evaluates source quality, and synthesises structured outputs. This vision paper argues that agentic AI can become a trustworthy partner in scientific inquiry only if its design satisfies research-specific requirements. Authors discuss the following requirements: verifiability, bias control, human oversight, workflow fit, temporal rigour, narrative scope, source quality, memory model; and the following limitations: unvetted inputs, temporal staleness, paywalls and licensing, hallucination, multimodal bottlenecks, scalability and cost, privacy and data protection. The paper concludes that without guardrails, the very speed and scale that make LLM agents attractive can increase misinformation and have a negative impact on reproducibility.

4. Keynote

For this 5th edition, we are honoured to have Dr. Jian Wu, Associate Professor of Computer Science at Old Dominion University, who will talk about the use of LLMs for mining scholarly data. Dr. Wu obtained his Ph.D. degree at Pennsylvania State University (Penn State) in 2011 and worked as a postdoctoral fellow with Dr. C. Lee Giles before joining ODU in 2018. Since then, his research has been supported by NSF, IMLS, DARPA, Los Alamos National Laboratory, Virginia Commonwealth, and the Open Philanthropy. Dr. Wu’s research interests include natural language processing, scholarly big data, information retrieval, digital libraries, and the science of science. He has published more than 90 peer-reviewed papers in ACM, IEEE, and AAAI venues, with best papers and nominations, in addition to his earlier publications in Astronomy and Astrophysics. Dr. Wu shared the British Computer Society Award 2021 for the Best Open Source Project with Dr. C. Lee Giles.

Title: Deep Mining Scholarly Big Data in the Large Language Model Era

Abstract: Since 2023, there has been a surge of public and research interest in large language models (LLMs) and recently vision language models, which significantly shifted the paradigm of mining scholarly big data, bringing both challenges and opportunities for this ever-growing field. This paradigm shift not only significantly improves the performance of traditional metadata-centred pipelines for knowledge extraction, classification, and downstream tasks, which usually served as core components for academic digital libraries, but it also opens doors to the content-centred tasks, mining fine-grained knowledge and data, which provides deeper insights and wider applications of scholarly publications for a broader audience beyond scientific researchers. We explore LLM-based solutions for several content-centred tasks related to knowledge and data from scholarly publications, and prospect how these solutions can shed light on supporting advanced services, such as data preservation, scholarly comparison, review generation, and science dissemination. We share preliminary work in this direction, including open-access datasets and software extraction, complex table data extraction, scientific claim verification, and research reproducibility assessment.

5. Sponsor

We are grateful to Digital Science for their financial support.

6. Program Committee

We would like to express our heartfelt gratitude to the members of the programme committee, who, in the middle of August, dedicated time to reviewing papers. • Allard Oelen (L3S Research Center, Leibniz University, Hannover) • Sarah Rebecca Ondraszek (FIZ Karlsruhe - Leibniz Institute for Information Infrastructure) • Fabrizio Pecoraro (CNR-IRPPS) • Etienne Posthumus (FIZ Karlsruhe - Leibniz Institute for Information Infrastructure) • David Pride (KMi, The Open University) • Sarah Rajtmajer (The Pennsylvania State University) • Diego Reforgiato (Università degli studi di Cagliari) • Stefan Reichmann (TU Graz - Graz University of Technology) • Gunjan Singh (FIZ Karlsruhe - Leibniz Institute for Information Infrastructure) • Ilaria Tiddi (Vrije Universiteit Amsterdam) • Tabea Tietz (FIZ Karlsruhe - Leibniz Institute for Information Infrastructure) • Jörg Waitelonis (FIZ Karlsruhe - Leibniz Institute for Information Infrastructure) • Giacomo Zamprogno (Vrije Universiteit Amsterdam)

Declaration on Generative AI

During the preparation of this preface, the authors used Gemini in order to: Grammar and spelling check. [9] E. Norouzi, N. Jung, A. M. Jacyszyn, J. Waitelonis, H. Sack, Ai4ditrare: Building the bfo-compliant chemotion knowledge graph, CEUR Workshop Proceedings Vol-4065 (2025). URL: https://ceur-ws. org/Vol-4065/paper4.pdf. [10] G. A. Gesese, Z. Chen, O. Zoubia, F. Limani, K. Silva, M. A. Suryani, B. Zapilko, L. J. Castro, E. Kutafina, D. Solanki, H. Fliegl, S. Schimmler, Z. Boukhers, H. Sack, A survey on metadata for machine learning models and datasets: Standards, practices, and harmonization challenges, CEUR Workshop Proceedings Vol-4065 (2025). URL: https://ceur-ws.org/Vol-4065/paper5.pdf. [11] A. Senaratne, O. Seneviratne, H. Z. Lim, L. Seneviratne, Cope: Chronic observation and progression events ontology, CEUR Workshop Proceedings Vol-4065 (2025). URL: https://ceur-ws.org/Vol-4065/ paper6.pdf. [12] J. Wu, S. Rajtmajer, What are research hypotheses?, CEUR Workshop Proceedings Vol-4065 (2025).

URL: https://ceur-ws.org/Vol-4065/paper7.pdf. [13] S. R. Ondraszek, J. Waitelonis, K. Keller, C. Niessner, A. M. Jacyszyn, H. Sack, Ontologies in motion: A bfo-based approach to knowledge graph construction for motor performance research data in sports science, CEUR Workshop Proceedings Vol-4065 (2025). URL: https://ceur-ws.org/Vol-4065/ paper8.pdf. [14] S. R. Ondraszek, T. Tietz, J. Waitelonis, H. Sack, Interlinking research data and services in the historical sciences with memo and the nfdi4memory knowledge graph, CEUR Workshop Proceedings Vol-4065 (2025). URL: https://ceur-ws.org/Vol-4065/paper9.pdf. [15] T. Tietz, E. Posthumus, L. Söhn, J. J. Steller, O. Bruns, J. Waitelonis, T. Schrade, H. Sack, Knowledge representation and discovery for cultural heritage research data with cto and shmarql, CEUR Workshop Proceedings Vol-4065 (2025). URL: https://ceur-ws.org/Vol-4065/paper10.pdf. [16] G. Burel, H. Alani, Climafactskg: Towards an interlinked knowledge graph of scientific evidence to ifght climate misinformation, CEUR Workshop Proceedings Vol-4065 (2025). URL: https://ceur-ws. org/Vol-4065/paper11.pdf. [17] S. Auer, A. Oelen, M. Y. Jaradeh, M. Khalid, F. Keya, S. K. Gaddipati, J. D’Souza, L. Schlüter, A. Alasti, G. Rabby, A. Jiomekong, O. Karras, Towards ai-supported research: a vision of the tib aissistant, CEUR Workshop Proceedings Vol-4065 (2025). URL: https://ceur-ws.org/Vol-4065/paper12.pdf. [18] M. Y. Jaradeh, S. Auer, Deep research in the era of agentic ai: Requirements and limitations for scholarly research, CEUR Workshop Proceedings Vol-4065 (2025). URL: https://ceur-ws.org/ Vol-4065/paper13.pdf.

References

[1] P. Manghi , A. Mannocci , F. Osborne , D. Sacharidis , A. Salatino , T. Vergoulis, Sci-k 2022- international workshop on scientific knowledge: Representation, discovery, and assessment , in: Companion Proceedings of the Web Conference 2022 , 2022 , pp. 735 - 738 .

[2] A. A. Salatino , Y. Bu , Y. Ding , Á. Horvát, Y. Huang , M. Liu, P. Manghi , A. Mannocci , F. Osborne , D. Romero , et al., 3rd international workshop on scientific knowledge representation, discovery, and assessment (sci-k 2023) , in: Companion Proceedings of the ACM Web Conference 2023 , 2023 , pp. 783 - 783 .

[3] A. Salatino , A. Mannocci , F. Osborne , G. Rehm, S. Schimmler , et al., 4th international workshop on scientific knowledge: Representation , discovery, and assessment, volume Vol- 3780 , CEUR, 2024 . URL: https://ceur-ws. org/ Vol- 3780 /preface.pdf.

[4] L. Bellatreche , F. Bentayeb , M. Bieliková , O. Boussaid , B. Catania , P. Ceravolo , E. Demidova, M. Halfeld Ferrari , M. T. G. Lopez, C. S. Hara , et al., Databases and information systems in the ai era: Contributions from adbis, tpdl and eda 2020 workshops and doctoral consortium, in: ADBIS, TPDL and EDA 2020 Common Workshops and Doctoral Consortium: International Workshops: DOING, MADEISD , SKG, BBIGAP, SIMPDA, AIMinScience 2020 and Doctoral Consortium , Lyon, France, August 25-27 , 2020 , Proceedings 24, Springer, 2020 , pp. 3 - 20 .

[5] P. Manghi , A. Mannocci , F. Osborne , D. Sacharidis , A. Salatino , T. Vergoulis , New trends in scientific knowledge graphs and research impact assessment , Quantitative Science Studies 2 ( 2021 ) 1296 - 1300 .

[6] J. Evans , M. Hofmann , S. Matter , A. Klinger , Mop: Augmenting and standardizing heterogeneous knowledge graph data sources , CEUR Workshop Proceedings Vol- 4065 ( 2025 ). URL: https://ceur-ws. org/ Vol- 4065 /paper1.pdf.

[7] J. T. Ciuciu-Kiss , D. Garijo , Are scientific annotations consistently represented across science knowledge graphs? , CEUR Workshop Proceedings Vol- 4065 ( 2025 ). URL: https://ceur-ws. org/ Vol- 4065 /paper2.pdf.

[8] S.-Y. Kim , M. Görz , S. Geisler , Konda: An llm-based tool for semantic annotation and knowledge graph creation using ontologies for research data , CEUR Workshop Proceedings Vol- 4065 ( 2025 ). URL: https://ceur-ws. org/ Vol- 4065 /paper3.pdf.