In recent years, machine learning (ML) methods have seen widespread adoption across various domains due to their ability to perform complex tasks with high accuracy. However, their applicability remains limited in certain critical fields, where the demand for reliable decision-making is crucial. Furthermore, the eficacy of ML models is often limited by data availability and quality, posing significant challenges especially in data-sensitive areas. Addressing these limitations, this thesis explores the integration of semantic knowledge through knowledge graphs to enhance the performance of ML models. Knowledge graphs, with their structured representation of domain-specific information, ofer a way to augment ML models with domain insights, improving their performance and potentially overcoming the issues of data scarcity. This research contributes to the field of Neurosymbolic AI and Semantic Web by not only demonstrating the feasibility and benefits of combining knowledge graphs with ML but also by ofering guidance on the efective construction and utilization of knowledge graphs for the purpose of ML enhancement.
CEUR ceur-ws.org
The application of artificial intelligence (AI) techniques across diverse fields has significantly
expanded the potential of Machine Learning (ML) models in addressing complex problems.
The third wave of AI, Neuro-symbolic (NeSy) AI [
Despite the promising advancements brought by NeSy AI, several challenges remain prevalent.
ML models require large datasets for training, which can be sparse, of low quality, or biased,
particularly outside mainstream applications [
Considering these challenges, an emerging area of research focuses on knowledge infusion into ML. In this thesis we explore how the infusion of KGs influence ML outcomes, especially concerning accuracy, data sparsity, and data quality. Moreover, we also investigate how the Proceedings of the Doctoral Consortium at ISWC 2024, co-located with the 23rd International Semantic Web Conference (ISWC 2024) nEvelop-O quality, granularity, and relevance of KGs influence the improvement of ML models when these KG are infused. To our knowledge, these dependencies have not been extensively investigated previously. Addressing these issues entails refining the ways in which knowledge is represented and utilized, ensuring that KGs not only supplement but also synergize with ML processes to drive more accurate and reliable results.
This research focuses on the settings where a supervised ML model is used for prediction
and investigates methods to enhance this through knowledge infusion. Illustrative use cases
examined include the prediction of heart disease using tabular data on patients and their clinical
features [
Problem Importance This thesis tackles a significant challenge in ML by enhancing model performance despite the constraints of insuficient or low-quality data. The main benefits of this thesis are three fold. First, by integrating semantically rich knowledge from KGs into ML pipelines, this thesis aims to reduce the reliance of ML models on large data sets, which is particularly crucial in sensitive domains where data is scarce, such as medical domain. The integration of KGs promises to improve decision-making processes, enhancing the accuracy and reliability of ML applications. Second, this thesis advances the field of NeSy AI by exploring efective methods and proposing an innovative approach to use KG insights into ML pipelines, thus enriching both the research community and practical applications. Third, the findings of this thesis are also beneficial for KG engineers, providing them with detailed insights on the characteristics that KGs should possess to optimally support ML models.
The broadest area related to this research is the integration of sybmbolic and sub-symbolic AI,
known as neurosymbolic AI. This integration combines reasoning with data-driven learning,
significantly advancing AI capabilities. As outlined by Henry Kautz [
Within the broader scope of NeSy described earlier, one specific focus is the integration of
semantic knowledge into ML pipelines. Examples of such successful integration are as follows.
Knowledge-based artificial neural networks (KBANN) [
Besides the use of semantic knowledge to augment the input of the ML pipeline, there have
been successful eforts to incorporate domain knowledge into the loss functions and weights of
neural networks [
These examples underline the diverse applications and potential of semantic structures in enhancing machine learning, setting a precedent for our research. However, there are only a limited number of approaches of KG based infusion. Additionally, there is minimal empirical evidence on how KG characteristics influence this integration.
In this thesis, we will focus on the use of the KGs to enhance the predictions of the ML models. We aim to investigate the following overall research question: How to efectively and eficiently infuse KG into ML pipeline? In particular, we will investigate the following sub-research questions:
RQ1: What are the approaches to infuse KGs into ML pipelines? Understanding the approaches to infuse KGs into ML pipelines is crucial, as KGs provide contextualized and structured information that can potentially enhance ML models. The explicit representation of knowledge in KGs enables more informed feature engineering, helps in data preprocessing, and can be used for better interpretation of model outcomes. To address this question, we aim to explore various ways to leverage KGs for enhancing the predicting performance of ML methods and introduce a novel approach. Our focus will be on identifying the most efective ways to integrate KGs into various parts of the ML pipeline, in order to improve the performance of ML models. The contribution of addressing this question is to identify and categorize various methods to leverage KGs efectively, thereby proposing an innovative approach to enhance ML workflows. This framework could serve as a guideline for practitioners and researchers to optimize the performance of their ML systems using KGs.
RQ2: To what extent does the infusion of KGs into ML pipelines impact model performance? Assessing the impact of KG infusion is critical to validate the theoretical benefits of KGs and understand their efectiveness in real-world applications. A key focus of our investigation will be to explore whether integrating KGs can enhance the ML performance in terms of accuracy and F-measures, compared to the baseline when no KG is used. Additionally, we aim to address the data-dependency problem inherent in ML models, enabling efective learning with fewer data by incorporating semantically rich, structured knowledge. The investigation of this question will empirically evaluate the performance changes in ML models with KG infusion across various use cases. This provides evidence-based insights into the advantages and possible limitations of KG integration, enabling more informed decisions in the development of ML models.
RQ3: How do the characteristics of KGs influence the performance of ML models? It is important to explore how specific KG characteristics, such as size and modeling techniques, afect ML model performance in order to guide the design of more efective KGs tailored to enhance specific ML tasks. This question is critical within the field of knowledge engineering, as it informs the development of optimized KGs tailored for integration into NeSy systems. By understanding the specific characteristics that contribute to performance improvements, we can design more efective and meaningful KGs suited for advanced ML applications. Exploring this question contributes by developing novel metrics or requirements for KGs to ensure their optimal infusion into ML models. This research could lead to a set of best practices for KG construction and refinement specifically aimed at enhancing ML applications.
In this thesis, we will adopt the Design Science methodology [20] to systematically develop and evaluate artifacts that infuse knowledge from KG into various steps of ML pipeline. In the following we show how we will address each cycles.
Relevance Cycle This cycle connects the research to the real-world by identifying the practical problems that the artifacts aim to solve and demonstrating the artifacts’ impact in a real application setting. In our case, the relevance cycle involves engaging with stakeholders in the ML and semantic web communities to identify key challenges in integrating KGs with ML models, using methods such as literature study. We will collaborate with industry professionals and academic peers to ensure that our solutions are applicable and address significant needs. Design Cycle We will design approaches that integrate KG data into ML systems, addressing our first research question (RQ1) which focuses on diferent ways of knowledge infusion. Our current investigation explores the infusion of KG embeddings as part of the input to the ML model, augmenting tabular data with semantics from KGs constructed using instances from the same dataset. Moreover, we will assess the impact of these integrations, and evaluate the enhancement of ML models to demonstrate how KGs can enhance ML model performance, directly tackling our second research question (RQ2). Currently we are focusing in medical domain, for heart and kidney disease prediction. Lastly, we will evaluate how diferent characteristics of the KGs influence ML methods’ improvement, thus answering our third research question (RQ3).
To evaluate the research questions, the improvements of ML methods and the impact of KGs, we will compare the baseline when no additional knowledge is being used, with diferent infusion methods. We will measure accuracy and F-scores as evaluation metrics. Accuracy reflects the proportion of correct predictions, while the F-score, a combination of precision and recall, provides a balanced measure of a model’s precision and its ability to identify all relevant instances. The choice for F-score is particularly crucial in fields such as medicine where the consequences of false negatives or positives are significant, making the F-score a more comprehensive indicator of model performance than accuracy alone.
Preliminary Results In our previous work [
Rigor Cycle We will ensure the rigor cycle by incorporating knowledge from existing literature and theoretical foundations in the fields of ML and KGs. Our approach will also utilize established methodologies and frameworks from previous research to guide the design and evaluation processes. Throughout the thesis, we will maintain a rigorous scientific approach, using well-defined metrics and evaluation methods to ensure that our findings are valid, reliable, and reproducible.
The planned work for this thesis is depicted in Figure 1, outlining our structured approach to explore, implement, and evaluate the proposed enhancements.
This thesis aims to contribute in the Neuro-Symbolic AI field by exploring methods to integrate knowledge from knowledge graphs into the machine learning pipeline, aiming to enhance their performance. Additionally, this thesis contributes in the knowledge engineering area, by identifying characteristics and modeling techniques for knowledge graphs, making them more efective for the purpose of enhancing machine learning methods in prediction.
As illustrated in Figure 2, I started my PhD in July 2022, initially focusing on familiarizing
myself with the field and identifying research gaps. Until now, I have developed essential
research skills, participated in PhD courses as part of my curriculum, and engaged in teaching
activities. Additionally, I have co-authored several papers in the neuro-symbolic area, enhancing
my understanding and expertise in this field. Moreover, I have contributed to publications and
presented preliminary findings on RQ1, RQ2, and RQ3 at the Airov Workshop [
I would like to thank my supervisors, Prof. Marta Sabou and Asst. Prof. Fajar J. Ekaputra for their valuable advises, expert insights and guidance. The work presented in this paper is supported by the FWF HOnEst project (No. V 745-N). [19] H. Wang, F. Zhang, M. Zhang, J. Leskovec, M. Zhao, W. Li, Z. Wang, Knowledge-aware graph neural networks with label smoothness regularization for recommender systems, in: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, 2019, pp. 968–977. [20] A. R. Hevner, S. T. March, J. Park, S. Ram, Design science in information systems research, MIS quarterly (2004) 75–105.