We propose a novel methodology that combines Graph Neural Networks (GNNs) with Fuzzy Logic to enhance the interpretability of deep learning models. GNNs handle structured data, while Fuzzy Logic provides a framework that excels in handling uncertainty and imprecision. To solve the challenge of interpretability in GNNs, we present a novel approach that marries GNNs' expressive power with Fuzzy Logic's readability. Preliminary experiments show promising results, indicating the potential of this approach to create AI systems that are transparent and trustworthy.
Fuzzy Inference Systems, on the other hand, grounded in Fuzzy Logic, excel in handling uncertainty and imprecision—traits inherent in real-world data. By articulating knowledge in the form of fuzzy rules, FIS provides a mechanism for reasoning that is both interpretable and adaptable, ofering insights into the decision-making process that are intuitively understandable.
Therefore, the synthesis of GNNs and FIS represents an innovative approach to surmounting
the barriers of interpretability and flexibility in AI. This paper explores this novel neuro-symbolic
approach, demonstrating its potential to advance the state of the art in XAI. By bridging the gap
between the rich learning capabilities of GNNs and the interpretability aforded by Fuzzy Logic,
we propose a hybrid method that incorporates Fuzzy Logic into a GNN so that the graph-based
knowledge acquired from data can be easily expressed in the form of interpretable fuzzy rules.
Our basic idea is to apply a “fuzzification” process to transform the input features into fuzzy
variables with associated linguistic terms. This granular form of input features allows for the
adaptation of tabular data to graph representations so that a GNN can be applied to learn a
classification model from data. To further enhance explainability, we apply GNNExplainer,
which efectively improves the transparency of the graph-based model by analyzing the nodes
that were most significant in the classification task [
The rest of this paper is structured as follows. Section 2 reviews the state of the art. Section 3 describes the proposed method. Section 4 presents preliminary quantitative and qualitative experiments. Section 5 summarizes our findings, draws conclusions, and outlines future works.
In recent years, Graph Neural Networks have emerged as powerful tools for modeling complex
relationships and structures in data represented as graphs [
In particular, Fuzzy Logic [
By incorporating Fuzzy Logic, which deals with uncertainty and imprecision in data, hybrid models aim to provide clearer explanations for AI systems’ decisions. This combination allows for a more intuitive understanding of how neural networks process information and make predictions, thus fostering trust and transparency in AI applications. Moreover, leveraging the complementary strengths of neural networks and Fuzzy Logic can lead to more robust and adaptable models that excel in various domains where interpretability is crucial, such as healthcare, finance, and social networks.
However, the achievements in hybrid methods that combine FIS with neural networks have
not been mirrored in GNNs. Specifically, using GNNs within neuro-symbolic approaches has not
been extensively explored in the literature [
To our knowledge, our work is the first attempt to embed fuzzy theory into GNNs to achieve accurate and interpretable models.
We introduce a novel method that integrates Fuzzy Logic with GNNs to improve model interpretability while retaining performance. The proposed method involves the following steps: 1. Fuzzification: The first step involves transforming crisp data inputs into fuzzy representations, allowing for the incorporation of uncertainty and imprecision inherent in real-world data. 2. Graph construction: Following fuzzification, the method constructs a graph representation of the data, where nodes represent input features and related linguistic terms. 3. Graph-based classification: Using the constructed graph, the method applies a GNN to make predictions based on the learned patterns and relationships encoded in the graph. This step leverages the rich expressiveness of GNNs to model complex relational structures within the data efectively. 4. Graph-based explanation: One of the distinguishing features of the method is its ability to provide intuitive and simple explanations for the classification decisions made by the model. By analyzing the graph structure and activations of the GNN, the method generates interpretable explanations, shedding light on the reasoning behind each prediction. 5. Fuzzy rule extraction: The method extracts salient fuzzy rules from the learned graphbased model, encapsulating the decision-making logic in a human-understandable format. These fuzzy rules capture the underlying patterns and relationships the model identifies, ofering valuable insights into the data and the decision-making process.
In this section, we detail the process of fuzzification [
We initially fuzzify the input values to address the inherent uncertainty in the data.
Specifically, each input feature , = 1 . . . , is granulated into fuzzy sets 1, 2, . . . , . For
this study, we maintain a uniform number of fuzzy sets [
The membership functions for these fuzzy sets are illustrated in Fig. 1. Specifically, we employ a z-function for the LOW term, a gaussian function for the MEDIUM term, and a s-function for the HIGH term. These functions were selected to reflect the domain of discourse accurately. The deployment of “open-ended” fuzzy sets at both extremities of the domain implies that our model can accommodate values that extend beyond the observed range in the dataset, thereby ensuring a holistic representation of the phenomenon under investigation.
To establish the parameters for the fuzzy sets, we determine the 25ℎ, 50ℎ, and 75ℎ percentiles of the data distribution. These percentiles serve as critical reference points for defining the core and support of each fuzzy set, ensuring that the membership functions are appropriately aligned with the underlying data distribution.
The outcome of the fuzzification phase is a “fuzzified” dataset = {(x, m, )}=1, wherein the original data points are augmented with fuzzy membership values for each input feature. This fuzzified dataset forms the basis for subsequent phases of our approach.
As a further data engineering phase, we represent each data point as a direct graph, defined as a couple = (, ℰ ), where is the set of vertices and ℰ ⊆ × is the set of edges. To this end, given a data point (x, m) ∈ , the associated graph instance is created by composing a graph the membership value nodes , symbolizing the membership of the given feature to the corresponding fuzzy set. structure with nodes for each feature and each fuzzy set . Nodes corresponding to fuzzy sets have the membership value as feature value. Edges are established by connecting each feature node to its corresponding fuzzy set nodes. Formally, for each data point, we define = {1, 2, . . . , , 11, 12, . . . , } and ℰ = {(, ) with = 1, . . . , , = 1, . . . , }.
Figure 2 shows an example of a graph created with three input features 1, 2, 3 each connected to three fuzzy sets, leading to the following configuration: 1, 2, 3, 11, 12, 13, 21, 22, 23, 31, 32, 33.
After applying the fuzzification process to the dataset and creating the corresponding graphs,
a new dataset, denoted as , is generated as a set of labeled graphs. Formally, we have
= {(, ) | = (, ℰ)}=1. This transformation converts the initial classification
task into a graph classification task. We employ a graph-based methodology to address this
issue, training a Graph Convolutional Network (GCN) [
⎛ ℎ (+1) = ⎝()
∑︁ ∈ ()∪{} ˆ , √︁ ˆ ˆ
⎞ ℎ () ⎠ , (+1) = ︂( ˆ− 21 ˆˆ− 21 () () , ︂) where is a general activation function (e.g., ReLU), ˆ = + is the adjacency matrix of the input graph with inserted self-loops, ˆ, with ˆ = ∑︀ ˆ =0 is the diagonal degree matrix associated with ˆ, () is the node embedding matrix calculated at the -th layer, () is a trainable parameter matrix, and is the neighborhood function defined as : ↦→ * such that () = { |(, ) ∈ ℰ ∨ ( , ) ∈ ℰ }.
The GCN acts as an encoder that, given an instance graph , refines its initial node feature matrix thanks to the graph convolutions to produce another instance graph , which has the same structure of but has a diferent node feature matrix, denoted as . Generating a distinct feature vector representing the graph in a vector space is essential to classify an entire input graph instance. To achieve this, we compress the entire refined feature matrix ∈ R× ℎ, where ℎ denotes the hidden embedding dimensionality, applying a pooling layer to compute a feature-wise average and producing a comprehensive graph feature vector p ∈ Rℎ. Finally, p ∈ Rℎ is fed to a classification head layer to compute the output class, represented as a softmax-activated class probability distribution. After training the GCN, we obtain a base model Φ that can classify test instances represented as graphs.
This phase of our framework is devoted to deriving explanations from the graphs corresponding
to test instances’ classifications using GNNExplainer [
After finalizing this optimization phase, parameters nearing 1.0 accentuate pivotal node characteristics or connections within the input graph, delineating the most significant subgraph for a specific prediction. Therefore, given any test instance, we obtain the corresponding subgraph containing only edges crucial for the specific prediction.
This corresponds to pruning of useless fuzzy sets for each input variable. Indeed, for all the data in the test set, we calculate the average of the activation values of the edges and prune of the edges with a value above a threshold (defined empirically) of 0.50. Due to edge pruning, scenarios may arise where all linguistic terms associated with an input variable are pruned. In such cases, the variable is consequently eliminated, leading to automatic feature selection.
Given a specific input instance, the final step is to convert the subgraph obtained from the graphbased explanation phase into a linguistic fuzzy rule. To achieve this, we assign the relevant fuzzy linguistic terms for each specific instance in the prediction to every node corresponding to a fuzzy variable. This is done using the “is” connector, which signifies the membership of a specific instance to a fuzzy linguistic term. In cases where multiple fuzzy linguistic terms for the same fuzzy variable are required for prediction, we employ the “OR” connector. Otherwise, we apply the “AND” connector. If an input variable’s value lacks representation by any linguistic term with suficient degree, it is deemed irrelevant and excluded from the antecedent of the fuzzy rule. This process continues until all input nodes have been processed, thereby establishing the antecedent of the fuzzy rule. For the definition of the fuzzy rule’s consequent, we append the name of the output variable, succeeded by the prediction outcome, following the “THEN” connector.
An example of extracting a fuzzy rule is shown in Fig. 4. The example has the same structure as Fig. 2, but since it is in the prediction phase of the result, the entire subgraph is represented, including the target class () with its corresponding result (1). GNNExplainer identifies the most important edges, which are colored in red. These edges are the connections from fuzzy variable 1 to linguistic term 13, from fuzzy variable 2 to linguistic terms 22 and 23, and from fuzzy variable 3 to linguistic term 32. Subsequently, the significant edges are connected to the target variable. The final result obtained is the following fuzzy rule:
IF 1 is 13 AND 2 is 22 OR 23 AND 3 is 32
THEN is 1
To show the efectiveness of the proposed graph-based neuro-symbolic method, preliminary experiments were carried out on two standard datasets: the Iris dataset1 and the Haberman
dataset.2 To better evaluate the quality of the models produced by the proposed method in terms of accuracy and interpretability, they were compared with models generated by the ANFIS neuro-fuzzy network. This four-layer feed-forward neural network reflects a fuzzy rule base in its parameters and topology [17].
The same experimental setup was used to train the ANFIS neuro-fuzzy network for a fair comparison. The dataset was divided into 60% training set, 10% validation set, and 30% test set using the holdout method. A search for optimal hyperparameters was performed using the Hyperopt algorithm3 [18]. The following hyperparameters of the GCN were optimized: number of layers, number of hidden channels, aggregation function, batch size, and learning rate. To train the GCN, we used the PyTorch Geometric (PyG)4 library that ofers various types of architectures. In this preliminary study, we limited our investigation to the basic Graph Convolutional Network, without testing diferent types of graph neural layers.
The Iris dataset consists of 150 samples of Iris flowers, each belonging to one of three species: Setosa, Versicolor, or Virginica. For each sample, four features were measured: sepal length, sepal width, petal length, and petal width. The goal is to predict the species of Iris flowers based on their feature measurements. 2Haberman dataset: https://www.kaggle.com/datasets/gilsousa/habermans-survival-data-set 3Hyperopt library: https://github.com/hyperopt/hyperopt 4PyG library: https://pytorch-geometric.readthedocs.io/en/latest/
Dataset Iris Haberman
Method Our method ANFIS Our method ANFIS
As shown in Table 1, the proposed graph-based approach and ANFIS exhibited similar performance levels in accuracy, with our method yielding a marginally lower accuracy score than ANFIS. However, our method overcomes ANFIS in terms of readability. Indeed, the model generated by ANFIS includes 81(= 34) rules, and each rule contains all the input variables in the antecedent part. Due to this high number of fuzzy rules, the ANFIS model can be hard to read and interpret. Figure 5 shows some of the 81 rules generated by ANFIS. Conversely, the model generated by our method is highly interpretable since it provides a single rule for classifying a test instance, and each rule contains only significant input variables and useful fuzzy sets. For example, Fig. 6 shows the sub-graph generated by GNNExplainer for classifying a specific test instance. It can be seen that only edges with strong activation (red edges) are retained because they are considered significant for the final classification. The input variable petal width is weakly represented by its linguistic terms (blue edges), and it is not connected to the output node, indicating that it does not afect the final prediction. The sub-graph can be easily translated into a compact linguistic form as follows:
IF sepal lenght is low OR medium AND sepal width is low OR medium
AND petal length is low OR high
THEN type is Setosa
Our method gives users an understandable description of the prediction for a given test instance, presented in both graphical form (the sub-graph) and textual form (the linguistic IF-THEN rule).
The Haberman dataset is a classic dataset in the field of medical research and machine learning, containing information about patients who underwent surgery for breast cancer at the University of Chicago’s Billings Hospital between 1958 and 1970. It consists of 306 instances and three input features: age (the age of the patient at the time of surgery), year (the year of the surgery), and nodes (the number of positive axillary lymph nodes detected). The target variable is the status, i.e., the survival status of the patient after surgery, categorized as either 0 (survived for 5 or more years) or 1 (died within 5 years).
Table 1 summarizes the comparative results. It can be seen that our method has slightly lower accuracy than ANFIS. However, our model outperforms ANFIS in terms of interpretability as it provides a synthetic rule for a given instance. Figure 7 shows an example of an explanation generated by our method for an instance of the Haberman dataset. It can be seen that the pruning mechanism has efectively removed the linguistic term high from the fuzzy partition of the variable age due to its activation exceeding the threshold of 0.50. Starting from this sub-graph, the following linguistic rule is derived: IF age is low OR medium AND years is medium OR high AND nodes is medium THEN status is 0
In this study, we proposed a new model that combines Graph Neural Networks with Fuzzy Logic to enhance the interpretability of deep learning models. To test our model, we conducted a preliminary study comparing our method with the ANFIS neuro-fuzzy network using two benchmark datasets (Iris and Haberman). The quantitative results indicate that our model achieved results similar to those of the neuro-fuzzy network. In detail, there is a 2% accuracy diference in favor of the neuro-fuzzy model for both tested datasets. We can confirm that our model achieves levels of robustness comparable to those of ANFIS.
Regarding the qualitative analysis, our model features a “one-shot” rule generation mechanism, explaining the prediction for each instance. On the other hand, the neuro-fuzzy model does not operate this way. It generates diferent fuzzy rules according to the number of fuzzy variables and fuzzy linguistic terms. Thus, ANFIS obtained 81 and 27 fuzzy rules for the Iris and Haberman datasets, respectively. Therefore, a significant reduction in interpretive complexity can be observed. Using our model in a real-world context would allow domain experts to explain the prediction outcome instantly.
In future work, it will be essential to compare our model with traditional machine learning models to extend the experiments by increasing the number of datasets. Lastly, particular attention will be given to scalability and computational complexity, critical aspects of the proposed model’s applicability in real-world contexts.
Giovanna Castellano and Gianluca Zaza acknowledge funding support from the FAIR - Future AI Research (PE00000013) project, Spoke 6 - Symbiotic AI (CUP H97G22000210007), under the NRRP MUR program funded by NextGenerationEU. The research objectives of this paper are in partial fulfilment of the project EXPLICIT (CUP H93C23000890005). A Ph.D. fellowship funds Rafaele Scaringi’s research within the Italian “D.M. n. 352, April 9, 2022” – under the NRRP, Mission 4, Component 2, Investment 3.3 – Ph.D. project “Automatic analysis of artistic heritage via Artificial Intelligence”, co-supported by Exprivia S.p.A. (CUP H91I22000410007). All authors are members of the INdAM GNCS research group. [17] J.-S. Jang, C.-T. Sun, Neuro-fuzzy modeling and control, Proceedings of the IEEE 83 (1995) 378–406. [18] J. Bergstra, D. Yamins, D. D. Cox, Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures, in: International Conference on Machine Learning, 2013.