It may be possible to extract knowledge from functional magnetic resonance (fMRI) data with artificial neural networks (ANNs) and explainable artificial intelligence (xAI). However, modeling fMRI data with ANNs has its hurdles. One is the unbalance between inputs (one typical volume encompasses hundreds of thousands of voxels) and training epochs (usually hundreds), turning the training stage intractable. In addition, fMRI data is noisy and highly correlated, both spatially and temporally. Such characteristics tend to hamper current deep learning techniques and, therefore, limit fMRI data modeling and their explanation. The research here reported relies on a process encompassing data splitting by training and testing, dimensionality reduction, feature extraction, ANN structuring, and its training and testing. After the procedure, two explaining methods are put side by side, path-weights and layer-wise relevance propagation (LRP). The two methods achieve similar results, i.e., identify the same inputs responsible for the ANN's correct predictions. Therefore, they support each other. An additional validation comes from neuroscientific established knowledge, which sanctions the results of the two explaining methods. A publicly accessible database, Human Connectome Project (HCP) - Young Adults, precisely the motor paradigm, is used to apply the procedure. In conclusion, the combined use of XAI techniques with ANNs modeling permits the extraction of knowledge from fMRI data, at least concerning motor tasks. The complete procedure is an improvement over traditional data analysis methods, which are correlational. The following steps will extend the procedure to cognitive tasks. Artificial neural networks (ANN), Explainable artificial intelligence (XAI), Layer-wise relevance propagation (LRP), Functional magnetic resonance imaging (fMRI)1 Late-breaking work, Demos and Doctoral Consortium, colocated with The 1st World Conference on eXplainable Artificial Intelligence: July 26-28, 2023, Lisbon, Portugal ∗ Corresponding author. up201908014@edu.fe.up.pt (J. D. Marques dos Santos) jpsantos@umaia.pt (J. P. Marques dos Santos) 0000-0003-4101-2748 (J. D. Marques dos Santos)
Although functional magnetic resonance imaging (fMRI) data analysis with artificial neural
networks (ANNs) is more than one decade old [
Addressing the explainability of ANN-built models of fMRI data has been recently tackled
[1315]. One computational model for such purpose is layer-wise relevance propagation (LRP) [16,
17], which was already applied in ANNs [
The fMRI data processing stages are represented in Figure 1: firstly, the raw data processing; next, the two datasets, train and test, are used to build the model (ANN); and finally, the model is explained with path-weights and LRP.
2.1. Raw Data Processing and Model Training and Testing
The first two stages were already described and discussed elsewhere [
The ANN has one hidden layer composed of 10 hidden nodes. Inputs are 46, and the outputs are five, each corresponding to a task (LF, LH, RF, RH, and T).
2.2. Explaining with Path-Weights and Layer-Wise Relevance Propagation
The path-weightijk is defined as the module of the product of all connection weights in a path
defined from the input Ii to the output Ok, passing by the hidden node Hj [
× = ∑ ∑ , where
is the weight between the input node Ii and the hidden node Hj, and weight between the hidden node Hj and the output node Ok. according to the basic rule (LRP-0) [17]:
LRP is a state-of-the-art method for explaining ANN’s predictions [16]. It is calculated where Rj is the relevance of node j, aj is the activation of node j, wjk is the weight of the connection between nodes j and k, and Rk is the relevance of node k.
LRP computation is implemented using R’s library innsight [23], version 0.2.0. 2.3. Grand-Weight (GW) and Grand-Relevance (GR) Computation There is a need for a metric that allows for the direct comparison of individual inputs between themselves for a given stimulus. In this way, the metric Grand-weight (GR) is the sum of the absolute values of the path-weights from each input to a specific output, according to the formula: where GWik is the Grand-weight from input i to output k, and path-weightijk is the path-weight that goes from input i to output k through hidden node j.
For the LRP-based analysis, to keep consistency with the path-weights-based analysis, the absolute values of the relevancy score for each input for a given output are summed, obtaining the metric Grand-relevance (GR), which allows the direct comparison between inputs regarding their relevance. GR formula is: = ∑ | | where GRik is the Grand-relevance from input i to output k, and Rikl is the relevance score for input i for output k for computational epoch l.
(1) is the (2) (3) (4)
Confusion matrix of the ANN predictions, including the partial and global accuracies and precisions (LF: left foot; LH: left hand; RF: right foot; RH: right hand; T: tongue).
t u p n I
LF LH RF RH T Accuracy (%) 67.5 Precision (%) 64.3 LF 27 3 8 0 4 42
LH 36 1 0 2 0 39 90.0 92.3
RF 31 6 0 1 1 39 77.5 79.5
RH 5 1 1 0 37 44 92.5 84.1
T 1 0 0 0 35 36 87.5 97.2
Total 40 40 40 40 40 200 83.0 although the magnitudes are not constant across all stimuli.
Overall, both methods yield the same inputs as the most important. So, the path-weights-based analysis is congruent with the state-of-the-art method, LRP. Hence, it is possible to conclude that the path-weight-based procedure explains the ANN model in coherence with the layer-wise relevance propagation-based method.
This work was partially financially supported by Base Funding - UIDB/00027/2020 of the Artificial Intelligence and Computer Science Laboratory – LIACC - funded by national funds through the FCT/MCTES (PIDDAC).