A Study on the Faithfulness of Feature Attribution Explanations in Pruned Vision-Based Multi-Task LearningXenia Demetrioux.demetriou3@gmail.com0Sophie Sananikonesophie.sananikone12@gmail.com0Vojislav Tobias Westmorelandvojo.westmoreland@gmail.com0Matthia Sabatellim.sabatelli@rug.nl0Marco Zullichmarco.zullich@gmail.com0Department of AI, Faculty of Science and Engineering, University of Groningen
,
Nijenborg 9, 9747 AG, Groningen
,
the Netherlands
As Artificial Neural Networks (ANNs) continue growing in size to enhance predictive accuracy, model compression techniques, such as pruning, counteract the increase in energy and computational costs. Multi-Task Learning (MTL) consists in training a single model on multiple tasks, providing regularization and increased generalization, especially in applications like robotics and autonomous driving. While compressed models often match the accuracy of their larger counterparts, they are usually evaluated on metrics related to task completion or eficiency, overlooking critical aspects such as fairness, and transparency-key requirements for high-stakes applications. Specifically, Explainable AI ofers tools for making model predictions more transparent, for instance, by means of feature importance. However, especially when AI models are complex, these tools can lead to unfaithful or unreliable explanations, potentially undermining trust in these models. In the present work, we propose to investigate whether pruning applied to vision-based MTL significantly afects the faithfulness of the explanations generated for the tasks the models are trained on. We train and prune diferent models on the benchmark datasets NYUv2 and CityScapes. Despite the hurdle of generating feature importance for tasks such as surface normal prediction and depth estimation, our results show that unstructured pruning maintains faithfulness across diferent sparsity percentages. Structured pruning with milder sparsity percentages preserves faithfulness, but can decrease more rapidly at higher sparsity percentages. Overall, sparsity up to a certain threshold (90%) does not compromise explanation faithfulness. Beyond this point, both faithfulness and performance decline significantly, making them unfit to be deployed, rendering the faithfulness risk negligible.
The deployment of artificial vision systems through Artificial Intelligence (AI) has transformed various
domains such as autonomous driving and medical imaging, resulting in remarkable performance
increases over the past decade. In Computer Vision (CV) tasks, Convolutional Neural Networks (CNNs)
have emerged as one of the cornerstones of this technology. However, they are characterized by high
computational costs and a lack of interpretability due to their complexity. Their computational footprint
can however be tackled by Model Compression (MC) techniques. One of the main techniques for MC
is pruning [
1
], which aims at removing (groups of) synapses in an Artificial Neural Network (ANN)
according to specific criteria. Another viable technique consists in training models in a Multi-Task
Learning (MTL) setting [
2
], where one ANN can generate predictions for multiple tasks given the same
input, instead of having task-specific ANNs, each responsible for generating one prediction per task.
MTL is useful in areas such as robotics and autonomous driving, where a machine learning model
may be tasked to perform multiple tasks at the same time (e.g., Depth Estimation—DE—and semantic
segmentation) in a resource-constrained environment. It can additionally be considered a MC tool,
since MTL models are typically less computationally demanding than using separate models for each
individual task [
3, 4
].
Explainable AI (XAI) is crucial for understanding complex model predictions, particularly in
applications requiring transparency [
5
]. Feature attribution, a key XAI tool, assigns “saliency scores” to
input features, highlighting their importance. The main model-agnostic methods are LIME [
6
], which
relies on surrogate models, and SHAP [
7
], which has solid game theoretical foundations. Additionally,
ANN-specific tools, such as Layerwise Relevance Propagation (LRP) [
8
], which tracks down components
of an ANN responsible for a given activation, have been developed. In ANNs, feature attribution can be
performed by backpropagating gradients of the output on the input features, like in the case of Guided
Backpropagation [
9
]. However, these gradient-based methods have often been found to be unreliable,
often acting similarly to edge detectors [
10
]. Grad-CAM [
11
], a feature attribution tool specific for CNNs,
combines gradients and activations, being more class-discriminative and faithful to the underlying
model than other gradient-based tools. If compared to LIME, SHAP, and LRP, gradient-based methods
are often much more eficient [
12
], which is a reason why Grad-CAM is often the de facto choice for
feature attributions in CNNs [
13
]. The present work utilizes and expands Grad-CAM to explain DE
and Surface Normal (SN) prediction tasks. One of the paramount issues is that all these methods are
approximations of the complex decision-making process of ANNs, and can hence produce unfaithful
outputs, which highlight irrelevant features [
14
].
Analyzing the efect of pruning on the quality of explanations has been researched previously:
Abbasi-Asl and Yu [
15
] studied the features learned by CNNs on image classification, noticing that
redundancy in CNN filters coalesced when applying filter pruning. Weber et al. [
16
] provided a
humangrounded evaluation of saliency maps generated on CNNs for image classification. Their findings
suggest that explanations produced by moderately pruned models are evaluated as superior by humans
with respect to their dense counterparts. Finally, Khakzar et al. [
17
] introduced a custom pruning
method, termed PruneGrad, shown to quantitatively produce more faithful explanations; however, this
is an input-specific method, thus reducing the real world practicality and scalability.
In the present work, we aim at answering the question: “How is the faithfulness of multi-task
model explanations afected by pruning? ”. We train three diferent CNNs for MTL on the NYUv2
and CityScapes datasets. We then proceed to apply diferent pruning methods, to obtain several pruned
models at diferent sparsity levels. Subsequently, we generate task-level saliency maps using variations
of the popular image-based XAI tool Grad-CAM [
11
]. Finally, we proceed to evaluate the faithfulness of
these maps with respect to the models using the Iterative Removal of Features (IROF) technique [
18
].
Our findings highlight how sparsity up to a certain threshold (dependent upon the specific pruning
technique) does not compromise the faithfulness of explanation; The main consequence is that, as far
as explainability is concerned, well-performing, moderately sparse models will likely yield similarly
faithful explanations.
Our contributions are twofold: (i) We carry out an analysis on the efect of pruning on the faithfulness
of models trained for vision-based MTL, and (ii) We provide an adaptation of Grad-CAM and its
assessment for monocular DE and SN, a task which Grad-CAM is not originally designed for.
2. Methods
In this section, we provide the materials and methods used in the present work, alongside a summary
of the experimental setup.
2.1. Datasets
We employed two datasets to train and evaluate our models; The NYU-Depth V2 (NYUv2) dataset [
19
],
and the Cityscapes dataset [
20
], both commonly used benchmarks for MTL and scene understanding in
indoor and urban settings. NYUv2 contains annotations for Semantic Segmentation, SN estimation, and
DE, while Cityscapes only provides labels for Semantic Segmentation and DE. We provide an example
(a) Original Image.
Each color represents a the angle components. diferent depths.
diferent class.
of one image and the corresponding annotations for NYUv2 in Figure 1. Semantic Segmentation is a
pixel-level classification task. We assess the quality according to the mean Intersection-over-Union
(mIoU) metric. The SN task is a pixel-level three-dimensional regression task, where surface angles
are described by their , , and coordinates in a 3D space. We evaluate it using the median angle
error over all pixels in an image. DE is a pixel-level regression task where the distance of each pixel
relative to the camera is measured. We assess it using the relative error between the ground truth and
the predicted depth.
2.2. Model Architectures
We made use of two CCN-based model architectures specifically crafted for MTL: DeepLab [
21
] and
SegNetMTAN [
22, 23
]. Both techniques can be used in combination weith the FAMO weighting method
[
24
] to correctly balance the contribution of the various tasks.
2.3. Pruning
Pruning compresses ANNs by removing synapses based on saliency. Unstructured pruning removes
individual connections, while structured pruning removes groups (neurons, filters, etc.). Structured
pruning ofers hardware-agnostic speed-ups via tensor dimension reduction, whereas unstructured
pruning requires specialized hardware. Pruning can be dynamic (allowing synapse regrowth) or static
(no regrowth). This work employs DiSparse [
3
] (unstructured, gradient-based MTL), Network Slimming
(NetSlim) [
25
] (structured, channel-level L1-norm), and HRank [
26
] (structured, filter-rank-based).
DiSparse avoids task-conflicting pruning, NetSlim uses L1-norm for channel importance, while HRank
prunes low-rank filters.
2.4. Explanations
For an RGB image , we generate a saliency map indicating pixel importance. Grad-CAM, using
CNN layer activations and output gradients, produces a matrix ′, normalized and rescaled to the
dimensions of , so each pixel is assigned a saliency from 0 to 1. Following Selvaraju et al. [
11
], we apply
it to the last convolutional layer. For semantic segmentation, we use SegGrad-CAM [
27
], explaining
per-class pixel groups. An example of the explanations generated using SegGrad-CAM is present in
Figure 2. In addition, we operate the following methodology to adapt Grad-CAM to DE and SN. We
group predictions according to their values into specific categories: for DE, we divide the [
0 − 1
] range
of depth prediction into quintiles and compute the gradients according to each of these groups; in
this way, we aim at explaining the feature importance in determining a given quintile of the depth
predictions. Similarly, for SNs, we group the predictions in the eight octants of the 3d plane and generate
explanations accordingly. Despite the problems of DE and SN estimation being pixel-level regression
tasks, the proposed methodology restructures the output into a classification problem, thus allowing us
to use Grad-CAM.
(a) Wall
(b) Floor
(c) Table
(d) Ceiling
representing feature importance, is shown on the scale at the bottom.
2.5. Assessing XAI
As highlighted by Nauta et al. [
28
], faithfulness is paramount in XAI evaluation. While other explanation
properties exist, unfaithful explanations highlight irrelevant features, potentially causing loss of trust
in AI systems by stakeholders. The assessment of faithfulness boils down to determining whether the
features marked as salient are efectively important for the model. One way of evaluating this is by
progressively masking out features according to their importance and measuring the change in the
model prediction. The expectation is that highly important features should cause a large change in
the prediction. This is the rationale behind IROF: after the process of iterative masking, the change in
prediction is plotted against the proportion of features removed, and faithfulness is computed as the
area-over-the-curve (AOC). A detailed visualization of the IROF procedure is shown in Figure 3.
Formally, every input is segmented in superpixels using SLIC [
29
]. Each segment is given a
saliency score according to the mean importance provided by Grad-CAM. Next, we compute the output
of the model on the perturbed input ′ )︁ , where is the number of segments removed, and divide
︁(
it by the output on the unperturbed input ′0 )︁ . By varying this procedure over all segments
︁(
∈ {1, . . . , } in order of saliency, the aforementioned curve can be plotted and AOC computed as a
faithfulness score. This metric is then averaged over a whole set of datapoints to produce the final
estimate:
IROF(, ) =
=1
1 ∑︁ AOC ⎜
⎛ (︁ ′ ⎟
)︁
⎝ (︀ ′0 ⎠
︀)
⎞
=0
.
In the present study, we used the IROF implementation from the Quantus library by Hedström et al.
[
30
]. Since IROF requires scalar outputs to be compared, for our three tasks, we adapted the procedure
as follows: (a) for Semantic Segmentation, we considered the mean of the predicted logits for each
specific class of interest; (b) for DE, we considered the relative error between unperturbed and perturbed
outputs, while (c) for SN, we considered the cosine similarity between the unperturbed and perturbed
outputs.
2.6. Experimental Setup
The present study investigates the efect of two independent variables (sparsity and pruning method)
on two dependent variables (model performance and IROF Scores). We tested this on three model
architectures to improve the robustness of the conclusions. Diferent models addressed diferent
combinations of tasks. We applied a diferent pruning method for each model architecture. Table 1 shows an
overview of the model architectures used, with their respective pruning method used, sparsity
percentages, tasks, and datasets. The specific optimizers, training hyperparameters, and data augmentation
strategy are instead presented in Appendix A. We point out that the aim of our experiments, as far as
[
21, 22, 23
], not to beat the state-of-the-art on the specific tasks. The goal of the present paper is indeed
the faithfulness of the explanations.
3. Results
Results, as shown in Figure 4 following the setup in Table 1, reveal diverse trends across tasks and
pruning methods. For DE, Relative Error rises exponentially with sparsity for both model/dataset
combinations. IROF scores remain stable until 75% sparsity; HRank maintains better IROF at higher
sparsity compared to NetSlim. For SN, DiSparse (DeepLab) achieves significantly higher performance
and IROF than HRank (SegNetMTAN), even with substantial pruning due to DiSparse’s unstructured
nature allowing for aggressive pruning. IROF scores show a similar diference between DiSparse and
HRank, with sparsity minimally afecting the score until 90%. In Semantic Segmentation, NetSlim
(SegNetMTAN) achieves stable, high performance (around 70% mIoU), with a drop at 90% sparsity.
HRank (SegNetMTAN) and DiSparse (DeepLab) perform worse (35-45% mIoU). IROF scores remain
largely stable across pruning rates for all methods, with declines at high sparsity (∼ 90%) for structured
pruning.
4. Discussion and Conclusions
In the present work, we investigated the ties between pruning and faithfulness of the explanations in
the case of MTL. We trained three diferent model architectures on the datasets CityScapes and NYUv2.
We then pruned these models using iterative structured and unstructured pruning techniques, reaching
sparsity levels above 90%. We then proceeded to generate per-task input attribution explanations
using variations of GradCAM. Finally, we assessed the faithfulness of these explanations, faithfulness
being the degree to which the explanations capture the complex predictive process of the underlying
black-box model. We evaluated the faithfulness using the popular Iterative Removal of Features
(IROF) metric, which proceeds to iteratively delete progressively more important (as measured by the
explanation) information from the input, expecting a variation in the model’s output in the process.
The results indicate how the DiSparse unstructured pruning technique seems not to particularly afect
the faithfulness of the explanations; however, the structured pruning techniques generally show a
decrement in faithfulness as the pruning rate increases—a trend which seems to be loosely connected to
the decrease in performance—which comes with excessive pruning—measured across the various tasks.
Our findings highlight several positive impacts. Firstly, it encourages the implementation of DNNs
with unstructured pruning in embedded systems and resource-constrained environments without
compromising explanation faithfulness. In turn, this also helps foster user trust and regulatory compliance
in such systems. We suggest positive findings in terms of AI Democratization through the decreased
computational demands that come as a result of compression, making these models more accessible to
a broader audience. For structured pruning, the results show that sparsity up to a certain threshold
maintains explanation faithfulness. Beyond this point, both faithfulness and performance decline
significantly, making such highly pruned models unlikely to be deployed. Thus, the overall risk from
reduced faithfulness is minimal.
However, our work comes with several limitations: first of all, it is strictly tied to vision-based MTL.
In addition, we assess faithfulness only according to IROF, while other methodologies exist for the
evaluation. Moreover, our analysis lacks considerations on other facets of evaluation of explanations,
such as robustness and coherence, which still should heavily be subject to considerations on faithfulness.
Our analysis is also strictly tied to functional assessment, while several XAI studies operate
humangrounded evaluation as a step for validating the explanations. These limitations also guide our next
steps in extending the project: (i) extend the study to non-vision tasks, (ii) include other metrics for
faithfulness and other facets of explanations, (iii) and carry out a human-grounded evaluation step.
Considering the paramount importance of having faithful feature importance explanations, we still
believe our findings to be of use for practitioners willing to implement MC on MTL.
5. Acknowledgements
We thank the Center for Information Technology of the University of Groningen for their support and
for providing access to the Hábrók high performance computing cluster.
Declaration on Generative AI
The author has not employed any Generative AI tools.
A. Training hyperparameters
We report here the optimizer and hyperparameters we used for training our models. The settings are as
similar as possible to the original implementations found in the source references [
21, 22, 23
].