The susceptibility of deep learning models to adversarial examples raises serious concerns over their application in safety-critical contexts. In particular, the level of understanding of the underlying decision processes often lies far below what can reasonably be accepted for standard safety assurance. In this work, we provide insights into the high-level representations learned by neural network models. We specifically investigate how the distribution of features in their latent space changes in the presence of distortions. To achieve this, we first abstract a given neural network model into a Bayesian Network, where each random variable represents the value of a hidden feature. We then estimate the importance of each feature by analysing the sensitivity of the abstraction to targeted perturbations. An importance value indicates the role of the corresponding feature in underlying decision process. Our empirical results suggest that obtained feature importance measures provide valuable insights for validating and explaining neural network decisions.
When neural networks are used in critical applications, the
reliability of their decision making becomes a major concern.
Various techniques have been developed to verify, falsify, enhance, or
explain the neural networks, see
Deep neuronal networks (DNNs) learn their decision rule
through training on a large dataset by gradually optimising
parameters until they achieve the required accuracy. Therefore, they do not
have a specific control-flow structure, which makes it difficult to
precisely define suitable test criteria. Neuron activation
This is a contribution to a recent trend to exploring the internal
logic of the learning model, such as eXplainable Artificial
IntelliCopyright © 2022, for this paper by its authors. Use permitted
under Creative Commons License Attribution 4.0 International (CC
BY 4.0).
gence (XAI)
We build on the work of Berthier et al. (2021), which elicited semantic assumptions by advancing an approach that relies on a Bayesian Network (BN) abstraction to examine whether latent features are adequately exercised by a set of inputs. The Bayesian view of statistics treats the latent parameters as random variables and seeks to learn a distribution of these parameters conditional on what is observed in the training data.
Contribution. Our key contribution is to propose a method that estimates the importance of a neural network’s latent features by analysing an associated Bayesian network’s sensitivity to distributional shifts. This allows us to define semantic testing metrics and to identify distributional shifts in the feature space through the effect they have on the random variables in BNs.
This provides us with a separation of concern: we can study the effect of a distributional shift in the latent feature space, which is typically low-dimensional, independent of potential shifts in the input distributions. This provides insight on the semantic mechanisms of decision making in the DNN as well as information for testing the sensitivity of features to distributional shifts.
A weighted scoring model is commonly applied in statistics when certain selected criteria are assigned more importance than others. The feature importance (FI) describes how much each feature influences the classifier’s decision, and thus indicates the importance of the feature for the classification. This is to be contracted with the existing notion of feature importance in explanation models, which assigns the importance value to the features that belong to the input space, e.g., age, sex, education. Instead, we investigate the learning models’ latent feature space and examine how much their deep representation relies on a specific hidden feature to change their prediction.
We seek to evaluate the learning models’ semantic robustness by developing a weight-based test metric that utilises the Bayesian Network model from Berthier et al. (2021). However, instead of directly using the extracted hidden features to measure some test coverage metric, we first compute a weight value, wi, for the i-th latent feature, by analysing the BN’s sensitivity to a controlled noise applied to this feature. In this paper, we develop several analyses that rely on the BN abstraction to estimate the relative impacts and sensitivity of the latent features. For example, we measure the relative impact one feature has on another for all feature pairs by estimating how a controlled noise impacts the BN’s probability distributions. As an alternative approach, we also estimate the sensitivity to a given latent feature by comparing the probability distributions of training samples before and after the feature has been perturbed. Figure 1 outlines the proposed feature sensitivity analysis approach.
This allows us to monitor the behaviour of a DNN via its associated BN. The structure of this BN is built based on a parameterisable abstraction scheme that defines a series of DNN layers to consider (conv2d, dense and dense 1 in the Figure), a feature extraction technique to identify a given number of latent features for each one of these layers (2 in the example), and a discretisation strategy that determines the granularity at which values of latent features are aggregated into distinct intervals of indistinguishable values. Combined with the feed-forward nature of the DNNs we consider, this scheme allows us to derive the structure of a BN, as shown in Figure 1.
In addition, this scheme provides us with a discretisation function Discr], that transforms a set of inputs X into a lowdimensional, discretised version FX . In the Figure, the vector of inputs X is transformed into FX , which associates each input x 2 X with six feature intervals, one for each latent feature represented by the BN. As the BN assigns a probability to an input sample that belongs to the distribution it represents, we can compare the probabilities of a sample under a given BN before and after a perturbation. To do so, we conduct the interior analysis on FX by calculating the probability of each sample under the BN B. After that, we iterate over all considered latent feature f , and shift the associated intervals in FX to produce a modified F X0f w.r.t. the feature f , and calculate its probability belonging to the BN B distribution. The term intervals shifting refers to a technique used to artificially simulate a controlled distribution shift by randomly shifting intervals in the selected feature space. To identify the impact of a perturbation, we compute a distance between the original probability vector and the probability vector obtained from the perturbed features. 2
Robustness of Neural Networks latent features. Features
play an essential role in the field of image processing and
classification. They are considered as the basic conceptual components
of the semantics of an image. Similar as this paper, there are
recent studies concentrated on the features space to study the hidden
semantic representation of intelligent models.
Bayesian Networks (BNs) and Neural Networks (NNs).
Current trends towards using Bayesian modelling to solve
challenging issues of neural networks have seen a growing
recognition of the vital links between them.
However, we cannot directly apply the sensitivity analysis in the traditional sense with Bayesian Networks, where the sensitivity is performed by changing the BN parameter from the input space and observing how that influences the final decision. Instead, since our analysis targets the latent features in the low-dimensional space, we measure how sensitive are the BN probability distributions to changes in the values of hidden features. This process gives insight into how such perturbations impact the inner Bayesian network distribution and hence reflects the ground truth of the neural networks’ behaviour in the presence of adversarial inputs.
Throughout this work, we use the Bayesian network probability distributions to study the neural networks latent features and analyse their deep representations.
3
At the core of our approach lies the proposed BN-based latent feature analysis algorithms. Before that, we introduce the scheme and Bayesian Network that have been used by Berthier et al. (2021) as an explainable abstraction of DNNs’ latent features.
Let N be a trained deep neural network with sequential layers L = (l1; : : : ; lK ) and X a training dataset. As an abstract model of N and X, a Bayesian Network (BN) is a directed acyclic graph B = (V; E; P ), where V are nodes, E are edges that indicate dependencies between features in successive layers, and P maps each node in V to a probability table representing the conditional probability of the current feature over its parent features w.r.t. X. Example 1 Figure 2 gives a simple neural network of 2 hidden layers and its Bayesian Network abstraction. hi is a function that gives the neuron activations at layer li from any given input sample, and i;j is a feature mapping from the set i = f i;j gj2f1;:::;j ijg. Each random variable i;j hi in the BN represents the j-th component of the value obtained after mapping hi into the latent feature space. Since each function i;j hi ranges over a continuous space, the respective feature components—which are the codomains of the i;j ’s—are discretised into a finite set of feature intervals.
Each node in BN abstractions represents an extracted feature, and we let Fi;j = ffi];1j ; : : : ; fi];mjg, for the j-th extracted feature ] from layer li, be a finite set of m intervals that partition the value range of the feature. We formally define a feature as a pair (i; j), where i indexes a layer li in L, and j identifies a component of the extracted feature space for layer li, i.e., j 2 f1; : : : ; j ijg. Each node in the BN model is associated with either a marginal probability table for hidden features of layer l1, or a conditional probability table for hidden or output layers. In Figure 2, the conditional probability table for the feature component 3;1 is defined for each feature interval f( 1; 3[; [3; +1)g for layer l3, w.r.t. each combination of the parent feature intervals from previous layer l2. Example 2 Figure 3 illustrates an example node in a BN, which corresponds to the second extracted feature from the first NN layer, i.e., i = 1; j = 2. The set F1;2 contains two intervals, f1];12 and ] f1];22, which partition the real line. The node is denoted as a random variable named 1;2 h1, which is associated with a probability table. The probability table is a marginal probability table because the features on the first layer do not have parent features. The table says that this feature has probability 0.7 to have a value smaller than 2 and probability 0.3 to have a value no less than 2.
The previous demonstrating examples were simple, which illustrated a BN built using each layer of the NN. In practice, we can select specific NN layers to be abstracted, as we did in our analysis experiments.
Using Bayesian Network Abstractions. We can fit the set of probability tables in a BN abstraction B by using a training sample X. This process first transforms X by means of the discretisation function to obtain a vector of elements from the discretised latent feature space FX = Discr](X). It then updates the probability tables in B in such a way that the joint probability distribution it represents fits the distribution of FX .
We can query the fitted BN B for the probabilities of the discretised input sample FX0 . We denote this query operation Pr(FX0 2 B), and may abuse this notation by defining Pr(X0 2 B) = Pr Discr](X0) 2 B .
Perturbation of Latent Features. Our developments in the next Section rely on the application of a controlled change of a feature (i; j) in an element Fx of the latent feature space. This operation simulates a distortion in single targeted component of the latent feature space by substituting its associated interval with an adjacent one. (We assume that each latent feature component is partitioned into at least two intervals). When an interval has two neighbours, we chose uniformly at random between them. We denote this operation with the function random shift(Fx; i; j), which replaces the feature interval fi];kj of Fx with either fi];kj 1 or fi];kj+1. For instance, assuming two hidden feature components extracted from activations at two layers of a NN, each component being discretised into small-enough intervals, i.e., 10 intervals, random shift((f0];40; f0];71; f1];10; f1];91); 1; 0) returns either (f0];40; f0];71; f1];00; f1];91) or (f0];40; f0];71; f1];20; f1];91).
4
BN-based Latent Feature Analysis In this section, we develop several BN-based analysis approaches we employ to gain insights on latent features. The first approach produces a pairwise comparison matrix that exhibits the relative impact the latent features have on each other. Next, we leverage the BN to estimate the sensitivity of individual features to a controlled distribution shift. We then describe how the sensitivity analysis technique can be applied to define feature importance based on a generic definition of weights on features. Finally, we formalise a concrete definition of weights based on our BN-based feature sensitivity. 4.1
This particular study is to assess the degree to which the extracted features can affect each other by comparing the parallelised Conditional Probability Tables (CPTs) of a sample, under a BN, with the CPTs of the same sample after perturbing the features intervals of the BN. The pairwise comparison method is used to make a recursive comparison. It begins by extracting a set of inputs X from training data, and computing its feature intervals with Discr]. This produces a sample FX of intervals w.r.t. X. To generate the probability tables, we fit the Bayesian network with FX , which gives the clean reference probability tables CPTs(FX ). Figure 4-(a) shows the CPT for feature (3; 0), which is the first extracted feature from the third NN layer, named dense 1 in the BN from Figure 1.
To extract knowledge about a given feature’s independence and robustness, we apply a controlled change to the targeted feature f , by using the random shift operation to shift f ’s intervals in FX to obtain F X0 . We then re-fit the BN’s probabilities with F X0 , which gives the modified probability tables CPTs(F X0 ) w.r.t. the perturbed feature f , examplified in Figure 4-(b). To identify the impact, we use the mean squared error (MSE) between each corresponding table in the reference CPTs(FX ) and generated CPTs(F X0 ).
We illustrate and give an example of pairwise comparison in Section 5 below. 4.2
The core benefit of relying on a Bayesian Network is to have a model that exhibits the relevant theoretical aspects of Bayesian analysis. To estimate the sensitivity of the abstraction scheme on a given latent feature, we measure the impact of artificially perturbing the intervals representing the selected feature on the probability distribution represented by the BN. In this algorithm, the BN is already fitted using a training dataset, and the distribution it represents does not change.
Algorithm 1: BN-based Feature Sensitivity Analysis Input: Bayesian network B and associated feature mapping & discretisation function Discr], training dataset X , distance metric dp.
Output: Mapping associating a distance measure with each considered latent feature 1: Compute the feature intervals w.r.t. X :
FX = Discr](X ) 2: Compute the reference probabilities of FX w.r.t. B:
Pref = Pr(FX 2 B) 3: for each considered feature f = (i; j) do 4:
Pf0 = hPr(random shift(Fx; i; j) 2 B)iFx2FX 5: df = dp Pref ; P 0
f 6: end for 7: return distances df , for all f
The feature sensitivity analysis is given in Algorithm 1. This procedure receives an input sample X, taken from the training dataset, and first performs the feature projection and discretisation step with Discr] to obtain the associated feature intervals FX . It then calculates the probability of each element of FX w.r.t. the BN B; this gives the vector of reference probabilities Pref , that associates a probability with each set of abstracted latent features that are elicited by each x in X. Then, for each extracted feature f , a random perturbation is performed in FX via the random shift function introduced in the previous Section. This leads to a second vector, that holds the probabilities of the resulting F X0f w.r.t. the BN B. The given distance dp between these two probability vectors for the perturbed feature f is eventually computed.
We chose to make the feature sensitivity analysis algorithm parametric in the distance metric p for the purposes of easing further experimental use of the FI measure. The considered distances are: • Lp’s with different norms, typically 1, 2, or 1; • JS is the Jensen-Shannon distance, that is a metric that measures the similarity between two probability distributions based on entropy computations; • corr is the correlation distance; • cos is the cosine distance; • MSE is the mean squared error; • RMSE is the root mean squared error; • MAE is the mean absolute error; • AF is a special purpose anti-fit divergence, which we define based on the coefficient of determination R2. R2 is a score that is typically used as a “goodness-of-fit” measure for regression models, and we refer to it as scoreR2 . While the maximal score is 1 (indicating a perfect fit), the score decreases with the amount of variance in P that is not in Q and can take negative values. With this we define dAF(P; Q) = 1 scoreR2(P; Q). The rationale of using scoreR2 as a basis for measuring the divergence is that we can view the probability vectors for perturbed features as output by a model. Divergence will be large when the effect of the perturbation is significant, and small when the model is not (very) sensitive to the perturbation. where T is the set of considered latent features. The soft-max weighting in Eq. (1) acts as a normalisation function, i.e., it ensures the sum of the feature components’ weights equals one. The normalised importance weight for each feature is usually positively correlated with the respective probabilities distances.
Example 3 Table 1 shows selected distance measures computed based on one experiment detailed in the next Section. Assuming the dcorr distance is chosen to determine feature importance, feature (1; 1) is assigned the largest weight at 0:192, followed by feature (2; 1) at 0:182, etc.
The importance weight for an extracted latent feature of a DNN’s layer may reflect some relevant amount of information/variance/ that the abstracted DNN uses at the considered layer. The current abstraction scheme, however, does not relate latent features with the DNNs’ decisions. Still, perturbing a specific part of the latent space and observing the implicit changes of the learning models’ distribution contributes to understanding their internal decisions. 5
In this Section, we first illustrate the results of pairwise comparison of latent features, and then turn to an empirical evaluation of the sensitivity of BN abstractions at detecting distribution shift induced by adversarial examples. 5.1
Illustration of Pairwise Comparison We first concentrate on the BN given in examples so far.
We report in Table 2 a pairwise comparison matrix for this example, where we arrange the perturbed features in the first column and compute their impact on each feature (i; j)’s probability tables. The numbers reported in this matrix represent the change in the probability values. For instance, our controlled perturbation of feature (2; 0) intervals has an impact on features (3; 0) and (3; 1) values. More specifically, the MSE between the (3; 0) probability tables for feature (3; 0), given in Figure 4 (a) before and (b) after perturbing feature (2; 0), is 0:0113.
Discussion. Suppose we set the diagonal line to zeros since the change is made from the feature itself. In that case, we can observe that the perturbations are not affecting the probability of features from the previous layer (parent features) or the same layer as expected. On the other hand, random shifting only influenced the immediate features in the next layer. The largest difference occurred on feature (3; 1) when perturbing feature (2; 1). Although this impact is relatively small, we can (as expected) observe the dependencies between latent feature values of the BN model. However, the perturbations do not change the features’ probability for deeper layers, e.g., features of Layer 3 are not affected by the perturbation made on features of Layer 1, which is surprising. 5.2
Let us now turn to our empirical assessment of the effectiveness of the BN sensitivity analysis method in examining the behaviour of the latent features under perturbation.
Datasets and Experimental Setup. We have selected two trained CNN models for our experiments: the first one targets the MNIST classification problem with 99.38% validation accuracy, and the second model targets the CIFAR-10 dataset with 81.00% validation accuracy. The models are reasonably sized, with more than 15 layers including blocks of convolutional and max-pooling layers, followed by a series of dense layers. They have 312 000 and 890 000 trainable parameters, respectively.
The Bayesian Network abstraction scheme accepts a wide range of feature extraction techniques and discretisation strategies. To explore their impact on our approach, we use a wide set of BN abstractions. We have selected two linear feature extraction techniques: Principal Component Analysis (PCA) and Independent Component Analysis (ICA), and one non-linear technique: radial basis functions (RBF) kernel-PCA. We also decided to fix the number of extracted features at three features per layer; this choice of a relatively small number of hidden features enables us to use many intervals (5 or 10) for their discretisation while still obtaining reasonably-sized probability tables. We applied both uniformand quantile-based discretisation strategies, with or without the addition of two left- and right-most intervals that do not contain any element of the training sample. Finally, we considered three hidden layers to construct the BN abstractions: for the two models, the first two selected layers directly follow a block of convolutions, while the last is a dense ReLU layer that is situated few layers before the NN’s output layer. The layers chosen criteria is based on a belief that the activation values at these layers capture relevant patterns w.r.t the NN decisions.
Example Distributions and Distances. We plot in Figure 5 example distributions of probabilities in vectors obtained from a BN abstraction of the MNIST model. We have annotated each one of these plots with various measures of distances between the reference probabilities Pref that is generated using a sample from the training data set, and the respective six perturbed features probabilities Pf0 . The shown difference between these two probability distributions illustrates the internal change in the distribution represented by the BN. For instance, when applying the random shift on the first feature that is extracted from the first selected layer i.e., perturbed feature (1,0), the calculated probability perturbed feature distribution P 0(1; 0), coloured with blue, shows a change on probabilistic causal relation that implies the change on the probability represented by the BN. Hence, we can determine the safety violation risk by comparing an input probability belonging to the BN probability distribution.
Sensitivity to Adversarial Distribution Shift. We have carried out a set of experiments to assess whether the set of three features extracted for each considered hidden layer allows us to capture relevant properties of the learnt representations. In particular, we wanted to check whether the BN abstraction allows us to detect the shift in the distribution of inputs that occurs when the NN is subject to adversarial examples. In other words, we want to discover whether some distance measures indicate that the BN abstractions capture relevant latent features (and their dependencies) with sufficient precision to associate diverging probabilities between “legitimate” inputs and adversarially perturbed ones. If such is the case, we shall conclude that our abstraction scheme and the associated BN are sufficiently precise to capture relevant dependencies in latent feature values that may not be matched (or matched too well, depending on the sign of the actual difference in probabilities) by some adversarial inputs.
To carry out these experiments, we have selected the following adversarial attacks: fgsm is the Fast Gradient Sign Method of Goodfellow, Shlens, and
Szegedy (2015); pgdlinf and pgdl2 are the Projected Gradient Descent approach of
Madry et al. (2017) with L1 and L2 norm, respectively;
cwlinf and cwl2 are
Attacks involving the L1 norm target a maximum of " = 0:1 perturbation in the input images, whereas pgdl2 targets a maximum perturbation " = 10.
For each attack, we have generated an adversarial dataset Xattack from the validation dataset Xtest for both the MNIST and CIFAR10 models, where each dataset consists of 10 000 inputs. Then, for each attack and BN abstraction B built and fit using 20 000 elements drawn from the respective training datasets, we measured a set of distances p between the vectors of probabilities Pr(Xtest 2 B) and Pr(Xattack 2 B), denoted dp(Pr(Xtest 2 B) ; Pr(Xattack 2 B)).
Results and Discussion. Figure 6 shows our results for three
selected distances L2, cos, and AF . We give more detailed results
in Appendix A. Each chart in the figure illustrates the calculated
distances with four colours according to the discretisation method
and the number of intervals in the vertical axis, using three sets
of feature extraction techniques (pca, ica, and rbf.kpca) in the
horizontal axis. The used distance metric and attack type are shown
at each chart’s top. First of all, we can observe that some
combinations of abstractions and distance measures exhibit notable
differences between the validation dataset and the adversarial one for
some attacks. For instance, every distance shown allows us to
measure a shift in input distribution for every attack, except
Overall, the experimental results show that computing distances between two BN probability distributions, clean and perturbed by intervals-shift or adversarial attacks, can detect the distribution shift where it exists. We emphasise that, in the case of adversarial shift, this is measured based on the latent features only. Given this empirically confirmed property, BN-based computation of feature importance appears to be one tool, which adds to the growing set of useful techniques for the detection of important features as well as of adversarial examples. What is more, it adds a semantic twist to this analysis and allows for explaining in which way the changes in the features contribute to the distribution shift. 6
In this section, we discuss a few aspects related to either the method we take or the potential application of the method.
Hyper-parameters in BN Construction. The parametric
nature of the scheme advanced by Berthier et al. (2021) enables
the exploration of a wide range of DNN abstractions. For instance,
in our experiments, the sensitivity to adversarial distribution shift
is relied most on the linear dimensionality reduction techniques
to extract latent features. We plan to conduct further experiments
with more non-linear feature extraction techniques, like manifold
learning
Hyper-parameters in Weight Quantification. There are a number of building blocks in the weight quantification method (Algorithm 1), including e.g., the perturbation made to generate new CPTs, the random shifting function, and the distance metrics for probabilities (Pref ) and (Pf0 ). In this paper, we have explored several different options of the distance metrics for a comparison. It would also be useful to study if and how the other hyper-parameters may affect the overall results.
Utility of Feature Weights. Quantifying the importance of the hidden features provides three advantages. First, visualising the most important features provides insight into the model’s internal decisions by highlighting dominating regions in the feature space.
Second, we can use the importance measurement to design highlevel testing metrics that evaluate the robustness of the DNN. Some attempts have been made in Berthier et al. (2021), where no feature weight is taken into consideration.
Third, with FI as a defence, we can utilise the obtained
importance in the training process and force the DNN to adjust its
parameters according to the features that are most relevant for the
prediction. This direction is the most widely adopted strategy. For
example,
Utility of Bayesian Network. As suggested, BN can be seen
as an abstraction of the original DNN. It is therefore imperative
to understand how this abstraction may help in either analysing or
enhancing the original DNN. In Berthier et al. (2021), test metrics
are designed over the BN by extending the MC/DC metrics
proposed by
In addition to testing, it would also be interesting to see if such
abstraction may bring any benefit to e.g., verification
Similar as the above discussion for testing and verification, the
potential for the BN to be used as an intermediate step for the
reliability assessment
7
In this study, we have advanced a novel technique that employs a BN abstraction to investigate how to measure the importance of high level features when they are used by the neural network to make classification decisions. In addition to the observed ability of detecting the distribution shifts before and after perturbation, this will open many doors for future exploration. For example, it will certainly be interesting to understand if the generated importance values can support the explanation of the black-box learning model.
It will also be useful if such importance values can be utilised to improve the training process.
A
Detailed Results for Sensitivity to
Adversarial Shift Experiments We have plotted in Figure 6 some statistics for a subset of the distances we have considered for comparing probability vectors. Figures 7, 8, and 9 show the distances computed for the MNIST model, and Figures 10, 11, and 12 show the results for the CIFAR10 model.
In these plots, hue still indicates the discretisation strategy. However, we have discriminated between extended and non-extended strategies: the prefix ‘-X’ denotes that latent features are discretised in such a way that left- and right-most intervals do not contain any (projected) training sample. ) ) B40 k2 (Xttca30 a r P ; ) B20 tts2 e X r(10 P ( p d discretization uniform5 uniform5-X quantile5 quantile5-X uniform10 uniform10-X quantile10 quantile10-X attack ilf n w c 2 l w c 0:0 ) ) 0:0 )0:7 B 2 k0:6 c a t t a X0:5 ( r 0:0 discretization attack w c 2 l w c attack w c 2 l w c B 5 2 k c a t t a 4 X ( r P 0:008 ) ) B0:007 2 ( r X0:002 P (0:001 p k c a t t a ( r P0:8 ; ) B 0:6 2 0:0 me5asure p = MSE — feat. extr.: pca discretization measure p = RMSE — feat. extr.: rbf kpca attack attack ) ) B 250 2 a200 X ( r P P ( 3:5 B 2 k ta3:0 c t a ( r 0:0 ) )0:07 B 2 k0:06 c a t t a X0:05 ( r discretization attack w c 2 l w c attack w c 2 l w c by selected adversarial attacks (horizontal axes). See Figure 7 for further details. 0:0 1:0 ) ) B 2 ck0:8 a t t a X ( 0:0 )0:8 ) B 20:7 k c a t a0:6 t X ( 0:0 discretization attack w c 2 l w c attack w c 2 l w c attack w c 2 l w c B 0:00050 X ( r ) ) X r 5 ( P ( measure p = MSE — feat. extr.: ica discretization measure p = RMSE — feat. extr.: rbf kpca attack