In the last decade, Deep Neural Networks (DNNs) have witnessed great advances in real-world applications like Autonomous Vehicles (AVs) to perform complex tasks such as object detection and tracking or vehicle control. Despite the progress introduced by DNNs in the previous decade, they still have significant safety shortcomings due to their complexity, opacity and lack of interpretability. Moreover, it is well-known that DNN models behave unpredictably under dataset shift [1]. Deep Learning (DL) models have training and data bias that directly impact model predictions and performance. This impedes ensuring the reliability of the DNN models, which is a precondition for safety-critical systems to ensure compliance with industry safety standards to avoid jeopardizing human lives [2].
As highly automated systems (e.g., autonomous vehicles or autonomous mobile robots) increasingly rely on DNNs to perform safety-critical tasks, different methods have been proposed to represent confidence in the DNN predictions. One way to represent DNN confidence is to capture the uncertainty associated with a prediction for a given input sample. Capturing information about "what the model does not know" is not only useful but essential in safety-critical tasks.
Bayesian Neural Networks (BNNs) and existing Bayesian approximate inference methods (Deep Ensem-bles, Monte-Carlo Dropout, etc.) offer a principled approach to model and quantify uncertainties in DNNs. However, quantifying uncertainty is challenging since we do not have access to ground-truth uncertainty estimates, i.e., we do not have a clear definition of what a good uncertainty estimate is. Moreover, computer vision tasks can add an extra level of complexity since tasks such as semantic segmentation require a pixel-level understanding of an image. In this case, a Bayesian Deep Learning model for semantic segmentation will classify each pixel in the input image and generate an uncertainty estimate for each classified pixel.
In semantic segmentation, uncertainty estimation has been used for Out-of-Distribution (OoD) detection under the assumption that samples that are far away from the training distribution (anomalous or OoD samples) provide higher predictive uncertainty than samples that are observed in the training data [3]. Approaches that use BNNs are able to capture aleatoric and epistemic uncertainties in the form of uncertainty maps (Figure 1-top) but still fail to detect anomalies accurately. BNN methods for semantic segmentation are prone to yield false-positive predictions, as well as miss-matches between anomaly instances and uncertain areas caused by the lack of information on semantic structures and contexts [4,5], as presented in Figure 1-middle. Recently, embedding density estimation methods have been proposed to estimate the connection to uncertainties from Bayesian methods [6,3]. In this direction, methods that leverage metrics or statistics from the nonparametric embedding space density have been proposed recently [7,8], in contrast to a distance-based method that often assumes a parametric embedding density [9,10,11].
The present work combines the benefits from Bayesian methods for uncertainty estimation with methods for la-
Probabilistic U-Net [12], is a DNN architecture for semantic segmentation that combines the U-Net architecture [13] with the conditional variational autoencoder (CVAE) framework [14]. The goal of Probabilistic U-Net is to handle input image ambiguities by leveraging the stochastic nature of the CVAE latent space. Figure 2 shows the Probabilistic U-Net architecture.
During training, depicted in Figure 2a, Probabilistic U-Net finds a useful embedding of the segmentation variants in the latent space by introducing a Posterior Net. This network learns to recognize a segmentation variant and to map it into a noisy position in the latent space (ππππ π‘, π 2 πππ π‘ ). In addition, KL divergence is used to penalize differences between the distributions at the output of prior and posterior nets. The idea here is to bring both distributions as close as possible so that the Prior Net distribution covers the spaces of all presented segmentation variants.
In general, the central component of this architecture is its latent space. Each value from the latent space encodes a segmentation variant. During inference, the Prior Net encodes each input image π₯π and estimates the probability of these segmentation variants (ππππππ, π 2 πππππ ). To predict a set of segmentation outputs, a set of samples are drawn from the Prior Net probability distribution. Interestingly, we can draw a connection from this approach to other related work that aims to model complex aleatoric uncertainty (ambiguity, multi-modality) by handling stochastic input variables [15,16,17].
Despite the benefits introduced by injecting random samples from the latent space into U-Net, aleatoric uncertainty alone is not enough. For the Out-of-Distribution detection task, epistemic uncertainty is needed [18,19].
Although the Prior Net encoder ππππππ employs Bayesian inference to obtain latent vectors z, it does not capture epistemic uncertainty since the encoder lacks a distribution over parameters π. To overcome this problem, we took inspiration from Daxberger and HernΓ‘ndez-Lobato [20], Jesson et al. [21], and propose to capture uncertainty in the Probabilistic U-Net Prior Net encoder using π Monte Carlo Dropout (MCD) samples [22], i.e., ππππππ(π§ | π₯, ππ).
In eq. 1, we adapt the Prior Net encoder to capture the posterior π(z | x, π) using a set Ξ¦ = {ππ} π π of encoder parameters samples ππ βΌ π(π | ππ) that are obtained applying MCD at test-time. During execution time, we forward-pass an input image π₯π multiple times into the ππππππ net. Each time we forward-pass the input image, we will generate a new dropout mask that in consequence will make a new (ππππππ, π 2 πππππ ) prediction. From each predicted (ππππππ, π 2 πππππ ) for the same image we sample a new latent vector z, as presented in Figure 3.
MCD has been applied extensively for simple epistemic uncertainty estimation. However, dropout was found to be ineffective on convolutional neural networks (CNNs). Standard dropout is ineffective in removing semantic information from CNN feature maps because nearby activations contain closely related information. On the other hand, dropping continuous regions in 2D feature maps can help remove semantic information and enforce remaining units to learn features for the assigned task [23]. This effect is also desired for capturing uncertainties, otherwise, we could get overconfident uncertainty estimates in the presence of samples that contain anomalies. To overcome the standard dropout limitation, we followed the approach from Deepshikha et al. [24], and used Drop-Block2D to capture uncertainty from the Probabilistic U-Net. We applied MC DropBlock2D in the last feature map from the Prior Net, as shown in Figure 2 and Figure 3 (in red).
The average surprise or uncertainty of a random variable π§ is defined by its probability distribution π(π§), and it is called the entropy of π§, i.e., H(π§). For continuous random variables, we use the differential entropy, as presented in Eq. 2,
To quantify uncertainty from Prior Net MCD samples, we used standard entropy estimators [25] on 32 Monte Carlo samples (32 image forward passes through Prior Net with MC DropBlock2D turned on). In Eq. 3, the entropy H ΛΞ¦(π§ | π₯) measures the average surprise of observing latent vector π§ at the output of Prior Net, given an input image π₯.
For OoD detection, we assume that we have access to a dataset of normal (InD) and anomaly (OoD) samples π = {normal, anomaly}, with which we can train a Bayesian generative classifier (Not so naive Bayes Classifier) using the empirical density of a metric or statistic π from latent representations z, i.e., π (z). To this end, we follow Morningstar et al. [7] approach and use a Kernel Density Estimation (KDE) method to obtain the π (z) densities. Since we aim at leveraging the uncertainty from intermediate latent representations, the π statistic is the entropy at the output of the Prior Net (described in the previous section) with which we build the monitoring function β³πππ·, as presented in Figure 2b.
For each label set, we fit a KDE to obtain a generative model of the data, i.e., use KDE to compute the likelihood π(π (z) | π¦). Then, we compute the class label prior probability π(π ), i.e., compute the marginal categorical distribution by counting frequencies (from the number of samples of each class in the complete training set). For an unknown latent vector, we can compute the posterior probability of each class π(π¦ | π (z)), using Baye's rule in Eq. 4. For the OoD task, we use Eq. 5
For a more details description of the approach for Bayesian generative classification we refer the reader to the works from VanderPlas [26] and Postels et al. [3].
Dataset Building. For training the DNN model for semantic segmentation we used the Valeo Woodscape dataset1 [27] with the semantic segmentation labels.
For training the monitoring function (i.e., Bayesian generative classifier), our first choice was to use Soiling Woodscape sub-dataset. However, after inspecting the dataset, we noticed that samples were taken in small sequences. To improve dataset diversity and implement our approach, we decided to create a new smaller sub-dataset by taking just one or two samples from the sampling sequences for each anomaly in soiling Woodscape. We called this new dataset OoD Woodscape, and it combines samples from the Woodscape training set (normal class) and samples from the Soiling Woodscape validation set (anomaly class). The ooD-Woodscape training set has 280 samples, 140 samples for each class; the validation set has 120 samples total, 60 samples for each class. The dataset-building procedure is depicted in Figure 4.
Experiments. We quantify the entropy from intermediate latent vectors. Using the entropy values, we estimate the entropy density for each sub-dataset, i.e., samples from normal and anomaly sub-datasets. First, we quantify the entropy assuming a multivariate Gaussian distribution π» Λπ(z | π₯), as presented in Figure 5 top-right. Next, we compute the entropy estimation for each variable in the latent vector π» Λπ(π§π | π₯), as shown in Figure 5-bottom. Finally, for comparison, we also use the Mahalanobis distance which is a multivariate measure of the distance between a point and distribution. In this last case, we built the reference distribution taking intermediate representations z i for each input image π₯π, from the Woodscape validation set (see Figure 5 top-left). Then, we measure the distance to this reference distribution using ππ = βοΈ (z * β πz val ) π Ξ£ β1 z val (z * β πz val ), for a new input image x * and its predicted latent vector z * .
For entropy, in both cases, we observe that the densities for InD and OoD samples are different. In the first case, the estimated latent vector density shows clear multimodality for OoD samples, with peaks in entropy inter- vals that denote under-confident (uncertainty high) and overconfident (uncertainty very low) predictions. In the latter case, the entropy from latent vector variables, we observe that some variables exhibit multimodal density predictions for OoD samples and density peaks in different entropy value intervals from those obtained with InD samples. Finally, the ππ density shows slight peaks or modes for OoD samples, however, densities for InD and OoD have a high degree of overlap.
Metrics. To evaluate our monitoring function, we used the validation set from OoD-Woodscape (the dataset we designed and built). We report the results using the following metrics, as suggested by Ferreira et al. [28] and Blum et al. [6]. In this regard, we report the Matthews correlation coefficient (MCC), the F1-score, the area under the Receiver Operating Characteristic (AUROC), and the False-Positive Rate at 90% True Positive Rate (FPR90) values. Table 1 summarizes the results used for each statistic or feature employed in our classifier (monitoring function), and Figure 6, shows the ROC curve.
Results & Discussion. We present the results of our monitoring function (classifier) in Table 1 and in Figure 6.
In the results, we can see that the latent vector entropybased methods outperform the Mahalanobis distancebased ππ method in almost all the performance metrics. We believe that the reason behind the poor performance of the ππ method is the strong assumption on the embedding space being class conditional Gaussian we building the reference distributions to compute the distance. On the hand, we can see that latent vector variable entropy has the best results. The reason behind the performance is that the classifier benefits from getting more expressive (entropy) information at the latent variable level.
In this work, we presented a method to use the uncertainty from intermediate latent representations for Outof-distribution detection in a semantic segmentation task.
Our early results show that using the entropy from latent features can be useful in building data-driven monitoring functions. In future work, we aim to explore the impact of the structure in the latent space by relaxing the Gaussian assumption [29] and its effect on the metrics and statistics used for the OoD detection task. Moreover, it is important to analyze the applicability of our approach in other semantic segmentation architectures that do not present generative blocks of neural networks.
This work has been supported by the French government under the "France 2030" program as part of the SystemX Technological Research Institute within the Confiance.ai Program (www.confiance.ai).