Recently, averaging the predictions of multiple stochastic or deterministic networks has become a standard approach for improving accuracy [1,2] and uncertainty estimates [3]. Generally, the quality of uncertainty estimates (e.g.: QoC) is assessed by the degree of calibration and/or the ability to detect FPs. Model averaging can yield well-calibrated confidence [4,5] and is one of the state-of-the-art methods for detecting FPs caused by out-of-distribution examples [4,3]. However, recent findings [6,7,8] show that model averaging does not necessarily lead to calibrated confidence, especially when the networks are built using modern regularization techniques, such as mixup [9] or label smoothing [10,11]. This is because modern regularization techniques can (strongly) regularize networks, resulting in underconfidence. Furthermore, averaging underconfident networks
Cedrique.NjieutcheuTassi@dlr.de (C. R. N. Tassi); Jakob.Gawlikowski@@dlr.de (J. Gawlikowski); Auliya.Fitri@dlr.de (A. U. Fitri); Rudolph.Triebel@dlr.de (R. Triebel) produce more underconfident networks. For example, [7] showed that averaging networks trained with modern regularization techniques resulted in more underconfident networks and therefore miscalibrated predictions. [12] supported this argument by theoretically and empirically showing that averaging calibrated networks do not always lead to calibrated confidences. Calibrating confidences of averaged networks has received little attention in the literature. Generally, post-processing calibration methods, such as temperature scaling [13], can be used to recalibrate the confidences of averaged networks, as demonstrated in [8,12]. From [14] and further supported by [8], confidence calibration in model averaging is correlated to diversity inherent in individual networks and the more diverse the networks, the better the calibration. Motivated by this observation, [14] promoted model diversity using structured dropout to reduce calibration errors. [7] proposed class-adjusted mixup that trains less confident networks by evaluating the difference between accuracy (estimated on a validation dataset after each training epoch) and the confidence of each training sample to activate or deactive mixup training for overconfidence (average confidence > accuracy) or underconfidence (average confidence < accuracy), respec-tively. All these methods for improving the calibration of combined networks focus on recalibrating, building, or sampling the calibrated networks. However, this work focuses on combining the networks. Specifically, we address the question: What is the impact of averaging logits instead of probabilities of multiple (stochastic or deterministic) networks on the QoC?
We hypothesized that averaging logits instead of probabilities of multiple networks increases the confidence of the averaged network. This is because logits (inputs to softmax), which can be interpreted as found evidence for possible classes [15], are continuous values normalized using the softmax to produce discrete probabilities. The softmax normalization of continuous values (logits) to discrete values (probabilities) causes information loss and possible robustness to changes in the magnitudes of logits. This implies that the softmax function is a nonlinear function that maps multiple logit vectors with large differences in magnitudes to the same discrete probability vector. We evaluated the impact of the increase in confidence caused by averaging logits instead of probabilities on the QoC. Specifically, we evaluated the QoC by assessing the degree of confidence calibration, which measures the difference between the predicted (average confidence) and true probabilities (empirical accuracy). Furthermore, we evaluated the QoC by assessing its ability to seperate TPs and FPs. To provide empirical evidence for evaluating the QoC, we considered the logit averaging against probability averaging and compared both approaches using different averaged models, such as ensemble, MCD, and MMCD. The comparison was based on results from different experiments conducted on three datasets, namely, MNIST, FashionMNIST, and CIFAR10 evaluated on VGGNet, ResNet, and DenseNet, respectively.
Results show that averaging logits instead of probabilities preserves accuracy, but increases confidence. For example, for MCD evaluated on CIFAR10 (see Table 2), the accuracy remained around 85.36% while the average confidence increased from 73.35% to 80.04% when we averaged logits instead of probabilities. Furthermore, given underconfident models, the increase in the degree of confidence reduces the calibration error on the test data. For example, for MCD evaluated on CIFAR10, ECE dropped from 12.04% to 5.40% when the average confidence increased from 73.35% to 80.04%. However, given overconfident models, the increase in the degree of confidence increased the calibration error on the test data. For example, for the ensemble evaluated on CIFAR10 (see Table 3), ECE increased from 3.03% to 7.40% when the average confidence increased from 89.43% to 96.17%. Finally, for underconfident or overconfident models, the increase in the degree of confidence can harm the separability between TPs and FPs. This is because averaging logits instead of probabilities increases the confidence of both TPs and FPs. Therefore, FPs can be made with high confidence similar to TPs. For example, for MMCD evaluated on FashionMNIST (see Table 4), the average confidence on FPs due to the noisy data increased from 51.31% to 94.58% when averaging logits instead of probabilities. In summary, we provide empirical evidence demonstrating how combining logits instead of probabilities of multiple (stochastic or deterministic) networks • preserves accuracy, but increases the confidence on TPs and FPs. • reduces the calibration error (given underconfident networks), but increases the calibration error (given overconfident networks). • can harm the separability between TPs and FPs.
The combination process describes how multiple members are combined and the information type (e.g., logits or probabilities) that is combined. Several approaches such as stacking [16] and voting [17,18,19]) have been reported for aggregating multiple predictions. Some of these approaches have been reviewed and discussed in [20,21] and experimentally compared in [18,16] to find the one with the best accuracy. It was found that one approach improves accuracy better than another depending on several factors, such as the number of members, diversity inherent in individual members, and accuracy of individual members. However, in [22], we compared approaches such as averaging, plurality voting, or majority voting to find the one that better captures uncertainty. We found that the averaging approach captures uncertainty better than voting approaches. Before our work, [23] argued that simple averaging approaches are more robust than voting approaches. This argument was further supported by [24]. This is because the averaging approach considers all members' predictions, whereas plurality/majority voting ignores uncertain predictions and therefore, reduces the uncertainty in the combined members' prediction. Although various combination approaches have been presented and compared in the literature, the information type that is combined has received relatively little attention. [25] showed that averaging quantiles rather than probabilities improve the predictive performance. Generally, for neural networks and classification problems in particular, multiple members (networks) are combined by averaging probabilities [16]. [16] evaluated the impact of combining logits instead of probabilities on accuracy, however, the impact on the QoC remains unclear. Thus, we investigated the impact of combining logits instead of probabilities on the QoC.
In the context of image classification, let the training data 𝐷𝑡𝑟𝑎𝑖𝑛 = {𝑥𝑖 ∈ R 𝐻×𝑊 ×𝐶 , 𝑦𝑖 ∈ 𝑈 𝐾 } 𝑖∈[1,𝑁 ] be a realization of independently and identically distributed random variables (𝑥, 𝑦) ∈ 𝑋 × 𝑌 , where 𝑥𝑖 denotes the 𝑖 𝑡ℎ input and 𝑦𝑖 its corresponding one hot encoded class label from the set of standard unit vectors of R 𝐾 , 𝑈 𝐾 . 𝑋 and 𝑌 denote the input and label spaces. 𝐻 × 𝑊 × 𝐶 denotes the dimension of input images, where 𝐻, 𝑊 , and 𝐶 refer to the height, weight, and number of channels, respectively. 𝐾 and 𝑁 denote the numbers of possible output classes and samples within the training data, respectively.
A CNN is a nonlinear function 𝑓 𝜃 parameterized by model parameters 𝜃, called the network weights. Here, it maps input images 𝑥𝑖 ∈ R 𝐻×𝑊 ×𝐶 to class labels 𝑦𝑖 ∈ 𝑈 𝐾 ,
The network parameters are optimized on the training dataset, 𝐷𝑡𝑟𝑎𝑖𝑛. Given a new data sample 𝑥 ∈ R 𝐻×𝑊 ×𝐶 , a trained CNN 𝑓 𝜃 predicts the corresponding target 𝑦 = 𝑓 𝜃 (𝑥) using the set of trained weights 𝜃. The network output (logit) is given by 𝑧 = 𝑓 𝜃 (𝑥), from which a probability vector 𝑝(𝑦|𝑥, 𝐷𝑡𝑟𝑎𝑖𝑛) = 𝑠𝑜𝑓 𝑡𝑚𝑎𝑥(𝑧), can be computed. In the following, this probability vector will be abbreviated by 𝑝 and its entries by 𝑝 𝑘 with 𝑘 = 1, . . . , 𝐾 and ∑︀ 𝐾 𝑘=1 𝑝 𝑘 = 1. Further, we get the predicted confidence 𝑐 = max 𝑘 (𝑝 𝑘 ) and predicted class label 𝑦 = arg max 𝑘 (𝑝 𝑘 )
MCD was investigated in [26,27,28] for uncertainty estimation. It is one of the most widespread Bayesian methods reviewed in [3]. It approximates the prediction 𝑝(𝑦|𝑥, 𝐷𝑡𝑟𝑎𝑖𝑛) using the mean of 𝑆 stochastic forward passes, 𝑝(𝑦|𝑥, 𝜃1), ..., 𝑝(𝑦|𝑥, 𝜃𝑆), representing 𝑆 stochastic CNNs parameterized by samples 𝜃1, 𝜃2,..., and 𝜃𝑆. That is
Specifically, MCD approximates the prediction with a dropout distribution realized by sampling weights with masks drawn from known distributions, such as Gaussian, Bernoulli, or a cascade of Gaussian and Bernoulli distributions [22]. For example, given the activation vector 𝑎 fed to a MCD layer (placed for example at the input of the first fully-connected layer) and assuming that sampling is realized with masks drawn from a cascade of Gaussian and Bernoulli distribution, the MCD layer samples the 𝑗 𝑡ℎ element of 𝑎 as
Here, 𝑞 denotes the dropout probability. In this work, we can refer to MCD as an average of 𝑆 stochastic CNNs.
An (explicit) ensemble was investigated in [4,27,28] for uncertainty estimation. It approximates the prediction 𝑝(𝑦|𝑥, 𝐷𝑡𝑟𝑎𝑖𝑛) by learning different settings. Given a set of CNNs 𝑓 𝜃𝑚 for 𝑚 ∈ 1, 2, ..., 𝑀 , the ensemble prediction is obtained by averaging over the predictions of the CNNs. That is,
In this work, we can refer to an ensemble as an average of 𝑀 deterministic CNNs.
MMCD was investigated in [29,30,31]
In this work, we can refer to MMCD as an average of 𝑀 • 𝑆 stochastic CNNs.
The output layer of a CNN-based classifier includes 𝐾 output neurons with a softmax activation function, which normalizes its inputs (continuous values) to produce discrete probabilities 𝑝 𝑘 (with 𝑘 = 1, . . . , 𝐾 and ∑︀ 𝐾 𝑘=1 𝑝 𝑘 = 1) representing the probability that the input image belongs to the class associated with the 𝑘 𝑡ℎ output neuron. The input to the softmax function are logits and interpreted as evidence for possible classes [15]. The discrete probability 𝑝 𝑘 is interpreted as the model confidence that the input belongs to the class associated with the 𝑘 𝑡ℎ output neuron. Given the logit vector 𝑧 = [︀ 𝑧1 . . . 𝑧𝐾 ]︀ 𝑇 , the softmax estimates
From Figure 1, given an ensemble of 𝑀 deterministic CNNs with logits 𝑧 𝑚 , the average logit 𝑧 can be estimated as
and the predicted probability vector of the ensemble of deterministic CNNs can be reformulated as
Given MCD representing an ensemble of 𝑆 stochastic CNNs with logits 𝑧 𝑠 , we can estimate the average logit 𝑧 as
and reformulate the predicted probability vector of MCD, as shown in (7). Similarly, given MMCD representing an ensemble of 𝑀 • 𝑆 stochastic CNNs with logits 𝑧 𝑚𝑠 , we can estimate the average logit 𝑧 as (9) and reformulate the predicted probability vector of MMCD, as shown in (7). From Figure 2, averaging logits instead of probabilities of multiple stochastic or deterministic CNNs increases the confidence of the averaged CNNs. Intuitively, logit averaging provides the best evidence (characterized by a low level of uncertainty caused by the reduction of inductive biases inherent in individual logits) for making decisions. However, probability averaging provides the best confidence associated with decisions made using weak evidence (characterized by a high level of uncertainty caused by inductive biases inherent in individual logits). This implies that a decision made using probability averaging considers more uncertainty than that made using logit averaging. In this work, we evaluated the impact of the possible increase in the degree of confidence caused by applying logit averaging instead of probability averaging on the QoC.
We hypothesized that the QoC of CNNs (strongly) depends on the task-difficulty (specified using the training data), the underlying architecture, and/or the training procedure (mostly influenced by the regularization 𝑚=1 𝑝 𝑚 with 𝑝 1 = 𝑠𝑜𝑓 𝑡𝑚𝑎𝑥(𝑧 1 ), 𝑝 2 = 𝑠𝑜𝑓 𝑡𝑚𝑎𝑥(𝑧 2 ), 𝑝 3 = 𝑠𝑜𝑓 𝑡𝑚𝑎𝑥(𝑧 3 ), and 𝑝 4 = 𝑠𝑜𝑓 𝑡𝑚𝑎𝑥(𝑧 4 ). One can see that averaging logits (𝑝 𝑧 ) results in more confident predictions than averaging probabilities (𝑝). This is attributed to averaging logits being more sensitive to the magnitude of logit values than averaging probabilities. Here, 𝑧 𝑚 with large values contributes most to 𝑧. In our example, 𝑧 is mostly influenced by the values of 𝑧 1 . That is, the contributions of 𝑧 2 , 𝑧 3 , and 𝑧 4 to 𝑧 are minor. However, 𝑝 is influenced by the values of all probability vectors 𝑝 𝑚 and therefore, is less sensitive to the magnitude of individual logits. strength). Therefore, we compared logits and probabilities averaging on three datasets to evaluate the impact of the task-difficulty on the QoC. Moreover, we compared logits and probabilities averaging using three different architectures to evaluate the impact of the underlying architecture on the QoC. Specifically, we evaluated MNIST [32] on VGGNets [1], FashionMNIST [33] on ResNets [2] and CIFAR10 [34] on DenseNets [35]. Finally, we compared logits and probabilities averaging on CNNs trained using two regularization strengths (strong and weak regularization summarized in Table 1) to evaluate the impact of the regularization strength on the QoC. We observed strong and weak regularization results in underconfident and overconfident CNNs, respectively. All CNNs were regularized using batch normalization [36] layers placed before each convolutional activation function. All CNNs were randomly initialized and trained with random shuffling of training samples. All CNNs were trained using the categorical cross-entropy and stochastic gradient descent with momentum of 0.9, learning rate of 0.02, batch size of 128, and epochs of 100. All images were standardized and normalized by dividing pixel values by 255. For all MCD and MMCD, we sampled activations of the first fully-connected layer using masks drawn from a cascade of Bernoulli and Gaussian distributions [22] and using a dropout probability of 0.5. We performed 100 stochastic forward passes (𝑆 = 100) and considered ensembles consisting of five deterministic CNNs (𝑀 = 5).
Summary of values assigned to regularization hyperparameters.
QoC was evaluated by assessing the degree of confidence calibration. Specifically, we evaluated the calibration error using measures, such as the negative log likelihood (NLL) applied in [4,5,31], expected calibration error (ECE) applied in [13,8,12], and Brier score (BS) applied in [4]. Low values of NLL, ECE, and BS indicate low calibration error and vice versa. Furthermore, we evaluated QoC by assessing its ability to separate TPs and FPs. Here, we evaluated the average confidence on evaluation data causing TPs or FPs. Given evaluation data causing TPs, we expect the average confidence on the evaluation data to be high. However, for the evaluation data causing FPs, we expect low average confidence on the evaluation data. Moreover, we evaluated the ability to separate TPs and FPs by evaluating the area under the receiver operator characteristic (AU-ROC) applied in [37,5]. AU-ROC summarizes the trade-off between the fraction of TPs that are correctly detected and those of FPs that are undetected using different thresholds. In summary, in addition to the NLL, ECE, and BS, we evaluated the accuracy, average confidence, and AUC-ROC.
We BS. We expect the accuracy to be high and NLL, ECE and BS to be low on test data.
include 1000 correctly classified test data from the experimental data. Since CNNs will make TPs on these data, we used these data for evaluating the average confidence on TPs.
Swapped data were simulated using subsets of the correctly classified test data structurally perturbed by dividing images into four regions and diagonally permuting the regions. From Figure 3b, the upper left and right are permuted with the bottom right and left regions, respectively. Swapped data include structurally perturbed objects within the given images. We expect CNNs to make FPs on swapped data. Therefore, we used these data for evaluating the average confidence on FPs caused by structurally perturbed objects.
Noisy data were simulated using subsets of the correctly classified test data perturbed by applying additive Gaussian noise with a standard deviation of 500. From Figure 3c, noisy data include noise within the given images. We expect CNNs to make FPs on these data. Therefore, we used these data for evaluating the average confidence on FPs caused by noisy objects.
Out-of-domain data were simulated using 1000 test data of CIFAR100 [34]. Since CNNs will make FPs on these data, we used these data for evaluating the average confidence on FPs caused by unknown objects.
In general, we expect the average confidence to be high on TPs and to be low on FPs.
We evaluate the conducted experiments with respect to accuracy and QoC. Table 2 and Table 3 summarize the accuracy, average confidence, NLL, ECE, and BS of different models using the two averaging approaches and CNNs trained using strong regularization (causing underconfidence) and weak (causing overconfidence). The results show that averaging logits instead of probabilities do not strongly affect the accuracy. This means that averaging logits can preserve accuracy. Furthermore, averaging logits instead of probabilities significantly increases the average confidence. Figure 2 illustrates why the confidence increases. Further, Table 2 shows that averaging logits instead of probabilities significantly decreases the NLL, ECE, and BS for undercondifent CNNs (trained using strong regularization). This means that averaging logits, unlike averaging probabilities, reduces the calibration error for undercondifent CNNs. This is because the stronger the regularization, the lower the confidence and the higher the gap between accuracy and average confidence. Here, the increase in the degree of confidence caused by averaging logits instead of probabilities reduces the gap between accuracy and average confidence. For example, Table 2 shows that averaging logits instead of probabilities of the ensemble reduces the gap between accuracy and average confidence from 18.24(= |88.75 − 70.51|)% to 9.52(= |88.94 − 79.42|)% on CIFAR10.
However, the increase in the degree of confidence caused by averaging logits instead of probabilities increases the calibration error for overconfident CNNs (trained using weak regularization). Table 3 provides empirical evidence for this claim by showing that, on CIFAR10 and FashionM-NIST, NLL, ECE, and BS of the ensembles increase when the logits are averaged instead of probabilities. We argued that the more overconfident the CNNs, the higher
Comparison of accuracy[%], average confidence[%] (in bracket), NLL[10 −2 ], ECE[10 −2 ], and BS[10 −2 ] of different models using two approaches for averaging underconfident CNNs trained using strong regularization: average probabilities (AP) and average logits (AL). The results were obtained using the test data described in Section 5.3. the confidence and the higher the gap between accuracy and average confidence. Here, the increase in the degree of confidence caused by averaging logits instead of probabilities further increases the gap between the accuracy and average confidence and therefore, increases the calibration error. For example, Table 3 shows that, on CIFAR10, averaging logits of the ensemble increases the gap between the accuracy and average confidence from 0.76(= |88.67−89.43|)% to 7.29(= |88.88−96.17|)%.
In Table 4, the average confidence on TPs and FPs is shown for underconfident models using both averaging approaches. The results show that averaging logits instead of probabilities increases the confidence level on TPs and FPs. The increase in the average confidence is sometimes very large for FPs due to the noisy data. For example, for MMCD evaluated on FashionMNIST, the average confidence on the noisy data increases from 51.31% to 94.58% when averaging logits. This is because noisy data can increase the magnitude of logits and averaging logits is more sensitive to changes in the magnitude of logits than averaging probabilities (see Figure 2). The increase in the degree of confidence caused by averaging logits can harm the separability of TPs and FPs. For example, the increase in the average confidence on the noisy data from 51.31% to 94.58% causes the AUC-ROC obtained based on the evaluation of the degree of confidence to decrease from 84.80% to 42.42%.
The term 'combination process' encompasses how multiple networks are combined and the information type combined. It was found in [23,24,22] that simple averaging is more robust and captures uncertainty better than voting approaches. This is because the simple averaging equally weights all predictions, while voting ignores uncertain predictions. In this work, we compared the process of averaging logits instead of probabilities. We empirically showed that averaging logits instead of probabilities increases the confidence while preserving the accuracy for underconfident or overconfident networks. This might be because logit averaging preserves the position of the max element of individual logit vectors, but is more sensitive to the magnitude of logit values than probability averaging. Thus, logit values with a large magnitude contribute the most to the average logit. In this way, the magnitude of logit values induces a nonuniform weighting (for logit averaging), which is lost (for probability averaging). Furthermore, we provided empirical evidence showing that for underconfident networks (trained using strong regularization), the increase in the confidence caused by averaging logits instead of
Comparison of average confidence[%] of different models using two approaches (average probabilities (AP) and average logits (AL)) for averaging underconfident networks trained using strong regularization and evaluated on TPs and FPs: TPs were obtained on subsets of the correctly classified test data, while FPs were obtained on swapped, noisy and out-of-domain (OOD) data described in Section 5.3. probabilities reduces the calibration error on the test data. This is because the increase in the degree of confidence reduces the gap between accuracy and average confidence. However, the increase in confidence caused by averaging logits instead of probabilities for overconfident networks (trained using weak regularization) increases the calibration error on the test data. This is because the increase in the confidence further increases the gap between the accuracy and average confidence. This finding suggests that for underconfident networks, we can average logits instead of probabilities to reduce the calibration error. However, we should average probabilities instead of logits for overconfident networks to avoid increasing the calibration error. Although the increase in the confidence caused by averaging logits reduces the calibration error on the test data for underconfident networks, we empirically showed that it can harm the separability of TPs and FPs. This is because averaging logits increases the confidence on both TPs and FPs. Therefore, FPs can also be made with high confidence similar to TPs. These findings suggest that reducing the calibration error on the test data and improving the separability of TPs and FPs can be two contradicting goals. Improving one may be at the detriment of the other. Furthermore, for two models 𝐴 and 𝐵, if 𝐴 is better calibrated than 𝐵, then 𝐴 does not necessarily separate TPs and FPs better than 𝐵. This implies that calibration methods may be insufficient for separating TPs and FPs and therefore, ensuring safe decision-making. Additionally, existing methods for confidence calibration may not help in separating TPs and FPs. Subsequently, future work will evaluate the ability of existing methods for confidence calibration to separate TPs and FPs. We also recommend researchers to evaluate both the calibration error of their proposed method for con-fidence calibration and the ability of their proposed method to separate TPs and FPs. Finally, for mission-and safetycritical applications where the separability of TPs and FPs is of paramount importance, we suggest to average probabilities to avoid the negative impact of logits averaging on the ability to separate TPs and FPs.
Due to averaging logits instead of averaging probabilities of stochastic or deterministic networks, the degree of confidence on TPs and FPs increased. This reduces the calibration error on the test data for underconfident networks but affects the separability of TPs and FPs. Our empirical results show that there is a trade-off between improving calibration on the test data and improving the separability of TPs and FPs. Additionally, the increase in the degree of confidence increases the calibration error on the test data for overconfident networks. Therefore, averaging logits should only be applied when combining underconfident networks. For example, we can average logits instead of probabilities of an ensemble of networks trained with mixup or other modern data augmentation techniques to improve calibration on the test data. Notwithstanding this, for mission-and safetycritical applications where the separability of TPs and FPs is essential, we suggest traditionally average probabilities. However, it remains unclear if the findings of this paper will change if the given networks or the average logit are calibrated, for example, with temperature scaling [13]. This suggests a new research direction.