Convolutional neural networks (CNNs) have become an established part of numerous safetycritical computer vision applications, including human robot interactions and automated driving. Real-world implementations will need to guarantee their robustness against hardware soft errors corrupting the underlying platform memory. Based on the previously observed efficacy of activation clipping techniques, we build a prototypical safety case for classifier CNNs by demonstrating that range supervision represents a highly reliable fault detector and mitigator with respect to relevant bit flips, adopting an eight-exponent floating point data representation. We further explore novel, non-uniform range restriction methods that effectively suppress the probability of silent data corruptions and uncorrectable errors. As a safety-relevant end-to-end use case, we showcase the benefit of our approach in a vehicle classification scenario, using ResNet50 and the traffic camera data set MIOVision. The quantitative evidence provided in this work can be leveraged to inspire further and possibly more complex CNN safety arguments.
With the widespread use of convolutional neural networks
(CNN) across many safety-critical domains such as
automated robots and cars, one of the most prevailing challenges
is the establishment of a safety certification for such
artificial intelligence (AI) components, e.g., with respect to the
ISO 26262 [
C1: Operational design domain: Inference of pretrained classifier networks with protection layers, input represented by given dataset.
C2: An appropriate independent dataset for bound extraction exists.
C3: “Sufficiently safe” is well defined by the end user and is proportional to the overall risk.
G2: System detects critical
soft errors.
E2a: SDC/DUE events appear in conjunction with oob events with a high conditional probability.
E2b: Oob events are detected by threshold-based protection layers.
E2c: DUE events can further be detected by Nan/Inf monitoring.
G1: System is sufficiently safe in the presence of soft errors.
C4: The chance of a soft error event to occur can be given.
C5: The simulated weight/neuron fault model appropriately represents realistic soft errors.
C6: The data representation has eight exponent bits (FP32, BF16).
C7: A fallback system/re-execution can be used for uncorrectable errors.
G3: System mitigates soft errors.
G4: System does not increase
the error severity.
E3a: The probability of SDC/DUE events is significantly reduced by restricting oob activations in
protection layers.
E3b: DUE events can further be mitigated by referring to a fallback system or via re-execution.
E4a: DUE events can be handled with negligible risk for any error
severity.
E4b: The severity of residual SDC depends on the application. As an example, we study the scenario of MioVision and ResNet50 and find that the average severity of errors is comparable or lower.
Soft errors typically manifest as single or multiple bit
upsets in the platform’s memory elements [5]. As a
consequence, network parameters (weight faults) or local
computational states (neuron faults) can be altered during inference
time, and invalidate the network prediction in a safety-critical
way, for example, by misclassifying a person as a background
image in an automated driving context [6–8]. This has led to
a search for strategies to verify CNN-based systems against
hardware faults at the inference stage [
In this paper, we evaluate range restriction techniques in CNNs exposed to platform soft errors with respect to the key elements of a prototypical safety case. This means that we formulate arguments (in the form of ”goals”) that constitute essential parts of a complete safety case, and provide quantitative evidence to support these goals in the studied context (see Fig. 1). Individual safety arguments can be reused as building blocks of more complex safety cases. The structure of our goals is based on the probabilistic, high-level safety No fault (a) No protection
Weight fault se d u it n g a m n o it a v itc
A
objective of minimizing the overall risk [
Risk = å Ploss(i) Severity(i): i (1) Explicitly, for a fault type i, this includes the sub-goals of efficient error detection and mitigation, as well as a consideration of the fault severity in a given use case. On the other hand, the probability of occurrence of a soft error (i.e., Pfailure in Eq. 1) is assumed to be a constant system property that cannot be controlled by run-time monitoring methods such as activation range supervision.
In a nutshell, range restriction builds on the observation
that silent data corruption (SDC) and detected uncorrectable
errors (DUE), e.g., NaN and Inf occurrences), stem primarily
from those bit flips that cause very large values, for example
in high exponential bits [6]. Those events result in large
activation peaks that typically grow even more during forward
propagation due to the monotonicity of most neural network
operations [
The focus of this paper is to examine alternative restriction schemes for optimized soft error mitigation. In a CNN, the output of every kernel is represented as a two-dimensional (2D) feature map, where the activation magnitudes encode specific features, on which the network bases its prediction. Soft errors will manifest as distortions of feature maps in all subsequent layers that make use of the corrupted value, as shown in Fig. 2(a)-(b). The problem of mitigating soft errors in a CNN can therefore be rephrased as restoring the faultfree topology of feature maps.
Previous analyses have adopted uniform range restriction schemes that truncate out-of-bound values to a finite threshold [7, 8], e.g., Fig. 2(c)-(d). We instead follow the intuition that optimized, non-uniform range restriction methods that attempt to reconstruct feature maps (see Fig. 2(e)-(g), and details in Sec. 5) can not only reduce SDC to a comparable or even lower level, but may also lead to less critical misclassifications in the case of an SDC. This is because classes with more similar attributes will display more similar high-level features (e.g., pedestrian and biker will both exhibit upright silhouette, in contrast to car and truck classes).
Finally, a safety analysis has to consider that not all SDC events pose an equal risk to the user. We study a safetycritical use case evaluating cluster-wise class confusions in a vehicle classification scenario (Sec. 6). The example shows that range supervision reduces the severe confusions proportionally with the overall number of confusions, meaning that the total risk is indeed mitigated.
In summary, this paper make the following contributions: • Fault detection: We quantify the correlation between SDC events and the occurrence of out-of-bound activations to demonstrate the high efficiency of fault detection by monitoring intermediate activations, • Fault mitigation: We explore three novel range restriction methods that build on the preservation of the feature map topologies instead of mere value truncation, • Fault severity: We demonstrate the benefit of range supervision in an end-to-end use case of vehicle classification where high and low severities are estimated by the generic safety-criticality of class confusions.
The article is structured as follows: Section 2 reviews relevant previous work while section 3 describes the setup used in this paper. Subsequently, the sections 4, 5, and 6 discuss error detection, mitigation, and an exemplary risk analysis, respectively, before section 7 concludes the paper.
Parity or error-correcting code (ECC) can protect memory
elements against single soft errors [
CNNs are the most commonly used network variant for
computer vision tasks such as object classification and detection.
We compare the three standard classifier CNNs ResNet-50
[
Experiments were performed in PyTorch (version 1.8.0)
deploying torchvision models (version 0.9.0). For
MIOVision, the ResNet-50 model was retrained [
We insert protection layers at strategic positions in the network such as after activation, pooling, reshape or concatenate layers, according to the model of Chen et al. [8]. Each protection layer requires specific bound values for the expected activation ranges as a parameter. We extract those by monitoring the minimal and maximal activations from a sepaDUE No faults Inject faults
Faults
SDC
rate test input, which is taken from the training data sets of
ImageNet (143K images used) and MIOVision (83K images
used), respectively. This step has to be performed only once.
Bound extraction depends on the data set and will in general
impact the safety argument (see Fig. 1). To check the
suitability of the bounds, we verify that no out-of-bound events
were detected during the test phase in the absence of faults,
so the baseline accuracy is the same with and without
protection. While all minimum bounds are zero in the studied
setup, the maximum activation values for ImageNet vary by
layer in a range of (see also Sec. 5) 1 < Tup < 45 for
ResNet50, 20 < Tup < 360 for VGG-16, and 65 < Tup < 170 for
AlexNet. For MIOVision and ResNet-50, we find maximum
bounds between 1 < Tup < 19.
In line with previous investigations, we distinguish two
different manifestations of memory bit flips referred to here as
weight faults and neuron faults. The former represent soft
errors affecting memory elements that store the learned network
parameters, while the latter refer to errors in memory that
holds temporary states such as intermediate network layer
outputs. While neuron faults may also impact states used
for logical instructions, it was demonstrated that bit flip
injections in the output of the affected layer are generally a
good model approximation [
We inject faults either directly in the weights of CNN
layers (weight faults) or in the output of the latter (neuron faults),
using a customized fault injection framework based on
PytorchFI [
To quantify the impact of faults on the system safety, we measure the rate of SDC events. Throughout, we consider the Top-1 prediction to determine SDC. In line with previous work [6, 8], SDC is defined as the ratio of images that are misclassified in the presence of faults (without exceptions) but correctly classified in the absence of faults and the overall number of images, p(sdc) = Nincorrect=Ntest , correct (Fig. 3).
During the forward pass, non-numerical exceptions in the form of Inf and NaN values can be encountered, due to the following reasons: i) Inf values occur if large activation values accumulate (for example during conv2d, linear, avgpool2d operations) until they exceed the maximum of the data representation. This effect becomes particularly apparent when flips of the most significant bit (MSB, position index 1) are injected. ii) NaN values are found when denominators are undetermined or multiple Inf values get added, e.g., in BatchNorm2d layers, iii) NaN values can be generated directly via bit flips in conv2d layers due to the fact that FP32 encodes NaN as all eight exponent bits being in state ”1”. In the studied classifier networks, the latter effect is very rare for single bit flips in weights (see Sec. 4) but not necessarily for single neuron bit flips or multiple flips of either type.
The creation of the above exceptions is found to differ
slightly between CPU and GPU executions, as well as
between experiments with different batch sizes on the
accelerator. We attribute this observation to algorithmic
optimizations on the GPU that are not necessarily IEEE754-compliant
and thus affect the floating point precision [
In a real system, DUE events can be readily monitored and the execution is typically halted on detection. However, due to the non-numerical nature of these errors we cannot apply the same mitigation strategy that is adopted for SDC events. We therefore make the assumption that either a fallback system (e.g., alternative classifier, emergency stop of vehicle, etc.) can be leveraged or a timely re-execution is possible to recover from transient DUE events. This in turn assumes that DUEs do not impact the system safety but may compromise the system availability when occurring frequently. To effectively protect the network against faults, we first verify the error detection coverage for silent errors. Those errors are detected by a given protection layer if the activation values exceed (fall short of) the upper (lower) bound. If at least one protection layer is triggered per inference run, we register an out-of-bound (oob) event. Otherwise, we have an in-bound (ib) event. In addition, we quantify the probabilities of SDC and regular correct classification (cl) events, as well as the respective conditional probabilities that correct and incorrect classifications occur given that oob or ib events were detected. This allows us to define true positive (Tp), false positive (Fp), and false negative (Fn) SDC detection rates as Tp = p(sdcjoob) p(oob); Fp = p(cljoob) p(oob); Fn = p(sdcjib) p(ib): (2) The fault detector then is characterized by precision, P = Tp=(Tp + Fp), and recall, R = Tp=(Tp + Fn).
The Tab. 1 displays the chances of oob and sdc events resulting from a single fault per image in the absence of range protection. For weight faults, we find that all three CNNs showcase a high correlation between oob situations and either SDC or DUE events (p(sdcjoob) + p(duejoob) > 0:99), which can be associated with the chance of a successful error detection, Pdetection (see Eq. 1). The chance of finding SDC after ib events is very small ( 1e 3), leading to a very high precision and recall performance (> 0:99). For neuron faults, while the recall remains very high, the precision is reduced (in particular VGG-16 and AlexNet) due to additional Fp events where non-MSB oob events still get classified correctly.
We further verify that SDC events from single weight faults are attributed almost exclusively to flips of the MSB. This can be explained with the distribution of parameters in the studied networks (Fig. 4). The weight values are closely centered around zero, and thus exhibit characteristic properties when represented in an eight-exponent data format. In the fault-free case, the MSB always has state “0”, while the exponent bits 2 to 4 are almost always in state “1”. This means that among the relevant exponential bits all single bit flips of the MSB will produce large values, while those of the other exponential bits will either be from “1” ! “0” or will be too small to have a significant effect.
For neuron faults, on the other hand, the distribution of fault-free values is input-dependent and broader, leading in general to a smaller quota of MSB flips to SDC, in favor of flips of other exponential bits and the sign bit. No SDC due to mantissa bit flips were observed in either weight or neuron faults. DUE events are unlikely (< 0:01) for a single bit flip as there are not multiple large values to add up. Further, network weights are usually < 1, meaning that at least two exponent bits are in state ”0”, and hence at least two bit flips are needed to directly generate a NaN value. We refer to a subset of the tensor given by a specific index in the batch and channel dimensions as a 2D feature map, denoted by f . Let x be an activation value from a given feature map tensor f 2 f f1; f2; : : : ; fCout g. Further, Tup and Tlow denote the upper and lower activation bounds assigned to the protection layer, respectively.
Ranger: For a given set of ( f ; Tup; Tlow), Ranger [8] maps out-of-bound values to the expected interval (see Fig. 2c), rranger(x) = 8<Tup
:x if x > Tup; if x < Tlow; otherwise:
Clipper: In a similar way, clipping truncates activations that are out of bound to zero [7], rclipping(x) = 0 if x > Tup or x < Tlow; x otherwise: The intuition is that it can be favorable to eliminate corrupted elements rather than to re-establish finite activations.
FmapRescale: While uniform restriction methods help in eliminating large out-of-bound values, the information encoded in relative differences of activation magnitudes is lost when all out-of-bound values are flattened to the same value. The idea of rescaling is to linearly map all large out-of-bound values back onto the interval [Tlow; Tup], implying that smaller out-of-bound values are reduced more. This follows the intuition that the out-of-bound values can originate from the entire spectrum of in-bound values.
rrescale(x) =
Tlow >:x 8 (x min( f ))(Tup Tlow) <> max( f ) min( f ) + Tlow if x > Tup; if x < Tlow; (5) otherwise:
Backflip: We analyze the underlying bit flips that may have caused out-of-bound values. This reasoning holds for (3) (4)
p(sdc) p(oob) p(sdcjoob) p(sdcjib) p(MSBjsdc) P R p(due) p(duejoob) p(MSBjdue) VGG-16: p(sdc) p(oob) p(sdcjoob) p(sdcjib) p(MSBjsdc) P R p(due) p(duejoob) p(MSBjdue)
p(sdc) p(oob) p(sdcjoob) p(sdcjib) p(MSBjsdc) P R p(due) p(duejoob) p(MSBjdue)
Weight faults
Neuron faults neuronal faults, where we may assume that a specific activation value is bit-flipped directly. For weight faults, on the other hand, the observed out-of-bound output activation is the result of a multiply-and-accumulate operation of an input tensor with a bit-flipped weight value. However, we argue that the presented back-flip operation will recover a representative product, given that the input component is of the order of magnitude of one. To restore a flipped value, we distinguish the following cases: rbackflip(x) = 80 > > >>2 > <
Tup >>>Tlow > >:x if x > Tup 264; if Tup 264 > x > Tup 2; if Tup 2 > x > Tup; if x < Tlow; otherwise: (6) VGG-16 AlexNet VGG-16
AlexNet The above thresholds are motivated by the following logic: Given appropriate bounds, an activation is < Tup before a bit flip. Any flip of an exponential bit i 2 f1 : : : 8g effectively multiplies a factor of pow(2; 28 i). Hence, any value beyond Tup 264 must have originated from a flip ”0” ! ”1” of the MSB, meaning that the original value was between 0 and 2. We then set back all out-of-bound values in this regime to zero, assuming that lower reset values represent a more conservative choice in eliminating faults. Next, flipped values that are between Tup 264 > x > Tup 2 can possibly originate from a flip of any exponential bit. Given that Tup is typically > 1, a bit flip has to produce a corrupted absolute value > 2 in this regime. This is possible only if either the MSB is flipped from ”0” ! ”1”, or the MSB is already at ”1” and another exponential bit is flipped ”0” ! ”1”. In all variants of the latter case, the original value had to be already > 2 itself, and hence we conservatively reset out-of-bound values to 2. Finally, corrupted values of Tup 2 > x > Tup may originate from any non-sign bit flip. Lower exponential or even fraction bit flips result from already large values close to Tup in this case, which is why we set back those values to the upper bound. As in Ranger, values that are too small are reset to Tlow.
FmapAvg: The last proposed range restriction technique uses the remaining, healthy fmaps of a convolutional layer to 1 FI 10 FI 1 FI 10 FI 21.1 1FI 10FI 16.1 1FI 10FI 8.2 24.2 0.3 0.2 1.0 FmapRescale (7) (8) reconstruct a corrupted fmap. The intuition behind this approach is as follows: Every filter in a given conv2d layer tries to establish characteristic features of the input image. Typically, there is a certain redundancy in the topology of fmaps, since not all features the network was trained to recognize may be strongly pronounced for a given image (instead mixtures of potential features may form), or because multiple features resemble each other at the given processing stage. Therefore, replacing a corrupted fmap with a non-corrupted fmap from a different kernel can help to obtain an estimate of the original topology. We average all healthy (i.e., not containing out-of-bound activations) fmaps by ind = fi = 1 : : :Coutj max( fi) Tup; min( fi)
Tlowg; favg = 1
å fi: jindj j2ind If there are no healthy feature maps, favg will be the zerotensor. Subsequently, we replace oob values in a corrupted fmap with their counterparts from the estimate of Eq. (7), rfavg(x) = favg(x) if x > Tup or x < Tlow; x otherwise: 5.2
In Fig. 5 we present results for the SDC mitigation experiments with different range supervision methods. Comparing 1 and 10 fault injections per input image, we note that the unprotected models are dramatically corrupted with an increasing fault rate (SDC rate becomes 0:50 for weights, 0:32 for neurons in the presence of 10 faults). We can associate the SDC rate with the chance of unsuccessful mitigation, 1 Pmitigation, in Eq. 1. Weight faults have a higher impact than neuron faults since they directly corrupt a multitude of activations in a layer’s fmap output (in contrast to individual activations for neuron faults) and thus propagate faster than neuron faults.
All the studied range restriction methods reduce the SDC rate by a significant margin, but perform differently for weight and neuron fault types: For weight faults, we observe that Clipper, Backflip, and FmapAvg are highly efficient in all three networks, with SDC rates suppressed to values of . 0:01 (SDC reduction of > 50 ). Ranger provides a much weaker protection, in particular in the more shallow networks VGG-16 and AlexNet. FmapRescale performs better than Ranger but worse than the aforementioned methods. The deepest studied network, ResNet-50, benefits the most from any type of range restriction in the presence of weight faults.
When it comes to neuron faults (Fig. 5b), we see that Clipper and Backflip provide the best protection (SDC rate is suppressed to < 0:005, reduction of > 38 ), followed by the also very effective Ranger (except for AlexNet). FmapAvg appears to be less efficient for higher fault rates in this scenario, while FmapRescale again falls behind all the above.
Overall, we conclude that the pruning-inspired mitigation techniques Clipper and Backflip represent the best choices among the investigated ranger supervision methods, as they succeed in mitigating both weight and neuron faults to very small residual SDC rates.
In the experiments of Fig. 5, the encountered DUE rates for 1 weight or neuron fault (0:003 for ResNet, 0:03 for VGG-16 or AlexNet) are only slightly reduced by range restrictions. However, for a fault rate of 10 we find the following trends: i) For weights, the DUE is significantly reduced in ResNet (from 0:15 to 0:002), while rates in VGG (0:22) and AlexNet (0:26) remain. ii) For neurons, Ranger, Clipper and Backflip suppress the DUE rate by a factor of up to 2 in all networks.
The studied range restriction techniques require different compute costs due to the different number of additional graph operations. In PyTorch, not all needed functions can be implemented with the same efficiency though. For example, Ranger is executed with a single clamp operation, while no equivalent formulation is available for Clipper and instead three operations are necessary (two masks to select oob values greater and smaller than the threshold, and a masked-fill operation to clip to zero). As a consequence, measured latencies are framework-dependent and a fair comparison cannot be made at this point. Given the complexity of the protection operations, we may instead give a qualitative performance ranking of the described methods: FmapRescale appears to be the most expensive restriction method due to the needed number of operations, followed by FmapAvg and Backflip. Clipper and Ranger are the least complex, with the latter outperforming the former in the used framework, due to its more efficient use of optimized built-in operations. 6
As a selected safety-critical use case, we study object
classification in the presence of soft errors with a retrained
ResNet50 and the MIOVision data set [
However, not every class confusion is equally harmful. To estimate the severity of an error-induced misclassification we establish three clusters of vulnerable, as well as nonvulnerable road users (VRU or non-VRU), and background, 0 20.0 ) i(n%17.5 s n iso15.0 u f n o lc12.5 a c ii t r -c10.0 y ft e sah7.5 t i w tae5.0 r C SD2.5 see Fig. 6. Misclassifications that lead to the prediction of a class in a less vulnerable cluster are assumed to be safetycritical (Severity 1 in Eq. 1, e.g., a pedestrian is misclassified as background), while confusions within the same cluster or towards a more vulnerable cluster are considered noncritical (Severity 0) as they typically lead only to similar or a more cautious behavior. This binary estimation allows us quantify the overall risk as the portion of SDC events associated with the respective critical class confusions.
From our results in Fig. 7 we make the following observations: i) The relative proportion of critical confusions is lower for weight than for neuron faults in the unprotected and most protected models. For weight faults, the most frequent confusions are from other classes to the class ”car” (the most robust class of MIOVision, with the most images in the training set), which are statistically mostly non-critical. Neuron faults, on the other hand, distort feature maps in a way that induces with the highest frequency misclassifications towards the class ”background”. Those events are all safety-critical (see Fig. 6), leading to a high critical-to-total SDC ratio. ii) Range supervision is not only effective in reducing the overall SDC count, but also suppresses the critical SDC count proportionally. For example, we observe that the most frequent critical class confusion caused by 1 or 10 weight faults is from the class ”pedestrian” to ”car” ( 0:2 of all critical SDC cases), where > 0:99 of those cases can be mitigated by Clipper or Backflip. For neuron faults, the largest critical SDC contribution is from ”pedestrian” to ”background” (1 fault) or ”car” to ”background” (10 faults), both in about 0:1 of all critical SDC cases. Clipper or Backflip are able to suppress > 0:91 of those events.
As a consequence, all studied range-restricted models exhibit a critical-to-total SDC ratio that is similar to or lower than one of the unprotected network (< 0:41 for weight, < 0:78 for neuron faults), meaning that faults in the presence of range supervision have on average a similar or lower severity than faults that do not face range restrictions. A lower ratio can be interpreted as a better preservation of the feature map topology: If the reconstructed features are more similar to the original features there is a higher chance of the incorrect class being similar to the original class and thus to stay within the same cluster. The total probability of critical SDC events – and therefore the relative risk according to Eq. 1 – is negligible in the studied setup in the presence of Clipper or Backflip range protection.
The mean DUE rates in the unprotected model are 0:0 (0:02) for 1 weight (neuron) fault and 0:11 (0:17) for 10 faults. Using any of the protection methods, the system’s availability increases as DUE rates are negligible for 1 fault, and reduce to < 0:03 (< 0:05) for 10 weight (neuron) faults. 7
In this paper, we investigated the efficacy of range supervision techniques for constructing a safety case for computer vision AI applications that use Convolutional Neural Networks (CNNs) in the presence of platform soft errors. In the given experimental setup, we demonstrated that the implementation of activation bounds allows for a highly efficient detection of SDC-inducing faults, most importantly featuring a recall of > 0:99. Furthermore, we found that the range restriction layers can mitigate the once-detected faults effectively by mapping out-of-bound values back to the expected intervals. Exploring distinct restriction methods, we observed that Clipper and Backflip perform best for both weight and neuron faults and can reduce the residual SDC rate to . 0:01 (reduction by a factor of > 38 ). Finally, we studied the selected use case of vehicle classification to quantify the impact of range restriction on the severity of SDC events (represented by cluster-wise class confusions). All discussed techniques reduce critical and non-critical events proportionally, meaning that the average severity of SDC is not increased. Therefore, we conclude that the presented approach reduces the overall risk and thus enhances the safety of the user in the presence of platform soft errors. [2] [4] [5] [6] [7] [8]
Our research was partially funded by the Federal Ministry of Transport and Digital Infrastructure of Germany in the project Providentia++ (01MM19008). Further, this research was partially supported by a grant from the Natural Sciences and Engineering Research Council of Canada (NSERC), and a research gift from Intel to UBC.