Autonomous vehicles (AV) have the potential of not only increasing the safety, comfort and fuel efficiency in a vehicle but also utilising the road bandwidth more efficiently. This, however, will require us to build an AV control software, capable of coping with multiple sources of uncertainty that are either preexisting or introduced as a result of processing. Such uncertainty can come from many sources like a local or a distant source, for example, the uncertainty about the actual observation of the sensors of the AV or the uncertainty in the environment scenario communicated by peer vehicles respectively. For AV to function safely, this uncertainty needs to be taken into account during the decision making process. In this paper, we provide a generalised method for making safe decisions by estimating and integrating the Model and the Data uncertainties.
In an AV’s software pipeline, the uncertainty arising
from various sources is critical for safe decision
making. Due to the recent advancement in Machine
Learning (ML) techniques, especially Neural Networks (NN),
the software pipeline of an AV is heavily dependent
on data related to its environment and this data comes
from the sensors. The key data sources in the software
pipeline of an AV are the LIDAR, RADAR, GPS,
camera, etc. As these sources are prone to measurement
fluctuations, there is always some uncertainty or noise
in the data which they provide, for example, uncertainty
due to variation in sensor resolution, internal sensor
noise, measurement fluctuations caused by changes in
the weather like rain, dust, etc. This gives rise to
uncertainty about how the sensor data corresponds to the
ground truth. Although recent advancements in sensor
technology have greatly reduced such inaccuracies, they
still remain of significant concern (Schwart
In the software pipeline of a typical AV, the perception task is heavily dependent on data and advanced ML model techniques, both of which are prone to uncertainty. This uncertainty can lead to incorrect predictions and therefore, jeopardize the safety of the AV. Hence, for the safety of an AV, it becomes imperative that we incorporate these uncertainties in the decision making process. (Macfarlane and Stroila 2016)
One recent ML technique, known as the Convolutional Neural Network (CNN), has been widely adopted across both the industry and the research, primarily because of its at par human level accuracy in dealing with various image recognition challenges and for providing robustness to the Data and Model Uncertainty. (CNN are robust to large variation in input data). The perception task of an AV also utilises CNN techniques for various classification and object detection tasks. (Stallkamp et al. 2012)
For a safety critical application like an AV, it becomes imperative that in perception tasks, such CNN models not only have high accuracy but are also able to estimate and utilise the Data and Model Uncertainty for decision making. Recent advances in the area of Probabilistic Convolutional Neural Networks (PCNN) have provided a way to estimate the Data and Model Uncertainty for object classification. (McAllister et al. 2017)
Data Uncertainty arises from sensor noise or
measurement fluctuations caused by changes in weather
conditions like rain, dust, etc., whereas, Model
Uncertainty arises because the ML models learn from data
and are not explicitly programmed to perform certain
tasks
Bayesian Networks (BN) are an effective technique
for decision making under uncertainty, and are utilised
heavily for such tasks across domains
In this paper, we present a method that addresses the challenge of managing the uncertainty from PCNN by using BN for decision-making. Our method links the outputs from a PCNN to a predefined BN. At runtime, the output from the PCNN is used as evidence for nodes of the BN. This allows us to estimate the probability of being in a certain state while taking into account uncertainties arising at runtime. These state probabilities can be used to ensure that safe decisions are taken. 2
The challenges of decision making in an AV, which
is safety critical in nature, is that they require
robust guarantees to assure safety, security, assurance
and other dependable characteristics
Though these techniques yield good results, none of these solutions address how to estimate uncertainties arising from perception tasks, or how to take these uncertainties into account during the decision making process.
Work done by
Using
Like
In
PCNN produces probabilistic understanding of Deep
Learning models by inferring the distribution over NN
parameters, i.e., Weights and Biases. This distribution
over NN parameters allows us to estimate the Model
and Data Uncertainty. This estimate of Model and Data
Uncertainty are added to get a single value for Total
Uncertainty, which is then normalised using logistic
regression to present probability of correct
classification
Model Uncertainty tells our ignorance about which
model parameter best fits the underlying data. In the
case of NN, where the model training (learning)
process is stochastic in nature, there can be different values
for model parameters leading to similar prediction
accuracy. Therefore, using PCNN, we can estimate our
ignorance regarding which model parameters generated our
underlying data
In a PCNN, exact inference of posterior distributions
over a large parameter space, like a Kernel in PCNN, is
intractable. Possible methods which exist consist of the
Sampling Methods, the Variational Inference Methods
or the Ensemble Methods
In a given dataset, the input feature space is defined
by X = [x1; ::; xn], and the output to be predicted is
defined as Y = [y1; ::; yn]. The usage of dropouts at
runtime allows us to use the distribution over Weights and
Biases which can later be used to calculate the Mean of
the Predictive Posterior Distribution (y ) for any new
data (x ) by taking the Mean of the SoftMax output
Score for N number of Forward Passes. Finally, the
Model Uncertainty can be captured in the form of
Shannon Entropy (SE)
Data Uncertainty captures the noise which is inherently
present in the sensor data. PCNN help us to quantify
the noise in the data as it can be trained to learn this
noise in an unsupervised manner. This uncertainty in the
data, which is learned by modifying the loss function of
PCNN, tells us the noise inherently present in the data
Unlike Model Uncertainty, we do not need to run multiple Forward Passes to capture Data Uncertainty. Also, in case of the latter, uncertainty cannot be reduced using additional data. 3.3
cases where the probabilities of two or more different states are equal, to avoid a deadlock, the system designer can define a set of rules. In cases where two states have highest and approximately equal probability, safety goals can be ensured by using predefined rules to choose a particular state. For instance, the more safetycritical state can be chosen in the case of a tie. 4
In this section, we describe the implementation of our
proposed method by using a conceptual platooning case
study used by
The case study we use is a Platooning Scenario
consisting of two vehicles, the Follower and the Leader. These
vehicles operate in Cooperative Adaptive Cruise
Control (CACC), tasked to ensure that a Safe Distance is
maintained between the two vehicles. For the
Platooning Scenario, the following conditions
– Condition 1: d ds, where d and ds are the distance between the two vehicles and the minimum safety distance respectively.
– Condition 2: Current Vehicle Speed Speed Limit, where former is the current speed of the vehicles and the latter is the speed limit on the road.
– Condition 3: Any ambiguity arising while checking the validity of the input data, is modeled to ensure the safety of the system and the system utilises only correct input data for decision making.
A SM is used to model the failure behaviour
An executable BN can be created to produce the system’s probability of being in a certain SM state. The BN model and the PCNN used, as shown in Figure 2, contain both the quantitative and the probabilistic safety parameters for inferring the system’s state at runtime. The BN nodes of “Speed”, “Speed Limit”, “Distance from Follower”, “Safe Distance”, are all quantitative parameters. These quantitative parameters are used for checking the safety condition related to Speed and Distance, as mentioned in the SM. The “Leader detected by Follower”, “Follower detected by Leader” and “Valid Speed Limit”, are all probabilistic parameters used for checking the validity of the input data.
The safety of the Platooning Scenario, as defined in the SM in Figure 1, is based on three conditions, i.e., Safe Speed, Safe Distance and Ambiguity. These condition are also represented in the BN. The “Speed Check” S0 S1 S2 S3 S4 S5 State Description
The safety condition of safe distance is fulfilled and the Follower is driving within the speed limit of the road.
The safety condition of safe distance is fulfilled but the Follower is driving above the speed limit of the road.
Action The state is safe, therefore, continue driving.
Decelerate to fall within the speed limit.
The safety condition of Decelerate to increase safe distance is not the distance with the fulfilled and the Follower Leader until safety is driving within the condition is fulfilled. speed limit of the road.
The safety condition of Decelerate to achieve a safe distance is not safe distance with the fulfilled and the Follower Leader and fall within is driving above the the speed limit. speed limit of the road.
The safety condition of safe distance is not fulfilled, the Follower is driving above the speed limit of the road, and is driving too close to the Leader.
Safety condition of safe distance and/or speed limit cannot be verified.
Brake to stop driving.
Switch to ACC mode.
node is responsible for producing probabilistic
guarantees of maintaining a safe speed. This is achieved
through two child nodes, namely, “Valid Speed Limit”,
representing a certificate about the validity of the speed
limit and “Speed Within Limit”, monitoring the legality
of the vehicle’s current speed by comparing it with the
current speed limit. Similarly, “IsSafe” is responsible
for producing probabilistic guarantees of maintaining
Safe Distance between the Leader and the Follower. The
condition for Ambiguity is monitored by the “Detection
Quality” node, which provides a guarantee about the
detection by both the Leader and the Follower vehicles.
In
In this section we discuss the implementation of the
PCNN for the Platooning Case Study. For the
platooning working example, we provide evidence of speed
detected from the speed sign board, which is used in the
“Speed Limit” node of the BN. Also, the
probabilistic confidence of this prediction is used in the “Valid
Speed Limit” node of the BN. The PCNN used was
trained using a Traffic Sign Dataset where traffic signs
were detected from images and uncertainty in the results
was quantified using PCNN discussed earlier. The
German Traffic Sign Recognition Benchmark Dataset
(Stallkamp et al. 2012)
For easy implementation of PCNN, we used AstroNN
API, which is built on top of Keras and Tensorflow.
For estimating Model Uncertainty, the runtime dropout
is implemented by ”MCdropout” layer of the AstroNN
API
The simple model with 20 epochs was producing a training accuracy of above 95 percent for multiple runs. Also, the test data accuracy was 90 percent and above. Fig 3a) shows how we have high accuracy corresponding to lower value of Total Uncertainty.
The uncertainty measures produced by PCNN are numeric values and not a probability distribution as is required for probabilistic inference in BN. To address this issue, we convert the uncertainty measure, i.e., the sum of the Model and Data Uncertainty, into a probability of correct classification by using a logistic regression and it is implemented with a popular pymc3 library. The results in Figure 3b), show how low uncertainty correlates highly with the probability of correct prediction. 224 0 0 0 0 0 0 409856 32896 5547 5547 5547 Layer Input Layer Conv2D Activation MCDropout Conv2D Activation MCDropout Flatten Dense MCDropout Dense Activation Dense Dense MaxPooling2D varianceoutput(Dense) (None,43)
The remaining setup and assumptions for the
experiment remain the same as used by
In the next section, we discuss the results we performed to show how our approach can incorporate data and model uncertainties to ensure the overall safety of Output Shape
Parameters (None,40,40,3) (None,40,40,8) (None,40,40,8) (None,40,40,8) (None,40,40,16) 1168 (None,40,40,16) 0 (None,40,40,16) 0 (None,10,10,16) 0 (None,1600) (None,256) (None,256) (None,128) (None,128) (None,43) (None,43) To test the working of the proposed method, we generated two Test Scenarios. Scenario A, where the evidence provided to “Validity of speed limit” is considered to be 100% for each test case, and Scenario B, where the the probabilistic uncertainty output from PCNN is used. The results for the tests performed for each scenario are summarised in Table 3 and Table 4.
Firstly, we discuss the results obtained for test cases in Scenario A, where the evidence provided to “Validity of speed limit” is considered to be 100% for each of the following test case:
– Test Case A1: the “Speed” of the Follower is more than the “Speed Limit” and all other safety conditions are met, therefore, State S1 (Decelerate to fall within the speed limit) is selected with 100% probability.
– Test Case A2: the “Distance detected by Leader” and “Distance detected by Follower” is less than the “Safe distance”, and all other safety conditions are met, therefore, State S2 (Decelerate to increase the distance with the Leader until safety condition is fulfilled) is selected with 100% probability.
– Test Case A3: the “Distance detected by Leader and “Distance detected by Follower” is less than the “Safe distance” and “Speed of the Follower” is more than the “Speed Limit”, therefore, State S3 (Decelerate to achieve a safe distance with the Leader and fall within the speed limit) is selected with 100% probability.
– Test Case A4: the Follower is driving above the “Speed Limit” and is also “Too close” to the Leader therefore, State S4 (Brake to stop driving) is selected with 100% probability.
In Scenario B, all the safety conditions, as described in the SM, are met, but instead of always considering the probability of the “Valid Speed Limit” detected as 100%, the probabilistic uncertainty output from PCNN is used. As seen below, the different test cases results in both a change in the probability of the output state and the output state selected:
– Test Case B1: all the safety conditions are met, and there is 100% confidence in the validity of the speed limit, therefore State S0 (The state is safe, therefore, continue driving) is selected with 100% probability.
– Test Case B2: as in Test Case B1, the safety conditions are met and the evidence provided to the nodes in the BN are the same except for the “Valid Speed Limit” node, which gets the normalised input from PCNN. Here, the “Valid Speed Limit” node receives the probability of correct “Speed Limit” detected as 70%. We see that the same final State S0 is selected, but with 70% probability. This result shows that with a sufficient probability from the PCNN, even when probability is less than the 100%, State S0 is correctly selected. This ensures that even when some uncertainty is observed, the car is still able to move.
– Test Case B3: as in Test Cases B1 ad B2, most of the safety conditions are met and the evidences provided to various nodes in a BN are the same, except for the “Valid Speed Limit” node. Here, the “Valid Speed Limit” node receives the probability of correct “Speed Limit” detected as 40% and therefore, we see that State S5 (Switch to ACC mode) is selected with 60% probability as the final output. Figure 2 shows that the state having the highest probability, i.e., State S5 (Switch to ACC mode), is selected. This represents the safest decision for this test case. Here, the output state selected changes because of low confidence in the validity of the speed limit (i.e. the evidence provided to the “Valid Speed Limit” node is below 50%, which is in this case, the acceptable safety threshold used in the BN). This test case shows that if blind trust is put into the “Speed Limit” detected from road sign boards, believing it to be always 100% accurate, then that is likely to lead to an unsafe output state. This was seen in the original implementation of the platooning case study, and would typically be the result if using advanced ML techniques like NN.
– Test Case B4: similar to the test cases above, most of the safety conditions are met, however, there is absolutely no confidence in the validity of the speed limit detected (“Validity of speed limit” is 0%) and therefore, final State S5 (Switch to ACC mode) is selected with 100% probability.
The Test Scenarios A and B show that while using BN, the Model and the Data Uncertainty (provided as normalised/probabilistic input to “Validity of speed limit” node) have a huge influence on the probability of the output state selected. The results show that our method of using a PCNN, to estimate both the Model and the Data Uncertainty, along with BN, enables us to make safe decisions. Unlike deterministic models, BN are capable of handling uncertainty in the input and therefore are an better choice for handling uncertainty generated from PCNN for making safe decisions. 6
In this paper, we have described how we can utilise the estimated uncertainty, arising from data and complex ML models, to improve safety in decision making. The proposed method allows designers of AV to improve the decision making process by integrating multiple sources of uncertainty. The efficacy of the proposed approach has been illustrated via an experimental analysis. 7
This research has received funding from Assuring Autonomy International Program (University of York) and the European Union’s EU architecture Program for Research and Innovation Horizon 2020 under Grant Agreement No. 812.788