Modeling the driver's behavior is an importance task in the automotive field. Most data-driven approaches build a general model that works well for the majority of drivers in the training set. Personalization, on the other hand, addresses the problem of adapting the model on the driver. A personalized model has to be able to adjust to changing user preferences over time. In this work, we propose an approach to extract and incorporate information from previous maneuver executions to improve the performance of prediction tasks in impending maneuvers. We apply our proposed adaptation method to predict the gap taken at a left-turn scenario, and show that it can boost the prediction of a neural network.
It is often the case that users have to adapt themselves to a new system or function to be able to use it. When the users’ expectation and preference of a system are not met, their trust in the system will decrease and eventually they may ignore or turn it off . For a standard not-adapting system, there could be a gap between the user’s expectation and the outcome of the system. Personalization systems aim at closing this gap.
The problem of personalization is already addressed in different field studies and
applications. In recommender systems, for example, the data are mostly sequences of
actions [
Driving style is an abstract concept that is not directly observable from the data. What
we can observe is the reflection of driving style on some measurable signals like speed,
acceleration, etc. A survey conducted by Martinez et al. [
Mi = f (Si, Dj + i) where Si represents all environmental factors, Dj constitutes the driver’s dispositional factors. And i is introduced to capture the fluctuations in the driver’s behavior.
To personalize the predictions of y0, we additionally have to consider the influences of the driver Dj + 0 in this situation, since y0 depends on both factors, i.e., y0 ← S0, Dj + 0. Now we have the same problem as in modeling maneuver execution, namely that the driver’s dispositional factors are not directly observable. This make it impossible to learn the individual impacts of driver Dj on y0.
As mentioned above, in real-world driving data, the drivers’ factors are encoded in each maneuver execution Mi. We therefore propose that instead of learning y0 as a function of S0 and Dj , we can learn y0 as a function of S0 and Mi, where Mi is a recently performed maneuver This temporal restriction of Mi allows us to approximate the current impact of the driver Dj + 0 with Dj + i, which was captured in the Mi.
y0 ← S0, Mi = S0, f (Dj + i, Si) ≈ S0, f (Dj + 0, Si) y0 can then be written as a function G of S0, Dj + 0 and Si:
y0 = g(S0, f (Dj + 0, Si)) = G(S0, Dj + 0, Si)
With the assumption that y0 does not depend on other situations than the current S0, there should be no information of y0 contained in Si. G will be forced to learn to extract useful information about the driver from past maneuvers and then how they will affect the decision of the driver in the current situation.
Extracting Driver’s Information. A maneuver execution Mi with length n is characterized by a multivariate time series (xi,t is a vector of sensor values at time t): (2) (3) (4) Mi = (xi,1, xi,2, ..., xi,n), n > 0 (5)
In this work, we used two approaches to extract a driver’s information from the last executed maneuvers. The first approach use the statistical information from each sensor (minimum, mean, maximum and standard deviation). The second approach makes use of recurrent network layers and takes the whole maneuver execution as an input. While the idea of the first approach is widely used in the literature for extracting driver’s information, the second one does not make any assumption about the statistical values but learns to extract useful information direct from raw data.
We have now formulated the prediction of y0 as a function of the current situation S0 and the previous maneuver Mi. This can be extended by incorporating the last k maneuvers. Fig 1 shows an example of a network that uses k = 3 last maneuvers as features. To validate this concept we test it with k = 1 which is easier to train and requires less data. past maneuvers M3 M2 Network Architectures. We design the network using two input layers sepa- M1 rately, one takes the current situation (S0) as input and one captures useful informa- input path 2 tion from the past maneuver Mi.
Each of these two input layers are concatenate then followed by hidden layers and form input path 1 two separate paths. For the ultimate purpose of predicting y0, these two paths are then combined in the deeper layers, fol- Prediction lowed by further hidden layers and lastly the output layer. The whole network is Fig. 1: Proposed architecture to capture the trained by back-propagating the classifi- driver information from previous maneuver cation error. for predicting the current situation current situation
S0
Extracting Driver Information with Neural Network. As mentioned in Section 2,
extracting driver information from Mi could be done in two different ways. We modeled
the first approach by configuring the first input path using fully connected layers that
take statistical values from the last maneuver execution as input. The function G could
then be written as or Gs for short. The second approach makes use of a recurrent layer
to capture the maneuver execution. In particular, we use Long short-term memory
networks (LSTM) [
Data set. The data used in this work were collected from 32 drivers covering a wide range of ages and driving experiences. Each driver drove 30 rounds on a pre-defined route. The data set was collected in real-world traffic. The chosen route is a common urban route which requires the driver to perform different maneuvers roundabout, left turn, or intersection with right of way. For our experiments, we focused on the two most complex maneuvers, roundabout and left-turn.
For capturing the driving style we use four signals as features for describing a maneuver execution, which include speed, longitude and latitude acceleration and steering wheel speed. As features for describing the intersection situation we additionally use the position of the ego vehicle to determine the distance to the middle of the intersection. Gap Acceptance at Left-Turn Maneuver. In an unsignalized intersection scenario, in which the incoming traffic has the right of way. We observe that the preference of the driver in choosing gaps differs from driver to driver. As shown in Fig. 2, gaps that are in range from approximately 3 to 7 seconds could either be taken or ignored. In such a scenario, the problem of predicting the gap acceptance of driver should not only consider the current traffic situation but also the driver’s individual preferences. 5
Evaluation Setup. We evaluate our method using 10-fold cross-validation to construct training and testing sets. The data are split into disjoint folds according to the driver ID. In total we evaluate four models. As the baseline model, we estimate the best possible threshold that separates the taken and ignored gaps in the training set. θb = arg min L(y, ti > θ) θ (6) where ti is the time available for turning left if gap i is taken. L is the loss function that computes the error rate for a given label y and the predictions based on threshold θ. The baseline is then compared to the three models described in the previous sections. Learning from previous maneuvers. The Table 1: Mean of F 1 and accuracy score on average F1 score and the accuracy of validation set over 10-fold cross-validation all four models on the validation sets are show in Table 1. Both variants that extract information from Mi show sig- Model F1 Score Accuracy nificant improvements. The overall best Threshold 81.9 % 89.8 % score is obtained by Glstm, which uses G(S0) 86.8 % 92.6 % an LSTM layer for capturing Mi. Various G(S0, hs(Mi)) 89.9 % 94.1 % network configuration are tested with dif- G(S0, hlstm(Mi)) 91.7 % 95.1 % ferent regularization parameters applied to each input path individually. In all settings, using extra information from Mi always leads to improvement of the prediction performance.
Reviewing the cross-validation results, we observe the improvement in terms of F1 score and accuracy of Gs and Glstm over G(S0) in eight out of ten folds. The maximal improvement reaches 18.5 % in F1 score which translates to 9.4 % accuracy. Fig 3a shows the validation score of all three models in one of these eight folds. Here we can see that G(S0) gets stuck and its best score is quite low, whereas both, the Gs and Glstm models benefit from the extra input Mi and reach much higher accuracy. (a) Incorporating Mi lead to improvement (b) Overfitting on training data In Fig 3b we observe the first slight effect of overfitting as the score ofGlstm and Gs keeps fluctuating and overfit on training data after 40 epochs. To deal with such effect, early stoping was also used for training the final model. 6
In this work we proposed a new approach to learn the dependency between maneuver execution, namely to extract the information about driver behavior and style, and use it to improve the performance of the prediction task in driver assistance systems. We implement our approach using neural networks as building blocks and empirically evaluate the model in a left-turn situation using on-road data. The results show that the model is able to extract the driver impact from the past maneuver executions and can use it improve the prediction quality by almost 10 % in terms of F1 score. We compared two approaches for extracting driver’s information from maneuver execution, which make use of statistical features and an LSTM layer. The LSTM model shows improvement over a model that only uses statistical features and fully connected layers.