In this paper a design framework for preventive safety systems (ADAS) is proposed. The design framework takes into account risk mitigation strategies, advanced driver's model, based on modern approaches and algorithms (machine learning and add-on functionalities), able to capture key aspects of human behavior, such as distraction, and to retain the fundamental characteristics of cognition and decision making.
Driver modeling is a scientific area involving several disciplines, such as psychology, physics, computer science, etc. The importance of adding the learning capability to information systems, in order to make them more effective and smarter, is confirmed by the variety of areas in which user’s modeling has already been applied: information retrieval, filtering and extraction systems, adaptive user interfaces, educational software, etc.
In relation to the problem formulated above, the aim of
this paper is to deeply understand the problem of ADAS
(Advanced Driver Assistance System Design) design, the
problem of developing an effective driver’s and driving
model supporting distraction mitigation. Such systems
would mitigate the effects of distraction and tolerate the
consequences of distraction thanks to a better road and
vehicle design
A feasible and promising solution is the use of Add-On functionalities, able to detect driving maneuvers that are indication of distraction, placing them in the framework of a cognitive model of human behavior.
In this paper a design framework for preventive safety
systems is proposed, following three main building
blocks:
1. new knowledge about driver behavior:
extensive empirical studies about the sources of
accidents and potential counter measures as a
basis for the driver model development.
2. risk mitigation strategies: implementation of a
human error risk based approach;
3. advanced driver modeling: development of
models for predicting correct and erroneous
driver behavior, based on modern approaches
and algorithms (machine learning), able to
capture key aspects of human behavior, and to
retain the fundamental characteristics of
cognition and decision making.
toward an automation failure. The automation failure is a
side effect of an effort to produce ‘‘safety’’
In addition to the safety issues associated with the driving task, the proliferation of complex in-vehicle functions itself poses a further challenge for the design of the driver-vehicle interface: one of the current research area in automotive is the development of preventive warning systems, also called ADAS (i.e. Advanced Driver Assistance Systems) adopted with the aim of improving driving safety. These systems are able to detect an incoming dangerous in advance, allowing a time to perform a repairing manoeuvre.
ADAS are aimed at “partly supporting and/or taking
over the driver’s tasks”
Lane Keeping Assistant: The function of a lane keeping assistant system includes the lane detection and the feedback to the driver if he/she is leaving a defined trajectory within the lane. An active steering wheel can help the driver with a force feedback to keep on this trajectory. The lane is detected by a video image processing system.
Local Hazard Warning : If a hazard occurs far away in front of the vehicle, so that the driver cannot see it, this system will warn him/her. By the means of communication it is possible, to transfer this information over long distances Lane Change Assistant: Before and during a dangerous lane change process, the lane change assistant will warn the driver. Several stages of such a system are possible from pure warning systems to even haptic feedback at the steering wheel to help the driver following a lane change trajectory.
Blind Sport Monitoring: This function detects if a vehicle is present in the so called “blind spot” area when the vehicle is starting a lane change and/or overtaking maneuvers. A camera is placed into the left rear-mirror and once the incoming vehicle is recognized, a warning is issued to the driver.
Obstacle & Collision Warning: The driver will be warned if a potential collision is detected via radar-based technology (e.g. another car or obstacle). The functional limits of these systems have to be clearly pointed out. The liability problem of these systems grows with the complexity of the detecting scenarios.
Obstacle and Collision Avoidance: This system has an extended functionality compared to the Obstacle and Collision Warning. An autonomous intervention takes over the control of the vehicle in critical situations in order to avoid an accident. Longitudinal and lateral control will be done by the system during the defined time while the dangerous event takes place.
Night Vision: Based on camera techniques like near or far infrared, it allows enhancing the perception of the driver in dark light conditions. The picture of the camera will be shown to the driver by monitors or head up displays.
Platooning: Several cars are connected electronically (e.g. by the means of communication) and follow one after the other in a platoon. An example is the connection of trucks in order to save space, fuel and to increase the traffic flow. As the following vehicles are driven automatically, the system is complex concerning all aspects. The takeover of the driver at e.g. gateways has to be taken into account as well as the behavior in mixed traffic at driveways.
Designing the trustiness: ADAS research issues
Interaction with these devices is one of the many
activities that constitutes driving and so it can represent
an additional source of driving-related distraction
The analysis of ADAS working conditions, architectures and performances leads towards the definition of a proper theoretical framework that is not yet present in current projects.
The reasoning behind is the following: Advanced
Driver Assistance Systems, or ADAS, are systems that
help the driver in its driving process: they detect a
dangerous situation and gives a warning. We can define
analytical
In the case of analytical ADAS we can consider there are two actors playing a role: • The driver • The warning system
In the case of behavioral ADAS we can consider there
are three actors playing a role:
• The driver
• The horse
In both cases, analytical and behavioral ADAS, there is a warning systems that detects the dangerous situation and then provides the driver, as safety warning, accidentavoiding maneuvers.
The purpose of these systems is to foresee and detect possible driver’s errors and mistakes, due to a misbehavior such as distraction, or resulting from too high workload, missing perception, wrong action/execution or poor operator skills.
The ADAS design is aimed at enhancing the driver’s perception of hazards and critical situations (in some cases, by partly automating the driving task as well). Of course the potential of such systems in reducing accidents depends on the effectiveness of their interaction with the driver. For example, in the case of an anti-collision systems it is safety-critical that the collision warning is able to generate the appropriate feedback (e.g. an avoidance maneuver).
Since ADAS can be actually considered recommending systems, the use of an appropriate driver’s or/and driving model will improve their effectiveness and consequently, human safety.
Risk mitigation strategies: recommending the accident avoiding actions ADAS can be considered an application of recommending systems that recommends the driver repairing maneuvers in order to avoid an accident. The most advanced systems are able to directly take part in the driving task, whether the driver doesn’t react on time. Also in this case, the systems follows a recommendation formulated by the system itself.
Recommender systems have become a promising
research area since the appearance of the first papers on
collaborative filtering in the mid 1990s
Shortly, the recommendation problem is formally represented as a space S of possible items that may be very big, ranging in hundreds of thousands or even millions of items in some applications, such as recommending books. An utility function measures the usefulness of each item for a certain user.
In recommender systems, the utility of an item is
usually represented by a rating, which indicates how a
particular user likes a particular item or how a particular
item is appropriate for a certain user, taking care of a set
of context conditions. Generally speaking , utility can be
an arbitrary function, including a profit function.
Depending on the application, the utility can either be
specified by the user, as is often done for the user-defined
ratings, or is computed by the application, as can be the
case for a profit-based utility function
To each element of the user space C can be associated a profile that includes the user characteristics that are relevant for the current application. Similarly each element of the item space S is defined by a set of relevant characteristics.
In recommender systems, utility is typically represented by ratings, therefore, the recommendation engine should be able to estimate (predict) the ratings of the nonrated user/item combinations and issue appropriate recommendations based on these predictions. Extrapolations from known to unknown ratings are usually done by: • specifying heuristics that define the utility function and empirically validating its performance • estimating the utility function that optimizes certain performance criterion, such as the mean square error.
Despite of the results nowadays achieved, the existing
generation of recommender systems still requires further
improvements including better methods for representing
user behavior and the information about the items to be
recommended, more advanced recommendation
modeling methods, incorporation of various contextual
information into the recommendation process, utilization
of multicriteria ratings, development of less intrusive and
more flexible recommendation methods that also rely on
the measures that more effectively determine
performance of recommender systems
In the case of services provided on board a car, we can
notice that they are rapidly growing. Almost all car
manufactures are offering systems that add functionality
to route planners, possibly integrated with internet and
web access or that support driver in high demanding
tasks, in order to increase safety and avoiding accidents.
The availability of these add-ons is an interesting
opportunity, considering that nowadays the amount of
time spent in the car (e.g., for commuting or for work and
vacation trips) is very high
If on one hand the driver and the other vehicle
occupants can actively use the time spent on the car, on
the other hand the use of these services can be distracting
and can create serious safety problems (Green 200)
Starting from this consideration, the introduction of
personalization and adaptation strategies and techniques
should be a feasible solution in the case of services in the
car. In fact, by considering the characteristics of the user
and the context of interaction, a personalized and
adaptive system may tailor the interaction to the way
which is most appropriate to avoid distractions, and as a
direct consequence, to avoid an accident
In the case of safety-critical systems which should recommend accident avoiding maneuvers, the adaptation of the recommendations to the specific user is crucial, according to the psychophysical parameters that are taken into account (i.e. mental workload, distraction, arousal level, situation awareness). In the case of advanced driving assistance systems one of the most important psychophysical parameter to be taken into account is distraction. The system should be able to assess driver’s distraction in order to estimate accident precondition (risk layout) and recommend driver appropriate actions, or in the case of adaptive automatic systems to perform a proper risk mitigation strategy.
If the recommending engine has not at its disposal a user behavior model, it can formulate recommendation that may lead towards no decisions or wrong decisions.
Whether the system prediction capability is augmented through a user behavior model it is possible to reduce errors and then the risk of accidents. This consideration is of paramount importance in complex safety critical systems as avionic and automotive, that commonly use different kind of recommending services.
The design of cognitive preventive safety systems Driving is considered as a complex and multitasking cognitive activity that can be summarized by four main sub-processes: perception, analysis, decision and action. To be performed, each phase presumes the achievement of the previous one. That said, it is likely that the demands of one element of driving will interfere with another element.
ADAS new technologies have great potential for enhancing road safety, however, when an ADAS or an
In-Vehicle information system (IVIS) is activated and the driver is asked to interact with it, the driver him/herself is distracted from the driving task, that is, his/her attention is moved from the driving task to the secondary task. A relevant part of vehicle crashes are estimated to
The Driver Assistance Systems have to be able to adapt their action to the context and to the driver and vehicle status. Thereby, they need a model of human behaviour that takes into account the model of the system performance and that is able to detect and classify driver’s intention and distraction, in order is essential to facilitate operating mode transition between users and driver assistance systems.
The need of an effective user model is a requirement for any recommending system, as faced and confirmed by the domain literature on user modeling and automatic recommending systems. This requirements is crucial for any recommending system that has to cope with timecriticality, that directly affects safety.
ADAS applications are examples of such systems and they represent a challenging test bed for the implementation and validation of user behavioral modeling systems realized by means of Machine Learning techniques.
Basically, the human behavior is characterized by the interactions between driver-vehicle and driver environment.
The first interaction is related to how the driver interacts with the vehicle and all systems and subsystems on-board.
The second interaction is related to how drivers perceive and process the data coming out from the surrounding scenario.
Hence, the driver model should be adaptive to different drivers’ style and preferences as well as to the external environment (including learning both from the driving experience and from the surrounding conditions), but overall it should allow to assess and foresee distraction.
Preventing distraction permit to prevent driving errors and accident risk, as a consequence, a risk based design approach (that follows a risk mitigation strategy) is crucial for the design of vehicles and transport systems in order to guarantee safety and efficiency of human mobility.
User modeling (UM) aims at improving system effectiveness and reliability by adapting the behavior of the system to the needs of the individual.
The importance of adding this capability to information systems is proven by the variety of areas in which user modeling has already been applied: information retrieval, filtering and extraction systems, adaptive user interfaces, educational software, safety-critical systems .
Machine learning (ML) techniques have been applied
to user modeling problems for acquiring models of
individual users interacting with an information system
and grouping them into communities or stereotypes with
common interests. This functionality is essential in order
to have a useful and usable system that can modify its
behavior over time and for different users
Driver Assistance Systems have to handle crucial aspects like timing and warning, therefore the development of an algorithm for the personalization of such aspects that takes into account for example is needed.
Regarding the drivers’ intention prediction, several
models have been proposed aiming at reproducing in a
virtual environment how the drivers could behave
according to specific Driver, Vehicle or Environment
conditions , that is DVE model
The development of a model of the human machine system is driven by the model of the Driver, which is the most complex element of the system.
Concerning the design of algorithms used to represent
the Driver behaviour, previous and ongoing studies
propose different approaches based on the real-time
monitoring of the drivers’ performance
Basing on these information, these approaches allow to predict specific drivers’ profiles (e.g. stressed, aggressive, tired, distracted, high workload etc.) and are developed following machine learning approaches. Thanks to machine learning, information on drivers’ profile can be automatically extracted from data, by computational and statistical methods applied to observable information (e.g. drivers’ performance data).
On one hand the assessment of the driver’s status and consequentially the prediction of his/her next behaviour is easier and most successful using driver’s physiological data, as for example the eye gaze and the eye movements measured by means of eye tracker. On the other hand eye-tracking is an intrusive measurement system of distraction, it represents a further equipment and a further cost that stand in the way of a next future massmarketing. What is really interesting and challenging is to obtain a driver index analysing driving performance data, realising what it may be considered an ADD-On Functionalities.
Understanding driver’s maneuvers by the use of
Add-On Functionalities
Research in driver comfort and performance
improvement understanding driver’s maneuvers is very
active. Usually, these targets are achieved through the
installation of further in-vehicle sensors and devices
An alternative is the use of the so-called Add-On
Functionalities (AOF). They do not require new sensors
but only information coming from the on-board network
and sub-systems (e.g. the elements of chassis,
suspensions, steering angle, etc.). This information is
computed in a well tuned algorithm and the results
provide some added-value supports to the drivers as
AOFs for pre-crash detection. They prepare the vehicle to
the impact in critical situation which can not be avoided,
for example, by pre-tensioning the seat belt
Add-On Functionalities can in fact be divided in two main categories: • Driving Behaviour: i.e. Add-On Functionalities related to driving performance. Main objective of these AOF is to estimate driving conditions concerning road, dynamics and current manoeuvre. • Driver Behaviour: these AOF deal with driver current state, mainly intended as mental effort (or workload) related to the driving task.
In order to be implemented, an Add-On function has to satisfy two conditions: • All Add-On inputs must be available and shareable. • At least one Add-On output can be received as input by a vehicle device.
An Add-On Function with m inputs and n outputs is defined as:
, , … or in a concise form: , , …
If Y is set from available device output and X is set from available device inputs, the two aforementioned conditions can be expressed as follows: In order to be implementable, an add-on functionality , defined as where outputs.
with , , … and , , …
must satisfy the following conditions:
1. , , …
2. , 1, … , |
and are respectively set of devices inputs and
In the study presented in
According to the level of stress it is possible to find strategies to assist the driver in particularly demanding maneuvers by modifying the steering force feedback, braking behavior and inhibition of secondary task to avoid possible safety-risk situations.
Particularly, modifying the steering force feedback or the braking behavior has an interesting impact in the human-machine interaction, because the feedback to the driver is haptic Empirical test confirmed that driving performance significantly improved when the system activated the force feedback models.
These results compared with data arose from other
studies in the literature
For a decade researchers
Traffic congestion (TC). Through the monitoring of
longitudinal vehicle parameters (e.g. brakes and speed
behavior) it is possible to state whether drivers are
driving in a heavy traffic situation or not. In this case,
strategies could be elaborated by optimizing both the
engine-fuel management at a low speed and the driver
comfort aiming at reduces the level of stress
Road Conditions (RC). The knowledge of road profile
characteristics through information coming from specific
sensors (i.e. suspensions, Roll Rate Sensor, Pitch Rate
Sensor, Sound Sensor Cluster, ESP intervention) allows
alerting active suspension system to smooth the impact of
obstacles, helping drivers to reduce the effect of this
critical situation
The table below
Table 1 AOF inputs, parameters and outputs
Speed Deceleration Jerks TC (DJ) Brake Pressure Braking Frequency TC (BF) Accelerator Accelerator TC Displacement Frequency (AF) Gear number Gear Index (GI) TC Z acceleration Frontal Obstacle RC
Preview (FR) Roll rate Roll Index (RI) RC Pitch rate Pitch Index (PI) RC Suspensions Frontal Obstacle RC Displacement Preview (FR)
Inputs were selected to compute specific AOF parameters with the aim to describe driver stress, traffic congestion and road conditions. AOF outputs are the result of the balanced sum among parameters; for instance, the Driver Stress (DS) index (2) was developed as follow 0:
DS = (RR x cRR) + (HFS x cHFS) + (SAR x cSAR) (2) Where cRR + cHFS + cSAR = 1 are the coefficients to be tuned in order to define the final value of the AOF output. Each AOF outputs (TC, DS and RC) and their related computed parameters (see “AOF parameters” in Table 1) were developed in a simulated environment using Matlab/Simulink (www.mathworks.com).
In order to test and tune these parameters, AOF models were interfaced with a Matlab/Simulink simulated vehicle. The whole model (AOF and simulated vehicle) is fed by real driving data coming from a professional driving simulator. Specific tests were carried out, aiming to provide driving situation where each parameter varies significantly; then, their effectiveness was assessed.
According to the result, information monitored by AOF outputs (DS, TC and RC) will be used as a basis for the development of strategies aiming at improve driving performance, safety and comfort.
AOF test and tuning: Driver Stress
In the following, test and tune of Driver Stress (DS)
parameters are described. The DS index is the balanced
sum of steering angle based parameters, in particular:
SAR (Steering Action Rate), HFS (High Frequency
Steering), RR (Reversal Rate). A default tuning of
coefficients related to these parameters has been applied
(CRR = 0,4; CHFS = 0,2; CSAR = 0,4). The effect of the
tuning was assessed by comparing the expected stress
profile in certain pre-determined conditions (i.e., the
points numbered from 1 to 5 in Figure 1) with the
drivers’ steering activity
Two tests were conducted on a driving simulator where
12 subjects, each of them was asked to drive for 10
minutes. Test environments were characterized by roads
with different curve radius, variable visibility (from 100
to 4500 m) reproduced with fog and variable traffic (from
10 to 50 vehicle/km)
In order to increase the steering activity, subjects were
also expected to complete a secondary visual task, which
consisted in pressing the left/right side of a touch-screen
on the left side of the vehicle cabin, according to the
position of a circle displayed among smaller ones. Data
regarding DS parameters were collected and compared
with the steering activity in a specific point of the road
(the number from 1…5 circled in Figure 2)
Due to the large amount of information, the comparison focused on a sub-sample of 3 subjects; mean values of steering activity and DS parameters are then depicted. An example of steering angle activity is showed in the top side of Figure 2, while Driver Stress index in the bottom. Both are related to scenario coordinates (xaxis). As foreseen, an increased steering activity leads to higher Driving Stress values 0. These peaks are pointed out in particularly critical situations (due to curves, high traffic, low visibility), highlighted in the circled number of the figures.
Results show that the first DS parameters’ tuning
produced an index able to detect driver stress status.
Since the analysis was carried out on a small sample of
subjects, in order to increase the significance of the
tuning the above deployed comparison will be extended
to all subjects
Driving Stress Index appears to be a good starting
point for developing a parameter able to detect driver
status in real-time even if a deeper test and tuning activity
is needed. Furthermore, together with the other AOFs
(Traffic Congestion and Road Conditions) can be easily
implemented on a vehicle ECU (Electronic Control Unit)
or a DSP (Digital Signal Processor)
Conducted simulations are just preliminary tests to find out the most promising indexes. It is very important to improve techniques aimed at monitoring of the status and the performance of both the driver and the vehicle in order to gather all data useful to customize the information provision strategies of the in-vehicle devices. In this way the vehicle may adapt his status, improving comfort or modifying driving performances.
This proactive behaviour paves the way to mechanisms
able to infer the driver’s distraction and situation
awareness, allowing the triggering of adaptive
automation strategies. The information provided by
AOFs are real-time and allows at dynamically
implementing such adaptive strategies and referring them
both to the path changes and to the driver’s status. The
AOFs added-value is the prospect to obtain information
in a not intrusive way
The framework for the design of preventive safety systems The aim of preventive safety system is to support drivers, especially in risky and critical situations, or whenever distraction may occur. The first step for the design of a vehicle able to assess the driver status and intentions is the development a model able to explain and reproduce driver’s characteristics. Based on the empirical results presented in the previous chapters, this research work aims at developing a little missing piece of the puzzle of the future intelligent vehicles: namely to identify the main elements for a feasible architecture of a “cognitive driving assistance system” which will substantially advance both integrated safety/assistance systems and the cooperation between human beings and highly automated vehicles.
The feasibility of such an architecture has been investigated analyzing:
The problem of accident precondition analysis ADAS existing applications and research issues Risk mitigation strategies for the accident avoiding Design issues of cognitive preventive safety systems The understanding of driver behavior from driving maneuvers by means of add-on functionalities
Hence the framework to develop an effective model of driver’s perception will be include four major functional areas: 1.
The core application, where motion-planning tools including enhanced personalisation, will be used to explore the maneuver space and to ultimately understand the driver, and if needed, to produce maneuvers compatible to human motion.
Improved sensing of driver input, where the control input (“input” in control theory sense) produced by the driver, both in longitudinal and lateral directions will be measured. The scope is • • • • •
to better discriminate between different motion
alternatives. The representation of driver input
will be given in abstract, vehicle independent
way; preferably in terms of the longitudinal and
lateral jerk
A model for driver perception, which is an ambitious additional function of the system. The scope of this module is to maintain a representation of the items the driver is aware of, which do not necessarily coincide with the real world. For this module, the feasibility to the expected accuracy is not sure (see risks section), but it does not cost much (here) and if it works it may provide additional very useful information (e.g., understanding that a mistake comes form a missing entity in driver world). The function combines eye gaze observations with information from the perception layer to determine which objects and points the driver watched. The gazed features are introduced into an alternative representation of the world (the driver mental model of the world) and then evolve according to rules that plausibly reproduce the assumptions a driver do about objects he is no longer looking at.
An interaction manager in the form of a variable plug-in. It will show how a variety of interactions, suited for different types of vehicles and different types of support, can all be built above the same unified situation assessment produced by the core application.