<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Evaluating a custom-made agent-based driving simulator</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Andreas Gregoriades</string-name>
          <email>andreas.gregoriades@cut.ac.cy</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Maria Pampaka</string-name>
          <email>maria.pampaka@manchester.ac.uk</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Cyprus University of Technology</institution>
          ,
          <addr-line>Limassol</addr-line>
          ,
          <country country="CY">Cyprus</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>The University of Manchester</institution>
          ,
          <addr-line>Manchester</addr-line>
          ,
          <country country="UK">UK</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>We present the design and evaluation of a custommade driving simulator, which was conducted through an experiment with users. Objective and self-reported measures of driving behaviour are used to validate the simulator. Objective data include situation awareness and workload measures, quantified with SAGAT and physiological estimates, while self-reported data focused on driving behaviour perceptions from a standardised driving style questionnaire. To evaluate the simulator, we firstly check that the synthetic environment does not overload the participants and enable them to have a sufficient level of situation awareness. Secondly, a correlation analysis is conducted between observed and self-reported driving style to examine the level of their covariance and similarity. Results showed that participants exhibited a similar driving behaviour as that reported with self-reports. This indicates that the simulator provides realistic driving conditions that encourage participants to behave in a realistic way.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>Driver related factors, according to the literature [Evans,
1991; NHTSA, 2015a] constitute the main cause of accident
in three out of five crashes, while they contribute to the
occurrence of 95% of all accidents. The National Highway
Traffic Safety Administration -NHTSA [NHTSA, 2015a],
classified driver-related accident causes into recognition
errors, decision errors, performance errors, and
nonperformance errors [Reason et al., 1990]. The most frequent
of these errors (more than 40%) are recognition errors which
include driver’s inattention, internal and external
distractions, and inadequate surveillance. Decision errors, such as
driving too fast under certain conditions, too fast for given
curves, false assumption of others’ actions, illegal
manoeuvres and misjudgement of gap between other vehicles or
others’ speed, account for more than 30% of accidents
[NHTSA, 2015b]. Performance related errors such as
overloading, poor directional control, etc., account for 11% of the
crashes. Sleep is the most common critical reason among
non-performance errors that accounts for about 7% of the
crashes. These categories however, are highly interrelated.
For instance, overloading will affect decision and recognition
that could lead to a crash.</p>
      <p>Driving style is directly linked to accidents. Many studies
analyse driving style, and particularly aggressive driving,
since this is highly related to crashes. Evaluating the effect of
different driving styles on accidents can be performed in
different ways, one of which is through driving simulation.
Alternatively surveys such as the Manchester Driving Style
questionnaire [Reason et al., 1990] can be used. Simulation
can be described as a method of reproducing a situation
similar to reality. To test driving style, it is necessary to design an
environment with identical stimuli to a real situation. In the
field of driving, simulations are used to generate situations
that produce the same response to participants as real-life
driving, without having the risks of injury. Driving
simulators have been developed ranging from small-scale such as
the NADS miniSim [FHA, 2013] to large scale models such
as the Daimler-Benz [Kading, 1995]. The advantages of
using simulators as a research tool include the design of
experiments that are easily replicated and the dynamic collection
of relevant drivers’ variables for safety analysis. Driving
simulators are designed for specific purposes and hence
require validation in order to produce correct results. However
due to the inherent complexity of such systems the prediction
of travellers and drivers’ behaviour is becoming increasingly
harder. For these reasons, agent-based simulation, which
adopts an individual-centered approach, is one of the most
relevant paradigms to design and implement such
applications [Mastio et al., 2018].</p>
      <p>Human driver behaviour modelling has been the subject of
many studies. The car-following model [Reuschel, 1950] has
been used to describe driver behaviour at the micro level and
is based on control theory (predictive control, optimal
control, etc.). The model expresses how vehicles follow one
another on a roadway, the minimum space and time gap
between them and the behaviour of the driver with regards to
keeping a “safe distance” from the leading vehicle, driving at
a desired speed, or choosing acceleration pattern to maintain
a comfortable range from the vehicle in front. The aim is to
mimic different driving styles, such as: Aggressive driving,
that has high accelerations and deceleration patterns with
almost no anticipation, eco driving style, where sufficient
anticipation is evident to avoid unnecessary acceleration and
braking, and normal driving, an intermediate driving style
between the above two styles. Alternative driver behaviour
modelling methods include neural networks, and fuzzy logic.
The latter models human perceptions by fuzzy sets and fuzzy
mathematics combined with knowledge-based logic. These
methods are often used to mimic processes that have
complicated mathematical models. Fuzzy logic has gained attention
in modelling tactical driver behaviour [Khaisongkram et al.,
2010], however, it requires a large number of experimental
data.</p>
      <p>The use of driving simulation in driver behaviour analysis is
considered essential due to the difficulty in eliminating
confounding effects on control measures in field experiments.
However, unrealistic simulation conditions may affect the
driving behaviour of participants in experiments which could
influence the validity of any study. Commercial driving
simulators, on the other hand do not provide the required level of
customisability necessary for researchers to design
experiments. Therefore, most experiments are designed using
custom made simulators. The main limitation of driving
simulation studies is that by removing the risk of harm to
participants, their driving behaviour may be altered. Therefore, the
conclusions made could be inaccurate. This paper contributes
in resolving this problem by addressing the following
research questions: (1) Does self-report driving behaviour of
participants differ from the observed objective driving
behaviour in the simulator? The assumption here is that drivers
should demonstrate similar driving style as their self-report
in the driving behaviour questionnaire. (2) Does the
simulator enable the drivers to have sufficient situation awareness?
The assumption is that a realistic driving simulator should
enable drivers to have a minimum situation awareness (SA),
which is the capability of understanding what is going on
around them and make decisions accordingly. (3) Is
aggressive driving associated with situation awareness? (4) Does
the simulation environment overload the drivers? This
examines if the workload of participants is within the accepted
levels. An indication of overloading in a normal driving
scenario could indicate a problem with the realism of the
simulator. (5) Do imprudent drivers consume more cognitive
resources than prudent drivers?
The paper is organised as follows. The next two sections
describe the literature relating to driver behaviour, SA,
workload and driver simulation. The next section describes the
process of designing a custom made driving simulator,
followed by a section that addresses its validation process. The
paper concludes with the main results of this study.</p>
    </sec>
    <sec id="sec-2">
      <title>2. Driving behaviour, Situation Awareness and workload</title>
      <p>Driving performance is associated with driving skills that are
manifested on driver behaviour which, in turn, affects
driving style. Driving skills include information processing and
motor skills, which improve with experience. Driving
behaviour describes driving habits that define the way a driver
chooses to drive [Lajunen et al., 2011]. The Driver
Behaviour Questionnaire (DBQ) [Reason et al., 1990] is one of the
most widely used instruments for measuring driving style.
According to Reason et al. [1990] driving errors and
violations are two different groups of behaviour, which is overall
categorised into violations, errors, slips and lapses.
Violations are deliberate deviations from practices believed
necessary to maintain safe operation of a potentially hazardous
system, while errors are defined as the failure of planned
actions to achieve intended outcomes. The research
instrument DBQ developed by Reason et al. [1990] considers this
classification. Slips and lapses refer to attention and memory
failures such as: attempt to drive away from the traffic lights
in wrong gear, forgetting where you park the car etc.
Violations are more serious and include close following vehicle
ahead (tailgating), speeding, risky overtaking etc. Errors
refer to behaviours such as failing to notice
pedestrianscrossing, missing Give Way signs etc. A further
classification [Lawton et al., 1997] divides violations into aggressive
violations, and ordinary violations, which are deliberate
deviations without aggressive behaviour.</p>
      <p>An important skill that affects driver safety is anticipation of
events. Experienced drivers can predict the traffic situation,
hence are ready when a hazardous event occurs. This ability
is referred to as driver’s situation awareness. Gaining
situation awareness involves perception and pattern recognition,
attention and comprehension, and decision-making [Ensley,
2012]. Hence, drivers identify, process, and comprehend the
critical information cues from the environment to predict
how future events could unfold. Drivers’ decision-making
process is not only based on the current environmental state,
but also extrapolates the current situation to future
projections. Well aware drivers analyse the current state of their
environment using multiple information sources, then predict
the next states. Situation awareness is an important feature in
driver safety. In normal condition an average driver has a
minimum level of situation awareness, which is required in
order to navigate the vehicle. This driver property can be
used as an indicator of the quality and realism of a driving
simulator, assuming that an unrealistic simulator will not
enable drivers to maintain minimum situation awareness. In
this study, self-reports of driver style gives an indication of
capability to maintain sufficient situation awareness, along
with driver behaviour. Therefore, a driver that reports in
DBQ that he/she is making a few errors and lapses is
expected to demonstrate sufficient situation awareness in the
simulator.</p>
      <p>Amongst the various methods for assessing drivers’ situation
awareness, this study employs the Situation Awareness
Global Assessment Technique SAGAT [Endsley, 2004;
Endsley &amp; Jones, 2012], which is a dynamic query technique
that questions participants’ recent memory of the situation
by freezing the simulation and hiding all information.
Measuring driver workload is of great significance for
improving the understanding of driver behaviours and
supporting the development of driver assistance systems. Workload
expresses the demands placed on the driver from secondary
tasks that could potentially interfere with the primary driving
task. Workload is defined as the competition in driver
resources (perceptual, cognitive, or physical) between the
driving task and a concurrent secondary task, occurring over that
task’s duration. Driving tasks for instance, require physical
and cognitive resources that are dynamically varied under
different driving conditions.</p>
      <p>There are three main methods to measure cognitive
workload: subjective, performance-based, and physiological.
Subjective knowledge acquisition techniques such as surveys,
interviews, and observations are commonly used to assess
cognition workload during tasks [Lehto et al., 1992].
Performance based measures are usually classified as either
primary task or secondary task performance. Depending on
the type of secondary task performed, objective measures of
workload include lane departures, and lateral deviations.
Additionally, performance based assessments include task
time, reaction time, accuracy, and error rate. Physiological
measures encompass audiology, cardiovascular, urodynamic,
gastrointestinal, respiratory, neurophysiology, and
ophthalmic physiology [Rusnock, 2018]. Using physiology is
advantageous, as assessment occurs continuously in real-time.
Physiological quantitative data of a subject’s state can be
linked to complex constructs such as mental workload,
fatigue, situation awareness, health, and emotion [Endsley,
1996; Kelly, 2003]. By assessing a user’s physiological state,
a designer will receive feedback that cannot be expressed
verbally or written by the user.</p>
      <p>In this study drivers’ workload was measured by
electroencephalography (EEG) and lateral deviation. The algorithm
implemented in the NeuroSky EEG device measures the
attentional resources consumed while the participant
performed a task. Data from the EEG was monitored on a
simulation time-step basis and automatically mapped to road
sections. The optimum level of driver performance is achieved
with a medium level of workload [Gregoriades et al., 2006],
which implies an EEG reading around 50%. Hence, as part
of the simulator validation, it is hypothesised that a normal
driving scenario should not overload the participants.
Overloading users in a simple scenario could indicate unrealistic
driving conditions that require participants to devote extra
cognitive resources to process unfamiliar task-related
situations (unexpected acceleration, steering etc).</p>
    </sec>
    <sec id="sec-3">
      <title>3 Driving simulation</title>
      <p>By definition, driving simulators are complex systems of
software and hardware which simulate real life
environments, behaviours and physical systems. Driving simulators
are used in a variety of applications, from training new
drivers in a safe environment to testing new car technologies.
They are often developed as part of traffic modelling and
driver behaviour research, prototype intelligent
transportation systems validation and training.</p>
      <p>There are different categories of driving simulations.
Microsimulation is widely considered as a method to study drivers’
behaviors, as in the example of parking choice simulators
PARKIT and PARKAGENT [Bonsall and Palmer, 2004].
Among micro-simulation programs, multi agent-based
modelling simulation environments, such as NetLogo [Sklar,
2007] and Archisim [Doniec et al., 2008], allow researchers
to investigate the connection between micro-level behaviors
of individuals, and macro-level patterns coming from their
interactions. Intelligent agents in multi agent systems
perform three functions: they perceive the dynamic conditions
from their environment, they perform actions, and reason to
interpret perceptions, solve problems, draw inferences, and
determine best course of actions.</p>
      <p>Traffic models are also classified into microscopic and
macroscopic models. The latter analyse traffic flow as a whole,
while the former focus on specific actions of the driver and
the physical laws of motion. Thus, in the case of
macroscopic models, overall shockwaves are analysed but do not
consider each car individually. Macroscopic models are not
suitable for driving behavior modeling since they do not
examine individual vehicle behaviours. Microscopic approaches
are more suitable for driving behaviour analysis and are
based on the models of: car-following [Brackstone and
McDonald, 1999], intelligent agents [Hidas, 2002], fuzzy
logic [McDonald et al., 1997], and cell transmission
[Daganzo, 1993] for simulation of traffic. Car-following theory
is an effective method to study the interaction between
vehicles in a microscopic simulator. The method used in this
work is based on a microscopic model utilising the
agentbased approach, with each vehicle represented by a software
agent having autonomy to behave based on some
predetermined rules that define basic driving styles.</p>
      <p>The aim of this work is to provide a simulation environment
that is fully customisable. This is necessary to eliminate the
effects of confounding variables from a driver behaviour
experiment due to unfamiliarity with the infrastructure.
Hence, it was necessary to model the road network in the
simulator prior to the analysis.</p>
    </sec>
    <sec id="sec-4">
      <title>4 Designing the driving simulator</title>
      <p>Much effort has been put in implementing driving simulators
in the last years [Biurrun-Quel et al., 2017; Rossetti et al.,
2013; Almeida et al., 2013; Gonçalves et al., 2012, 2013;
Alves et al., 2013]. These methods and tools allow the
representation of complex, realistic traffic situations for
evaluating specific traffic situations or testing new technological
applications and their influence on the driver. The simulator
was implemented using UNITY game engine which
appraises rapid application development through a component-based
software engineering approach. The driving environment
was designed using generic models that make up driving
conditions and road infrastructure. The modelling of the road
network was achieved by extracting a section of the Nicosia
road network from OpenStreetMap to generate a 3D model
of the cropped area in UNITY. The selection of the road
network was based on identified accident black spots
[Gregoriades, 2013] on the road network: roads suffering from
high accident rates. The assumption is that drivers consume
more cognitive resources at these locations hence they are
more susceptible to accidents. The selection of the car
models was based on car types and brands currently used in
Cyprus, in order to enhance the realism factor of the simulated
environment. Traffic conditions were specified though the
use of autonomous agent-based vehicles that are able to
navigate independently in the network based on pre-set driving
behaviours. The vehicle behaviours were based on a
preliminary analysis of traffic routing in the modelled traffic
network. The accident time statistics of the modelled section of
the road network were used to pinpoint the most critical time
on the selected black spot and accordingly replicate the
traffic conditions in the simulator.
Interactivity between the user and the simulator was realised
in Unity through C# scripting languages. Finally, the
simulator was designed with the capability to record in log files the
driving behaviour of users in real-time. Specifically, for each
simulation time-step the simulator records drivers’ headway,
lateral deviations, speed, acceleration and deceleration. Thus,
it enables the analysis of the data collected on a section-by
section basis. A screenshot of the simulator’s user interface
from the driver’s perspective is depicted in Figure 1 along
with the road network under study divided into 63 sections.
The main components of the simulator are: i) the Unity game
engine that controls the physical and environmental aspects
of the simulation; ii) the host vehicle controller that enables
the navigation of the host vehicle using the pedals and
steering wheel; iii) the data-logger that records the driving
behaviour of participants in experiments, along with additional
data relating to the traffic conditions; iv) the Multi screen
controller, that is responsible for the synchronization of the 4
screens in the cave automatic virtual environment (CAVE)
facility; v) the autonomous multi agent vehicle controller,
that controls the vehicle-agents in the simulation. This
component is responsible for recreating different traffic
conditions depending on the scenarios that need to be modelled.
Each autonomous vehicle agent dynamically decides its
route, avoids obstacles in its way and alters its speed
depending on the traffic. vi) The final component, is the road
infrastructure manager component is the facility used for the
development of the road network and the surrounding
environment.</p>
    </sec>
    <sec id="sec-5">
      <title>4.1 Autonomous vehicle agents</title>
      <p>In order to mimic a real driving experience, it is essential to
model all external environmental factors such as surrounding
vehicles dynamics, traffic lights and so on. The behaviour of
vehicles around a car is modelled based on the car-following
model and using the agent-based paradigm. The goal here is
for participants to experience the same feelings as if they
were in the real vehicle in naturalistic settings. This is
achieved by embedding each agent with a driving behaviour
model with the following features: path finding, speed
selection, obstacle avoidance, and acceleration and deceleration
models. Each vehicle agent interacts with other vehicle
agents and with infrastructure and traffic control agents as
shown in Figure 3. The exchanged messages enable each
agent to achieve its goals which are to avoid colliding with
other vehicles or the infrastructure, maintain a normal speed,
abide to the traffic regulations (drive on left lane etc.). The
route followed by each agent is dynamically defined,
however collectively all agents device routes that mimic realistic
traffic conditions.</p>
      <p>Infrastructure</p>
      <p>AGENT Ii</p>
      <p>Msg</p>
      <p>Msg
Msg</p>
      <p>AVGeEhNicTleAi</p>
      <p>AVGeEhNicTleAi</p>
      <p>Msg
Msg</p>
      <p>Traffic</p>
      <p>AcGoEnNtrToAli
Msg</p>
      <p>AVGeEhNicTleAi
Path finding refers to the process of finding the path to
follow in order to reach a destination or an objective. For
instance, an agent might be seeking the shortest path to a
destination, or the path with the smaller number of obstacles or
traffic. Path finding agents analyse all available paths, and
based on their objectives and restrictions, decide which one
to follow in a similar way as Navmesh method [He et al.,
2016]. Agents choose their path at runtime, hence deciding
the path based on what is currently happening around them.
For this study vehicle agents had no specific destination.
Their role is to move autonomously, in a non-predefined
path, in the road network under study, to mimic the traffic
condition at the specific network. The driving behaviour
model used was the car-following and the traffic volumes for
the particular part of the road network was specified based
on results from a previous study using the VISTA
macroscopic simulation model [Gregoriades et al., 2013].
For the path finding model to be operational, the road
network was modelled using waypoints (Figure 3). This enables
vehicle agents to know their location on the network, the
number of lanes at each point and the flow direction at each
lane. Waypoints represent all the possible paths a car can
take on the network. Waypoints are connected in a way that
each road lane has a predefined direction. Two types of
waypoints were used: simple and connector waypoints. Simple
waypoints are used by the vehicle agents as targets to follow
on a path, which resembles the road lane they are currently
on (Figure 4). Connector waypoints are used as simple
waypoints, with the added functionality to connect two different
road sections. For example, in an intersection, you exit the
road section you are currently on to enter a different section
on your path. In this case, these two sections are connected
with connector waypoints.</p>
      <p>Vehicle agents follow waypoints, to create a path to follow.
As mentioned before, the path is not specified at the
beginning of an agent’s life. Instead they start by defining the first
two target waypoints and each time a target is reached, the
agent changes its target to the next waypoint it had already
selected before, while choosing a new “next” target. The
selection of a target waypoint is done randomly, based on
where the target waypoint is connected to, along the
direction of the car. All possible connections from each waypoint
are stored in an array, and are dynamically accessed by the
vehicle agent at each time-step of the simulation.</p>
      <p>To implement this functionality, all waypoints were
prespecified on the network model in the form of invisible
event-based UNITY objects (Figure 4). Waypoints’ objects
act as placeholders of infrastructural information that
autonomic cars utilise. Vehicle agents access this information
when targeting a waypoint, or when they are in the process
of selecting a new target waypoint. In order to mimic the
driving conditions of the road network under study, the
autonomous agents’ controller assesses the number of vehicles
that are on the road at each time-step of the simulation and
accordingly increase or reduce the traffic volume so as to
replicate the expected traffic conditions.</p>
      <p>Vehicle steering and acceleration is performed after the
vehicle has selected its next target. As soon as a car has a new
target to reach, it starts calculating the steering angle and
acceleration required to effectively reach the target waypoint.
The steering angle is adjusted dynamically depending on the
position of the vehicle, its desired destination, and speed.
The steering functionality also addresses issues with regards
to obstacles or bottlenecks. In case of obstacles, the steering
to be applied is calculated based on the direction the car
needs to follow in order to avoid the obstacle. Acceleration
and speed are calculated based on distance to the preceding
vehicle. Lane change behaviour is stochastic.</p>
    </sec>
    <sec id="sec-6">
      <title>5. Validating the Simulator</title>
      <p>To be confident that the driving simulator correctly mimics
reality, two validation studies were conducted: a preliminary
validation and a more extensive human factors validation.
For the former, a number of professional taxi drivers were
asked to drive in a modelled road section in the VR settings
using the virtual host vehicle. Experts tested vehicle’s
steering sensitivity, acceleration and deceleration, and evaluated
the realism factor of the virtual environment. Initially,
several problems were identified with regards to vehicle steering,
acceleration and deceleration behaviours. In addition, the
early versions of the driving simulator suffered from low
refresh rate that led to motion sickness. In order to overcome
these issues, several modifications were performed to the
simulation scripts until a satisfactory vehicle behaviour was
achieved. The revised version of the simulator was
revalidated by 5 taxi drivers who all agreed that its behaviour was
realistic.</p>
      <p>The main simulator validation study aimed to identify the
suitability of the developed synthetic environment for human
factors analysis. Therefore, for this purpose an experiment
was conducted with participants in a hypothetical driving
scenario of a replica road section of Nicosia, with the same
infrastructure, traffic control and similar traffic conditions.
The simulation was performed in the VR cave with physical
steering wheel and petals. The research was conducted in
three stages: before, during and after the experiment. Before
the experiment, participants completed the Manchester
Driving Style questionnaire [Reason et al., 1990] and after the
driving experience questionnaire.</p>
      <p>Seventeen participants from the local population, with a
valid driver’s licence and either 20/20 vision or wearing
corrective glasses or lenses were involved in all stages of the
experiment. Given that driving skill is a significant factor in the
visual search strategies of drivers, and subsequently SA
[Underwood, 2007], the subjects selected had at least seven
years’ driving experience and were under 55 years old. Prior
to the experiment, participants were screened for colour
blindness. They were introduced to the various simulator
controls, made adjustments to the seat and were given a
fiveminutes training session in a road section other than the
section used in the experiment. The average age of participants
was 37.1 years and the gender distribution was 55% female
to 45% male.</p>
      <p>The main variables of interest in this study were workload
and Situation awareness (SA), hypothesising that a realistic
driving environment would enable participants to have
adequate level of SA and workload. During the experiment
participants were informed to drive in their normal driving style
in a pre-specified path in the road network. During the
experiment the simulator was collecting data regarding their
speed, acceleration, deceleration, EEG, headway, lateral
movements and breaking patterns. Upon completion of the
experiment participants completed the post-test questionnaire
about their driving experience in the simulator.
Postexperiment questionnaire addressed the following
dimensions: realism of the simulator’s general features, user
interface, ease of learning, capabilities, usefulness, ease of use,
how the simulator supports their situation awareness. Each
dimension was assessed on a 1-7 point response scale with 1
being negative ratings and 7 positive (figure 5). Results show
percentage of positive scores (scores of 5 and above).
Participants’ post-test response shown as percentage of
positive responses (above 4) in Figure 5, reveal that overall the
simulator was perceived as satisfactory in mimicking a
realistic driving situation. Moreover, the level of realism was
adequate (71%). However, in one case the participant
suffered of a minor incident of motion sickness.</p>
      <p>During the objective SA assessment, the simulator was
stopped at different points and participants were asked a
number of questions relevant to the driving situation to the
freezing point. Questionnaire responses from this process
were assessed on a 0-100 score and analysed by comparing
the actual situation with what the participants reported in
their results for the 3 freezing points. Answers from these
questions were analysed and an average collated score for all
questions designated the level of SA. Results showed that all
participants maintained an adequate level of SA with an
average score of (69.6%) in 3 freezing points. This was slightly
less than the subjective rating of participants as shown in
Figure 5 which was about 72%. However both indicate a
satisfactory level of SA. An additional evaluation of SA was
conducted using objective data from lateral deviations as
recorded by the simulator for all 63 road sections. These
were analysed to identify points of reduced SA due to sharp
lateral movements. This is phenotype behaviour related to
both overloading and low SA. From the diagram in Figure 6
it is evident that the deviations are relatively smooth which
indicates an acceptable level of SA. This, in turn, shows that
participants were actively engaged with the driving task.
Moreover, smooth deviations also indicate a relatively easy
task undertaken by participants. The three points with high
deviations (sections 23, 47 &amp; 58) represent the points with
the pre-set obstacles.</p>
      <p>Average leteral deviations per Section</p>
      <sec id="sec-6-1">
        <title>Driver Style Analysis</title>
        <p>To answer the first research question it was necessary to
investigate the extent of the association between participants’
self-reported and observed driving style. The assumption is
that, if self-reported and observed driving behaviours are
similar then the simulator provides the means for participants
to behave in a realistic manner and hence is considered as
valid.</p>
        <p>For the self-report stage, participants were asked prior to the
experiment, to fill in the Manchester Driving Style
questionnaire [Reason et al., 1990]. This aimed to elicit the driving
style of participants, along with demographic information.
Their observed driving style data were collected by the
simulator for each time-step of the simulator and assigned to
relevant road sections.</p>
        <p>Collected data underwent pre-processing and subsequently
analysed in SPSS to investigate the magnitude and
significance of the link between observed (simulator) and
selfreport (questionnaires) behaviours. Results in Table 1
indicate that aggression variable is correlated positively (and
significantly) with the variables “serious violation”, “errors”,
“lapses” and “aggressive acceleration”. Observed aggressive
Speed (O)
Aggression
(SR)
serious(SR)
errors(SR)
lapses(SR)
Aggressive
acceleration
(O)
acceleration was positively correlated with “errors”, “lapses”
and negatively correlated with “SA”. This means that
aggressive driving reduces drivers’ SA while it increases errors
and lapses. Essentially, our initial assumptions regarding
self-report driver behaviour and observed driver behaviour
were met. Specifically, self-reported aggressive behaviour
was found to be positively related to increased speed and
acceleration patterns in the simulator, hence indicating that
the simulation environment provides a realistic setting that
enables participants to drive in the same manner as they do
in their everyday life. This, as a result, is a promising
indicator towards the validity of the designed simulator</p>
        <p>Serious
(SR)</p>
      </sec>
      <sec id="sec-6-2">
        <title>Workload analysis</title>
        <p>To answer the fourth research question in relation to drivers’
workload, both EEG readings and lateral deviations per road
section (Figure 6 &amp;7) were utilised. The former is a
physiological objective measure and the latter a phenotype
objective measure. Given that the participants were driving in a
normal driving scenario with easy traffic conditions, the
assumption here was that there would be no overloading of
participants. If that occurred then it could indicate a problem
with the simulator’s level of realism. The hypothesis is that
drivers under optimum driving condition (no hazards and
low traffic flow) should not experience overloading. If this
occurs then it could designate that the simulator requires the
drivers to utilise extra cognitive resources to figure out how
to drive optimally in the synthetic environment. It is evident
from these results that on average all participants experience
an optimum level of workload. This was between 45 to 65 in
terms of EEG readings (Figure 6). Similar results are
depicted in the 3D analysis of the frequency distribution of EEG
ratings (Figure 7) per road section. This shows that the
majority of participants experience optimum level of workload
in all road sections. The EEG ratings are slightly high at the
first road sections but still within the acceptable range of
optimality. The second measure of workload utilised here is
lateral deviations. Results of Figure 6 show that there was no
significant deviations by participants and hence indicating
that the level of workload was optimal throughout the
experiment.
To answer the fifth research question, an analysis was
conducted to examine the link between drivers’ style and
workload. Correlation results showed that drivers who are
characterised as inattentive (i.e. commit high level of lapses) in
their self-report driving style experiences high readings of
EEG (Table 2). This confirms the assumption that careless
and inattentive drivers (imprudent) consume more cognitive
resources to engage with the driving scenarios.</p>
      </sec>
    </sec>
    <sec id="sec-7">
      <title>6. Conclusions</title>
      <p>The paper describes the design and validation of a custom
made driving simulator for driver behaviour analysis. The
developed driving simulator is agent-based with the
infrastructure being developed using a component based
approach. This allows the analyst to easily customize the road
infrastructure for what-if scenario analyses and the design of
experimental settings for a variety of scenarios.</p>
      <p>Results from the analysis of the data collected during the
experiment, revealed that the simulator satisfies the
minimum requirements for vehicle control since participants
maintain satisfactory level of SA and workload.
Additionally, results indicate that what the users experienced during
their interaction with the simulator and what they actually
denoted as their opinion in the post-test questionnaire point
to the same conclusion. Finally, self-reported driver style of
participants was correlated with observed behaviour during
the use of the simulator, pointing to the conclusion that the
artificial settings did not alter their driving style, hence it is
realistic and considered as valid.</p>
      <p>Limitations of this work concentrate on the simulator’s level
of immersion factors and the issue of motion sickness known
in VR settings. Simulated settings do not currently offer the
resolution of the real world, and so these may affect driving
behaviour and human factors analyses.
[Plochl, M., et al, 2007] Driver models in automobile
dynamics application, Vehicle Syst. Dynamics, 45, 699-741
[Reuschel, 1950], Vehicle Movements in a Platoon with
Uniform Acceleration or Deceleration of the Lead Vehicle,
Zeitschrift des Oesterreichischen Ingenieur-und
Architekten-Vereines, No.95, 59-62 and 73-77
[Khaisongkram, W.,et al, 2009]. Driver behavior
modeling and parameter identification during car-following
situation on urban road, in Proc. of 27thAnnual Conf.
of the Robotics Society of Japan.
[Lajunen, T., et al, 2011]. Self-report instruments and
methods. In B. E. Porter (Ed.), Handbook of Traffic
Psychology (pp. 43-59). London: Elsevier.
[Mastio M., et al, 2018], "Distributed Agent-Based Traffic
Simulations," in IEEE Intelligent Transportation Systems
Magazine, vol. 10, no. 1, pp. 145-156, Spring 2018.
[Lawton, R., et al, 1997] "The Role of Affect in Predicting
Social Behaviors: The Case of Road Traffic Violations.",
Blackwell Publishing Ltd,
[Bonsall, P., et al, 2004] Modelling drivers car parking
behaviour using data from a travel choice simulator.
Transportation Research Part C, 12, pp. 321–347.
[Sklar E, et al, 2007] NetLogo, a multi-agent simulation
environment. Artif. Life. 13(3)
[McDonald, M., et al, 1997] Development of a Fuzzy Logic
Based Microscopic Motorway Simulation Mode.
Proceeding of the ITSC97 Conference, Boston, U.S.A.
[Wu, J., et al, 2000] Fuzzy Sets and Systems for a Motorway
Microscopic Simulation Model. Fuzzy Sets and Systems,
special issue on fuzzy sets in traffic and transport systems
Vol. 116 (Issue 1)
[Hidas, P., 2002] Vehicle Interactions by Intelligent Agents,</p>
      <p>University of New South Wales.
[Daganzo C.F. 1993]. The cell-transmission model. Part I: A
simple dynamic representation of highway traffic.
California PATH Research report, UCB-ITS-PRR-93-7, Inst.</p>
      <p>Trans. Studies, University California, Berkeley
[Evans, L. 1991]. Traffic Safety and the Driver. Van
Nostrand Relnhold, New York .
[Rusnock C, et al 2018] Workload profiles: A continuous
measure of mental workload, International Journal of
Industrial Ergonomics, Volume 63, 2018,
[FHA, 2013] Making Driving Simulators More Useful for
Behavioral Research— Simulator Characteristics
Comparison and Model-Based Transformation, Technical Report,
USDT 2013</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [Alves,
          <string-name>
            <given-names>J.</given-names>
            <surname>Gonçalves</surname>
          </string-name>
          , et al 2013],
          <article-title>"Forward collision warning systems using heads-up displays: Testing usability of two new metaphors," IEEE Intelligent Vehicles Symposium (IV), Gold Coast</article-title>
          ,
          <string-name>
            <surname>QLD</surname>
          </string-name>
          ,
          <year>2013</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>6</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [Davenne D el al,
          <year>2012</year>
          ].,.
          <article-title>“Reliability of simulator driving tool for evaluation of sleepiness, fatigue and driving performance”</article-title>
          ,
          <source>Accident Analysis and Prevention</source>
          ,
          <volume>45</volume>
          , pp.
          <fpage>677</fpage>
          -
          <lpage>682</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <string-name>
            <surname>[Doniec</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Mandiau</surname>
          </string-name>
          , et al,
          <year>2008</year>
          <article-title>] “A behavioural multiagent model for road traffic simulation</article-title>
          ,
          <source>” Eng. Appl. Artif. Intell.</source>
          ,vol.
          <volume>21</volume>
          , no.
          <issue>8</issue>
          , pp.
          <fpage>1443</fpage>
          -
          <lpage>1454</lpage>
          ,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <string-name>
            <surname>[Eksler</surname>
            <given-names>V.</given-names>
          </string-name>
          et al,
          <year>2008</year>
          ] “
          <article-title>Regional analysis of road mortality in Europe”</article-title>
          , Public Health,
          <volume>122</volume>
          , pp.
          <fpage>826</fpage>
          -
          <lpage>837</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <string-name>
            <surname>Endlsey</surname>
            <given-names>M. R.</given-names>
          </string-name>
          (
          <year>2004</year>
          ).
          <article-title>Designing for situation awareness: an approach to user-centered design</article-title>
          , CRC press
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <string-name>
            <surname>[Endsley</surname>
            <given-names>MR</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jones</surname>
            <given-names>DG</given-names>
          </string-name>
          (
          <year>2012</year>
          )
          <article-title>Designing for situation awareness: an approach to human-centered design, 2nd edn</article-title>
          . Taylor and Francis, London
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [Gonçalves et al.,
          <year>2014</year>
          ]
          <article-title>"Testing Advanced Driver Assistance Systems with a serious-game-based human factors analysis suite,"</article-title>
          <source>IEEE Intelligent Vehicles Symposium Proceedings</source>
          , Dearborn, MI,
          <year>2014</year>
          , pp.
          <fpage>13</fpage>
          -
          <lpage>18</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [Gonçalves,
          <string-name>
            <surname>R. J. F.</surname>
          </string-name>
          et al 2012]
          <article-title>"IC-DEEP: A serious games based application to assess the ergonomics of in-vehicle information systems,"</article-title>
          <source>2012 15th International IEEE Conference on Intelligent Transportation Systems</source>
          , Anchorage,
          <string-name>
            <surname>AK</surname>
          </string-name>
          ,
          <year>2012</year>
          , pp.
          <fpage>1809</fpage>
          -
          <lpage>1814</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          <string-name>
            <given-names>[</given-names>
            <surname>Gregoriades</surname>
          </string-name>
          . A, et al,
          <year>2007</year>
          ] “
          <article-title>Workload prediction for improved design and reliability of complex systems</article-title>
          ,” Reliab. Eng. Syst. Saf.,
          <volume>39</volume>
          , n.4, pp.
          <fpage>530</fpage>
          -
          <lpage>549</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          <string-name>
            <surname>[Gregoriades</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , et al,
          <year>2013</year>
          <article-title>] Black spots identification through a Bayesian Networks quantification of accident risk index</article-title>
          .
          <source>Transportation Research Part C</source>
          <volume>28</volume>
          ,
          <fpage>28</fpage>
          -
          <lpage>43</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          <string-name>
            <surname>[Gregoriades</surname>
            <given-names>A</given-names>
          </string-name>
          , et al,
          <year>2010</year>
          ] “
          <article-title>Human-Centred Safety Analysis of Prospective Road Designs”</article-title>
          ,
          <source>IEEE Transactions on Systems, Man and Cybernetics</source>
          ,
          <string-name>
            <surname>Part</surname>
            <given-names>A</given-names>
          </string-name>
          , Vol
          <volume>40</volume>
          ,
          <issue>2</issue>
          , pp
          <fpage>236</fpage>
          -
          <lpage>250</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          <string-name>
            <surname>[He</surname>
            <given-names>Z</given-names>
          </string-name>
          ,
          <string-name>
            <surname>M.</surname>
          </string-name>
          et al,
          <year>2016</year>
          ]
          <article-title>"Research and application of pathfinding algorithm based on unity 3D,"</article-title>
          <source>2016 IEEE/ACIS 15th International Conference on Computer and Information Science (ICIS)</source>
          ,
          <year>Okayama</year>
          ,
          <year>2016</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>4</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          <string-name>
            <surname>[Reason</surname>
            <given-names>J</given-names>
          </string-name>
          , et al,
          <year>1990</year>
          ],
          <article-title>Errors and violations on the road: a real distinction?</article-title>
          <source>Ergonomics</source>
          <volume>33</volume>
          :
          <fpage>1315</fpage>
          -
          <lpage>1332</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          <source>[NHTSA</source>
          , 2015a],
          <source>The National Highway Traffic Safety Administration.</source>
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          <source>[NHTSA</source>
          , 2015b],
          <article-title>Critical Reasons for Crashes Investigated in the National Motor Vehicle Crash Causation Survey, The traffic safety facts</article-title>
          ,
          <source>DOT HS 812 115</source>
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <string-name>
            <surname>[Kading</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <year>1995</year>
          ].
          <article-title>The Advanced Daimler-Benz driving simulator</article-title>
          .
          <source>PC-8 Technical Paper No 9530012. Society of Automotive Engineers of Japan</source>
          , Inc
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>