=Paper=
{{Paper
|id=Vol-3156/short41
|storemode=property
|title=Intellectual tools to prevent road accidents by monitoring the driver's physiological state
|pdfUrl=https://ceur-ws.org/Vol-3156/short41.pdf
|volume=Vol-3156
|authors=Anatoliy Melnyk,Yurii Morozov,Bohdan Havano,Petro Hupalo
|dblpUrl=https://dblp.org/rec/conf/intelitsis/MelnykMHH22
}}
==Intellectual tools to prevent road accidents by monitoring the driver's physiological state==
https://ceur-ws.org/Vol-3156/short41.pdf
Intellectual Tools to Prevent Road Accidents by Monitoring the
Driver's Physiological State
Anatoliy Melnyka,b, Yurii Morozova, Bohdan Havanoa and Petro Hupaloa
a
Lviv Polytechnic National University, Stepana Bandery St, 12, Lviv, 79013, Ukraine,
b
The John Paul II Catholic University of Lublin, Al. Racławickie 14, 20–950, Lublin, Poland
Abstract
Road accidents cause a large number of deaths and injuries every year. Creation of intelligent
tools for preventing traffic accidents by monitoring the physiological state of the driver is an
important step toward improving road safety. The paper proposes methods and tools to warn
drivers about their physiological state dangerous for driving. This uses the sensory capabilities
of modern wearable mobile devices, namely smart watches and bracelets. The method of
notifying drivers of dangerous driving conditions includes the following steps: collecting real-
time data from the driver of the vehicle using a mobile device worn by the driver; determining
whether real-time data indicates hazardous driving conditions; informing the driver if real-time
data indicate dangerous driving conditions. Real-time data can also be collected and used to
study driver characteristics. These characteristics can be compared with previously collected
data to help determine in real time whether the driver's behavior is normal and whether there
is a dangerous situation. It is important to use serial wearable mobile devices with different
types of data that are normalized before processing.
Keywords 1
Road accident prevention, intellectual tools, wearable mobile devices, smart watches,
bracelets, monitoring the driver's physiological state.
1. Introduction
On average, about 10 people die on the roads of Ukraine every day [1]. During the period from
01.01.2021 to 30.11.2021 there were 170972 road accidents in which 2926 people died and 27296
people were injured. Thus, in the Lviv region there were 10616 road accidents in which 217 people died
and 2238 people were injured [1].
During 2021, 11,744 road accidents were registered in the Lviv region, 228 people died and 2,419
were injured [2]. Every day in Lviv there are an average of 17 accidents and about 6 thousand per year
[3].
The National Highway Traffic Safety Administration (NHTSA) estimates that in 2017, 91,000
police-reported crashes involved drowsy drivers. These crashes led to an estimated 50,000 people
injured and nearly 800 deaths in the United States [4]. More than 30% of car crashes observed are due
to the driver low vigilance level [5].
This number of accidents leads to negative socio-economic consequences. Increasingly, this is due
to the human factor - driving fatigue, lack of sleep and falling asleep behind the wheel, illness, as well
as driving under the influence of alcohol [6]. Driving requires significant perceptual, physical and
cognitive requirements from the driver. The human nervous system has limitations in controlling a large
amount of information, and the driver is a major factor in most accidents. Therefore, we need to define
IntelITSIS’2022: 3rd International Workshop on Intelligent Information Technologies and Systems of Information Security, March 23–25,
2022, Khmelnytskyi, Ukraine
EMAIL: aomelnyk@gmail.com (A. Melnyk); yurii.v.morozov@lpnu.ua (Y. Morozov); havano.bohdan@gmail.com (B. Havano);
gypalo911@gmail.com (P. Hupalo)
ORCID: 0000-0002-8981-0530 (A. Melnyk); 0000-0002-3670-411X (Y. Morozov); 0000-0002-2546-1917 (B. Havano); 0000-0003-4984-
3220 (P. Hupalo)
©️ 2022 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
the limits beyond which the functional state of the driver can no longer satisfy safe driving. The impact
of the human factor is especially great when it comes to public transport drivers, as their critical
functional state can endanger the lives and health of many people. Therefore, constant monitoring of
the functional state of the driver will increase safety and reduce mortality and injuries on the roads.
2. State-of-the-art
People want to avoid car accidents, including those caused by their own physiological state, which
can negatively affect their successful driving. Insurance companies would also like to have the tools to
obtain a more reliable driver risk profile.
Modern driving efficiency tracking technology is usually based on data collected from the car. See,
for example [7], in which driving data is collected using the vehicle's on-board data collector (OBD).
However, this technology is related to the vehicle itself, not the driver. Thus, the data collected relate
only to the driver's driving style, if he is alone.
In addition, since the information is collected through the vehicle itself, the data only applies to the
movement and operation of the vehicle.
Therefore, methods and means of monitoring the physiological state of drivers themselves and
warning drivers in the event of potentially dangerous situations would be desirable.
3. Rationale for the approach
Although it is assumed that the operation of the vehicle is a direct result of the driver's actions, there
are many factors related to driving safety that cannot be taken into account. For example, distraction,
drowsiness, stress, shortness of breath, dizziness, fever, etc. can affect the driver in such a way that the
car-based system will not notice them. It is desirable to be able to detect such situations and warn drivers
of any potential dangers.
The proposed intelligent tools of preventing traffic accidents by monitoring the physiological state
of the driver provide warning to drivers about dangerous driving conditions. They use the touch
capabilities of wearable mobile technology, such as smart watches, to inform the driver in real time
based on the data collected. The method of warning drivers about dangerous driving conditions
proposed in the work includes the following steps: collecting real-time data from the driver of the
vehicle (data is collected using a mobile device worn by the driver); determining whether real-time data
indicates hazardous driving conditions; informing the driver if real-time data indicate dangerous driving
conditions.
Real-time data can also be collected and used to study driver characteristics. These characteristics
can be compared with the data collected to help determine in real time whether the driver's behavior is
normal and whether there is a dangerous situation.
4. Structure and functioning of the system
As noted above, there is a need for technology, based on monitoring the physiological state of the
driver to warn drivers of potentially dangerous situations. This technology would collect data from the
driver himself and not from the vehicle he / she is driving. For example, as will be described in detail
below, there are indicators of stress, fatigue, physiological disorders, difficulty breathing, dizziness,
fever and so on. These figures can be obtained from data collected from the driver, which may not be
obvious at the level of vehicle operation.
The proposed tools use the capabilities of new technology of wearable devices to obtain data from
the driver and the introduction of informing the driver about dangerous driving conditions. As will be
described in detail below, wearable devices include a variety of sensors capable of collecting a huge
amount of data from the user. In addition, drivers can wear wearable devices daily, including while
driving. By comparison, a smartphone, for example, is often worn as an accessory and is remote from
the driver when he / she is driving. In addition, the wearable device is usually worn on the user's wrist.
The proposed wearable devices are used to collect (real-time) data on the physiological state of the
driver and to provide him with real-time warnings about potentially dangerous situations.
In addition to data on the physiological state of the driver, other important data may be collected, in
particular, data on hand movements, and driving speed (e.g. detected by GPS positioning device of the
wearable device, etc.). Therefore, the proposed tools can be used to notify the driver (using a wearable
device and / or other available communication technology) about the need to slow down, the need for
rest, call a doctor, etc.
The structure of the driver's physiological state monitoring system is shown in Fig.1. The system
operates in two ways to handle and alert the driver to critical changes in health. The first involves the
connection between the wearable device and the smartphone. Processing and analysis of indicators in
this case takes place on a smartphone and is based on preliminary data and analysis algorithms available
at the time of measurement. This approach is needed when there is no possibility of constant access to
the Internet. The use of this method is required for the driver to respond independently to his state of
health and well-being.
Figure 1: The structure of the system for monitoring the physiological state of the driver.
The second type adds a server part to the system. Thanks to this combination, it is possible to save
data not only on the local device of the driver; backup data; in-depth data analysis and continuous
improvement of algorithms, as the server resource is much higher than in the smartphone. As well as a
notification system for persons or organizations related to the driver.
Both methods involve fulfilling their main task - monitoring the critical performance of the driver
and instantaneous response to their change.
It is assumed that wearable devices will include a number of sensors for health analysis. Among the
most common indicators that are already collected thanks to modern devices are heart rate, ECG, skin
temperature, pressure, and saturation (Fig. 2).
Their collection is carried out in parallel and independently, which allows a comprehensive approach
to the assessment of the driver's health.
Examples of wearable devices are smart watches, fitness bracelets, and more.
Smart watches that can be used according to these technologies are available from companies such
as Motorola™ (for example, MOTO 360), Samsung™ (for example, Samsung Gear™), Apple™ (for
example, Apple Watch™), Garmin (for example, Garmin Vivo Active™) etc.
Figure 2: Example of measuring tools using a wearable device.
5. Sensory and informative capabilities of the smartwatch
The following is a non-exhaustive list of smartwatch features that can be used to monitor a driver's
physiological state. Different smart watches (or other suitable technology for wearing) have different
capabilities: different sensors, and interactive user functions (such as voice commands, beeps / traffic
signals, etc.). As an example, some smart watch technologies that can be used in the proposed tools
include the following.
5.1. Electrodermal activity sensor
A smartwatch uses one or more sensors located on the driver's body that can measure his
physiological state. These sensors usually require contact with the driver's body to perform their
functions, and are therefore called contact sensors. For example, one such contact sensor is an
electrodermal activity sensor or EDA sensor, the use of which in mobile devices is described in [8,9].
EDA sensors measure the electrical characteristics of the skin. The electrical characteristics of the skin
are controlled, at least in part, by the condition of the sweat glands in the skin, which in turn are
regulated by the sympathetic nervous system. Thus, EDA sensors can assess sympathetic and nervous
responses.
More specifically, based on the sweat gland circuit, the EDA measures the strength of the change in
skin's conductivity to electrical charge as a reflection of the sympathetic nervous system's response to
sensation. This change is due to the activity of the eccrine sweat glands, which are innervated by the
sympathetic branch of the autonomic nervous system. Reactions cannot be controlled instantly by the
mind, so measurements reliably record the stress caused by external stimuli.
EDA data are classified as a tonic - low-amplitude, low-frequency waveforms, typical in the relaxed
state, or phase - high-amplitude and high-frequency waveforms that occur 1-3 seconds after sensory
stimulation. This phase measurement is a skin conductivity reaction (SCR). Sensory stimuli can be
auditory, visual, olfactory, tactile, or vestibular (vertigo, imbalance).
In the context of this technique, EDA sensors can be used to collect real-time data that indicate the
driver's stress level, for example, the driver is in a relaxed state or a state of high anxiety / stress.
For example, based on the above sensory stimulus, the driver may be under stress when he / she
drives aggressively, is in an accident, or witnesses an accident or other dangerous situation on the road
(for example, other motorists drive incorrectly or exceed speeding, slippery road, garbage on the road,
etc.) - based on the above-described sensory stimulus.
5.2. Pulse oximeters and heart rate sensors
Other contact sensors useful in this technique include heart rate monitors and heart rate sensors [10].
A pulse oximeter often measures the level of oxygen in a person's blood using a sensor placed on the
body while measuring the heart rate. Similarly, a heart rate sensor measures your heart rate (usually in
beats per minute).
As for modern methods, the driver's heart rate can be an indicator of stress. As described above,
driver stress can be caused by his driving condition and / or the condition of others, road conditions and
hazards, and so on. These vital indicators are also indicators of disorders, such as when the driver has a
disease such as heart failure. It is assumed that under normal driving conditions the user will have a
constant pulse. Thus, sudden changes (jumps or decreases) in the driver's heart rate can be indicators of
dangerous driving conditions.
In addition, the pulse oximeter measures the level of oxygen in human blood by determining the
pulmonary physiological characteristics of the person, which are indicators of such disorders as
shortness of breath, dizziness and the like. That is, the departure of these characteristics beyond safe
limits means an immediate threat to driving safety.
5.3. Temperature sensors
Also wearable devices may include human body temperature sensors [11,12].
When contact measurement of skin temperature, it is about 24 ° C, i.e. very different from body
temperature. It should be noted that, as a rule, skin temperature changes according to changes in human
body temperature.
Measuring a person's temperature can detect states of hypothermia and hyperthermia. Hypothermia
(low temperature) can indicate a stroke, nervous system damage, alcohol and drug use, septic condition,
liver and gallbladder dysfunction, and so on.
Hyperthermia (fever) can indicate food poisoning, blood diseases, chronic inflammation in the body,
internal bleeding and more.
The departure of human body temperature from safe limits does not mean an immediate threat to
driving safety. But in combination with other characteristics of the human body can provide more
information for analysis.
For example, high heart rate and low body temperature often mean hypothermia, but with high body
temperature you can detect fever.
6. The principle of analysis of vital signs
After receiving periodic data sent from all sensors, the critical event detection algorithm must
analyze them in real time and alert the driver in case of an emergency. To check for abnormal situations,
it is recommended to use the Early Warning Score manual (EWS).
EWS is a guide based on vital signs, such as 𝑉, and it is used to monitor the level of criticality of the
𝑝
functional state of the human body. For each vital sign 𝑣 ∈ 𝑉, the collected record 𝑟𝑖 ∈ 𝑡𝑅𝑣 is compared
with the normal range for calculating the estimate 𝑠𝑖 between 0 and 3; 0 means a normal record, where
other values indicate abnormal situations with increasing severity relative to the increase in score.
𝑝 𝑝
Therefore, for 𝑡𝑅𝑣 the set of records of 𝑡𝑆𝑣 = [𝑠1 , 𝑠2 , ⋯ , 𝑠𝜏 ] is calculated.
Table 1 shows a scale for assessing the vital indicators of the human state, which is based on one of
the most widely used EWS manuals, developed in the UK and distributed worldwide, and is called
National EWS (NEWS).
After calculating the scores set for each period, it is suggested to directly analyze the collected
records and warn the driver.
𝑝
For example, if the set score calculated during the period is 𝑡𝑆𝑣 = [0,2, 1,2], then the total score is
5 (i.e. 0 + 2 + 1 + 2), means that the indicators cross the established allowable limits with a certain
criticality level (see Table 2).
Table 1
Scale for assessing vital signs of the driver’s state.
Vital sign Score
3 2 1 0 1 2 3
Respiratory rate ≤8 9-11 12-20 21- ≥25
(breaths per 24
minute)
𝑺𝒑𝑶𝟐 , ≤91 92-93 94- ≥96
(%) 95
Systolic blood ≤90 91-100 101- 111- ≥ 220
pressure (mm 110 219
Hg Art.)
Heart rate, ≤40 41- 51-90 91- 111- ≥131
(beats per 50 110 130
minute)
Body ≤35.0 35.1- 36.1- 37.1- 38.1- ≥ 39.1
temperature, 36.0 37.0 38.0 39.0
(°C)
Table 2
Dependence of the level of criticality and recommendations on the overall assessment of vital signs
of the driver’s state.
Aggregate score Critical level Driver's state
0 1 Satisfactory
1-4 2 Dangerous
Find out the reason for the deterioration of
indicators.
5+ or at least one of the 3 Critical
indicators 3 Requires the driver's reaction to restore a
satisfactory state.
7. Operation of the driver warning system
To ensure the functioning of the warning system about the dangerous state of the driver, you can use
the monitoring tools described in [13,14]. The wearable device periodically transmits data to the
smartphone. The transmission period can be set on a wearable device. Different data may be transmitted
at different intervals and with different volumes. For example, heart rate data can be transmitted every
15s, 5 impressions in one transmission. Data on the level of oxygen in human blood can be transmitted
once every 5 minutes, 1 reading per transmission. On the smartphone, the data is received by an
application that works according to the algorithm of Fig.3.
After initialization, the application enters the mode of continuous data acquisition from the wearable
device. The data obtained from wearable devices from different manufacturers differ in presentation
format, so they must be reduced to a universal format. Then the data is clustered according to a given
table of criteria.
If all the data is in satisfactory condition, then nothing happens. If there is data in other groups, an
integrated assessment of the deviation from the norm is calculated. If the integrated assessment is
critical, the driver is immediately informed by light and sound signals to draw the driver's attention to
the critical state of the indications of the functional state of his body. In the threatening case, a message
is displayed on the screen about the deviation of the readings of the functional state of the driver's body.
Figure 3: Algorithm for informing the driver about his functional state.
8. Conclusion
In our opinion, continuous monitoring of health will soon become one of the leading areas of health
care. The proposed solution relates to this area and may have an immediate technological and social
impact to assist health and public care systems in preventing accidents by monitoring the driver's
physiological state and alerting him to threatening or critical conditions.
We have developed intellectual tools that warn drivers about their physiological states that are
dangerous to drive. The intellectual tools are based on wireless devices and are designed to monitor the
driver's physiological state while driving. The driver’s physiological state is estimated on the continuous
analysis of the following parameters: the oxygen level in the blood, pulse frequency, and temperature.
These data from the smart devices wirelessly connected to the driver’s smartphone will be processed.
The results of processing data are shown to the driver. The monitoring system includes two components:
a sensor network of wireless smart sensors and a linked smartphone with a special mobile application.
Optionally, implemented in the mobile application algorithms process results of monitoring,
avoiding duplicates, and store them for future usage on the server. It allows a better representation of
driver’s indicators and the ability to overview his indicators history which is highly important for
insurance companies and doctors.
To check for abnormal situations, we used the Early Warning Score manual. After calculating the
scores set for each period, it is suggested to directly analyze the collected records and warn the driver.
Dependence of the level of criticality and recommendations on the overall assessment of vital signs of
the driver’s state is also proposed.
9. References
[1] Accident statistics in Ukraine, 2021, [online] Available: https://patrol.police.gov.ua/statystyka/.
[2] Lviv Region Patrol Police, 2021, [online] Available:
https://www.facebook.com/lvivpolice/photos/ a.1485055981796169/28472060833890689.
[3] Articles, 2021, [online] Available: https://zaxid.net/statti_tag50974/.
[4] Drowsy Driving: Avoid Falling Asleep Behind the Wheel | NHTSA, 2021, [online] Available:
https://www.nhtsa.gov/risky-driving/drowsy-driving.
[5] P. S. Rau, “Drowsy driver detection and warning system for commercial vehicle drivers: Field
operational test design, data analyses, and progress,” in Proc. Nat. Highway Traffic Safety Admin.,
U.S., SAE Government, Ind. Meeting, Apr. 2005, paper 05-0192.
[6] WHO. Risk factors for traffic injuries, 2021, [online] Available:
https://www.who.int/violence_injury_prevention/road_traffic/activities/roadsafety_training_man
ual_unit_2.pdf.
[7] U.S. Patent No. 8,090,598 to Bauer et al., entitled “Monitoring System for Determining and
Transferring the Cost of Insurance”.
[8] A. Affanni, A. Piras, R. Rinaldo and P. Zontone, "Dual channel Electrodermal activity sensor for
motion artifact removal in car drivers’ stress detection," 2019 IEEE Sensors Applications
Symposium (SAS), 2019, pp. 1-6, doi: 10.1109/SAS.2019.8706023.
[9] M. F. Canabal, J. A. Miranda, J. M. Lanza-Gutiérrez, A. I. Pérez Garcilópez and C. López-Ongil,
"Electrodermal Activity Smart Sensor Integration in a Wearable Affective Computing System,"
2020 XXXV Conference on Design of Circuits and Integrated Systems (DCIS), 2020, pp. 1-6, doi:
10.1109/DCIS51330.2020.9268662.
[10] A. Chatterjee, S. C. Kolay, M. Bandyopadhyay and S. Chattopadhyay, "IoT Based Pulse Oximeter
System," 2021 International Conference on Industrial Electronics Research and Applications
(ICIERA), 2021, pp. 1-4, doi: 10.1109/ICIERA53202.2021.9726732.
[11] S. Marathe, D. Zeeshan, T. Thomas and S. Vidhya, "A Wireless Patient Monitoring System using
Integrated ECG module, Pulse Oximeter, Blood Pressure and Temperature Sensor," 2019
International Conference on Vision Towards Emerging Trends in Communication and Networking
(ViTECoN), 2019, pp. 1-4, doi: 10.1109/ViTECoN.2019.8899541.
[12] S. Jun and J. Lee, "Towards Measuring Body Temperature Using COTS Mobile Devices," 2021
International Conference on Information and Communication Technology Convergence (ICTC),
2021, pp. 1847-1849, doi: 10.1109/ICTC52510.2021.9621105.
[13] A. Melnyk, Y. Morozov, P. Hupalo, B. Havano. (2021) “HealthyLungs: Mobile Applications for
Round-the-Clock Remote Monitoring of Lung Function in Patients with COVID-19”, 2021 11th
International Conference on Advanced Computer Information Technologies, ACIT 2021 –
Proceedings, 15 September 2021, Pages 554-559. doi: 10.1109/ACIT52158.2021.9548596
[14] A. Melnyk, Y. Morozov, P. Hupalo, B. Havano. HealthSupervisor: Mobile Application for Round-
the-Clock Remote Monitoring of the Human Functional State, 2021, CEUR-WS, 2853, pp. 24–37.
URL: http://ceur-ws.org/Vol-2853/keynote3.pdf.