Intellectual Tools to Prevent Road Accidents by Monitoring the Driver's Physiological State Anatoliy Melnyka,b, Yurii Morozova, Bohdan Havanoa and Petro Hupaloa a Lviv Polytechnic National University, Stepana Bandery St, 12, Lviv, 79013, Ukraine, b The John Paul II Catholic University of Lublin, Al. Racławickie 14, 20–950, Lublin, Poland Abstract Road accidents cause a large number of deaths and injuries every year. Creation of intelligent tools for preventing traffic accidents by monitoring the physiological state of the driver is an important step toward improving road safety. The paper proposes methods and tools to warn drivers about their physiological state dangerous for driving. This uses the sensory capabilities of modern wearable mobile devices, namely smart watches and bracelets. The method of notifying drivers of dangerous driving conditions includes the following steps: collecting real- time data from the driver of the vehicle using a mobile device worn by the driver; determining whether real-time data indicates hazardous driving conditions; informing the driver if real-time data indicate dangerous driving conditions. Real-time data can also be collected and used to study driver characteristics. These characteristics can be compared with previously collected data to help determine in real time whether the driver's behavior is normal and whether there is a dangerous situation. It is important to use serial wearable mobile devices with different types of data that are normalized before processing. Keywords 1 Road accident prevention, intellectual tools, wearable mobile devices, smart watches, bracelets, monitoring the driver's physiological state. 1. Introduction On average, about 10 people die on the roads of Ukraine every day [1]. During the period from 01.01.2021 to 30.11.2021 there were 170972 road accidents in which 2926 people died and 27296 people were injured. Thus, in the Lviv region there were 10616 road accidents in which 217 people died and 2238 people were injured [1]. During 2021, 11,744 road accidents were registered in the Lviv region, 228 people died and 2,419 were injured [2]. Every day in Lviv there are an average of 17 accidents and about 6 thousand per year [3]. The National Highway Traffic Safety Administration (NHTSA) estimates that in 2017, 91,000 police-reported crashes involved drowsy drivers. These crashes led to an estimated 50,000 people injured and nearly 800 deaths in the United States [4]. More than 30% of car crashes observed are due to the driver low vigilance level [5]. This number of accidents leads to negative socio-economic consequences. Increasingly, this is due to the human factor - driving fatigue, lack of sleep and falling asleep behind the wheel, illness, as well as driving under the influence of alcohol [6]. Driving requires significant perceptual, physical and cognitive requirements from the driver. The human nervous system has limitations in controlling a large amount of information, and the driver is a major factor in most accidents. Therefore, we need to define IntelITSIS’2022: 3rd International Workshop on Intelligent Information Technologies and Systems of Information Security, March 23–25, 2022, Khmelnytskyi, Ukraine EMAIL: aomelnyk@gmail.com (A. Melnyk); yurii.v.morozov@lpnu.ua (Y. Morozov); havano.bohdan@gmail.com (B. Havano); gypalo911@gmail.com (P. Hupalo) ORCID: 0000-0002-8981-0530 (A. Melnyk); 0000-0002-3670-411X (Y. Morozov); 0000-0002-2546-1917 (B. Havano); 0000-0003-4984- 3220 (P. Hupalo) ©️ 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) the limits beyond which the functional state of the driver can no longer satisfy safe driving. The impact of the human factor is especially great when it comes to public transport drivers, as their critical functional state can endanger the lives and health of many people. Therefore, constant monitoring of the functional state of the driver will increase safety and reduce mortality and injuries on the roads. 2. State-of-the-art People want to avoid car accidents, including those caused by their own physiological state, which can negatively affect their successful driving. Insurance companies would also like to have the tools to obtain a more reliable driver risk profile. Modern driving efficiency tracking technology is usually based on data collected from the car. See, for example [7], in which driving data is collected using the vehicle's on-board data collector (OBD). However, this technology is related to the vehicle itself, not the driver. Thus, the data collected relate only to the driver's driving style, if he is alone. In addition, since the information is collected through the vehicle itself, the data only applies to the movement and operation of the vehicle. Therefore, methods and means of monitoring the physiological state of drivers themselves and warning drivers in the event of potentially dangerous situations would be desirable. 3. Rationale for the approach Although it is assumed that the operation of the vehicle is a direct result of the driver's actions, there are many factors related to driving safety that cannot be taken into account. For example, distraction, drowsiness, stress, shortness of breath, dizziness, fever, etc. can affect the driver in such a way that the car-based system will not notice them. It is desirable to be able to detect such situations and warn drivers of any potential dangers. The proposed intelligent tools of preventing traffic accidents by monitoring the physiological state of the driver provide warning to drivers about dangerous driving conditions. They use the touch capabilities of wearable mobile technology, such as smart watches, to inform the driver in real time based on the data collected. The method of warning drivers about dangerous driving conditions proposed in the work includes the following steps: collecting real-time data from the driver of the vehicle (data is collected using a mobile device worn by the driver); determining whether real-time data indicates hazardous driving conditions; informing the driver if real-time data indicate dangerous driving conditions. Real-time data can also be collected and used to study driver characteristics. These characteristics can be compared with the data collected to help determine in real time whether the driver's behavior is normal and whether there is a dangerous situation. 4. Structure and functioning of the system As noted above, there is a need for technology, based on monitoring the physiological state of the driver to warn drivers of potentially dangerous situations. This technology would collect data from the driver himself and not from the vehicle he / she is driving. For example, as will be described in detail below, there are indicators of stress, fatigue, physiological disorders, difficulty breathing, dizziness, fever and so on. These figures can be obtained from data collected from the driver, which may not be obvious at the level of vehicle operation. The proposed tools use the capabilities of new technology of wearable devices to obtain data from the driver and the introduction of informing the driver about dangerous driving conditions. As will be described in detail below, wearable devices include a variety of sensors capable of collecting a huge amount of data from the user. In addition, drivers can wear wearable devices daily, including while driving. By comparison, a smartphone, for example, is often worn as an accessory and is remote from the driver when he / she is driving. In addition, the wearable device is usually worn on the user's wrist. The proposed wearable devices are used to collect (real-time) data on the physiological state of the driver and to provide him with real-time warnings about potentially dangerous situations. In addition to data on the physiological state of the driver, other important data may be collected, in particular, data on hand movements, and driving speed (e.g. detected by GPS positioning device of the wearable device, etc.). Therefore, the proposed tools can be used to notify the driver (using a wearable device and / or other available communication technology) about the need to slow down, the need for rest, call a doctor, etc. The structure of the driver's physiological state monitoring system is shown in Fig.1. The system operates in two ways to handle and alert the driver to critical changes in health. The first involves the connection between the wearable device and the smartphone. Processing and analysis of indicators in this case takes place on a smartphone and is based on preliminary data and analysis algorithms available at the time of measurement. This approach is needed when there is no possibility of constant access to the Internet. The use of this method is required for the driver to respond independently to his state of health and well-being. Figure 1: The structure of the system for monitoring the physiological state of the driver. The second type adds a server part to the system. Thanks to this combination, it is possible to save data not only on the local device of the driver; backup data; in-depth data analysis and continuous improvement of algorithms, as the server resource is much higher than in the smartphone. As well as a notification system for persons or organizations related to the driver. Both methods involve fulfilling their main task - monitoring the critical performance of the driver and instantaneous response to their change. It is assumed that wearable devices will include a number of sensors for health analysis. Among the most common indicators that are already collected thanks to modern devices are heart rate, ECG, skin temperature, pressure, and saturation (Fig. 2). Their collection is carried out in parallel and independently, which allows a comprehensive approach to the assessment of the driver's health. Examples of wearable devices are smart watches, fitness bracelets, and more. Smart watches that can be used according to these technologies are available from companies such as Motorola™ (for example, MOTO 360), Samsung™ (for example, Samsung Gear™), Apple™ (for example, Apple Watch™), Garmin (for example, Garmin Vivo Active™) etc. Figure 2: Example of measuring tools using a wearable device. 5. Sensory and informative capabilities of the smartwatch The following is a non-exhaustive list of smartwatch features that can be used to monitor a driver's physiological state. Different smart watches (or other suitable technology for wearing) have different capabilities: different sensors, and interactive user functions (such as voice commands, beeps / traffic signals, etc.). As an example, some smart watch technologies that can be used in the proposed tools include the following. 5.1. Electrodermal activity sensor A smartwatch uses one or more sensors located on the driver's body that can measure his physiological state. These sensors usually require contact with the driver's body to perform their functions, and are therefore called contact sensors. For example, one such contact sensor is an electrodermal activity sensor or EDA sensor, the use of which in mobile devices is described in [8,9]. EDA sensors measure the electrical characteristics of the skin. The electrical characteristics of the skin are controlled, at least in part, by the condition of the sweat glands in the skin, which in turn are regulated by the sympathetic nervous system. Thus, EDA sensors can assess sympathetic and nervous responses. More specifically, based on the sweat gland circuit, the EDA measures the strength of the change in skin's conductivity to electrical charge as a reflection of the sympathetic nervous system's response to sensation. This change is due to the activity of the eccrine sweat glands, which are innervated by the sympathetic branch of the autonomic nervous system. Reactions cannot be controlled instantly by the mind, so measurements reliably record the stress caused by external stimuli. EDA data are classified as a tonic - low-amplitude, low-frequency waveforms, typical in the relaxed state, or phase - high-amplitude and high-frequency waveforms that occur 1-3 seconds after sensory stimulation. This phase measurement is a skin conductivity reaction (SCR). Sensory stimuli can be auditory, visual, olfactory, tactile, or vestibular (vertigo, imbalance). In the context of this technique, EDA sensors can be used to collect real-time data that indicate the driver's stress level, for example, the driver is in a relaxed state or a state of high anxiety / stress. For example, based on the above sensory stimulus, the driver may be under stress when he / she drives aggressively, is in an accident, or witnesses an accident or other dangerous situation on the road (for example, other motorists drive incorrectly or exceed speeding, slippery road, garbage on the road, etc.) - based on the above-described sensory stimulus. 5.2. Pulse oximeters and heart rate sensors Other contact sensors useful in this technique include heart rate monitors and heart rate sensors [10]. A pulse oximeter often measures the level of oxygen in a person's blood using a sensor placed on the body while measuring the heart rate. Similarly, a heart rate sensor measures your heart rate (usually in beats per minute). As for modern methods, the driver's heart rate can be an indicator of stress. As described above, driver stress can be caused by his driving condition and / or the condition of others, road conditions and hazards, and so on. These vital indicators are also indicators of disorders, such as when the driver has a disease such as heart failure. It is assumed that under normal driving conditions the user will have a constant pulse. Thus, sudden changes (jumps or decreases) in the driver's heart rate can be indicators of dangerous driving conditions. In addition, the pulse oximeter measures the level of oxygen in human blood by determining the pulmonary physiological characteristics of the person, which are indicators of such disorders as shortness of breath, dizziness and the like. That is, the departure of these characteristics beyond safe limits means an immediate threat to driving safety. 5.3. Temperature sensors Also wearable devices may include human body temperature sensors [11,12]. When contact measurement of skin temperature, it is about 24 ° C, i.e. very different from body temperature. It should be noted that, as a rule, skin temperature changes according to changes in human body temperature. Measuring a person's temperature can detect states of hypothermia and hyperthermia. Hypothermia (low temperature) can indicate a stroke, nervous system damage, alcohol and drug use, septic condition, liver and gallbladder dysfunction, and so on. Hyperthermia (fever) can indicate food poisoning, blood diseases, chronic inflammation in the body, internal bleeding and more. The departure of human body temperature from safe limits does not mean an immediate threat to driving safety. But in combination with other characteristics of the human body can provide more information for analysis. For example, high heart rate and low body temperature often mean hypothermia, but with high body temperature you can detect fever. 6. The principle of analysis of vital signs After receiving periodic data sent from all sensors, the critical event detection algorithm must analyze them in real time and alert the driver in case of an emergency. To check for abnormal situations, it is recommended to use the Early Warning Score manual (EWS). EWS is a guide based on vital signs, such as 𝑉, and it is used to monitor the level of criticality of the 𝑝 functional state of the human body. For each vital sign 𝑣 ∈ 𝑉, the collected record 𝑟𝑖 ∈ 𝑡𝑅𝑣 is compared with the normal range for calculating the estimate 𝑠𝑖 between 0 and 3; 0 means a normal record, where other values indicate abnormal situations with increasing severity relative to the increase in score. 𝑝 𝑝 Therefore, for 𝑡𝑅𝑣 the set of records of 𝑡𝑆𝑣 = [𝑠1 , 𝑠2 , ⋯ , 𝑠𝜏 ] is calculated. Table 1 shows a scale for assessing the vital indicators of the human state, which is based on one of the most widely used EWS manuals, developed in the UK and distributed worldwide, and is called National EWS (NEWS). After calculating the scores set for each period, it is suggested to directly analyze the collected records and warn the driver. 𝑝 For example, if the set score calculated during the period is 𝑡𝑆𝑣 = [0,2, 1,2], then the total score is 5 (i.e. 0 + 2 + 1 + 2), means that the indicators cross the established allowable limits with a certain criticality level (see Table 2). Table 1 Scale for assessing vital signs of the driver’s state. Vital sign Score 3 2 1 0 1 2 3 Respiratory rate ≤8 9-11 12-20 21- ≥25 (breaths per 24 minute) 𝑺𝒑𝑶𝟐 , ≤91 92-93 94- ≥96 (%) 95 Systolic blood ≤90 91-100 101- 111- ≥ 220 pressure (mm 110 219 Hg Art.) Heart rate, ≤40 41- 51-90 91- 111- ≥131 (beats per 50 110 130 minute) Body ≤35.0 35.1- 36.1- 37.1- 38.1- ≥ 39.1 temperature, 36.0 37.0 38.0 39.0 (°C) Table 2 Dependence of the level of criticality and recommendations on the overall assessment of vital signs of the driver’s state. Aggregate score Critical level Driver's state 0 1 Satisfactory 1-4 2 Dangerous Find out the reason for the deterioration of indicators. 5+ or at least one of the 3 Critical indicators 3 Requires the driver's reaction to restore a satisfactory state. 7. Operation of the driver warning system To ensure the functioning of the warning system about the dangerous state of the driver, you can use the monitoring tools described in [13,14]. The wearable device periodically transmits data to the smartphone. The transmission period can be set on a wearable device. Different data may be transmitted at different intervals and with different volumes. For example, heart rate data can be transmitted every 15s, 5 impressions in one transmission. Data on the level of oxygen in human blood can be transmitted once every 5 minutes, 1 reading per transmission. On the smartphone, the data is received by an application that works according to the algorithm of Fig.3. After initialization, the application enters the mode of continuous data acquisition from the wearable device. The data obtained from wearable devices from different manufacturers differ in presentation format, so they must be reduced to a universal format. Then the data is clustered according to a given table of criteria. If all the data is in satisfactory condition, then nothing happens. If there is data in other groups, an integrated assessment of the deviation from the norm is calculated. If the integrated assessment is critical, the driver is immediately informed by light and sound signals to draw the driver's attention to the critical state of the indications of the functional state of his body. In the threatening case, a message is displayed on the screen about the deviation of the readings of the functional state of the driver's body. Figure 3: Algorithm for informing the driver about his functional state. 8. Conclusion In our opinion, continuous monitoring of health will soon become one of the leading areas of health care. The proposed solution relates to this area and may have an immediate technological and social impact to assist health and public care systems in preventing accidents by monitoring the driver's physiological state and alerting him to threatening or critical conditions. We have developed intellectual tools that warn drivers about their physiological states that are dangerous to drive. The intellectual tools are based on wireless devices and are designed to monitor the driver's physiological state while driving. The driver’s physiological state is estimated on the continuous analysis of the following parameters: the oxygen level in the blood, pulse frequency, and temperature. These data from the smart devices wirelessly connected to the driver’s smartphone will be processed. The results of processing data are shown to the driver. The monitoring system includes two components: a sensor network of wireless smart sensors and a linked smartphone with a special mobile application. Optionally, implemented in the mobile application algorithms process results of monitoring, avoiding duplicates, and store them for future usage on the server. It allows a better representation of driver’s indicators and the ability to overview his indicators history which is highly important for insurance companies and doctors. To check for abnormal situations, we used the Early Warning Score manual. After calculating the scores set for each period, it is suggested to directly analyze the collected records and warn the driver. Dependence of the level of criticality and recommendations on the overall assessment of vital signs of the driver’s state is also proposed. 9. References [1] Accident statistics in Ukraine, 2021, [online] Available: https://patrol.police.gov.ua/statystyka/. [2] Lviv Region Patrol Police, 2021, [online] Available: https://www.facebook.com/lvivpolice/photos/ a.1485055981796169/28472060833890689. [3] Articles, 2021, [online] Available: https://zaxid.net/statti_tag50974/. [4] Drowsy Driving: Avoid Falling Asleep Behind the Wheel | NHTSA, 2021, [online] Available: https://www.nhtsa.gov/risky-driving/drowsy-driving. [5] P. S. Rau, “Drowsy driver detection and warning system for commercial vehicle drivers: Field operational test design, data analyses, and progress,” in Proc. Nat. Highway Traffic Safety Admin., U.S., SAE Government, Ind. Meeting, Apr. 2005, paper 05-0192. [6] WHO. Risk factors for traffic injuries, 2021, [online] Available: https://www.who.int/violence_injury_prevention/road_traffic/activities/roadsafety_training_man ual_unit_2.pdf. [7] U.S. Patent No. 8,090,598 to Bauer et al., entitled “Monitoring System for Determining and Transferring the Cost of Insurance”. [8] A. Affanni, A. Piras, R. Rinaldo and P. Zontone, "Dual channel Electrodermal activity sensor for motion artifact removal in car drivers’ stress detection," 2019 IEEE Sensors Applications Symposium (SAS), 2019, pp. 1-6, doi: 10.1109/SAS.2019.8706023. [9] M. F. Canabal, J. A. Miranda, J. M. Lanza-Gutiérrez, A. I. Pérez Garcilópez and C. López-Ongil, "Electrodermal Activity Smart Sensor Integration in a Wearable Affective Computing System," 2020 XXXV Conference on Design of Circuits and Integrated Systems (DCIS), 2020, pp. 1-6, doi: 10.1109/DCIS51330.2020.9268662. [10] A. Chatterjee, S. C. Kolay, M. Bandyopadhyay and S. Chattopadhyay, "IoT Based Pulse Oximeter System," 2021 International Conference on Industrial Electronics Research and Applications (ICIERA), 2021, pp. 1-4, doi: 10.1109/ICIERA53202.2021.9726732. [11] S. Marathe, D. Zeeshan, T. Thomas and S. Vidhya, "A Wireless Patient Monitoring System using Integrated ECG module, Pulse Oximeter, Blood Pressure and Temperature Sensor," 2019 International Conference on Vision Towards Emerging Trends in Communication and Networking (ViTECoN), 2019, pp. 1-4, doi: 10.1109/ViTECoN.2019.8899541. [12] S. Jun and J. Lee, "Towards Measuring Body Temperature Using COTS Mobile Devices," 2021 International Conference on Information and Communication Technology Convergence (ICTC), 2021, pp. 1847-1849, doi: 10.1109/ICTC52510.2021.9621105. [13] A. Melnyk, Y. Morozov, P. Hupalo, B. Havano. (2021) “HealthyLungs: Mobile Applications for Round-the-Clock Remote Monitoring of Lung Function in Patients with COVID-19”, 2021 11th International Conference on Advanced Computer Information Technologies, ACIT 2021 – Proceedings, 15 September 2021, Pages 554-559. doi: 10.1109/ACIT52158.2021.9548596 [14] A. Melnyk, Y. Morozov, P. Hupalo, B. Havano. HealthSupervisor: Mobile Application for Round- the-Clock Remote Monitoring of the Human Functional State, 2021, CEUR-WS, 2853, pp. 24–37. URL: http://ceur-ws.org/Vol-2853/keynote3.pdf.