=Paper=
{{Paper
|id=Vol-2804/short3
|storemode=property
|title=Unsupervised-based Framework for Aged Worker’s Stress Detection (short paper)
|pdfUrl=https://ceur-ws.org/Vol-2804/short3.pdf
|volume=Vol-2804
|authors=Gabriele Rescio,Alessandro Leone,Pietro Siciliano
|dblpUrl=https://dblp.org/rec/conf/aiia/RescioLS20
}}
==Unsupervised-based Framework for Aged Worker’s Stress Detection (short paper)==
https://ceur-ws.org/Vol-2804/short3.pdf
Unsupervised-based framework for aged worker’s stress
detection
Gabriele Rescioa, Alessandro Leonea and Pietro Sicilianoa
a
National Research Council of Italy, Institute for Microelectronics and Microsystems, Via Monteroni, c/o Campus
Ecotekne, Palazzina A3, Lecce, Italy
Abstract
The new industry 4.0 paradigm provides for totally automated and interconnected industrial
production processes that require greater human-machine interaction. This involves the onset
of new problems related to the stress evaluation of the aged worker which is found to operate
in new and more complex work contexts. In literature several works for human stress
detection are presented, they use above supervised machine learning with high accuracy level
detection, but needed a complicated training phase. Moreover, a relevant issue in the field of
stress detection lies in the model validation, indeed the commonly questionnaires used to
record perceived stress levels are prone to subjective inaccuracies. To reduce this limitation,
in this paper an unsupervised machine learning based stress detection system, in which the
labels from perceived stress levels are not needed, is presented. It analyses heart rate,
galvanic skin response and electrooculogram signals, relevant for the detection of excessive
stress and cognitive load. The developed architecture software has been experimented in
laboratory contest and preliminary obtained results appear promising.
Keywords 1
Stress monitoring, Unsupervised learning, Wearable sensors.
1. Introduction
The Industry 4.0 represents a fundamental shift in manufacturing that will transform both
processes and people. This emphasizes the need to integrate the human operator in a socially
sustainable way into the new production paradigms [1]. Nowadays, it is common to design a
workplace according to the requirements of the operator and the increase of automation (e.g., cobots,
exoskeletons) is reducing the physical effort of workers. However, the activities of the operator 4.0
will entail an increased share of complex cognitive tasks. This is a challenge especially for aged
workers who have to operate in this new context. Therefore, more attention should be paid to
emerging stressors and risks to mental health [2].
Over the years, the most widespread technique to assess human stress concerns the use of
psychological questionnaires. The main limitation of this type of survey is that it does not allow real-
time and continuous monitoring. This makes it impossible to identify the causes and work tasks that
are stressful for the worker. A potential solution to overcome this problem is to consider bodily
responses, such as physiological signals (e.g. skin temperature, breathing rate, heart rate, …) in terms
of stress index. These parameters can be measured accurately and continuously through ambient or
wearable sensors-based devices. The ambient sensor-based monitoring technologies (e.g. Ultra-Wide
Band systems, video cameras, …) are less intrusive with respect to the wearable devices, but they
require a complex and ad hoc design of environments to allow accurate measurements. In the last
years, through the development of miniaturized technologies, minimally invasive wearable sensors
Workshop on Artificial Intelligence for an Ageing Society (AIxAS 2020), November 25–27, 2020
EMAIL: gabriele.rescio@cnr.it (G. Rescio); alessandro.leone@cnr.it (A. Leone); pietro.siciliano@le.imm.cnr.it (P. Siciliano)
ORCID: 0000-0003-3374-2433 (G. Rescio); 0000-0002-8970-3313 (A. Leone); 0000-0002-1312-4593 (P. Siciliano)
© 2020 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
have been realized and this has favored their diffusion in the monitoring of health parameters and to
prevent dangerous events in the healthcare field [3, 4].
In literature, several studies adopt supervised machine learning technique for the stress event
classification. The main analysis physiological parameters are heart rate, breathing rate, blood
pressure, skin temperature, galvanic skin response and recently electrooculogram. The accuracy
values measured for these works is very high, up to 99% [5]. However, they required a complex
training phase, based on labelled dataset of simulated events that it could be inaccurate and different
from real-world data [6]. This issue could be reduced using an unsupervised approach. Unsupervised
learning technique is particularly suitable for analyzing data that doesn’t need to be associated to a
labelling information. It can also be used in many applications related to data mining, such as the
analysis of electrophysiological signals. In literature few works deal with unsupervised detection of
mental stress and to the best of our knowledge, deep learning framework based on unsupervised
detection of acute mental stress, in industrial context, using heart rate (HR), electrical dermal activity
(EDA) and electrooculogram (EOG) signals has not been investigated before.
This paper proposes a stress assessment system based on an unsupervised learning technique to
monitor the operator’s mental load with the aim to increase his wellbeing in the working environment.
It exploits minimally wearable devices and combines HR, EDA and EOG measurements in order to
automatize stress detection conditions. To analyze the framework performance, traditional mental
stress test and simulations of manufacturing activities were performed.
2. Materials and methods
In this section the hardware-software architecture and the procedure used for the physiological
signal acquisition are described.
2.1. Multisensory platform
The main computational steps of the software architecture are a) pre-processing, b) calibration, c)
feature extraction and d) Clustering. The algorithmic framework has been developed using three low
invasive commercial wearable devices: 1) the ES_R glasses produced by J!NS MEME [7] for EOG
signals, 2) the E4 wristband produced by Empatica [8] for EDA signal, 3) the Bioharness 3.0 chest
strap band produced by Zephyr [9] for HR signal. The HR signal is also acquired from the E4 device,
but it was verified that its accuracy is lower than that of Bioharness. Therefore, it was preferred to use
3 devices (shown in Figure.1) instead of 2.
Figure 1.Wearable devices. a) J!NS MEME ES_R glasses, b) Zephyr Bioharness 3.0 chest strap, c)
Empatica E4 wristband.
The J!NS MEME ES_R acquires the EOG data through three-point electrooculography sensors
placed on the nose pads glasses for the evaluation of vertical and horizontal EOG components. The
glasses send data through a low energy Bluetooth connection. The autonomy of the rechargeable
lithium-ion battery is about 16 hours. The sample rate for the presented work is 50Hz.
The Empatica E4 wristband acquires EDA signal to measure the sympathetic nervous system
arousal through two metal electrodes placed on the skin. The sample rate of EDA signal is 4Hz. The
wristband is equipped with a low energy Bluetooth connection and a proprietary mobile App, for the
visualization and the storing of data. Moreover, it is possible to record the data using the on-board
memory.
The Bioharness 3.0 band measures the HR through electrode sensors housed within a chest trap
and an electronic module integrated in the trap. The Bioharness 3.0 operates either in streaming mode
for real-time data viewing on a mobile device using Bluetooth low energy or in recording mode using
its internal memory.
2.2. Procedure
In order to analyse the signals coming from the device and to test the framework software, a data
collection has been defined in controlled (simulated) conditions by involving 7 female and 4 male
volunteers (age 31.3±5.8). They were recruited at multidisciplinary laboratory of Engineering
department of Ancona’s University within a funded project by the Italian ministry for Education and
Research. All participants were asked to wear the aforementioned wearable devices and perform five tasks,
separated by a rest period of two minutes. The experiment procedures are detailed in [10] and they have
provided the simulation of manufacturing activities, by using LEGO bricks and a toolbox. Moreover,
the volunteers accomplished mental arithmetic test, frequently used in the literature to induce stress
[11]. At the end of the experiments, users were asked to judge the cognitive load generated by all tasks. The
mental arithmetic tests and complex LEGO assembly, without instructions, resulted the most stressful
unanimously. So, more than 90 stress and 130 rest events were identified and accordingly labelled for
the performance evaluation. During the data collection each user wore the three devices: the data
coming from the wristband and chest strap devices were stored on the memory integrated on board,
while the glasses sent the data to an embedded PC. The data so acquired were synchronized and
analyzed in off-line mode through the mathematical computing software Mathworks Matlab [12].
2.3. Data Analysis
In this section it is detailed the analysis of the raw data, acquired during the laboratory tests
described in the previous paragraph. The main computational steps of the software framework
architecture are listed below.
2.3.1 Pre-processing and calibration
The aims of data pre-processing step are to reduce environmental electrical noise and the artefacts
due to device movements. Regarding EDA signal, the technique implemented in Kocielnik et al [13]
was used to remove signal artefacts. This technique eliminates the anomalous signals with respect to
the shape characteristics of conductance spikes in normal skin. In particular, it was eliminated the
signals that present changes more than 20% per second or decreased more than 10% per second. The
EDA signal consists of phasic and tonic components. The tonic component is related to the slow
changing baseline levels and to the individual background characteristics. While the phasic
component shows the fast skin conductance response due to a stimulus measured over a short duration
of time and it can be event related. To extract the phasic component a Butterworth band-pass filter
with cut frequencies between 0.16Hz and 2Hz was implemented.
Considering the EOG signals, the Empirical Mode Decomposition (EMD) technique [14] and the
glasses acceleration data were examined in order to reduce the noise and the artifact due to the sudden
head movements or to the contact losses between glasses electrodes and nose skin. Basically, it was
eliminated the Vertical and Horizontal EOG peak signals in correspondence of excessive
accelerometer module peaks, calculated by using the data coming from the tri-axial accelerometer
integrated in the glasses. Ten EMD IMFs from the EOG data were extracted, and the IMFs 2 to 6
empirically were selected and added to obtain the filtered data.
For HR signal the interval between successive heart beats (RR intervals) coming from the chest
trap were considered and the artefacts were filtered from RR sequences acquired, analyzing relative
trusted RR intervals.
In order to correctly handle pre-processed data and to reduce the inter-individual variability
between different users, a calibration procedure was accomplished by recovering the initial conditions
after the device mounting. To obtain this, the baseline of EDA, HR and EOG signals was calculated
as the mean of the data acquired for 10 seconds, while the user was in rest condition, in the first phase
of each data collection trial. Then the baseline value was used to normalize the pre-processed acquired
signals.
2.3.2 Feature extraction and selection
For the purpose of stress events assessment, attention was focused on several time domain
features, commonly used in the analysis of stress detection [15-19] and the most significant features
were chosen. They were calculated within a sliding window of 120 seconds for HR signal and of 5
seconds for both EDA and EOG signals.
For EDA signal the following features were extracted: 1) the peak amplitude sum, 2) peak energy
sum, 3) peak rise time sum, 4) normalized root mean square, 5) the mean of the first signal derivative,
6) the standard deviation of the first signal derivative, 7) variance.
Regarding EOG signal the calculated features are: 1) eye blink frequency, 2) the averages of EOG
peak-to-peak voltage amplitude of vertical and horizontal signal components 3) Variance of vertical
and horizontal EOG components.
Finally, for the HR the heart rate variability was calculated and then the following features were
extracted: 1) Root Mean Square of the Successive Differences, 2) Standard Deviation of NN intervals,
3) the proportion of interval differences of successive NN intervals greater than 50 ms, 4) mean heart
rate.
To reduce the complexity of the signal processing and to enhance the performance of the system, a
feature selection technique was applied. For this phase, the Laplacian score was used, it is suitable for
ranking features for unsupervised learning [20]. In the Figure 2 the ranking for the signals is reported,
the x-axis shows the number equivalent to the features described above, while the y-axis represents
the features score; for each EOG signal feature the vertical and horizontal components were
considered.
Figure 2. Feature rank for a) EDA signal, b) HR signal, c) EOG signal (V and H indicate the Vertical and
Horizontal EOG components). The x-axis shows the numbers equivalent to the feature lists described
in this paragraph.
In this way it was possible to choose the most important features for each signal. In particular, the
following features were selected: 3), 2) and 1) for EDA; 3) and 2) for EOG; 1) and 3) for HR (the
numbers refer to the feature lists described above).
2.3.3 Clustering
After the features were extracted for all three devices, the feature vectors were applied as input to
the classifier, implemented to distinguish the rest condition from stress condition, for each signal. The
algorithm chosen for this phase is the commonly used and effective K-means clustering [21]. It is an
unsupervised algorithm that identifies k number of centroids, through an iterative procedure of
positioning optimization, and then allocates each data sample to the nearest cluster, based on the
distance between the sample and centroid. The algorithm uses specific types of distance metric
(sqeuclidean, cityblock, cosine, correlation, ...), whose can be chosen to improve the clustering
performance.
3 Results
To evaluate the performance of the stress detection system, compared to different distance
functions of k-means, the average sensitivity and specificity according to the equation (1) and (2) [10]
were calculated. The sensitivity indicates the ability of the framework to recognize a human stress
condition, while the specificity is the ability to reduce the false event detections.
= (1)
= (2)
where TP (True Positive) represents that a stress phase occurs and the algorithm detects it; FP
(False Positive) indicates that a stress phase does not occur and the algorithm activates an alarm; TN
(True Negative) means that a stress phase does not occurs and the algorithm does not detect it; FN
(False negative) implies that stress phase occurs but the algorithm does not detect it.
Moreover, the average clustering accuracy was calculated in percentage according to the equation
(3).
Accuracy = (3)
These metrics were calculated for each device, considering the squeclidean, cityblock and cosine
distance metrics. The acquired data was split and labelled in “no-stress” and “stress” phases as described in
“Procedure” section. In Table 1 the results obtained with the best k-means parameters setting were
reported for each signal.
Table 1
Detection performance in terms of Sensitivity, Specificity and Accuracy
Signal Sensitivity (%) Specificity (%) Accuracy (%)
HR 72.9 65.3 70.6
EDA 76.8 70.6 74.6
EOG 67.5 63.4 63.7
As it is possible to verify from the table, the best performance was achieved by the EDA signals.
To increase the reliability and efficiency of the stress detection system, the final prediction is given by
a combination of the three outputs classifiers by using a weighted voting approach in which the EDA
classifier has higher coefficient value (according to its clustering performance on a validation set). So,
summing up all weighted votes and by selecting the class with the highest aggregate the new values
for sensitivity, specificity and accuracy are 83.3%, 74.3% and 79.7% respectively. The algorithm was
also tested by using the other unselected features, but performance has not improved. These
performances are slightly lower than those of existing approaches for stress detection [5], but the use
of unsupervised approaches makes the proposed system suitable in a real industrial context.
Moreover, minimally invasive devices have been tested that also allow to identify stress conditions
due to a high cognitive load, through the EOG signal.
4 Conclusions
The paper proposes a multisensory and unsupervised machine learning based system for the
stressful condition detection in industrial 4.0 context. The system is mainly addressed to elderly
workers, who have to operate in this new context. It could be useful to select the most suitable tasks
for each worker, optimizing its performance and wellbeing. The heart rate, electrodermal activity and
electrooculogram signals were considered for the evaluation of cognitive load and stress. The
developed framework was tested on data acquired during laboratory tests, in which mental arithmetic
test and manufacturing activities by using LEGO bricks and a toolbox were simulated. The
unsupervised k-means approach was used, and the results show good performance in terms of
specificity, sensitivity and accuracy (about 80%) and it does not need the training phase, that it is
usually inaccurate and heavy. Future researches will be addressed to investigate other unsupervised
machine learning classifiers in order to improve the clustering performance. Moreover, the system
will be test in a real industrial case study
5 Acknowledgements
This work has been carried out within the PON REACT project, with the co-financing of the
European Union – FESR or FSE, PON Research and Innovation 2014-2020. Authors would like to
thank the staff of Università Politecnica delle Marche, for supporting the data collection.
6 References
[1] D. Romero, J. Stahre, T. Wuest, O. Noran, P. Bernus, Å. Fast-Berglund, and D. Gorecky,
“Towards an operator 4.0 typology: a human-centric perspective on the fourth industrial
revolution technologies” In International conference on computers & industrial engineering, pp.
1–11, 2016.
[2] T. Arai, R. Kato, and M. Fujita, “Assessment of operator stress induced by robot collaboration in
assembly” CIRP Annals - Manufacturing Technology vol. 59, pp. 5–8, 2010.
[3] B. Andò, S. Baglio, C. O. Lombardo, V. Marletta, “A Multi-Sensor Data Fusion Approach for
ADL and Fall classification”, IEEE Transactions on Instrumentation and Measurement, vol. 65,
N.9, pp. 1960-1967, 2016.
[4] A. Leone, G. Rescio, L. Giampetruzzi and P. Siciliano, "Smart EMG-based Socks for Leg
Muscles Contraction Assessment," 2019 IEEE International Symposium on Measurements &
Networking (M&N), 2019, pp. 1-6.
[5] Y. S. Can, .t Arnrich, C. Ersoy, “Stress detection in daily life scenarios using smart phones and
wearable sensors: A survey”, Journal of Biomedical Informatics, Volume 92, 2019, 103139.
[6] E., Kallio J., Mäntyjärvi J., Kyllönen V., Lindholm M., Gimel’farb G. (2017) “Unsupervised
Stress Detection Algorithm and Experiments with Real Life Data”. In: Oliveira E., Gama J., Vale
Z., Lopes Cardoso H. (eds) Progress in Artificial Intelligence. EPIA 2017. Lecture Notes in
Computer Science, vol 10423. Springer, Cham.
[7] https://jins-meme.com/.
[8] https://www.empatica.com/.
[9] https://www.zephyranywhere.com/.
[10] A. Leone, G. Rescio, P. Siciliano, A. Papetti, A. Brunzini and M. Germani, "Multi sensors
platform for stress monitoring of workers in smart manufacturing context," 2020 IEEE
International Instrumentation and Measurement Technology Conference (I2MTC), Dubrovnik,
Croatia, 2020, pp. 1-5.
[11] S. Serino, A. Matic, D. Giakoumis, G. Lopez, P. Cipresso (eds) “Pervasive Computing
Paradigms for Mental Health”. MindCare 2015. Communications in Computer and Information
Science, vol 604. Springer, Cham.
[12] https://www.mathworks.com
[13] R. Kocielnik, N. Sidorova, F. M. Maggi, M. Ouwerkerk and J. H. D. M. Westerink, "Smart
technologies for long-term stress monitoring at work," Proceedings of the 26th IEEE
International Symposium on Computer-Based Medical Systems, Porto, 2013, pp. 53-58.
[14] A. Zeiler, R. Faltermeier, I. R. Keck, A. M. Tomé, C. G. Puntonet and E. W. Lang, "Empirical
Mode Decomposition - an introduction," The 2010 International Joint Conference on Neural
Networks (IJCNN), Barcelona, 2010, pp. 1-8.
[15] R. R. Singh, S. Conjeti, R. Banerjee, “A comparative evaluation of neural network classifiers for
stress level analysis of automotive drivers using physiological signals”, Biomedical Signal
Processing and Control, Vol 8, Issue 6, 2013, Pages 740-754.
[16] T. Ogawa, M. Takahashi, and R. Kawashima, “Human cognitive control mode estimation using
JINS MEME” IFAC-PapersOnLine vol. 49(19), pp. 331-336, 2016.
[17] H. Ledger, “The effect cognitive load has on eye blinking” The Plymouth Student Scientist, vol.
6(1), pp. 206-223, 2013.
[18] Kim, Hye-Geum et al. “Stress and Heart Rate Variability: A Meta-Analysis and Review of the
Literature.” Psychiatry investigation vol. 15,3 (2018): 235-245.
[19] R. Zangróniz, A. Martínez-Rodrigo, J.M. Pastor, M.T. López, A. Fernández-Caballero.
“Electrodermal Activity Sensor for Classification of Calm/Distress Condition”. Sensors (Basel).
2017;17(10):2324. Published 2017 Oct 12.
[20] Xiaofei He, Deng Cai, and Partha Niyogi. 2005. “Laplacian score for feature selection”. In
Proceedings of the 18th International Conference on Neural Information Processing Systems
(NIPS'05). MIT Press, Cambridge, MA, USA, 507–514.
[21] K. P. Sinaga and M. Yang, "Unsupervised K-Means Clustering Algorithm," in IEEE Access, vol.
8, pp. 80716-80727, 2020.