Comparison of Machine Learning approaches for Stress Detection from Wearable Sensors Data

Abstract

Stress is a prevalent and growing phenomenon in the modern world potentially leading to significant repercussions on both physical and mental health. The analysis of physiological signals, collected from wearable sensors, has emerged as a promising approach to predicting and managing stress. Methods based on machine learning techniques have been defined in the literature and achieved promising results by using handcrafted features extracted from the signal. However, there is no consensus on the list of features, while deep learning approaches that overcomes the problem require significant computational power and a large amount of data. In this paper, we present a comprehensive view of the most common representative machine learning algorithms applied to the stress detection domain by giving a reference point for both academia and industry professionals in this application field. This study considers fragments of signals without extracting any features and uses a public dataset, WESAD, that contains high-resolution physiological, including blood volume pulse, electrocardiogram and electromyogram. The data collected from 15 subjects during a lab study are heterogeneous and characterized by diferent frequencies and noises due to some devices. After preprocessing, we assess the performance of ten machine learning algorithms belonging to four models (tree, ensemble, linear and neighbours) on the WESAD by facing the problem as binary (stress/no-stress) and multiclass (baseline, stress, and amusement) classifications. Our results, evaluated in terms of classical metrics, show that Random Forest outperforms the others in binary and multi-class approaches.

Keywords

• eol>Physiological Signals
• Binary and multi-class classification
• Wearable Sensor Data
• time series

1. Introduction

tonomic Nervous System, allow us to detect and monitor stress. Hovsepian et al. [ 4 ] pioneered the stress detection Stress is a non-specific body reaction to any demand by using physiological signals. Both faced the problem upon it. Its efects influence overall behaviour, well-being, as a binary classification problem, whereas Gjoreski et and potential personal and professional successes [ 1 ]. al. [ 5 ] aimed at distinguishing diferent levels of stress Chronic stress may give rise to significant physical and (no stress versus low stress versus high stress). Such mental health issues, such as cancer, cardiovascular dis- bioignals can be captured non-invasively by wearable ease, depression, and diabetes. It is an increasingly preva- devices, such as smartphones and smartwatches, comlent and pervasive phenomenon in the modern world: monly used among people. Such devices can monitor more than 50% of all work-related ill health cases in some physiological parameters, such as Blood Volume 2020/21 are due to stress [ 2 ]. Assessments based on psy- Pulse (BVP), Electrodermal Activity (EDA), temperature chologically designed questions, such as the Perceived (TEMP), and heart rate (HR) etc. In the scenario of stress Stress Scale (PSS) [ 3 ], are frequently used to detect stress. detection, machine learning and deep learning methodHowever, these methods may be time-consuming, psy- ologies achieve promising results by analyzing these data. chologically invasive and lack reliability. Therefore, the These approaches include support vector machines, randefinition of non-invasive approaches for rapid and accu- dom forest and k-nearest neighbours and use handcrafted rate stress detection influences the quality and wellness features extracted from the pre-processed signal in order of people’s lives: managing stress before it causes health to reduce the data noises [ 6 ]. Moreover, no consensus issues is fundamental. In the literature, it has been demon- on the list of features to extract from physiological data strated that physiological signals, a response to the Au- has been reached [ 7 ]. To solve the problem, advanced Ital-IA 2024: 4th National Conference on Artificial Intelligence, orga- deep learning approaches have been applied since they nized by CINI, May 29-30, 2024, Naples, Italy have the ability to automatically comprehend patterns * Corresponding author. and, thus extract features. Nevertheless, these require † These authors contributed equally. significant computational power and a large amount of $ [e-mail] (M. Quadrini); data. The appropriate machine learning algorithm choice [e-mail] (D. Falcone); for a particular problem task is not trivial: no single clasmic0h0e0l0a-.q00u0a3d-r0in53i@9-u0n29ic0a(mM.i.t Q(Gu.adGreinrai)r;d0)000-0003-0539-0290 sifier works best across all possible scenarios, as stated (G. Gerard) by no free lunch theorem states [ 8 ]. To the best of our © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License knowledge, no scientific work compares machine learnAttribution 4.0 International (CC BY 4.0). ing methods for stress detection on the same datasets without feature extraction or dimensionality reduction.

In this paper, we present a comprehensive view of the most common representative machine learning algorithms applied to the stress detection domain by giving Figure 1: The two protocol versions used to collect data a reference point for both academia and industry professionals in this application field. In the analysis, we consider fragments of signals without extracting any fea- lar disorders. Furthermore, the females subjects were tures due to the nature of the problem: stress determines not pregnant. The dataset includes blood volume pulse nonspecific human responses and the feature selection (BVP), electrocardiogram (ECG), electrodermal activity depends on the subject and do not can be generalized. (EDA), electromyogram (EMG), respiration (RESP), body Such signal fragments contain samples of all the physio- temperature (TEMP), and three-axis acceleration (ACC). logical parameters measured. After appropriate resam- ECG, EDA, EMG, RESP, TEMP and ACC were recorded pling and noise reduction, these values are linearized by a chest-worn device (RespiBan) and sampled at 700 and constitute the input of the considered ML model by Hz, whereas a wrist-worn device (Empatica E4) recorded following the neural network approach. This study uses BVP (sampled at 64 Hz), EDA (at 4 Hz), TEMP (at 4 Hz), the WESAD [ 9 ] dataset that is public and stores 12 phys- and ACC (at 32 Hz). The dataset comprises 14 time series, iological signals, such as blood volume pulse and electro- each spanning approximately 2 hours, total experimental cardiogram, collected from 15 subjects during a lab study. duration. The experiments were conducted to capture After preprocessing (consisting of resampling, outlier re- three distinct afective states: baseline, stress, and amusemoval, and normalization), we determine a dataset of ment with durations of 20 minutes, 392 seconds and 7 samples that are signal fragments obtained using the slid- minutes, respectively. They also included two meditation ing window approach. Over these entries, we evaluate periods. To capture the data during the experiment, a the most common and popular methods widely in various particular protocol, depicted in Figure 1, has been used. It application areas. We consider eight machine learning consists of two diferent versions, where amusement and algorithms, i.e, Decision Tree (DT), Random Forest (RF), stressful conditions are interchanged between diferent Adaboost (AB), Extratree (ExT), Passive Aggressive Clas- subjects to avoid the efects of order. sifier (PA), Logistic Regression (LR), K-kneighbors (NKE) and Nearest Centrod (NC). We face the binary (stress/no- 2.2. Preprocessing stress) and multi-class (baseline, stress, and amusement) problem classifications. The results, evaluated in terms of classical metrics, show that RF outperforms the others in binary and multi-class approach. We also compare the results obtained with the ones in the literature [ 9 ].

The paper is organized as follows. Section 2 describes the materials and the methods used in this study. The pipeline of the approach used in the study with the main results are described in Section 3. The paper ends with some conclusion and future work, Section 4.

The varied sampling frequencies in WESAD, as detailed

in Section 2.1, necessitated a harmonization step. We resampled all data to match the 700Hz frequency of the RespiBAN. Therefore, the resampling is applied only to the time series detected by Empatica E4 using Fourier method as an unsampling technique.

After the resampling, we remove the outliers due to occasional anomalous peaks in some signals, which may be attributed to instrumental errors or measurement noise. We removed the anomalies from each time series by using a Hampel filter, discussed in [ 10 ]. Such a filter uses 1-minute sliding windows as input and calculates the mean ( ) and standard deviation ( ) of the values within the corresponding interval. Observations higher than the threshold of 3 from the mean within the respective window are classified as outliers (following Pearson’s rule) and are substituted with the nearest chronological value. This strategy ensures that outlier substitution doesn’t introduce significant high-frequency variations.

After outliers removal, we normalize all signals in the interval [ − 1, 1 ] to treat all inputs equally.Let = {1, 2, . . . , } be the considered time series with components, where each component corresponds to a biophysical signal. Each of them are rescaled to the in

2. MATERIALS AND METHODS

This work proposes a comparative evaluation of ML approaches to understand the best approach for real-time analytics. For this study, we consider the WESAD dataset.

2.1. Dataset

WESAD is a public dataset designed for stress and afective detection. It is a high-quality multimodal dataset storing physiological and movement data of 15 subjects (12 male and 3 female) during a controlled lab experiment [ 9 ]. All the participants were not heavy smokers and did not sufer from chronic mental or cardiovascurepresents a classification or decision. The root of the tree corresponds to the best predictor. Usually, a DT is pruned by combining the adjacent nodes to avoid overfitting. 2.4.2. Ensemble models terval [ − 1, 1 ] by applying the mean normalization: Random Forest Random Forest is an ensemble model by Breiman [ 14 ] for both classification and regression.

( − ()) + (( − ()) It constructs a set of decision trees during training and ˜ = () − () determines the prediction by selecting the most common class in the classification problem or calculating the where () and () is the maximum and mean/average prediction in the regression problem of minimum value among each component of , respec- the classes output by individual trees. This model comtively. Therefore, the input is a the scaled time series, bines the bagging approach with the random selection ˜ = {˜1, ˜2, . . . , ˜}. of features to ensure the uncorrelation among the decision trees of the forest. Feature randomness generates a 2.3. Dataset Entry random subset of features by ensuring low correlation among decision trees. In bagging, the decision trees deAfter the data preprocessing phase, we create two pend on trees created from a diferent bootstrap sample, datasets: one for binary classification and the other for i.e., samples that may appear more than once in the enmulticlass. All entries are obtained by applying the slid- tries of the training dataset. Diferently from decision ing window technique to preprocessed signals. Specifi- trees that consider all the possible feature splits, random cally, the entries consist of time series fragments charac- forests only select a subset of those features. terized by only an emotional state (or label) obtained by a slide of 60 seconds and a stride of 30 seconds, according to the study in [ 11 ]. To create the multiclass dataset, we consider parts of the time series associated with stress, Baseline and Amusement, as described in Section 2.1. For the binary classification, both the Baseline and Amusement states were aggregated under a single ’non-stress’ label. The labels distribution of the two datasets are shown in Fig. 2.

AdaBoost AdaBoost, Adaptive Boosting, is an ensemble models developed by Yoav Freund et al. [ 15 ]. It employs an iterative approach to improve poor classifiers by learning from their errors. Unlike the random forest that uses parallel ensembling, Adaboost uses “sequential ensembling”. Therefore, it is not possible to parallelize jobs on a multiprocessor machine like Random Forest. It creates a classifier by combining many poorly performing classifiers to obtain a good classifier of high accuracy. 2.4. Machine Learning Algorithms Such resulting classifier is accomplished with sequenIn this section, we describe some machine learning clas- tial weight adjustments, individual voting powers and a sification techniques. Interested readers can refer to [ 12 ] weighted sum of the final algorithm classifiers. for a complete treatment of machine learning approaches. 2.4.1. Decision Tree A DT is a non-parametric supervised learning algorithm for classification and regression in the form of a tree structure [ 13 ]. It predicts the value of a target variable by learning simple decision rules inferred from the data features. The method exploits the “divide et impera” approach to learning: it learns from data with a set of if-then-else decision rules. The depth directly correlates with the complexity of these decision rules. The output is a tree comprising decision nodes and leaf nodes: a decision node has two or more branches, and a leaf node Extremely Randomized Trees Extremely Randomized Trees, introduced in [ 16 ], are ensembling methods that perform regression or classification. It creates a large number of unpruned decision trees from the training dataset and uses majority voting to select the decision trees for the classification. Diferent from Random Forest, it uses the entire dataset to train decision trees. Moreover, it randomly selects the values at which to split a feature and create child nodes to ensure suficient diferences between individual decision trees. 2.4.3. Linear Models Logistic Regression Logistic Regression, introduced in [ 17 ], is a supervised learning algorithm mainly used for classification tasks where the aim is to estimate the probability of an instance belonging to a specific class based on the values of the input features. The method uses the sigmoid function to map any real-valued number into a value between 0 and 1. More specifically, it calculates a weighted sum of the input features, applies the logistic function to this sum, and then classifies the input as belonging to one of the two classes based on a chosen threshold.

Passive Aggressive The passive-aggressive algorithm, introduced in [ 18 ], is one of the few "online learning algorithms": the input data comes in sequential order, and the model is updated step-by-step. It is useful in applications that receive data as a continuous flow and need to adapt to change rapidly or autonomously or if you have limited computing resources. The algorithm is based on based on Passive and Aggressive approches.

If the prediction is correct, keep the model and do not make any changes (passive), while If the prediction is incorrect, make changes to the model. 2.4.4. Neighbors-based Models Supervised neighbors-based models can be applied for classification and regression. The principle behind nearest neighbor methods is to find a predefined number of training samples closest in distance to the new point, and predict the label from these. each class (target label). The training data is divided into clusters based on their class labels, and then the centroid is computed for each data cluster. Each centroid is simply the mean value of each of the input variables. Such a centroid represents the "model": given new examples, the algorithm assigns the label by computing the distance between a given data and each centroid.

2.5. Metrics

We evaluate the performance and efectiveness of the approaches by using Accuracy (), Precision ( ), Recall (), and F-measure (1), defined as follows =

+ + + + = =

+

+ 1 = 2

· · + where represents the number of true positive, denotes the number of false negative, represents the number of false positive, denotes the number of true negative.

3. RESULTS

The work aims to compare various machine learning algo

rithms to detect stress from signals captured by wearable devices. The workflow is described in Section 3.1, while K-Nearest Neighbors The k-nearest neighbours al- the results of the experiments are described in Section 3.2. gorithm, introduced by Fix and Hodges in 1951 [ 19 ] and expanded by [ 20 ], is a non-parametric supervised learning method for classification and regression. K-nearest 3.1. Methodology neighbours algorithm exploits proximity to make classifi- Our pipeline, depicted in Fig. 3, is implemented in Python cations or predictions about the grouping of an individual using the scikit-learn package for the machine learning data point. KNN searches for the k-nearest labelled train- approaches and SciPy for data manipulation and analying data by using the distance metric and attributes the sis. In particular, some methods of the SciPy library is label which appears the most to the new observation. In used in the data preprocessing phase. The method resamour study, we use the Minkowski distance as a metric. ple permits the resampling of signals. In our approach, The input consists of the k closest training examples in a all signals are resampled at 700 Hz. About the outlier data set, whereas the output depends on the task, classi- remotion, the Hampel filter is implemented using the ifcation or regression. Such output is a class membership ‘rolling‘, ‘mean‘, ‘std‘, ‘fillna‘, ‘mask‘, and ‘interpolate‘ or the property value for the entry, respectively. methods from the Pandas library. The ‘MinMaxScaler‘ class of the scikit-learn package is used to perform data Nearest Centroid Nearest Centroids, defined in [ 21 ], normalization. The machine learning methods Decision is arguably the simplest classifier. It operates on an intu- Tree, Random Forest , K-Nearest Neighbors and Logisitive principle: it takes data samples as input and classifies tic Regression are implemented via the tree, ensemble, them into the class of training examples whose centroid neighbors and linear model modules, respectively. The (a geometric centre of a data distribution) is closest to it. method K-Folds is used to split the dataset into conThe algorithm assumes that the centroids are distinct for secutive folds without shufling and then each fold is DT RF KNN DT RF KNN

4. CONCLUSIONS AND FUTURE

WORK

then used once as a validation while the − 1 remaining folds form the training set. The code used in this manuscript are available from the corresponding author upon reasonable request.

3.2. Experiments

Given the small number of subjects involved in the ex

periment, we consider the Leave-One-Subject-Out CrossValidation (LOSOCV), i.e., an approach that utilizes each subject as a “test” set and the remaining 14 as a “training” set. The experiments have been performed considering the decision tree, random forest, K-Nearest Neighbors and logistic regress as machine learning methods. For all experiments, we use the default parameters.

We evaluate such experiments by considering Accuracy, Precision, Recall and F1-Score as metrics. Tables 1 shows the average values with the standard deviation of the considered metrics obtained for binary and multiclass classification, respectively. Appendix A reports the values for each experiment.

In this work, we have compared various classical ma

chine learning algorithms. We have used a public dataset, WESAD, to perform our study. Analyzing the results, we have noted the best results have been archived by the random forest algorithm. This evidence is in line with the

Binary Classification results proposed in the literature [ 9 ]. We have observed RDFT 00..9826A09cc±±ur00a..11c05y30 00..9942P44r e±±ci00si..o1100n05 00..984658R±±ec00a..l02l9029 00..9848F021-±±Sc00o..01re7660 tohnaetsctlhaastsicficoantisoidnesrbsaisgendalofneatthueressig.nal values outcome EAxBT 00..980496 ±± 00..110594 00..984833 ±± 00,,018493 00..991057 ±± 00..111298 00..982855 ±± 00..019122 In future work, we intend to conduct additional experLR 0.822 ± 0.232 0.843 ± 0.208 0.925 ± 0.199 0.871 ± 0.186 iments to discern the most relevant physiological signals. PA 0.823 ± 0.225 0.842 ± 0.200 0.934 ± 0.187 0.874 ± 0.173 It represents another fundamental aspect of detecting KNNCN 00..982495 ±± 00..110903 00..995339 ±± 00..019178 00..984196 ±± 00..028531 00.,895415±± 00..200735 stress for real-time analysis using wearable sensors and

Multiclass Classification smartphones. In this case, the aim is to store the min

Accuracy Precision Recall F1-Score imum information to be non-invasive and reduce the RDFT 00..760279 ±± 00..127212 00..666538 ±± 00..115975 00..760279 ±± 00..127212 00..656949 ±± 00..127333 space while maintaining high model performance. We LEKRxNTNe 000..,656278307±±± 000...221436196 000...766014352 ±±± 000...122504289 000...656278307 ±±± 000...221436295 000...556864835 ±±± 000...221446925 aplrsooacinhetesn,sdutcoh caosngsriadpehr caonndveomluptiloony ndeetewpolrekasronrinrgecaupr-NC 0.680 ± 0.219 0.685 ± 0.242 0,680 ± 0.220 0.662 ± 0.228 rent neural networks, motivated by the results obtained Table 1 in other scenarios [ 22, 23 ]. Moreover, we also intend to Average value with metrics with their standard deviation re- study the role of the length of the sliding windows from lated to the binary and multiclass classification a theoretical perspective by taking into account various entropy-based methods that have produced evaluable out

The Random Forest model outpaces its counterparts in comes in the scenario of protein-protein interaction site both binary and multiclass classification scenarios. For prediction [ 24 ]. Another crucial future investigation is the RF model, the obtained accuracy stands at 92% (bi- to explore and define approaches to extract and describe nary) and 70% (multiclass). Corresponding F1-scores are the correlation that sliding windows represent. Other 88.2% and 60% , respectively. While multiclass classifica- representations, like arc-annotated sequences, strings tion ofers insights for emotion detection via wearables, and simplicial complexes, will be explored. We will exthere remains room for improvement. Comparing re- plore other representations like arc-annotated sequences sults from Schmidt et al.’s benchmark on the WESAD for the analysis and comparison of time utilizing tools dataset [ 9 ], which utilized standardized machine learn- like [ 25 ] and strings or simplicial complexes, which aling techniques and features, our study finds that the RF low applying techniques from formal methods to identify patterns [ 26 ] or verify properties [ 27 ].

Acknowledgements. This work has been funded by

the European Union - NextGenerationEU under the Italian Ministry of University and Research (MUR) National Innovation Ecosystem grant ECS00000041 - VITALITY CUP J13C22000430001

References

[1] B. S. McEwen , Protective and damaging efects of stress mediators , New England journal of medicine 338 ( 1998 ) 171 - 179 .

[2] Health and Safety Executive, HSE on work-related stress . 2021 , http://www.hse.gov.uk/statistics/ causdis/-fstress/index.htm, ???? Accessed on March 7 , 2022 .

[3] E.-H. Lee , Review of the psychometric evidence of the perceived stress scale , Asian nursing research 6 ( 2012 ) 121 - 127 .

[4] K. Hovsepian , M. Al'Absi , E. Ertin , T. Kamarck , M. Nakajima , S. Kumar, cstress: towards a gold standard for continuous stress assessment in the mobile environment , in: Proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing , 2015 , pp. 493 - 504 .

[5] M. Gjoreski , M. Luštrek , M. Gams , H. Gjoreski , Monitoring stress with a wrist device using context , Journal of biomedical informatics 73 ( 2017 ) 159 - 170 .

[6] L. Shu , J. Xie , M. Yang , Z. Li , Z. Li , D. Liao , X. Xu , X. Yang , A review of emotion recognition using physiological signals , Sensors 18 ( 2018 ) 2074 .

[7] R. Li , Z. Liu , Stress detection using deep neural networks , BMC Medical Informatics and Decision Making 20 ( 2020 ) 1 - 10 .

[8] D. H. Wolpert , The lack of a priori distinctions between learning algorithms , Neural computation 8 ( 1996 ) 1341 - 1390 .

[9] P. Schmidt , A. Reiss , R. Duerichen , C. Marberger , K. Van Laerhoven , Introducing wesad, a multimodal dataset for wearable stress and afect detection , in: Proceedings of the 20th ACM international conference on multimodal interaction , 2018 , pp. 400 - 408 .

[10] J. Astola , P. Kuosmanen , Fundamentals of nonlinear digital filtering , CRC press, 2020 .

[11] M. Quadrini , S. Daberdaku , A. Blanda , A. Capuccio , L. Bellanova , G. Gerard, Stress detection from wearable sensor data using gramian angular fields and cnn , in: International Conference on Discovery Science, Springer, 2022 , pp. 173 - 183 .

[12] S. Shalev-Shwartz , S. Ben-David , Understanding machine learning: From theory to algorithms , Cambridge university press, 2014 .

[13] J. R. Quinlan , Induction of decision trees , Machine learning 1 ( 1986 ) 81 - 106 .

[14] L. Breiman , Random forests, Machine learning 45 ( 2001 ) 5 - 32 .

[15] Y. Freund , R. E. Schapire , et al., Experiments with a new boosting algorithm , in: icml, volume 96 , Citeseer , 1996 , pp. 148 - 156 .

[16] P. Geurts , D. Ernst , L. Wehenkel, Extremely randomized trees , Machine learning 63 ( 2006 ) 3 - 42 .

[17] D. R. Cox , The regression analysis of binary sequences , Journal of the Royal Statistical Society Series B: Statistical Methodology 20 ( 1958 ) 215 - 232 .

[18] K. Crammer , O. Dekel , J. Keshet , S. Shalev-Shwartz , Y. Singer , Online passive aggressive algorithms ( 2006 ).

[19] E. Fix , J. L. Hodges , Discriminatory analysis. nonparametric discrimination: Consistency properties , International Statistical Review/Revue Internationale de Statistique 57 ( 1989 ) 238 - 247 .

[20] T. Cover , P. Hart , Nearest neighbor pattern classification , IEEE transactions on information theory 13 ( 1967 ) 21 - 27 .

[21] R. Tibshirani , T. Hastie , B. Narasimhan , G. Chu, Diagnosis of multiple cancer types by shrunken centroids of gene expression , Proceedings of the National Academy of Sciences 99 ( 2002 ) 6567 - 6572 .

[22] M. Quadrini , S. Daberdaku , C. Ferrari , Hierarchical representation and graph convolutional networks for the prediction of protein-protein interaction sites , in: Machine Learning, Optimization, and Data Science: 6th International Conference, LOD 2020 , Siena, Italy, July 19-23 , 2020 , Revised Selected Papers , Part II 6 , Springer, 2020 , pp. 409 - 420 .

[23] M. Quadrini , S. Daberdaku , C. Ferrari , Hierarchical representation for ppi sites prediction , BMC bioinformatics 23 ( 2022 ) 96 .

[24] M. Quadrini , M. Cavallin , S. Daberdaku , C. Ferrari , Prosps: protein sites prediction based on sequence fragments , in: International Conference on Machine Learning, Optimization, and Data Science , Springer, 2021 , pp. 568 - 580 .

[25] M. Quadrini , L. Tesei , E. Merelli, Aspralign: a tool for the alignment of rna secondary structures with arbitrary pseudoknots , Bioinformatics 36 ( 2020 ) 3578 - 3579 .

[26] M. Quadrini , E. Merelli, R. Piergallini , Loop grammars to identify rna structural patterns ., in: Bioinformatics, 2019 , pp. 302 - 309 .

[27] M. Loreti , M. Quadrini , A spatial logic for simplicial models , Logical Methods in Computer Science 19 ( 2023 ).