Towards Exploring Stress Reactions in Teamwork using Multimodal Physiological Data

Abstract

In education, while teams of students are learning in a real scenario, many different factors are happening in real-time and can have a significant impact on the way students can improve their skills. Realistic simulated scenarios can help them to achieve their learning goals. However, these close-to-real situations can make them experience stress that can be confronting and hinder learning. In other cases, these stressful experiences are meant to reflect the kinds of pressures they will encounter in authentic workplaces, thus becoming authentic training experiences. There is strong evidence that stress has an important effect on the student's engagement and motivation and consequently influences learning outcomes. In the particular educational context of healthcare (e.g. nursing), teachers commonly have a series of expectations about the moments in which students will have a higher cognitive load and stress that can impact on their learning, depending on the phase of the simulation in which they are and how they move around the space prepared for the simulation. This paper introduces a study with nursing students carrying out a practice teamwork in a simulated scenario divided into 5 different phases with a critical patient, in which students must learn to make life-to-death decisions timely. This paper discusses the multimodal data processing that is being performed to identify if the arousal levels match the teachers' expectations regarding the students' affective situation in each phase.

Keywords

• affective computing
• physiological sensors
• non-intrusive devices
• teamwork
• nursing
• simulation
• spatial behavior
• learning design
• multimodal

-

A significant aspect of life is emotion: it influences decision-making, perception, human intelligence and human interaction. Physiologically and mentally, feelings control the status of humans [ 1 ] and diverse models have been proposed to describe emotions [ 2, 3, 4 ]. In the educational system, practice in real settings is a fundamental added value that provides competencies at the level of those that would be obtained in a work experience. This can result in better training that will improve the skills and future employability of students. Simulation-based learning strategies are designed to provide safe training spaces for students and professionals to develop the skills they will need in authentic clinical practice [ 5 ]. In the case of nursing education, nurses in training are often immersed in simulated rooms to practice a series of clinical procedures [ 6 ]. In these simulations, students are asked to act out different team roles according to a fictional scenario and to perform a variety of tasks with the purpose of improving the outcome of a simulated patient which usually imply moving around the space prepared for the simulation (e.g., the box of hospital).

In this sense, we have previously worked on exploring how to gain educational insights from indoor positioning data based on the learning design and the educators’ expectations regarding variations in the affective state [ 7, 8 ]. These expectations consider the cognitive load the students should manage according to the corresponding instructional design (that is, depending on which phase of the teamwork practice they are in), which can elicitate emotions that students have to learn to deal with in a real situation. Moreover, the place they take up in the classroom might influence the information they are able to obtain during the activity, and thus, impact on the cognitive load and the emotions. Indoor positioning data can be captured through wearable tags to provide metrics about teaching and learning based on x and y coordinates [ 9, 10, 11 ]. Moreover, we have also explored the potential of physical analytics for teaching and learning considering proximity, motion and location analytics [ 12 ]. For the processing, we have also proposed a multimodal data modelling method called the multimodal matrix, based on quantitative ethnography and which provides means to model different types of modalities [ 13 ].

In this context, we aim to study the changes in stress levels of the students during a practice teamwork in the different phases of a simulated scenario considering both the physiological information of the students and their position in the classroom using the information collected with several devices. We are analyzing the data collected from a study [ 14 ] in which nursing students carried out a practice teamwork during five predefined phases with a critical patient to learn how to make life-to-death decisions timely.

In the current contribution of this paper, we focus on the processing of the emotional information. In particular, we present the on-going works to process the multimodal data collected with the goal to identify if the arousal levels gained with electrodermal devices matched the teachers' expectations regarding the students’ stress in each phase. Since stress coping mechanisms are individually dependent and need to be personalized to each student's needs [ 15 ], in this research we are carrying out both inter-subject (analyzing the users as a group) and intra-subject (analyzing each user individually) analysis. As an initial step, we are analyzing whether the electrodermal data would be able to match the teacher expectations as the teacher expectations are considered the ground truth and thus, our first goal is to evaluate the validity of electrodermal data for this purpose.

The rest of this paper is organized as follows. Section 2 describes some work that can frame our proposal. Section 3 focuses on the simulation learning scenario we are analyzing in our research and explains how the data was collected. Section 4 presents the progress on the data analysis presented in Section 3. Section 5 discusses the current progress and presents potential avenues of future work. The paper finalizes with some concluding remarks in Section 6. 2

Related works

Psychological information can be obtained with wearable devices, such as body temperature and galvanic skin response or electrodermal activity (EDA), as done in our previous work [ 7, 8, 16 ]. With these datasets, it is possible to analyze whether the educators’ expectations correspond with the arousal data of the students' devices. This analysis of EDA shows generalized changes in the state of arousal, which can be caused by emotional, cognitive or physical stimulation [ 17 ].

EDA can be measured through skin conductance (SC), widely used in psychophysiology as an expression of psychological arousal due to its connection with the social network sites [ 18 ]. SC can be described by two components: skin conductance level (SCL) and skin conductance response (SCR). The SCL describes a tonic activity that varies slowly and the variations in EDA are more of the order of minutes [ 19 ]. In contrast, the SCR characterizes a rapidly varying phasic activity, on the order of seconds, which may reflect a specific stimulus response and thus, are more interesting in learning scenarios as they reflect punctual situations that take place [ 20, 21 ]. They can be combined with clearly identifiable external events that arise in a predefined window, seconds after the start of the stimulus [ 19, 22 ].

The quality of the gathered EDA and the quality of the both amplitude of the SCL and SCR depends on the next items: the density of the sweat glands in the chosen skin area, the degree of the psychophysiological activity in this area and the size of the skin that has contact with the electrodes [ 20 ]. In order to identify the quality of a SCR, it is necessary to define its components first: a latency, an amplitude, a rise time and a half recovery time [ 23, 24 ]. The latency refers to the time between the onset of the stimulus and the start of a SCR which is typically about 1 to 3 seconds [ 22, 25 ]. The non-specific SCRs occur just about 1 to 3 times per minute. The rise time describes the time between the onset of the SCR and its peak amplitude which also takes about 1 to 3 seconds. To be identified as a SCR, the deflection has to reach a certain threshold which is commonly around 0.04µS, 0.03µS or 0.01µS. Deflections below this value are not a SCR. The amplitude refers to the difference between the conductivity at the onset (baseline) and the peak whereby a phasic increase in conduction occurs which is around 0.2µS and 1.0µS [ 25 ].

The peaks in the EDA signal provide information about the arousal of the person, and thus, it is relevant to analyze how to label arousal peaks. A simple way to work with arousal levels is to focus on how arousal is computed by the increasing slope of the EDA [ 26 ]. The more positive the slope of the EDA in a given time window is, the higher the arousal is.

In addition, in a collaboration scenario as in [ 27 ], arousal cross-recurrence with stress can be checked to see what is going on between the teams and between the team and the teacher. Stress can be computed as temperature is decreasing slope. The more negative the slope of the temperature is in a given time window, the higher the stress is. In this way, acute stress triggers peripheral vasoconstriction, causing a rapid, shortterm drop in skin temperature in homeotherms. In [ 28 ] it was tested whether this response had the potential to quantify stress, by exhibiting proportionality with stressor intensity.

In this context, next we present our current research to identify if the arousal levels obtained with an EDA sensor matches the teachers' expectations in the different phases of the learning activity analyzed. 3 3.1

Method

Learning Design

Healthcare simulation is a pedagogical approach that uses a constructivist learning model to provide students with opportunities to experience teamwork and patient situations without compromising the care of real patients [ 5 ]. Simulations often start with a description of learning goals, followed by the simulation itself, concluding with a debrief aimed at provoking students’ reflection on performance and errors made. Although video-based products to support this reflection exist, they are commonly impractical for class use, resulting in students rarely using such evidence to inform reflection [ 29 ]. 3.2

Learning Scenario: Allergic Reaction to Antibiotics Simulation

This simulation was run in 5 classes taught by 3 teachers (including the same subject coordinator). A total of 25 students in their third year (21 females and 4 males) volunteered to participate. The aim of this simulation was to help student nurses learn how to react when a patient is having an allergic reaction to some medication. Students in each team played the roles of team leader, recovery nurses (RN1, RN2), scribe (RN3) and the patient (not tracked). According to the assessment criteria, a highly effective team should have performed the following 6 actions: (i) perform an initial set of vital signs, after the teacher reads the initial handover; (ii) administer the intravenous (IV) fluid antibiotics; (iii) perform another set of vital signs after the patient complains of chest tightness; (iv) stop the IV antibiotic after the patient reacts with chest tightness; (v) perform an electrocardiogram after the patient complains of chest tightness; and (vi) call the doctor after stopping the IV antibiotic.

Considering the 6 actions, the simulation was therefore divided into 5 phases, as follows: • Phase 1: patient assessment, from the beginning of the simulation to the moment nurses realize the patient needs IV antibiotic; • Phase 2: IV fluid preparation; • Phase 3: IV fluid administration; • Phase 4: patient adverse reaction (since the patient starts complaining about the allergic reaction until the moment nurses stop the IV antibiotic); and • Phase 5: patient recovery. 3.3

Data collection

For the study, we used the dataset obtained and described in [ 14 ] for collocated teamwork based on multiple sources of data captured via a combination of sensor signals (e.g. positioning and physiological markers), system logs and human logs during group situations. Detecting arousal/stress from the physiological state requires monitoring body condition by tracking different parameters. In this case, students’ arousal levels were captured with the Empatica® E4 wristband (Empatica Inc., Cambridge, MA, USA). Similar to E3 [ 30 ], the E4 is a research quality multisensor wristband that allows to record multimodal data, namely electrodermal activity (EDA, GSR sensor) at 4Hz, wrist acceleration at 32 Hz, body temperature and photoplethysmogram (which registers flow changes in blood volume). Arousal peaks have been labelled, discriminating high and low values to compare the labelling with educators’ expectations regarding arousal in each phase. 4

On-going works

This paper contribution focuses on analyzing if the arousal levels measured in the students with physiological data collected using wearables devices during a simulationbased learning matches the teachers' expectations of students’ reactions to stressful situations in each phase of the training scenario. The research question behind aims to observe whether the EDA data would be able to match the teacher expectations, being these considered as the ground truth and thus, the validity of the EDA for our research is evaluated.

For the analysis of electrodermal activity we have used EDA Explorer1, an open site where anyone with EDA data can upload it for automatic artifact and peak detection and visualization. Settings can be customized and the results can be downloaded, for example to train a classifier. This tool uses the temperature and accelerometer data for the labeling. All 3 data streams (EDA, skin temperature and accelerometer) are shown to the labeler to provide more context.

Our current research work focuses on the intra-subject analyses with the EDA values obtained with the E4 wristband. As an example, Figure 1 shows the peaks during a simulation using EDA-Explorer [ 31 ]. The peak analysis was conducted with the webapplication of the EDA-Explorer. Thereby, a minimum amplitude threshold of 0.07µS was used with a maximum rise-time of four seconds and an offset-value of 1s. Instead of downsampling, in order to preprocess the data, the EDA-Explorer provided a lowpass Butterworth filter, which is mandatory to use. It was chosen for a filter frequency of 1 Hz and a filter order of 6.

1 https://eda-explorer.media.mit.edu

Fig. 1. EDA graphic obtained with the EDA-Explorer tool showing the peaks and phases marked for one of the students.

An increase in stress, cognitive load, or emotion can cause body sweating. This produces a Skin Conductance Response (SCR), which means abrupt increases in the conductance of the skin. The EDA-Explorer algorithm detects these SCRs or "peaks" in an EDA signal and computes features related to them, allowing to perform machine learning on the computed features.

Figure 1 shows the different phases in which the training session was divided. Each is separated by a red line. It can be seen how the EDA values differ in each of them.

Table 1 shows the values used to generate the graphics presented in Figure 1. Each row shows information about each peak. Columns from left to right contain: ─ Peak: time where peak took place. ─ EDA: the EDA amplitude at the apex in µSiemens. ─ Rise_time: the time, in seconds, it takes for the SCR to rise from the start of the SCR to the apex. The start of the SCR is computed by going backwards from the apex of the peak to point where derivative is less than 1% of its maximum value. ─ Max_deriv: maximum derivative of SCR, in µSiemens per second. ─ Amp: Amplitude of peak; that is [amp = (EDA at apex) - (EDA at start of the SCR)], in µSiemens. ─ Decay_time: The time, in seconds, that it takes for the SCR to decay to 50% of its amplitude. Note that this is blank if an SCR does not decay to 50% before another peak starts or before the maximum decay time is reached. ─ SCR_width: The time in seconds between the 50% of the amplitude on the incline side of the peak to 50% of the amplitude on the decline side of the SCR. Note that this is blank if a Decay_time was not computed. ─ AUC: Area under the Curve; approximated by multiplying the Amplitude by the SCR_width. Note that this is blank if a Decay_time was not computed.

Peak 02:46:52 02:54:54 02:55:08 03:02:40 03:04:49 03:27:05 03:29:43

Interview with educators

Using the methodology described in [ 8 ] and motivated by designing meaningful analytics, the five educators (females: 4, average years teaching: 12.6), who had taught the simulation beforehand, were interviewed to elicit their perceptions about the stress and cognitive load they expected from students in each phase of the simulation. Each interview was recorded using an online video conferencing platform (i.e., Zoom) and had an approximate duration of 60 minutes. Following a semi-structured format, the interview was structured as follows: (1) educators were explained the purpose of the session, (2), then, they were presented with the phases of the simulation according to the learning design, and (3) they were asked to respond to the following questions for each phase of the simulation: (i) What are the potential triggers of stress/arousal for the team or specific roles in phase X ( , ranging from 1 to 5), if any?, and (ii) What can make team members experiment cognitive load in phase X ( , ranging from 1 to 5), if any? The interviews were fully transcribed using a professional service.

Then, the educators’ responses were grouped and categorized to identify the expected behaviors in relation to each phase using NVIVO2, a qualitative data analysis tool. This resulted in a set of descriptions of the potential triggers of stress/arousal and the events (actions) that can make students experiment cognitive load per phase that were discussed by the rest of the research team. The team found consistent descriptions of expected behaviors across educators, which are compiled in Table 2.

Not much stress. It can variate, it

depends on the patient too (e.g., how willing is the patient to answer questions or to allow nurses to approach him/her).

Cognitive Load

What can make team members experiment with cognitive load? Reading through the notes.

Validating compatibility of medicine.

Validating dosage. 2 https://www.qsrinternational.com/nvivo-qualitative-data-analysis-software/home

Phase 2

Phase 3

Phase 4

Phase 5

Trying to figure out which medication does the patient need. Trying to figure out what antibiotic to use and the appropriate dose.

Working out how antibiotics

work together.

Reading through the notes. Validating compatibility of medicine and dosage.

Working out how antibiotics work together.

Validating all equipment is adequate.

Administering the IV antibiotic.

Probably if they have not’ given antibiotics, or using the equipment correctly, they should be a bit nervous.

Administering the IV-antibiotic is a technical skill. It could be the first/second/third time for them practicing. For this course, they should have practiced before. Some of them are confident, some not.

Critical trigger. The allergic reac- Put everything together to idention can be very surprising for tify what was causing the situanurses. tion.

Majority of stress peaks should Figuring out what is going on happen in phase 4, due to changing with the patient. conditions of the patient. Coordinating the team.

They should be aroused all the time during this phase.

Depends on experience. Writing reports Less stress because at this point the critical moment had happened.

From the description that has been defined based on the educators’ expectations of stress and cognitive load (Table 2), and considering previous analysis of spaces of interest [ 9, 7 ] in ward locations, there is potential in running further analysis aiming to gain additional meaning based on the relationship between stress, cognitive load and spaces of interest. What is expected from this research is to validate how accurate were the expectations of educators in terms of stress and cognitive load and if additional insights can be generated from the triangulation of different modalities.

The main point is to use these datasets with EDA-Explorer in order to gain visual information through graphics with relevant arousal peaks that allow determine if, once crossed with the teachers' expectations of stress reactions (Table 2), this answers the main question of this research: “Can the students’ reactions to stress be determined in base of psychological data collected during a real learning simulation divided in meaningful didactic phases?”.

In the end, we expect to confirm a twofold objective: i) the challenging situations of students can be previously known by educators and thus, allow to properly design the training simulations, and ii) provide evidence for students and teachers to manage their arousal states to help them improve learning outcomes, and thus, take the best decision in the given learning situation.

Discussion

Through the EDA-Explorer tool, the datasets of the students are being analyzed, obtaining information on arousal peaks by phases for each of the participants. This initial intra-subject analysis will be compared with the results expected by the teaching staff, thus determining if it is possible, in a case study such as the one at hand, to know the stress levels in each of the phases. For this comparison, an inter-subject analysis is to be used. Otherwise, we will have a first impression of the phases where the arousal peaks are more pronounced, hence, it will make it easier for the teachers involved to work on them. Note that in this research we use the arousal to map emotions, but there are other approaches in which arousal is being considered a emotion itself together with valence.

An important issue is to focus on the risk of gaining data out of context during the capture of data by sensors, which should be managed to minimize it. We considered that the learning design and the educator’s expectations can be used to guide the modelling and the analysis of the multimodal data collected. Each modality can be validated separately based on a specific assessment criteria, but the combination of different modalities in conjunction with the assessment criteria can perhaps provide additional insights that can be used for reflection. Educators’ expected behaviors can also be used to guide and focus attention on specific aspects of the simulation such as critical actions that can be of interest for educators and nurses to reflect on.

At this point, we raised a series of questions aimed for discussion at the workshop: • Can we gain additional insights via triangulating different modalities and assessment criteria defined by educators? • How can we contextualize and gain meaning from multimodal data? • How can we promote the generation of meaningful analytics using the learning designs and the expectations from educators? • How can we extend the processing of the emotional information to consider the indoor positioning during the teamwork practice? • Would it be possible to extrapolate the results to other settings outside of nursery? In addition, as future work we also propose to focus on the five phases trying to synchronize the multimodal data collections with some more meaningful data (e.g., watching the moments with peaks) as in [ 32 ]. 6

Conclusions

In this paper we have presented our progress on the data analysis obtained from a realistic simulated learning scenario on nursing training designed to make students experience stress similar to those that arise in a real situation. Data collected includes physiological information of the participants and their movements around the classroom. In the current work we report the analysis on intra-subject variations of the arousal levels experimented by one participant along the different phases of the training obtained through an E4 wristband. We also report the teachers' expectations regarding the students’ stressful situation along the training session. Next steps in our research are to cross these expectations with the arousal peaks identified with the EDA-Explorer tool following both an intra-subject and an inter-subject analysis. In addition, there is potential in running further analyses aiming to gain additional meaning based on the relationship between stress, cognitive load and spaces of interest, which refer to the positions within the classroom from where the students should carry out the different expected actions of the learning scenario that is simulated in the teamwork practice.

Acknowledgements

The work is partially supported by the project “INTelligent INTra-subject development approach to improve actions in AFFect-aware adaptive educational systems” INT2AFF funded under Grant PGC2018-102279-B-I00 (MCIU/AEI/FEDER, UE) by the Spanish Ministry of Science, Innovation and Universities, the Spanish Agency of Research and the European Regional Development Fund (ERDF). Roberto Martinez-Maldonado’s research is partly funded by Jacobs Foundation.

References

1. Kumar J , Kumar J. A machine learning approach to classify emotions using GSR . Advanced Research in Electrical and Electronic Engineering 2 ( 12 ), 72 - 76 , 2015 .

2. Graesser , A. C. , and D'Mello , S. Emotions during the learning of difficult material . In B. H. Ross (Ed.), The psychology of learning and motivation : Vol. 57 . The psychology of learning and motivation (p. 183 - 225 ). Elsevier Academic Press, 2012 . https://doi.org/10.1016/B978- 0 -12-394293-7 . 00005 - 4

3. Van Kleef , G.A. How Emotions Regulate Social Life: The Emotions as Social Information (EASI) Model . Current Directions in Psychological Science, 18 ( 3 ): 184 - 188 , 2009 . doi: 10 .1111/j.1467- 8721 . 2009 . 01633 .x

4. Gross , J.J. Emotion Regulation: Current Status and Future Prospects , Psychological Inquiry, 26 : 1 , 1 - 26 , 2015 . doi: 10 .1080/1047840X. 2014 .940781

5. Berragan , L. Simulation: An effective pedagogical approach for nursing? Nurse Education Today 31 , 7 ( 2011 ), 660 - 663 . https://doi.org/10.1016/j.nedt. 2011 . 01 .019

6. Foster , M. , Gilbert , M. , Hanson , D. , Whitcomb , K. Graham , C. Use of simulation to develop teamwork skills in prelicensure nursing students: an integrative review . Nurse educator 44 , 5 ( 2019 ), E7 - E11 .

7. Fernandez-Nieto , G.M. , Martinez-Maldonado , R. , Kitto , K. and Shum , S.B. Modelling spatial behaviours in clinical team simulations using epistemic network analysis: Methodology and teacher evaluation , in ACM International Conference Proceeding Series, Apr. 2021 , pp. 386 - 396 , doi: 10.1145/3448139.3448176.

8. Fernandez-Nieto , G. , Martinez-Maldonado , R. , Echeverria , V. , Kitto , K. , An , P. , and Shum , S.B. What Can Analytics for Teamwork Proxemics Reveal About Positioning Dynamics In Clinical Simulations? Proc. ACM Hum.-Comput. Interact . 5 , CSCW1 , Article 185 ( April 2021 ), 24 pages. DOI: https://doi.org/10.1145/3449284

9. Echeverria V. , Martinez-Maldonado R. , Power T. , Hayes C. , Shum S .B. Where Is the Nurse? Towards Automatically Visualising Meaningful Team Movement in Healthcare Education . In: Penstein Rosé C. et al. (eds) Artificial Intelligence in Education. AIED 2018. Lecture Notes in Computer Science , vol 10948 . Springer, Cham. https://doi.org/10.1007/978-3- 319 - 93846-2_ 14

10. Yan , L. , Martinez-Maldonado , R. , Gallo-Cordoba , B. , Deppeler , J. , Corrigan , D. , Fernandez-Nieto , G. and Gasevic , D. Footprints at School: Modelling In-class Social Dynamics from Students' Physical Positioning Traces . LAK21: 11th International Learning Analytics and Knowledge Conference. Association for Computing Machinery , New York, NY, USA, 43 - 54 , 2021 . doi: https://doi.org/10.1145/3448139.3448144

11. Martinez-Maldonado R. , Echeverria V. , Schulte J. , Shibani A. , Mangaroska K. , Buckingham Shum S. Moodoo: Indoor Positioning Analytics for Characterising Classroom Teaching . In: Bittencourt I. , Cukurova M. , Muldner K. , Luckin R. , Millán E . (eds) Artificial Intelligence in Education. AIED 2020. Lecture Notes in Computer Science , vol 12163 , 2020 . Springer, Cham. https://doi.org/10.1007/978-3- 030 -52237-7_ 29

12. Martinez-Maldonado , R. , Echeverria , V. , Santos , O.C. , Dias Pereira Dos Santos , A. , and Yacef , K. Physical learning analytics: a multimodal perspective . In Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK '18) . Association for Computing Machinery , New York, NY, USA, 375 - 379 , 2018 . DOI:https://doi.org/10.1145/3170358.3170379

13. Echeverria , V. , Martinez-Maldonado , R. and Shum , S.B. Towards Collaboration Translucence: Giving Meaning to Multimodal Group Data, Proceedings of the CHI , pp. 39 : 1 -- 39 : 16 , 2019 . http://doi.org/10.1145/3290605.3300269

14. Martinez-Maldonado , R. , Echeverria , V. , Fernandez-Nieto , G. and Shum , S.B. From Data to Insights: A Layered Storytelling Approach for Multimodal Learning Analytics , Apr. 2020 , doi: http://doi.org/10.1145/3313831.3376148.

15. Santos , O.C. Emotions and Personality in Adaptive e-Learning Systems: An Affective Computing Perspective . In: Emotions and Personality in Personalized Systems. Editors: Tkalčič, M. , De Carolis , B. , de Gemmis , M. , Odić , A. , and Košir . A. Springer, p. 278 - 279 , 2016 . doi: 10 .1007/978-3- 319 -31413-6_ 13

16. Santos , O.C. , Uria-Rivas , R. , Rodriguez-Sanchez , M.C. , Boticario , J.G. An Open Sensing and Acting Platform for Context-Aware Affective Support in Ambient Intelligent Educational Settings . IEEE Sensors Journal , vol. 16 ( 10 ), p. 3865 - 3874 , 2016 , doi: 10.1109/JSEN. 2016 .2533266

17. Picard , R.W. Future affective technology for autism and emotion communication . Philosophical Transactions of the Royal Society , Vol. 364 , 3575 - 3584 , 2009 , doi: https://doi.org/10.1098/rstb. 2009 .0143

18. Chen , W. , Jaques , N. , Taylor , S., Sano , A. , Fedor , S. , and Picard , R.W. Wavelet-based motion artifact removal for electrodermal activity . Paper presented at the Engineering in Medicine and Biology Society , 2015 37th Annual International Conference of the IEEE, Milan. doi: https://doi.org/10.1109/EMBC. 2015 .7319814

19. Benedek , M, and Kearnbach , C. A continuous measure of phasic electrodermal activity . Journal of Neuroscience Methods , Vol. 190 ( 1 ), 80 - 91 , 2010 . doi: https://doi.org/0.1016/j.jneumeth. 2010 . 04 .028

20. Lykken D.T. , Venables P.H.. Direct measurement of skin conductance: a proposal for standardization . Psychophysiology . 1971 Sep; 8 ( 5 ): 656 - 72 . PMID: 5116830 . doi: 10 .1111/j.1469- 8986 . 1971 .tb00501.x.

21. Braithwaite , J. J. , Watson , D. G. , Jones , R. , and Rowe , M. A guide for analysing electrodermal activity (EDA) & skin conductance responses (SCRs) for psychological experiments . Psychophysiology 49 , 1017 - 1034 , 2013 .

22. Kappeler-Setz , C. , Gravenhorst , F. , Schumm , J. , Arnrich , B. , and Tröster , G. Towards long term monitoring of electrodermal activity in daily life . Personal and Ubiquitous Computing , Vol. 17 ( 2 ), 261 - 271 , 2013 , doi: 10.1007/s00779-011-0463-4

23. Ishchenko , A.N. , and Shevev , P.P. Automated complex for multiparameter analysis of the galvanic skin response signal . Biomedical Engineering , Vol. 23 ( 3 ), 113 - 117 , 1989 , doi: 10.1007/BF00562429

24. Setz , C. , Arnich , B. , Schumm , J. , La Marca , R. , and Tröster , G. Disciminating Stress From Cognitive Load Using a Wearable EDA Device . IEEE Transactions on Information Technology in Biomedicine , Vol. 14 ( 2 ), 410 - 417 , 2010 , doi: 10.1109/TITB. 2009 .2036164

25. Dawson , M.E. , Schell , A.M. , and Filion , D.L. The Electrodermal System . In Cacioppo, J.T., Tassinary , L.G. , & Berntson , G. (3rd Ed.), Handbook of Psychophysiology (159-181) . Cambridge: University Press, 2007

26. Leiner , D. , Fahr , A. and Früh , H. EDA Positive Change: A Simple Algorithm for Electrodermal Activity to Measure General Audience Arousal During Media Exposure . Communication Methods and Measures. 6 . 237 - 250 , 2012 . doi: 10 .1080/19312458. 2012 . 732627 .

27. Sharma , K. , Pappas , I. , Papavlasopoulou , S. , and Giannakos , M. Towards automatic and pervasive physiological sensing of collaborative learning . In Computer-Supported Collaborative Learning , 2019 .

28. Herborn , K.A. , Graves , J.L. , Jerem P. , Evans , N.P. , Nager , R. , McCafferty , D.J. , McKeegan , D.E. Skin temperature reveals the intensity of acute stress . Physiol Behav. Dec 1 ; 152 (Pt A ): 225 - 30 , 2015 . doi: 10 .1016/j.physbeh. 2015 . 09 .032.

29. Mariani , B. , Doolen , J. Nursing Simulation Research: What Are the Perceived Gaps? , Clinical Simulation in Nursing , vol 12 ( 1 ), 2016 , 30 - 36 , https://doi.org/10.1016/j.ecns. 2015 . 11 .004.

30. Garbarino , M. , Lai , M. , Tognetti , S. , Picard , R. W. , and Bender , D. Empatica E3 -A wearable wireless multi‐sensor device for real‐time computerized biofeedback and data acquisition . In Wireless Mobile Communication and Healthcare (Mobihealth) , 2014 EAI 4th International Conference on (pp. 39 - 42 ), 2014 . Athens, Greece: IEEE. https://doi.org/10.4108/icst.mobihealth. 2014 .257418

31. Taylor , S., Jaques , N. , Chen , W. , Fedor , S. , Sano , A. , and Picard , R. Automatic Identification of Artifacts in Electrodermal Activity Data" In EMBC , August 2015 .

32. Lee-Cultura , S. , Sharma , K. , Cosentino , G. , Papavlasopoulou , S. and Giannakos , M. Children's Play and Problem Solving in Motion-Based Educational Games: Synergies between Human Annotations and Multi-Modal Data . In Interaction Design and Children (IDC '21) , June 24-30, 2021 , Athens, Greece. ACM, New York, NY, USA, 19 pages. https://doi.org/10.1145/3459990.3460702