=Paper=
{{Paper
|id=Vol-2302/paper3
|storemode=property
|title=Hacia una educación consciente a través de la detección fisiológica de emociones (Towards an Emo-aware Education Through Physiological Emotion Detection)
|pdfUrl=https://ceur-ws.org/Vol-2302/paper3.pdf
|volume=Vol-2302
|authors=Franci Suni Lopez,Veronica Marisol Collanqui Puma,Luis Enrique Ancco Calisaya,Betsy Carol Cisneros Chávez
|dblpUrl=https://dblp.org/rec/conf/citie/LopezPCC18
}}
==Hacia una educación consciente a través de la detección fisiológica de emociones (Towards an Emo-aware Education Through Physiological Emotion Detection)==
https://ceur-ws.org/Vol-2302/paper3.pdf
Towards an Emo-aware Education Through
Physiological Emotion Detection
Franci Suni Lopez1 , Veronica Marisol Collanqui Puma1 , Luis Enrique Ancco
Calisaya2 , and Betsy Carol Cisneros Chávez1
1
Universidad Nacional de San Agustı́n de Arequipa, Perú
fsunilo@unsa.edu.pe, vcollanqui@unsa.edu.pe, bcisnerosc@unsa.edu.pe
2
Universidad Privada de Moquegua José Carlos Mariátegui, Perú
luis.ancco@ujcm.edu.pe
Abstract. During the different educative processes in the high school or
university, a student feels different emotions. For instance, physiological
stress has mostly experimented during exam periods, when there is aca-
demic overload, in new topics or learning too focused on memorization.
Additionally, the stress has been associated with chronic diseases (e.g.,
heart diseases, faults in the immune system, anxiety or headaches). In
line with these notions, in this paper, we introduce the idea of measur-
ing emotions in order to empower the educative processes by providing
relevant emotional information of all stakeholders involved in the educa-
tional task. This kind of information is highly useful for analyzing new
educational methodologies or for evaluating the current educational ap-
proaches (educational institutions), and for students because it will allow
a possible optimization in their teaching-learning process. Regarding the
results, we present a preliminary experiment to evaluate the emotion
detector, which obtained an accuracy of 79.17%.
Keywords: educational approach · emo-aware architecture · real-time
emotion detection · physiological stress
1 Introduction
The stress and its influence on the life of the humans have been resumed at
present with great force, driven by the new theoretical conceptions assumed, its
recognition as a disease or its association with multiple alterations of the normal
functioning of the organism. Despite its insertion in the field of medical, social
and educational sciences, a general consensus among experts on the definition
of the term stress has not been achieved. This situation has generated a con-
ceptual, theoretical and methodological diversity reflected in a wide range of
research collected in many studies. Academic stress is a systemic process, of an
adaptive and essentially psychological nature, which occurs when the student is
subjected, in school contexts, to a series of demands that, under the assessment
of the student, are considered stressors; when these stressors cause a systematic
2 Suni Lopez et al.
imbalance (stressful situation) that manifests itself in a series of symptoms (in-
dicators of imbalance); and when this imbalance forces the student to carry out
coping actions to restore the systemic equilibrium [22].
According to Arias-Gundı́n and Vizoso-Gómez [3], the main factors that gen-
erate stress in people are: poverty, constant changes in the employment situa-
tion and social, pollution and competition among co-workers and classes. Several
studies agree that entering university or school represents a set of highly stress-
ful situations, due to a lack of adaptation to the new environment. This kind
of stress can be classified as academic stress, it is expressed for example, during
exam periods [5] [16] [23], when there is academic overload [5] [16] at the begin-
ning of the in the courses [23], a teaching and learning focused on memorization
[16], when there is a lack of time [23], the demands of some subjects [16], during
the interventions in public [23], at the moment that there are methodological de-
ficiencies of the teaching staff [23] and when unsatisfactory results are obtained
[23]. For an educational institution, it is important to know the main academic
stressors in its students, given that stress has been associated with chronic dis-
eases [20], heart diseases [20], faults in the immune system [20] , anxiety [16]
[21], headaches [16], anger [21], metabolic and hormonal disorders, depression
[16] [20] [2] , sadness [16] [21]; irritability [22] [2], decrease in self-esteem, insom-
nia [22] [2], even with asthma [22], memory and concentration disturbances [2],
affecting both the health and the academic performance of the students [22].
Therefore, it is important then to carry out these kinds of studies that will
be useful; firstly for the students, because it will allow them to increase the
theoretical knowledge on the subject and with it, a possible optimization in
their teaching-learning process and secondly, for the institution, because it will
allow them to have knowledge about stress of the students who are part of it.
Finally, the paper is organized as follows: Section 2 discusses the background
and the related works on human emotions. Section 3 presents the architecture
and the algorithms used in our stress detector. The description of the experiment
and results are presented in Section 4. Finally, conclusions and future work are
discussed in Section 6.
2 Human emotions
Emotions are located in many parts of the brain. Cognitive responses are located
in the cerebral cortex, mainly in the prefrontal area. Also, they imply changes
in human behavior, autonomic nervous system, and neuroendocrine alterations.
The cerebral centers involved in these processes are located in subcortical re-
gions, in the limbic system and the brain stem [7]. The amygdala is a brain
structure located in the limbic system that has historically been directly related
to emotions, it has the size and shape of an almond and its direct electrical stim-
ulation produces subjective reactions of fear and apprehension [7]. Additionally,
the autonomic nervous system is responsible for the physiological activation of
the person. It is a basic survival mechanism that allows us to mobilize many of
the resources available for rapid action. Before the perception of a threat acti-
Towards an Emo-aware Education Through Physiological Emotion Detection 3
vates the sympathetic autonomous nervous system that would produce a series
of changes in the viscera that are detailed below. While if there is no perception
of threat and everything goes smoothly, the parasympathetic nervous system
remains activated. According to specific stimuli, the autonomic nervous system
changes the behavior of a determined physiological signal.
In this context, human emotions recognition has been investigated in different
computer science fields. For instance, in video games, Tognetti et al. proposed
to detect enjoyment in a racing game [24]. In software engineering, Muller and
Fritz presented a method to recognize the perceived difficulty of developers [17]
and in other work the frustration and the happiness [18]. Also, we can find
more proposals in the literature such as Healey and Picard [11], Tognetti [24],
Muller and Fritz [18], Lee et al. [13] or Leon et al. [14]. Overall, the different
proposed works use different data sources to recognize emotions (e.g., images,
microphone data, physiological signals or text). However, according to [7] physi-
ological signals (e.g., heart rate, electroencephalography, electrodermal activity,
electromyogram or electrocardiogram) provides a reliable way to recognize emo-
tions because this theory is based on detecting automatic physiological responses
of the body. For our practical case of education, we use Electrodermal activity
(EDA) as a source of data because EDA is one of the best real-time correlates
of stress [11]. EDA is a psychophysiological parameter that reflects the activity
of the sympathetic nervous system [8]. It could be interpreted as the level of
activation of the subject. In other words, when the subject is very activated
(i.e. high emotionality) the electrical conductance of the skin increases; on the
contrary, when the subject is little activated (relaxed), the conductance of the
skin decreases.
3 Emo-aware education approach
Figure 1 shows the architecture of our proposal; which is addressed not only
for one person but also for many students (e.g., students of a course). On the
left side, each user uses one or more physiological sensors (e.g., E4-wristband3 or
Moodmetric ring4 ); in case of the E4-wristband, this device is placed on the wrist
of the non-dominant hand of the subject. Also, the collected signals are the input
of the real-time emotion detector module. This module has the responsibility to
determine whether the user feels an emotion or not; for this paper, the physiolog-
ical stress was selected as target emotion, in other words, the detector will mark
a label of ”stressed” or ”not stressed”; for that objective, we have implemented
the pre-processing steps (see Section 4 for details) proposed by Bakker et al. [4]
for arousal detection in an integrated pipeline to enable real-time processing.
Next, this information is input for the inference engine component, which ac-
cording to the set of assessment rules in the rule base, it decides which metrics
will be sent to the messaging application module. This last module provides rele-
vant information about each student in the experiment to the stakeholders (e.g.,
3
https://www.empatica.com/research/e4/
4
http://www.moodmetric.com/
4 Suni Lopez et al.
professors, teachers or researchers). Finally, it is important to remark that each
physiological sensor placed on the subjects is directly connected by Bluetooth
to the inference engine module; in other words, the inference engine also works
as a server to receive all information.
Rule base
Real-time
Inference Messaging
emotion
engine application
detector
…
Sensors
Subjects
Fig. 1. Architecture of the emo-aware proposal.
4 Preliminary results
In this section, we present the first stage of experiments to analyze the perfor-
mance of the primary module (i.e., real-time emotion detector) of our approach.
First, it is presented the description of the algorithms used for detecting physio-
logical stress; next, it is presented the details of the experiment and the obtained
results.
4.1 Real-time stress detector
As the real-time emotion detector is the primary module of our approach, then it
is necessary to ensure a good performance of this module for the correct working
of the complete pipeline. Therefore, the goal of this preliminary experiment
is to evaluate the performance of the real-time stress detector in terms of its
accuracy. As was mentioned before, the stress detector is based on a change
arousal detection approach proposed by Bakker et al. [4].
As we use signals is required a previous step of noise filter applying a median
filter over a moving window of size n = 100 EDA samples. After it is applied
an aggregation process of one value for each 240 EDA samples. Next, the data
is discretized using the symbolic aggregate approximation (SAX) method [15].
Towards an Emo-aware Education Through Physiological Emotion Detection 5
Finally, we use a change detection algorithm based on ADaptive WINdowing
(ADWIN) method [6]; basically, this algorithm analyzes the statistically signifi-
cant difference between two consecutive splits. For instance, given φ1 and φ2 as
the means of two splits of a sequence of EDA signals, then |φ1 − φ2 | >∈cut is
the condition for a change detection that is computed with the Equation 1.
r
2 2 2 2 2
∈cut = .σ .ln + ln (1)
m W δ0 3m δ 0
2
where σW is the variance of the elements of W. δ is the desired confidence
0
and δ = δ/(ln n) [4]. Figure 2 shows the output the algorithm detecting a stress
change.
Fig. 2. Stress detection using ADWIN algorithm.
4.2 Experiment
The goal of this preliminary experiment was to evaluate the accuracy of the
real-time stress detector, for that objective the following research question was
proposed:
What is the accuracy of the real-time stress detector able to recognize physi-
ological stress changes in semi-controlled conditions?
To achieve this objective, we use two different stressful5 (i.e., the Stroop Task
[12] and an environmental noise [19]) for generating stress on participants, and
the stress detector can detect these emotional changes. Also, the used stressful is
5
A stressful is any stimulus that generates stress on the user.
6 Suni Lopez et al.
defined as the independent variable, and as dependent variables, the user stress
state (measured by the stress detector) and the reported stress by the subjects.
The experiment involved 14 subjects (i.e., master students and Ph.D. candi-
dates), whose ages ranged between 21 and 32 years old. The experiment lasted
about 30 minutes; where the subjects interacted with the two stressful by five
minutes each one. All subjects used the E4-wristband, that is a wearable device
that offers real-time physiological data acquisition. Additionally, after of the in-
teraction with the stressful, all participants are asked to complete a questionnaire
about their stress perception (self-report stress). Overall, comparing the results
of the stress detector and the reported stress by the subjects, the real-time stress
detector obtained an accuracy of 79.17% (to compute the accuracy we use the
Equation 2).
TP + TN
accuracy = (2)
TP + TN + FP + FN
Where TP indicates true positives, TN true negatives, FP false positive and
FN false negatives. In this case, examples where reported stress and stress de-
tector are labeled as stressed are considered as true positive.
5 Applications
The recognition of physiological stress on students is a valuable information for
different stakeholders and this information could be used with different proposes.
In line with this notion, in this paper we identify two potential applications,
which are explained as follow.
5.1 Evaluation of teaching-learning processes
In the last decades, from economic, political and social spheres, one of the main
objectives of the education, in any of its levels, is the quality. The legal educa-
tional regulations have emphasized this demand and for which various projects
and institutions have been launched, in order to achieve the highest levels of
quality in Peruvian institutions. In addition, there is a need for accountability,
demanded by society, with the interest of planning improvement processes that
provide as a result the increase in the quality of the education system. In which
undoubtedly, the evaluation plays an important and necessary role.
Initially, the evaluation was an activity carried out by those who were in a
position of power, authority or superiority over the people who are evaluated.
In this way, the evaluation has served and serves for the selection of people, for
the qualification of the apprenticeships, for the promotion within the system or
for the certification of socially recognized qualifications. However, the evalua-
tion is not a process that consists of controlling and demanding the evaluated,
but it is a process of reflection that requires us all to commit to knowledge and
improvement. One of the intrinsic reasons for the need for evaluation is that
Towards an Emo-aware Education Through Physiological Emotion Detection 7
an educational program cannot be designed and developed effectively and effi-
ciently without the evaluation phase is naturally present. Therefore, the value
of evaluation as a quality factor can hardly be denied. Being aware of the need
to promote evaluation procedures that address the needs, already mentioned, of
accountability and improvement of teaching, a potential use of this emotional
information is in the evaluation of educational processes.
Contextualizing in the educational area, it is possible to evaluate the negative
emotions that could generate the methodologies that use a teacher, the teaching
process, the selected educational competencies, the educational material, or the
number of hours in which the students stay in an institution. In line with this
notion, it is important to carry out these type of evaluations together with
psychologies who can supportand orient about emotional intelligence topics that
could be useful for managing emotions in relation to students behavior.
5.2 Prevention of chronic diseases
Stress as a psychophysiological response of the organism due to external or in-
ternal factors (classified as stressors). Stress can complicate the health of people
and consequently, it could result in chronic diseases. Regarding the educational
area, stress can affect both teachers and students even reaching more aggressive
pathologies such as Burnout Syndrome [9] which is an emotional disorder that
is linked to the workplace (e.g., schools). This syndrome can have very serious
consequences, both physically and psychologically.
Therefore, the collection of emotional information takes on a crucial im-
portance in order to make quick decisions in order to avoid and/or prevent
this problem from worsening. In our proposal, the collection of information is
obtained objectively, appropriately and in real-time, unlike other conventional
procedures, to provide true data that help identify cases that require prompt
help and derive this information to the corresponding professionals avoiding so
that the problem becomes chronic. This information collection could be done
during a month, approximately, both to professors and students and from the
information that is obtained begin to plan actions oriented to the prevention
and treatment of stress.
6 Conclusions and future works
The academic stress whose source is in the educational environment is a question
that affects the students’ learning and their well-being.
In this paper, we present an emo-aware architecture for providing emotional
information of students (from high school or university) during a teaching-
learning process. We argue that this kind of emotional information is funda-
mental to understand the different behaviors in the student reactions involved
in a teaching-learning process. Another important contribution of our approach
is for the students, which can use the information collected from themselves
8 Suni Lopez et al.
to analyze their learning difficulties and after to make decisions about how to
optimize their educational processes.
In the first stage of experiments (analyzing only the real-time emotion de-
tector component), an experiment was conducted with 14 subjects using the
E4-wristband device to gather physiological data (physiological sensors). Com-
paring the outcome of our stress detector with the reported by each subject
(perceive stress), the real-time stress detector obtained an accuracy of 79.17%.
Overall, we can conclude the real-time emotion detector based on stress recog-
nition has had a good performance for detecting physiological stress in semi-
controlled conditions (i.e., in a room), because this result show a good accuracy
in comparison with other machine-learning based on recognition methods, due
to it oscillates between 70% and 85% [1, 10], values reported in the literature of
stress recognition using physiological data.
As part of our future work, it is important to generate interest in future
research in which academic stress is the focus of attention, because it is necessary
to prevent or even cushion the effects of stress in students, although it may seem
to some to be unimportant in comparison with others, it is closely related to
undesirable alterations, such as memory failures at the moment of performing a
stressful test, or failures in the learning process itself. Also, we plan to conduct
a series of simulation-based experiments to assess our inference rules. Then, we
plan to conduct experiments with multiple groups of subjects for evaluating the
relevance of the information delivered by our prototype application.
References
1. Alberdi, A., Aztiria, A., Basarab, A.: Towards an automatic early stress recogni-
tion system for office environments based on multimodal measurements: A review.
Journal of Biomedical Informatics 59, 49–75 (feb 2016)
2. Amat V, Fernández C, O.I.P.M.R.M.R.D.: Estrés en estudiantes de enfermerı́a.
Rev. Rol enferm. 133, 75–78 (1990)
3. Arias-Gundı́n, O., Vizoso-Gómez, C.: Causas de estrés académico en estudiantes
universitarios (2016)
4. Bakker, J., Pechenizkiy, M., Sidorova, N.: What’s your current stress level? detec-
tion of stress patterns from gsr sensor data. In: Proceedings of the 2011 IEEE 11th
International Conference on Data Mining Workshops. pp. 573–580. ICDMW ’11,
IEEE Computer Society, Washington, DC, USA (2011)
5. Barraza A, Martı́nez JL, S.J.C.E.A.R.: Estresores académico y género: un estudio
exploratorio de su relación en alumnos de licenciatura. VE-IUNAES 5(12), 33–43
(2011)
6. Bifet, A., Gavaldà, R.: Learning from time-changing data with adaptive windowing.
In: Proceedings of the 2007 SIAM International Conference on Data Mining, pp.
443–448. Society for Industrial and Applied Mathematics (apr 2007)
7. Boucsein, W.: Electrodermal Activity. Springer US (2012)
8. Dawson, M.E., Schell, A.M., Filion, D.L., Berntson, G.G.: The elec-
trodermal system. In: Cacioppo, J.T., Tassinary, L.G., Berntson, G.
(eds.) Handbook of Psychophysiology, pp. 157–181. Cambridge Uni-
versity Press (2007). https://doi.org/10.1017/cbo9780511546396.007,
https://doi.org/10.1017/cbo9780511546396.007
Towards an Emo-aware Education Through Physiological Emotion Detection 9
9. Embriaco, N., Papazian, L., Kentish-Barnes, N., Pochard, F., Azoulay, E.: Burnout
syndrome among critical care healthcare workers. Current Opinion in Critical
Care 13(5), 482–488 (oct 2007). https://doi.org/10.1097/mcc.0b013e3282efd28a,
https://doi.org/10.1097/mcc.0b013e3282efd28a
10. Garcia-Ceja, E., Osmani, V., Mayora, O.: Automatic stress detection in working
environments from smartphones x2019; accelerometer data: A first step. IEEE
Journal of Biomedical and Health Informatics 20(4), 1053–1060 (July 2016)
11. Healey, J.A., Picard, R.W.: Detecting stress during real-world driv-
ing tasks using physiological sensors. Trans. Intell. Transport. Sys.
6(2), 156–166 (Jun 2005). https://doi.org/10.1109/TITS.2005.848368,
http://dx.doi.org/10.1109/TITS.2005.848368
12. Lattimore, P.: Stress-induced eating: an alternative method for inducing ego-
threatening stress. Appetite 36(2), 187–188 (apr 2001)
13. Lee, M., Kim, K., Rho, H., Kim, S.J.: Empa talk: A physiological data
incorporated human-computer interactions. In: CHI ’14 Extended Abstracts
on Human Factors in Computing Systems. pp. 1897–1902. CHI EA ’14,
ACM, New York, NY, USA (2014). https://doi.org/10.1145/2559206.2581370,
http://doi.acm.org/10.1145/2559206.2581370
14. Leon, E., Clarke, G., Callaghan, V., Sepulveda, F.: A user-
independent real-time emotion recognition system for software
agents in domestic environments. Eng. Appl. Artif. Intell. 20(3),
337–345 (Apr 2007). https://doi.org/10.1016/j.engappai.2006.06.001,
http://dx.doi.org/10.1016/j.engappai.2006.06.001
15. Lin, J., Keogh, E., Lonardi, S., Chiu, B.: A symbolic representation of time series,
with implications for streaming algorithms. In: Proceedings of the 8th ACM SIG-
MOD Workshop on Research Issues in Data Mining and Knowledge Discovery. pp.
2–11. DMKD ’03, ACM, New York, NY, USA (2003)
16. Marı́n MM, Álvarez CG, L.A.A.A.L.B.: Estrés académico en estudiantes: El caso de
la facultad de enfermerı́a de la universidad michoacana. rev. iberoam. producción
académica gest. educ 1(17) (2014)
17. Müller, S.C.: Measuring software developers’ perceived difficulty with biometric
sensors. In: Proceedings of the 37th International Conference on Software Engi-
neering - Volume 2. pp. 887–890. ICSE ’15, IEEE Press, Piscataway, NJ, USA
(2015), http://dl.acm.org/citation.cfm?id=2819009.2819206
18. Müller, S.C., Fritz, T.: Stuck and frustrated or in flow and happy: Sensing develop-
ers’ emotions and progress. In: Proceedings of the 37th International Conference on
Software Engineering - Volume 1. pp. 688–699. ICSE ’15, IEEE Press, Piscataway,
NJ, USA (2015), http://dl.acm.org/citation.cfm?id=2818754.2818838
19. Passchier-Vermeer W, P.W.: Noise exposure and public health. Environ Health
Perspect. p. 108(suppl 1): 123–31 (2000)
20. Pulido MA, Serrano ML, V.E.C.M.H.P.V.F.: Estrés académico en estudiantes uni-
versitarios. Psicologı́a y Salud 21(1), 31–37 (2011)
21. Rivadeneira C, Minici A, D.J.: Algunas puntualizaciones sobre el estrés. Revista
de terapia cognitivo conductual 23, 1–7 (2013)
22. y Rodrigo Mazo Zea, N.B.G.: Estrés académico. Revista
de Psicologı́a Universidad de Antioquia 3(2), 55–82 (2012),
https://aprendeenlinea.udea.edu.co/revistas/index.php/psicologia/article/view/11369
23. Rodrı́guez B, González MP, B.L.: Estresores académicos percibidos por estudiantes
pertenecientes a la escuela de enfermerı́a de Ávila, centro adscrito a la universidad
de salamanca. Rev. enferm. CyL 6(2), 98–105 (2014)
10 Suni Lopez et al.
24. Tognetti, S., Garbarino, M., Bonanno, A.T., Matteucci, M., Bonar-
ini, A.: Enjoyment recognition from physiological data in a car rac-
ing game. In: Proceedings of the 3rd International Workshop on Affec-
tive Interaction in Natural Environments. pp. 3–8. AFFINE ’10, ACM,
New York, NY, USA (2010). https://doi.org/10.1145/1877826.1877830,
http://doi.acm.org/10.1145/1877826.1877830