In this paper, we present the main achievements and analysis of our ongoing piece of research about the use of wearables in educational contexts. This work has led us to explore the use of wearables in educational environments with the aim of enriching the student profile with new information, such as sleep and stress indicators. These indicators can be estimated through the use of widely available wearable device sensors, namely Commercial-Off-The-Shelf wrist wearables. The first step has been to validate the use of these devices and their sensors as good predictors of stress and sleep. Once this has been validated, we have proposed some indicators that offer the user the opportunity to get information such as the sleepiness quality, the chronotype, the latent stress of the stress regularity. These are different features that contribute to get a better knowledge of the learner and his state. In this paper, we will present the main research lines that make up our project, the elements implemented in our system and how these indicators can be applied in educational environments.
The appearance of the first Commercial-Off-The-Shelf (COTS) wearables as electronic devices and systems incorporated in some part of our body or clothes [1, 2] became a reality projects like Google Glasses and the first wrist wearables. The commercialization of this new type of devices generated a high expectation in all areas, from sport [3] to medical environments [4, 5]. These devices offer the ability to see the world and interact with the environment in a completely different way. The inclusion of portable sensors such as the accelerometer, Heart Rate (HR), skin temperature (ST), Galvanic Skin Response (GSR) and other sensors, offer users new self-quantification and training systems never seen before. But, certainly among all the available wearables wrist wearables (watches and wrist bands) are the reference COTS wearable par excellence.
This type of devices has a greater presence in the market, leaving aside any other available option [6, 7].
Copyright © 2017 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
Currently, wrist wearables are the main COTS wearables and among sensors available on them there is one whose presence is clearly superior to any other one: the accelerometer. This sensor is without a doubt the most present sensor in COTS wearables [8]. This sensor allows to quantify the movement in three axes and therefore to detect the level of activity of the subject (number of steps, distance walked, speed, etc.). It can also be used for other purposes, for example allows to study the level of rest while sleeping through the application of actigraphy techniques using the accelerometer data [9]. These analyses allow us to provide an estimate of sleep quality. Although the estimated results are not as accurate as in the case of polysomnography [10], they are acceptable [11]. Therefore, the use of wearables including this sensor allows to obtain a daily estimate of physical condition and rest without the need of any medical equipment, which is inaccessible and expensive, although much more precise, of course.
Other sensor commonly present in COTS wrist-wearables is the HR [9]. The use of this sensor in combination with the accelerometer allows to obtain more precise results about features such as the training level, calories consumed (during the sport) or the level of deep rest during sleep. In addition, this sensor combined with GSR could be used to detect the level of agitation and anxiety in work and educational settings. The variation of these physiological signals is completely related to stress, as indicated in the following papers [12–16].
Focusing on education, both stress and sleep have an extensive amount of scientific papers explaining how they affect students in educational settings. The existing literature demonstrates that sleep has an influence on the ability to acquire new knowledge as well as in the academic achievement of students [17, 18]. Students’ cognitive functions may be affected by levels of sleepiness as a result of poor sleep quality [19]. Furthermore, as indicated in [20, 21], sleep deprivation affects the abilities to memorize and think and even low motivation of students may be related to sleepiness [22]. Related to stress, it can be experienced by students through an emotional exhaustion and a decrease in professional effectiveness as a result of trying to reach challenges too demanding without the necessary resources. This type of stress is described as Burnout syndrome [23, 24]. The effect of stress on students is especially important among firstyear college students [25, 26] and for this reason between a 20% and 30% of the students drop out during such a year of college studies [25, 27]. As a result, it seems of great importance to analyse both stress and sleep and to provide tools so that students are able to manage their time, resources and environment in a better way.
Following these ideas our research group is interested on exploring the use of consumer electronics wrist wearables, to feature learners and provide educational profiles that can be used to enrich the educational support and improve learning performance.
Since the field of application is broad, we decided to start our study from a global perspective to focus on the applicability of existing results in the educational environment.
In this way, this article aims to provide a summary of all those questions that we have studied over the last few years and our proposal of application for education, with the aim of improving academic performance and increasing self-regulated learning.
The rest of the paper is organized as follows. First, in section 2, research points and related work are introduced, describing important issues that need to be taken into account when working with wearables. In section 3, it is provided an overview of the developed system. Then, in section 4 application areas in education for indicators extracted from the wearables explained. The paper finishes with some conclusions in section 5. 2
Our focus of study combines a total of 4 points of interest. Three of them are analysed from a general point of view, while a third point is focused on the applicability of the previous ones to educational scenarios: • Homogenization of COTS wearable devices [9]. The use of different devices is not direct, because they present very heterogeneous features, sensors, software platforms, protocols, etc. In order to work with COTS wearables from different vendors, a platform is needed that unifies the incoming data and that offers a service of homogenization independent of the type of wearable and the data that it supplies. • Estimation of sleep indicators [28]. As it has been described in the introduction, accelerometer and HR data can be used to estimate the level of sleepiness of a subject. We have been working on the development specific sleep indicators, such as: sleep quality, sleepiness, chronotype and sleep regularity. These indicators are proposed to create a profile of the student sleep. • Estimation of stress indicators. This is focused on estimating a subject stress levels through collecting data from wearable sensors on which useful indicators can be offered, such as: snapshot stress, aggregated stress, latent stress and stress regularity. • Applicability of sleep and stress indicators in education. Through a student enriched profile based on sleep and stress indicators, it could be possible to offer valuable information and services for education, such as: to support the creation of groups with homogeneous students according to the chronotype, to support the management of classroom activities to reduce the global stress, to include reactivating tasks against sleepiness, etc.
The result is a system that is able to collect information from a set of wearables and offer a much more detailed student profile. 2.1
In order to start this project, we reviewed the state of the art about similar projects in this field. This work has led us to understand that the applicability of wearables, not just COTS wearable wristbands, is still at a very early stage, with enormous possibilities, but also with a major effort on the part of developers and researchers. These studies are divided into two types of classes: (i) studies that use wearables as an educational tool to develop specific projects; and (ii) analysis of the use of wearable sensors and capabilities to feature students and enrich their profile.
As examples of the use of wearables to be used as an educational tool, it is possible to find from the use of T-shirt to represent human body organs and their relations [29], use of small microchips to carry out small educational projects [30], and use of Bluetooth Low Energy BLE beacons for interaction with real physical objects [31] to projects whose main focus are Google Glasses with the aim of using augmented reality for experiments [32].
Meanwhile, as examples of wearable sensors and capabilities to feature students, it is possible to find examples such as the use of wearables by teachers in accordance to a new approach known as Teaching Analytics [33]. Also, projects focused on our research line, using basic variables collected from a wearable such as number of steps, HR, and the time and answers for a brief questionnaire, in order to estimate student performance and stress [34], analysing the engagement in different educational activities [35], or the social relations among the students [36]. We emphasize in this line StudentLife project [37], that uses smartphones (not wearables) to collect the data of apps usage, user location, physical activity, number of conversations and duration of each conversation per day, and so on, and relates them with the GPA, mental health (e.g. stress) and behavioural patterns. In this sense, even if the variables collected are not wearable, their objectives are closely related to our project.
As it can be seen, the literature on this subject is still at an early stage, and the existing projects focus on analysing the most intuitive and quick inputs that can be used with wearables, marking the path of more complex studies that will arise as a result of them. From our knowledge, there are no studies that use wrist wearables to enrich the student profile and improve academic environments through indicators of stress, sleep and the analysis of activities that are more sleepy or stressful. Once these indicators are proposed, we believe that there are innumerable works that will take advantage of them and eventually will provide useful information and services for education and teaching purposes. 2.2
Before analysing the proposed system and its applicability in educational environments we believe it necessary to transmit to the reader the most important ideas that surround the wearables, analysing their diversity and their main problems. Although these and other aspects have been addressed in an earlier publication [9], here it is a brief summary of the most relevant issues.
COTS wearables are devices that have a wide range of possibilities, although it is true
that wrist bands and watches are over the rest as indicated in [7]. These devices had
approximately 50.5% and 41% of the mar
The diversity of devices is followed by the high diversity of sensors. GPS, accelerometer, Heart Rate (HR), gyroscope, barometer or thermometer are just some examples of the multiple sensors available. Notice that they do not have the same degree of penetration in the market. Sensors such as the accelerometer and the HR are included respectively in more than 86% and 33% of the devices [9]. This fragmentation of devices is accompanied by the fragmentation of operating systems included on them. Different companies like Google, Samsumg, Apple, have presented their own software platforms and their operating systems for this type of private devices. This also contributes to increase the level of fragmentation.
Unlike PCs or smartphones, the world of wearables presents many difficulties because the problems of fragmentation explained above. This results in many issues when data from sensors of different devices needs to be collected and processed. These problems are referred to as homogenization problems. Although two wearables have the same sensors, collection of their data may be very different [9]. Moreover, even if the data is accessible on both devices, the delivery of the data to our analysis server will depend to a great extent on which tools are available, protocols, authentication tokens, etc. Once the server is reached, a new problem arises, the data of different wearables follows different data models, vocabularies, and these must be homogenized. For example, the sleep states are named by Fitbit as ("asleep", "awake" and "really awake") and By Microsoft as ("awake", "light" and "deep").
Finally, another of the main problems in the world of wearables is its accuracy. Each sensor has a certain precision, but this information in many cases is not available on the manufacturers' websites, and for this reason their accuracy is discussed in some papers [38, 39]. Nevertheless, there are also recent studies that validate the wearables' HR accuracy within a reasonable margin of error [40–42], especially in conditions of low movement or rest, common situations in a classroom. In any case, their use has not yet been accepted in medical settings yet [43–45]. 3
Taking into account the current state of similar projects and the issues with COTS wearables, in this section we will present the state of development of our research project. We reviewed the COTS wearable market to select the wearables to be used in our studies. The following ones were tested: Fitbit Charge, Fitbit Charge 2, Fitbit Surge, LG Watch R, Microsoft Band, Microsoft Band 2, Moto 360 v1 and Jawbone.
Initially, to perform the data collection we developed a system running on a Linux machine, written in Java and running on a Tomcat server. This development allows us to automatically collect the data from the wearables directly, via an intermediate smartphone or PC, as well as some data from wearable vendor servers, because in some cases they store and provide services to get users’ data on-line. Therefore, in our system, using a REST API and several applications running on Android devices, the data is transferred to our server. Later this data is analysed and processed to allow its visualization in a teacher and student dashboard and the estimation of indicators.
To get the information we need to compile for stress and sleep indicators, we developed several applications for Android: • Survey app for the sleep study. Our first goal involved offering a set of sleep indicators, including Sleep Quality. This indicator required a validation based on user responses. In addition, the system is trained and improved through the responses of the user perceived slept at each night. For this reason, we made a survey-app, which every day asked at the same time how tired the subject felt on that day. A capture of the application with only one answer and no data about the sleep is shown in the next Fig. 1. • App for collecting data from wearable. This application is used to collect in real time the values of the sensors and once collected, through internet access, send them to our server. It is important to notice, as indicated in [9], not all wearable devices allow this type of access in real time, in particular. The device offering greater flexibility was Microsoft Band 1 and 2, because it is possible to retrieve all the sensor data of these bracelets in real time. In addition, this app is prepared to serve as the student dashboard application, showing data about stress and sleep features, as shown in the Fig. 2. Moreover, this app enables the user to register daily, the stress level, or the level of sleepiness, and thus improve the classification system to predict news states of stress and sleepiness. • Application for the study of stress. In order to validate stress or as a pre-training phase of the model, we perform an application that combines several activities to provoke relaxation, stress and concentration in the users who perform them. These activities have been described in the literature in stress-related studies: a relaxing video, Stroop Colour-Word Interference Test [46], the PASAT text [47] and an Hyperventilation exercise [16]. A capture of two layouts of the application is shown in the Fig. 3.
In addition to the applications mentioned above, we have also implemented a small dashboard, where the student and the teacher can see the variation of physiological signals and several of the proposed indicators. This dashboard is still under development, although several of the features are already available. In Fig. 4, we show an example of what would be the dashboard of a teacher-subject, visualizing the average level of stress / concentration / relaxation of the class and its variations during teaching hours.
In this section, we explain how the stress indicators, such as the Snapshot Stress level (SS) Aggregated Stress (AS), and Latent Stress (LS), and the indicators of sleep, such as the Sleep Quality (SQ), Sleepiness (S) or Chronotype (CT)), can be applied in educational environments with the aim of assisting student learning tasks, teachers and developers of educational content.
To understand the possibilities offered by these indicators, it is necessary to briefly explain what the function of each one of them is. On the one hand, the SS will indicate the level of stress that the users feel at a particular moment, the AS indicates the stress percentage along the day while the LS indicates the levels of stress that a student can accumulate after several days of activities and consecutive exams. The first indicator provides an instant measure of stress while the second and the third offers a cumulative value. Related to sleep, SQ evaluates the quality of sleep during a night; S offers the level of tiredness and sleepiness that the student experiences; and finally, the CT indicates the most suitable hourly range for going to sleep and get up. While S is an instant value, SQ value is daily and CT is a value analysed in broad temporal periods, as for example 4 weeks.
The final purpose of our work is to support teachers, students and developers on the performance of their educational activities. Next sub-sections provide more details about our goals focusing on services for these roles. Our main focus is to apply these indicators to enrich the student profile and enable him to manage and organize by himself his activities and efforts more accurately and therefore increase his performance and learning. This idea fits perfectly in the Self-Regulated Learning (SRL) approach [48, 49]. As indicated in [50], Self-regulated learning (SRL) autonomy and control by the individual is emphasized, because it is the student who monitors, directs, and regulates actions towards goals of information acquisition, expanding expertise, and self-improvement. But even if a user knows how he feels, or what activities are the ones that stress him the most, sometimes he does not know how to interpret these values, or what decisions he should make. That is the reason to offer some technical support that can help him to monitor and reflect about his psychological and physiological states. This kind of information can also be useful in the context of affect-aware tutors [51], as long as these systems base their actions on the features and states of the students. In both cases, the use of stress and sleep indicators can lead to enhance the performance of tasks autonomously and with a high performance. We conceive a system that will offer the following tools: • Values of indicators and graphs. Sometimes a simple value of an indicator is not enough. The representation of values and their variation along time in a graph are necessary for the user to detect some problem in their levels of rest and stress. That is the reason why graphs are proposed to represent snapshot stress, latent stress, sleep quality or sleepiness indicators. • Notifications and alerts. Depending on the levels of stress and sleep indicators, notifications will be generated that the user can understand as suggestions of the system to improve some behaviours. Similarly, alerts that will show a worrying warning about some of the behaviours can be triggered, indicating how they could be tackled. A notification would be a low level notice, while an alert will show a high level of danger. Some conditions to trigger alerts could be high values of latent stress, aggregated stress with constant increases, sleep quality lower than 50% for more than one week, a high snapshot stress time rates during classes or a sudden variation in the chronotype during a certain period of time. • Through the analysis of the activities that have generated higher levels of sleepiness and snapshot stress, a work agenda can be proposed to the student interleaving activities to maintain an adequate level of stress. The idea would be neither to generate too many monotonous and boring activities, nor to saturate students with difficult and costly activities that generate too much stress. 4.2
Although our main focus is situated in the context of self-regulated and affect-aware tutors, the proposed indicators can be useful for other purposes in educational systems. For example, these indicators could help teachers to select and improve learning resources. In this way, the indicators can also be useful for content providers, because the stress and sleepiness reactions of students while experiencing contents can be very useful to improve them
For these reasons, we have explored how the proposed indicators can offer support during teaching and in the creation of learning contents. In this case, we conceive the following tools: • Values of indicators and graphs. As in the case of students, the representation of graphs in a dashboard will allow the teacher to get a global view of the students and their average indicators. With indicators, such as the sleepiness, the teacher could know if his last lecture has been as engaging as hoped. Depending on the results and variations over time the teacher could change his plans, for example, trying to get students stay on average in low levels of sleepiness and stress. • Arousal level detection. Based on papers such as [34], detection of stress and mood states during exercises, exams or during lectures, allows to draw a map of the psychological profile for each student. In our case, we could use latent stress, variations in chronotype and regularity of sleep, to understand the mood of students. These indicators provide a snapshot of how the previous days have been and therefore of the accumulated fatigue and stress. • Creation of working groups. One of the most direct utilities of the use of indicators such as the chronotype is the creation of working groups. Creating working groups with similar sleep profiles will allow students to agree on a broad range of hours. If people with different sleep profiles are joined this can introduce difficulties for arranging meetings and plan activities. In other scenario, if it is detected that the activities of oral presentation in front of the classroom are especially traumatic for a student, it could be wise to join him in a same group with students whose speech facility and level of stress in this type of activities is smaller. • Timetable. Analysing the level of sleepiness and stress along with accumulated values (latent stress, sleep regularity) both outside and inside the classroom, and combining this information with the school and extracurricular hours could allow to discover specific difficulties for a student, higher workload day, or hours of the day at which arousal levels are higher. With this, the teacher could make better personalized work plans, or decide about the best days for the exams. From a more institutional point of view, the educational institution itself could structure teaching hours based on these indicators, avoiding tasks of high level of concentration in the hours where students do not present high performance • Improving the learning resources. As it has been said about the teacher, there is also the task of producing the educational content to be used by learners. A good way to know if this content is adequate would be to evaluate the levels of stress and sleepiness during their use. Those contents that generate very high levels of stress should be reconditioned, thus avoiding that these levels could lead to frustration and abandonment by the student. Likewise, activities that do not generate any stress and that raise the levels of sleepiness of the student should be eliminated or placed between tasks that generate a depletion above the average. 5
The existing literature demonstrates the role of stress and sleep in academic and learning performance. Proper stress control and a good level of rest would reduce dropout rates and improve student achievement. To enable the measurement of stress and sleep related features, devices such as wearables are presented as a good alternative because of their great advantages: they are comfortable, programmable, and have a large number of sensors. Nevertheless, studies focused on the use of these devices have revealed that this field of research is still at an early stage where it is possible to explore new alternatives of applicability in educational settings. Our proposal is focused on studying if the detection of stress and sleep through COTS wrist wearables is possible. To achieve this, we have developed our own homogenization / analysis server and several apps on Android to collect, adapt and analyse data from wearables sensors. This complete system allows us to calculate indicators of sleep and stress, such as sleep quality, sleepiness, chronotype, snapshot stress, aggregated stress, latent stress, etc. In general, the use of these indicators results in an enrichment of the student profile. This enrichment allows the student to know better his levels of stress and sleep, to understand what routines and strategies provide better results and helps him to achieve an efficient management of his learning time. Similarly, the teacher could know first-hand his students, efficiently distribute the contents of his subject throughout the term and know what activities are better and worse for the students.
Finally, although the feasibility of using these devices seems valid, our studies also show that the use of COTS wearables has limitations. On the one hand, the precision of the sensors, or the low penetration of interesting sensors such as the GSR or ST, prevent the same advantages for any wearable device. Factors such as the lack of homogenization, or the programming efforts required to set up a system, are also a problem. On the other hand, solved all the previous problems, there are other factors like the ambient conditions (e.g. temperature) or the level of movement that affect the physiological signals measured by the wearable and therefore to the possible results obtained. 6
The authors acknowledge the support of SNOLA, officially recognized Thematic
Network of Excellence (TIN2015-71669-REDT) by the Spanish Ministry of Economy and
Competitiveness, the Spanish State Research Agency, the European Regional
Development Fund (ERDF) un
Mastrandrea, R., Fournet, J., Barrat, A.: Contact Patterns in a High School: A Comparison between Data Collected Using Wearable Sensors, Contact Diaries and Friendship Surveys. PLoS One. 10, e0136497 (2015).
Dror Ben-Zeev, Andrew Campbell, Fanglin Chen, Zhenyu Chen, Tianxing Li, Rui Wang and Xia Zhou (Dartmouth College), Gabriella Harari (University of Texas at Austin), Stefanie Tignor (Northeastern University)Wang, R., Campbell, A.T.: StudentLife Study, http://studentlife.cs.dartmouth.edu/, last accessed 2016/07/29.
Guo, F., Li, Y., Kankanhalli, M.S., Brown, M.S.: An evaluation of wearable activity monitoring devices. In: Proceedings of the 1st ACM international workshop on Personal data meets distributed multimedia - PDM ’13. pp. 31– 34. ACM Press, New York, New York, USA (2013).
Richmond, S.: The real world wrist-based heart rate monitor test: Are they accurate enough?, http://www.wareable.com/fitness-trackers/heart-ratemonitor-accurate-comparison-wrist, last accessed 2016/11/16.
Wang, R., Blackburn, G., Desai, M., Phelan, D., Gillinov, L., Houghtaling, P., Gillinov, M.: Accuracy of Wrist-Worn Heart Rate Monitors. JAMA Cardiol. 2, 104 (2017).
Stahl, S.E., An, H.-S., Dinkel, D.M., Noble, J.M., Lee, J.-M.: How accurate are the wrist-based heart rate monitors during walking and running activities? Are they accurate enough? BMJ Open Sport Exerc. Med. 2, e000106 (2016). Wallen, M.P., Gomersall, S.R., Keating, S.E., Wisløff, U., Coombes, J.S., Smith, B.: Accuracy of Heart Rate Watches: Implications for Weight Management. PLoS One. 11, e0154420 (2016).
Rudner, J., McDougall, C., Sailam, V., Smith, M., Sacchetti, A., O’Brien, T., Troutman-Jordan, M., Hathaway, D., al., et, Shin, D.W., Joh, H.K., Yun, J.M., al., et, Evenson, K.R., Goto, M.M., Furberg, R.D., El-Amrawy, F., Nounou, M.I., Wang, J.B., Cadmus-Bertram, L.A., Natarajan, L., al., et, Rowe-Roberts, D., Cercos, R., Mueller, F.: Interrogation of Patient Smartphone Activity Tracker to Assist Arrhythmia Management. Ann. Emerg. Med. 68, 292–294 (2016).
Wallis Snowdon, C.N.: Apple watch saved Alberta man’s life, makes international headlines - Edmonton - CBC News, http://www.cbc.ca/news/canada/edmonton/apple-watch-saved-alberta-man-slife-makes-international-headlines-1.3495397, last accessed 2016/11/16. Paul Burton: Apple Watch Saves Life Of Tabor Academy Football Player, http://boston.cbslocal.com/2015/09/20/apple-watch-tim-cook-heart-rate-taboracademy-marion-massachusetts-paul-houle/, last accessed 2016/11/16. Zhai, J., Barreto, A.: Stress Detection in Computer Users Based on Digital Signal Processing of Noninvasive Physiological Variables. In: 2006 International Conference of the IEEE Engineering in Medicine and Biology Society. pp. 1355–1358. IEEE (2006).
Tombaugh, T.N.: A comprehensive review of the Paced Auditory Serial Addition Test (PASAT). Arch. Clin. Neuropsychol. 21, 53–76 (2006).
Schunk, D., Zimmerman, B.: Self-regulated learning: From teaching to self