The potential of Ambient Intelligence to deliver Interactive Context-Aware Affective Educational support through Recommendations

Abstract

There is a challenge and opportunity to research if the ambient intelligent support that can be deployed with a recommender system extended with an open hardware infrastructure that can sense and react within the learners' context is of value to supports learners' affectively. In this paper, we summarize the status of our research on eliciting an interactive recommendation for a stressful scenario (i.e., oral examination of a foreign language) that can be delivered through the Ambient Intelligence Context-aware Affective Recommender Platform (AICARP), which is the infrastructure we have designed and implemented with Arduino, an open-source electronic prototyping platform.

Eliciting Interactive Recommendations with TORMES

We have reported elsewhere [ 1 ] our progress on analyzing the potential of Ambient Intelligence to deliver more interactive educationally oriented recommendations that can deal with the affective state of the learner. In particular, following the TORMES methodology [ 2 ], we elicited an educational scenario focused on helping the learner when preparing for the oral examination in a second language learning course, which is widely considered as a stressful situation.

The recommendation identified in this scenario consists in suggesting the learner to breathe slowly (at a rate of 4 breaths/minute) and is aimed to calm her down when she is nervous. The applicability conditions that trigger the recommendation take into account physiological (i.e., heart rate, pulse, skin temperature, skin conductance) and behavioral (facial/body movements and speech speed) information that show evidence of restlessness. The recommendation output has been coded in a multisensory way by simultaneously modulating light, sound and vibration behavior at aforementioned breath rate, so the learner can perceive the recommended action through alternative sensory channels (i.e., sight, hearing and touch) without interrupting her activity.

Delivering Interactive Recommendations with AICARP

To deliver the aforementioned recommendation elicited with TORMES, the Ambient Intelligence Context-aware Affective Recommender Platform (AICARP) is being implemented with open source software and open hardware following a modular design controlled by an Arduino board (see [ 1 ] for details). In the current version, AICARP receives information from physiological sensors regarding changes in the learner affective state through corresponding physiological signals. The sensors are integrated into the e-Health platform [ 3 ] and a wireless electrocardiogram system [ 4 ]. Taking into account this information, AICARP is able to provide the elicited interactive recommendation to the learner by modulating the output of alternative sensorial actuators with the recommended breath rhythm. In particular, the following actuators have already been integrated into AICARP: i) white and red flashlights, ii) an array of blue LEDs, iii) a buzzer that vibrates and sounds, and iv) a speaker reproducing a pure tone at 440 Hz (i.e., “La” musical note).

To get some insight on the users’ perception on the recommendation delivery, we have deployed the educational scenario outlined in Section 1 in order to deliver the corresponding recommendation elicited with TORMES. So far, in this context we have carried out 2 pilot studies, one with 6 university students with various interaction needs -including a blind participant-, and another with 4 participants within the 2014 Madrid Science Week. Since we wanted to test the potential of this approach in detecting not only the physiological information but also the behavioral information, we used the Wizard of Oz method [ 5 ]. In this way, the recommendation was triggered by the wizard (in our case, a psycho-educational expert) considering participants’ information on both physiological evidences detected with AICARP, as well as body/facial movements and speech speed that the wizard observed while the participants carried out the two tasks defined in the pilots (i.e., talking aloud in English about two specific given topics selected from those usually considered in oral exams). 3

Evaluation Outcomes and Open Issues identified

We evaluated AICARP in the 2 pilot studies with the analysis of the participants’ responses to the System Usability Scale [ 6 ] and to a post-study consisting in a semi structured interview led by the psycho-educational expert. This evaluation showed that the implemented infrastructure can actually sense the physiological state of the learner (which seems to be related to some affective state) and deliver ambient intelligent interactive feedback aimed to transform a negative affective (i.e., nervousness) state into a positive one (i.e., relaxation) (see [ 1 ] for details on the evaluation results). To the latter, actuators considered aim to provide a natural interaction support not interfering with the participant’s task, and consisted of visual, audio and/or tactile feedback.

As discussed in [ 1 ], the analysis of the evaluation outcomes has identified several open issues to be addressed in future research, as follows:

Acknowledgements

This work was supported by the Spanish Ministry of Economy and Competitiveness (MINECO) under Grant TIN2011-29221-C03-01 (MAMIPEC project).

References

1. Santos , O.C. , Saneiro , M. , Rodriguez-Sanchez , M.C. and Boticario , J.G. ( 2015 ) Towards Interactive Context-Aware Affective Educational Recommendations in Computer Assisted Language Learning. New Review of Hypermedia and Multimedia , in press.

2. Santos , O.C. , Boticario , J.G. ( 2015 ) Practical guidelines for designing and evaluating educationally oriented recommendations . In Computers and Education , vol. 81 , 354 - 374 .

3. Cooking Hacks. E-Health Platform . Available from: http://www.cooking-hacks.com.

4. Torrado-Carvajal , A. , Rodriguez-Sanchez , M.C. , Rodriguez-Moreno , A. , Borromeo , S. , Garro-Gomez , C. , Hernandez-Tamames , J. A. , and Luaces , M. ( 2012 ) Changing communications within hospital and home health care . In 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) , 6074 - 6077 .

5. Dahlbäck , N. , Jönsson , A. , and Ahrenberg , L. ( 1993 ) Wizard of Oz studies: why and how . In Proceedings of Intelligent User Interfaces , 193 - 200 .

6. Brooke , J. ( 1996 ) SUS: a 'quick and dirty' usability scale . In Jordan , P.W., Thomas , B. , Weerdmeester , B.A. and McClelland , A.L. Usability Evaluation in Industry . London: Taylor and Francis.

7. Salmeron-Majadas , S. , Arevalillo-Herráez , M. , Santos , O.C. , Saneiro , M. , Cabestrero , R. , Quirós , P. , Arnau , D. and Boticario , J.G. ( 2015 ) Filtering of Spontaneous and Low Intensity Emotions in Educational Contexts . 17th Int. Conf. on Artificial Intelligence in Education (AIED 2015). Lecture Notes in Artificial Intelligence , vol. 9112 , 429 - 438 .