Children with Autism Spectrum Disorder can nd very difcult focusing their attention towards activities they nd less attractive than the ones they like. In this paper we will introduce a prototype designed to reinforce their mental skills through neurofeedback using EEG data and a small humanoid robot to stimulate attention towards a joint activity.
As autism Spectrum Disorder is characterized by restricted and repetitive
patterns of behaviors, interests and activities [
Past studies highlighted the possibility of employing a small humanoid robot
to elicit joint attention [
Methodology The system presented in this paper is a prototype designed to reinforce the attention of a child towards a particular activity, thanks to the feedbacks o ered by a small humanoid robot. Among the possible choices of joint activities, paying attention towards a movie or a cartoon has been selected as a simple, stereotyped, joint activity that the child can be reasonably able to perform. A measure of attention is retrieved through the real-time exploitation of EEG signals. The small humanoid robot, as a proactive companion of the child, would return feedbacks in case of attention breakdown, with the goal of re-inducing attention towards the joint activity.
More in detail, the experimental protocol is de ned as in Figure 1: 1. A movie or a cartoon, is used as audio-visual stimulus for the child. At the same time, his EEG activity is captured. 2. EEG data is exploited to extract a measure able to represent the attention the child is giving to the current task. The selected metric is able to highlight the child's attention breakdowns. 3. In case of attention breakdown, the robot produces a feedback to trigger back the attention towards the task. The feedback is de ned upon a set of well pre-de ned mix of verbal and non-verbal behaviours. { An EEG cap, Enobio from NeuroElectrics; { A screen for the audio-visual stimulation; { A small humanoid robot, Nao from Aldebaran Robotics;
Two computer are also employed: one to capture and exploit the EEG data; one to control the robot and store logs and sensors data. All the systems are synchronized through Network Time Protocol.
The session is composed by two main stages: { Training stage: The child watches an audio-visual content that assures his attention. This data is used to build the baseline for the child attention measure, to train the system how to recognize attention breakdowns. { Test stage: The child watches an audio-visual content that does not ensure his attention. When the system recognizes an attention breakdown, a feedback behavior of the robot is triggered.
At the end of each session, the child behavior is evaluated in terms of information retained, of number of attention breakdowns, and through the average of the attention measure. 2.1
The audio-visual content employed to stimulate the child is one of the most important component of the system. A stronger stimulus or a weaker one in terms of attention induced or cognitive load required to understand it, can compromise the capabilities of the system on recognizing the attention breakdowns. Several contents have been selected, distinguishing, in particular, the ones for the training stage of the system. In this case, the stimulus employed should be particularly e ective on inducing attention in the children because the data captured will be used as baseline of the attention. Consequently, the stimulus should be very personalized according to the particular preferences of the child. On the contrary, the stimulus employed during the Test stage will be less attractive and engaging.
The developed prototype employed documentaries (Length: 3min) and storytelling videos (Length: 5min), as in Figure 3.
The EEG signal captured is exploited to identify attention breakdowns.
Attention has been characterized through the Theta - Beta ratio of the signals
captured by the central, frontal and temporal EEG sensors (c3, c5, f3, f4, t7,
t8) [
Attention breakdown are identi ed in a sliding window of 5sec of the real-time attention measure captured. The breakdown is de ned by a prevalence ( 50% of the window) of measures lower than 0.5 (not attentive). 2.3
During the experiment, the robot is placed at the side of the child, attentive towards the screen, to stimulate on him imitation behaviors and joint attention towards the audio-visual stimulus. When the attention breakdown is identi ed, the system triggers a feedback by executing one of some speci c, prede ned behaviors, able to re-induce attention towards the joint activity. This set of behaviours mixes gestures, pointing in particular, and voice, as: Ah!, Look!, I love it!, Ah! I love this!, as in Figure 4.
In the presented pilot, only 8 behaviors have been implemented: 4 coupling gestures and voice; 4 proposing just non-verbal behaviors. The selection of the behavior after the attention breakdown detection is randomly chosen among the developed behaviors. To evaluate this pilot system, an exploratory study has been conducted involving an adolescent (17 years old) from the Ecole Georges Heuyer, part of the Child and Adolescent Psychiatry Service of Piti-Salptrire Medical Hospital in Paris, France. While not statistically signi cants, the results obtained in this pilot study are extremely important to evaluate the feasibility of the system, the possible protocols that would be employed, the adverse events that could happen and an appropriate sample size before a more exhaustive experiment. The child has has been diagnosed with autism spectrum disorder, with a particular evidence of attentional troubles. The child has been selected because he does not show any motor disability, or any other serious behavioral trouble.
The experiments involving the child have been carried on in 3 sessions during a single week: each session has been divided in an experimental condition and in a control condition.
After a common training stage, in which the system learns the attention model, according to the previous introduced methodology, the child faces the two conditions. In the experimental condition, the child is in front of the screen, side by side with the robot, that eventually intervene by giving feedbacks according to the attentional state revealed by the EEG-based measure. In the control condition, the robot does not intervene: it just stand on the side of the child, watching carefully the audio-visual stimulus. The two condition are opportunely randomized among the days of the week.
In both conditions, a trained clinician ask questions about the content proposed to the child, in order to obtain a rough measure of the retained information. This measure is coupled with the number of attention breakdowns captured by the system and with an attention score, de ned as the mean of the attention measure among the whole condition.
Figure 5 shows that the amount of information retained by the child among the di erent sessions is higher in the case of proactive robot rather than in the case of static robot.
Table 1 shows that the number of attention breakdown events increase among the time. It shows also that these events are more frequent in the control condition than the experimental condition. Table 2 shows how the score of attention among the days is almost constant in the control condition while it highlights uctuations in the experimental condition.
Experiment Day Control Condition Experimental Condition
In this paper we introduced a prototype neurofeedback system designed to
stimulate attention of a child towards an audio-visual stimulus. The system exploits
data from EEG to estimate the attention, while employs a robot as convenient
and natural interface to stimulate attention in the child. Results obtained seem
to highlight a perturbing e ect of the robot on the attention of the child. Such
e ect could be a result of the eventual presence of false positives on the attention
breakdown detection system. In any case, due to the limited samples collected
in this pilot study, it is di cult to obtain statistically reliable conclusion. Those
results encourage the development of more precise measures of attention, mixing
together EEG data and behavioral information, as the psycho-motor agitation [