Designing robot-based treatments for children with Autistic Spectrum Disorder (ASD) is a growing research field. This paper presents an artificial intelligence system based on a robot-assisted treatment of autism. The robot acts as a social mediator, trying to elicit specific behaviors in autistic children. A first preliminary evaluation of the system has been performed involving 3 high functioning children with autism spectrum disorders. The experiments carried out make it possible to evaluate the behavioral response of the children in the eye contact exercise.
Autism is a severe disorder of development that is characterized by social
interaction/communication difficulties and tendency to engage in repetitive patterns of
behavior. A quite large number of early diagnosis and treatment protocols have been designed
empirically tested and published in the autism literature. The most recent protocols are
derived from Applied Behavior Analyses (ABA) [
The protocol has been partially used in the SARACEN (Social Assistive Robots for Autistic Children EducatioN) project aimed at developing innovative methods for early diagnosis of ASD and therapy support for autistic children with socially assistive robots. SARACEN has been partially supported by the Italian Ministry for Education, University and Research (MIUR) and by the European Union in the framework of Smart Cities and Communities and Social Innovation of 2007-2013.
This paper is organized as follows. Section 2 reports the state of the art relative to the robot-assisted treatments for autistic children. Section 3 presents an overview of the Artificial Intelligence in Robot-Child Interaction. Section 4 describes the experimental setup and the preliminary results. Finally, conclusions are drawn. 2
Positive effects of social robots in autistic children treatment are already reported in
the literature to elicit specific behavior in ASD children such as emotion generation
[
The proposed system for ASD treatment includes four main modules: the RGB-D camera, the workstation, and the social robot and the robot camera. The overview of the system is illustrated in the figure 1. The child’s behavior is captured by two cameras: a 5 mega-pixel auto focus camera on board the robot, and a RGB-D camera. The social robot is the Softbank NAO H25 humanoid robot 1. NAO has the following technical characteristics: – 25 degrees of freedom (11 for the lower part and 14 for the upper part); – x86 AMD GEODE 500MHz CPU with 256 MB of RAM and 2GB of storage;
– Ethernet and Wi-Fi connections.
The workstation is equipped with: – Intel Core i7-4700MQ CPU (2.40 GHz), with 8GB of RAM; – 1TB of storage; – Ubuntu GNU/Linux 16.10 as operating system; and the following software modules: – NAOqi API; – Kinect SDK 2; – the Robot Intelligence Module (RIM); – the Behavior Manager (BM).
The workstation uses the NAOqi API to communicate with the robot in order to capture the video streaming from the robot camera and in order to activate specific robot behaviors. The Kinect SDK 2 installed on the workstation is used to acquire depth streaming from the RGB-D camera. The video and depth frames acquired via the sensors are then sent to the RIM. This is composed by four software components: head pose, body posture, eye contact, and facial expression. Each module use specific computer vision algorithms. The RIM detects the child’s non verbal signals and transfer them to the BM. In this module is implemented the treatment protocol. A log file in the BM reports an anonymous code for the child, the behavior performed from the robot, the behavioral response of the child, and the exercise performed by the social robot. The protocol is designed to improve a difficult behavior for an autistic child. It is based on the ABA program that includes: a stimulus presentation, a behavioral response, and a reinforcement. The new aspect is the presence of a social robot as a partner to perform the treatment. The protocol has five exercises with three levels of difficulty (see figure 2). The exercises focus on: eye contact, joint attention, body imitation, facial imitation, and facial expression imitation. The child has to performs each level for 5 times and when he/she performs the exercise correctly he/she can pass the next level. The therapist can assign each exercise or a set of exercise to a child according the functioning level.
In this study only the eye contact exercise has been carried out with autistic children. Therefore, only the eye contact exercise is described in this subsection. 3.2
The eye contact exercise is design to improve the eye contact behavior typically reduced
in ASD children. This behavior is essential for interpersonal communication [
Medium Level In this level, the stimulus is changed: the robot call the child by name and it does not say "Look at me".
3.3
A description of the computer vision based algorithm used to implement the eye contact detection is provided in this section. Our algorithm for eye contact detection needs the RGB camera of the robot placed close (max 40 50cm:) to the face of the child.
The pipeline of the eye contact detector, illustrated in the figure 3, is composed of 5 steps: eye detection, preprocessing, iris detection, and pupil detection, pupil position.
The algorithm takes images from the camera as an input (raw images) and as a first step the eyes are detected using the well-known Viola-Jones detector implemented in OpenCV. Then, in the preprocessing step the right and left eye patches are converted in 8-bit-deep gray-level image and several filters are applied. The filters applied in the preprocessing step are: thresholding, erosion, and morphological gradient. The next step of the pipeline draws the contours of the iris finding the dark part of the eye (iris) from the white background (sclera). Once the location of the iris has been obtained the pupil can be detected by calculating the centroid of the iris. In the final step the eye contact detection is performed. The eye patches are divided in 8x8 sections: if the pupils are in the center of this grid the eye contact occurred (see figure 4) otherwise the child is looking into something else. 4
In this section an assessment of the system is provided analyzing the behavioral responses of the autistic children. The analyze has the following goals: 1. test the artificial intelligent system components in a real environment; 2. evaluating the behavioral responses elicited of ASD children.
Three children (C1, C2, C3) with a diagnosis of high functioning ASD (age range of 6-13 years) have been involved in this study. According to the ethical guidelines the personal data related to the children have been anonymized so that the individual identity can not be revealed. The parents of the children signed the informed consent, written in Italian (the participants’ mother tongue) of which one copy has been kept by the therapists and the other one by the parents of the child. Participants have been asked to perform three sessions (S1, S2, and S3) with the interface. Each child tests the Eye Contact exercise (Easy, Medium, and Hard levels) to achieve the first assessment of the child with the robotic interface. Subsequently, the robotic treatment program will be tailored on the specific needs of the child. Each children played 15 eye contact exercises for each session. A session lasted 20 minutes on average. The experiment was conducted by expert therapist. The children were admitted one at a time in the experimental room. The therapist and the child entered the room together, the child were placed in front of the robot sitting on a chair. Beforehand, all children participated in a familiarization session lasting 10 minutes. Then, the therapist introduced the social robot providing a simple description of it and answer any child’s questions. Subsequently, once the children felt comfortable in the presence of the social robot (usually 10 minutes), the first experimental session (S1) under the supervision of the therapist started. In the first session the robot started with the Easy Level of the Eye Contact exercise as detailed in Section 3.1. In the second session the robot started the Medium Level of the same exercise. Finally, the third session start with the Hard Level of the Eye Contact exercise. For each session the robots repeated 15 times the corresponding level. At the end of each session, a debriefing was given to each participant. 4.2
In general, the system was able to operate well in the treatment environment for all the ASD children. To evaluate the behavior of the children during the interaction with the system, the focus was on the number of the eye contact correctly performed (nEC). The system considers 1000 seconds as the maximum time (tMAX) to perform the eye contact.
With respect to nEC, it has been observed that, in percentage, the eye contact act had an average success rate equal to 73.33% in S1, 55.55% in S2, and 28.89% in S3 as reported in Table 1.
Figure 5 depicts the nEC achieved by each child in S1, S2, and S3.
A first preliminary evaluation of the system has been performed involving 3 high functioning children with autism spectrum disorders. Results were encouraging, analyzing the nEC in the three different sessions. In fact, it is possible to understand in an objective way the level of difficulty of the child involved in the treatment. This measure can be useful to adjust the treatment in the next session with the robot. In the experiment all children are able to perform well the Eye Contact exercise at the easy level, but they need help when they perform the eye contact at the medium level and at the hard level. In this paper an Artificial Intelligence system for Robot-Child interaction based on a behavioral treatment protocol has been proposed. Results show as a social robot playing the role of mediator can be successful in robot-assisted treatment of autistic children. The same children who involved in this experiment will be interact with the social robot in the same exercises to test the follow up of the treatment. Moreover, investigations including experiments with a larger sample of autistic children that interact with whole protocol (eye contact joint attention, body imitation, facial imitation, and facial expression imitation) will allow us to test the exercises completely. 6
This work has been partially supported by Italian Ministry for Education, University and Research (MIUR) and European Union under Grants PON04a3_00201, SARACEN project.