The development of robots that are truly sociable requires understanding how human interactions can be applied to the interaction between humans and robots. A sociable robot must be able to interact with people taking into account aspects like verbal and non-verbal communications (emotions, postures, gestures). This work presents a social robot which main goal is to provide assistance to older people in carrying out their daily activities (through suggestions or reminders). In addition, the robot presents non-verbal communications like perceiving emotions and displaying human identifiable emotions in order to express empathy. A prototype of the robot is being tested in a daycare centre in the northern area of Portugal.
Globally, the elderly population is increasing, according to demographic
projections [
The common human resources needed are healthcare professionals, including
formal and informal caregivers. Which currently are in short supply, and there is
no indication of increase of the caregivers numbers in the foreseeable future. The
lack of these professionals represents a serious issue. One way of dealing with
this issue is using technology that empowers people to overcome problems that
they encounter on their daily lives. Various areas can be of help to the elderly,
like Social Robotics [
These areas try to solve these problems by introducing new technologies that can help older people living alone or in nursing homes. Some of these solutions include voice assistants such as Amazon’s Alexa, Google Voice Assitant and Apple’s HomePod. However, other emerging technologies such as assistant robots and therapy robots are becoming increasingly popular. These devices integrate speech recognition, artificial vision, text chat and gestural interactions. These mechanisms help users to interact with these devices. However, in human interaction the way that emotion is expressed plays an important role, mainly because emotions are themselves a communication channel.
In this paper we present a social robot, called EmIR, which main goal is to provide assistance to the elderly performing their daily activities, presenting human-like features like perceiving emotions of a group of people and displaying human identifiable emotions. Moreover, our social robot makes use of an e-Health platform (Cognitive Life Assistant - CLA) which integrates a persuasive module that makes use of argumentative techniques similar to reasoning procedures that physicians and caregivers use to recommend activities to patients.
The rest of the paper is structured as follows. Section 2 analyzes previous works in the area of assistants robots. Section 3 presents in detail the main functionalities of the robot with particular emphasis on the emotion detection and the user interaction. Section 4 describes the hardware employed to build the robot and it illustrates the functionalities of the robot through an example. Finally, the conclusions are presented in Section 5. 2
Recently, it is observable the increase on interest in assistive robots and their development. Current sociological shift demands technological solutions that are able to interact with elderly people. Thus, the sudden increase of the number of robots available that have as goal solving this sociological issue. Furthermore, they are increasingly refined and advanced in terms of aspect and features, being more human-like in both fields.
In recent years, we have seen a growing interest in robots, many of which
are available to people. These robots have nice looks, a powerful background of
artificial intelligence, navigation in complex environments, artificial vision, etc.
Some of them, such as Pepper [
From these projects we have observed that while the physical aspect, and its underlying impact (as displayed by PARO), is very advanced and the target of a careful development, they lack in terms of functionality. The actions that makes one human and their emotions are not explored by these projects and we believe that after the initial impact of the visual aspect it is what makes them really human-like. We, as humans, strive for human empathy, and is what our project aims for. 3
In this section we describe the proposed social robot, called EmIR (Emotional Intelligent Robot). This robot has been developed in order to provide assistance to the elderly to perform their daily activities. To validate the proposed robot, a prototype is being tested in a real scenario, in a daycare institution in the north of Portugal.
The main features of the robot are the following: perceives the emotions of each individual and also calculates the social emotion of a group of people; displays human identifiable emotions according to the emotional states of the users; assists users recommending healthy activities and also, remembering when to do them; and finally, persuades users by giving them arguments to justify recommended actions. The robot was built using a lattepanda board3 and an Arduino mega 2560 board4. These development boards are used to control the different engines and also to detect emotions and to identify people. Figure 1 shows a picture of the robot with its different elements. 3 http://www.lattepanda.com/ 4 https://store.arduino.cc/arduino-mega-2560-rev3
The functionalities of the proposed robot are divided into three main modules which provides the services that our robot uses to perceive and interact with the environment. The first two modules are the Emotion Detection and the Emotion Display modules which are in charge of detecting, processing and displaying emotions. On the other hand, the other module is the User Interaction module which has been implemented using the CLA framework [10]. The CLA framework improves the user experience and gives a fluid visual interface to the robot. 4
In order to illustrate the behavior of the robot, this section presents an example which describes the different processes made by the robot as an emotional-based persuasive recommender. In the proposed example, the robot will interact with humans by identifying them, detecting their emotions and suggesting activities.
Figure 2 shows the different processes followed by the robot for supporting the described interaction.
In a general view, the robot will follow the following steps: 1. The way to initiate the interaction with the robot consists of saying the name of the robot, which in our case is EmIR (Emotional Interactive Robot).As soon as the robot hears its name it starts interacting with the caller(s). 2. The first step is to take a picture through its webcam. This image is used to identify the person and to track the face. If the robot does not know the person, the robot initiates the following dialogue (as an example): – EmIR: I’m sorry, but I don’t know you. – EmIR: What’s your name?.
– User: Hi, my name is Jaime. 3. After this, the image is then introduced as input into a neural convolutional network that returns the emotional state. If the robot detects more than one face in the same image, the process is repeated for each face. Once the emotional state of all the users are obtained, the robot will adapt its emotion in an attempt to show empathy. To do this the robot can move its eyebrows and change the lights of its cheeks. 4. Next, the robot makes use of its persuasive recommender system. It can recommend the following items: – Events that promote active aging. The suggestion of events follows a set of configurations (age, likes, medical condition, etc.) that make them appropriate for each user. – Activities on the user calendar, e.g., medication reminders, medical appointments.
During this recommendation process, a dialogue with the user is started where the user can accept or refuse the suggestions. If the user refuses a suggestion, the robot tries to suggest another activity or generates arguments in order to persuade the user. In parallel, it keeps identifying the user’s emotion and producing visual cues (eyebrows, etc.) to enforce the information delivered. 5. After this process, the robot goes to a waiting state until it is requested again by the user. In the meanwhile the robot presents practical information like the current weather state or news, in an effort to interact with the user(s). 5
This paper has presented the EmIR robot, a low cost desktop robot devised to interact with humans taking into account the emotion of the people it interacts with and even being able to express some emotion. The application area where is assistance to an elderly community.
In the interaction with people in this application area, the robot is able to talk with the persons it perceives and help those people by recommending new activities along with giving reminders about scheduled activities.
Although the robot is still in a prototype form, its design and abilities have been improved presenting human-like features like perceiving emotions of a group of people and displaying human identifiable emotions. The functionalities of the proposed robot are divided into three main aspects. The first one is the Emotion Detection that allows the robot to estimate the emotional state of the people that is in from of the robot. The second one is the Emotion Display, that allows the robot to express empathy with people according to the emotional states previously detected. The third and last one, is the User Interaction that allows the robot to enhance the user experience suggesting activities/events to the user based on his/her profile and medical condition and trying to persuade the users into accepting the suggested activities/events.
This work is being validated by workers and patients of a daycare centre in the northern area of Portugal. Specifically in the centre Centro Social Irmandade de S. Torcato. The validation is being performed through simple interactions with the patients under the supervision of caregivers.
As future work, we want to improve the detection of emotions by introducing speech recognition as another input in the process of identifying emotional states. Another aspect to be introduced, and currently supported by CLA, is the direct connection with caregivers or doctors to share information on scheduled activities or events as well as the patient progress. Finally, we aim for the adaptation of the proposed system to people with disabilities who cannot use the touchscreen. 6
This work is partially supported by the MINECO/FEDER TIN2015-65515-C41-R and the FPI grant AP2013-01276 awarded to Jaime-Andres Rincon. Angelo Costa thanks the FCT - Fundacao para a Ciencia e Tecnologia for the PostDoc grant SFRH/BPD/102696/2014. This work is supported by FCT within the projects UID/CEC/00319/2013 and COMPETE: POCI-01-0145-FEDER007043. 10. Angelo Costa, José Carlos Castillo, Paulo Novais, Antonio Fernández-Caballero, and Ricardo Simões. Sensor-driven agenda for intelligent home care of the elderly. Expert Syst. Appl., 39(15):12192–12204, 2012.