=Paper=
{{Paper
|id=Vol-2166/afcai18-paper11
|storemode=property
|title=EmIR: An Emotional Intelligent Robot Assistant
|pdfUrl=https://ceur-ws.org/Vol-2166/afcai18-paper11.pdf
|volume=Vol-2166
|authors=Jaime A. Rincon,Alberto Martin,Angelo Costa,Paulo Novais,Vicente Julian,Carlos Carrascosa
|dblpUrl=https://dblp.org/rec/conf/afcai/RinconMCNJC18
}}
==EmIR: An Emotional Intelligent Robot Assistant==
https://ceur-ws.org/Vol-2166/afcai18-paper11.pdf
EmIR: An Emotional Intelligent Robot Assistant
Jaime A. Rincon, Alberto Martin, Angelo Costa, Paulo Novais, Vicente Julian,
and Carlos Carrascosa
1
Universitat Politècnica de València. D. Sistemas Informáticos y Computación
jrincon, vinglada, carrasco@dsic.upv.es
2
Centro ALGORITMI, Escola de Engenharia, Universidade do Minho, Braga acosta,
pjon@di.uminho.pt
Abstract. The development of robots that are truly sociable requires
understanding how human interactions can be applied to the interaction
between humans and robots. A sociable robot must be able to interact
with people taking into account aspects like verbal and non-verbal com-
munications (emotions, postures, gestures). This work presents a social
robot which main goal is to provide assistance to older people in car-
rying out their daily activities (through suggestions or reminders). In
addition, the robot presents non-verbal communications like perceiving
emotions and displaying human identifiable emotions in order to express
empathy. A prototype of the robot is being tested in a daycare centre in
the northern area of Portugal.
Keywords: Social Robots, Emotional Models, Ambient Assisted Living
1 Introduction
Globally, the elderly population is increasing, according to demographic projec-
tions [1]. The OMS prdeicts that the amount of people aged over 60 is expected
to double between 2000 and 2050 [2]. As less developed countries start to evolve,
this trend is onset immediately [2]. A common societal issue that emerges from
a rapid elderly population growth is the exponential demand or care (medical
and otherwise).
The common human resources needed are healthcare professionals, including
formal and informal caregivers. Which currently are in short supply, and there is
no indication of increase of the caregivers numbers in the foreseeable future. The
lack of these professionals represents a serious issue. One way of dealing with
this issue is using technology that empowers people to overcome problems that
they encounter on their daily lives. Various areas can be of help to the elderly,
like Social Robotics [3], Virtual Assistants [4], Artificial Intelligence [5], etc.
These areas try to solve these problems by introducing new technologies that
can help older people living alone or in nursing homes. Some of these solutions
include voice assistants such as Amazon’s Alexa, Google Voice Assitant and Ap-
ple’s HomePod. However, other emerging technologies such as assistant robots
and therapy robots are becoming increasingly popular. These devices integrate
speech recognition, artificial vision, text chat and gestural interactions. These
mechanisms help users to interact with these devices. However, in human in-
teraction the way that emotion is expressed plays an important role, mainly
because emotions are themselves a communication channel.
In this paper we present a social robot, called EmIR, which main goal is
to provide assistance to the elderly performing their daily activities, presenting
human-like features like perceiving emotions of a group of people and displaying
human identifiable emotions. Moreover, our social robot makes use of an e-Health
platform (Cognitive Life Assistant - CLA) which integrates a persuasive module
that makes use of argumentative techniques similar to reasoning procedures that
physicians and caregivers use to recommend activities to patients.
The rest of the paper is structured as follows. Section 2 analyzes previous
works in the area of assistants robots. Section 3 presents in detail the main
functionalities of the robot with particular emphasis on the emotion detection
and the user interaction. Section 4 describes the hardware employed to build
the robot and it illustrates the functionalities of the robot through an example.
Finally, the conclusions are presented in Section 5.
2 Related Work
Recently, it is observable the increase on interest in assistive robots and their
development. Current sociological shift demands technological solutions that are
able to interact with elderly people. Thus, the sudden increase of the number of
robots available that have as goal solving this sociological issue. Furthermore,
they are increasingly refined and advanced in terms of aspect and features, being
more human-like in both fields.
In recent years, we have seen a growing interest in robots, many of which
are available to people. These robots have nice looks, a powerful background of
artificial intelligence, navigation in complex environments, artificial vision, etc.
Some of them, such as Pepper [6] and Romeo [7] from Aldebaran or Aido [8] or
Buddy [8] from Frog Robotics, are presented as a new generation of home robots.
One of the main applications of these robots is to assist older people in their daily
life. Usually, these robots are used as a form of therapy or health care. In therapy
field, we can find the Paro robot [9]. The Paro robot is an advanced interactive
robot developed by AIST, which offers the well-known benefits of animal therapy
to be administered to patients in environments such as hospitals. This robot was
built to be a companion to children and elderly, prompting a positive emotional
response due to its visual aspect. In the healthcare field we can find the Mabu
robot of Catalia Health, who learns over time about the personality, interests and
therapeutic challenges of each patient. This allows Mabu to create conversations
adapted to each patient. The structure of these conversations is based on well-
known behavioral models of psychology to promote behavioral change.
From these projects we have observed that while the physical aspect, and its
underlying impact (as displayed by PARO), is very advanced and the target of a
careful development, they lack in terms of functionality. The actions that makes
one human and their emotions are not explored by these projects and we believe
that after the initial impact of the visual aspect it is what makes them really
human-like. We, as humans, strive for human empathy, and is what our project
aims for.
3 Emotional Robot Assistant Description
In this section we describe the proposed social robot, called EmIR (Emotional
Intelligent Robot). This robot has been developed in order to provide assistance
to the elderly to perform their daily activities. To validate the proposed robot, a
prototype is being tested in a real scenario, in a daycare institution in the north
of Portugal.
The main features of the robot are the following: perceives the emotions
of each individual and also calculates the social emotion of a group of people;
displays human identifiable emotions according to the emotional states of the
users; assists users recommending healthy activities and also, remembering when
to do them; and finally, persuades users by giving them arguments to justify
recommended actions. The robot was built using a lattepanda board3 and an
Arduino mega 2560 board4 . These development boards are used to control the
different engines and also to detect emotions and to identify people. Figure 1
shows a picture of the robot with its different elements.
Fig. 1: Robot hardware.
3
http://www.lattepanda.com/
4
https://store.arduino.cc/arduino-mega-2560-rev3
The functionalities of the proposed robot are divided into three main modules
which provides the services that our robot uses to perceive and interact with the
environment. The first two modules are the Emotion Detection and the Emotion
Display modules which are in charge of detecting, processing and displaying
emotions. On the other hand, the other module is the User Interaction module
which has been implemented using the CLA framework [10]. The CLA framework
improves the user experience and gives a fluid visual interface to the robot.
4 Application Example
In order to illustrate the behavior of the robot, this section presents an example
which describes the different processes made by the robot as an emotional-based
persuasive recommender. In the proposed example, the robot will interact with
humans by identifying them, detecting their emotions and suggesting activities.
Figure 2 shows the different processes followed by the robot for supporting
the described interaction.
Fig. 2: A schema of the main processes followed by the robot
In a general view, the robot will follow the following steps:
1. The way to initiate the interaction with the robot consists of saying the name
of the robot, which in our case is EmIR (Emotional Interactive Robot).As
soon as the robot hears its name it starts interacting with the caller(s).
2. The first step is to take a picture through its webcam. This image is used
to identify the person and to track the face. If the robot does not know the
person, the robot initiates the following dialogue (as an example):
– EmIR: I’m sorry, but I don’t know you.
– EmIR: What’s your name?.
– User: Hi, my name is Jaime.
3. After this, the image is then introduced as input into a neural convolutional
network that returns the emotional state. If the robot detects more than
one face in the same image, the process is repeated for each face. Once the
emotional state of all the users are obtained, the robot will adapt its emotion
in an attempt to show empathy. To do this the robot can move its eyebrows
and change the lights of its cheeks.
4. Next, the robot makes use of its persuasive recommender system. It can
recommend the following items:
– Events that promote active aging. The suggestion of events follows a
set of configurations (age, likes, medical condition, etc.) that make them
appropriate for each user.
– Activities on the user calendar, e.g., medication reminders, medical ap-
pointments.
During this recommendation process, a dialogue with the user is started
where the user can accept or refuse the suggestions. If the user refuses a sug-
gestion, the robot tries to suggest another activity or generates arguments
in order to persuade the user. In parallel, it keeps identifying the user’s emo-
tion and producing visual cues (eyebrows, etc.) to enforce the information
delivered.
5. After this process, the robot goes to a waiting state until it is requested again
by the user. In the meanwhile the robot presents practical information like
the current weather state or news, in an effort to interact with the user(s).
5 Conclusions and future work
This paper has presented the EmIR robot, a low cost desktop robot devised to
interact with humans taking into account the emotion of the people it interacts
with and even being able to express some emotion. The application area where
is assistance to an elderly community.
In the interaction with people in this application area, the robot is able to
talk with the persons it perceives and help those people by recommending new
activities along with giving reminders about scheduled activities.
Although the robot is still in a prototype form, its design and abilities have
been improved presenting human-like features like perceiving emotions of a group
of people and displaying human identifiable emotions. The functionalities of the
proposed robot are divided into three main aspects. The first one is the Emotion
Detection that allows the robot to estimate the emotional state of the people
that is in from of the robot. The second one is the Emotion Display, that allows
the robot to express empathy with people according to the emotional states
previously detected. The third and last one, is the User Interaction that allows
the robot to enhance the user experience suggesting activities/events to the user
based on his/her profile and medical condition and trying to persuade the users
into accepting the suggested activities/events.
This work is being validated by workers and patients of a daycare centre in the
northern area of Portugal. Specifically in the centre Centro Social Irmandade de
S. Torcato. The validation is being performed through simple interactions with
the patients under the supervision of caregivers.
As future work, we want to improve the detection of emotions by introducing
speech recognition as another input in the process of identifying emotional states.
Another aspect to be introduced, and currently supported by CLA, is the direct
connection with caregivers or doctors to share information on scheduled activities
or events as well as the patient progress. Finally, we aim for the adaptation of
the proposed system to people with disabilities who cannot use the touchscreen.
6 Acknowledgements
This work is partially supported by the MINECO/FEDER TIN2015-65515-C4-
1-R and the FPI grant AP2013-01276 awarded to Jaime-Andres Rincon. Angelo
Costa thanks the FCT - Fundacao para a Ciencia e Tecnologia for the Post-
Doc grant SFRH/BPD/102696/2014. This work is supported by FCT within
the projects UID/CEC/00319/2013 and COMPETE: POCI-01-0145-FEDER-
007043.
References
1. United Nations, Department of Economic and Social Affairs, Population Division.
World population ageing 2015. (ST/ESA/SER.A/390), 2015.
2. United Nations, Department of Economic and Social Affairs, Population Division.
World population prospects: The 2017 revision, key findings and advance tables.
(ESA/P/WP/248), 2017.
3. J. Broekens, M. Heerink, and H. Rosendal. Assistive social robots in elderly care:
a review. Grontechnology, 8:94–103, 2009.
4. Nicole C. Krämer. Social effects of virtual assistants. a review of empirical results
with regard to communication. In Helmut Prendinger, James Lester, and Mitsuru
Ishizuka, editors, Intelligent Virtual Agents, pages 507–508, Berlin, Heidelberg,
2008. Springer Berlin Heidelberg.
5. Eduardo Lupiani, Jose M. Juarez, Jose Palma, and Roque Marin. Monitoring
elderly people at home with temporal case-based reasoning. Knowledge-Based
Systems, 134:116 – 134, 2017.
6. Fumihide Tanaka, Kyosuke Isshiki, Fumiki Takahashi, Manabu Uekusa, Rumiko
Sei, and Kaname Hayashi. Pepper learns together with children: Development of
an educational application. IEEE-RAS International Conference on Humanoid
Robots, 2015-December:270–275, 2015.
7. Giovanni Claudio, Fabien Spindler, and Francois Chaumette. Vision-based manip-
ulation with the humanoid robot Romeo. In 2016 IEEE-RAS 16th International
Conference on Humanoid Robots (Humanoids), pages 286–293. IEEE, nov 2016.
8. Ester Martinez-Martin and Angel P. del Pobil. Personal Robot Assistants for
Elderly Care: An Overview, pages 77–91. Springer International Publishing, Cham,
2018.
9. K. Wada, T. Shibata, T. Saito, and K. Tanie. Effects of robot assisted activity to
elderly people who stay at a health service facility for the aged. In 3 IEEE/RSJ
International Conference on Intelligent Robots and Systems (IROS), pages 2847–
2852, 2003.
10. Angelo Costa, José Carlos Castillo, Paulo Novais, Antonio Fernández-Caballero,
and Ricardo Simões. Sensor-driven agenda for intelligent home care of the elderly.
Expert Syst. Appl., 39(15):12192–12204, 2012.