Workshop "From Objects to Agents" (WOA 2019)
Assessing Usability of a Robotic-Based AAL
System: A Pilot Study with Dementia Patients
Claudia Di Napoli1 , Emanuela Del Grosso1 , Giovanni Ercolano2 , Federica Garramone3 ,
Elena Salvatore4 , Gabriella Santangelo3 , and Silvia Rossi2
1
Istituto di Calcolo e Reti ad Alte Prestazioni, C.N.R., Naples, Italy
Email: claudia.dinapoli@cnr.it, emanuela.delgrosso@icar.cnr.it
2
DIETI, Universit degli Studi di Napoli Federico II, Naples, Italy
Email: {silvia.rossi, giovanni.ercolano}@unina.it
3
Department of Psychology, Universit degli Studi della Campania Luigi Vanvitelli, Caserta, Italy
Email: {gabriella.santangelo, federica.garramone}@unicampania.it
4
Department of Neuroscience, Universit degli Studi di Napoli Federico II, Naples, Italy
Email: elena.salvatore@unina.it
Abstract—Ambient Assisted Living is playing a crucial role in In this work, we report the approach adopted in the
supporting dementia patients to live in their preferred environ- User-Centred Profiling and Adaptation for Socially Assistive
ment, so limiting the involvement of careers and/or relatives. In Robotics (UPA4SAR) project, whose objective is to develop
order for such systems to become a reality, patients need to feel
comfortable when interacting with them, and so an agent-based an affordable, easy to deploy, and well accepted AAL system
modular approach is adopted to make it possible to personalize based on a social robot to deliver assistive services for home
the provision of digital services for each specific patient’s needs. patients affected by Alzheimer’s disease. The aim of the
Here, we experiment with a robotic-based ambient assisted project is to improve the level of acceptability of social
environment to analyze the perceived usability when real patients robotics through the possibility of adapting robots’ behavior
interact with it in a controlled research laboratory where the
system is deployed, by taking into account both their personality to the patient. The innovative character of the project concerns
traits and cognitive status. The perceived usability is evaluated the realization of new models of assistance and provision of
through a survey with a set of patients filling a questionnaire services in the health sector, which aim at a “patient-centric”
specifically designed for the experimentation that is based on the vision. To this end, the project proposes the use of a robotic
Unified Theory of Acceptance and Use Technology (UTAUT). The system that allows the automatic adaptation of the robot’s
preliminary obtained results show that the perceived usability of
the system is related to some traits of patients’ personality, while behavior to the personality profile, preferences, and cognitive
their cognitive status impacts the provided assistive services. status of the user, in order to provide an adaptive interaction.
Index Terms—Assistive robotics, workflow of services, person- The main task of the robot is the monitoring of the patient’s
alization, QoS adaptation quality of life (through the recognition of the activities he/she
performs) and cognitive support. In this work, we present the
I. I NTRODUCTION results of a first experimentation performed with the complete
Ambient Assisted Living systems equipped with social prototype of the system.
robots are reaching a great potential due to the advances in
the Information and Communication Technologies, so making II. A ROBOTIC - BASED AAL S YSTEM
it possible their adoption for supporting home care of patients In order to support home-care assistance for patients af-
with mild neurological disorders, or at initial stages of the fected by neurological disorders, a robotic-based ambient
Alzheimer’s disease. Such systems may help to provide cog- assisted environment has been developed within the UPA4SAR
nitive and physical stimulation to patients, crucial to limiting project whose goal is to provide an affordable and well-
their cognitive reserve, to remind tasks that have to be carried accepted robotic assistive system, able to generate and execute
out during the day, and to monitor their activities, so allevi- assistive plans and actions that are personalized for each
ating the already heavy burden on careers and/or relatives. specific patient, and that can be adapted during execution to
Nevertheless, the interaction of a very vulnerable category changing conditions in both the home environment, and the
of users, such as Alzheimer’s patients, with these systems patient’s conditions [2].
may strongly impact their effective use [1]. In fact, interactive At this purpose, a service-oriented approach (SOA) is
devices, as robotic systems, whose behavior is not compliant adopted to decouple a required functionality from its concrete
with the needs and the characteristics of each patient, may implementation that is characterized both by the device that
cause discomfort so preventing her/him from using it. provides it, and by Quality of Service (QoS) parameters
59
Workshop "From Objects to Agents" (WOA 2019)
referring to the way it is delivered [3]. This service-based III. AGENT- BASED A BSTRACT S ERVICES FOR H OME
approach allows both the execution of services on computing A SSISTANCE
resources outside the robot, and the integration of services Starting from the patient’s daily routine (as reported in
that can be provided by the robot with services provided by Table I, the Assistive Workflow Management subsystem is
other devices in the house, or even by services provided by a responsible for the generation of high level plans listing the
caregiver, thanks to the possibility of representing an assistive necessary daily assistive tasks required for assisting patients.
action as a service according to an interface and standard It is an agent-based middleware that, once assistive tasks
communication protocols. are declared in an XML format, it generates plans to per-
The system architecture is composed of different layers: form them, composed of the required functionalities, and
their execution order requirements. According to the adopted
service-oriented approach, such assistive task is represented
• the Data Computational Model, i.e. the knowledge base
as a workflow of abstract services, i.e. abstract functionality
containing static and dynamic information both on the
necessary to perform the task. An example of daily routine
patient, and on the home environment; dynamic informa-
and the corresponding high level plan, known as an Abstract
tion includes the current user activity, his/her physical
Assistive Plan, represented in XML, are here reported.
and emotional state, his/her current location, that are
collected through the available sensors (including the
TABLE I
robot) and updated from time to time; static information DAILY ROUTINE
contains: the patient Daily Routine, i.e. the set of daily
Time Range Daily Activity
activities that he/she has to carry out throughout the
07:00 - 07:30 Wake-up
day at given times, the Personality Profile, reporting 07:30 - 08:00 Breakfast
measures of five personality traits, i.e. Neuroticism, Ex- 09:30 - 11:00 Cognitive Entertainment
traversion, Openness, Agreeableness, and Conscientious- 11:30 Take medicine
12:30 - 13:30 Lunch
ness, assessed through the NEO Personality Inventory 14:00 - 16:00 Resting
test [4]; the Cognitive Profile, characterizing cognitive 16:00 - 18:00 Go out
and functional performance of Alzheimer’s patients and 18:30 - 19:30 Physical Entertainment
assessed through the ACE-R (Addenbrooke’s Cognitive 20:00 - 21:00 Dinner
Examination) test [5];
• the Assistive Workflow Management, i.e., the middleware
responsible for the execution of personalized assistive
plans represented as a workflow of services [6]; WakeUp
• the Daily Assistive Actions, i.e. the set of concrete service WakeUpMonitoring
2019−04−24
implementations provided by the technological devices, 7 : 0 0 :
that is periodically updated to take into account their 7 : 3 0 :
dynamic availability;
• the Smart Environment, i.e. the set of technological
B r e a k f a s t
devices deployed in the home environment that are: a B r e a k f a s t M o n i t o r i n g
Sanbot Elf, that is a low-cost mobile humanoid robot 2019−04−24
7 : 3 0 :
endowed with a tablet and an RGB-D camera (see Figure 8 : 0 0 :
3), iBeacons used for the indoor positioning system, able
to transmit a signal using Bluetooth Low Energy (BLE)
C o g n E n t e r t a i n m e n t
technology, a Polar M-600 smartwatch, and a Samsung S u g g e s t C o g n S t i m u l
Smartphone. 2019−04−24
09 : 3 0
11 : 0 0
All the considered devices are android-based, hence an-
droid applications are developed to communicate with their
T a k e M e d i c i n e
sensors and actuators. The workflow management subsys- MedicineRemind
tem is running on a PC where also user data are stored. 2019−04−24
The communication among all components of the system, 11 : 3 0
11 : 3 0
the services and the IoT devices, is based on Web Socket
protocol using Socket.IO, a JavaScript library for real-time .....
web applications that enables real-time, bi-directional and .....
event-based communication between web clients and servers,
and JSon messages. A server, running on the PC, manages Each assistive task is referred to as an AbsWorkflow,
the communication between the workflow manager and the a graph of abstract services, i.e. high level functionalities
concrete services. each one managed by an AbstractServiceAgent responsible
60
Workshop "From Objects to Agents" (WOA 2019)
Fig. 1. The abstract workflow for suggesting a stimulation activity
Fig. 2. The concrete workflow for suggesting a stimulation activity
for the lookup of the concrete services available to provide A. Selection of Concrete Services
the required functionality, and to contact the corresponding ConcreteServicesAgents, within the android applications,
providers represented as a ConcreteServiceAgent. are android services (i.e., threads working in background that
In Figure 1 the SuggestCognitiveStimulation ab- can be invoked via socket messages) and activities (for GUI).
stract workflow is reported, with the abstract services required A web interface used to activate concrete service individually
to perform it. For each abstract service, the corresponding Ab- is shown in Figure 4.
stractServiceAgent issues a request to the available Concrete- Currently, the types of available services are: 1) Monitoring
ServiceAgents, and it selects the first ConcreteServiceAgent Services that include activity recognition via a wearable de-
that replies or more complex mechanisms for ordering the vice or via camera using pose/skeleton recognition, emotion
replies can be implemented [7]. Once selected, the Con- and disengagement recognition, 2) Navigation Services for
creteServiceAgent is responsible for instantiating the QoS searching and approaching the user, 3) Interaction Services
parameters of the concrete service it provides, according to for speech recognition and speech synthesis using multimodal
the cognitive status and the personality profile of the specific interaction with the user.
patient. Currently the only QoS parameter considered is the 1) Monitoring Services: Monitoring services are developed
repetition frequency at which each service is executed. to monitor and recognize the current state of the user and the
performed Activities of Daily Living (ADLs) and instrumental
This selection process is repeated for all abstract services in Activities of Daily Living (IADL).
the workflow, until all of them are instantiated with a concrete In the current version of the system the state of the user is
implementation resulting in a concrete workflow, as the one evaluated through HR Detection service and Emotion Recog-
reported in Figure 2, ready to be executed by the selected nition service. The first aims at getting the current heart-rate
providers. The type of entertainment activities are randomly of the patient from the smartwatch and the average value of
selected among the ones preferred by the patient and reported 15 second of lectures is provided as a result. In the second,
in the user profile. the robot takes a video of the person that is analyze by the
During the execution of the concrete workflow, as shown in Affectiva SDK; this service returns the emotion (joy, surprise,
Figure 3, concrete services may fail due to either a timeout contempt, disgust, sadness, anger) of the person with the
for getting the service output, or for an unavailability of the highest mean on the whole video.
device delivering the service. In such cases, the corresponding ADL and IADL can be monitored by using either Pose
AbstractServiceAgent receives the error message and it selects Detection, Activity Classification, or Dialogue Check. The
the second ConcreteServiceAgent that replied to the initial Pose Detection service aims at evaluating the current pose
call, and so on. If all available devices fail, then the workflow of the user from wearable data. The robot communicates with
execution ends with a failure and a notification to a human the smartwatch via Bluetooth to take 512 data samples from
career is signaled. the smartwatch accelerometer and use a deep neural network
61
Workshop "From Objects to Agents" (WOA 2019)
Fig. 3. Selection of different devices for the same abstract service
for the classification process [8]; the recognized activities are:
standing up, getting up, walking, running, going up, jumping,
going down, lying down, sitting down. The Activity Classifica-
tion service implements an Activity Recognition algorithm by
analyzing a video sampled from the robot camera to detect Fig. 4. Server interface to control individual services
the skeleton pose for each video frame [9]. The activity
recognition works again with a deep neural network on a
window of 140 frames of human skeleton data with 30 fps. The the Dialogue Suggest service is used to suggest the activities,
activity recognized are watching tv, relaxing on couch, ironing, while the services Video Entertainment, Audio Entertainment,
making coffee, working at PC, and talking on the phone. and Game Entertainment are invoked to show respectively,
Finally, the Dialogue Check service allows to use the robot video, audio or games to the user.
GUI to directly ask a question to the user regarding his/her IV. T HE P ILOT S TUDY
activity. Two buttons (for positive and negative answers) are
A prototype implementation of the robotic-based AAL sys-
showed on the robot tablet (e.g. a question can ask if the user
tem has been developed and tested in a controlled environment
has taken the medicines).
with real patients with different degrees of the Alzheimer’s
Two additional services, In Room Detection and In Room
disease, recruited by the team of neurologists of the project1 .
Detection with Robot, are developed to infer additional infor-
The purpose of this preliminary experimentation was to
mation on the user state and activity. In details,with the In
evaluate a set of ratings to assess the general acceptance
Room Detection service the robot contacts the smartwatch via
degree of the system according to the different cognitive and
Bluetooth to know the position of the person in the house; the
personality characteristics of the selected patients, by varying
smartwatch communicates with the beacons placed in each
the repetition frequency of the interaction services suggesting
room of the house to calculate the distances between the
various entertainment activities, in a user-transparent manner.
user (that wears the smartwatch) and the beacon; this service
To determine the strength of predictors for elderly participants’
returns the label of the room with the beacon at minor distance
intention to accept and use the system, we adopted a modified
from the smartwatch, so locating the person into the house.
version of the Unified Theory of Acceptance and Use of
The same behavior can be exploited by the robot itself, when
Technology (UTAUT) questionnaire [11] specifically designed
it is in the presence of the user.
2) Navigation Services: These services are developed in or- by the team of psychologists of the project. This represents an
der to make the robot able to navigate within the environment instrument to measure the variety of perceptions of information
and to locate the user [10]. Look user is a simple service that technology innovations.
moves only the camera, that is located in the robot head, to The rationale of the experimentation is to collect informa-
scan if there is a user nearby the robot. Images from the camera tion on possible relationships between the modalities in which
are processed by the use of the PoseNet service to identify a services are executed, and the cognitive and personality traits
possible user. On the contrary, the service Find User is used of patients. Such information will be used for a customization
by the robot to randomly navigate within the environment, of the assistive tasks delivered by the system, so to improve
avoiding obstacles, and searching for a user. Once the user has its acceptance level, by selecting behaviors that result less
been detected, the Approach service can be requested to make disturbing for each patient because more compliant with
the robot move to the correct interacting/monitoring position. his/her profile.
The patients were left alone in an area of the controlled
Finally, the Stop Robot service is used to start the search for
environment resembling a house room, where they could
the charging station in order for the robot to charge its battery.
3) Interaction Services: Interaction modalities of the robot perform different entertainment activities (read a book or a
(voice interaction and GUI) can be used to suggest and show 1 This experimental study has been approved by the ethical committee of
personalized entertainment activities to the user. In details, the University Federico II with protocol number 167/18.
62
Workshop "From Objects to Agents" (WOA 2019)
TABLE II
UTAUT RESULTS
Code Construct Max Min Avg Std
ANX Anxiety 5 3 3.9 0.8
ATT Attitude 4 2 3.2 0.8
FC Facilitating conditions 4 2 2.8 0.9
ITU Intention to use 4 3 3.3 0.5
PAD Perceived adaptability 4.3 3 3.7 0.6
PENJ Perceived enjoyment 4.6 2.8 3.4 0.8
PEOU Perceived ease of use 3.6 2.4 3.1 0.5
PS Perceived sociability 5 2.5 3.8 1.0
PU Perceived usefulness 3.7 3 3.3 0.3
Fig. 5. One of the selected patients interacting with the system (left). The SI Social influence 4 1 2.5 1.3
robot (right) SP Social presence 4.4 1.6 3.1 1.1
TR Trust 4 2 3.1 0.9
magazine, watching tv, listen to music, playing at a PC, take
a refreshment). From time to time, a monitoring workflow
was executed to check if the user was doing some activity. In The questionnaire was translated from English to Italian
case he/she was not performing any activity, an entertainment by two psychologists and an engineer, that were proficient
workflow was executed by varying the frequency when the in English and Italian and familiar with HRI. The translation
entertainment was suggested. Each user interacted with the was examined at a consensus meeting, back-translated, and
system for approximately three hours. approved at a second consensus meeting. A comprehension
test was carried out in a subgroup of 15 individuals aged
A. Patients’ Traits Classification 18 years. This consisted of a face-to-face interview during
The present study included four subjects (two men and two which the interviewer inquired whether the subject had any
women) between 71 and 75 years old. All subjects performed difficulty in understanding the questions and the pre-coded
a cognitive screening test (ACE-R) [5], and a personality answers. A comprehension rate was obtained as the percentage
questionnaire (NEO PI -3) [4]. The first subject (man; age of questions and pre-coded answers of all items correctly
= 74; years of education = 8) showed a cognitive decline with understood by subjects. In the test, more than 90% of subjects
significant difficulties of memory and fluency. The personality found the questions easy to understand and had no difficulty
profile showed low levels of neuroticism and normal levels in interpreting the answer modes. The final Italian version of
of openness, without depressive symptoms. The second sub- the questionnaire is available from the authors upon request.
ject (woman; age = 75; years of education = 5) showed a Results of the UTAUT evaluation is shown in Table II.
cognitive decline with significant difficulties of memory, at- We consider a positive perception of a participant when the
tention, fluency and language. The personality profile showed construct score is greater than 3, while a negative perception
higher levels of neuroticism and low levels of openness, with is when average score is lower than 3 (in a scale from 1
depressive symptoms. The third subject (woman; age = 71; to 5). Facilitating Conditions and Social Influence are the
years of education = 8) did not show a cognitive decline, only two constructs obtaining a score lower than 3. However,
but light memory difficulties. The personality profile showed these results are compatible with our experimental setting
normal levels of neuroticism and high levels of openness, that was tested by the patients left alone, without involving
without depressive symptoms. The last subject (man; age = caregivers, in a simulated home environment in the research
75; years of education = 18) did not show a cognitive decline, laboratory, and so not a real domestic environment. Indeed,
but only slight difficulties in memory tasks. The personality Facilitating Conditions refers to factors in the environment
profile showed lower levels of neuroticism and normal levels that facilitate use of the system, while Social Influence refers
of openness, without depressive symptoms. to the perception that people who are important for the subject
B. A UTAUT-based Usability Test think he/she should or should not use the system.
We adopted the version of the UTAUT questionnaire pro- In addition, it was noticed that the patient with a low
posed by [12] because it was already adapted and validated education level, depressive symptoms, and a high neuroticism
in the similar context of assistive robotics applied to elderly was less incline to be engaged in the interaction with the robot,
users. This UTAUT questionnaire consists of 41 items and and so to accept its suggestions. On the contrary, patients
explores 12 constructs: Anxiety (ANX), Attitude (ATT), Fa- with similar cognitive decline, but with a higher education
cilitating conditions (FC), Intention to use (ITU), Perceived level and lower neuroticism showed a better acceptance and
adaptability (PAD), Perceived enjoyment (PENJ), Perceived a stronger engagement with the robotic system. These results
ease of use (PEOU), Perceived sociability (PS), Perceived are encouraging in showing that personality traits, more than
usefulness (PU), Social influence (SI), Social Presence (SP) cognitive decline, play a crucial role in the acceptance of the
and Trust. Subjects are required to reply to each item on a robotic system and so it has to be taken into consideration for
Likert type scale (range: 1-5). personalization of patient-system interaction.
63
Workshop "From Objects to Agents" (WOA 2019)
V. C ONCLUSIONS
R EFERENCES
In this work, we presented a general overview of the
proposed approach and the software services developed in [1] E. Broadbent, R. Stafford, and B. MacDonald, “Acceptance of healthcare
the context of the UPA4SAR project. Moreover, we presented robots for the older population: Review and future directions,” Interna-
the results of a pilot study that we conducted in a research tional Journal of Social Robotics, vol. 1, no. 4, p. 319, Oct 2009.
[2] S. Rossi, G. Ercolano, L. Raggioli, M. Valentino, and C. Di Napoli,
laboratory environment with 4 real end-users recruited for the “A framework for personalized and adaptive socially assistive robotics,”
project. in Proceedings of the 19th Workshop ”From Objects to Agents” - Vol.
The experimentation highlighted a general good acceptance 2215. CEUR, 2018, pp. 90–95.
rate of the system, even if, in the case of the patient with an [3] C. Di Napoli, “Software agents to enable service composition through
negotiation,” Studies in Computational Intelligence, vol. 170, pp. 275–
high neuroticism, the interaction with the robot was limited. 296, 2009.
Moreover, from the users’ free comments, we observed a [4] R. R. McCrae, J. Paul T. Costa, and T. A. Martin, “The neo-pi-3: A
immediate attachment to the robot. These results are encour- more readable revised neo personality inventory,” Journal of Personality
Assessment, vol. 84, no. 3, pp. 261–270, 2005.
aging for the next experimental stage that will be conducted [5] E. Mioshi, K. Dawson, J. Mitchell, R. Arnold, and J. R. Hodges, “The
directly in the patients’ homes. In this stage, 40 patients will Addenbrooke’s cognitive examination revised (ace-r): a brief cognitive
be recruited (20 with a mild level of Alzheimer, 10 with a test battery for dementia screening,” International Journal of Geriatric
Psychiatry, vol. 21, no. 11, pp. 1078–1085, 2006.
moderate level and 10 with a subjective memory disorder), and [6] C. Di Napoli, M. Valentino, L. Sabatucci, and M. Cossentino, “Adaptive
the robot will stay in their home for 14 days each. This longer workflows of home-care services,” in Proceedings of the 27th IEEE
period, will allow us to collect more reliable data regarding International Conference on Enabling Technologies: Infrastructure for
Collaborative Enterprises (in press). IEEE, 2018.
the acceptance rate of the robotic-based AAL system to better [7] C. Di Napoli, P. Pisa, and S. Rossi, “Towards a dynamic negotiation
tune the adaptation of the robot’s behavior with respect to mechanism for qos-aware service markets,” in Trends in Practical Ap-
each single patient. Indeed, the robot is also able to adapt its plications of Agents and Multiagent Systems, ser. Advances in Intelligent
Systems and Computing. Springer International Publishing, 2013, vol.
own social interaction parameters, such as the proxemics, the 221, pp. 9–16.
speed of movements, and the interaction modalities, so future [8] S. Rossi, L. Bove, S. Di Martino, and G. Ercolano, “A two-step
work are planned to consider also these QoS parameters to be framework for novelty detection in activities of daily living,” in Social
tuned to validate the reliability of the proposed design choices. Robotics. Cham: Springer International Publishing, 2018, pp. 329–339.
[9] G. Ercolano, D. Riccio, and S. Rossi, “Two deep approaches for adl
It is expected that the adaptation of these parameters to the recognition: A multi-scale lstm and a cnn-lstm with a 3d matrix skeleton
specific user’s profile will allow to overcome the novelty effect representation,” in 26th IEEE International Symposium on Robot and
caused by the robot presence and to evaluate its long-term Human Interactive Communication (RO-MAN), Aug 2017, pp. 877–882.
[10] G. Ercolano, L. Raggioli, E. Leone, M. Ruocco, E. Savino, and S. Rossi,
effectiveness. “Seeking and approaching users in domestic environments: Testing a
reactive approach on two commercial robots,” in 27th IEEE International
ACKNOWLEDGMENT Symposium on Robot and Human Interactive Communication (RO-
This work has been supported by MIUR (Italian Ministry of MAN), Aug 2018, pp. 808–813.
[11] V. Venkatesh, M. G. Morris, G. B. Davis, and F. D. Davis, “User
Education, Universities, and Research) within the PRIN 2015 acceptance of information technology: Toward a unified view.” MIS
research project “UPA4SAR - User-centered Profiling and Quarterly, vol. 27, no. 3, pp. 425–478, 2003.
Adaptation for Socially Assistive Robotics” (grant n. 2015KB- [12] M. Heerink, B. Kröse, V. Evers, and B. Wielinga, “Assessing acceptance
L78T). The authors thank OMITECH s.r.l. for providing the of assistive social agent technology by older adults: the Almere model,”
International Journal of Social Robotics, vol. 2, no. 4, pp. 361–375, Dec
Sanbot Elf robots for the project experimentation phase. 2010.
64