=Paper=
{{Paper
|id=Vol-3794/paper07
|storemode=property
|title=Enhancing Cognitive Training: Investigating the Impact of a
Suggestion-Offering Robot on Performance and Satisfaction
|pdfUrl=https://ceur-ws.org/Vol-3794/paper7.pdf
|volume=Vol-3794
|authors=Flavio Ruggiero,Marco Matarese, Alessandra Sciutti,Mariacarla Staffa
|dblpUrl=https://dblp.org/rec/conf/rfh/RuggieroMSS24
}}
==Enhancing Cognitive Training: Investigating the Impact of a
Suggestion-Offering Robot on Performance and Satisfaction==
https://ceur-ws.org/Vol-3794/paper7.pdf
Enhancing Cognitive Training: Investigating the Impact of a
Suggestion-Offering Robot on Performance and Satisfaction⋆
Flavio Ruggiero1 , Marco Matarese2,* , Alessandra Sciutti2 and Mariacarla Staffa1
1
University of Naples Parthenope, Centro Direzionale of Naples, Isola C4, 80143, Naples, Italy.
2
Italian Institute of Technology, Via Morego 30, 16163, Genoa, Italy.
Abstract
Advancements in technology have created opportunities to enhance cognitive training through robotic assistance, offering personalized
suggestions. This paper investigates how such robots impact performance and satisfaction in cognitive training. Understanding their
effects holds significance for psychology, human-computer interaction, and education. Through a review of relevant literature and
empirical study, insights into human-robot interaction dynamics and cognitive enhancement are gained. This exploration contributes to
optimizing human cognitive abilities in assistive settings. Our results demonstrate that robot tutors’ competence and periodic help
offerings are insufficient unless human tutees recognize the task’s difficulty.
Keywords
Human-Robot Interaction, Robot Tutoring, Cognitive Training, Assistive Robotics
1. Introduction cations and potential avenues for future research. Through
this exploration, we aim to contribute to the growing body
In the realm of cognitive training, advancements in technol- of knowledge concerning integrating robotics in cognitive
ogy have ushered in new opportunities to enhance human enhancement endeavors, ultimately striving towards opti-
performance and satisfaction. Among these advancements, mizing human cognitive abilities in assistive settings.
the integration of robotic assistance stands out as a promis-
ing avenue. Robots equipped with artificial intelligence
(AI) capabilities not only offer practical assistance but also 2. Related Works
possess the potential to provide personalized suggestions
tailored to individual needs. This intersection of technol- The utilization of robots for training and assistance has expe-
ogy and cognitive training prompts an intriguing question: rienced a notable surge, spanning various application areas
How does the presence of a suggestion-offering robot influ- such as education [1] and cognitive training customized
ence individuals’ performance and satisfaction in cognitive for individuals with specific requirements, including those
training tasks? diagnosed with autism and the elderly [2, 3]. The manifold
This paper delves into the exploration of this question, benefits of these applications are apparent, encompassing ca-
aiming to shed light on the implications of integrating such pabilities such as enabling concurrent usage at home under
robotic systems into cognitive training environments. By remote supervision by instructors or healthcare providers
examining the effects of a suggestion-offering robot on and facilitating personalized and adaptable training pro-
both performance metrics and subjective satisfaction levels, grams tailored to meet individual needs.
we seek to uncover valuable insights into the dynamics of Nonetheless, a significant challenge arises in finding the
human-robot interaction within the context of cognitive ideal equilibrium between the robot’s assistance and the
enhancement. Understanding the impact of a suggestion- individual’s active participation. This predicament is widely
offering robot on cognitive training tasks holds significance acknowledged in the field of robotics rehabilitation, where
for various fields, including psychology, human-computer interventions must carefully balance support to prevent in-
interaction, and education. Insights gleaned from this inves- advertently encouraging passivity (or idleness) [4]. Offering
tigation could inform the design of future cognitive training on-demand assistance is a common strategy, but it carries
programs and the development of more effective human- the risk of improper system usage, leading to suboptimal
robot collaboration frameworks. utilization of assistance, including aversion or misuse [5].
In this paper, we will first review relevant literature on Even within the context of robot-assisted gaming, there is
cognitive training methodologies, human-robot interaction, considerable variation in individuals’ reliance on technology
and the influence of technology on cognitive performance. for support, influenced by factors such as their comfort with
Subsequently, we will outline the methodology employed technology and personality traits [6, 7]. It is imperative to
in our study, including the experimental design, participant address instances where individuals overly depend on assis-
recruitment, and data collection procedures. We will then tance, as it can hinder the learning process. Therefore, it is
present and analyze the results obtained, discussing impli- essential to understand the impact of a suggestion-offering
robot on users and to understand if it can be considered a
helpful supporter for clinicians and patients in such tasks.
Robots for Humans Workshop at the International Conference on Advanced This study endeavors to explore the quantitative and qualita-
Visual Interfaces, June 03–07, 2024, Arenzano, Italy.
⋆ tive impacts of a suggestion-offering robot in the completion
You can use this document as the template for preparing your publica-
tion. We recommend using the latest version of the ceurart style. of a cognitive task by aiming to assess how it influences both
*
Corresponding author. the participants’ learning outcomes and their perception of
$ flavio.ruggiero@studenti.uniparthenope.it (F. Ruggiero); the robot as a helper, as well as their inclination to use it
marco.matarese@iit.it (M. Matarese); alessandra.sciutti@iit.it again in the future.
(A. Sciutti); mariacarla.staffa@uniparthenope.it (M. Staffa)
0000-0003-1719-3745 (M. Matarese); 0000-0002-1056-3398
(A. Sciutti); 0000-0001-7656-8370 (M. Staffa)
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribu-
tion 4.0 International (CC BY 4.0).
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
Figure 1: Computerized version of the London Tower test in
Android.
3. Materials and Methods
3.1. Tower of London Test (ToL) Figure 2: Pepper Robot during the administration of the test.
The Tower of London (ToL) test is a cognitive test devel-
oped in the 1980s by Tim Shallice and Rosaleen A. Mc-
Carthy. The test aims to evaluate the capacity for strategic correctly, the number of moves used to solve them, and the
decision-making and effective planning to solve tasks by time spent to solve the tasks.
anticipating and considering the consequences of actions on In each trial, the subject is asked to move the balls across
interconnected elements. The interdependence of elements a display to reach the configuration in the upper half of
in complex problems mirrors situations in daily life. The the screen. Participants dragged the ball with their fingers
ToL test presents a graded difficulty problem where partici- from the free position to one of the other rods to move it.
pants must move perforated balls arranged on a structure Incorrect movements were recorded on the computer as
to achieve a new configuration, requiring the adoption of non-responses. Subjects are asked to solve the task in the
appropriate strategies. minimum possible number of moves.
Specifically, three operations are vital: (a) formulating
a general plan, (b) identifying and organizing sub-goals in
a sequence of movements, and (c) storing both sub-goals
3.2. Stimuli
and the general plan in working memory [8, 9, 10]. Shallice The characteristics of the task we used allow for several
challenges the Supervisory Attentional System (SAS) as a ways of administration and types of measurements. In this
central mechanism in the prefrontal cortex, directing at- study, we aim to investigate to what extent the adjustable
tention towards necessary sub-goals and shifting attention social autonomy of a humanoid social robot employed to
from sub-goals to the general scheme [10]. administer the London Tower test can impact the users’
In the classic version of Shallice the material consists performance.
of three pegs of different lengths mounted on a wooden Considering this objective, we examined the effect of
structure and three balls of different colors (red, green, and Suggestion-Offering Robot users. Specifically, to prevent
blue), which are inserted. The test consists of a series of help aversion, the robot proactively asks the users whether
twelve tests of increasing gradual difficulty depending on they needed a suggestion every 10 seconds. However, we
the number of moves that must be executed to arrive at the instructed both the participants that they could ask the robot
solution. In this work, we propose a computerized version for help in every step of the task.
of the test consisting of a support with three vertical posts
of different heights and three colored balls (green, red, and 3.3. Measures
blue), as shown in Figure 1. The test was displayed on a
Pepper robot’s tablet as shown in Figure 2. For the evaluation of the test performance, the following
The objective of the test is to move the balls from an initial metrics will be considered:
configuration to a diverse target configuration, following
four rules: • Time: time to accomplish the task.
• Number of Moves (NMOV: total number of moves
1 Only one ball can be moved at a time. performed to complete the problem.
2 Only one ball can be moved from one post to another • Number of Successes (S): the number of completed
to prevent the user from placing the balls on the table problems.
or having more than one in his hand at a time. • Number of Suggestions requested (NS): the num-
3 It is possible to insert only one ball on the smaller ber of success w.r.t. the total number of tests.
post, two on the intermediate, and three on the larger • Performance Measure (PM): the number of suc-
one. cess w.r.t. the total number of tests.
4 The maximum number of movements allowed to
solve a problem can not be exceeded Before and after the interaction with the robot, we sub-
mitted questionnaires to participants to collect self-reported
To calculate the patient’s final score at the end of the test, measures about the HRI and the participants themselves.
it is necessary to consider the number of problems solved In the pre-experiment questionnaire, we submitted partici-
pants the Big Five personality traits test [11] and the Sense
of Agency scale [12]. Moreover, we collected some items Measure Phase AVG STD
about their perception of the robot, such as the Godspeed Time Assessment 14.35 10.46
questionnaires [13] and the Inclusion of Others in the Self Baseline 17.35 13.19
test (IOS) [14]. We also submitted these latter two during Training 11.38 9.87
the post-experiment questionnaire to measure the change Moves Assessment 7.24 3.25
in participants’ perception of the robot. Baseline 7.14 3.58
Training 5.49 3.76
Suggestions Assessment 0.00 0.00
4. Experimental Procedure Baseline 0.00 0.00
Training 0.11 0.44
Score Assessment 2.55 0.65
4.1. Procedure Baseline 2.48 0.72
The ToL test is extensively used in the context of cognitive Training 2.72 0.57
training for patients with neurodegenerative disorders, such
Table 1
as Parkinson’s disease [15]. Indeed, cognitive training is Descriptive Statistics of the Metrics.
a promising nonpharmacological treatment option for pa-
tients with dementia [16, 17]. Although testing on patients
was our goal, we wanted to test the protocol with healthy
participants to ensure the feasibility of the study. Thus, we the description of such an action. There was no limit about
asked 20 computer science students at the University of the number of suggestions that participants could ask; ba-
Naples Parthenope to participate in the study. sically, they could ask for suggestions at each step of the
In our experimental procedure, the ToL Test is adminis- tests.
tered by the Pepper robot. After an initial phase in which
the robot provides participants with an explanation of the 4.3. Hypotheses
rules, the robot shows a test configuration to be used to
Our hypothesis regarded both participants’ performance
verify understanding of the structure of the test by admin-
and satisfaction. We aim to examine the degree of assis-
istering it subsequently. At this point, the user views the
tance participants require in completing the task. This holds
configuration to be replicated on Pepper’s tablet (Figure
paramount significance because, while we acknowledge that
2), reproduces the one included, and continues in this way
support might facilitate an immediate understanding of how
until all the configurations provided by the humanoid are
to solve the test, we are cautious about the potential long-
exhausted (12 configurations for each participant). During
term impact on participants’ performance. We want to avoid
the ToL test execution, the social robot occasionally offers
the risk of participants continually relying on suggestions,
move suggestions to users.
which may not lead to sustained improvement and even
Namely, participants underwent three experimental
exacerbate their overall performance.
phases:
• Baseline: a practice phase where requesting sugges-
tions is not allowed, aiming to help the user become
5. Results and Discussion
familiar with the test. We found that the minority of participants asked the robot
• Training: an intermediate phase in which partici- for help during training (45% on average), with an average
pants train to complete the test where it is allowed number of suggestions asked of 1.53 (with 𝜎 = .717) among
to request suggestions. those who asked, and a maximum number of 3 suggestions
• Assessment: a final phase where requesting sug- asked per test. Hence, to investigate the difference between
gestions is not allowed, and the user, with the expe- the tests in which participants asked the robot for help
rience gained from the previous two phases, should and those in which they never asked, we introduced in our
demonstrate greater proficiency in completing the analysis a binary categorical variable representing whether
test. participants asked for help at least once during each test
(the single configuration to solve) of the training.
4.2. Robot’s Suggestions To measure participants’ improvements, we defined the
delta between the number of moves needed to solve a ToL
We needed an expert robot to provide useful and correct test and those performed by participants to solve it as their
on-demand suggestions for our training phase. For this pur- difference in absolute value. Considering this metric, par-
pose, we designed a graph-based representation of the ToL ticipants’ performance showed an improvement from the
test. Subsequently, we implemented a solving algorithm Baseline and Training phases (𝜇 = 1.89 with 𝜎 = 2.85 re-
that used a breadth-first search (BFS) strategy to find the garding the Baseline phase, and 𝜇 = 0.988 with 𝜎 = 2.73
optimal solution for each ToL test. Starting from the initial regarding the Training phase). Through a two-way ANOVA
ToL configuration representation, the BFS algorithm could test, we found an effect of the experimental phase on such
find the optimal path to the representation of the final one. measure of learning (𝐹 (2) = 3.57 with 𝑝 = .029). With a
Although a BFS-based resolution of the ToL test has a lin- post-hoc Tukey correction, we found that such an improve-
ear time complexity on the number of the graph’s edges ment was statistically significant (𝑡 = 2.548 with 𝑝 = .03),
and nodes, it was completely solvable since we considered while there was no statistical difference between perfor-
a small input space with only three balls and three spots. mance in Baseline/Training and Assessment. Hence, we
Hence, when participants press the button to ask for a sug- did not find significant improvements in Assessment on
gestion, the robot queries the algorithm to retrieve the first average.
action to optimally reach the final game configuration (the Regarding the Training phase, the number of suggestions
goal) from the current one. The robot suggestion involved
Figure 3: Delta (in absolute values) between participants’ actions Figure 4: Participants’ completion times (in seconds) in both
and those needed to solve the relative ToL configurations during the experimental groups, divided depending on whether they
the Training phase, considered as a measure of performance. The asked the robot for help at least once, or they never asked it for
closer the delta values are to zero, the higher the participants’ suggestions.
performance since it reflects the number of moves they performed
to solve the configuration in addition to those required. The 𝑥-
axis represents the trials in which participants asked for help at
least once, or they never asked for help. ticipants performing more moves than the others, as shown
in Figure 5.
Participants who asked the robot for help are those who
perceived its suggestions more useful as the correlation
asked significantly impacted as a covariate on the number between the perceived usefulness of suggestions and the
of moves they performed to solve the configurations (AN- number of suggestions asked (Pearsons’ 𝜌 = .492 with 𝑝 =
COVA test 𝐹 (1) = 13.9 with 𝑝 < .001) and completion .028). Moreover, by comparing participants’ answers to the
time (ANCOVA test 𝐹 (1) = 44.4 with 𝑝 < .001). We IOS test, we found that participants felt closer to the robot
found significantly worse performance in participants who after having interacted with it, regardless of whether they
asked the robot for help compared to those who never re- asked it for help (Repeated Measures ANOVA, 𝐹 (1) = 5.716
quested (Independent Samples t-test, 𝑡(238) = −2.16 with with 𝑝 = .028). A post-hoc Tukey correction highlighted a
𝑝 = .032), as shown in Figure 3. Moreover, we observed significant difference between the pre- and post-experiment
that the number of suggestions asked positively correlated test answers (𝑡 = −2.39 with 𝑝 = .028).
with participants completion time (Pearsons’ 𝜌 = .406 with Finally, we found that participants’ personality dimen-
𝑝 < .001), number of moves to solve the configurations sions (Big 5 and Sense of Agency tests) did not affect their
(Pearsons’ 𝜌 = .226 with 𝑝 < .001), and the configura- behavior during the training and assessment phases. The
tions’ difficulty (Pearsons’ 𝜌 = .238 with 𝑝 < .001). two robot modalities did not elicit differences in participants
Since the robot suggestions were optimal, these latter cor- perception of it (Godspeed test). We neither found differ-
relations explain the worse performance of those who asked ences between the groups regarding the perceived closeness
for help. We hypothesize that the participants who asked to the robot (IOS test).
for the robot’s suggestions also did not know the game or
obtained worse performance. Thus, only bad players needed
to ask the robot for help. This hypothesis is supported by
the latter correlations, which highlight how the number of
suggestions asked grew with the difficulty of the game (and
vice versa) and that participants who asked the robot for
help more were those who obtained worse performance in
solving the game.
As a further measure of performance, we collected par-
ticipants’ completion times, which were defined as the time
taken to complete each ToL test. We performed an Inde-
pendent Samples t-test to compare participants’ completion
times depending on whether they asked the robot for help
at least once during the tests or never asked. We found
that assisted participants took more time to complete the
ToL configurations than the other (𝑡(398) = −4.79 with
𝑝 < .001), as shown in Figure 4.
We found similar results regarding the number of moves
Figure 5: Number of moves participants performed to solve
participants performed to solve the ToL tests. We conducted the ToL configurations in both the experimental groups divided
an Independent Samples t-test on the number of moves par- depending on whether they asked the robot for help at least once
ticipants performed to solve the configurations and found or they never asked it for suggestions.
a significant effect on whether they asked for help at least
once (𝑡(398) = −2.7 with 𝑝 = .008) with the assisted par-
6. Conclusions Robotics and Automation Letters 3 (2018) 3701–3708.
Publisher: IEEE.
Despite the robot’s availability to provide suggestions and [7] S. Rossi, D. Conti, F. Garramone, G. Santangelo,
its correct and complete knowledge of the task, a large num- M. Staffa, S. Varrasi, A. Di Nuovo, The role of personal-
ber of participants (55%) never asked for help during the ity factors and empathy in the acceptance and perfor-
game. However, they could have gained from the robot’s mance of a social robot for psychometric evaluations,
support, as their performance during the training was sub- Robotics 9 (2020). doi:10.3390/robotics9020039.
optimal. Interestingly, participants who asked for help had [8] R. G. Morris, A. D. Baddeley, Primary and working
more difficulty with the task (as highlighted by an aver- memory functioning in alzheimer-type dementia, Jour-
age longer completion time and a larger number of moves). nal of Clinical and Experimental Neuropsychology 10
Furthermore, they also recognized the usefulness of the (1988) 279–296. doi:10.1080/01688638808408242,
suggestions at the end of the game. Nonetheless, the vast pMID: 3280591.
majority did not take advantage of the robot’s help but pre- [9] A. M. Owen, J. J. Downes, B. J. Sahakian, C. E. Polkey,
ferred to play autonomously. This was probably due to the T. W. Robbins, Planning and spatial working memory
perception that the task was sufficiently easy to be addressed following frontal lobe lesions in man, Neuropsycholo-
and by the desire to challenge themselves. Although this gia 28 (1990) 1021–1034. doi:https://doi.org/10.
is certainly positive, it also raises the question of how to 1016/0028-3932(90)90137-D.
best provide robot support to maximize its utility. Our re- [10] T. Shallice, Specific impairments of planning, Philo-
sults demonstrate that perfect competence and periodic help sophical Transactions of the Royal Society of London.
offerings are insufficient unless the participants recognize B, Biological Sciences 298 (1982) 199–209. doi:http:
their difficulty. Future work should focus on identifying //doi.org/10.1098/rstb.1982.0082.
novel strategies to counteract help aversion. [11] S. D. Gosling, P. J. Rentfrow, W. B. Swann, A very brief
measure of the big-five personality domains, Journal of
Research in Personality 37 (2003) 504–528. doi:https:
Acknowledgments //doi.org/10.1016/S0092-6566(03)00046-1.
The work was supported by the research RESTART project [12] A. Tapal, E. Oren, R. Dar, B. Eitam, The sense of agency
(Robot Enhanced Social abilities based on Theory of mind scale: A measure of consciously perceived control over
for Acceptance of Robot in assistive Treatments) (CUP: one’s mind, body, and the immediate environment,
I53D23003780001), funded by the MIUR with D.D. no.861 un- Frontiers in psychology 8 (2017) 1552. doi:10.3389/
der the PNRR and by the European Union - Next Generation fpsyg.2017.01552.
EU. [13] C. Bartneck, D. Kulić, E. Croft, S. Zoghbi, Measurement
instruments for the anthropomorphism, animacy, like-
ability, perceived intelligence, and perceived safety of
References robots, International journal of social robotics 1 (2009)
71–81. doi:10.1007/s12369-008-0001-3.
[1] T. Belpaeme, J. Kennedy, A. Ramachandran, B. Scas- [14] A. Aron, E. N. Aron, D. Smollan, Inclusion of other in
sellati, F. Tanaka, Social robots for education: A the self scale and the structure of interpersonal close-
review, Science Robotics 3 (2018) eaat5954. doi:10. ness., Journal of personality and social psychology 63
1126/scirobotics.aat5954. (1992) 596. doi:10.1037/0022-3514.63.4.596.
[2] F. Yuan, E. Klavon, Z. Liu, R. P. Lopez, X. Zhao, A sys- [15] R. Biundo, L. Weis, G. Abbruzzese, G. Calandra-
tematic review of robotic rehabilitation for cognitive Buonaura, P. Cortelli, M. C. Jori, L. Lopiano, R. Mar-
training, Frontiers in Robotics and AI 8 (2021). coni, A. Matinella, F. Morgante, et al., Impulse control
[3] S. Rossi, G. Santangelo, M. Staffa, S. Varrasi, D. Conti, disorders in advanced parkinson’s disease with dyski-
A. Di Nuovo, Psychometric evaluation supported by a nesia: the althea study, Movement Disorders 32 (2017)
social robot: Personality factors and technology accep- 1557–1565. doi:10.1002/mds.27181.
tance, in: 2018 27th IEEE International Symposium on [16] J. V. Hindle, A. Petrelli, L. Clare, E. Kalbe, Nonpharma-
Robot and Human Interactive Communication (RO- cological enhancement of cognitive function in parkin-
MAN), 2018, pp. 802–807. doi:10.1109/ROMAN.2018. son’s disease: a systematic review, Movement Disor-
8525838. ders 28 (2013) 1034–1049. doi:10.1002/mds.25377.
[4] D. Piovesan, A computational index to describe slack- [17] B. Guglietti, D. Hobbs, L. E. Collins-Praino, Optimiz-
ing during robot therapy, in: J. Laczko, M. L. Latash ing cognitive training for the treatment of cognitive
(Eds.), Progress in Motor Control: Theories and Trans- dysfunction in parkinson’s disease: current limita-
lations, Advances in Experimental Medicine and Biol- tions and future directions, Frontiers in Aging Neuro-
ogy, Springer International Publishing, Cham, 2016, pp. science 13 (2021) 709484. doi:10.3389/fnagi.2021.
351–365. doi:10.1007/978-3-319-47313-0\_19. 709484.
[5] A. Ramachandran, C.-M. Huang, B. Scassellati, To-
ward effective robot–child tutoring: Internal motiva-
tion, behavioral intervention, and learning outcomes,
ACM Transactions on Interactive Intelligent Systems
9 (2019) 1–23. doi:10.1145/3213768.
[6] A. M. Aroyo, F. Rea, G. Sandini, A. Sciutti, Trust
and social engineering in human robot interaction:
Will a robot make you disclose sensitive information,
conform to its recommendations or gamble?, IEEE