In the realm of cognitive training, advancements in technology have ushered in new opportunities to enhance human performance and satisfaction. Among these advancements, the integration of robotic assistance stands out as a promising avenue. Robots equipped with artificial intelligence (AI) capabilities not only offer practical assistance but also possess the potential to provide personalized suggestions tailored to individual needs. This intersection of technology and cognitive training prompts an intriguing question: How does the presence of a suggestion-offering robot influence individuals' performance and satisfaction in cognitive training tasks?
This paper delves into the exploration of this question, aiming to shed light on the implications of integrating such robotic systems into cognitive training environments. By examining the effects of a suggestion-offering robot on both performance metrics and subjective satisfaction levels, we seek to uncover valuable insights into the dynamics of human-robot interaction within the context of cognitive enhancement. Understanding the impact of a suggestionoffering robot on cognitive training tasks holds significance for various fields, including psychology, human-computer interaction, and education. Insights gleaned from this investigation could inform the design of future cognitive training programs and the development of more effective humanrobot collaboration frameworks.
In this paper, we will first review relevant literature on cognitive training methodologies, human-robot interaction, and the influence of technology on cognitive performance. Subsequently, we will outline the methodology employed in our study, including the experimental design, participant recruitment, and data collection procedures. We will then present and analyze the results obtained, discussing impli-
The utilization of robots for training and assistance has experienced a notable surge, spanning various application areas such as education [1] and cognitive training customized for individuals with specific requirements, including those diagnosed with autism and the elderly [2,3]. The manifold benefits of these applications are apparent, encompassing capabilities such as enabling concurrent usage at home under remote supervision by instructors or healthcare providers and facilitating personalized and adaptable training programs tailored to meet individual needs.
Nonetheless, a significant challenge arises in finding the ideal equilibrium between the robot's assistance and the individual's active participation. This predicament is widely acknowledged in the field of robotics rehabilitation, where interventions must carefully balance support to prevent inadvertently encouraging passivity (or idleness) [4]. Offering on-demand assistance is a common strategy, but it carries the risk of improper system usage, leading to suboptimal utilization of assistance, including aversion or misuse [5]. Even within the context of robot-assisted gaming, there is considerable variation in individuals' reliance on technology for support, influenced by factors such as their comfort with technology and personality traits [6,7]. It is imperative to address instances where individuals overly depend on assistance, as it can hinder the learning process. Therefore, it is essential to understand the impact of a suggestion-offering robot on users and to understand if it can be considered a helpful supporter for clinicians and patients in such tasks. This study endeavors to explore the quantitative and qualitative impacts of a suggestion-offering robot in the completion of a cognitive task by aiming to assess how it influences both the participants' learning outcomes and their perception of the robot as a helper, as well as their inclination to use it again in the future.
The Tower of London (ToL) test is a cognitive test developed in the 1980s by Tim Shallice and Rosaleen A. Mc-Carthy. The test aims to evaluate the capacity for strategic decision-making and effective planning to solve tasks by anticipating and considering the consequences of actions on interconnected elements. The interdependence of elements in complex problems mirrors situations in daily life. The ToL test presents a graded difficulty problem where participants must move perforated balls arranged on a structure to achieve a new configuration, requiring the adoption of appropriate strategies.
Specifically, three operations are vital: (a) formulating a general plan, (b) identifying and organizing sub-goals in a sequence of movements, and (c) storing both sub-goals and the general plan in working memory [8,9,10]. Shallice challenges the Supervisory Attentional System (SAS) as a central mechanism in the prefrontal cortex, directing attention towards necessary sub-goals and shifting attention from sub-goals to the general scheme [10].
In the classic version of Shallice the material consists of three pegs of different lengths mounted on a wooden structure and three balls of different colors (red, green, and blue), which are inserted. The test consists of a series of twelve tests of increasing gradual difficulty depending on the number of moves that must be executed to arrive at the solution. In this work, we propose a computerized version of the test consisting of a support with three vertical posts of different heights and three colored balls (green, red, and blue), as shown in Figure 1. The test was displayed on a Pepper robot's tablet as shown in Figure 2.
The objective of the test is to move the balls from an initial configuration to a diverse target configuration, following four rules:
1 Only one ball can be moved at a time. 2 Only one ball can be moved from one post to another to prevent the user from placing the balls on the table or having more than one in his hand at a time. correctly, the number of moves used to solve them, and the time spent to solve the tasks.
In each trial, the subject is asked to move the balls across a display to reach the configuration in the upper half of the screen. Participants dragged the ball with their fingers from the free position to one of the other rods to move it. Incorrect movements were recorded on the computer as non-responses. Subjects are asked to solve the task in the minimum possible number of moves.
The characteristics of the task we used allow for several ways of administration and types of measurements. In this study, we aim to investigate to what extent the adjustable social autonomy of a humanoid social robot employed to administer the London Tower test can impact the users' performance.
Considering this objective, we examined the effect of Suggestion-Offering Robot users. Specifically, to prevent help aversion, the robot proactively asks the users whether they needed a suggestion every 10 seconds. However, we instructed both the participants that they could ask the robot for help in every step of the task.
For the evaluation of the test performance, the following metrics will be considered: Before and after the interaction with the robot, we submitted questionnaires to participants to collect self-reported measures about the HRI and the participants themselves. In the pre-experiment questionnaire, we submitted participants the Big Five personality traits test [11] and the Sense of Agency scale [12]. Moreover, we collected some items about their perception of the robot, such as the Godspeed questionnaires [13] and the Inclusion of Others in the Self test (IOS) [14]. We also submitted these latter two during the post-experiment questionnaire to measure the change in participants' perception of the robot.
The ToL test is extensively used in the context of cognitive training for patients with neurodegenerative disorders, such as Parkinson's disease [15]. Indeed, cognitive training is a promising nonpharmacological treatment option for patients with dementia [16,17]. Although testing on patients was our goal, we wanted to test the protocol with healthy participants to ensure the feasibility of the study. Thus, we asked 20 computer science students at the University of Naples Parthenope to participate in the study.
In our experimental procedure, the ToL Test is administered by the Pepper robot. After an initial phase in which the robot provides participants with an explanation of the rules, the robot shows a test configuration to be used to verify understanding of the structure of the test by administering it subsequently. At this point, the user views the configuration to be replicated on Pepper's tablet (Figure 2), reproduces the one included, and continues in this way until all the configurations provided by the humanoid are exhausted (12 configurations for each participant). During the ToL test execution, the social robot occasionally offers move suggestions to users.
Namely, participants underwent three experimental phases:
• Baseline: a practice phase where requesting suggestions is not allowed, aiming to help the user become familiar with the test. • Training: an intermediate phase in which participants train to complete the test where it is allowed to request suggestions. • Assessment: a final phase where requesting suggestions is not allowed, and the user, with the experience gained from the previous two phases, should demonstrate greater proficiency in completing the test.
We needed an expert robot to provide useful and correct on-demand suggestions for our training phase. For this purpose, we designed a graph-based representation of the ToL test. Subsequently, we implemented a solving algorithm that used a breadth-first search (BFS) strategy to find the optimal solution for each ToL test. Starting from the initial ToL configuration representation, the BFS algorithm could find the optimal path to the representation of the final one. Although a BFS-based resolution of the ToL test has a linear time complexity on the number of the graph's edges and nodes, it was completely solvable since we considered a small input space with only three balls and three spots. Hence, when participants press the button to ask for a suggestion, the robot queries the algorithm to retrieve the first action to optimally reach the final game configuration (the goal) from the current one. the description of such an action. There was no limit about the number of suggestions that participants could ask; basically, they could ask for suggestions at each step of the tests.
Our hypothesis regarded both participants' performance and satisfaction. We aim to examine the degree of assistance participants require in completing the task. This holds paramount significance because, while we acknowledge that support might facilitate an immediate understanding of how to solve the test, we are cautious about the potential longterm impact on participants' performance. We want to avoid the risk of participants continually relying on suggestions, which may not lead to sustained improvement and even exacerbate their overall performance.
We found that the minority of participants asked the robot for help during training (45% on average), with an average number of suggestions asked of 1.53 (with 𝜎 = .717) among those who asked, and a maximum number of 3 suggestions asked per test. Hence, to investigate the difference between the tests in which participants asked the robot for help and those in which they never asked, we introduced in our analysis a binary categorical variable representing whether participants asked for help at least once during each test (the single configuration to solve) of the training.
To measure participants' improvements, we defined the delta between the number of moves needed to solve a ToL test and those performed by participants to solve it as their difference in absolute value. Considering this metric, participants' performance showed an improvement from the Baseline and Training phases (𝜇 = 1.89 with 𝜎 = 2.85 regarding the Baseline phase, and 𝜇 = 0.988 with 𝜎 = 2.73 regarding the Training phase). Through a two-way ANOVA test, we found an effect of the experimental phase on such measure of learning (𝐹 (2) = 3.57 with 𝑝 = .029). With a post-hoc Tukey correction, we found that such an improvement was statistically significant (𝑡 = 2.548 with 𝑝 = .03), while there was no statistical difference between performance in Baseline/Training and Assessment. Hence, we did not find significant improvements in Assessment on average.
Regarding the Training phase, the number of suggestions The closer the delta values are to zero, the higher the participants' performance since it reflects the number of moves they performed to solve the configuration in addition to those required. The 𝑥axis represents the trials in which participants asked for help at least once, or they never asked for help.
asked significantly impacted as a covariate on the number of moves they performed to solve the configurations (AN-COVA test 𝐹 (1) = 13.9 with 𝑝 < .001) and completion time (ANCOVA test 𝐹 (1) = 44.4 with 𝑝 < .001). We found significantly worse performance in participants who asked the robot for help compared to those who never requested (Independent Samples t-test, 𝑡(238) = −2.16 with 𝑝 = .032), as shown in Figure 3. Moreover, we observed that the number of suggestions asked positively correlated with participants completion time (Pearsons' 𝜌 = .406 with 𝑝 < .001), number of moves to solve the configurations (Pearsons' 𝜌 = .226 with 𝑝 < .001), and the configurations' difficulty (Pearsons' 𝜌 = .238 with 𝑝 < .001).
Since the robot suggestions were optimal, these latter correlations explain the worse performance of those who asked for help. We hypothesize that the participants who asked for the robot's suggestions also did not know the game or obtained worse performance. Thus, only bad players needed to ask the robot for help. This hypothesis is supported by the latter correlations, which highlight how the number of suggestions asked grew with the difficulty of the game (and vice versa) and that participants who asked the robot for help more were those who obtained worse performance in solving the game.
As a further measure of performance, we collected participants' completion times, which were defined as the time taken to complete each ToL test. We performed an Independent Samples t-test to compare participants' completion times depending on whether they asked the robot for help at least once during the tests or never asked. We found that assisted participants took more time to complete the ToL configurations than the other (𝑡(398) = −4.79 with 𝑝 < .001), as shown in Figure 4.
We found similar results regarding the number of moves participants performed to solve the ToL tests. We conducted an Independent Samples t-test on the number of moves participants performed to solve the configurations and found a significant effect on whether they asked for help at least once (𝑡(398) = −2.7 with 𝑝 = .008) with the assisted par- ticipants performing more moves than the others, as shown in Figure 5.
Participants who asked the robot for help are those who perceived its suggestions more useful as the correlation between the perceived usefulness of suggestions and the number of suggestions asked (Pearsons' 𝜌 = .492 with 𝑝 = .028). Moreover, by comparing participants' answers to the IOS test, we found that participants felt closer to the robot after having interacted with it, regardless of whether they asked it for help (Repeated Measures ANOVA, 𝐹 (1) = 5.716 with 𝑝 = .028). A post-hoc Tukey correction highlighted a significant difference between the pre-and post-experiment test answers (𝑡 = −2.39 with 𝑝 = .028).
Finally, we found that participants' personality dimensions (Big 5 and Sense of Agency tests) did not affect their behavior during the training and assessment phases. The two robot modalities did not elicit differences in participants perception of it (Godspeed test). We neither found differences between the groups regarding the perceived closeness to the robot (IOS test).
Despite the robot's availability to provide suggestions and its correct and complete knowledge of the task, a large number of participants (55%) never asked for help during the game. However, they could have gained from the robot's support, as their performance during the training was suboptimal. Interestingly, participants who asked for help had more difficulty with the task (as highlighted by an average longer completion time and a larger number of moves). Furthermore, they also recognized the usefulness of the suggestions at the end of the game. Nonetheless, the vast majority did not take advantage of the robot's help but preferred to play autonomously. This was probably due to the perception that the task was sufficiently easy to be addressed and by the desire to challenge themselves. Although this is certainly positive, it also raises the question of how to best provide robot support to maximize its utility. Our results demonstrate that perfect competence and periodic help offerings are insufficient unless the participants recognize their difficulty. Future work should focus on identifying novel strategies to counteract help aversion.
The work was supported by the research RESTART project (Robot Enhanced Social abilities based on Theory of mind for Acceptance of Robot in assistive Treatments) (CUP: I53D23003780001), funded by the MIUR with D.D. no.861 under the PNRR and by the European Union -Next Generation EU.