Recently, the market of telepresence robots has a great growth especially given the increasing demand for improving patient care by providing a direct link with doctors and caregivers. However, driving the current commercial telepresence robots seems not trivial given the limited (or even absent) robot autonomy capabilities. This paper presents and evaluates a preliminary field-trial to enhance a commercial telepresence robot with limited computational resources with a semi-autonomous navigation system. The system is designed to facilitate teleoperation by relying on enhanced robot capabilities of contextualizing the environment and the human intervention via high-level directional commands. Participants were required to drive the robot in a real house from the living room to the bedroom passing through a corridor, open space and not target rooms. The robot moved in autonomy, the participants were asked to intervene by sending commands when they thought it was necessary to complete the task. The navigation task was accomplished with success thanks to the cooperation between the robot awareness and the operators. Results on the human evaluation in terms of trust and the workload suggest a low level of risk perception, efort, and frustration.
CEUR ceur-ws.org
Telepresence robots serve as service bodies for remote people, to allow them to experience
a location without physically being present [
However, current commercial telepresence robots require the continuous presence of a remote and heedful human operating the robot and in charge of taking deliberative choices about its behaviors. As a result, only manual or direct teleoperation is supported, where user commands are executed by the robot without any intermediary processing (e.g., no robot’s awareness, no robot’s autonomy). This means, for instance, that if the user gives a “turn right” command and a table is on the right, the robot may collide or, at best, if equipped with bumpers, stop as an emergency measure. Given the absence of robot’s autonomy and ability to contextualize the situation, successful interaction with telepresence robots strongly depends on two main factors: a) the level of familiarity of the secondary users in driving a robot properly, b) the prompt feedback reception during the communication to control the robotic platform coherently. In real-life applications, typically, the secondary users have no experience in driving robotic platforms. Therefore, a proper training phase is required especially taking into account the basic way of interfacing with the current commercial robots. Nevertheless, delivering improper (or involuntary) commands to telepresence robots is not so uncommon during interactions also with expert users, considering the delays due to a slow internet connection, as seen in recent experiments such as, e.g., [12, 13]. Finally, a third limitation arises from the lack of any level of autonomy and context awareness in the current commercial telepresence robots. Even in the best scenario, where a remote user has perfectly learned to control the robot and the internet connection is suficiently stable, the continuous focus of the remote operator is required to deliver the expected command. Therefore, teleoperating these robots one at a time can be demanding and frustrating as already reported in previous studies [14, 15, 16]. Controlling multiple telepresence robots simultaneously seems to be out of reach for a single remote human.
In this paper, we investigate the feasibility to enhance a commercial telepresence robot with limited computational resources with a semi-autonomous navigation to facilitate remote teleoperation in a domestic environment from only high-level directional commands delivered by the operator if needed. Indeed, we hypothesize the operator would like the robot to be endowed with a certain level of autonomy by perceiving the full control of the platform. Thus, we tested the proposed human-mediated autonomy system in a preliminary field-trial experimentation in a real home characterised by doorways, corridors and multiple rooms with the typical domestic furniture and with the available network connection (e.g., no network enhancement). We focus on analysing the robot’s motion, the human intervention and the human evaluation in terms of trust and the workload required during the interaction.
Over the years, manual teleoperation has been widely assessed also in the domestic scenario. For instance, many research projects, including GirafPlus 1,SYMPARTNER2, ExCITE3 have evaluated the robustness and the impact of using such platforms to monitor elderly people at home as integrated care services [17, 18, 19]. Recently, with the same purpose of visiting elderly people at home, Fiorini et al. [12] tested the technology readiness level of a commercial telepresence robot Double without any kind of environment contextualization and assistance and after training the formal caregiver to teleoperation in a lab setting. More advanced are the services proposed in the previous projects SERROGA4, RAMCIP5 and SYMPARTNER2, EnrichMe [20], that aim at modifying the robot’s navigation according to diferent situations including the person search, detection of a fallen person, reaching a goal, etc. Despite the advancements and the tests in the ecological scenarios, the proposed systems need advanced and powerful hardware that is not comparable with the commercial telepresence platform that we target in this work. Moreover, in their experimentations, the robot acts in autonomy without mediation from the operator. Our system is designed to be lighter and to run in a low-cost robot and, therefore, is more similar to the one proposed in [21] for the Giraf-X robot. Indeed, both systems rely on the ROS navigation stack at low-level for handling the robot’s motion and computing the robot’s trajectory. However, diferently from [ 21], herein, we exploit a human-mediated autonomy system where the robot’s navigation is influenced both by the user’s commands and the robot’s awareness.
In this work, we use Ohmni robot6 by OhmniLabs shown in Figure 1a, that we have already exploited in the role of a companion and coach for cognitive and physical exercises in [22]. It is a diferential mobile robot endowed with two RGB cameras (specifications: 2MP as resolution, 3.0 micron as pixel dimension, and 5865 x 3276 um as sensor’s dimension), placed on the top of robot’s tablet, and, the Navigation Camera under the robot’s neck. The streaming video from both cameras is used as visual feedback for the operator driving the robot. Furthermore, the Ohmni robot is equipped with a 2D lidar (i.e., Rplidar A1: 360 °, 12 m range, 1 °as angular resolution) for obstacle avoidance. Given the low-cost, the Ohmni robot is more afordable than other commercial robots. In addition, its limited computational resources make more challenging the technological transfer of advanced navigation services as the ones investigated in this work. Specifically, the Ohmni robot is equipped with Intel Atom X5 Z8350 as processor with 1.92 GHz, with 4 cores, DDR3L memory with 2 GB.
The commercial Ohmni robot was endowed with a semi-autonomous navigation system based on two main components in Figure 1a: (a) a policies-based system in charge of computing the robot’s target direction according to the context awareness from sensors’ data and the direction commands from the human; (b) the ROS navigation stack7 for Simultaneous Mapping and Localization (SLAM) and motion planning. The first component returns a subgoal (i.e., the arrow in Figure 1(b) which is the most probable position toward moving from the current robot’s context awareness. Once computed the subgoal, the navigation stack determines the best trajectory along which the robot can move in autonomy. The robot’s context awareness is modeled as probabilistic maps as proposed in [23] and previously tested in a lab setting with a brain-computer interface as user interface for delivering inputs. In this work, instead, the human delivers directional commands via keyboard when necessary. Given the higher interaction rate of the keyboard interaction, the subgoal is updated more frequently based on a ifxed frequency (i.e., 20 Hz) and every time the human delivers a command to ensure reactivity. in addition, in this work, the probabilities maps are modified based on: • the distances from the obstacles: the probability is inversely proportional to the closest distance of each detected obstacle to avoid collisions; • the human’s inputs: the probability exponentially increases in the quadrant associated with the command’s direction; • the robot’s orientation: the positions in front of the robot get higher probability than the other (e.g., exponential weight based on the angle between the robot and the position). As regards the computation of the robot’s trajectory, the combination of the Timed Elastic Band8 for the local planner [24] and Dijkstra’s algorithm for the global planner determines the speed commands for the robot. We chose Timed Elastic Band than Dynamic Window Approach planner in the navigation stack because it made the robot reach the goals smoothly in preparatory tests for the experiments, probably due to the diferential robot’s base.
Given the multiple displacements that arise in a domestic environment, the robot does not rely on a global map of the house, rather it exploits the local occupancy grids computed in real-time based on the 2D lidar detections (see Figure 4b).
Both modules of the semi-autonomous navigation, implemented inside the Robot Operating System (ROS) ecosystem9, were deployed on the Android based Ohmni-robot by using Docker10.
The user interface presents the following information (see Figure 1b) that is displayed on an external screen for the operator: a) the streaming video from both robot’s cameras (the frontal and the navigation areas), b) the surrounding map that is created in real-time while the robot is moving (i.e., the obstacles are marked in black), c) the subgoal indicating the direction towards the robot is navigating and d) the path plan returned by the ROS navigation stack the robot follows. The user can intervene by sending high-level directional commands, turn left and turn right, by pressing the corresponding arrows keys on the keyboard.
Three participants (age = 40 ± 16.46, 2 female) accepted to take part in the experiment. The experiment was conducted in accordance with the principles of the Declaration of Helsinki. Only one has previously teleoperated robots.
In this study, both quantitative metrics based on the data collected by the robot during the interaction and human evaluation based on a questionnaire administrated to the participants were assessed. Specifically, we evaluate: a) the robot’s trajectory, b) the time required to perform the experiment, c) the number and the kind of human intervention (e.g., high-level directional commands). As regards the questionnaire, the metrics investigated in this study were related to the trust and the workload in teleoperating the robot. With this purpose, we adapted the Human-Computer Trust Model (HCTM) from Gulati et al. [25] and the NASA Task Load Index (TLX) from Hart and Staveland [26]. The detailed questions are reported in Table 1. Finally, we asked via an open question if they would like to provide suggestions/qualitative feedback about 8http://wiki.ros.org/teb_local_planner 9https://www.ros.org/ 10https://www.docker.com/ the overall system. All the data were collected, processed, and stored according to the GDPR (Regulation EU 679/2016).
After signing the consent form, the system was introduced to the participants. The navigation task was verbally explained while walking in the domestic environment to make participants aware of the navigation area. Specifically, the participants were required to drive the robot from the living room to the bedroom at the end of a corridor (see Figure 1c). On the sides, there are multiple opening areas and other rooms. In our protocol, the robot did not know any information on the navigation task a priori (e.g., the final destination), therefore the intervention of the humans is required when the autonomous robot’s behavior is not in line with his/her will. We design the task experimented in this work, in order that the human intervention is required to reach the target room to test the semi-autonomous capabilities of the system. To get familiar with the system, the participants tried to navigate the robot in an initial familiarization run that was not recorded. Then, participants were asked to perform the navigation task per ifve runs (i.e., repetitions). In each run, the participants sat in a closed room and did not see the physical robot navigating. Participants only received the stream video cameras, the robot’s position in a real-time built map, and the subgoal computed by the system toward the robot was moving (see Section 3.3) as a feedback.
All the participants carried out the five runs with success by making the robot reach the destination without causing any damage. Figure 2 represents the human’s intervention in terms of turn left and turn right commands during the teleoperation along the robot’s trajectories. The time for delivering commands appears consistent with the required navigation task given the robot’s motion. Instead, the robot would prefer moving toward the open spaces as emerged from the trajectories (e.g., notice the curve on the left). Nevertheless, the robot’s trajectories were reactively corrected thank to the user’s input. Despite the tendency of moving far from obstacles, in this domestic scenario with very tight spaces, we notice that the robot’s trajectories appear close to obstacles especially in two main points that correspond to a passage through a door connecting the living room to the corridor and one passage through curtains in the middle of the corridor. In addition, during the experiment, we experienced that the curtains were not always detected by the lidar (i.e., the obstacle was not perceived by the robot), for which subject S1 encountered one collision with them in the fourth run. Given this result, in the future, the system should be strengthened by introducing the semantic information from the robot’s cameras. However, the robot was smoothly able to navigate up and down two diferent carpets in the corridors. As regards, the number of commands, we can notice diferent behaviors among the participants. Specifically, by analysing the number of commands, we achieve that S1 delivered on average 9 ± 2, S2 27.80 ± 10.47 and S3 13.20 ± 1.92 inputs. While the results of 1 and 3 are comparable, it seems that the subject S2 would like to have more control over the robot’s motion and trusts less in the robot’s capabilities of computing the subgoal according to the context by intervening even when it is not strictly necessary. However, this diferent behavior did not excessively afect the time to complete the task that is shown in Figure 3. On average, the navigation task was completed in 168.36 s ± 19.38 by S1, 217.34 s ± 54.00 by and 153.66 s ± 32.71 by S3. In addition, the percentage of the number of the left commands over the total was consistent among the participants: 35.50% ± 5.20 for S1, 33.87% ± 12.26 for S2 and 27.84% ± 8.74 for S3.
This section presents the results from the questionnaire administrated to the participants at the end of the experiment (i.e., after completing the five runs). The results per each subject are shown in Figure 4. The first five columns represent the scores of the sub-scales of the HumanComputer Trust Model (HCTM), while the last ones are related to the NASA Task Load Index (TLX) (see Section 4.2). The average score for each item in each sub-scale of the Human-Computer Trust Model (HCTM) is computed. It can notice that, overall, the level of risk perception was very low (i.e., S1: 1 ± 0.0, S2: 1.40 ± 0.55, S3: 1.40 ± 0.55). In terms of “collaboration”, the scores are medium/high by focusing on the benevolence (i.e., S1: 6 ± 1.0, S2: 4 ± 2.0, S3: 5.66 ± 1.53) and the reciprocity (i.e., S1: 5 ± 2.83, S2: 6.50 ± 0.71, S3: 6.50 ± 0.71). In terms of competency of the system, all the participants expressed a neutral opinion (S1, S2, S3: 4.67 ± 0.58). Finally, the score of the general trust are medium/high (i.e., S1: 5.67 ± 1.53, S2: 5.67 ± 1.15, S3: 5.67 ± 1.53).
As regards the NASA Task Load Index (TLX), the mental demand appears low (i.e., S1: 1.0, S2: 2.0, S3: 1.0). The feedback about the temporal demand is contrasting among the participants (i.e., S1: 1.0, S2: 3.0, S3: 1.0). In line with the successful accomplishment of the navigation task, the performance score was the maximum for all the subjects. Finally, the level of the efort (i.e., S1, S2, S3: 1.0) and the frustration seem low (i.e., S1, 2.0, S2: 1.0, S3: 1.0). 5.2.1. Qualitative feedback In this section, we briefly report the qualitative feedback provided by the participants. S1 revealed that experienced possible delays in the sending of the commands during the experimentation. This aspect could be due to the low-speed connection. Furthermore, S1 mentioned that would like to have additional feedback to know if the command is received by the robot and then implemented. Instead, S2 would prefer to increase the robot’s speed. In addition, S2 reported that would like to add further functionalities to the robot, for instance using it to transport objects among the rooms. However, despite her inexperience, S2 told that was able to orient herself well thanks to the combined visualization of the position of the robot on the map and the streaming video of the cameras.
This paper presents a pilot experiment in the field to investigate the feasibility of enhancing a commercial telepresence robot with few computational resources with semi-autonomous navigation and collect the preliminary feedback related to trust and the workload required during the teleoperation. Indeed, with respect to previous works, herein, we focus on a human-mediated autonomy system where the user intervention through high-level commands is required to ifnalize the navigation task. In this way, the human can keep the perception of having full control of the robot but with limited workload, since the robot moves in autonomy as default by handling the obstacle avoidance and planning its motion. From this point of view, despite the very small samples, we noticed diferent attitudes among the participants in terms of human intervention (S2 vs. S1 and S3). These results were confirmed by the questionnaire evaluation. The mental workload for S2 was higher than the other two subjects. However, we can notice that in the last runs the number of inputs decreases which might indicate a higher level of trust and familiarization with the system at the end. Interestingly it is also the diference in the temporal demand. Moreover, S2 qualitatively proposed to increase the robot’s speed. As regards the remaining scales, the results are quite consistent among the participants suggesting a low level of risk perception, efort, and frustration. Further tests are needed to validate the system with more participants, once modified based on this emerging qualitative feedback and to compare the performance with other semi-autonomous approaches.
This work was supported in part by the SI-Robotics: Social robotics for active and healthy ageing, Italian M.I.U.R., PON—Ricerca e Innovazione 2014-2020—G.A. ARS01 01120, in part by Cleverness (FOE 2020) under Grant DUS.AD016.133, and in part by FOCAAL Projects. [2] V. A. Newhart, M. Warschauer, L. Sender, Virtual inclusion via telepresence robots in the classroom: An exploratory case study, The International Journal of Technologies in Learning 23 (2016) 9–25. [3] P. Thompson, S. Chaivisit, Telepresence robots in the classroom, Journal of Educational
Technology Systems (2021) 00472395211034778. [4] G. Claudio, G. Luca, L. M. Luce, Interaction design for cultural heritage. a robotic cultural game for visiting the museum’s inaccessible areas., The Design Journal 20 (2017) S3925–S3934. [5] E. Chang, Museums for everyone: Experiments and probabilities in telepresence robots, in: Exploring Digital Technologies for Art-Based Special Education, Routledge, 2019, pp. 65–76. [6] A. Cesta, G. Cortellessa, F. Fracasso, A. Orlandini, M. Turno, User needs and preferences on AAL systems that support older adults and their carers, J. Ambient Intell. Smart Environ. 10 (2018) 49–70. URL: https://doi.org/10.3233/AIS-170471. doi:10.3233/AIS- 170471. [7] S. Laniel, D. Létourneau, F. Grondin, M. Labbé, F. Ferland, F. Michaud, Toward enhancing the autonomy of a telepresence mobile robot for remote home care assistance, Paladyn, Journal of Behavioral Robotics 12 (2021) 214–237. [8] K. Winterstein, L. Keller, K. Hufstadt, N. H. Müller, Acceptance of social and telepresence robot assistance in german households, in: International Conference on Human-Computer Interaction, Springer, 2021, pp. 326–339. [9] B. Isabet, M. Pino, M. Lewis, S. Benveniste, A.-S. Rigaud, Social telepresence robots: A narrative review of experiments involving older adults before and during the covid-19 pandemic, International Journal of Environmental Research and Public Health 18 (2021) 3597. [10] C. Esterwood, L. Robert, Robots and covid-19: Re-imagining human–robot collaborative work in terms of reducing risks to essential workers, Available at SSRN 3767609 (2021). [11] M. Niemelä, L. Van Aerschot, A. Tammela, I. Aaltonen, H. Lammi, Towards ethical guidelines of using telepresence robots in residential care, International Journal of Social Robotics 13 (2021) 431–439. [12] L. Fiorini, A. Sorrentino, M. Pistolesi, C. Becchimanzi, F. Tosi, F. Cavallo, Living with a telepresence robot: results from a field-trial, IEEE Robotics and Automation Letters 7 (2022) 5405–5412. [13] M. Wang, C. Pan, P. K. Ray, Technology entrepreneurship in developing countries: Role of telepresence robots in healthcare, IEEE Engineering Management Review 49 (2021) 20–26. [14] M. Pérez-Cruz, L. Parra-Anguita, C. López-Martínez, S. Moreno-Cámara, R. del PinoCasado, Burden and anxiety in family caregivers in the hospital that debut in caregiving, International Journal of Environmental Research and Public Health 16 (2019) 3977. [15] W. Moyle, C. Jones, M. Cooke, S. O’Dwyer, B. Sung, S. Drummond, Connecting the person with dementia and family: a feasibility study of a telepresence robot, BMC geriatrics 14 (2014) 1–11. [16] W.-Y. G. Louie, J. Li, T. Vaquero, G. Nejat, Socially assistive robots for seniors living in residential care homes: User requirements and impressions, Human-Robot Interactions: Principles, Technologies and Challenges (2015) 75–108. [17] S. Coradeschi, A. Cesta, G. Cortellessa, L. Coraci, J. Gonzalez, L. Karlsson, F. Furfari, A. Loutfi, A. Orlandini, F. Palumbo, et al., Girafplus: Combining social interaction and long term monitoring for promoting independent living, in: 2013 6th international conference on Human System Interactions (HSI), IEEE, 2013, pp. 578–585. [18] G. Cortellessa, F. Fracasso, A. Sorrentino, A. Orlandini, G. Bernardi, L. Coraci, R. De Benedictis, A. Cesta, Robin, a telepresence robot to support older users monitoring and social inclusion: development and evaluation, Telemedicine and e-Health 24 (2018) 145–154. [19] D. Fischinger, P. Einramhof, K. Papoutsakis, W. Wohlkinger, P. Mayer, P. Panek, S. Hofmann, T. Koertner, A. Weiss, A. Argyros, et al., Hobbit, a care robot supporting independent living at home: First prototype and lessons learned, Robotics and Autonomous Systems 75 (2016) 60–78. [20] S. Coşar, M. Fernandez-Carmona, R. Agrigoroaie, J. Pages, F. Ferland, F. Zhao, S. Yue, N. Bellotto, A. Tapus, Enrichme: Perception and interaction of an assistive robot for the elderly at home, International Journal of Social Robotics 12 (2020) 779–805. [21] M. Luperto, M. Romeo, J. Monroy, J. Renoux, A. Vuono, F.-A. Moreno, J. Gonzalez-Jimenez, N. Basilico, N. A. Borghese, User feedback and remote supervision for assisted living with mobile robots: A field study in long-term autonomy, Robotics and Autonomous Systems 155 (2022) 104170. [22] G. Beraldo, R. De Benedictis, A. Cesta, F. Fracasso, G. Cortellessa, Toward ai-enabled commercial telepresence robots to combine home care needs and afordability, IEEE Robotics and Automation Letters 8 (2023) 6691–6698. doi:10.1109/LRA.2023.3310396. [23] G. Beraldo, L. Tonin, J. d. R. Millán, E. Menegatti, Shared intelligence for robot teleoperation via bmi, IEEE Transactions on Human-Machine Systems 52 (2022) 400–409. doi:10.1109/ THMS.2021.3137035. [24] C. Rösmann, F. Hofmann, T. Bertram, Integrated online trajectory planning and optimization in distinctive topologies, Robotics and Autonomous Systems 88 (2017) 142–153. [25] S. Gulati, S. Sousa, D. Lamas, Design, development and evaluation of a human-computer trust scale, Behaviour & Information Technology 38 (2019) 1004–1015. [26] S. G. Hart, L. E. Staveland, Development of nasa-tlx (task load index): Results of empirical and theoretical research, in: Advances in psychology, volume 52, Elsevier, 1988, pp. 139–183.