Frictional Design in Human-Robot Interaction: Delayed Movement in a Turn-Taking Game

Abstract

Our contribution concerns frictional design in human-social robot interaction. Exploring the use of frictional design in an empirical study, we argue designing friction can contribute to the cognitive empowerment of users. Our contribution concerns frictional design in human-social robot interaction. Exploring the use of friction in an empirical study, we argue the design of friction can be valuable for social robotics too. Frictional design aims to raise the users' awareness that AI and autonomous systems sometimes function to influence behaviour in specific ways that do not align with their preferences or values [1]. In decision support systems, manipulation may involve recommendations that encourage certain behaviour of the human that they would not show unless influenced by the AI. Whereas traditional seamless design emphasizes eficiency, useability, and ease-of-use and is directed at increasing acceptance and trust, Norman [2] cautioned that making systems too efortless to use might lead to complacency. Stimulating users to engage thoughtfully with recommendations and the underlying decision-making processes can prevent long-term deskilling and inhibition of upskilling due to indiscriminate ofloading of cognition and might equally promote acceptance and trust in new technologies. Park et al. [3] found that participants used the waiting time caused by reducing algorithm speed to reflect about the task and compare their own processes with the algorithm, and that more time generated better decisions, and Cabitza et al. [4] developed pro-hoc explanations to force medical doctors to develop their understanding and enable incidental learning. In our view, cognitive empowerment involves developing users' reasoning skills, critical awareness, and intrinsic motivation, preserving autonomy and self-reliance. We submit that the problems that motivate the use of frictional design equally occur in social robotics. The capacity for reciprocating emotion displays has increased the acceptance of social robots in society despite their error prone nature. Applying frictional design to human-robot interaction aims at leveraging humans to engage thoughtfully with social robots [1, 4]. Manipulation may involve provoking positive emotions for the robot that will increase the users' inclination to interact in spite of errors or reveal private information about themselves. Social robots are designed to assist humans on typically

Keywords

• eol>frictional design
• human-robot interaction
• robotic failures
• time delay
• timing
• reflection
• cognitive awareness
• empowerment
• social robot

-

human tasks, capable of influencing human behaviour directly by human means – physical action, emotion, nonverbal communication, and verbal language [ 5 ]. Despite HRI being framed as a collaboration between teammates, the fact that eficiency is the measure of success in HRI means human users must comply with the conditions imposed by the technology. To facilitate making sense of the behaviour their appearance is designed to be humanlike with head, face, arms, hands, and body. Then people think that because social robots have humanlike bodies, they function similarly to humans and have similar experiences, but this is wrong: similarity pertains to the surface only. Social robots ofer projection surfaces that make them deceptive as to their actual abilities, leading to overestimation of their capabilities. Thus Billing et al. [ 6 ] showed that assistant nurses held overly optimistic expectations about the capabilities of social robots. The design of time delay promises significant benefit to a low cost[ 7 ]. Delays are relatively common in Human-Robot Interaction, but usually investigated in terms of their negative efects, since humans quickly (approx. 2 seconds) become frustrated and lose engagement [ 8 ]. Here we are looking for a positive efect. We investigated participants’ reactions to delay in a hominoid robot’s performance of motor action (reach-to-place) in the context of playing Tic-Tac-Toe [ 9 ]. The experiment used the robot Epi developed at the Cognitive Robotics Lab at Lund [ 10 ]. Epi has a head, a LED mouth and eyes with pupils, and a torso with two arms and hands that can grasp objects. Sensory and motor processing is done at 40 Hz which yields smooth movement and human movements are reproduced with very high precision. The video recordings show a human and a robot facing each other across a table with a grid for playing Tic-Tac-Toe between them. Each participant played eight consecutive rounds of the game. To begin playing, the robot would grasp a ball with its right hand and place the ball in the grid. Once Epi had made its move, the human would grasp another ball and place it. This went on till one of the players had won or the grid was full. Having grasped the ball, Epi’s arm movement towards the grid sometimes would stall. The conditions of the experiment concerned variations in the robot’s arm movement when performing reach-to-place. In the ifrst condition, the entire movement was smooth and continuous, in the second condition the movement stopped just before placing the ball, the hand hovering above the grid with a delay of 4 seconds and then placing the ball, and in the third condition the movement came to a halt in the same position with a delay of 10 seconds before the arm started to move again and Epi ifnished its turn by placing the ball. Delays would occur several times in the second and third conditions. Epi showed two forms of behaviour that might cause friction and count as robotic failure. The one came across as clumsiness and concerned at times failing to grasp the ball properly due to insuficient calibration of the device. The other one was the delay of 10 seconds, a purposively programmed failure, that defies the distinction between interactional/social and technical failures. We expected it would draw the participant’s attention from playing the game to thinking about the process of playing. There are video recordings from the sessions (approx. 12 minutes) of five participants in the third condition of long delay. We analyzed the videos moment-by-moment looking for (1) behaviors that signal cognitive processing and cognitive engagement, and (2) emotional behavior such as frustration, excitement, interest or curiosity that relate to motivation. We carefully attended to how the interaction continuously develops, looking for interdependencies that explain the order of appearance of behaviour and their function or meaning in the context of the emerging interaction. In addition to eye gaze, body posture, dynamic movement, intention movement, self-touch to face and head, and oral (mouth) gesture typically provide significant information. The behaviors presented next constitute a first attempt at a coding schema for the empirical investigation of frictional design in human-robot interaction. We found the participants repeatedly showed the following cognitive and emotional behaviours (corresponding to the two categories mentioned above): • leaning back: disengaging (Fig.1). • gazing into the blue: cognitive processing: reflection. • leaning forward: re-engaging (Fig. 2). • touching face and head: self-regulation during increased cognitive processing. • rotating the ball in the hands before placing the ball: excitement, interest in the game, self-regulation. • hovering the hand over the board before placing the ballball cognitive processing, control of selected action. • increasing looks to board and hand-following: interest in co-player’s move. • fidgeting: arousal, excitement. • longer fixations while looking: intensity, increased attention, cognitive processing. • brief looks to face just before or at moment of co-player’s placing the ball: checking status of co-player, excitement.

Thus, we have reasons to believe that delays can lead to increased cognitive processing and encourage other scholars to look into these issues. We observed cognitive engagement after a delay e.g., people attend more to the board and hand movement, less to the face (social engagement). In addition, we observed improved playing e.g., trying out diferent strategies when playing or even winning the game.

Furthermore, we observed increased cognitive engagement after the long delays e.g., participants attended more to the board and hand movement, and comparably less to the face of the robot than before the delay (less social engagement). In addition, we observed improved playing e.g., trying out new strategies and using several strategies after each other as if comparing them, leading to better chances at winning the game. Apparently, the design of time delay caused the participants to increase their cognitive engagement in the interaction with the robot. One of the participants did not show any strong reactions to the delay and previously to the delay showed comparatively few signals of engagement in the game or of disliking it. While completing the session without complaining, the participant showed signs of freezing, or behavioral inhibition due to stress. Therefore, we do not take the data from that session into account here. Thus, we have reasons to believe that time delays can lead to increased cognitive processing in human-robot interaction and be a means for increased self-eficacy and competence of the users. We encourage looking into the function and efect of programmed delay for cognitive awareness and empowerment, and moreover, ask for the participants’ experiences of the long delay. We believe the number of delays during a session is critical. Frictional design runs the risk that the participants might get tired and familiarized over time, which might nullify positive efect. We also believe that when during a session the delays are introduced is critical for the outcome, depending on the status of the interaction. The wrong moment may have a detrimental efect. These issues certainly stand in need of further research. Mirning et al. [ 11 ] found that unexpected behavior, though making the robot seem more erroneous, also makes it appear more social to people interacting with it. This observation does not agree with our data. Beyond its social and emotional dimensions, we argue that robotic failure has implications for epistemic agency, particularly regarding participants’ sense of self-eficacy and perceived competence. In the research on human-robot interaction it is common to focus on the uncertainty caused by delay, assuming it will afect performance negatively. We were attracted by the transparency of the behaviour: Nobody would mistake it for regular interaction behaviour. Humans perceive longer delays than 1 second as frustrating. In most cases there are no pauses but turns are tight, sometimes overlap and agents are active simultaneously in diferent roles showing diferent and complementary behaviour. Hence, we expected 10 seconds to be blatant, indicating its own impossibility or out of placeness. We argue that letting the robot prove its competence and commitment to play in the first rounds, before the error occurs, amplifies the participants’ resilience to error and help them engage cognitively with the situation. In addition, Epi’s behaviour may have functioned to display thinking to co-present agents [ 12 ] , which would further the participants’ patience with failure. In conclusion, it appears that frictional design in HRI, in this case by introducing time delays into the interaction, can contribute to cognitive empowerment by developing the skills for interacting with a social robot. The waiting time helped the participants develop useful insights for the task.

Acknowledgments This work was partially supported by the Wallenberg AI, Autonomous Systems and Software Program - Humanities and Society (WASP-HS) funded by the Marianne and Marcus Wallenberg Foundation and the Marcus and Amalia Wallenberg Foundation.

References

[1] A. L. Cox , S. J. Gould , M. E. Cecchinato , I. Iacovides , I. Renfree , Design frictions for mindful interactions: The case for microboundaries , in: Proceedings of the 2016 CHI conference extended abstracts on human factors in computing systems , 2016 , pp. 1389 - 1397 .

[2] A. Norman Donald , The design of everyday things , MIT Press, 2013 .

[3] J. S. Park , R. Barber , A. Kirlik , K. Karahalios , A slow algorithm improves users' assessments of the algorithm's accuracy , Proceedings of the ACM on Human-Computer Interaction 3 ( 2019 ) 1 - 15 .

[4] F. Cabitza , C. Natali , L. Famiglini , A. Campagner , V. Caccavella , E. Gallazzi, Never tell me the odds: Investigating pro-hoc explanations in medical decision making , Artificial Intelligence in Medicine 150 ( 2024 ) 102819 .

[5] B. J. Ingar Brinck , Christian Balkenius, Making place for social norms in the design of human-robot interaction , in: Proceedings of RoboPhilosophy 2016/TRANSOR 2016 , 2016 , pp. 303 - 312 .

[6] E. Billing , J. Rosén , J. Lindblom , Expectations of robot technology in welfare, in: The second workshop on social robots in therapy and care in conjunction with the 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI 2019 ), Daegu, Korea, March 11 -14 2019 , 2019 .

[7] B. Frischmann , S. Benesch , Friction-in-design regulation as 21st century time, place, and manner restriction , Yale JL & Tech. 25 ( 2023 ) 376 .

[8] T. Kazuaki , O. Motoyuki , O. Natsuki, The hesitation of a robot: A delay in its motion increases learning eficiency and impresses humans as teachable , in: 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI) , IEEE, 2010 , pp. 189 - 190 .

[9] S. Stedtler , V. Fantasia , T. A. Tjøstheim , B. Johansson , I. Brinck, C. Balkenius , Is there really an efect of time delays on perceived fluency and social attributes between humans and social robots? a pilot study , in: Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction , 2024 , pp. 1013 - 1017 .

[10] B. Johansson , T. A. Tjøstheim , C. Balkenius , Epi: An open humanoid platform for developmental robotics , International Journal of Advanced Robotic Systems 17 ( 2020 ) 1729881420911498 .

[11] N. Mirnig , G. Stollnberger, M. Miksch , S. Stadler , M. Giuliani , M. Tscheligi , To err is robot: How humans assess and act toward an erroneous social robot , Frontiers in Robotics and AI 4 ( 2017 ) 21 .

[12] H. Pelikan , E. Hofstetter, Managing delays in human-robot interaction , ACM Transactions on Computer-Human Interaction 30 ( 2023 ) 1 - 42 .