In this work, we introduce the Evaluating tRust weaRing Of in Robots (ERROR) project. This ERROR project aims to address the lack of users' trust in robots, starting from the need for more persuading and personalized robotic mechanisms that can favour people's behavioural changes and compliance with instructions in order to improve their health, social and work lives. We aim to investigate how to deploy trustworthy and transparent robot behaviours in these contexts with particular attention to the techniques for mitigating peoples' trust after a loss of trust whether this was intentional (i.e., deceptive behaviours) or unintentional (i.e., erroneous and unexpected behaviours) for a balanced trustworthy and successful long-lasting interaction with people. We present our initial works in this direction, where participants (n. 37) played an assistive game with a deceptive robot endowed with Theory of Mind (ToM). We observed that a deceptive robot was less trusted by the participants, even though not all participants recognised the intentionality of the robot to deceive them.
In the ever-evolving landscape of robotics, a fundamental prerequisite for successful
integration of robots into assistive applications is their ability to facilitate behavioural changes and
encourage compliance with instructions, especially in medical and health contexts [
Factors influencing trust include personal diferences, task utility, prior experiences, reliability, and the nature of the relationship with the robot [8, 9, 10]. The integration of robots into society, mimicking human behaviour and cognitive capabilities, in contrast, also raises concerns about over-trust in these machines. Both under- and over-trust can hinder technology adoption, necessitating mechanisms for balanced trust [9].
Another technique explored to ensure compliance with robot requests and advice is deception in human-robot interaction. Controlled deception can prevent conflicts, reduce emotional distress, and enhance working relationships in contexts like education, healthcare, and rescue operations [11, 12, 13, 14]. At the same time, however, deception is also a topic of philosophical and psychological controversy [15]. Diferent forms of deception, from white lies to tactical and intentional deception, may influence trust in diverse ways. Deceptive behaviours, whether intentional or unintentional, can lead to misunderstandings, negative attributions, and a loss of confidence in robots, potentially eroding trust and discouraging their use. A mismatch between user expectations and robot behaviour can alter the perception of trust, jeopardizing the success of the interaction [16]. To this extent, it is essential to investigate both the mechanisms for recalibrating trust and preventive measures to avoid trust erosion. Ethical considerations surrounding the use of deception for behavioural change in human-robot interactions cannot be overlooked. Transparency, consent, and respect for user autonomy are paramount in responsibly employing deceptive mechanisms [17].
The ERROR project aims to provide a comprehensive assessment of deceptive mechanisms’ impact on trust in autonomous robots, emphasizing responsible use and alignment with existing regulations for the safe development of autonomous agents in human-centred environments [17]. To this extent, we started investigating whether the ability of mentalizing - which can be defined as a multi-modal system that allows people to naturally communicate and understand each other by inferring others’ intentions, desires, and beliefs [18] - may be used by a robot to foster trust and mitigate the potential issues connected to deception.
As a first step, we investigated how people’s perception of trust in a robot with the ability to mentalize varies when it has deceptive behaviours. Specifically, we decided to use verbal deception (i.e., lies), instead of non-verbal deception, and the robot’s perspective and people’s perception of the deception. In this initial work, we decided to focus on the definition of lies as “a false statement made by an individual which knows that the statement is not true” [19], but not to take advantage of the lie. We chose an assistive gaming scenario (i.e., Memory Game1), in which the robot does not compete against humans, and it used a Q-learning-based approach to provide advice relying on people’s beliefs and their intended game strategies. The robot’s ToM was shown to the players by generating move suggestions (e.g., the column or position of 1Open source GIT repository of the game https://github.com/yunkii/animal-memory-game the first or the second card) based on the robot’s knowledge of the game. The robot keeps track of the cards discovered by the player, the frequency and during which turn they were flipped. This info is used by the robot to generate a suggestion based on the current state of the game, possible beliefs and intentions of the participant. Participants were assigned to one of the two following conditions: 1) without a deceiving behaviour (ND-ToM condition) in which the robot provided assistance in the game by generating suggestions; and 2) with a deceiving behaviour (D-ToM condition) in which the robot suggested the wrong cards to the player. In order to not create an entirely faulty robot that would have never gained trust [20], the robot provided only 20% of wrong suggestions.
We recruited 37 people, aged between 18 and 59 years old (avg. 29, std. 11), and they identified themselves as female (43.3%) and male (56.7%). The majority of participants had no previous or close experience with robots. Participants were distributed as 20 participants in the ND-ToM condition, and 17 participants in the D-ToM condition.
We observed that the increase in wrong suggestions negatively afected people’s trust in the robot, as people did not accept the robot’s suggestion to choose a card ( (37) = − 0.483, < 0.001). We also asked participants to state whether they relied on the robot and had faith that the robot is able to succeed in performing even in situations in which it is untried. Participants had higher trust in the robot’s reliability and capabilities in the ND-ToM condition compared to those in D-ToM condition (respectively, (35) = 2.701, = 0.011, and (35) = 2.071, = 0.046).
Our first step has been to investigate whether and how people’s trust in a deceptive robot vary when they share the same awareness of the situational context, and the robot can mentalize people. Our next step is to identify diferent types of deception that an autonomous embodied agent, such as a robot, may provide while interacting with a human being, and evaluate how these types of deception afect a loss of people’s trust in robots based on people’s exposure to deceiving behaviours in tasks with diferent criticality. [4] M. Deutsch, Trust and suspicion, Journal of Conflict Resolution 2 (1958) 265–279. [5] J. D. Lee, K. A. See, Trust in automation: Designing for appropriate reliance, Human Factors 46 (2004) 50–80. URL: https://doi.org/10.1518/hfes.46.1.50_30392. doi:10.1518/hfes.46. 1.50\_30392. arXiv:https://doi.org/10.1518/hfes.46.1.5030392, pMID: 15151155. [6] D. McAllister, Afect- and cognition-based trust formations for interpersonal cooperation in organizations, Academy of Management Journal 38 (1995) 24–59. doi:10.2307/256727. [7] J. D. Lewis, A. Weigert, Trust as a Social Reality, Social Forces 63 (1985) 967–985. doi:10.
1093/sf/63.4.967. [8] A. Rossi, P. Holthaus, K. Dautenhahn, K. Koay, M. Walters, Getting to know pepper: Efects of people’s awareness of a robot’s capabilities on their trust in the robot, 2018, pp. 246–252. doi:10.1145/3284432.3284464. [9] A. Rossi, K. Dautenhahn, K. L. Koay, M. L. Walters, How the timing and magnitude of robot errors influence peoples’ trust of robots in an emergency scenario, in: A. Kheddar, E. Yoshida, S. S. Ge, K. Suzuki, J.-J. Cabibihan, F. Eyssel, H. He (Eds.), Social Robotics, Springer International Publishing, Cham, 2017, pp. 42–52. [10] P. Hancock, D. Billings, K. Schaefer, J. Chen, E. de Visser, R. Parasuraman, A meta-analysis of factors afecting trust in human-robot interaction, Human factors 53 (2011) 517–27. doi:10.1177/0018720811417254. [11] M. Coeckelbergh, How to describe and evaluate deception phenomena: Recasting the metaphysics, ethics, and politics of icts in terms of magic and performance and taking a relational and narrative turn, Ethics and Information Technology 20 (2018) 71–85. [12] H. S. Saetra, Social robot deception and the culture of trust, Paladyn, Journal of Behavioral Robotics 12 (2021) 276–286. URL: https://doi.org/10.1515/pjbr-2021-0021. doi:doi:10. 1515/pjbr-2021-0021. [13] E. Adar, D. Tan, J. Teevan, Benevolent deception in human computer interaction, 2013, pp.
1863–1872. doi:10.1145/2470654.2466246. [14] J. Shim, R. Arkin, The benefits of robot deception in search and rescue: Computational approach for deceptive action selection via case-based reasoning, 2015, pp. 1–8. doi:10. 1109/SSRR.2015.7443002. [15] G. Airenti, The cognitive bases of anthropomorphism: From relatedness to empathy,
International Journal of Social Robotics 7 (2015). doi:10.1007/s12369-014-0263-x. [16] M. H. Kneer, Can a robot lie? exploring the folk concept of lying as applied to artificial agents, Cognitive Science 45 (2021) e13032. doi:10.1111/cogs.13032. [17] J. Danaher, Robot betrayal: A guide to the ethics of robotic deception, Ethics and Inf.
Technol. 22 (2020) 117–128. doi:10.1007/s10676-019-09520-3. [18] S. Baron-Cohen, A. M. Leslie, U. Frith, Does the autistic child have a “theory of mind” ?, Cognition 21 (1985) 37–46. doi:https://doi.org/10.1016/0010-0277(85)90022-8. [19] T. L. Carson, The definition of lying, Noûs 40 (2006) 284–306. URL: http://www.jstor.org/ stable/3506133. [20] A. Rossi, K. Dautenhahn, K. L. Koay, M. L. Walters, How the timing and magnitude of robot errors influence peoples’ trust of robots in an emergency scenario, in: A. Kheddar, E. Yoshida, S. S. Ge, K. Suzuki, J.-J. Cabibihan, F. Eyssel, H. He (Eds.), Social Robotics, Springer International Publishing, Cham, 2017, pp. 42–52.