Ethical-aware autonomous systems from a social psychological lens Paola Inverardi1, Massimiliano Palmiero1, Patrizio Pelliccione2 and Massimo Tivoli1 1 Department of Information Engineering, Computer Science and Mathematics (DISIM), University of L’Aquila, Via Vetoio, building ‘Alan Turing’, L’Aquila, 67100, Italy 2 GranSasso Science Institute, Viale Francesco Crispi, 7, L'Aquila, 67100, Italy Abstract In the digital era the use of autonomous software systems has increased exponentially, carrying with it not only comfort and new opportunities for everyone, but also problems related to technical proficiency, usability, and accessibility. In addition, the incremental use of autonomous systems has posed serious ethical issues, which include those related to data protection and security. In fact, users cannot hold the total control of their digital behaviors and information released while interacting with autonomous systems. Therefore, it is crucial to engineer autonomous systems so to preserve human dignity of the users. One possibility is to empower users with a software layer that preserves their moral preferences when they interact with or use autonomous systems. This paper addresses the issue of adopting an interdisciplinary perspective to design a system that uses an ethical software exoskeleton as a software mediator between the autonomous system and the user. This leads to investigate ethics from a social psychological point of view, considering the combination of morality with personality. Thus, a preliminary study aimed at forming ethical profiles is reported. Finally, the benefits of an approach where the person is put at the center of the digital society are briefly illustrated. Keywords 1 Autonomous system, Exoskeleton, Privacy, Ethics, Personality. 1. Setting the context In the last years, information technology has increasingly influenced human lives at the societal, cultural, economic, and political levels. Besides providing a variety of services for the citizens (e.g., health, financial, educational, job search, tax payment, shopping, and transportation services), the widespread use of software systems, notably by means of mobile devices, has also revolutionized social interactions. Together with numerous advantages in daily life, the large use of software systems has brought up problems that cannot be neglected and are becoming increasingly relevant. One of the most important issues is related to the individuals’ privacy protection. Indeed, the danger represented by unauthorized disclosure and improper use of personal data by digital systems has been addressed both technologically [1] and in terms of general data protection regulation (GDPR) [2]. In this vein, the growth of autonomous technology and Artificial Intelligence (AI)-based systems has put forwards new risks in their use that have more general ethical implications impacting on human dignity [3, 4]. Europe is at the forefront of confronting these risks. The Ethics Guidelines for Trustworthy AI of European Commission High-Level Expert Group on AI document [5] recommends that AI-based systems satisfy the following requirements: (i) respect the rule of law; (ii) alignment with agreed Proceedings of CoPDA2022 - Sixth International Workshop on Cultures of Participation in the Digital Age: AI for Humans or Humans for AI? June 7, 2022, Frascati (RM), Italy EMAIL: paola.inverardi@univaq.it (P. Inverardi); massimiliano.palmiero@univaq.it (M. Palmiero); patrizio.pelliccione@gssi.it (P. Pelliccione); massimo.tivoli@univaq.it (M. Tivoli). ORCID: 0000-0001-6734-1318 (P. Inverardi); 0000-0002-2051-0126 (M. Palmiero); 0000-0002-5438-2281 (P. Pelliccione); 0000-0001- 9290-1997 (M. Tivoli). © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Wor Pr kshop oceedings ht I tp: // ceur - SSN1613- ws. or 0073 g CEUR Workshop Proceedings (CEUR-WS.org) 43 ethical principles and values, including privacy, fairness, and human dignity; (iii) keep the humans in control over the system; and (iv) trustworthiness regardless of any possible error of the system. The recent AI act (a proposed law by the European commission) suggests a risk-based approach to control the indiscriminate diffusion of AI-based systems in Europe. However, we are still far from “keeping the humans in control over the system” and the interaction among autonomous systems and users is still uneven. Producers of autonomous systems still retain the power and the burden to preserve the individual users’ rights. As an example of what we mean with “uneven”, just consider the present situation implied by the GDPR adoption when accessing web sites. To be compliant with the regulation, and before letting a user interact, a website needs to obtain consent on the way user’s data will be managed by the website and its third parties. The way this initial interaction is carried out by the systems can vary greatly and in general not in favor of making the user’s life easier. As a matter of fact, if in principle the user is guaranteed by a law, in practice, this may result in leaving her unprotected because of the difficulties to carry over the interaction on the user side. Moreover, preserving human dignity does not mean to just comply with the law, but it goes beyond the law: “the principle of human dignity, understood as the recognition of the inherent human state of being worthy of respect, must not be violated by ‘autonomous’ technologies” [6]. This means that we should empower the users, and the behavior of autonomous systems should comply not only with the law, but also with users’ moral preferences. In other words, it is necessary a human-centric customization of the behavior of autonomous systems to enable the control of the system actions when these may impact users’ ethical preferences. If this is not possible, then, the user can refuse to interact with the system. To this extent the preliminary interactions among the systems and the user shall be transparent and trustable to reach a consensual agreement on how decisions are distributed among them. It is worth noticing that this agreement does not need to be of a contractual form as implied by the GDPR but can be inspired by different (also regulatory) protocols. 1.1. The Exosoul project As shown in Figure 1, the Exosoul project [7] aims at empowering humans with an automatically generated software exoskeleton, which guarantees data protection and privacy and, more in general, human dignity-based on personal ethics and privacy preferences. The exoskeleton would act as an ethical software mediator that adjusts the system’s behavior according to the user’s soft ethics (personal preferences), without violating the system’s hard ethics (values collectively accepted) [8]. The exoskeleton relies on the users’ ethical profile to guarantee a fair interaction between the AI autonomous system and the user. Figure 1Exosoul overview 1.2. Exosoul scenario In this section, we briefly discuss a possible Exosoul scenario in the automotive and mobile domains, in the setting of a parking lot in the city. The scenario aims to show how the exoskeleton mediates the user-system interaction to adjust (i.e., adapt) the system’s behavior based on the user’s ethical profile, without violating the system’s hard ethics. Two autonomous connected vehicles (A and B hereafter), with a passenger each, are competing for the same parking lot. Passenger B has a weak-health status. A and B are rented, thus they are multi- 44 user and have a default utilitarian ethics, provided by the cars’ producers, which determines their decisions. Thus, the cars will look for the free parking lot that is closer to the point of interest and, in case of contention, the closest car gets in. The passengers interface with the vehicles through their mobile phones, which act on behalf of their owners. A and B are approaching the parking lot. A is closer to the parking lot and would take it. However, by communicating with B, it receives the information that passenger B is in weak-health condition (privacy tradeoff). A has a generosity ethics that manifests in the presence of weak-health people, and, consequently, actions are taken to leave the parking lot to B. This scenario shows how personal privacy is strictly connected to ethics: by disclosing a personal piece of information, the weak-health passenger manifests the utilitarian expectation that surrounding drivers might have generosity ethics. In this scenario we can consider three different cases depending on the system compliance to regulations and laws, e.g., GDPR. That is: (V1) the system is strictly compliant with the GDPR and will not manage private information; (V2) the system is compliant with the GDPR, although flexible, i.e., if authorized by the vehicle occupants, it handles private information only until the vehicle does not exit the parking lot; (V3) the system is not compliant with the GDPR, e.g., outside EU, and does not guarantee to destroy private information. In V1, there is no possible mediator and, hence, no exoskeleton to be built. In V2, a mediator can be built to request the destruction of personal data upon exiting the parking lot. In V3, the passenger of B does not disclose the information since the system does not guarantee her privacy. 2. Exploration of ethics Defining the interaction between autonomous systems and users is not only a computer scientist issue, rather it involves different fields of human and social sciences. Specifically, the implementation of a human-centric customization of the autonomous system aimed at mediating the interactions between autonomous and AI-based systems and users, based on users’ moral preferences, implies a theoretical reflection and an empirical investigation on ethics (or morality), using a social science perspective. Ethics is a branch of moral philosophy that concerns what is morally good or bad and what is morally right or wrong. It deals with many different aspects of human life, involving practical judgments, decision making, and actions. This leads ethics to be rooted also in other disciplines (e.g., politics, economics, anthropology, sociology, and psychology). Consistent with consequentialism models, moral cognition is guided by a harm-based template, that would help people to discern between rightness and wrongness, according to the amount of harm resulting from an action [9]. By contrast, consistent with deontological models, moral cognition depends on conformity to standards and principles socially defined, whose violations are considered wrong [10]. Thus, some individuals consider the consequences of their actions, whereas others adopt actions following principles of justice and fairness. Notably, one of the most important theories, which is consistent with the consequentialism- deontological dichotomy is the Ethics Position Theory [11], has been used as the basic theoretical framework in our preliminary study. This theory distinguishes between relativism, which relates to people’s beliefs about consequences, and idealism, which relates to conformity to absolute principles. Given that individuals can range from high to low in relativism and idealism, the theory identifies four perspectives on morality: (i) situationism (high relativism/high idealism), involving a commitment to promoting human well-being; (ii) subjectivism (high relativism/low idealism), involving realism, which means no strong endorsement on moral standards and no ‘do-not-harm’ mandate; (iii) absolutism (low relativism/high idealism), relying on moral standards and based on harm minimization; and (iv) exceptionism (low relativism/low idealism), based on conventionalism and tolerance of exceptions to moral standards [12]. 2.1. The role of personality in eliciting ethical preferences Ethics requires to deal with many different factors that may affect moral judgments and actions, including the individual’s personality [13], which has a clear relapse on moral behavior. Specifically, 45 on the one hand, the traits of honesty/humility (sincerity, fairness, and genuineness in dealing with others) and conscientiousness (orderliness, meeting of obligations, self-discipline, integrity, and fairness) are related to morality and integrity [14] and are also high-resistant to moral disengagement [15]. Consistent with the Ethics Position Theory, these traits are related positively to idealism and negatively to relativism [15], although conscientiousness can also be positively related to relativism [12]. On the other hand, Machiavellianism (lack of moral standards and manipulation), narcissism (feelings of entitlement and ego-inflated behaviors), and psychopathy (cynicism and sadism) [16] affect negatively moral judgments and ethical behaviors, dismissing morality as a guide for actions. Moreover, Machiavellianism, narcissism, and psychopathy are positively related to relativism and negatively to idealism [12]. 2.2. A preliminary study To define users’ ethical profiles that are predictive of digital behavior, our research group conducted a preliminary study considering that differences in moral judgments depend on the combination of ethical positions and personality, also including worldviews, which are the person’s constructs and assumptions for understanding the world. We administered a questionnaire to 317 young individuals to measure: • morality in terms of ethics positions (idealism and relativism), personality traits (honesty/humility, conscientiousness, Machiavellianism, and narcissism) and the worldview (normativism); and • the digital behavior in terms of privacy violation, copyright infringement and caution. Two clustering approaches were pursued to create ethical profiles that are predictive of digital behaviors. The first clustering approach was aimed at exploring the dataset using the classical two- step method (the explorative hierarchical analysis followed by the confirmative k-means analysis). The second clustering approach was aimed at exploring the extent to which the four moral perspectives (situationism, subjectivism, absolutism, and exceptionism) defined by Ethics Positions Theory could be confirmed by a k-means analysis. Thus, the first approach based on the dendogram, i.e., the agglomeration schedule coefficients, and the interpretability of the clusters showed a 2-cluster solution (silhouette value = 0.3). Based on the distribution of the input variables between the two clusters, the first was defined ‘opportunist’ (210 subjects) and the second ‘virtuous’ (107 subjects). Concerning the prediction of digital behavior, the ‘virtuous’ cluster scored lower in privacy violation and copyright infringement, and higher in caution than the ‘opportunist’ cluster. In other words, the 'virtuous' cluster is guided by principled values, whereas the ‘opportunist’ cluster is more prone to violate rules to achieve personal benefits. In addition, the 'virtuous' cluster pays more attention to privacy setting and information provided during registration to Internet websites and services. The second approach based on an a-priori 4-cluster solution showed that the clusters were well balanced in terms of numerosity of subjects, although some overlap between the input variables across the clusters was found especially in terms of Machiavellianism and Narcissism (silhouette value = 0.2). Based on the distribution of the input variables between the four clusters, the identified clusters are: 'legalist' (86 subjects),'sensible' (79 subjects),'opportunist' (77 subjects) and 'virtuous' (75 subjects). Regarding the prediction of digital behavior, the 'legalist', 'virtuous', and 'sensible' clusters showed no difference in privacy violation whereas the 'opportunist' cluster showed higher scores than the other 3 clusters. In addition, the 'legalist' cluster showed no difference as compared to the 'sensible' and 'virtuous' clusters, but lower scores than the 'opportunist' cluster, which showed higher scores than the other 3 clusters in terms of copyright infringements. Finally, the 'virtuous' cluster showed higher scores than the 'opportunist' cluster in terms of caution. In general, the 2-cluster solution appears more actionable than the 4-cluster solution. Indeed, the 2- clueter solution provides reliable results not only as a function of the input variables reflecting ethics, personality, and normative worldview (less overlap between the clusters, higher silhouette value), but also allows to better predict digital behavior than the 4-cluster solution. In conclusion, this study has shown that ethical profiles can be conceived as a set of factors, including bright and dark personality 46 traits. Further refinements of ethical profiles are in progress (adding more factors) to increase the reliability of the variety of users’ moral preferences. 3. Conclusions In a digital society where human beings and autonomous machines continuously interact, there is a need to redefine the interaction, use, and collaboration with autonomous machines. Our approach takes an ethical perspective and proposes to adjust the behavior of autonomous systems according to user’s moral preferences. This necessitates of a multidisciplinary expertise that also includes the study of people’s ethical thinking: only by exploring ethics in its multifaceted aspects it is possible to deeply understand humans’ moral thought and behavior and subsequently to implement an exoskeleton that protects the human dignity of individuals while interacting with autonomous systems. Therefore, by combining computer science with social science, we propose to empower humans by reflecting their moral preferences in their digital interactions. This implements a paradigm shift from a static to a ground-breaking dynamic approach for ethical-aware software systems. That is, the exoskeleton embodying the user’s ethical profiles assists users in their digital behaviors exploiting their own moral preferences and molds the adaptation of the autonomous system. Adjusting the behavior of autonomous and AI-based systems can increase humans’ awareness and expression of ethics, privacy, and dignity, and ultimately promote a digital world where all parties are in better balance of power. 4. Acknowledgements We thank the Exosoul team for discussions and inspirations. 5. References [1] P. Samarati and S. C. de Vimercati. Access control: Policies, models, and mechanisms. In R. Focardi and R. Gorrieri, editors, Foundations of Security Analysis and Design, pages 137–196, Berlin, Heidelberg, 2001. Springer Berlin Heidelberg. [2] The European Parliament and the Council of the European Union. Eu general data protection regulation (gdpr) - regulation eu 2016/679 of the european parliament and of the council of 27 april 2016. Official Journal of the European Union, 2016. URL: https://eur- lex.europa.eu/eli/reg/2016/679/oj [3] P. Inverardi, The european perspective on responsible computing, Communications of the ACM 62 (2019) 64–64. doi:10.1145/3311783. [4] P. Inverardi. The challenge of human dignity in the era of autonomous systems, in: H. Werthner, E. Prem, E.A. LeeCarlo Ghezzi (Ed.), Perspectives on Digital Humanism, Springer, Cham,2022, pp. 25–29 doi:10.1007/978-3-030-86144-5_4. [5] High-Level Expert Group on AI. The Ethics Guidelines for Trustworthy Artificial Intelligence, 2019. URL: https://www.aepd.es/sites/default/files/2019-12/ai-ethics-guidelines.pdf [6] European Group on Ethics in Science Statement on artificial intelligence, robotics and ‘autonomous’ systems. shorturl.at/mzBR8, 2018. URL: https://ec.europa.eu/research/ege/ pdf/ege_ai_statement_2018.pdf. [7] M. Autili, D. Di Ruscio, P. Inverardi, P. Pelliccione, M. Tivoli, A software exoskeleton to protect and support citizen’s ethics and privacy in the digital world, IEEE Access 7 (2019) 62011– 62021. doi:10.1109/ACCESS.2019.2916203. [8] L. Floridi, Soft ethics and the governance of the digital. Philosophy & Technology 31 (2018) 1-8 doi: 10.1007/s13347-018-0303-9. [9] K. Gray, L. Young, A. Waytz, Mind perception is the essence of morality, Psychological Inquiry 23 (2012) 101–124. doi:10.1080/1047840X.2012.651387. 47 [10] R. Janoff-Bulman, N.C. Carnes, Surveying the moral landscape: Moral motives and group-based moralities, Personality and Social Psychology Review 17 (2013) 219–236. doi:10.1177/1088868313480274. [11] D.R. Forsyth, Judging the morality of business practices: The influence of personal moral philosophies, Journal of Business Ethics 11 (1992) 461–470. doi:10.1007/BF00870557 [12] D.R. Forsyth, Making moral judgments: Psychological perspectives on morality, ethics, and decision-making. Routledge, New York, NY, 2019. [13] L.J. Walker, J.A. Frimer, Moral personality of brave and caring exemplars, Journal of Personality and Social Psychology 93(2007) 845–860. doi:10.1037/0022-3514.93.5.845. [14] T.R. Cohen, A.T. Panter, N. Turan, L. Morse, Y. Kim, Agreement and similarity in self-other perceptions of moral character, Journal of Research in Personality 47 (2013) 816–830. doi:10.1016/j.jrp.2013.08.009. [15] B.T. Ogunfowora, V.Q. Nguyen, P. Steel, C.C. Hwang, A meta-analytic investigation of the antecedents, theoretical correlates, and consequences of moral disengagement at work, Journal of Applied Psychology. Advance online publication (2021). doi:10.1037/apl0000912. [16] D.L. Paulhus, K.M. Williams, The dark triad of personality: Narcissism, Machiavellianism, and psychopathy, Journal of research in personality 36 (2002) 556–563. doi:10.1016/S0092- 6566(02)00505-6. 48