Acceptability of Symbiotic Artificial Intelligence: Highlights from the FAIR project Francesca Alessandra Lisi1,∗ , Antonio Carnevale2 , Abeer Dyoub1 , Antonio Lombardi2 , Piero Marra3 and Lorenzo Pulito4 1 University of Bari Aldo Moro, DiB Dept., via E. Orabona 4, Bari, 70125, Italy 2 University of Bari Aldo Moro, DIRIUM Dept., Piazza Umberto I, Bari, 70121, Italy 3 University of Bari Aldo Moro, LAW Dept., Piazza C. Battisti 1, Bari, 70121, Italy 4 University of Bari Aldo Moro, DJSGE Dept., Via Duomo 259, Taranto, 74123, Italy Abstract In this work we report the highlights of the work done at the University of Bari within the FAIR project and concerning the acceptability of Symbiotic Artificial Intelligence. Keywords Symbiotic AI, AI Ethics, Trustworthy AI, Philosophical foundations of AI 1. Introduction tive partnership between humans and machines within a broader social and technological context, where the focus The notion of symbiosis originated in the 19th century is not on a substantial peer-to-peer relationship but on to indicate a relationship between two taxonomically integrating technology into human-centric processes and separate life forms that nevertheless give rise to a sin- systems. In this context, symbiosis involves humans and gle organism. Life forms in a symbiotic relationship are machines working together as a cohesive unit, each play- not isolated but coexist in ways that are more or less ing a specific role and contributing to the team’s overall essential to their survival and development. The first performance. On one hand, humans provide the cogni- to advocate a symbiosis between humans and machines tive and emotional capabilities necessary for creativity, was J.C.R Licklider in 1960 [1]. In his view, this kind empathy, ethical decision-making, and adaptability. On of symbiosis would allow the computer to become an the other hand, machines offer computational power, data active part of the thinking process that leads to resolving processing, and automation capabilities that can handle technical problems and not just an executor of solutions repetitive and data-intensive tasks efficiently. thought up beforehand. Licklider was mainly thinking When applied to AI, the concept of symbiosis becomes of human-computer interfaces that would allow greater more complex, posing a whole series of foundational real-time collaboration and shorten the distance between questions. Addressing these questions is one of the goals human and machine language. He was pointing to a of the research done by the University of Bari (together road that has since been successfully travelled, bringing with INFN) within the project Future AI Research (FAIR). us to the so-called Symbiotic Artificial Intelligence (SAI). In particular, the acceptability of SAI is the subject of Human-AI symbiosis promises to boost human-machine research for our investigation within a dedicated work collaboration and socio-technical teaming, with mutually package (WP 6.5) of FAIR. Acceptability involves value beneficial relationships, by augmenting (and valuing) hu- alignment between AI and humans. It is related, e.g., to man cognitive abilities rather than replacing them [2]. In understanding AI decisions, the algorithmic bias, the re- particular, socio-technical teaming refers to the collabora- spect of privacy policies for data collected by AI systems, the struggle between security ensured by AI systems and Ital-IA 2024: 4th National Conference on Artificial Intelligence, orga- fundamental freedoms, the mitigation of possible safety nized by CINI, May 29-30, 2024, Naples, Italy and health risks. In FAIR, studies on the acceptability ∗ Corresponding author. Envelope-Open francesca.lisi@uniba.it (F. A. Lisi); antonio.carnevale@uniba.it of SAI adopt an interdisciplinary approach involving re- (A. Carnevale); abeer.dyoub@uniba.it (A. Dyoub); searchers in AI, Law, and Philosophy. antonio.lombardi@uniba.it (A. Lombardi); piero.marra@uniba.it In this paper, we briefly report the main achievements (P. Marra); lorenzo.pulito@uniba.it (L. Pulito) of our research on ethical and legal acceptability of SAI in Orcid 0000-0001-5414-5844 (F. A. Lisi); 0000-0003-2538-5579 the 1st year of the project (Sections 2-3) and outline the (A. Carnevale); 0000-0003-0329-2419 (A. Dyoub); 0000-0003-1803-5423 (A. Lombardi); 0009-0003-6365-2129 steps needed to go from general principles to operational (P. Marra); 0009-0000-3979-8716 (L. Pulito) definitions for ethical acceptability (Section 4). Section 5 © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). concludes the paper with final remarks. CEUR Workshop Proceedings http://ceur-ws.org ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org) CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings 2. Ethical acceptability of SAI but are an integral part of the same evolutionary process and are responsible for it. We think that this approach is The philosophical approach to AI is contributing to the keeping with the provisions, ex multis, of memorandum debate on the identification and analysis of the ethical no. 38 of the Proposal for a EU Reg. on artificial intelli- implications of algorithms. We have continued the inves- gence. A procedural condition ensures the fairness and tigation aiming to build the proposal of a methodological transparency of decision-making and it allows recipients framework grounded in process-oriented evaluations to to understand and respect the decision itself. Indeed, in assess the human-centricity and acceptability of SAIs law, it is not sufficient the content of the decision, but together with their societal benefit. also its enforcement. Thus, the effectiveness remains a The research carried out concerned two different sci- constitutive element of legality [9]. entific lines: Furthermore, some legal issues raised by the interac- Questioning the notion of “symbiosis” in SAI sys- tion between humans and AI were addressed in some tems. The research focused mainly on the meaning of areas of law (such as those that most require judgments “symbiosis” and its applicability to AI [3]. To this end, of predictive type, like the assessment of dangerousness preliminary research has been carried out on the trans- aimed, for example, at commensurate punishment and/or formation of the concept of intelligence in the history granting alternative measures). It has been so possible of ideas [4]. In several internal meetings, the notion to observe and identify some essential conditions that of symbiosis was explored both from a biological and should be taken into account in designing AI systems in phenomenological point of view, with reference to the this field, necessary to promote the symbiosis between key recent AI-driven technological developments (AI and humans and AI as well as to improve the trustworthiness, drones, AI and robotics, LLM, ML, etc.). fairness and efficiency of the interaction (for example, Assessing the ethical impact of SAI in terms of enriching the methods of responding to the crime in acceptability and human-centricity. Defining the fun- compliance with the fundamental principles of propor- damental conceptual stages of a methodology for eval- tionality and dignity of the person, realizing the requests uating AI systems involves comparing and studying a for individualization of the punishment) [10]. series of international regulatory frameworks – inter alia Finally, we would like to mention that, the European AI HLEG, Ethics Guidelines for Trustworthy AI (2018- legal framework for AI gives minimal consideration to 19). We have outlined a model with different fundamen- regulating AI based technologies where there is a recipro- tal steps: (a) onto-epistemic foundation of the method; cate relationship between human and machine (symbio- (b) screening; (c) risk evaluation; (d) impact assessment. sis). The research field of symbiotic AI is technologically Now, we need to work within each step to refine proce- challenging. In [11], we have undertaken a foundational dures and metrics further. study with the aim of conceptualizing and designing a The efforts in this direction have led to a joint paper comprehensive symbiotic approach to AI, with the goal of presented at the BEWARE workshop organized in Rome producing fair, legitimate, and effective outcomes while within the 22nd International Conference of the Italian ensuring their ethical and legal acceptability. This theo- Association for Artificial Intelligence (AI*IA 2023) [5], an retical research is expected to influence the development article accepted for publication in the journal Intelligenza of Symbiotic AI systems and technological governance Artificiale [6], and different book chapters in the final through model assessment. stages of publication [7], [8]. 4. Towards Operational 3. Legal acceptability of SAI Definitions of Ethical In line with the ethical and philosophical considerations Acceptability of SAI on symbiosis, moving from the perspective of human- machine interaction to a procedural model of construc- The ethical implications of Human-AI symbiosis are mul- tion and assessment of SAI decision, within a legal tifaceted and complex. Thus, it has become increasingly methodology theory we have identified the first legal paramount to take in consideration the ethical issues sur- pragmatic conditions of algorithmic decision-making, rounding SAI development, deployment, and impact. The such as that of the significant human control, a notion concept of ‘SAI Ethics’ offers a nuanced perspective that borrowed from the international debate within the UN emphasizes the harmonious coexistence and collabora- on autonomous weapons. In this way, symbiosis trans- tion between humans and AI systems. Operationalizing lates also a techno-procedural legal principle capable of SAI Ethics involves translating abstract ethical princi- formalizing a human-centric value where persons do not ples and values into concrete guidelines and practices remain behind technological development and society that govern every stage of the AI lifecycle, including data collection, algorithm design, model training, evalua- the integration of ethical principles into the design and tion, and deployment [12]. It requires a multidisciplinary development of AI algorithms and models. This means approach, involving collaboration between computer sci- translating ethical principles, values, and guidelines into entists, ethicists, policymakers, and other stakeholders to actionable and measurable practices or procedures. We ensure their alignment with societal values and human need to define specific rules, standards, or protocols that well-being, and to foster harmony and mutual benefit guide the behavior and decision-making in ethical dilem- between humans and machines. mas or concrete situations [16, 17]. Moreover, SAI Ethics emphasizes the importance of continuous learning and 4.1. Operationalizing SAI Ethics adaptation. As AI technologies evolve and their societal impact unfolds, ethical standards and norms must evolve From a practical perspective, operationalizing SAI Ethics in tandem [18, 19]. This requires interdisciplinary re- requires the establishment of governance frameworks, search, ethical reflection, and stakeholder engagement standards, and regulations to govern the responsible de- to address emerging challenges and dilemmas. velopment, deployment, and use of AI technologies. This includes the development of ethical guidelines, codes 4.2. Building a Computational Model of of conduct, and best practices to guide AI practitioners and organizations in navigating ethical dilemmas and SAI Ethics decision-making processes [13]. These tools should be Ethical Principles are abstract rules intended for guiding domain specific. Moreover, fostering interdisciplinary ethical decision making and judgement. There are a vari- collaboration and stakeholder engagement is essential ety of techniques used for technical implementation of to ensure that ethical considerations are adequately ad- ethical principles. In the previous literature of machine dressed and that AI technologies serve the broader soci- ethics, ethical principles are integrated into machines etal interest. in a top-down, bottom-up, or hybrid architectures (see One key aspect of operationalizing SAI Ethics is the [20] for a survey). However, so far, no model seems to development of robust frameworks and methodologies satisfy ethical judgement and decision making needs for for ethical risk assessment and mitigation. This involves an acceptable and responsible AI system. Approaches identifying potential ethical risks associated with AI sys- to encode principles into a format that computers can tems, such as bias, discrimination, privacy violations, and understand include logical reasoning, probabilistic rea- unintended consequences, and implementing strategies soning, learning, optimisation, and case-based reasoning to address these risks proactively [14]. Thus, it is im- [21]. portant to design algorithms and systems that are trans- We argue that it is impossible to build a ’general ethical parent, interpretable, and accountable, enabling stake- AI’, i.e., a machine that is generally ethical, a machine that holders to understand how AI decisions are made and to can reason and take ethical decisions in any domain and detect and rectify ethical issues when they arise. Here in every context. We believe that we need to concentrate we would like to highlight the role of logic program- on building domain-based ethical machines, i.e., machines ming for designing such models [15]. Additionally, op- that are able of ethical reasoning and decision making in erationalizing SAI Ethics requires ongoing monitoring any context and situation in a specific domain, which is, and evaluation of AI systems in real-world contexts to any way, still a very challenging task. Considering the ensure that they continue to operate ethically and re- purpose and the specific domain for which the AI system sponsibly throughout their lifecycle. From a technical is developed, developers should consider codes of ethics perspective, operationalization should focus on human- and conduct of the domain (domain ethics, e.g. medical centricity through the development of AI systems that ethics) as a guiding framework. Furthermore, the key are transparent, interpretable, and accountable. This en- aspects of SAI, such as the collaborative and cooperative tails implementing mechanisms for explainability and nature between human and machine, the human-centric interpretability, allowing users to understand how AI approach, the mutual benefit, the adaptability and respon- algorithms make decisions and providing insights into siveness of SAI, and the interdisciplinary perspective, their underlying processes. Techniques such as model in- should be taken in consideration in the design decisions terpretability, transparency tools, and algorithmic audits to be taken by the developers. enable stakeholders to scrutinize AI systems and iden- To build a computational model of domain ethics to tify potential biases, errors, or unintended consequences. be integrated into the AI system; the ethical principles of Additionally, ensuring the robustness and reliability of the domain should be operationalized. The operational- AI systems through rigorous testing, validation, and ver- ization task should be carried out involving all stake- ification processes is essential to minimize the risk of holders and domain ethical experts. Developers should harmful outcomes and instil confidence in their use. also decide on the architecture to adopt for integrating Furthermore, operationalizing SAI Ethics necessitates the ethical principles. Being clear about which princi- ple is being used will help designers to further specify revise a previously learned rule and present it to the hu- what inputs are necessary for their application, which in man. Through a collaborative dialogue, The human can turn will improve the ethical reasoning capabilities and correct the ethical behavior of the machine, but also the explainability of how decisions have been made [22]. machine can automatically demonstrate to the humans However, defining principles in an intentional manner their errors in reasoning. In this way both will learn so that they may be applied in a deductive manner, is and improve their reasoning capabilities (mutual benefit). often challenging and, in many cases, appears to be an This adaptability aspect will be tested and evaluated in impossible task. The issue lies in the gap between ab- our experiments. stract, open-textured principles and tangible, concrete facts. The abstract principles should be operationalized by linking them to the facts. When ethical experts jus- 5. Conclusions and Future Work tify their conclusions in particular cases, they frequently In this work, we reported on ongoing work in the Work- connect ethical principles directly to the specific facts of Package 6.5 of the project FAIR. A model of ethical ac- those cases. Essentially, these established connections ceptability of SAI was outlined. Many legal issues raised between ethical principles and relevant facts serve as by SAI systems were addressed. Currently, we are con- operational (concrete) definitions of the principles. The centrating on SAI ethics operationalization. Next, we experts operationalize the abstract principles by tying will work on the operationalization of legal aspects in them directly to the factual context. SAI by the development of a framework for embedding We are going to investigate, computationally, the pos- the considerations of legal issues in SAI, then on real- sibility of operationalizing abstract ethical principles by izing a computational model of legal reasoning for our inducing practical rules for ethical judgement and deci- SAI system to be ultimately integrated in the SAI system sion making in SAI systems from real-life interactions be- together with the ethical model. tween human and machine in different domains [19, 23]. By operationalizing SAI Ethics and legal issues, we These rules evolve overtime through the interaction be- can foster a collaborative and mutually beneficial rela- tween human and machine which is an important aspect tionship between humans and AI systems, promoting to SAI ethics. SAI recognizes the dynamic nature of responsible and trustworthy AI development for the ben- human-AI interactions and the need for AI systems to efit of the society. This requires a multifaceted approach adapt and respond to human preferences, values, and that integrates technical, organizational, regulatory, and feedback overtime. To achieve this, we are going to con- societal perspectives. sider different domains as case studies, collect and ana- A socio-technical approach to SAI systems develop- lyze a large set of domain ethics cases and build a com- ment will be adopted which leads to an increased ac- putational model employing different operationalization ceptability of these systems [24]. To capture the socio- techniques. Then, we are planning to carry out exper- technical complexity we are planning to adopt Multi- iments to test our hypothesis that the computational Agent Systems (MAS) for modelling the SAI system at model will accurately classify actions as ethical or uneth- hand [25]. The ethical and legal components in the sys- ical. The model will be developed using a foundational tem will be implemented as a MAS, which will act as set of cases that will be collected for this purpose. The an ethical and legal over-layer in the overall decision system performance will be evaluated using quantitative making process. A starting point might be the MAS pro- measures like precision and recall. totype presented in [26, 27] for the ethical evaluation and An important aspect, mentioned above, is the model monitoring of dialogue systems. adaptability overtime. In the context of SAI systems, hu- Finally, since a human-centric approach is central man and machine (as agents) work as a team, collaborate to SAI, transparency and explainability are key require- and learn from each other, evolve together. The machine ments for establishing trust in SAI systems which leads to (as well as the human) will learn concrete ethical rules acceptability. We would like to emphasize the the promi- from interaction with humans, the machine will apply nent role of computational logic in the development of the previously learned ethical rules on concrete cases, the computational model of ethical and legal acceptabil- will also revise and update the previously learned rules ity of SAI. Logic Programming (LP) has a great potential if needed. Here, it is important to emphasize the col- for developing such perspective ethical and legal SAI laborative aspect of SAI in revising and correcting the systems, as in fact logic rules are easily comprehensible ethical behavior overtime by both the human and the by humans. Furthermore, LP is able to model causality, machine. In fact, this task is, in reality, a collaborative which is crucial for ethical and legal decision making task, the machine will extract the case facts (the facts of [15]. the real-life case at hand), present them to the human, the human will provide an ethical judgment of the case at hand. Then the machine will learn a new rule and/or Acknowledgments ing, Cham, 2022, pp. 1–19. doi:10.1007/978- 3- 3 19- 31739- 7_142- 1 . This work was partially supported by the project FAIR - [10] L. Pulito, Algoritmi predittivi e valutazione della Future AI Research (PE00000013), under the NRRP MUR pericolosità, L’Ircocervo (2024). Invited essay, sub- program funded by the NextGenerationEU. mitted. [11] P. Marra, L. Pulito, A. Carnevale, F. Lisi, A. Lom- bardi, A. Dyoub, A procedural idea of decision- References making in the context of symbiotic ai, in: Pro- [1] J. C. R. Licklider, Man-computer symbiosis, IRE ceedings of the 1st International Workshop on De- Transactions on Human Factors in Electronics HFE- signing and Building Hybrid Human-AI Systems, 1 (1960) 4–11. doi:10.1109/THFE2.1960.4503259 . co-located with 17th International Conference on [2] S. S. Grigsby, Artificial intelligence for advanced Advanced Visual Interfaces (AVI 2024), Arenzano human-machine symbiosis, in: D. Schmorrow, (Genoa), Italy, June 3rd, 2024., CEUR Workshop Pro- C. Fidopiastis (Eds.), Augmented Cognition: In- ceedings, 2024. URL: https://synergy.trx.li/ceur-w telligent Technologies, volume 10915 of Lecture s/paper9.pdf. Notes in Computer Science, Springer, Cham, 2018. [12] J. Morley, L. Kinsey, A. Elhalal, F. Garcia, M. Ziosi, doi:10.1007/978- 3- 319- 91470- 1_22 . L. Floridi, Operationalising AI ethics: barriers, en- [3] A. Carnevale, Condizione e struttura del nos- ablers and next steps, AI Soc. 38 (2023) 411–423. tro rapporto con le macchine. dieci proposizioni URL: https://doi.org/10.1007/s00146-021-01308-8. per una filosofia critica dell’intelligenza artificiale doi:10.1007/S00146- 021- 01308- 8 . antropocentrica, in: S. Barone, et al. (Eds.), L’uomo [13] J. Mökander, L. Floridi, Operationalising AI gov- animale tecnologico, Sciascia Editore, Caltanissetta- ernance through ethics-based auditing: an indus- Rome, 2024. Invited chapter, accepted, in publica- try case study, AI Ethics 3 (2023) 451–468. URL: tion. https://doi.org/10.1007/s43681- 022- 00171-7. [4] A. Lombardi, L’origine dell’io. il “mistero” dell’intel- doi:10.1007/S43681- 022- 00171- 7 . ligenza da Darwin al riduzionismo contemporaneo, [14] C. Novelli, F. Casolari, A. Rotolo, M. Taddeo, Studium/Ricerca 119 (2023) 651–688. L. Floridi, AI risk assessment: A scenario-based, [5] A. Carnevale, A. Lombardi, F. A. Lisi, Exploring eth- proportional methodology for the AI act, Digit. Soc. ical and conceptual foundations of human-centred 3 (2024) 13. URL: https://doi.org/10.1007/s44206-0 symbiosis with artificial intelligence, in: G. Boella, 24-00095-1. doi:10.1007/S44206- 024- 00095- 1 . et al. (Eds.), Proceedings of the 2nd Workshop on [15] A. Dyoub, S. Costantini, F. A. Lisi, Logic pro- Bias, Ethical AI, Explainability and the role of Logic gramming and machine ethics, in: Proceedings and Logic Programming co-located with the 22nd 36th International Conference on Logic Program- International Conference of the Italian Association ming (Technical Communications), ICLP Techni- for Artificial Intelligence (AI*IA 2023), volume 3615 cal Communications 2020, (Technical Communica- of CEUR Workshop Proceedings, 2023, pp. 30–43. tions) UNICAL, Rende (CS), Italy, 18-24th Septem- URL: https://ceur-ws.org/Vol-3615/paper3.pdf. ber 2020, volume 325 of EPTCS, 2020, pp. 6–17. [6] A. Carnevale, A. Lombardi, F. A. Lisi, A human- doi:10.4204/EPTCS.325.6 . centred approach to symbiotic AI: Questioning the [16] A. Dyoub, S. Costantini, F. A. Lisi, Learning answer ethical and conceptual foundation, Intelligenza set programming rules for ethical machines, in: Artificiale (2024). Invited paper, in publication. A. Casagrande, E. G. Omodeo (Eds.), Proceedings [7] A. Carnevale, Assessing the impacts of symbiotic of the 34th Italian Conference on Computational AI (SAI) on individual and societal well-being, in: Logic, Trieste, Italy, June 19-21, 2019, volume 2396 H. Webb, et al. (Eds.), AI Impact Assessment: meth- of CEUR Workshop Proceedings, CEUR-WS.org, 2019, ods and practices, Oxford University Press, 2024. pp. 300–315. URL: http://ceur-ws.org/Vol-2396/pa Invited chapter, accepted, in publication. per14.pdf. [8] C. Falchi Delgado, M. T. Ferretti, A. Carnevale, Be- [17] A. Dyoub, S. Costantini, F. A. Lisi, Towards an yond one-size-fits-all: Precision medicine and novel ILP application in machine ethics, in: Inductive technologies for sex and gender-inclusive covid-19 Logic Programming - 29th International Confer- pandemic management, in: D. Cirillo, et al. (Eds.), ence, ILP 2019, Plovdiv, Bulgaria, September 3-5, Innovating Health against Future Pandemics, Else- 2019, Proceedings, volume 11770 of Lecture Notes vier, 2024. Invited chapter, accepted, in publication. in Computer Science, Springer, Netherlands, 2019, [9] P. Marra, I. Galatola, Effectiveness as Threat to Con- pp. 26–35. doi:10.1007/978- 3- 030- 49210- 6 . stitutional Systems, Springer International Publish- [18] A. Dyoub, S. Costantini, I. Letteri, Care robots learn- ing rules of ethical behavior under the supervision of an ethical teacher (short paper), in: P. Bruno, tions), ICLP Technical Communications 2021, Porto F. Calimeri, F. Cauteruccio, M. Maratea, G. Ter- (virtual event), 20-27th September 2021, volume 345 racina, M. Vallati (Eds.), Joint Proceedings of the of EPTCS, 2021, pp. 182–188. doi:10.4204/EPTCS. 1st International Workshop on HYbrid Models for 345.32 . Coupling Deductive and Inductive ReAsoning (HY- [27] A. Dyoub, S. Costantini, F. A. Lisi, G. De Gasperis, DRA 2022) and the 29th RCRA Workshop on Exper- Demo paper: Monitoring and evaluation of ethi- imental Evaluation of Algorithms for Solving Prob- cal behavior in dialog systems, in: Y. Demazeau, lems with Combinatorial Explosion (RCRA 2022) T. Holvoet, J. M. Corchado, S. Costantini (Eds.), Ad- co-located with the 16th International Conference vances in Practical Applications of Agents, Multi- on Logic Programming and Non-monotonic Reason- Agent Systems, and Trustworthiness. The PAAMS ing (LPNMR 2022), Genova Nervi, Italy, September Collection - 18th International Conference, PAAMS 5, 2022, volume 3281 of CEUR Workshop Proceed- 2020, L’Aquila, Italy, October 7-9, 2020, Proceed- ings, CEUR-WS.org, Germany, 2022, pp. 1–8. URL: ings, volume 12092 of Lecture Notes in Computer http://ceur-ws.org/Vol-3281/paper1.pdf. Science, Springer, UK, 2020, pp. 403–407. doi:10.1 [19] A. Dyoub, S. Costantini, F. A. Lisi, Learning domain 007/978- 3- 030- 49778- 1\_35 . ethical principles from interactions with users, Dig- ital Society 1 (2022) 28. doi:10.1007/s44206- 022 - 00026- y . [20] S. Tolmeijer, M. Kneer, C. Sarasua, M. Christen, A. Bernstein, Implementations in machine ethics: A survey, ACM Computing Surveys 53 (2020) 1–38. URL: http://dx.doi.org/10.1145/3419633. doi:10.114 5/3419633 . [21] S. J. Russell, P. Norvig, Artificial Intelligence: A Modern Approach, Pearson Education Limited, 2016. [22] D. Leben, Normative principles for evaluating fairness in machine learning, in: Proceedings of the AAAI/ACM Conference on AI, Ethics, and So- ciety, AIES ’20, Association for Computing Ma- chinery, New York, NY, USA, 2020, p. 86–92. URL: htt ps: //d oi. org /10 .11 45/ 337 562 7.3 375 808. doi:10.1145/3375627.3375808 . [23] A. Dyoub, S. Costantini, F. A. Lisi, I. Letteri, Logic- based machine learning for transparent ethical agents, in: F. Calimeri, S. Perri, E. Zumpano (Eds.), Proceedings of the 35th Italian Conference on Com- putational Logic - CILC 2020, Rende, Italy, October 13-15, 2020, volume 2710 of CEUR Workshop Pro- ceedings, CEUR-WS.org, 2020, pp. 169–183. URL: https://ceur-ws.org/Vol-2710/paper11.pdf. [24] G. Baxter, I. Sommerville, Socio-technical sys- tems: From design methods to systems engineer- ing, Interacting with Computers 23 (2011) 4–17. doi:10.1016/j.intcom.2010.07.003 . [25] K. Van Dam, I. Nikolic, Z. Lukszo, Agent-Based Modelling of Socio-Technical Systems, 2013. doi:10 .1007/978- 94- 007- 4933- 7 . [26] A. Dyoub, S. Costantini, I. Letteri, F. A. Lisi, A logic-based multi-agent system for ethical monitor- ing and evaluation of dialogues, in: A. Formisano, Y. A. Liu, B. Bogaerts, A. Brik, V. Dahl, C. Do- daro, P. Fodor, G. L. Pozzato, J. Vennekens, N. Zhou (Eds.), Proceedings 37th International Conference on Logic Programming (Technical Communica-