Towards Ethical Risk Assessment of Symbiotic AI Systems with Fuzzy Rules Abeer Dyoub1,* , Francesca Alessandra Lisi1 1 University of Bari Aldo Moro, DiB Dept., via E. Orabona 4, Bari, 70125, Italy Abstract Artificial Intelligence (AI) based systems are expanding rapidly in all domains of life. They are entering our everyday life and performing tasks on our behalf. AI-based systems such as personal healthcare assistants are increasingly engaging in close symbiotic relationships with humans. Symbiotic AI (SAI) promises improved outcomes in various domains such as healthcare, education, and business. However, as the degree of symbiosis increases, so does the ethical risk. To ensure that these systems behave ethically and do not cause harm of any kind (physical, mental, violation of privacy, etc.), we need to find ways to assess the ethical risk (risk of causing harm), then choose the right action to mitigate that risk. In this work, we propose an approach based on fuzzy logic for ethical risk assessment (ERA) of SAI systems. The approach is illustrated by means of a case study taken from the healthcare domain. Keywords Symbiotic AI, AI Ethics, Ethical Risk Assessment, Fuzzy Logic, Fuzzy Rules 1. Introduction Artificial Intelligence (AI) systems are rapidly expanding across all areas of life, becoming integral to our everyday activities and performing tasks on our behalf. These AI-based systems are increasingly forming close symbiotic relationships with humans, exemplified by digital twins, personal healthcare assistants, and virtual avatars. The term symbiosis has emerged as a result of the ongoing debate over whether AI will replace or enhance human abilities. Human-AI Symbiosis (known as Symbiotic AI, here after SAI) is about human-AI teaming, enabling people and AI to collaborate together for achieving better results together than they could separated [1]. SAI holds the promise of improved outcomes in various sectors such as healthcare, education, and business. However, SAI poses not only several technological challenges but also many philosophical questions. A deeply ingrained symbiosis guided by ethics will enhance human experience while respecting our values, which will make AI technologies ethically acceptable [2]. It is important to conceptualize and design holistic symbiotic frameworks for AI aiming at generating fair, legitimate, and effective outcomes while ensuring ethical and legal compliance. Such frameworks are expected to shape the development of SAI systems and influence technological governance through rigorous model assessment [3] . Over decades, AI developers have been considering various moral or ethical theories for developing artificial moral agents (AMA). When people think about moral theories, they usually consider three primary schools of academic ethics that aim to explain what is good or bad, right or wrong, and why. Briefly, these three schools are [4]: • Consequentialism: This theory, with utilitarianism as a notable example, asserts that the right action is the one that brings about the best overall consequences. • Deontology: Exemplified by Immanuel Kant’s theory, deontology states that an act is right or wrong based on its adherence to a set of principles, independent of its consequences. Breaking the rules is wrong, even if the outcome is positive. 3rd Workshop on Bias, Ethical AI, Explainability and the Role of Logic and Logic Programming (BEWARE24), co-located with AIxIA 2024, November 25-28, 2024, Bolzano, Italy * Corresponding author. $ abeer.dyoub@uniba.it (A. Dyoub); francesca.lisi@uniba.it (F. A. Lisi)  0000-0003-0329-2419 (A. Dyoub); 0000-0001-5414-5844 (F. A. Lisi) © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings • Virtue Ethics: Represented by Aristotle’s view, virtue ethics posits that the right action is what a virtuous person would do. If a courageous, generous, or kind person would perform the act, it is deemed right; if not, it is considered wrong. These theories are fundamentally incompatible with each other. If we believe an act is right based on its consequences, we are not deontologists. Conversely, if we believe an act can be right regardless of its consequences, we are not utilitarians. Therefore, we believe that the idea of solving an ethical problem by combining elements of utilitarianism, Kantianism, and perhaps a bit of virtue ethics is a flawed approach to developing ethically sound AI. These theories primarily aim to explain what is right or wrong and why. They are not meant to inform ethical decision making procedures. Ethical reasoning is far more complicated than following a moral philosophical theory. Ethical decision-making and judgment is a complex process involving numerous factors, blend- ing reasoning and emotions. Additionally, moral decision-making is highly flexible, contextual, and culturally diverse. Since the beginning of this century, various approaches have been attempted to integrate ethical decision-making into intelligent autonomous agents. However, no fully descriptive and widely accepted model of moral judgment and decision-making has been established, and none of the developed solutions has proven entirely convincing in providing trusted moral behavior. Anytime the actions/decisions of an AI-based system have potential to impact humans positively or negatively, it is a matter of ethical concern. In the ethical context, it is crucial to prevent AI-based systems from causing harm. The potential risk of causing harm of any kind to humans is what we refer to as ’ethical risk’ in this paper. There are different categories of ethical risks involving different types of harm, some examples are: • physical harm (e.g. injury or death) • mental harm (e.g. depression, anxiety, addiction) • autonomy • violation of privacy or confidentiality • violation of trust and respect • violation of fairness (discrimination) There is no single, definitive concept of risk. Instead, risk can be conceptualized and analyzed in various ways, with each approach offering different levels of usefulness depending on the context [5]. When building ethical AI systems, we need to identify the ethical risks associated with these systems and their use. Our primary concern should be ethical risk identification, not the ethical theory that explain why something is ethical. What are the ethical risks involved in what we are developing? How people might use our system/product in ways that are ethically risky? (deployment matter). In light of this, the development team should think about what features to (not)include in the AI system to mitigate these risks [6]. We need to develop frameworks for AI-related ethical risks understanding and analysis. In [5], authors presented a framework for epistemological analysis of AI-related risks. Their framework is a multi-component framework that distinguishes between three dimensions of hazard, exposure, and vulnerability. The sources of the potential harm, what could be harmed (humans), and how much the exposed humans are susceptible to the impacts of this potential harm. This three dimensional analysis allows us to better understand the AI-related risks and effectively intervene to mitigate them. With SAI systems, as the level of symbiosis increases, so does the ethical risk. To ensure these systems behave ethically and do not cause harm of any kind, we must develop methods to assess ethical risks and choose appropriate actions to mitigate these risks. No one can precisely estimate possible ethical risks without a comprehensive understanding of all aspects of the risk system being studied. In practical scenarios, it is impossible to completely eliminate gaps in Ethical Risk Assessment (ERA), resulting in fuzziness (imprecision, vagueness, incompleteness, etc.). Therefore, it is essential to address and manage the inherent fuzziness within ethical risk systems. In this work, we propose an approach based on fuzzy logic for ERA. Fuzzy logic offers a flexible framework capable of capturing and processing vague, imprecise, and uncertain information, leading to more nuanced and comprehensive ethical risk assessments. We focus here on ERA as a step in the overall ethical decision making and judgement process/system. The decision/action to be taken is based on the ethical risk assessment, and aims to mitigate this possible ethical risk according to the calculated level of this risk. The paper is organized as follows. Section 2 is devoted to background information. In particular, in subsection 2.1 we recall basic notions about fuzzy logic and overview its applications with special emphasys on those in risk assessment, whereas in subsection 2.2 we give a brief overview of the state of the art of ethical decision making and judgement. Section 3 is dedicated to our ERA approach. In subsection 3.1 we show the architecture of our ERA model. while in subsection 3.2, we illustrate the ERA approach using a case study from the medical domain. Finally, Section 4 is dedicated to discussion and conclusion. 2. Background 2.1. Fuzzy Logic and Applications Developed by Lotfi Zadeh1 in the 1960s, fuzzy logic [7] is based on fuzzy set theory, which is a generalization of the classical set theory. The classical sets are also called clear sets, as opposed to vague, and similarly classical logic is also known as Boolean logic or binary. A fuzzy set is a mathematical construct that allows an element to have a gradual degree of membership within the set, as opposed to the binary inclusion found in classical sets [8]. Formally, a fuzzy set 𝐴 in a universe of discourse 𝑋 is defined by a membership function 𝜇𝐴 : 𝑋 → [0, 1], where each element 𝑥 ∈ 𝑋 is assigned a degree of membership 𝜇𝐴 (𝑥). This value represents the extent to which 𝑥 belongs to the fuzzy set 𝐴. Membership functions (MF) can take various shapes, such as triangular, trapezoidal (this is the MF chosen for our case study, see section 3), or Gaussian, depending on the problem domain and the nature of the input data [9]. For instance, a trapezoidal MF (trapmf ) is defined as follows: ⎧ ⎪ 0 𝑥<𝑎 ⎪ 𝑥−𝑎 𝑎 ⩽ 𝑥 ⩽ 𝑏 ⎪ ⎪ ⎨ 𝑏−𝑎 𝑡𝑟𝑎𝑝𝑚𝑓 (𝑋; 𝑎, 𝑏, 𝑐, 𝑑) = 1 𝑏⩽𝑥⩽𝑐 𝑑−𝑥 ⎪ 𝑑−𝑐 𝑐 ⩽ 𝑥 ⩽ 𝑑 ⎪ ⎪ ⎪ ⎩ 0 𝑑⩽𝑥 Here, 𝑥 represents a real value (crisp value) within the universe of discourse, whereas 𝑎, 𝑏, 𝑐, 𝑑 represent x-coordinates of the four heads of the trapezoidal which should satisfy the following condition: 𝑎 < 𝑏 < 𝑐 < 𝑑. By using min and max, we can have an alternative expression for the preceding equation: 𝑡𝑟𝑎𝑝𝑚𝑓 (𝑋; 𝑎, 𝑏, 𝑐, 𝑑) = 𝑚𝑎𝑥(𝑚𝑖𝑛( 𝑥−𝑎 𝑑−𝑥 𝑏−𝑎 , 1, 𝑑−𝑐 ), 0) The concept of MF discussed above allows us to define fuzzy systems in natural language, as the MF couples fuzzy logic with linguistic variables. Let 𝑉 be a variable (e.g., quality of service in a restaurant, tip amount), 𝑋 the range of values of the variable, and 𝑇𝑉 a finite or infinite set of fuzzy sets. A linguistic variable corresponds to the triplet (𝑉, 𝑋, 𝑇𝑉 ). In fuzzy logic, reasoning, also known as approximate reasoning, is based on fuzzy rules that are expressed in natural language using linguistic variables such as "HIGH" or "LOW", which we have defined above. A fuzzy rule has the form: If 𝑥 ∈ 𝐴 and 𝑦 ∈ 𝐵, then 𝑧 ∈ 𝐶, where 𝐴, 𝐵, and 𝐶 are fuzzy sets. For example: ’If (the quality of the food is HIGH), then (tip is HIGH)’. Fuzzy logic is particularly effective in systems that must emulate human decision-making. It enables computers and other systems to make decisions based on imprecise or incomplete information, reflecting 1 https://spectrum.ieee.org/lotfi-zadeh the way humans process information and make judgments in everyday situations. Fuzzy logic is used in a variety of applications, including consumer electronics (e.g., washing machines, cameras) to industrial control systems (e.g., chemical plant processes, automotive systems), control systems, decision support systems, and pattern recognition [10, 11]. In healthcare, fuzzy logic can be applied to diagnose conditions, tailor treatments, and optimize resource allocation, ensuring that decisions accommodate the nuances of human health and well-being [12]. Fuzzy logic offers a flexible framework for handling uncertainties and ambiguities associated with complex decision making processes. Notably, it has been applied for risk assessment and management in many domains. Herein, we highlight some of these applications. One of the main applications is in the evaluation of environmental risks, such as pollution levels or the impact of climate change. For instance, fuzzy logic has been used to assess the risk of water pollution by integrating various indicators, such as chemical concentrations, water PH, and temperature, into a single risk index [13]. Another example of application for fuzzy logic is the assessment of risks in work places where data might be vague or incomplete. A fuzzy framework was used for assessing the risk of injury due to machinery, considering hazardous factors such as the skill level of the operator, and the working environment [14]. This approach allows safety managers to better prioritize risks and implement more effective mitigation strategies. Financial risk management is another area in which fuzzy logic was applied. Precise financial risk prediction is very challenging because financial markets are characterized by high levels of uncertainty and volatility. Fuzzy logic helps in modeling such uncertainty, allowing for better decision-making in areas such as portfolio management and credit risk assessment [15]. Fuzzy logic has been also used for assessing and managing the risks associated with project timelines, costs, and resources. Project managers can develop more realistic schedules and budgets, by incorporating fuzzy inputs like the likelihood of delays, cost overruns, and resource availability. In large and complex projects, traditional risk management approaches may fall short due to the high levels of uncertainty involved, fuzzy logic can offer a valuable solution [16]. 2.2. Ethical Decision Making: State of the Art In this section, we give a brief overview of the landscape of the machine ethics field. There are three design approaches to programming ethical decision-making into agent systems, with different attempts to build moral agents classified under one of these approaches, according to the first high-level classification by Wallach and Allen [17]: I) Top-down approaches: These implement a specific normative theory of ethics into autonomous agents, ensuring that the agent acts according to the principles of this theory. II) Bottom-up approaches: These are developmental or learning approaches where ethical mental models emerge through the activity of individuals. III) Hybrid approaches: These integrate both top-down and bottom-up approaches. Authors in [18] present a model in which they qualify the good in two modes, one is based on rights, and the other is based on values. Then, for quantifying the good, they introduce a method in which they define weighing parameters for the good and the bad ramifications of events caused by the actions, then calculate the total sum. Greater weights correspond to more participation in the good, while negative weights do more harm than good. This work was implemented in Answer Set Programming (ASP) [19] JEREMY [20] is an implementation of the Hedonistic Act Utilitarianism. This theory states that an action is morally right if and only if that action maximizes the pleasure, i.e. the one with the greatest net pleasure consequences. The authors of JEREMY, to respond to critics of act utilitarianism, have created another system, W.D. [20] which avoids a single absolute duty, by following several duties. Their system follows the theory of prima facie duties of Ross [21] and is implemented in Inductive Logic Programming (ILP) [22]. Tom Powers in [23] assesses the viability of using deontic and default logics, to implement Kant’s categorical imperative. In [24], the authors suggest that mechanized multi-agent deontic logics might be an appropriate vehicle for engineering ethically correct robot behaviours. Other attempts tried to formalize ethical systems using modal logic formalisms [25] and then trying to operationalize these formalizations on the computer, like in [24] and [23]. These formalizations are mainly based on the use of deontic logics. In [26], the authors formalized three ethical conceptions (the Aristotelian rules, Kantian categorical imperative, and Constant’s objection) using non-monotonic logic, particularly ASP. Pereira and Saptawijaya have proposed the use of different logic-based features, for representing diverse issues of moral facets, such as moral permissibility, doctrines of Double Effect and Triple Effect, the Dual-process Model, and counterfactual thinking in moral reasoning. They investigated the use of abduction, probabilistic logic programming, logic programming updating, and tabling. These logic-based reasoning features were synthesized in three different systems: ACORDA, Probabilistic EPA, and QUALM [27]. Model Checking was used to provide guarantees about whether an autonomous aircraft would create and execute plans that involved ethical decision-making [28]. In the context of the HERA (Hybrid Ethical Reasoning Agents) Project [29], many ethical principles were implemented to help an agent judge the moral permissibility of its actions. The implementation was done in Python. In [30], Sergot provides an alternative representation to the argumentative representation of a moral dilemma case concerning a group of diabetic persons, presented in [31], where the authors used value-based argumentation to solve this dilemma. According to Sergot, the argumentation framework representation does not work well and does not scale. The proposed solution was implemented in ASP. In [32], the authors use CP-nets to model both ethical principles and subjective preferences of an individual in two separate CP-nets. If we want the individual to behave ethically in a certain scenario: firstly the individual determines whether she can use her most preferred choice by checking if her CP-net is “sufficiently close” to the ethical CP-net. In [33] the authors propose a novel hybrid method using symbolic judging agents to evaluate the ethical behaviour of reinforcement learning agents. However, to do this, judging agents need to access extensive data about the learning agents like their actions, perceptions, etc. which raises other ethical issues related to privacy. TRUTH-TELLER [34] is a Case-Based Reasoning (CBR) System. It compares cases presenting ethical dilemmas about whether to tell the truth or not. Its comparisons list ethically relevant similarities and differences. SIROCCO [35] is another system that employs case-based (casuistic) reasoning. But, differently from TRUTH-TELLER, it retrieves ethical principles and past cases relevant to the new case situations. Dancy suggested that neural network models of cognition might offer a profitable way to explore some concerns pertaining to learning and generalizing without principles [36]. In [37] specific actions concerning killing and allowing to die were classified as ethical or unethical depending upon different motives and consequences. A simple recurrent artificial neural network (ANN) trained on a series of such cases was able to provide reasonable responses to a variety of previously unseen cases. In [38], the author has used reinforcement learning for an agent to learn the correct ethical response in a given situation. One limitation is having to design ethical utility functions that can be expressed in the observation function of the agent. That is since the learned behaviour is derived from what the agent can observe, the designer has to ensure that ethical behaviour can also be, at least potentially, derived from the agent’s observations. This greatly limits the complexity of situations that the agent can conceivably handle. In [39], the authors considered two potentially complementary paradigms, for designing moral decision-making methodologies. Namely, extending game-theoretic solution concepts to incorporate ethical aspects, and using machine learning on human-labelled instances. MedEthEx [40], and EthEl [41] are two systems based on a more specific theory of prima facie duties viz., the principle of Biomedical ethics of Beauchamp and Childress, and implemented in ILP. In these systems, the strength of each duty is measured by assigning it a weight, capturing the view that one duty may take precedence over another. Then computes, for each possible action, the weighted sum of duty satisfaction, and the right action is the one with the greatest sum. However, it is not really clear the basis for assigning weights to duties. Dyoub et. al. [42, 43, 44, 45] proposed a hybrid approach based on ASP and ILP for the evaluation of the ethical behavior of AI-based systems. In a later work and based on this approach, the authors proposed a logic-based multi agent system for ethical monitoring and evaluation of dialogue systems (chatbots). In [46] and [47], the authors have used constraints to implement ethical behaviour in robots, partic- ularly those that have the capability to exhibit lethal force, so that we can be guaranteed that robots will always obey the Laws Of War. Recent works have highlighted the ethical and legal limits for the diffusion of self-made autonomous weapons [48]. In [49], the authors described a moral reasoner that calculates an estimated morality level (between -1 and 1) of an action based on the influence on the total amount of utility in the world by the believed contribution of the action to the following three duties (also called moral goals): Autonomy, Non-maleficence and Beneficence. The moral reasoner is capable of balancing conflicting moral goals, but there is no guarantee that the resulting blend of principles will end up being coherent. For systematic surveys of the machine ethics field, the reader can refer, among many, to [50]. 3. Our Proposed ERA Approach Assessing ethical risks in general and in the medical domain in particular is a complex task as it is a qualitative concept. We propose an approach based on fuzzy rules to compute the ethical risk level. The computed level can be subsequently considered to make the appropriate decision/action to mitigate that risk. In this section, we present our proposal for ERA. In subsection 3.1 we show the architecture of a fuzzy system for ERA, while in subsection 3.2, we illustrate the ERA approach using a case study from the medical domain. The fuzzy ERA (fERA) model will be part of the overall ethical decision making (EDM) model, see Figure 1. The fERA model will receive from the EDM engine the relevant facts of the case at hand, and return to the EDM engine the calculated risk level to be used for computing the best action/decision to mitigate this risk. Figure 1: Architecture of the overall EDM system 3.1. A Fuzzy System for ERA Figure 2 shows the building blocks of a fuzzy system for ERA. The main components of our ERA system are: Inputs These are the factors/parameters relevant for the ethical risk calculation. Figure 2: Architecture of our fuzzy system for ERA Fuzzification In this stage crisp input values are converted into fuzzy sets, , allowing real-world data (e.g., temperature, speed) to be interpreted in a way that accounts for uncertainty or vagueness. This is done using membership functions that map input values to a degree of membership between 0 and 1. For example, in a temperature control system, a crisp input of 75°F might be partially categorized as both “warm” and “hot,” with different membership degrees for each. Inference Engine The inference engine will consults the Fuzzy Rules Base that contains a set of "if- then" rules that define the system’s behavior. These rules describe how fuzzy inputs relate to fuzzy outputs based on expert knowledge. The engine will apply these rules to the fuzzified input to derive fuzzy output sets. It determines which rules are relevant based on the degree of membership of the input values. There are different methods to infer rules, such as the Mamdani or Sugeno inference methods, which handle how the rules combine to produce a result. 2 We use the Mamdani method in our case study. Fuzzy rules could be automatically generated from data. In the current implemented version these rules are manually written. Defuzzification Converting the fuzzy output sets back into crisp values to implement actions or decisions. Common defuzzification methods include centroid, mean of maximum, and bisector, etc.3 Centroid method is the most widely used methods amongst all the de-fuzzification methods [51]. This method provides a center of the area under the curve of the membership function. The centroid is computed using the following formula, where 𝜇(𝑥𝑖 ) is the membership value for point 𝑥𝑖 in the universe of discourse: ∑︀ 𝑥𝐶𝑒𝑛𝑡𝑟𝑜𝑖𝑑 = ∑︀𝑖 𝜇(𝑥𝑖 )𝑥𝑖 𝑖 𝜇(𝑥𝑖 ) Output The only Output in our fuzzy system is the ethical risk level. The system was implemented in python using the library Scikit-Fuzzy. 3.2. Case Study To illustrate our proposed ERA approach, we will use the following case study adapted from [40] which is a common type of ethical dilemma that a care robot may face. Patient Dilemma Problem: A care robot approaches her competent adult patient to give her her medicine in time and the patient rejects to take it. Should the care robot try again to change the patient’s mind or accept the patient’s decision as final? 2 https://it.mathworks.com/help/fuzzy/types-of-fuzzy-inference-systems.html 3 https://it.mathworks.com/help/fuzzy/defuzzification-methods.html The dilemma arises because, on the one hand, the care robot may not want to risk diminishing the patient’s autonomy by challenging her decision; on the other hand, the care robot may have concerns about why the patient is refusing the treatment. Three of the four Principles/Duties of Biomedical Ethics are likely to be satisfied or violated in dilemmas of this type: the duty of Respect for Autonomy, the duty of Nonmaleficence and the duty of Beneficence. See section 2.2 for the description of the approach of [40] in handling ethical decision making in such cases. In this case study, ERA addresses the ethical risk of causing harm to the patient (in this case study, the ethical risk is of the kind ’physical harm’, namely patient physical health or life risk) and works as the basis for decision making by the care robot. In order to evaluate the risk, the care robot can consider different parameters like the severity of health condition of the patient, the mental/psychological condition of the patient, physiological indicators of well-being, etc. These parameters can be all considered fuzzy concepts. In this case study, we choose the following inputs to the system: 1. the severity of the health condition of the patient. The value of this parameter is given by the human medical doctor based on a periodical medical check and stored in the patient medical record, to which the care robot can have access to retrieve the value of severity, to be used for ERA. 2. the mental/psychological condition of the patient. This parameter is evaluated by the care robot based on a dialogue with the patient and on certain observations like movement or face expressions, etc. Both inputs are rated on a scale between 0 and 10. More precisely, the crisp values in input are the answer to the following questions: How severe is the health condition of the patient, on a scale of 0 to 10? Where zero refers that the patient is in a very good health conditions in terms of severity, while ten indicates that the patient is in a very severe conditions. How is the patient on the psychological level, is the patient lucid or depressed, on a scale of 0 to 10? Where zero indicates that the patient is in a very good mental condition, while the ten indicates that the patient is in a very bad mental condition, i.e. depressed. The crisp values are then fuzzified, both into 5 sets. For the severity of the health condition the fuzzy sets are: VERY LOW, LOW, MEDIUM, HIGH, VERY HIGH. For the mental/psychological condition the fuzzy sets are: VERY BAD, BAD, AVERAGE, GOOD, VERY GOOD. Starting from these two inputs, once fuzzified, the ERA system calculates the risk level on a scale between 0 % and 100 %. Also for the output there are 5 fuzzy sets: VERY LOW, LOW, MEDIUM, HIGH, VERY HIGH. The inputs and the output are the antecedents and the consequent, respectively, of the rules employed by the fuzzy system. Rules: The following rules are the fuzzy inference rules stored in the Fuzzy Rule Base. As mentioned above, these rules are used to derive the output from the input. Table 1 shows the rules matrix, which is a simplified representation of these rules: 1. If severity is VERY HIGH or (severity is HIGH and mental is average ) or (severity HIGH and mental is VERY BAD), then the risk is VERY HIGH. 2. If severity is VERY LOW or (severity is LOW and mental is VERY GOOD), then the risk is VERY LOW. 3. If (severity is LOW and mental is GOOD) or (severity is LOW and mental is AVERAGE) or (severity is MEDIUM and mental is VERY GOOD) or (severity is LOW and mental is BAD), then the risk is LOW. 4. If (severity is MEDIUM and mental is VERY BAD) or (severity is MEDIUM and mental is AVERAGE) or (severity is MEDIUM and mental is BAD) or (severity is HIGH and mental is BAD), then the risk is HIGH. 5. If (severity is MEDIUM and mental is GOOD) or (severity is HIGH and mental is GOOD) or (severity is LOW and mental is VERY BAD) or (severity is HIGH and mental is VERY GOOD), then the risk is MEDIUM. Usage: The initial inputs, in this case severity and mental, are provided by the user (the care robot in this case). These values are then fuzzified by using an MF. In this paper we choose to use the trapezoidal MF (see Section 2) because there is an interval of input crisp values for which the membership degree to the fuzzy set is 100%. Figure 3 shows the fuzzification of input values using the trapezoidal function. The fuzzified input is then processed through the rule matrix (Table 1, ’-’ values means the value here does not have effect on the output) which comprises the above specifically designed rules. The final output is subsequently de-fuzzified using centroid method to find a single crisp value which defines the output of a fuzzy set. This final value provides the level of ethical risk on the life of the patient. (a) Severity of Health Condition (b) Mental Condition Figure 3: Fuzzification of inputs using the trapezoidal MF. Table 1 Rules Matrix: fuzzy rules for ethical risk evaluation (risk of causing physical harm) Severity Mental Risk VERY HIGH - VERY HIGH HIGH AVERAGE VERY HIGH HIGH VERY BAD VERY HIGH VERY LOW - VERY LOW LOW VERY GOOD VERY LOW LOW GOOD LOW LOW AVERAGE LOW MEDIUM VERY GOOD LOW LOW BAD LOW MEDIUM VERY BAD HIGH MEDIUM AVERAGE HIGH MEDIUM BAD HIGH HIGH BAD HIGH MEDIUM GOOD MEDIUM HIGH GOOD MEDIUM LOW VERY BAD MEDIUM HIGH VERY GOOD MEDIUM As an example, if the severity is rated as 3.5, and mental as 2, this system will infer that the risk is 35% which is in area between LOW and MEDIUM (see Figure4). Based on this calculated level of ethical risk, the care robot decides what to do (try again/insist, accept, consult the doctor, etc.). Figure 4: Output: Risk Level 4. Conclusion and Discussion This paper is part of an ongoing work on ethical decision making and judgment in SAI systems. In this paper, we presented a novel approach for ethical risk assessment. The approach based on fuzzy rules, calculates a quantitative value for ethical risk (mapped later to a qualitative value). The approach recognizes the fuzzy nature of the significant factors that affect the ethical risk and computes the ethical risk level based on the values of these input factors. Higher values represent higher risk levels. The approach is illustrated via a case study in the medical field. In the future, this ERA model will be one block in the overall ethical decision making system that aims to mitigate the possible ethical risks. In this paper, we considered a simple case study (ethical dilemma) in the medical field, adapted from the literature on machine ethics. We decided, in this work, to maintain the case study simple because our objective was only to illustrate our ERA approach. Our ERA approach is a general approach that can be used by SAI systems in any domain for the assessment of the possible ethical risk. Then based on this assessment, the appropriate action is chosen to mitigate this risk. Back to the case study adopted in this work, it considers a patient with a chronic disease for which she was prescribed a certain medicine to take every day at a certain time to maintain her disease under control, so she can live her life normally. The patient has no other health issues. The care robot approaches the patient to give her her medicine on time. If the patient refuses to take the medicine, the care robot should decide to accept or insist based on the risk level. But, what if the patient has other health issues? what if there are many other physiological indicators that can interfere in the disease and should be monitored like blood pressure? what if the patient has the blood pressure or the temperature, etc. very high or out of the normal ranges? In the future, we are going to extend this case study considering more complex and realistic scenarios and consulting with domain experts. Furthermore, we are currently working on various case studies in which we might have different types of ethical risks. Acknowledgments This work was partially supported by the project FAIR - Future AI Research (PE00000013), under the NRRP MUR program funded by the NextGenerationEU. References [1] H. J. Wilson, P. R. Daugherty, Creating the symbiotic ai workforce of the future, MIT Sloan Management Review (2019). URL: https://sloanreview.mit.edu/article/creating-the-symbiotic-ai-w orkforce-of-the-future/. [2] A. Carnevale, A. Lombardi, F. A. Lisi, Exploring ethical and conceptual foundations of human- centred symbiosis with artificial intelligence, in: G. Boella, et al. (Eds.), Proceedings of the 2nd Workshop on Bias, Ethical AI, Explainability and the role of Logic and Logic Programming co- located with the 22nd International Conference of the Italian Association for Artificial Intelligence (AI*IA 2023), volume 3615 of CEUR Workshop Proceedings, 2023, pp. 30–43. URL: https://ceur-ws.or g/Vol-3615/paper3.pdf. [3] P. Marra, L. Pulito, A. Carnevale, A. Lombardi, A. Dyoub, F. A. Lisi, A procedural idea of decision- making in the context of symbiotic AI, in: A. J. Dix, M. Roach, T. Turchi, A. Malizia, B. Wilson (Eds.), Proceedings of the 1st International Workshop on Designing and Building Hybrid Human-AI Systems co-located with 17th International Conference on Advanced Visual Interfaces (AVI 2024), Arenzano (Genoa), Italy, June 3rd, 2024, volume 3701 of CEUR Workshop Proceedings, CEUR-WS.org, 2024. URL: https://ceur-ws.org/Vol-3701/paper9.pdf. [4] M. Frischhut, Normative Theories of Practical Philosophy, Springer International Publishing, Cham, 2019, pp. 21–30. URL: https://doi.org/10.1007/978-3-030-10582-2_2. doi:10.1007/978-3 -030-10582-2_2. [5] G. Zanotti, D. Chiffi, V. Schiaffonati, Ai-related risk: An epistemological approach, Philosophy & Technology 37 (2024) 1–18. [6] R. Blackman, Ethical Machines: Your Concise Guide to Totally Unbiased, Transparent, and Re- spectful AI, G - Reference,Information and Interdisciplinary Subjects Series, Harvard Business Review Press, 2022. URL: https://books.google.it/books?id=gYK0zgEACAAJ. [7] L. A. Zadeh, Fuzzy logic, Computer 21 (1988) 83–93. [8] H.-J. Zimmermann, Fuzzy set theory—and its applications, Springer Science & Business Media, 2011. [9] T. J. Ross, Fuzzy logic with engineering applications, John Wiley & Sons, 2009. [10] H. Singh, M. M. Gupta, T. Meitzler, Z.-G. Hou, K. K. Garg, A. M. G. Solo, L. A. Zadeh, Real- life applications of fuzzy logic, Advances in Fuzzy Systems 2013 (2013) 581879. doi:https: //doi.org/10.1155/2013/581879. [11] D. E. Tamir, N. D. Rishe, A. Kandel, Fifty years of fuzzy logic and its applications, volume 326, Springer, 2015. [12] S. Thukral, J. S. Bal, Medical applications on fuzzy logic inference system: a review, International Journal of Advanced Networking and Applications 10 (2019) 3944–3950. [13] P. Rea, Risk assessment of water pollution engineering emergencies based on fuzzy logic algorithm, Water Pollution Prevention and Control Project 3 (2022) 1–11. [14] D. Tadic, M. Djapan, M. Misita, M. Stefanovic, D. D. Milanovic, A fuzzy model for assessing risk of occupational safety in the processing industry, International journal of occupational safety and ergonomics 18 (2012) 115–126. [15] T. Korol, Fuzzy logic in financial management, INTECH Open Access Publisher, 2012. [16] B. M. Moreno-Cabezali, J. M. Fernandez-Crehuet, Application of a fuzzy-logic based model for risk assessment in additive manufacturing r&d projects, Computers & Industrial Engineering 145 (2020) 106529. URL: https://www.sciencedirect.com/science/article/pii/S0360835220302631. doi:https://doi.org/10.1016/j.cie.2020.106529. [17] W. Wallach, C. Allen, Moral machines: Teaching robots right from wrong, Oxford University Press, England, 2008. [18] F. Berreby, G. Bourgne, J. Ganascia, A declarative modular framework for representing and applying ethical principles, in: Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems, AAMAS 2017, São Paulo, Brazil, May 8-12, 2017, ACM, USA, 2017, pp. 96–104. URL: http://dl.acm.org/citation.cfm?id=3091145. [19] A. Dyoub, S. Costantini, G. De Gasperis, Answer set programming and agents, Knowledge Eng. Review 33 (2018) e19. doi:10.1017/S0269888918000164. [20] M. Anderson, S. L. Anderson, C. Armen, Towards machine ethics, in: AAAI-04 workshop on agent organizations: theory and practice, San Jose, CA, 2004. [21] W. D. Ross, The Right and the Good, Oxford University Press, Oxford, UK, 1930. doi:10.2307/21 80065. [22] S. Muggleton, L. De Raedt, Inductive logic programming: Theory and methods, J. Log. Program. 19/20 (1994) 629–679. doi:10.1016/0743-1066(94)90035-3. [23] T. M. Powers, Prospects for a kantian machine, IEEE Intelligent Systems 21 (2006) 46–51. [24] S. Bringsjord, K. Arkoudas, P. Bello, Toward a general logicist methodology for engineering ethically correct robots, IEEE Intelligent Systems 21 (2006) 38–44. URL: https://doi.org/10.1109/MI S.2006.82. [25] H. J. Gensler, Formal Ethics, Psychology Press, UK, 1996. [26] J.-G. Ganascia, Modelling ethical rules of lying with answer set programming, Ethics and information technology 9 (2007) 39–47. doi:10.1007/s10676-006-9134-y. [27] L. M. Pereira, A. Saptawijaya, Programming Machine Ethics, volume 26 of Studies in Applied Philosophy, Epistemology and Rational Ethics, Springer, Switzerland, 2016. doi:10.1007/978-3-3 19-29354-7. [28] L. Dennis, M. Fisher, M. Slavkovik, M. Webster, Formal verification of ethical choices in autonomous systems, Robotics and Autonomous Systems 77 (2016) 1–14. [29] F. Lindner, M. M. Bentzen, B. Nebel, The hera approach to morally competent robots, in: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE, 2017, pp. 6991–6997. [30] M. Sergot, Prioritised Defeasible Imperatives, Dagstuhl Seminar 16222 Engineering Moral Agents – from Human Morality to Artificial Morality, 2016. URL: https://materials.dagstuhl.de/files/16/16 222/16222.MarekSergot.Slides.pdf, schloss Dagstuhl-Leibniz-Zentrum fuer Informatik. [31] K. Atkinson, T. J. M. Bench-Capon, Addressing moral problems through practical reasoning, in: Deontic Logic and Artificial Normative Systems, 8th International Workshop on Deontic Logic in Computer Science, DEON 2006, Utrecht, The Netherlands, July 12-14, 2006, Proceedings, volume 4048 of Lecture Notes in Computer Science, Springer, Netherlands, 2006, pp. 8–23. doi:10.1007/11 786849\_4. [32] A. Loreggia, N. Mattei, F. Rossi, K. B. Venable, Preferences and ethical principles in decision making, in: Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, AIES 2018, New Orleans, LA, USA, February 02-03, 2018, ACM, USA, 2018, p. 222. doi:10.1145/3278721. 3278723. [33] R. Chaput, J. Duval, O. Boissier, M. Guillermin, S. Hassas, A multi-agent approach to combine reasoning and learning for an ethical behavior, in: M. Fourcade, B. Kuipers, S. Lazar, D. K. Mulligan (Eds.), AIES ’21: AAAI/ACM Conference on AI, Ethics, and Society, Virtual Event, USA, May 19-21, 2021, ACM, USA, 2021, pp. 13–23. doi:10.1145/3461702.3462515. [34] K. D. Ashley, B. M. McLaren, Reasoning with reasons in case-based comparisons, in: Case- Based Reasoning Research and Development, First International Conference, ICCBR-95, Sesimbra, Portugal, October 23-26, 1995, Proceedings, volume 1010 of Lecture Notes in Computer Science, Springer, USA, 1995, pp. 133–144. URL: https://doi.org/10.1007/3-540-60598-3. [35] B. M. McLaren, K. D. Ashley, Case representation, acquisition, and retrieval in SIROCCO, in: Case- Based Reasoning and Development, Third International Conference, ICCBR-99, Seeon Monastery, Germany, July 27-30, 1999, Proceedings, volume 1650 of Lecture Notes in Computer Science, Springer, USA, 1999, pp. 248–262. URL: https://doi.org/10.1007/3-540-48508-2. [36] J. Dancy, Can a particularist learn the difference between right and wrong?, in: The proceedings of the twentieth world congress of philosophy, volume 1, 1999, pp. 59–72. [37] M. Guarini, Particularism and the classification and reclassification of moral cases, IEEE Intelligent Systems 21 (2006) 22–28. URL: https://doi.org/10.1109/MIS.2006.76. doi:10.1109/MIS.2006.76. [38] D. Abel, J. MacGlashan, M. L. Littman, Reinforcement learning as a framework for ethical decision making, in: AI, Ethics, and Society, Papers from the 2016 AAAI Workshop, Phoenix, Arizona, USA, February 13, 2016, volume WS-16-02 of AAAI Technical Report, AAAI Press, USA, 2016. [39] V. Conitzer, W. Sinnott-Armstrong, J. S. Borg, Y. Deng, M. Kramer, Moral decision making frameworks for artificial intelligence, in: The Workshops of the The Thirty-First AAAI Conference on Artificial Intelligence, Saturday, February 4-9, 2017, San Francisco, California, USA, AAAI Workshops, AAAI Press, USA, 2017. [40] M. Anderson, S. L. Anderson, C. Armen, Medethex: Toward a medical ethics advisor, in: Caring Machines: AI in Eldercare, Papers from the 2005 AAAI Fall Symposium, Arlington, Virginia, USA, November 4-6, 2005., volume FS-05-02 of AAAI Technical Report, AAAI Press, USA, 2005, pp. 9–16. URL: https://www.aaai.org/Library/Symposia/Fall/fs05-02.php. [41] M. Anderson, S. L. Anderson, ETHEL: toward a principled ethical eldercare system, in: AI in Eldercare: New Solutions to Old Problems, Papers from the 2008 AAAI Fall Symposium, Arlington, Virginia, USA, November 7-9, 2008, volume FS-08-02 of AAAI Technical Report, AAAI, USA, 2008, pp. 4–11. URL: http://www.aaai.org/Library/Symposia/Fall/fs08-02.php. [42] A. Dyoub, S. Costantini, F. A. Lisi, Learning domain ethical principles from interactions with users, Digital Society 1 (2022) 28. doi:10.1007/s44206-022-00026-y. [43] A. Dyoub, S. Costantini, F. A. Lisi, Learning answer set programming rules for ethical machines, in: A. Casagrande, E. G. Omodeo (Eds.), Proceedings of the 34th Italian Conference on Computational Logic, Trieste, Italy, June 19-21, 2019, volume 2396 of CEUR Workshop Proceedings, CEUR-WS.org, 2019, pp. 300–315. URL: http://ceur-ws.org/Vol-2396/paper14.pdf. [44] A. Dyoub, S. Costantini, F. A. Lisi, I. Letteri, Logic-based machine learning for transparent ethical agents, in: F. Calimeri, S. Perri, E. Zumpano (Eds.), Proceedings of the 35th Italian Conference on Computational Logic - CILC 2020, Rende, Italy, October 13-15, 2020, volume 2710 of CEUR Workshop Proceedings, CEUR-WS.org, Germany, 2020, pp. 169–183. URL: http: //ceur-ws.org/Vol-2710/paper11.pdf. [45] A. Dyoub, S. Costantini, F. A. Lisi, Towards an ILP application in machine ethics, in: Inductive Logic Programming - 29th International Conference, ILP 2019, Plovdiv, Bulgaria, September 3-5, 2019, Proceedings, volume 11770 of Lecture Notes in Computer Science, Springer, Netherlands, 2019, pp. 26–35. doi:10.1007/978-3-030-49210-6. [46] R. C. Arkin, Governing lethal behavior: embedding ethics in a hybrid deliberative/reactive robot architecture, in: Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction, HRI 2008, Amsterdam, The Netherlands, March 12-15, 2008, ACM, USA, 2008, pp. 121–128. [47] A. K. Mackworth, Architectures and ethics for robots: constraint satisfaction as a unitary design framework, Mach Ethics 30 (2011) 335. [48] E. Falletti, C. Gallese, Ethical and legal limits to the diffusion of self-produced autono-mous weapons, Social Science Research Network: SSRN (2022) 22–28. [49] M. Pontier, J. F. Hoorn, Toward machines that behave ethically better than humans do, in: Proceedings of the 34th Annual Meeting of the Cognitive Science Society, CogSci 2012, Sapporo, Japan, August 1-4, 2012, volume 34, cognitivesciencesociety.org, Seattle, USA, 2012. URL: https: //mindmodeling.org/cogsci2012/papers/0383/index.html. [50] S. Tolmeijer, M. Kneer, C. Sarasua, M. Christen, A. Bernstein, Implementations in machine ethics: A survey, CoRR abs/2001.07573 (2020). URL: https://arxiv.org/abs/2001.07573. [51] C.-C. Lee, Fuzzy logic in control systems: fuzzy logic controller. i, IEEE Trans. Syst. Man Cybern. 20 (1990) 404–418. URL: https://api.semanticscholar.org/CorpusID:38662846.