On Representing Humans’ Soft-Ethics Preferences As Dispositions Donatella Donati1,∗ , Ziba Assadi2 , Simone Gozzano1 , Paola Inverardi2 and Nicolas Troquard2 1 University of L’Aquila (UNIVAQ), L’Aquila, Italy 2 Gran Sasso Science Institute (GSSI), L’Aquila, Italy Abstract The aim of this paper is to represent humans’ soft-ethical preferences by means of dispositional properties. We begin by examining real-life situations, termed as scenarios, that involve ethical dilemmas. Users engage with these scenarios, making decisions on how to act and providing justifications for their choices. We adopt a dispositional approach to represent these scenarios and the interaction with the users. Dispositions are properties that are instantiated by any kind of entity and that may manifest if properly triggered. In particular, the dispositional properties we are interested in are the ethical and behavioural ones. The approach will be described by means of examples. The ultimate goal is to implement the results of this work into a software exoskeleton solution aimed at augmenting human capabilities by preserving their soft-ethical preferences in interactions with autonomous systems. Keywords ethics, moral preferences, software, dispositions 1. Introduction be done over and above the existing regu- lation, not against it, or despite its scope, The constant growth of interaction between human and ar- or to change it, or to by-pass it (e.g. in tificial agents poses ethical challenges for our society. The terms of self-regulation). In other words, autonomy that intelligent systems are increasingly acquir- soft ethics is post-compliance ethics: in ing allows human agents to delegate tasks and decisions this case, ‘ought implies may’. to them. This delegation is, prima facie, very convenient. Nevertheless, it deprives human beings of one of their It is therefore crucial to collect and represent indi- most defining ethical aspects: their autonomy. vidual soft ethics. In [2] it has been shown that it is To contrast this situation, approaches that try to em- possible to collect excerpts of people’s moral and be- power humans in their interactions with autonomous ma- havioural preferences from their responses to a question- chines are sought. In this direction, we are interested naire. Roughly, they developed a questionnaire composed in building personalised software solutions that allow an of thirteen morally-loaded scenarios describing a context ethical mediation between human beings and automatic that involves a moral decisions to make. The user is then systems. That is, we want individuals’ moral and be- asked whether they would or would not undertake a cer- havioural preferences to be respected in the course of tain action in that given context, and to justify their reply interactions that have moral significance. We are there- by assigning a value from 1 to 5 to four different parame- fore in the domain of soft ethics. Clearly, respect of the ters. norms and accepted procedures is taken for granted and absorbed in the so-called “hard ethics”. Hard ethics is A dispositional and behaviourist approach. In this what may contribute to making or shaping the law. To paper, starting from the questionnaire, we aim at construct- make the difference between hard ethics and soft ethics ing a tentative model showing how this users’ feedback clearer, consider this quote from [1]: can help in capturing users’ soft ethics. The model we propose represents individual soft ethics as dispositions Soft ethics covers the same normative that, as explained in the following section, are well suited ground as hard ethics, but it does so by to capture the contextual nature of soft ethics. considering what ought and ought not to Dispositionality can be acquired and probed through Ital-IA 2024: 4th National Conference on Artificial Intelligence, experience [3]. This is analogous to the behaviourist organized by CINI, May 29-30, 2024, Naples, Italy approach to learning agents’ utilities in decision theory, ∗ Corresponding author. where preferences are revealed by one’s choices [4]: an $ donatella.donati@univaq.it (D. Donati); ziba.assadi@gssi.it agent prefers x to y if and only if they choose x over y (Z. Assadi); simone.gozzano@univaq.it (S. Gozzano); whenever given the opportunity. In our study, the question- paola.inverardi@univaq.it (P. Inverardi); nicolas.troquard@gssi.it (N. Troquard) naire is the probing method to elicit moral dispositions. © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Through experience, one can build the ethical profile of Attribution 4.0 International (CC BY 4.0). CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings an agent. This moral profile would be akin to a repertoire situation, the courageous individual would face the chal- of (dispositional) rules indicating what action the agent lenges.” would tend to take in a given context. Another tenet of dispositionalism is that dispositions are gradable properties: a thin glass is more fragile that Outline. We provide an overview of what dispositions a sturdy vase, gasoline is more flammable than wood, are in Section 2. In Section 3 we present the questionnaire some people are more courageous than others, etc. The of [2] and a clear identification of the pieces of informa- dimension per se is what Vetter calls potentiality, that is tion in the scenarios and the human agents’ feedback. In the fact something may manifest shattering, combustion Section 4, we specify what we may call a ‘moral oracle’ or facing challenges. Depending on the context, types which is used as a step for eliciting soft-ethics preferences of objects differentiate themselves, for instance, in their from the existing questionnaire and feedback. The instru- fragility. Fragility is a determinate that is manifesting mental role delegated to this oracle motivates the future more or less easily the breakability (a neutral potentiality) work, which is presented in a conclusion in Section 5. of objects. This could be understood as a variation along the determinable-determinate dimension. There are many ways in which something may manifest breakability or 2. Dispositions combustion, so such manifestation could be further de- termined in nuances of fragility and inflammability. The Dispositionalism is a philosophical theory of properties. specific way in which, for example, gasoline manifests According to this theory, properties are potentialities of combustion is its determinate way, which is different from the objects that instantiate them: e.g., the fragility of glass, the way in which wood manifests combustion. the solubility of a sugar cube, and the bravery of an indi- There are various theories about dispositions and dif- vidual. Fragility, solubility and bravery are potentialities ferent versions of those theories. However, for the project that dispose the entities instantiating them to exhibit par- at hand, using this standard conception is enough. This ticular behaviours under specific circumstances. The glass minimal version of dispositionalism is already helpful in is disposed to break if dropped on a hard surface, the sugar representing the soft-ethical preferences of individuals. cube is disposed to dissolve if immersed in a cup of hot An attempt to connect ethics and dispositions has been tea, and the courageous person is disposed to face chal- made before in [6]. lenges in a dangerous situation. Dispositional properties The questionnaire is the method to elicit the soft-ethical are modal in nature, which means that they individuate dispositions of human agents. The next section presents it potential behaviours of the entities possessing them, that in detail. is, what those entities could do within a given context. We can summarise all this with two claims that represent what Vetter calls “standard conception of dispositions”; 3. A formal analysis of the in her own words [5]: questionnaire’s scenarios 1. A disposition is individuated by the pair of its stimulus condition and its manifestation (or, if it This section presents the questionnaire and the scenarios is a multi-track disposition, by several such pairs): that compose it, what human agents’ feedback about the it is a disposition to M when S (or a disposition scenarios is made of, and what can be inferred from all to M1 when S1, to M2 when S2, etc., if it is a this. As anticipated in the introduction, the questionnaire mutli-track disposition). in [2] is made of scenarios. A human agent provides 2. Its modal nature is, in some way or another, linked feedback by answering to the questionnaire one scenario to or best characterised (to a first approximation) at a time. We report here two of those scenarios that will by a counterfactual conditional “if x were S, x be used in the paper. would M” (or if it is a multi-track disposition, by Scenario 1. As I am about to leave the post office, the several such conditionals). queue-eliminating machine breaks down. A messy line is Let us clarify with an example: the courage of the in- forming, and a clerk starts hand-writing numbered cards dividual (disposition D) is individuated by the pair of its for people coming in. Do I stop and help him? Let us call stimulus condition that is the dangerous situation (S) and this scenario postoffice. its manifestation that is the facing of the challenges by the individual (M). The relation between the disposition, Scenario 2. There are trees with ripe fruit in a private the stimulus and the manifestation can be, roughly, in- park with private access. The gate is open and there are dividuated by the following counterfactual conditional: no people around. Do I go in and steal some? Let us call “if the courageous individual were placed in a dangerous this scenario fruits. Action(fruits)? Response(a, fruits) Justification(a, fruits) ⏞ ⏟⏟ ⏞ feedback fruits agent a no elicited ⏞ ⏟⏟ ⏞ disposition false a is law defying ‘yes’ soundj(fruits, Response(a, fruits), Response(a, fruits)? Justification(a, fruits)))? true a is law abiding ‘no’ Figure 1: From scenarios to soft ethics. Depiction of Scenario 2, called fruits, with some elements of formalisation, and commented in more details in Example 4. After answering the questions of the scenarios by ‘yes’ decision maker from whom the soft ethics is revealed or ‘no’, the subjects are asked to justify their answer by through their answers to the questionnaire. Then we have assigning values to four parameters. The parameters used the scenario, which is the central concept. A scenario s is in [2] are viewed as fundamental principles upon which made of: ethics theories are usually constructed. The four parame- • A setting S ETTING(s) which is a description in ters are as follows: natural language of the setting of the scenario s. p1 How much did the potential consequences of the • A problem P ROBLEM(s) which is a description in action on others weigh on my choice? natural language of the problem of the scenario s. • An action ACTION(s) which is a description in p2 How much did the potential consequences of the natural language of an hypothetical action that the action on me weigh on my choice? human agent might perform of not. p3 How much did my personal experiences weigh on The set of scenarios is noted Scenarios. my choice? Example 1. Consider Scenario 1. We have the setting p4 How much did respect for the law weigh on my S ETTING(postoffice) which is “As I am about to leave choice? the post office, the queue-eliminating machine breaks down.”, the problem P ROBLEM(postoffice) which is “A Since we are interested in an overall representation of messy line is forming and a clerk starts hand-writing the soft-ethics preferences, we make a particular effort at numbered cards for people coming in.”, and the action extracting the concepts and the relations among them that ACTION(postoffice) which is “stop and help him”. are involved in the more informal presentation from [2]. All the provided information can be interpreted as stim- The first concept in the domain of the questionnaire uli. That is, as properties that may trigger some disposi- is that of a human agent a. It is typically an individual tion of the individual. S ETTING(postoffice) provides the property of a state-based disposition “readiness to leave”, Scenarios and “machine broken”. P ROBLEM(postoffice) provides the properties “messy line forming”, and “clerk hand- p3 p2 writes numbers and is needing help”. ACTION(postoffice) provides the action “stop and help”. postoffice The properties of a scenario, and the moral and be- fruits havioural properties of the agent are stimuli-disposition p4 p1 partners, as bearers of properties that may reveal them- selves by interacting with each other. Once the agent is in the given setting, their dispositions, in the form of po- tential behaviours, are triggered by the properties of the Figure 2: Categories of scenarios. overall scenario: the setting, the problem, and the action. A scenario is qualified with the help of a set of param- eters Params . As in the questionnaire of [2] which in- forms our study, we are interested in the social and ethical personal experiences: It takes values on an interval domain. The questionnaire uses the set {p1 , p2 , p3 , p4 } scale of expertness over a pragmatism dimension. to justify the actions in the scenarios. However, we pre- p4 refers to the human agent’s consideration about the fer to reformulate the wording of the parameters. For, as law: It takes values on an interval scale of obedi- we said in Section 2, dispositions are gradable properties. ence over a legality dimension. For instance, p1 is about whether one is willing to help others. As an extreme case, one in which the subject It should be further noticed that a scenario s may stress assigns the top value (5) to the parameter, it reveals the one or more of the parameters in Params . The set user’s altruistic disposition. Moreover, in order for an P RESS(s) ⊆ Params is the set of parameters that the sce- action to be altruistic, the action should be considered nario s puts pressure on. positive. So, to satisfy gradability each parameter must Example 2. Consider Scenario 1. The scenario run along a determinate-determinable dimension. (This is postoffice presses the parameter about the consequences analogous to a physical object parameter running along of the action on others. Hence, P RESS(postoffice) = a breakable–fragility dimension, where the ‘fragility’ is {p1 }. determinate and ‘breakability’ is a determinable dimen- sion.) With respect to parameter p1 that would be good This is typically intended and determined by the de- willingness–altruism; with respect to parameter p2 we signer of the scenario. But this could also be determined propose self-servingness–egoism; as to parameter p3 we experimentally if need be. propose pragmatism–expertness; finally, parameter p4 is The human agents’ feedback on a scenario uses an legality–obedience. So, we may re-phrase the parameter interval scale from 1 to 5: Scale = {1, 2, 3, 4, 5}. by adding also the positivity of the action in the first two A feedback f on a scenario s provided by a human parameters, those that are other-regarding. By “positive agent a is made of: effect” we notice that this should be from the point of • a response R ESPONSE(a, s) ∈ {yes, no} indicating view of the other human agents. Moreover, the action whether the human agent a would perform action should not be taken for granted from all parties (so it is ACTION(s) if confronted to scenario s. not obvious that the action is going to be performed) and, • a justification J USTIFICATION(a, s) ∈ being positive means that they are desirable from other ScaleParams , where the integer value human agents’ point of view. J USTIFICATION(a, s)(pi ) indicates the level Summing up: of relevance of pi for human agent a in choosing p1 refers to the human agent’s consideration about the R ESPONSE(a, s).1 positive effect of their action on others. It takes The category C ATEGORY(s) ⊆ Scenarios of scenario s values on an interval scale of altruism over a good- is the set of scenarios s′ such that P RESS(s) = P RESS(s′ ). will dimension. The 16 categories of scenarios can be visualised with the p2 refers to the human agent’s consideration about the Venn’s 4-set diagram represented on Figure 2. positive effect of their action on themselves: It Each category is thus intended as an abstraction of a takes values on an interval scale of egoism over a scenario. A soft-ethics preference elicited from a sce- self-servingness dimension. nario is intended to apply to all scenarios belonging to the 1We use the standard notation where X Y denotes the set of functions p3 refers to the human agent’s consideration about their from set Y to set X. same category. This is the primary mechanism to handle • SOUNDJ(s, R ESPONSE(a, s), J USTIFICATION(a, s)) new situations encountered by human agents. Elicited is true, dispositions are then to be implemented into an ethical • SOUNDJ(s, R ESPONSE(b, s), J USTIFICATION(b, s)) software profile that augments human capabilities by pre- is true, serving their soft-ethical preferences in interactions with • SOUNDJ(s, R ESPONSE(c, s), J USTIFICATION(c, s)) autonomous systems. is false, • SOUNDJ(s, R ESPONSE(d, s), J USTIFICATION(d, s)) is false. 4. A moral oracle In the previous example, a ‘yes’ answer has an ethically Before we can elicit a dispositional soft-ethics preference ‘positive´ connotation. This is in contrast with the next from a feedback, we will eventually need a mechanism example. to decide whether a scenario and a feedback follow a certain consistency. For now, we treat this mechanism as Example 4. Let us consider fruits Scenario 2, also an oracle, SOUNDJ, that stands for “sound justification”. depicted on Figure 1. Agent a is considering en- The difficulty resides in analysing formally the scenarios tering the private park and stealing a fruit. The as described in [2], and the ’direction’ of the actions. E.g., scenario presses on parameter p4 , that is, the le- in Scenario 1, an answer ‘yes’ has a positive overtone, gality of the action. If R ESPONSE(a, fruits) is ‘yes’ while in Scenario 2, answer ‘no’ has a negative overtone. and J USTIFICATION(a, fruits) gives a high value to p4 , We only start specifying what this mechanism should do. SOUNDJ(. . .) is false, and then we cannot elicit any Let us consider a scenario s, that presses on the pa- disposition. Instead, if R ESPONSE(a, fruits) is ‘yes’ rameters P RESS(s), and includes the action ACTION(s) and J USTIFICATION(a, fruits) gives a low value to p4 , that can or cannot be performed. Let us also consider SOUNDJ(. . .) is true, and since R ESPONSE(a, fruits) is a human agent a and a’s feedback that includes the ‘yes’, we can elicit the disposition of agent a to be law answer yes or no R ESPONSE(s) and the justification defying. J USTIFICATION(a, s) in terms of parameters values. Re- member that if R ESPONSE(a, s) is yes, the agent takes ac- The SOUNDJ function thus occupies an in- tion ACTION(a), and if R ESPONSE(a, s) is no, the agent strumental role in our soft-ethics preferences does not take action ACTION(a). from questionnaire feedback. Before elicit- We can define the boolean function ing a disposition, SOUNDJ (s, R ESPONSE (x, s), SOUNDJ (s, R ESPONSE (a, s), J USTIFICATION (a, s))) J USTIFICATION(x, s)) filters out the responses by a which captures the judgement about whether the human agent x that are not consistent with the intended justification is sound with respect to the action taken in meaning of the scenario s. the scenario s by the human agent a. For the time being, we assume the existence and com- putability of this function. As it may appear clear, the Example 3. For example let us consider again postoffice actual implementation of the function must account for from Scenario 1 (post office). Remember that the scenario a nuanced setting of the parameters, and some informa- presses on p1 , that is, good-willingness. Let us assume tion about the ‘direction´ of the action in a scenario. We that:2 discuss future work related to the function SOUNDJ in the next section. • agent a helps the clerk (R ESPONSE(a, s) = ‘yes′ ) with justification (4, _, _, _), • agent b does not help the clerk 5. Outlook (R ESPONSE(b, s) = ‘no′ ) with justification (1, _, _, _), We clarified the ontology of the questionnaire of [2]. • agent c does help the clerk (R ESPONSE(c, s) = Guided by a pre-formalisation, we have also proposed ‘yes′ ) with justification (1, _, _, _), how the empirical data collected through this question- • agent d does not help the clerk naire permits to elicit the feedback from the subjects into (R ESPONSE(d, s) = ‘no′ ) with justification soft-ethics preferences. To this end, we have adopted a (4, _, _, _). behavioural approach. Furthermore, we have argued for a dispositional perspective of these soft-ethics preferences. Then The work done so far has permitted us to identify the necessary pieces of information present in a scenario and 2 The placeholder value _ indicates that the exact value does not matter. in the feedback to derive a soft-ethics preference. Nonethe- We suppose that the oracle takes 1 as a low value and 4 as a high less, we found a stumbling block, inasmuch that those are value. not sufficient. We have indeed recourse to an oracle to inform us about the soundness of the feedback with a Autonomous-Agent Rules, Minds Mach. 32 (2022) given scenario. This is the first natural course of action 683–715. for future work. [8] N. Troquard, M. De Sanctis, P. Inverardi, P. Pellic- cione, G. L. Scoccia, Social, Legal, Ethical, Empa- Future work. We plan to work on an concrete imple- thetic, and Cultural Rules: Compilation and Reason- mentation of SOUNDJ function. Working with existing ing, in: Thirty-Eighth AAAI Conference on Artifi- questionnaires, we will need methods to extract the rele- cial Intelligence, AAAI 2024, Thirty-Sixth Confer- vant pieces of information from scenario written in natural ence on Innovative Applications of Artificial Intel- language. This includes understanding the ‘direction’ of ligence, IAAI 2024, Fourteenth Symposium on Ed- the action, whether either a ‘yes’ or ‘no’ should be con- ucational Advances in Artificial Intelligence, EAAI sidered a ‘positive’ action. 2024, AAAI Press, 2024, pp. 22385–22392. We also envisage that missing information from the [9] L. A. Zadeh, A computational theory of disposi- questionnaire could be easily filled in by the designers. tions, in: I. B. Turksen, K. Asai, G. Ulusoy (Eds.), In a future iteration of this work, we anticipate making Computer Integrated Manufacturing, Springer Berlin recommendations on how to design a questionnaire with Heidelberg, Berlin, Heidelberg, 1988, pp. 215–241. additional data. This would enable the fully automated elicitation of feedback for soft-ethics preferences. Another perspective for future work lies in developing a formal language to represent the soft-ethics preferences elicited from such a questionnaire. It could be an adap- tation of so-called SLEEC rules [7] to personal ethics, formalised along the ideas presented in [8]. We anticipate that classical logic might be too coarse to capture their dispositional nature. Instead, we will explore the use of probabilistic rules or fuzzy logic [9]. Finally, we want to use the gathered preferences as dispositions to create a software profile that enhances human abilities by respecting their ethical choices when they interact with autonomous systems. References [1] L. Floridi, Soft ethics and the governance of the digital, Philosohy & Technology (2018). [2] C. Alfieri, D. Donati, S. Gozzano, L. Greco, M. Segala, Ethical preferences in the digital world: The EXOSOUL questionnaire, in: HHAI 2023: Aug- menting Human Intellect - Proceedings of the Second International Conference on Hybrid Human-Artificial Intelligence, volume 368 of Frontiers in Artificial Intelligence and Applications, IOS Press, 2023, pp. 290–299. [3] S. Mumford, R. L. Anjum, Lebenswelt und Wis- senschaft, 2011, pp. 380–394. [4] M. Peterson, An Introduction to Decision Theory, Cambridge University Press, 2009. [5] B. Vetter, Potentiality: From Dispositions to Modal- ity, Oxford University Press, Oxford, England and New York, NY, USA, 2015. [6] R. L. Anjum, S. A. N. Lie, S. Mumford, Powers and Capacities in Philosophy, Routledge, 2012. [7] B. A. Townsend, C. Paterson, T. T. Arvind, G. Ne- mirovsky, R. Calinescu, A. Cavalcanti, I. Habli, A. Thomas, From Pluralistic Normative Principles to