=Paper=
{{Paper
|id=Vol-3762/551
|storemode=property
|title=On Representing Humans' Soft-Ethics Preferences As Dispositions
|pdfUrl=https://ceur-ws.org/Vol-3762/551.pdf
|volume=Vol-3762
|authors=Donatella Donati,Ziba Assadi,Simone Gozzano,Paola Inverardi,Nicolas Troquard
|dblpUrl=https://dblp.org/rec/conf/ital-ia/DonatiAGIT24
}}
==On Representing Humans' Soft-Ethics Preferences As Dispositions==
https://ceur-ws.org/Vol-3762/551.pdf
On Representing Humans’ Soft-Ethics Preferences As
Dispositions
Donatella Donati1,∗ , Ziba Assadi2 , Simone Gozzano1 , Paola Inverardi2 and Nicolas Troquard2
1 University of L’Aquila (UNIVAQ), L’Aquila, Italy
2 Gran Sasso Science Institute (GSSI), L’Aquila, Italy
Abstract
The aim of this paper is to represent humans’ soft-ethical preferences by means of dispositional properties. We begin by
examining real-life situations, termed as scenarios, that involve ethical dilemmas. Users engage with these scenarios, making
decisions on how to act and providing justifications for their choices. We adopt a dispositional approach to represent these
scenarios and the interaction with the users. Dispositions are properties that are instantiated by any kind of entity and that may
manifest if properly triggered. In particular, the dispositional properties we are interested in are the ethical and behavioural ones.
The approach will be described by means of examples. The ultimate goal is to implement the results of this work into a software
exoskeleton solution aimed at augmenting human capabilities by preserving their soft-ethical preferences in interactions with
autonomous systems.
Keywords
ethics, moral preferences, software, dispositions
1. Introduction be done over and above the existing regu-
lation, not against it, or despite its scope,
The constant growth of interaction between human and ar- or to change it, or to by-pass it (e.g. in
tificial agents poses ethical challenges for our society. The terms of self-regulation). In other words,
autonomy that intelligent systems are increasingly acquir- soft ethics is post-compliance ethics: in
ing allows human agents to delegate tasks and decisions this case, ‘ought implies may’.
to them. This delegation is, prima facie, very convenient.
Nevertheless, it deprives human beings of one of their It is therefore crucial to collect and represent indi-
most defining ethical aspects: their autonomy. vidual soft ethics. In [2] it has been shown that it is
To contrast this situation, approaches that try to em- possible to collect excerpts of people’s moral and be-
power humans in their interactions with autonomous ma- havioural preferences from their responses to a question-
chines are sought. In this direction, we are interested naire. Roughly, they developed a questionnaire composed
in building personalised software solutions that allow an of thirteen morally-loaded scenarios describing a context
ethical mediation between human beings and automatic that involves a moral decisions to make. The user is then
systems. That is, we want individuals’ moral and be- asked whether they would or would not undertake a cer-
havioural preferences to be respected in the course of tain action in that given context, and to justify their reply
interactions that have moral significance. We are there- by assigning a value from 1 to 5 to four different parame-
fore in the domain of soft ethics. Clearly, respect of the ters.
norms and accepted procedures is taken for granted and
absorbed in the so-called “hard ethics”. Hard ethics is A dispositional and behaviourist approach. In this
what may contribute to making or shaping the law. To paper, starting from the questionnaire, we aim at construct-
make the difference between hard ethics and soft ethics ing a tentative model showing how this users’ feedback
clearer, consider this quote from [1]: can help in capturing users’ soft ethics. The model we
propose represents individual soft ethics as dispositions
Soft ethics covers the same normative that, as explained in the following section, are well suited
ground as hard ethics, but it does so by to capture the contextual nature of soft ethics.
considering what ought and ought not to Dispositionality can be acquired and probed through
Ital-IA 2024: 4th National Conference on Artificial Intelligence, experience [3]. This is analogous to the behaviourist
organized by CINI, May 29-30, 2024, Naples, Italy approach to learning agents’ utilities in decision theory,
∗ Corresponding author.
where preferences are revealed by one’s choices [4]: an
$ donatella.donati@univaq.it (D. Donati); ziba.assadi@gssi.it agent prefers x to y if and only if they choose x over y
(Z. Assadi); simone.gozzano@univaq.it (S. Gozzano);
whenever given the opportunity. In our study, the question-
paola.inverardi@univaq.it (P. Inverardi); nicolas.troquard@gssi.it
(N. Troquard) naire is the probing method to elicit moral dispositions.
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Through experience, one can build the ethical profile of
Attribution 4.0 International (CC BY 4.0).
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
an agent. This moral profile would be akin to a repertoire situation, the courageous individual would face the chal-
of (dispositional) rules indicating what action the agent lenges.”
would tend to take in a given context. Another tenet of dispositionalism is that dispositions
are gradable properties: a thin glass is more fragile that
Outline. We provide an overview of what dispositions a sturdy vase, gasoline is more flammable than wood,
are in Section 2. In Section 3 we present the questionnaire some people are more courageous than others, etc. The
of [2] and a clear identification of the pieces of informa- dimension per se is what Vetter calls potentiality, that is
tion in the scenarios and the human agents’ feedback. In the fact something may manifest shattering, combustion
Section 4, we specify what we may call a ‘moral oracle’ or facing challenges. Depending on the context, types
which is used as a step for eliciting soft-ethics preferences of objects differentiate themselves, for instance, in their
from the existing questionnaire and feedback. The instru- fragility. Fragility is a determinate that is manifesting
mental role delegated to this oracle motivates the future more or less easily the breakability (a neutral potentiality)
work, which is presented in a conclusion in Section 5. of objects. This could be understood as a variation along
the determinable-determinate dimension. There are many
ways in which something may manifest breakability or
2. Dispositions combustion, so such manifestation could be further de-
termined in nuances of fragility and inflammability. The
Dispositionalism is a philosophical theory of properties. specific way in which, for example, gasoline manifests
According to this theory, properties are potentialities of combustion is its determinate way, which is different from
the objects that instantiate them: e.g., the fragility of glass, the way in which wood manifests combustion.
the solubility of a sugar cube, and the bravery of an indi- There are various theories about dispositions and dif-
vidual. Fragility, solubility and bravery are potentialities ferent versions of those theories. However, for the project
that dispose the entities instantiating them to exhibit par- at hand, using this standard conception is enough. This
ticular behaviours under specific circumstances. The glass minimal version of dispositionalism is already helpful in
is disposed to break if dropped on a hard surface, the sugar representing the soft-ethical preferences of individuals.
cube is disposed to dissolve if immersed in a cup of hot An attempt to connect ethics and dispositions has been
tea, and the courageous person is disposed to face chal- made before in [6].
lenges in a dangerous situation. Dispositional properties The questionnaire is the method to elicit the soft-ethical
are modal in nature, which means that they individuate dispositions of human agents. The next section presents it
potential behaviours of the entities possessing them, that in detail.
is, what those entities could do within a given context.
We can summarise all this with two claims that represent
what Vetter calls “standard conception of dispositions”; 3. A formal analysis of the
in her own words [5]: questionnaire’s scenarios
1. A disposition is individuated by the pair of its
stimulus condition and its manifestation (or, if it This section presents the questionnaire and the scenarios
is a multi-track disposition, by several such pairs): that compose it, what human agents’ feedback about the
it is a disposition to M when S (or a disposition scenarios is made of, and what can be inferred from all
to M1 when S1, to M2 when S2, etc., if it is a this. As anticipated in the introduction, the questionnaire
mutli-track disposition). in [2] is made of scenarios. A human agent provides
2. Its modal nature is, in some way or another, linked feedback by answering to the questionnaire one scenario
to or best characterised (to a first approximation) at a time. We report here two of those scenarios that will
by a counterfactual conditional “if x were S, x be used in the paper.
would M” (or if it is a multi-track disposition, by Scenario 1. As I am about to leave the post office, the
several such conditionals). queue-eliminating machine breaks down. A messy line is
Let us clarify with an example: the courage of the in- forming, and a clerk starts hand-writing numbered cards
dividual (disposition D) is individuated by the pair of its for people coming in. Do I stop and help him? Let us call
stimulus condition that is the dangerous situation (S) and this scenario postoffice.
its manifestation that is the facing of the challenges by
the individual (M). The relation between the disposition, Scenario 2. There are trees with ripe fruit in a private
the stimulus and the manifestation can be, roughly, in- park with private access. The gate is open and there are
dividuated by the following counterfactual conditional: no people around. Do I go in and steal some? Let us call
“if the courageous individual were placed in a dangerous this scenario fruits.
Action(fruits)?
Response(a, fruits)
Justification(a, fruits)
⏞ ⏟⏟ ⏞
feedback
fruits agent a no elicited
⏞ ⏟⏟ ⏞
disposition
false
a is law defying
‘yes’
soundj(fruits, Response(a, fruits), Response(a, fruits)?
Justification(a, fruits)))?
true
a is law abiding
‘no’
Figure 1: From scenarios to soft ethics. Depiction of Scenario 2, called fruits, with some elements of formalisation, and
commented in more details in Example 4.
After answering the questions of the scenarios by ‘yes’ decision maker from whom the soft ethics is revealed
or ‘no’, the subjects are asked to justify their answer by through their answers to the questionnaire. Then we have
assigning values to four parameters. The parameters used the scenario, which is the central concept. A scenario s is
in [2] are viewed as fundamental principles upon which made of:
ethics theories are usually constructed. The four parame-
• A setting S ETTING(s) which is a description in
ters are as follows:
natural language of the setting of the scenario s.
p1 How much did the potential consequences of the • A problem P ROBLEM(s) which is a description in
action on others weigh on my choice? natural language of the problem of the scenario s.
• An action ACTION(s) which is a description in
p2 How much did the potential consequences of the natural language of an hypothetical action that the
action on me weigh on my choice? human agent might perform of not.
p3 How much did my personal experiences weigh on The set of scenarios is noted Scenarios.
my choice?
Example 1. Consider Scenario 1. We have the setting
p4 How much did respect for the law weigh on my S ETTING(postoffice) which is “As I am about to leave
choice? the post office, the queue-eliminating machine breaks
down.”, the problem P ROBLEM(postoffice) which is “A
Since we are interested in an overall representation of messy line is forming and a clerk starts hand-writing
the soft-ethics preferences, we make a particular effort at numbered cards for people coming in.”, and the action
extracting the concepts and the relations among them that ACTION(postoffice) which is “stop and help him”.
are involved in the more informal presentation from [2]. All the provided information can be interpreted as stim-
The first concept in the domain of the questionnaire uli. That is, as properties that may trigger some disposi-
is that of a human agent a. It is typically an individual tion of the individual. S ETTING(postoffice) provides the
property of a state-based disposition “readiness to leave”, Scenarios
and “machine broken”. P ROBLEM(postoffice) provides
the properties “messy line forming”, and “clerk hand- p3 p2
writes numbers and is needing help”. ACTION(postoffice)
provides the action “stop and help”.
postoffice
The properties of a scenario, and the moral and be- fruits
havioural properties of the agent are stimuli-disposition p4 p1
partners, as bearers of properties that may reveal them-
selves by interacting with each other. Once the agent is
in the given setting, their dispositions, in the form of po-
tential behaviours, are triggered by the properties of the Figure 2: Categories of scenarios.
overall scenario: the setting, the problem, and the action.
A scenario is qualified with the help of a set of param-
eters Params . As in the questionnaire of [2] which in-
forms our study, we are interested in the social and ethical personal experiences: It takes values on an interval
domain. The questionnaire uses the set {p1 , p2 , p3 , p4 } scale of expertness over a pragmatism dimension.
to justify the actions in the scenarios. However, we pre- p4 refers to the human agent’s consideration about the
fer to reformulate the wording of the parameters. For, as law: It takes values on an interval scale of obedi-
we said in Section 2, dispositions are gradable properties. ence over a legality dimension.
For instance, p1 is about whether one is willing to help
others. As an extreme case, one in which the subject It should be further noticed that a scenario s may stress
assigns the top value (5) to the parameter, it reveals the one or more of the parameters in Params . The set
user’s altruistic disposition. Moreover, in order for an P RESS(s) ⊆ Params is the set of parameters that the sce-
action to be altruistic, the action should be considered nario s puts pressure on.
positive. So, to satisfy gradability each parameter must Example 2. Consider Scenario 1. The scenario
run along a determinate-determinable dimension. (This is postoffice presses the parameter about the consequences
analogous to a physical object parameter running along of the action on others. Hence, P RESS(postoffice) =
a breakable–fragility dimension, where the ‘fragility’ is {p1 }.
determinate and ‘breakability’ is a determinable dimen-
sion.) With respect to parameter p1 that would be good This is typically intended and determined by the de-
willingness–altruism; with respect to parameter p2 we signer of the scenario. But this could also be determined
propose self-servingness–egoism; as to parameter p3 we experimentally if need be.
propose pragmatism–expertness; finally, parameter p4 is The human agents’ feedback on a scenario uses an
legality–obedience. So, we may re-phrase the parameter interval scale from 1 to 5: Scale = {1, 2, 3, 4, 5}.
by adding also the positivity of the action in the first two A feedback f on a scenario s provided by a human
parameters, those that are other-regarding. By “positive agent a is made of:
effect” we notice that this should be from the point of • a response R ESPONSE(a, s) ∈ {yes, no} indicating
view of the other human agents. Moreover, the action whether the human agent a would perform action
should not be taken for granted from all parties (so it is ACTION(s) if confronted to scenario s.
not obvious that the action is going to be performed) and,
• a justification J USTIFICATION(a, s) ∈
being positive means that they are desirable from other
ScaleParams , where the integer value
human agents’ point of view.
J USTIFICATION(a, s)(pi ) indicates the level
Summing up:
of relevance of pi for human agent a in choosing
p1 refers to the human agent’s consideration about the R ESPONSE(a, s).1
positive effect of their action on others. It takes The category C ATEGORY(s) ⊆ Scenarios of scenario s
values on an interval scale of altruism over a good- is the set of scenarios s′ such that P RESS(s) = P RESS(s′ ).
will dimension. The 16 categories of scenarios can be visualised with the
p2 refers to the human agent’s consideration about the Venn’s 4-set diagram represented on Figure 2.
positive effect of their action on themselves: It Each category is thus intended as an abstraction of a
takes values on an interval scale of egoism over a scenario. A soft-ethics preference elicited from a sce-
self-servingness dimension. nario is intended to apply to all scenarios belonging to the
1We use the standard notation where X Y denotes the set of functions
p3 refers to the human agent’s consideration about their from set Y to set X.
same category. This is the primary mechanism to handle • SOUNDJ(s, R ESPONSE(a, s), J USTIFICATION(a, s))
new situations encountered by human agents. Elicited is true,
dispositions are then to be implemented into an ethical • SOUNDJ(s, R ESPONSE(b, s), J USTIFICATION(b, s))
software profile that augments human capabilities by pre- is true,
serving their soft-ethical preferences in interactions with • SOUNDJ(s, R ESPONSE(c, s), J USTIFICATION(c, s))
autonomous systems. is false,
• SOUNDJ(s, R ESPONSE(d, s), J USTIFICATION(d, s))
is false.
4. A moral oracle
In the previous example, a ‘yes’ answer has an ethically
Before we can elicit a dispositional soft-ethics preference
‘positive´ connotation. This is in contrast with the next
from a feedback, we will eventually need a mechanism
example.
to decide whether a scenario and a feedback follow a
certain consistency. For now, we treat this mechanism as Example 4. Let us consider fruits Scenario 2, also
an oracle, SOUNDJ, that stands for “sound justification”. depicted on Figure 1. Agent a is considering en-
The difficulty resides in analysing formally the scenarios tering the private park and stealing a fruit. The
as described in [2], and the ’direction’ of the actions. E.g., scenario presses on parameter p4 , that is, the le-
in Scenario 1, an answer ‘yes’ has a positive overtone, gality of the action. If R ESPONSE(a, fruits) is ‘yes’
while in Scenario 2, answer ‘no’ has a negative overtone. and J USTIFICATION(a, fruits) gives a high value to p4 ,
We only start specifying what this mechanism should do. SOUNDJ(. . .) is false, and then we cannot elicit any
Let us consider a scenario s, that presses on the pa- disposition. Instead, if R ESPONSE(a, fruits) is ‘yes’
rameters P RESS(s), and includes the action ACTION(s) and J USTIFICATION(a, fruits) gives a low value to p4 ,
that can or cannot be performed. Let us also consider SOUNDJ(. . .) is true, and since R ESPONSE(a, fruits) is
a human agent a and a’s feedback that includes the ‘yes’, we can elicit the disposition of agent a to be law
answer yes or no R ESPONSE(s) and the justification defying.
J USTIFICATION(a, s) in terms of parameters values. Re-
member that if R ESPONSE(a, s) is yes, the agent takes ac- The SOUNDJ function thus occupies an in-
tion ACTION(a), and if R ESPONSE(a, s) is no, the agent strumental role in our soft-ethics preferences
does not take action ACTION(a). from questionnaire feedback. Before elicit-
We can define the boolean function ing a disposition, SOUNDJ (s, R ESPONSE (x, s),
SOUNDJ (s, R ESPONSE (a, s), J USTIFICATION (a, s))) J USTIFICATION(x, s)) filters out the responses by a
which captures the judgement about whether the human agent x that are not consistent with the intended
justification is sound with respect to the action taken in meaning of the scenario s.
the scenario s by the human agent a. For the time being, we assume the existence and com-
putability of this function. As it may appear clear, the
Example 3. For example let us consider again postoffice actual implementation of the function must account for
from Scenario 1 (post office). Remember that the scenario a nuanced setting of the parameters, and some informa-
presses on p1 , that is, good-willingness. Let us assume tion about the ‘direction´ of the action in a scenario. We
that:2 discuss future work related to the function SOUNDJ in the
next section.
• agent a helps the clerk (R ESPONSE(a, s) = ‘yes′ )
with justification (4, _, _, _),
• agent b does not help the clerk 5. Outlook
(R ESPONSE(b, s) = ‘no′ ) with justification
(1, _, _, _), We clarified the ontology of the questionnaire of [2].
• agent c does help the clerk (R ESPONSE(c, s) = Guided by a pre-formalisation, we have also proposed
‘yes′ ) with justification (1, _, _, _), how the empirical data collected through this question-
• agent d does not help the clerk naire permits to elicit the feedback from the subjects into
(R ESPONSE(d, s) = ‘no′ ) with justification soft-ethics preferences. To this end, we have adopted a
(4, _, _, _). behavioural approach. Furthermore, we have argued for a
dispositional perspective of these soft-ethics preferences.
Then The work done so far has permitted us to identify the
necessary pieces of information present in a scenario and
2 The placeholder value _ indicates that the exact value does not matter. in the feedback to derive a soft-ethics preference. Nonethe-
We suppose that the oracle takes 1 as a low value and 4 as a high less, we found a stumbling block, inasmuch that those are
value. not sufficient. We have indeed recourse to an oracle to
inform us about the soundness of the feedback with a Autonomous-Agent Rules, Minds Mach. 32 (2022)
given scenario. This is the first natural course of action 683–715.
for future work. [8] N. Troquard, M. De Sanctis, P. Inverardi, P. Pellic-
cione, G. L. Scoccia, Social, Legal, Ethical, Empa-
Future work. We plan to work on an concrete imple- thetic, and Cultural Rules: Compilation and Reason-
mentation of SOUNDJ function. Working with existing ing, in: Thirty-Eighth AAAI Conference on Artifi-
questionnaires, we will need methods to extract the rele- cial Intelligence, AAAI 2024, Thirty-Sixth Confer-
vant pieces of information from scenario written in natural ence on Innovative Applications of Artificial Intel-
language. This includes understanding the ‘direction’ of ligence, IAAI 2024, Fourteenth Symposium on Ed-
the action, whether either a ‘yes’ or ‘no’ should be con- ucational Advances in Artificial Intelligence, EAAI
sidered a ‘positive’ action. 2024, AAAI Press, 2024, pp. 22385–22392.
We also envisage that missing information from the [9] L. A. Zadeh, A computational theory of disposi-
questionnaire could be easily filled in by the designers. tions, in: I. B. Turksen, K. Asai, G. Ulusoy (Eds.),
In a future iteration of this work, we anticipate making Computer Integrated Manufacturing, Springer Berlin
recommendations on how to design a questionnaire with Heidelberg, Berlin, Heidelberg, 1988, pp. 215–241.
additional data. This would enable the fully automated
elicitation of feedback for soft-ethics preferences.
Another perspective for future work lies in developing
a formal language to represent the soft-ethics preferences
elicited from such a questionnaire. It could be an adap-
tation of so-called SLEEC rules [7] to personal ethics,
formalised along the ideas presented in [8]. We anticipate
that classical logic might be too coarse to capture their
dispositional nature. Instead, we will explore the use of
probabilistic rules or fuzzy logic [9].
Finally, we want to use the gathered preferences as
dispositions to create a software profile that enhances
human abilities by respecting their ethical choices when
they interact with autonomous systems.
References
[1] L. Floridi, Soft ethics and the governance of the
digital, Philosohy & Technology (2018).
[2] C. Alfieri, D. Donati, S. Gozzano, L. Greco,
M. Segala, Ethical preferences in the digital world:
The EXOSOUL questionnaire, in: HHAI 2023: Aug-
menting Human Intellect - Proceedings of the Second
International Conference on Hybrid Human-Artificial
Intelligence, volume 368 of Frontiers in Artificial
Intelligence and Applications, IOS Press, 2023, pp.
290–299.
[3] S. Mumford, R. L. Anjum, Lebenswelt und Wis-
senschaft, 2011, pp. 380–394.
[4] M. Peterson, An Introduction to Decision Theory,
Cambridge University Press, 2009.
[5] B. Vetter, Potentiality: From Dispositions to Modal-
ity, Oxford University Press, Oxford, England and
New York, NY, USA, 2015.
[6] R. L. Anjum, S. A. N. Lie, S. Mumford, Powers and
Capacities in Philosophy, Routledge, 2012.
[7] B. A. Townsend, C. Paterson, T. T. Arvind, G. Ne-
mirovsky, R. Calinescu, A. Cavalcanti, I. Habli,
A. Thomas, From Pluralistic Normative Principles to