As autonomous systems rapidly become ubiquitous, there is a growing need for a legal and regulatory framework that addresses when and how such a system harms someone. There have been several attempts within the philosophy literature to define harm, but none of them has proven capable of dealing with the many examples that have been presented, leading some to suggest that the notion of harm should be abandoned and “replaced by more well-behaved notions”. As harm is generally something that is caused, most of these definitions have involved causality at some level. Yet surprisingly, none of them makes use of causal models and the definitions of actual causality that they can express. In our full paper [1] we formally define a qualitative notion of harm that uses causal models and is based on a well-known definition of actual causality [2]. The key features of our definition are that it is based on contrastive causation and uses a default utility to which the utility of actual outcomes is compared. We show that our definition is able to handle the examples from the literature, and illustrate its importance for reasoning about situations involving autonomous systems.
The notion that one should not cause harm is a central tenet in many religions; it is enshrined in
the medical profession’s Hippocratic Oath, which states explicitly “I will do no harm or injustice
to [my patients]” [
Unfortunately, when we look at attempts to explain the nature of harm, we find a mess. The most widely discussed account, the comparative account, faces counterexamples that seem fatal. But no alternative account has gained any currency. My diagnosis is that the notion of harm is a Frankensteinian jumble . . . It should be replaced by other more well-behaved notions.
The situation has not improved much since Bradley’s paper (see, e.g., recent accounts like
[
Imagine a UAV used by the military has to decide whether or not it should bomb a suspected enemy encampment. The problem is that the target is not clearly identified, because there are two camps close to each other: one consisting of civilian refugees, another consisting of a rebel group that is about to launch a deadly attack on the refugee camp, killing all of its inhabitants. The UAV’s decision is based only on the expected utility of the refugees, and therefore it bombs the camp. Tragically, as it turns out, the camp was that of the refugees. Here we have the intuition that the UAV harmed these refugees, despite the fact that both actions would have led to all the refugees being killed. Examples in which one event (the bombing) preempts another event (the attack) from causing an outcome are known as Late Preemption examples in the causality literature; we discuss them in the full paper.
In the healthcare domain, autonomous systems are used for, among other things, classifying MRI brain images suspected of containing a tumor. If an image is classified as having a tumor, the system decides whether to recommend a surgery. While the overall accuracy of the system is superior to that of humans, in some instances the system overlooks an operable tumor. Imagine a patient who has such a tumor and dies from brain cancer as the result of not undergoing surgery, leading to a dispute between the patient’s family and the hospital regarding whether the patient was harmed. Even if both parties agree that the patient would probably have been alive if the diagnosis had been performed by a human, the hospital might claim that using the system is the optimal policy, and therefore one should compare the actual outcome only to those that could have occurred under the policy.
Fortunately, the formal tools at our disposal to develop a formal notion of harm have also
improved over the past few years; we take full advantage of these developments in our full
paper [
There is one set of problems that arise from using counterfactuals that also arise with causality,
and can be dealt with using the by-now standard approaches in defining causality. For example,
Carlson, Johansson, and Risberg [
Just applying the definition of causality does not deal with all problems. The other key step
that we take is to assume that there exists a default utility. Roughly speaking, we define an
event to cause harm whenever it causes the utility of the outcome to be lower than the default
utility. The default may be context-dependent, and there may be disagreement about what the
default should be. We view that as a feature of our definition. For example, we can capture the
fact that people disagree about whether a doctor euthanizing a patient in great pain causes harm
by taking it to be a disagreement about what the appropriate default should be. Likewise, the
dispute between the family and the hospital described above can be modeled as a disagreement
about the right default. Moreover, by explicitly bringing utility into the picture, we can connect
issues that that have been discussed at length regarding utility (e.g., aggregating utilities of
diferent individuals, weighting losses vs gains, etc.) to issues of harm. We develop a quantitative
notion of harm in a follow-up paper that focuses on harm in the context of uncertainty and
when involving multiple individuals [
Sander Beckers was supported by the German Research Foundation (DFG) under Germany’s Excellence Strategy – EXC number 2064/1 – Project number 390727645, and by the Alexander von Humboldt Foundation. Hana Chockler was supported in part by the UKRI Trust-worthy Autonomous Systems Hub (EP/V00784X/1) and the UKRI Strategic Priorities Fund to the UKRI Research Node on Trustworthy Autonomous Systems Governance and Regulation (EP/V026607/1). Joe Halpern was supported in part by NSF grant IIS-1703846, ARO grant W911NF-22-1-0061, and MURI grant W911NF-19-1-0217.