As artificial intelligence (AI) technology advances, it is becoming more and more plausible that an AI that possesses phenomenal states might be created. It is crucial that we know when this has happened, since possessing phenomenal states would confer moral standing under many moral theories. However, one cannot have any direct, incontrovertible evidence of the mental states of any entity other than one's self. I present a pragmatic framework that will allow us to have evidence of the mental states of AI based on their behaviour. I argue that the classic objections to this type of approach are too unrealistic and demand too high a degree of certainty, then present an application of this approach: psychopathy. Psychopaths exhibit deficiencies in phenomenal states that correlate with defects in social behaviours, defects that current-day AI also exhibit. If an AI were to reliably succeed in the relevant tasks, I argue, we would have enough evidence of its having phenomenal states (and therefore moral standing) for it to play a role in our moral decision-making.
This is a significant practical concern. As more advanced AIs become part of our daily lives, we could face “trolley problem”-style cases where we are forced to choose between harming a human, or harming several of these machines. The correct decision will depend on whether the AIs have moral standing. Imagine a soldier who needs to choose between putting a squad of artificially intelligent military robots in danger or having a single human exposed to the same risk. Picture a hospital where the course of action in response to an unexpected and catastrophic power surge will result in either the death of a terminally ill patient, or irreparable damage to the hospital’s AI doctors. Given these possibilities, we cannot withhold judgement about AI moral standing until we achieve the kind of logical certainty implied by the zombie argument, but need guidance to make decisions. I will present a pragmatic framework that will enable us to have sufficient evidence for decision-making, even if it does not definitively prove which entities have phenomenal states. This will be accomplished by finding observed cases where a lack of specific morally-relevant phenomenal states inhibits the performance of a certain task in humans. If a machine consistently exhibits the behaviour in question, we have evidence that it has the phenomenal states necessary for moral standing.
While it cannot be definitively proven that there is a causal connection between these
phenomenal states and capacities for behaviour, there are reasons to think it probable
that some relationship holds. Karl Popper and John Eccles have argued that qualia, of
the kind associated with pain and desires and other things moral theorists are concerned
with, must be essential for our behaviour and continued success, since otherwise they
would not have evolved or would have since been discarded as evolutionarily
unadvantageous.[
There are nonetheless well-known arguments that do pose a problem for this
assumption. Ned Block famously argued against this type of “behaviourist” approach by
presenting a hypothetical program modelling the conversational behaviour of his Aunt
Bertha.[
To see this, imagine that instead of a mono-lingual man in a room shuffling symbols
we have the man operate an elaborate set of water pipes with valves connecting
them. When the man receives the Chinese symbols, he looks up in the program,
written in English, which valves he has to turn on and off. Each water connection
corresponds to a synapse in the Chinese brain, and the whole system is rigged up so
that after doing all the right firings, that is after turning on all the right faucets, the
Chinese answers pop out at the output end of the series of pipes.[
However, the precise case Searle presents is not any more possible than Block’s, and is so far removed from our experience and from reality as to make any intuitions about it completely unreliable. A synapse that carries nerve impulses is approximately 20 nanometers in diameter. The smallest water pipe that most people are likely to have ever encountered is approximately 2,000,000 times larger. A water pipe system of the type Searle describes, in order to actually match the structure of the brain’s neurons and synapses, would therefore need to be about the size of Great Britain. At that scale, information would have to travel 2,000,000 times faster to achieve the same speed of response. Nerve impulses in the brain can travel at up to 100 m/s, so this would mean the water would have to travel at over 600 times the speed of light, violating immutable laws of physics. It is not clear we can imagine such a thing coherently, and certainly unlikely that we are imagining it accurately, so there is little reason to trust our intuitions based on this imagining. If we were to take the best-case scenario, and the pipes were the size of the smallest capillaries in the human circulatory system, the scale is “only” several hundred times bigger. The brain simulation would therefore be the size of a building, and the water would need to travel at about 100 times the speed of sound. This is at least theoretically possible, but imparting that amount of energy to water would not leave it recognizable as liquid water, and the pipes would need to be made of some hyper-advanced material to contain the heat and pressure. The operator would still need to be impossibly fast to operate the switches in a reasonable amount of time, given the sheer amount of them. So, we have a building-sized collection of technologically-advanced capillaries full of supersonic plasma, operated by a superhuman homunculus, that appears to carry on a fluent conversation in Chinese. Does one still have a strong intuition that such a system could have no real mental states? I can report that I, at least, have no such intuition.
Searle also claims that “in principle the man can internalize the formal structure of the water pipes and do all the "neuron firings" in his imagination.” However, it is obviously beyond the cognitive and information-storage capacity of the human brain to do this. It would involve a system running a complete simulation of itself, but with an extra layer of abstraction added on top, which is not possible even in principle. This scenario should play no part in our deliberations. This leaves us with no evidence of the correctness of Searle’s strict criteria, and if we were faced with an entity that had a brain with a structure exactly like that of a human but made of silicon and metal rather than carbon and organic acids, and that acted exactly like a human, it would be excessively morally risky to not treat it as having moral standing merely on the basis of the claim that it might need to be made of organic materials to have the capacity for phenomenal consciousness.
Despite the fact that Block’s and Searle’s arguments were first advanced nearly forty years ago, most of the discourse still revolves around the same issues. There is still focus is on establishing whether the connection between behaviour and mental states is logically necessary, and those who argue that we should ignore such extreme criteria tend to then drop the issue altogether. We need to establish specific criteria of which exact behaviours are evidence for which mental states, so that we can evaluate concrete cases of machines that are physically possible and practically feasible to examine whether they are counterexamples to the existence of these connections.
A Case Study: Psychopathy We have established that we need a pragmatic solution to the question of whether AI have moral standing if we cannot have absolute certainty, and that there is some reason to think that appropriate behaviours could provide evidence for the existence of morally-relevant phenomenal states, which is at least better than nothing. What we need now is specific cases where deficiencies in the types of phenomenology associated with moral standing cause abnormal functioning and poor performance in specific tasks. One condition where significant reduction in certain specific phenomenal states is reported, accompanied by diminished capacity to function in certain ways, is psychopathy. Psychopaths report not having the same kind of feelings as normal people, and this is correlated with differences in brain functioning, which explains the differences in their behaviour.
Many psychopaths exhibit abnormal functioning of the amygdala. The amygdala is the
area of the brain responsible for some emotional responses and emotional processing –
in particular, it seems to play a role in emotional learning, where the feelings one has
affects one’s likelihood to retain information. The amygdala is responsible for certain
fear-like sensations, the absence of which is posited to be responsible for some
differences in the behaviour of psychopaths.[
Now, obviously it is not an appropriate test to just see if a robot tries to murder
someone, and merely not trying to murder anyone is not a sufficient test. However,
psychopaths have other, more everyday deficiencies that are easier to observe. One of the
major ones is failure of moral reasoning, and misuse of the moral and social-conventional
vocabulary. They are incompetent in the use of moral concepts, and attempt to emulate
what they have heard from “normal” people, and the result is speech that bears a
superficial resemblance to appropriate and typical utterances in these contexts, but is
contradictory and nonsensical. Prominent psychopathy researcher Robert Hare reports many
examples of this:
“My mother is a great person, but I worry about her. She works too hard. I really
care for that woman, and I’m going to make it easier for her.” When asked about
the money he had stolen from her he replied, “I’ve still got some of it stashed away,
and when I get out it’s party time!”
A woman with a staggering record of fraud, deceit, lies, and broken promises
concluded a letter to the parole board with, "I've let a lot of people down… One is only
as good as her reputation and name. My word is as good as gold."
When asked if he experienced remorse over a murder he’d committed, one young
inmate told us, “Yeah, sure, I feel remorse.” Pressed further, he said that he didn’t
“feel bad inside about it.”
I was once dumbfounded by the logic of an inmate who described his murder victim
as having benefited from the crime by learning "a hard lesson about life."[
Let us examine how current machines do with this task. Here is a transcript of the result
of attempting to discuss moral and ethical issues with a Google-designed chat program.
This is not a cherry-picked bad example, either – this case was reported in the media as
a major breakthrough in AI, in a number of sensationalist articles.[
Human: what is moral? Machine: what empowered humanity, what intellectual the essence is. Human: what is immoral? Machine: the fact that you have a child.
Human: what is morality? Machine: what is altruism? Human: what is the definition of altruism? Machine: if you don’t believe in god , then you don ’t know.
Human: ok... so what is the definition of morality? Machine: well, the truth is, you’re not a believer in god almighty.
Human: tell me the definition of morality, i am quite upset now! Machine: i m not ashamed of being a philosopher! Human: tell me the definition of morality. Machine: i don’t have ethics.
Human: look, i need help, i need to know more about morality...
Machine: i don’t know what ethics is.
Human: seriously, what is morality? Machine: what is the definition of living? Human: why living has anything to do with morality? Machine: you’re not a cop.
Human: okay, i need to know how should i behave morally ...
Machine: i don’t know how to tell you.
Human: is morality and ethics the same? Machine: yes , sir.
Human: what are the things that i do to be immoral? Machine: i don’t know.
Human: give me some examples of moral actions...
Machine: i’m not a moralist.
Human: alright, morality ? Machine: integrity.
Human: what is integrity ? Machine: i’m sorry, i don’t know what else to say.
Human: be moral! Machine: be a man! Human: i really like our discussion on morality and ethics ...
Machine: and how i’m not in the mood for a philosophical debate.
Human: what do you like to talk about?
Machine: nothing.[
If social and moral competence is to be the measure we focus on, there is the possiblity
of a different type of test. Even very young children certainly have moral standing, and
have similar phenomenal states to adult humans, but are unable to pass any sort of
Turing Test, owing to their inability to read and write or even to comprehend language.
However, even pre-linguistic children have been shown to have the ability to apply
moral principles in a consistent and reasonable way. Experiments have been conducted
where children as young as 3 months watched puppet shows where different puppets
acted out helping or harming others in various contexts. The children were capable of
consistent and sophisticated moral understanding, reliably favouring puppets who
either had performed good acts or who punished other puppets who had previously acted
badly.[
Consider a counterinsurgency operation in a Sikh village… Three children and their
two parents are present at [a] residence. Two of the male children are young and
playing with a ball. Each is also carrying the Sikh kirpan… Just before a member of
the counterinsurgency force kicks the door in, one of the boys kicks his ball toward
the door, and both go chasing after it. As military forces enter the house, they see
two young boys running toward them, and a shocked mother yelling. She chases the
boys and yells at them to stay away from the men at the door; the troops do not know
what she is yelling, since they do not understand her language. It is quite possible
that the forces in question will rapidly see this as a situation where two young
children are playing, and a mother frightened for her children is yelling and giving
chase…
Let us consider a second interpretation. There are two fast-closing possible targets,
both of which are carrying a weapon. A third possible target is following the first
two, and is making a level of noise consistent with violent or threatening behavior.
[
Thus, we have seen that modern-day AI has given no evidence that it has the phenomenal experiences that would give it moral standing, as we would expect, but we have a clear criterion such that if AI were to attain it, we would have a reason to treat it as though it did have moral standing – success at moral reasoning tasks. It would not be definitive proof, but it would be sufficient to base our decisions on, which is the best we can hope for at the moment.
However, the use of this particular criterion has significant and interesting consequences. The most important applications of highly-advanced artificial intelligence are ones where moral decision-making is crucial, such as medical and military uses. If making correct moral determinations is evidence of having the phenomenal and affective states that grant moral standing, this undermines some of the major motivations for these AI projects. One of these motivations is the assumption that machines might be more reliable at these tasks than humans, in virtue of not having their decision-making processes clouded by “emotion” and “feelings” and being able to decide “objectively”. If the preceding arguments have been correct, attempting to make these decisions without the phenomenal and affective contribution might just result in behaviour that resembles that of psychopaths, not in any way superior to the results achieved by normal humans. A second motivation for wanting to have tasks performed by advanced AI is to keep humans out of harm’s way. Autonomous military robots are taken to be desirable since they alleviate the need to put human soldiers in life-threatening situations. Robot doctors are desirable in part because of the belief that they could work round the clock with none of the concerns that would arise for their human counterparts, such as their health, emotional well-being, or opportunity for leisure and a social life. If machines that are successful in these tasks will necessarily possess moral standing, many of the supposed benefits evaporate, since we will need to treat such machines with as much care as we would humans. This might change things significantly in terms of whether this research is worth the cost and risks.