=Paper=
{{Paper
|id=Vol-1419/paper0049
|storemode=property
|title=A Clarification on Turing's Test and its Implications for Machine Intelligence
|pdfUrl=https://ceur-ws.org/Vol-1419/paper0049.pdf
|volume=Vol-1419
|dblpUrl=https://dblp.org/rec/conf/eapcogsci/MaguireMM15
}}
==A Clarification on Turing's Test and its Implications for Machine Intelligence==
https://ceur-ws.org/Vol-1419/paper0049.pdf
A clarification on Turing’s test and its implications for machine intelligence
Phil Maguire (pmaguire@cs.nuim.ie)
Philippe Moser (pmoser@cs.nuim.ie)
Department of Computer Science
National University of Ireland, Maynooth
Rebecca Maguire (rebecca.maguire@ncirl.ie)
School of Business, National College of Ireland
IFSC, Dublin 1, Ireland
Abstract a live link was nothing more than a historical recording be-
ing played back. The scenario is unlikely, yet it remains a
Turing’s (1950) article on the Turing test is often interpreted as possible explanation for any observation of finite behaviour.
supporting the behaviouristic view that human intelligence can
be represented in terms of input/output functions, and thus em- According to Aaronson (2006): “In principle, we can always
ulated by a machine. We show that the properties of functions simulate a person by building a huge lookup table, which en-
are not decidable in practice by the behaviour they exhibit, a codes the person’s responses to every question that could ever
result, we argue, of which Turing was likely aware. Given that
the concept of a function is strictly a Platonic ideal, the ques- be asked...So there’s always some computational simulation
tion of whether or not the mind is a program is a pointless one, of a human being”. The observation that “physical action
because it has no demonstrable implications. Instead, the in- is always reducible to some kind of Turing-machine action”
teresting question is what intelligence means in practice. We
suggest that Turing was introducing the novel idea that intel- (Hodges, 2013) is therefore a misleadingly simplistic charac-
ligence can be reliably evidenced in practice by finite interac- terisation of Turing’s (1950) work.
tions. In other words, although intelligence is not decidable in We suggest that Turing’s idea was not about machines em-
practice, it is testable in practice. We explore the connections
between Turing’s idea of testability and subsequent develop- ulating intelligence, but about the possibility of intelligence
ments in computational complexity theory. being evidenced in practice. His point was that, although in-
Keywords: Turing test, functionalism, mathematical objec- telligence is a Platonic ideal (i.e. cannot be decided in prac-
tion, artificial intelligence, Chinese room argument, P versus tice), it is somehow manifested in finite objects, meaning that
NP.
finite tests can detect it with high confidence. Hodges (2009)
succinctly expresses this alternative idea: “operations which
Introduction are in fact the workings of predictable Turing machines could
The Turing test (Turing, 1950) is often interpreted as sug- nevertheless appear to the human observer as having the char-
gesting that the mind can be viewed as a program, and that acteristics of genuine intelligence and creativity”.
intelligence can, in effect, be emulated by a machine. For The first argument we will make is that, as regards the
example, Hodges (2013) states that a “fair characterisation” mind-program debate, Turing supported the mathematical ob-
of the implicit assumptions in Turing’s paper is the idea that jection and held the intuitive view that the mind could not be
all physical action is, in effect, computable. He also states a program.
that Turing’s underlying argument was that “the organisation The second argument we will make is that Turing (1950)
of the brain must be realisable as a finite discrete-state ma- was pointing out that the mind-program debate concerns Pla-
chine.” Penrose (1994) presents Turing’s argument as that of tonic ideals and hence is not relevant in practice. At the same
viewing “physical action in general - which would include time he was also conjecturing that although intelligence is not
the action of a human brain - to be always reducible to some decidable in practice, it is somehow testable in practice.
kind of Turing-machine action.” Such views imply that Tur-
ing would have supported the philosophy of functionalism, Turing and the mathematical objection
the idea that the mind can be viewed as a function delivering Lucas (1961) and Penrose (1994) have argued that Gödel’s
a particular mapping between input and output. first incompleteness theorem shows that the human mind can-
We will show that, in fact, the properties of functions can- not be a program. Lucas makes the observation that, given
not be decided in practice by the behaviour they exhibit. any formal system which claims to emulate the mind, its
Hence, the observation that all physical actions are com- Gödel sentence can be produced mechanically by a Turing
putable is a useless one. Physical action is finite, meaning machine. People can ’see’ that this sentence is true, but the
that any such action will always have the potential of hav- system cannot, meaning that there is at least one thing humans
ing been produced by trivial mechanical means. This arti- can do that formal systems cannot. Penrose (1994) develops
cle could, in principle, have been written by chance by an this argument using Turing’s theorem, arguing that no pro-
algorithm selecting random characters. It’s possible. Al- gram can fully account for the set of all humanly accessible
ternatively, you could chat for an hour with somebody over methods for ascertaining mathematical truth. These ideas fall
video link and only later find out that what you thought was into the category of argument which Turing (1950) charac-
318
terises as “the mathematical objection”: swer at all. On the other hand the human intelligence seems
“There are a number of results of mathematical logic which to be able to find methods of ever-increasing power for deal-
can be used to show that there are limitations to the powers of ing with such problems, ‘transcending’ the methods available
discrete-state machines...The short answer to this argument to machines”.
is that although it is established that there are limitations to In his last article published before his death in 1954, Tur-
the powers of any particular machine, it has only been stated, ing again emphasises the role of intuition beyond effective
without any sort of proof, that no such limitations apply to the method. He argues that Gödel’s theorem shows that ‘common
human intellect.” sense’ is needed in interpreting axioms, something a Turing
One requirement for Lucas’s argument to succeed is that machine can never demonstrate:
human minds are consistent, the explicit assertion of which “The results which have been described in this article are
would seem to be ruled out by Gödel’s second incompleteness mainly of a negative character, setting certain bounds to what
theorem. Lucas (1976) responds by suggesting that “we must we can hope to achieve purely by reasoning. These and some
assume our own consistency, if thought is to be possible at other results of mathematical logic may be regarded as go-
all...” From this perspective, consistency is not something to ing some way towards a demonstration, within mathematics
be established, but rather the starting point for understanding. itself, of the inadequacy of ‘reason’ unsupported by common
Another assumption in Penrose’s (1994) use of Turing’s sense.”
theorem is that Church and Turing’s conception of effective Separating the Platonic and the practical
method (i.e. computation) is genuinely universal. This as-
Turing’s writings reveal him to be a consistent proponent of
sumption, known as the Church-Turing thesis, does not ap-
the mathematical objection. Nevertheless, it was not some-
pear to be something that can be proved in the traditional
thing he ever attempted to prove. Instead, in 1950 he made
sense. Nevertheless, Church considered his description of
a surprising statement: “I do not think too much importance
computation as a definition, and even Turing was convinced
should be attached to it”.
of its veracity. According to Turing (1954): “The statement
As we will show, the question of whether the mind is a
is...one which one does not attempt to prove. Propaganda is
program is unrelated to the question of whether physical ma-
more appropriate to it than proof, for its status is something
chines can demonstrate intelligent behaviour. A problem with
between a theorem and a definition.”
the associated philosophical debate is that it confuses Platonic
As one of the founders of the discipline of computer sci- ideals, such as ‘function’ and ‘program’, with concepts that
ence, Turing was in a good position to evaluate the mind- can be manifested in practice through finite means, such as
program debate. Although he notes in 1950 that assertions mechanical machines. For example, Searle (1980) seeks to
of the mind’s superiority are “without any sort of proof”, ad- address a question concerning Platonic ideals (is the mind
herence to the mathematical objection is a consistent theme a program?) via a thought experiment involving a practical
of his writings. In his 1938 PhD thesis, carried out under the mechanism (his Chinese room argument).
supervision of Church, Turing makes clear that the mind has To be clear, when Turing (1936) refers to a ‘machine’, he
an intuitive power for performing uncomputable steps beyond is referring to a Turing machine, or program, not a physical
the scope of a Turing machine: machine. His idea of a rule-following automaton represents
“Mathematical reasoning may be regarded rather schemat- what a human following instructions would be able to achieve
ically as the exercise of a combination of two faculties, which using pen and paper. Turing machines cannot be built in prac-
we may call intuition and ingenuity. The activity of the intu- tice. For a start, they require an infinitely long read/write
ition consists in making spontaneous judgments which are not tape. They also require infinite mechanical precision, with
the result of conscious trains of reasoning...In consequence the head being able to move up and down the tape in perpe-
of the impossibility of finding a formal logic which wholly tuity without its position slipping. All of the ‘computers’ that
eliminates the necessity of using intuition, we naturally turn we encounter in everyday life are merely physical finite state
to non-constructive systems of logic with which not all the machines which are projections of a Platonic ideal. Accord-
steps in a proof are mechanical, some being intuitive.” ingly, when we use the word ‘machine’ in this article, we are
After the Second World War, Turing’s view on the role of referring to a finite physical machine which can be realised in
intuition in reasoning appears unchanged. In a 1948 report to practice. The Platonic concept of a rule-following automaton
the National Physical Laboratory, Turing again clarifies that we refer to as a ‘Turing machine’ or ‘program’.
mathematicians’ ability to decide the truth of certain theo- In his 1950 article, Turing is referring, not to Turing ma-
rems appears to transcend the methods available to any Tur- chines, but to finite physical machines. He is wondering
ing machine: about the practical implications of the Platonic theory of com-
“Recently the theorem of Gödel and related results...have putation. For example, can the mathematical objection be
shown that if one tries to use machines for such purposes as demonstrated to have any observable practical implications?
determining the truth or falsity of mathematical theorems and Below, we provide a modification of the use theorem (see
one is not willing to tolerate an occasional wrong result, then Oddifreddi, 1992) to show that no finite set of interactions
any given machine will in some cases be unable to give an an- is sufficient for checking the behaviour of a black-box sys-
319
tem: properties of functions cannot be decided in practice. the idea that the test can decide if a system is intelligent. We
No matter how many questions we are allowed to ask, it is suggest that this is a straw man argument. Turing never made
not possible to deduce that the black-box system is capable any statements about the ‘passing’ of his test, or about its ca-
of solving a given problem. As stated informally by Block pacity to reliably decide the emulation of human understand-
(1981), “two systems could be exactly alike in their actual ing.
and potential behaviour, and in their behavioural dispositions Turing seeks merely to establish the possibility of “satis-
and capacities and counterfactual behavioural properties...yet factory performance” at the imitation game over a finite pe-
there could be a difference in the information processing that riod; not perfect performance, nor the idea that satisfactory
mediates their stimuli and responses that determines that one performance is a reliable indicator of subsequent perfect per-
is not at all intelligent while the other is fully intelligent”. formance. He never goes beyond claims for the finite simula-
More formally, let O be an observer, A a set of strings, and tion of intelligence: “My contention is that machines can be
f : Σ∗ → {0, 1} be a Boolean function. O can adaptively ask constructed which will simulate the behaviour of the human
finitely many queries f (x) =? (O has access to A), after which mind very closely” (Turing, 1951).
O decides whether f computes the set A, i.e. f (x) = A(x) One source of misunderstanding is that Searle and Turing
for every x. The following standard argument shows every use different meanings for the word ‘thinking’. For Searle, a
observer is wrong on some function. machine can be described as thinking when it is capable of
emulating human performance. For Turing, a machine can be
described as thinking when it successfully simulates a finite
Formal argument For any observer O, and any set A, there
set of human performance. While Searle treats ‘thinking’ as
is a function f : Σ∗ → {0, 1} such that O is wrong on f .
a Platonic ideal, Turing identifies that, in practice, there can
be nothing to thinking beyond acting indistinguishably from
Proof. Let O be as above, A be a set. If O rejects all functions the way a thinker acts.
(i.e. thinks all functions do not compute A) then O is wrong
on f , where f (x) = A(x) for every x. So let g be accepted Dismissing the mathematical objection
by O. O queries g on finitely many strings x1 , x2 , . . . , xn . On Whereas Turing’s 1936 article established the Platonic con-
all the strings x1 , x2 , . . . , xn , g is equal to A, otherwise O is cept of computation, his 1950 paper investigates the practical
wrong about g. Choose y different from x1 , x2 , . . . , xn , and implications of this concept. There are two components to his
construct f : Σ∗ → {0, 1}, by letting g(x) = f (x) for all x 6= y, conjecture. The first is that the mathematical objection (i.e.
and f (y) = 1 − A(y). f does not compute A, because f is a possible separation between programs and minds) has no
different from A on input y. Because f equals g on inputs implications in the finite, physical world. The second is that
x1 , x2 , . . . , xn (the ones queried by O), O will make the same intelligence does have certain practical implications, which
decision about f than about g, i.e. O decides that f can com- are somehow testable.
pute A. By construction of f , O is wrong. Dealing with the first component, Turing (1948) notes that
the mathematical objection based on Gödel’s theorem rests
In summary, not only is a Turing-style test not capable on a proviso that the Turing machine is not allowed to make
of deciding intelligence, a finite sequence of input/output is mistakes. However, “the condition that the machine must
not even sufficient for deciding the program that computed not make mistakes...is not a requirement for intelligence”.
it. Accordingly, we can see that, while Searle’s (1980) Chi- In other words, Turing is pointing out that, although physi-
nese room argument presents a valid observation of the limi- cal machines cannot emulate intelligence, they can, when en-
tations of Turing-style tests for deciding the origins of strings, gineered with sufficient precision, simulate it to any desired
it cannot possibly say anything about the mind-program de- level. At the limit, mistakes are inevitable, but in practice
bate. The separation, or lack of, between minds and programs those mistakes can be pushed back as far as one wants. Tur-
has no practical implications that could be exposed by a Chi- ing (1947), in his earliest surviving remarks concerning AI,
nese room scenario. points out that this would allow machines to play very good
Because our argument uses a similar diagonal proof to Tur- chess:
ing’s (1936) theorem, it seems likely that he was aware of the “This...raises the question ‘Can a machine play chess?’ It
general idea. If the properties of functions cannot be decided could fairly easily be made to play a rather bad game. It
in practice, it is clear that the relevant real-world question is would be bad because chess requires intelligence. We stated...
not about establishing once and for all whether the mind is a that the machine should be treated as entirely without intel-
program, but about what intelligence means in practice. This ligence. There are indications however that it is possible to
is the issue that, we believe, Turing (1950) was addressing. make the machine display intelligence at the risk of its mak-
ing occasional serious mistakes. By following up this as-
Intelligence in practice pect the machine could probably be made to play very good
Searle (1980) makes the case that the automaton in the Chi- chess.”
nese room, which intuitively appears to have no understand- Rather than dismissing the mathematical objection, Turing
ing, would “pass the Turing test”, thus supposedly refuting (1950) conjectures that it does not result in practical limita-
320
tions: in the physical world there will always be some ma- Professor Jefferson does not wish to adopt the extreme and
chine which is up to the job of simulating intelligence to a re- solipsist point of view. Probably he would be quite willing to
quired level. The mathematical objection is a Platonic rather accept the imitation game as a test.”
than practical one:
Testing for intelligence
“There would be no question of triumphing simultaneously
over all machines. In short, then, there might be men cleverer Some have interpreted Turing (1950) as suggesting that infi-
than any given machine, but then again there might be other nite testing is required to establish intelligence, spread over
machines cleverer again, and so on.” an infinite space of time (e.g. Harnad, 1992). But, again,
At first blush, this withdrawal of the mathematical objec- this conceptualisation of intelligence as being infinitely dis-
tion appears to eliminate the possibility of evidencing intelli- tant holds no value, because it never delivers practical results.
gence in practice. For instance, if stupid machines can simu- Instead, Turing is saying something far more significant. He
late any finite set of behaviour, and pass any test, then it can is saying that, although intelligence is not something that can
be argued that behaviour alone is never sufficient for estab- be decided, it is something that can be reliably tested.
lishing intelligence. Nothing we can do in the real world, no Let us consider the question of “what is a test”? A test is
behaviour we can perform, can offer conclusive evidence of of finite duration. Applying it to an object yields results that
non-trivial origin. Could it be that intelligence is useless? enable inferences to be drawn about that object. Somehow,
the results hold significance for other aspects of the object,
This is exactly the attitude adopted by Professor Jefferson
beyond those which have been directly tested. One could say
in his 1949 Lister Oration, whom Turing (1950) cites: “Not
that the test succeeds in succinctly ‘characterising’ the object
until a machine can write a sonnet or compose a concerto
through a finite set of responses.
because of thoughts and emotions felt, and not by the chance
For example, students are asked to sit tests to reveal how
fall of symbols, could we agree that machine equals brain -
much they know about a particular subject. Because of the
that is, not only write it but know that it had written it.”
short duration, it is not possible to ask them every question
Here, Jefferson is arguing that finite behaviour alone is that could possibly be asked. Instead, questions are chosen
not sufficient for establishing intelligence. Instead, we must cleverly so that responses can be relied on to draw inferences
‘know’ what strings mean and ‘feel’ emotions. Because such about students’ ability to answer all the other potential ques-
properties can never be represented symbolically, there is no tions which haven’t been asked.
possibility of any system, human or otherwise, evidencing its Of course, a particular student might get lucky on a test.
intelligence in practice. But this doesn’t seem right. Intu- They might fortuitously have learned off the answers to the
itively, our intelligence is something useful. It lets us achieve exact questions which came up, but no others. Thus, as we
things in the real world that less intelligent systems cannot. have already shown, a test can never decide whether a student
The big question is whether there is there any reliable test understands a subject. What a cleverly crafted test can do is
that can somehow validate this intuition regarding the practi- offer a very high level of confidence that the student would
cal utility of intelligence. have answered other questions correctly.
What Turing (1950) is pointing out is that, although intelli- What are the properties of a good test that would lead us to
gence can never be emulated in practice, it must somehow be have such confidence? In short, a good test is one for which
possible to evidence it in practice with high confidence. For there is no easy strategy for passing it, other than full mastery
example, it seems feasible that a finite signal beamed from a of the subject. For a start, there should be no way for the stu-
distant solar system could convince us that it harbours intelli- dent to get a copy of the test in advance, or predict what will
gent life. Granted, we could never be absolutely 100% sure, be on it so that they can just learn off the relevant responses.
but it seems plausible that there exist signals that could lead In addition, the test should be well diversified, bringing to-
us to be very, very confident. gether material from many different areas of the subject. For
Indeed, all the communication we have ever had with instance, the answers should draw on different aspects of un-
other human beings can be summarized as a finite string of derstanding and not betray a simple pattern which allow them
symbols. If intelligence could not be evidenced in practice to all be derived using the same technique. Furthermore, to
through finite interactions, it would preclude humans from be as hard to compute as possible, successive answers should
identifying each other as intelligent, reducing us to solipsists. be integrated with each other, rather than addressing totally
According to Aaronson (2006), “people regularly do decide separate chunks of knowledge.
that other people have minds after interacting with them for The next question is whether testing for a property can con-
just a few minutes...there must be a relatively small integer n tinue to yield dividends in terms of raising confidence closer
such that by exchanging at most n bits, you can be reasonably to certainty. For example, it seems intuitive that a short two-
sure that someone has a mind”. hour test can provide a clear picture of a student’s ability in a
It seems that in order for the concept of intelligence to be subject. But what if we extended the length of the test to three
meaningful, there must be some practical means of identify- hours? Can a test be designed such that it continues to build
ing and engaging with intelligent systems in the real world. confidence continually higher? Is it conceivable that there are
Having realised this, Turing (1950) remarks “I am sure that some properties, such as intelligence, that would support con-
321
tinuous testing of this nature to any level of confidence? Such This explains why Turing (1950) was upbeat on the immi-
a mechanism could ‘bridge’ the gap between the Platonic and nent prospect of artificial intelligence. No matter what elab-
physical worlds and render the concept of intelligence mean- orate tests we conceive of, there will always be feasible ma-
ingful in practice. We propose that this is the fundamental chines that succeed in passing them: “...there might be men
premise underlying Turing’s (1950) conjecture. cleverer than any given machine, but then again there might
be other machines cleverer again, and so on.” The surpris-
Opening the door for machine intelligence ing success of IBM’s Deep Blue over chess champion Gary
Kasparov in 1997 can be seen as a vindication of Turing’s
An important implication of Turing’s testability is that it
principle of unending testability. According to Turing, no
paves the way for machines to display intelligent behaviour.
matter how we shift the goalposts for intelligence tests in the
Tests are finite. The ability to pass hard tests can therefore
future, we will never be able to rule out the possibility of
be encoded in finite machines, which are themselves hard to
machine success. The intuition that intelligence must some-
construct and hard to understand, yet still feasible in practice.
where trump machine will remain simply that: an intuition.
In a BBC radio debate in 1952, Turing connects the idea of
‘thinking’ with this capacity to do things which are reliably
difficult. Even when the mechanics of a machine are exposed,
Computational complexity theory
it can still retain the ability to do ‘interesting things’, which In 1956 Gödel wrote a letter to the dying von Neumann,
are not rendered trivial by the overt mechanisation. In other echoing Turing’s remarks on a potential gap between the
words, explicitly finite objects can still pass hard tests: Platonic theory of computation and its practical implica-
“As soon as one can see the cause and effect working them- tions. In the letter Gödel identified a finite analogue of
selves out in the brain, one regards it as not being thinking, the Entscheidungsproblem which Turing (1936) originally
but a sort of unimaginative donkey-work. From this point of addressed by demonstrating the existence of uncomputable
view one might be tempted to define thinking as consisting of problems. Gödel realised that, although this uncomputability
‘those mental processes that we don’t understand’. If this is must kick in at the Platonic limit, it did not necessarily ap-
right then to make a thinking machine is to make one which ply in practice for deciding the existence of solutions of finite
does interesting things without our really understanding quite length. This would present the possibility of using an algo-
how it is done.” rithm to quickly decide if a given mathematical statement had
When Jefferson confuses this concept of hardness with the a proof of feasible length. He explained to von Neumann that
difficulty of identifying the implementation, Turing immedi- this “would have consequences of the greatest importance”
ately corrects him: because “the mental work of a mathematician...could be com-
“No, that isn’t at all what I mean. We know the wiring pletely replaced by a machine”.
of our machine, but it already happens there in a limited sort These novel ideas that Turing and Gödel struggled to ex-
of way. Sometimes a computing machine does do something press have since developed into a field known as compu-
rather weird that we hadn’t expected. In principle one could tational complexity theory. This discipline now provides a
have predicted it, but in practice it’s usually too much trouble. framework that can be used to formally define concepts such
Obviously if one were to predict everything a computer was as ‘smart questions’ and ‘high confidence’, which are integral
going to do one might just as well do without it.” to the Turing test. Smart questions are, for example, those that
What Turing (1952) is getting across is that finite objects involve solving instances of an NP-hard problem (e.g. com-
can have a form whose mechanical implications are in prin- puting a Hamiltonian path or a subset-sum solution).
ciple predictable, but in practice are hard to anticipate. We In 1950 Turing didn’t have the formal tools needed to ex-
know exactly what the program is, we can see its structure, press these ideas, he was relying on his intuition. What is
yet its relationship with potential input is a complex one. interesting is that many of the key questions in computational
Even when the validity of a machine’s responses can be eas- complexity theory, such as that raised by Gödel in 1956, con-
ily verified, and we can see it computing the answers, the tinue to lie unresolved. For example, it not yet known if there
reason the machine works can still be hard to fathom, other are problems whose solutions can be easily verified, yet are
than doing the calculations oneself. This property is what, for hard to compute. Do smart questions really exist? Can hard
Turing, constitutes ‘thinking’. tests be created that engender high confidence from finite re-
What this implies is that hard tests may actually be hard to sponses? This is known as the P versus NP problem, which
find. When we put forward what appears to be a challenging remains the biggest unsolved problem in computer science to-
test for AI, such as chess, we cannot know for sure how hard day. While computational complexity theory has succeeded
it is. As soon as a machine succeeds in defeating the test, the in formalising the key components in Turing’s conjecture,
associated limitations become apparent. At that point we go which concern the interaction between the Platonic and prac-
back to the drawing board to develop a harder test. Yet the tical domains, it has not yet succeeded in answering the diffi-
process never ends. In the same way that it is not possible to cult questions that ensue. Aaronson (2006) eloquently sums
decide intelligence, it is not possible to decide the reliability up the impasse: “All ye who would claim the intractability of
of a test for intelligence. Tests must themselves be tested. finite problems: that way lieth the P versus NP beast, from
322
whose 2n jaws no mortal hath yet escaped”. ever being able to decide the reliability of a test. The assump-
tion that Turing’s (1950) concept can be addressed by a lo-
Conclusion calised testing event involving untrained and unsophisticated
judges is thus a serious misinterpretation of the basic idea.
Interpretations of Turing’s (1950) work have focused strongly
In conclusion, we have proposed that the Turing test does
on the idea of running the test. The article has often been
not aim to decide a yes/no answer to the question of whether
interpreted either as being supportive of functionalism (e.g.
or not a system is intelligent. It does not address, and was
Searle, 1980), or of advocating a trite, deeply flawed test for
never intended to address, the question of whether the mind is
evaluating the intelligence of artificial systems through the
a program. The Turing test is the observation that finite inter-
process of imitation (e.g. French, 2012).
actions can result in very high confidence in a system’s ability
In this article we have argued that Turing (1950) was nei- to exhibit intelligent behaviour. Hence, intelligent ‘thinking’
ther claiming that the mind is a program, nor providing a machinery is feasible in practice.
heuristic for evaluating the progress of AI resting on human
psychology. Instead, Turing was making the observation that References
although the mathematical objection collapses in practice, it
Aaronson, S. (2006). PHYS771 lecture 10.5: Penrose.
is somehow possible for intelligence to be evidenced with
Block, N. (1981). Psychologism and behaviorism. The Philo-
high confidence through finite interactions.
sophical Review, 5–43.
Philosophers such as Searle (1980) have confused Turing’s French, R. M. (2012). Moving beyond the Turing test. Com-
definition of ‘thinking’ with the emulation of intelligence. We munications of the ACM, 55(12), 74–77.
have shown that the question of whether the mind is a pro- Harnad, S. (1992). The Turing test is not a trick: Turing
gram is not one that has implications in the real world. Any indistinguishability is a scientific criterion. ACM SIGART
finite set of behaviour could have been produced by a triv- Bulletin, 3(4), 9–10.
ial process that simply outputs the behaviour from a database Hayes, P., & Ford, K. (1995). Turing test considered harmful.
or produces it randomly. Turing’s key idea is that, although In Ijcai (1) (pp. 972–977).
intelligence is not decidable in practice, an observer’s confi- Hodges, A. (2009). Alan Turing and the Turing test.
dence in testing for intelligence can increase quickly with the Springer.
length of the interaction. In other words, our intelligence pro- Hodges, A. (2013). Alan Turing. In E. N. Zalta (Ed.), The
vides us with a means of quickly posing and responding to stanford encyclopedia of philosophy (Winter 2013 ed.).
finite tests which are reliably hard to pass. At the core of this Lucas, J. R. (1961). Minds, machines and Gödel. Philosophy,
conjecture lies the idea that intelligence gives us the ability 36(137), 112–127.
to quickly and easily verify, with high confidence, properties Lucas, J. R. (1976). This Gödel is killing me: A rejoinder.
that appear hard to compute. Philosophia, 6(1), 145–148.
Because it was not possible for Turing in 1950 to present Odifreddi, P. (1992). Classical recursion theory: The theory
his conjecture mathematically, he instead chose to publish of functions and sets of natural numbers. Elsevier.
these ideas as a philosophical article in the journal Mind. An Penrose, R. (1994). Shadows of the mind (Vol. 52). Oxford
unfortunate outcome of this choice is that Turing’s (1950) ar- University Press Oxford.
ticle seems whimsical. Reading it quickly, one might almost Searle, J. R. (1980). Minds, brains, and programs. Behavioral
imagine that Turing was playing the gender-based imitation and brain sciences, 3(03), 417–424.
game at a party and stumbled by chance upon the idea of us- Turing, A. M. (1936). On computable numbers, with an ap-
ing it as a test for intelligence. Hayes and Ford (1995) go so plication to the Entscheidungsproblem. J. of Math, 58(345-
far as to suggest Turing was proposing “a test of making a 363), 5.
mechanical transvestite” and state that “Turing’s vision from Turing, A. M. (1947). Lecture on the ACE.
1950 is now actively harmful to our field”. Turing, A. M. (1948). Intelligent machinery.
Turing’s trite presentation betrays the sophisticated theory Turing, A. M. (1950). Computing machinery and intelli-
behind the concept. From his extended writings we can see gence. Mind, 433–460.
that he was concerned with, not the idea of a psychological Turing, A. M. (1951). Intelligent machinery, a heretical the-
standard for AI, but the more general concept of how intel- ory.
ligence can be evidenced in practice. In particular, Turing Turing, A. M. (1954). Solvable and unsolvable problems.
(1950) was not claiming that every test that humans come up Turing, A. M., Braithwaite, R., Jefferson, G., & Newman,
with is reliable. Inevitably, if a Turing-style test is run us- M. (1952). Can automatic calculating machines be said to
ing laypeople, the programs that get furthest will be those think?
that exploit the weaknesses of human psychology. Turing’s Turing, A. M., & Copeland, B. J. (2004). The essential Tur-
conjecture instead concerns the actual concept of testability - ing: seminal writings in computing, logic, philosophy, ar-
the idea that if a group of world-leading experts got together tificial intelligence, and artificial life, plus the secrets of
and laboured for long enough, they would be able to distil Enigma. Clarendon Press Oxford.
stronger and stronger tests for intelligence, though without
323