=Paper=
{{Paper
|id=Vol-2521/paper-05
|storemode=property
|title=Using AI to Understand Intelligence: The Search for a Catalog of Intelligence Capabilities
|pdfUrl=https://ceur-ws.org/Vol-2521/paper-05.pdf
|volume=Vol-2521
|authors=Dagmar Monett,Christian Winkler
|dblpUrl=https://dblp.org/rec/conf/aiia/MonettW19
}}
==Using AI to Understand Intelligence: The Search for a Catalog of Intelligence Capabilities==
https://ceur-ws.org/Vol-2521/paper-05.pdf
Using AI to Understand Intelligence:
The Search for a Catalog of Intelligence Capabilities
Dagmar Monett1,2 and Christian Winkler3
1
Computer Science Dept., Berlin School of Economics and Law, Germany
2
AGISI.org
dagmar.monett@agisi.org
3
datanizing GmbH, Germany
christian.winkler@datanizing.com
Abstract. Artificial Intelligence (AI) algorithms permeate many of the systems
and devices we interact with in everyday life. Since its conception as a field, a
wide variety of these intelligent algorithms have sought to simulate or even sur-
pass human cognitive abilities and behavior. However, there is as yet no widely
accepted definition of what intelligence in machines means, nor of human intel-
ligence. The primary goal of this paper is to propose an intelligence vocabulary
or catalog in the quest for the boundaries that shape the current discourse of the
experts on intelligence. The idea is to provide and inform researchers, practition-
ers, journalists, and policymakers, among many others, with a terminology that
can be used when defining intelligence. Considering these challenges, we ana-
lyze the data from hundreds of experts around the world, provided when they
were asked to give both their opinions on existing definitions of intelligence in
the scientific literature and their own suggested definitions. All of the answers
that were gathered (despite being subjective views) were evaluated using state-
of-the-art text analytics processing and AI mechanisms. This ensures objectivity
and allows us to extend (or even reproduce) the study in the future. Ultimately,
our work contributes to strengthen the bridge between human and automatic rea-
soning: we examine experts’ opinions on definitions of human and machine in-
telligence with both manual and automatic methods. Some linguistic tasks like
normalizing and clustering the opinions are performed automatically, thereby the
twofold goal being to find an overview and to enable a drill-down to the most
interesting answers. These individual artefacts are then interpreted manually. We
hope that the proposed intelligence vocabulary will not only contribute to defin-
ing (machine) intelligence better but also to an understanding of both the current
views of experts on intelligence and intelligence itself. This would help to frame
a common language around AI, which has unfortunately been absent thus far. In
the future, extending the procedure presented in this paper might lead to an inter-
disciplinary machine-assisted method for extracting knowledge from subjective
opinions.
Keywords: Intelligence · Human intelligence · Machine intelligence · NLP · In-
telligence catalog.
Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International
(CC BY 4.0).
2 D. Monett and Ch. Winkler
1 Introduction
Over the last 100 or so years, the concept of intelligence has been defined on numer-
ous different occasions and in different fields, both formally and informally. There are
many informal definitions of both human and machine or artificial intelligence (AI).
For example, Legg and Hutter [11] collected 71 definitions that were divided into three
broad categories: collective definitions, psychologists’ definitions, and AI researchers’
definitions. Despite many attempts and suggestions, there is still no generally accepted
definition of intelligence. Defining intelligence has been a rather controversial topic in
the AI community, and this is one of the fundamental problems that has remained un-
solved since the creation of the field. It is also a perceived stumbling block to the pursuit
of understanding intelligence and building machines that replicate and exceed human
intelligence, as addressed by Brooks [4]. In their study more than 25 years later, Wol-
lowski et al. [27] outlined a stark difference of opinion with respect to the definition of
AI. Very little has changed since, and this gap is further reflected in a recent research
study on defining intelligence [19].
Several works have tried to characterize and structure the field of AI. For example,
researchers at Elsevier [6] have applied text mining and machine learning techniques
for this purpose. The authors first identified meaningful concepts and then extracted
field-specific keywords from relevant and non-relevant AI publications that reflect four
perspectives on AI: how it is taught, researched, talked about in the media, and de-
scribed in patents. As a result, a diverse AI vocabulary was created and then used to
shape AI subfields and areas of research. The authors wondered, however, about the
greater divergence than commonality that arises when comparing the four perspectives,
and noted that the absence of a common language around AI was a missing piece for a
wide-ranging understanding.
This was not the first time that the evolution of AI as a discipline had been the
focus of research. In [15], relevant AI publications were investigated, and information
was drawn from two leading AI conferences and around 50 years of documents from
the AAAI’s AITopics database on research, people, and applications of AI.4 Martı́nez-
Plumed et al. propose a framework for defining this field that consists of nine facets (or
intersections between multiple dimensions), such that those that help to characterize
the functionality of AI systems, their generalizability when solving problems, and the
paradigm or approach used, among others. By plotting the relevance of field-specific
words (although these were limited to substrings appearing in titles, abstracts, key-
words, and conference topics), the authors analyzed AI past trends and theorized about
its future. Other authors have also used AI to analyze and track the development of
specific AI areas, such as deep learning [10], and even to predict the decline in their
popularity [9]. A more ambitious initiative even aimed to create an atlas or map of
intelligence [3] that allowed for the categorization, specification, comparison, and ex-
perimental reproduction not only of artificial intelligent systems but also of other kinds
of intelligence.
However, although there has been a recent and growing interest in characterizing,
structuring and tracking the evolution of AI in particular and of intelligent systems in
4
See https://aitopics.org/ for more.
Using AI to Understand Intelligence 3
general, there has been very little research on finding a common language around intel-
ligence and AI. The two-fold purpose of this paper is therefore to use AI to understand
intelligence and to help to frame a common language around defining it. To reach these
goals, we apply various AI techniques to extract the most representative verbs, nouns,
adjectives, and adjective-noun phrases contained in experts’ opinions that might indi-
cate cognitive and behavioral abilities (or capabilities) that could be used when defining
intelligence.
1.1 Data and Relevance of the Study
Our research starts with the analysis of two data sources: respondents’ written opinions
to the AGISI research survey “Defining (machine) Intelligence” [19] to justify their
level of agreement with definitions of human and machine intelligence from the liter-
ature; and respondents’ suggested definitions of human and machine intelligence, i.e.
new suggestions for defining intelligence. Both these data sources and the definitions
from the literature that were provided in the survey were made available by the survey’s
authors.
The diversity of opinions that were collected reflects the current vastly mixed land-
scape of research, development and theory in AI. Most of the respondents to the survey
were AI experts with several years of experience in the field, and an overwhelming ma-
jority were explicitly invited to participate in the survey due to their research in AI or
intelligence-related areas. They originated from 57 different countries and more than
184 institutions around the world. They worked mainly in academia (N =441, 79.3%)
and industry (N =114, 20.5%), and had primary roles as researchers (N =424, 76.3%)
and/or educators (N =193, 34.7%). This was the first time that such a contemporary col-
lection of opinions on and new suggested definitions of human and machine intelligence
had been gathered.
A thorough analysis of this information could have a significant impact on the AI
field for several reasons. Firstly, to obtain clarity around defining intelligence (and AI as
a field), we must consider experts’ opinions and the current debate on AI. Definitions of
intelligence (and especially machine intelligence) should not rely solely on published
content, since the scientific literature does not fully reflect the evolving dynamics of the
field. Shaping the boundaries of the current scientific discourse is necessary, and thus is
important for the advancement of the field.
Secondly, there is no consensus definition of AI, let alone of intelligence; this is
primarily because the former has evolved with a fluid definition due to the varied human
conceptions of the latter [15]. Furthermore, the interdisciplinary nature of the field may
conspire against the possibility of a consensus definition [14]. This has both positive
and negative implications; as mentioned in [23], “the lack of a precise, universally
accepted definition of AI probably has helped the field to grow, blossom, and advance
at an ever-accelerating pace.” However, there are several pressing reasons for coming
to a consensus definition that can help to tackle the various well-known limitations on
the development of the field, such as the poor public knowledge and understanding of
AI [20] and the often misleading media coverage that generates misconceptions about
what is and what is not AI [1,5,13,22], to name only a few. An examination of experts’
4 D. Monett and Ch. Winkler
opinions and suggested definitions may contribute to better insights into intelligence
and to a wider understanding of the current discourse on AI.
Thirdly, it matters how and by whom the AI field is defined [15]. There is no mono-
lithic approach to the development of AI, although this has historically been concen-
trated on a few actors worldwide. The current AI landscape shows not only uneven
progress within and among a variety of application domains [6,23] but also different
interests and research focus together with different perceptions of how and by whom
AI is being developed. This also directly affects how AI is being defined; sounded ex-
perts’ opinions are simply a reflection of the key cultural, societal, and technological
differences driving AI.
Due to the overwhelming response rate to the survey, a manual analysis of the data
was difficult. We therefore decided to apply AI techniques ourselves, and to use text
mining to analyze the results. The following sections describe in more detail how these
techniques were applied to extract and analyze the experts’ opinions on and their new
suggested definitions of intelligence, as introduced above. As a result of this work, an
intelligence catalog that includes cognitive and behavioral capabilities is proposed that
can be considered when defining intelligence, and this forms the main contribution of
this work. With this, we hope to contribute to a common language around both intelli-
gence and AI.
2 Data Pre-Processing
Two corpora were created: the first consisted of 4,041 experts’ opinions, of which 2,424
were on definitions of machine intelligence (MI) and 1,617 on definitions of human
intelligence (HI) from the literature; the second consisted of 338 new, suggested defini-
tions of intelligence, of which 213 were suggested for machine intelligence and 125 for
human intelligence. The easiest way to look at the data in these corpora was by applying
simple text analytics techniques for gathering information about the number of words
that are used, together with their frequency of occurrence, the number of sentences and
so on. The corpus of opinions, for example, contains more than 71,000 words. This
work would be very exhausting to do manually, and the separation of verbs, nouns, and
other parts of the speech into unigrams, bigrams, etc. for later processing would be very
difficult and time-consuming. Other techniques and algorithms should therefore be used
instead.
Our pre-processing of the data from the corpora consisted of the following phases
and steps (see Figure 1):
Data preparation: First, the raw data from the survey was converted into a format
ready for further processing. Then, all personal information about the survey partici-
pants (like name, age, ethnicity, institution, email, etc.), available in separate fields, was
removed.
Cleaning: The data was cleaned. For example, opinions containing a URL as only
content were deleted. Some intelligence-specific words might be losing their signifi-
cance if they are not spell checked or have grammatical errors in their occurrences.
Thus, a spelling and grammar check was carried out.
Using AI to Understand Intelligence 5
Natural Language Processing (NLP): Part-of-speech (POS) tagging was performed.
This allows for the identification of verbs, adjectives, nouns, etc. in a sentence accord-
ing to their meaning and context. Other techniques that were applied in this phase in-
cluded tokenization and lemmatization. Stop words and unimportant characters like
non-printable characters and emojis were also removed.
Analysis: This phase comprises the following steps: i) Statistical analysis: Word
clouds, heat maps and histograms were generated, among other graphics, together with
some general statistics. This allowed for an initial understanding of the data and the
subsequent inclusion of other analyses; ii) Unsupervised learning: We used a slight
modification of topic modeling, aggregating the content written by individual authors
and using these documents as basis. We refer to these as data-driven persona models;
iii) Semantic analysis: A word embedding was created and trained to convert words to
vectors. This is useful for finding semantic relations between definitions of intelligence,
respondents’ opinions, etc. We started with a simple bag-of-words model and refined
it using TF-IDF to penalize frequent words with little distinction quality. To obtain a
better understanding of the semantics, we analyzed both unigrams and bigrams.
Fig. 1: Pre-processing phases and steps in analyzing the corpora of intelligence data.
6 D. Monett and Ch. Winkler
Our analysis pipeline depended mainly on open source software. Data preparation
was carried out with Pandas (https://pandas.pydata.org), and for cleaning we used GNU
Aspell (http://aspell.net). NLP was carried out using spaCy (https://spacy.io) and we
chose scikit-learn (https://scikit-learn.org) for machine learning and topic modeling. Se-
mantics were extracted using word2vec [17], especially to detect phrases, what worked
well despite the very domain-specific, small vocabulary. Even though the dataset is not
very large, the vocabulary is highly repetitive which makes word2vec a suitable technol-
ogy. We also used Jupyter notebooks (https://jupyter.org) as an integration platform.5
3 Data Analysis
Our initial approach to interpreting the data consisted of the analysis of 18 word clouds
created in the final phase of the pre-processing. They included the most common verbs,
nouns, adjectives, and adjective-noun combinations used in the experts’ opinions on
definitions of human and machine intelligence from the literature, the most common
verbs used in opinions about the three most agreed upon definitions of human intelli-
gence (Gottfredson’s [8], Anastasi’s [2], and Wechsler’s [25]) and machine intelligence
(Wang’s [24], Winston’s [26], and Legg and Hutter’s [11]), and the most common words
used in all new, suggested definitions of human and machine intelligence.
3.1 Most used Verbs when Defining Intelligence
Intelligent capabilities are mostly described in ostensive and operational ways, i.e. by
exemplifying and using the cognitive and behavioral characteristics of humans, agents,
or intelligent systems in general that denote or are important in achieving intelligence
[7]. Verbs are often used to denote which these intelligent capabilities should be. Figure
2 shows the word clouds for the verbs in the experts’ opinions on definitions of human
and machine intelligence from the literature, and in their new, suggested definitions, as
created in the pre-processing phase.
A first analysis of these verb word clouds indicates the presence of two types of
verbs used by experts in their arguments for or against existing definitions and in their
suggested definitions: not only verbs that exemplify intelligent capabilities, such as
think, anticipate, and feel, but also those that complement respondents’ written speech
when justifying their opinions, such as define, agree, and include. The former are im-
portant in constructing a catalog of intelligence. Furthermore, the verbs used in the sug-
gested definitions shape the current discourse on intelligence, whilst those used when
criticizing definitions from the literature may refer to issues that are no longer relevant
or that should not be considered when defining intelligence.
It is worthwhile to analyze in depth those definitions for which the level of agree-
ment was positive, and thus less polarized. Although comments expressing positive
agreement were many fewer than those expressing disagreement [18], and “[a] good
argument is an argument that is not refuted” [16], we expected that they would con-
tain words that both supported the positive opinion of their authors and indicated what
should be considered when defining intelligence, rather than what should not.
5
The source code and data are available upon request.
Using AI to Understand Intelligence 7
(a) (b)
(c) (d)
Fig. 2: Verb word clouds for (a) all opinions on the definitions of human and (b) machine intel-
ligence from the literature, and for the new, suggested definitions of (c) human and (d) machine
intelligence.
A closer look at the verbs used in the comments on the most agreed-upon definitions
shows a similar trend. Again, two types of verbs can clearly be identified. The process
of constructing the catalog of intelligence should identify only those verbs that denote
cognitive and behavioral capabilities. This is not a task that can be automated easily,
however, and depends heavily on the context. There are sometimes verbs that could be
of both types, such as define: compare “The goal is to define intelligence easily” to “An
agent can define its own goals.” In the latter example, there is a cognitive capability that
could denote intelligence. This would mean that the data pre-processing and application
of NLP techniques in conjunction with visualization using word clouds can provide “the
first filter,” but the remainder would mostly depend on human work.
3.2 Most used Adjectives and Adjective-Noun Bigrams
Other words besides verbs can indicate cognitive and behavioral capabilities that de-
note intelligence, for example adjective and nouns. We analyzed the most commonly
used adjectives and their combinations with nouns for all of the experts’ comments on
definitions of intelligence from the literature and for their suggested definitions in the
corpora. Again, the most frequently used words included the two types discussed above:
those that could be used when constructing the catalog of intelligence, and those used
to complement the respondents’ written speech. For example, the adjectives cognitive,
rational, and adaptive may be related to properties of intelligence, but vague, fuzzy, and
restrictive may form part of a negative opinion on a concrete definition that is being
criticized. A property of intelligence should not be vague, although a definition may be
8 D. Monett and Ch. Winkler
vague. This distinction is more evident in the adjective-noun phrases, since bigrams give
more information about the context than unigrams. Furthermore, adjectives and nouns
arise in respondents’ opinions for or against definitions from the literature that are used
to qualify those definitions (for example good, narrow, general definition) and that are
not used at all in the respondents’ new, suggested definitions of intelligence. Thus, the
parts of speech from the latter can better contribute to a catalog of intelligence.
3.3 Measuring Similarity Between Experts’ Opinions
Figure 3 shows a heat map of the similarities between the opinions given by respondents
to the definitions of human (top left side) and machine (bottom right side) intelligence,
in descending order of similarity in each category. The higher the similarity values
(i.e. closer to 1), the more similar the level of agreement between two definitions. The
similarity values were calculated using the cosine similarity measure of summed TF-
IDF vectors. These vectors give information on the frequency of the words used by
respondents in their opinions against their importance with respect to the other opinions
in the corpus. No other similarity measures were considered at this stage; further work
could include an extended comparison to other methods.
Fig. 3: Heat map of the cosine similarities between experts’ opinions. The x- and y-axis use the
order of perceived similarity between definitions.
There is a remarkable low similarity in the opinions on definitions of machine intel-
ligence and those given for human intelligence. The similarities between the arguments
that were given to justify the level of agreement with definitions of machine intelligence
Using AI to Understand Intelligence 9
(lower right-hand side) are much more evident (have higher values) than when they are
compared to human intelligence (lower values in the top right and lower left quadrants).
Correspondingly, the similarities between the opinions given for definitions of human
intelligence (top left-hand side) are similar to each other but not to the opinions given
for machine intelligence (except for HI8 and MI6). Once again, this confirms the find-
ings in [12] and supports the need for two separate definitions of machine and human
intelligence: it is clear that we view, understand, react to, and judge these differently. In
other words, a definition of intelligence may need to differentiate between machine and
human intelligence. Furthermore, the heat map shows that the arguments provided by
respondents to justify their level of agreement with definition MI7 (Russell and Norvig’s
[21] definition of machine intelligence) are the least similar to other comments, i.e. the
similarity values to other opinions are the lowest on average (see the last column and
last row of the heat map). Since this definition was both the most commented on and
second least accepted definition overall, a possible explanation for these low similarity
values may be that the polarized comments contain “special” information about what
should not be considered when defining intelligence that the other comments do not in-
clude. This may have an explanation that is consistent with results from argumentative
theory: “When participants want to prove a conclusion wrong, they will find ways to fal-
sify it [. . . ] If they disagree with [the conclusion], they try to prove it wrong” [16], i.e.
by providing counterexamples that falsify the wrong conclusion. In this case, we expect
that the counterexamples use a (different) vocabulary that contradicts the conclusion.
3.4 Measuring Similarity Between Experts’ Opinions and Their Suggested
Definitions of Intelligence
Figure 4 shows a heat map of the similarities between the experts’ opinions and their
new, suggested definitions of machine and human intelligence; the higher the similarity
values, the more similar the terminology used in the opinions on the definitions in the
literature to the new, suggested definitions. As can be interpreted from the figure, the re-
spondents tended to (re-)define both machine and human intelligence using much of the
same terminology (and thus intelligent capabilities) that was used when commenting on
definitions of machine intelligence from the literature (see the higher similarity values
on the left-hand side). However, the vocabulary used in their opinions on the definitions
of human intelligence was much less often used in their suggestions for new defini-
tions (see lower similarity values on the right-hand side). In other words, the suggested
definitions of both machine and human intelligence seem to require much of the same
vocabulary that was previously used to subjectively evaluate definitions of machine in-
telligence from the research literature. This may be another reason for supporting two
separate definitions of intelligence. We note that 41.9% of the respondents to the re-
search survey on defining intelligence supported the need for only one definition, in
contrast to 48.2% who preferred two separate definitions, as concluded in [19].
Interestingly, the new, suggested definitions of human intelligence used a vocabu-
lary that was very similar to that used to comment on the definitions of machine intel-
ligence that were not the most agreed upon. Three of the four highest similarity values
10 D. Monett and Ch. Winkler
(see the darker colors at the top left-hand side of Figure 4) corresponded to opinions
given on the definitions of machine intelligence that were not the most agreed upon.6
Fig. 4: Heat map of the cosine similarities between experts’ opinions (horizontal axis) and their
new, suggested definitions of intelligence (vertical axis).
3.5 Analyzing Other Variables
The question arose as to whether the ratings given by experts, i.e. their level of agree-
ment with the definitions of intelligence from the literature, were correlated with the
number of words they wrote in their opinions. However, these were not correlated, i.e.
a significant relationship between the length of the opinions and the level of agreement
was not observed. Due to space constraints, detailed results are not include here. The
interested reader can receive more information about the concrete calculations upon
request.
Does the vocabulary used in the opinions depend on respondents’ years of experi-
ence in the field? To answer this question, we again aggregated the TF-IDF vectors, but
this time for respondents with similar numbers of years of experience. It turns out that
novices with less than five years of experience and experts used a similar vocabulary.7
The same is true for respondents with a considerable amount of experience. However,
6
The most agreed upon were MI3 [24], MI1 [26], and MI2 [11], in that order.
7
Respondents without experience in AI used a different vocabulary and were viewed as outliers.
Using AI to Understand Intelligence 11
the similarity values decreased with the level of experience in the field. One possible
explanation is that greater expertise not only means a deepened knowledge of AI but
also a wider knowledge of other fields, which may enrich the experts’ point of view.
This may empower the use of a more sophisticated, distinctive or selective vocabulary
that occasionally also contributes to the discourse with new terminology. We can sur-
mise that experts not only have knowledge that goes beyond others’ grasp, but that they
also use it actively to produce new information, in this case opinions on definitions of
intelligence from the scientific literature.
4 Intelligence Vocabulary or Catalog
The intelligence catalog was created with the most representative verbs, nouns, adjec-
tives, and adjective-noun phrases (i.e. bigrams) that were extracted after applying text
mining and machine learning techniques, and the analysis introduced above. They were
manually filtered out from the top 100 occurrences of each word type after considering
the techniques presented in this paper. These extracted words help us to distinguish the
boundaries of the discourse around definitions of intelligence from the scientific litera-
ture and to shape the current experts’ view around how intelligence should be defined
instead. Figure 5 shows the main categories and subcategories of the intelligence cat-
alog and Table 1 shows a list with the most common words that were extracted. For
example, the top five verbs that are related to the internal processing of the intelligent
entity are learn, solve, achieve, adapt, and understand, in this order. The more gen-
eral categories are a suggestion from the authors and relate to their expert knowledge
in the intelligent agents and multi-agent systems domains (i.e. they are not extracted
automatically nor any ontology engineering is used, yet).
Fig. 5: Intelligence vocabulary or catalog: categories and subcategories for the corpus of experts’
suggested definitions.
12 D. Monett and Ch. Winkler
Verbs related to perception (with at least five occurrences):
perceive [19] acquire [7] recognize [5]
Verbs related to the internal processing of the intelligent entity (top 15 and with at least five occurrences):
learn [54] adapt [26] improve [14] evolve [10] think [9] explain [5]
solve [52] understand [20] generate [14] reason [9] plan [6] predict [5]
achieve [37] find [15] maintain [13]
Verbs related to action (top 10 and with at least five occurrences):
act [32] exhibit [17] behave [9] interact [7] execute [5] respond [5]
change [20] perform [15] pursue [8] communicate [7]
Nouns (top 30):
machine [284] system [82] information [45] reasoning [34] learning [29] context [23]
ability [146] behavior [77] time [42] agent [34] capacity [28] situation [23]
human [96] capability [60] action [36] world [32] decision [28] range [22]
environment [96] knowledge [55] process [35] experience [30] reason [27] domain [21]
goal [96] problem [54] task [34] artificial [30] resource [24] entity [21]
Adjectives (top 30):
human [183] wide [21] real [18] specific [12] effective [11] mental [9]
cognitive [32] physical [21] computational [17] limited [12] optimal [11] right [8]
general [24] complex [20] social [15] available [12] adaptive [10] possible [8]
artificial [24] good [19] rational [14] capable [11] evolutionary [10] autonomous [7]
high [22] broad [18] natural [14] useful [11] efficient [9] biological [7]
Bigrams indicating or related to properties of intelligence:
Properties:
prior knowledge [5] future behavior [3] adaptive behavior [3] previous experience [3] common sense [2]
social interaction [4] autonomous behavior [2]
Measuring success:
right thing [4] right time [3] efficient plan [2] effective goal [2] useful action [2]
meaningful existence [4] ongoing success [2]
Representation:
own goal [7] short goal [3] new solution [3] heuristic approach [3] current state [2] inner narrative [2]
Human-related:
human level [15] human being [10] human condition [6] human observer [3] natural language [3]
Bigrams related to the environment, thus external to the intelligent entity:
Environment:
real time [11] real world [4] natural world [3] physical world [2]
Complexity:
complex problem [4] new situation [3] different situation [3] recurrent situation [3] complex dimension [2]
complex environment [3] unknown situation [2]
Availability of resources:
wide range [12] limited resource [5] wide variety [4] sensory information [4] available resource [3]
broad range [6] limited information [3]
What is intelligent?
Types of intelligence:
human intelligence [22] artificial intelligence [17] general intelligence [9] high intelligence [3]
Intelligent entity:
intelligent behavior [11] intelligent machine [9] machine intelligent [8] intelligent system [7] system intelligent [4] artificial agent [2]
ability, capability:
cognitive capability [6] cognitive process [4] cognitive ability [3] human capability [3] broad capability [2]
Table 1: Intelligence vocabulary or catalog: top words extracted from the corpus of experts’
suggested definitions that may be considered when defining intelligence. The words are ordered
based on their frequency; some of them could be related to more than one category.
Using AI to Understand Intelligence 13
5 Conclusions
The findings presented in this paper clearly suggest that the current discourse on intelli-
gence (and on AI) requires a deeper analysis of how intelligence is defined. This should
be the first step in understanding intelligence. The aim of the present study was there-
fore to use AI in helping to frame a common language around AI and intelligence by
analyzing thousands of experts’ opinions on definitions from the scientific literature and
their new, suggested definitions of intelligence. As a result, an intelligence vocabulary
is suggested that could be used when defining intelligence.
The possible uses of such a vocabulary are many. For example, it could be used by
lecturers when introducing the concept of (machine) intelligence in their courses; or
by researchers when defining the goals of their artificial intelligence-related research;
or by journalists when reporting scientific results in the neurosciences, psychology, or
AI fields, to name a few; or by policy makers, lawyers, and ethicists when delineating
recommendations and regulations on how to design, deploy, and use intelligent systems,
because for regulating something, that something must be well-defined. In short, we are
of the opinion that the intelligence vocabulary could be essential when developing a
common language around AI and intelligence.
A wider discussion remains open, however, and it includes questions such as: to
what degree can the resulting capabilities be considered properties of intelligence?
Which capabilities or parts of the intelligence catalog are related to which specific ar-
eas from the AI field? Is there any distinction between capabilities that relate only to
humans or only to machine or non-human intelligence? The intelligence vocabulary of-
fers one of several initial approaches to these questions, but further research is needed
before we can achieve a common language around AI and a satisfactory understanding
of what intelligence is.
References
1. Committee on AI, S.: AI in the UK: ready, willing and able? Report of Session 2017–19,
The Authority of the House of Lords, UK (2018)
2. Anastasi, A.: Intelligence as a quality of behavior. In: Sternberg, R.J., Detterman, D.K. (eds.)
What is intelligence?: Contemporary viewpoints on its nature and definition, pp. 19–21.
Ablex, Norwood, NJ (1986)
3. Bhatnagar, S., Alexandrova, A., Avin, S., Cave, S., Cheke, L., Crosby, M., Feyereisl, J.,
Halina, M., Loe, B.S., Ó hÉigeartaigh, S., Martı́nez-Plumed, F., Price, H., Shevlin, H.,
Weller, A., Winfield, A., Hernández-Orallo, J.: Mapping intelligence: Requirements and Pos-
sibilities. In: Müller, V.C. (ed.) Philosophy and Theory of Artificial Intelligence 2017, vol.
SAPERE 44, pp. 117–135. Springer, Berlin (2018)
4. Brooks, R.A.: Intelligence without representation. Artificial Intelligence 47, 139–159 (1991)
5. Darwiche, A.: Human-level intelligence or animal-Like abilities? Communications of the
ACM 61, 56–67 (2018)
6. Elsevier: Artificial intelligence: How knowledge is created, transferred, and used. Trends in
China, Europe, and the United States. https://bit.ly/2Lajwaa (2018)
7. Fox, J., Beveridge, M., Glasspool, D.: Understanding intelligent agents: analysis and synthe-
sis. AI Communications 16, 139–152 (2003)
14 D. Monett and Ch. Winkler
8. Gottfredson, L.S.: Mainstream science on intelligence: An editorial with 52 signatories, his-
tory, and bibliography. Intelligence 24, 13–23 (1997)
9. Hao, K.: We analyzed 16,625 papers to figure out where AI is headed next. Intelligent Ma-
chines, MIT Technology Review. http://bit.ly/2TfLfK1 (2019)
10. Klinger, J., Mateos-Garcia, J., Stathoulopoulos, K.: Deep learning, deep change?
Mapping the development of the Artificial Intelligence General Purpose Technology.
arXiv:1808.06355 (2018)
11. Legg, S., Hutter, M.: Universal Intelligence: A Definition of Machine Intelligence. Minds
and Machines 17, 391–444 (2007)
12. Lewis, C.W.P., Monett, D.: Text Analysis of Unstructured Data on Definitions of Intelligence.
In: Proceedings of The 2018 Meeting of the International Association for Computing and
Philosophy, IACAP 2018. Warsaw, Poland (2018)
13. Lipton, Z.C.: From AI to ML to AI: On Swirling Nomenclature & Slurried Thought.
Approximately Correct: Technical and Social Perspectives on Machine Learning. https:
//bit.ly/2QWDs6t (2018)
14. Luckin, R., Holmes, W., Griffiths, M., Forcier, L.B.: Intelligence Unleashed: An argument
for AI in Education. Pearson, London (2016)
15. Martı́nez-Plumed, F., Loe, B.S., Flach, P.A., Ó hÉigeartaigh, S., Vold, K., Hernández-Orallo,
J.: The Facets of Artificial Intelligence: A Framework to Track the Evolution of AI. In:
Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence,
IJCAI 2018. pp. 5180–5187. Stockholm, Sweden (2018)
16. Mercier, H., Sperber, D.: Why do humans reason? Arguments for an argumentative theory.
Behavioral and Brain Sciences 34, 57–111 (2011)
17. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient Estimation of Word Representations
in Vector Space. arXiv:1301.3781 (2013)
18. Monett, D., Hoge, L., Lewis, C.W.P.: Cognitive Biases Undermine Consensus on Defini-
tions of Intelligence and Limit Understanding. In: Furbach, U., Hölldobler, S., Ragni, M.,
Rzepka, R., Schon, C., Vallverdu, J., Wlodarczyk, A. (eds.) Joint Proceedings of the IJCAI-
2019 Workshops on Linguistic and Cognitive Approaches to Dialog Agents and on Bridging
the Gap Between Human and Automated Reasoning. pp. 51–58. CEUR-WS, Macau, China
(2019)
19. Monett, D., Lewis, C.W.P.: Getting clarity by defining Artificial Intelligence–A Survey. In:
Müller, V.C. (ed.) Philosophy and Theory of Artificial Intelligence 2017, vol. SAPERE 44,
pp. 212–214. Springer, Berlin (2018)
20. Nemitz, P.: Constitutional democracy and technology in the age of artificial intelligence.
Phil. Trans. R. Soc. A 376, 20180089 (2018)
21. Russell, S.J., Norvig, P.: Artificial Intelligence: A Modern Approach. Prentice Hall, third
edn. (2010)
22. Sloane, M.: Making artificial intelligence socially just: why the current focus on ethics is not
enough. The London School of Economics and Political Sciences. https://bit.ly/2NF2amR
(2018)
23. Stone, P., Brooks, R., Brynjolfsson, E., Calo, R., Etzioni, O., Hager, G., Hirschberg, J.,
Kalyanakrishnan, S., Kamar, E., Kraus, S., Leyton-Brown, K., Parkes, D., Press, W., Sax-
enian, A., Shah, J., Tambe, M., Teller, A.: Artificial intelligence and life in 2030. One
Hundred Year Study on Artificial Intelligence. Stanford University, Stanford, CA. http:
//ai100.stanford.edu/2016-report (2016)
24. Wang, P.: What do you mean by “AI”? In: Wang, P., Goertzel, B., Franklin, S. (eds.) Artificial
General Intelligence 2008, Proceedings of the First AGI Conference, Frontiers in Artificial
Intelligence and Applications. vol. 171, pp. 362–373. IOS Press Amsterdam, The Nether-
lands (2008)
Using AI to Understand Intelligence 15
25. Wechsler, D.: The measurement of adult intelligence. Williams & Wilkins, Baltimore (1939)
26. Winston, P.H.: Artificial Intelligence. Addison-Wesley Publishing Company, third edn.
(1992)
27. Wollowski, M., Selkowitz, R., Brown, L.E., Goel, A., Luger, G., Marshall, J., Neel, A.,
Neller, T., Norvig, P.: A survey of current practice and teaching of “AI”. In: Schuurmans,
D., Wellman, M. (eds.) Proceedings of the Thirtieth AAAI Conference on Artificial Intelli-
gence, AAAI 2016. pp. 4119–4124. AAAI Press, Palo Alto, CA (2016)