Fabio Massimo Zanzotto[Viewpoint: Human-In-The-Loop Arti cial IntelligenceUniversity of Rome Tor Vergata
,
Italy000000028494
Little by little, newspapers are revealing the bright future that Arti cial Intelligence (AI) is building. Intelligent machines will help everywhere. However, this bright future may have a possible dark side: a dramatic job market contraction before its unpredictable transformation. Hence, in a near future, large numbers of job seekers may need nancial support while catching up with these novel unpredictable jobs. This possible job market crisis has an antidote inside. In fact, the rise of AI is sustained by the biggest knowledge theft of the recent years. Many learning AI machines are extracting knowledge from unaware skilled or unskilled workers by analyzing their interactions. By passionately doing their jobs, many of these workers are shooting themselves in the feet. In this paper, we propose Human-in-the-loop Arti cial Intelligence (HITAI) as a fairer paradigm for AI systems. Recognizing that any AI system has humans in the loop, HIT-AI will reward these aware and unaware knowledge producers with a di erent scheme: decisions of AI systems generating revenues will repay the legitimate owners of the knowledge used for taking those decisions. As modern Merry Men, HIT-AI researchers should ght for a fairer Robin Hood Arti cial Intelligence that gives back what it steals. The paper appeared in Journal of Arti cial Intelligence Research, Vol 64 (2019) https://doi.org/10.1613/jair.1.11345
We are on the edge of a wonderful revolution: Arti cial Intelligence (AI) is
breathing life into helpful machines, which will relieve us of our need to perform
mundane activities. Self-driving cars [
24,27,26
] are taking their rst steps in our
urban environment and their younger brothers, that is, assisted driving cars
[
35,42,15
], are already a commercial reality. Robots are vacuum cleaning and
mopping the oors of our houses [
44,41,18
]. Chatbots [
49,47
] have conquered our
new window-on-the-world { our smartphones { and, from there, they help with
everyday tasks such as managing our agenda, answering our factoid questions or
being our learning companions [
20,4
]. In medicine, computers can already help
in formulating diagnoses [
2,22,12
] by looking at data doctors generally neglect.
Copyright c 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
AI is preparing a wonderful future where people are released from the burden of
repetitive jobs.
The bright AI revolution may have a possible dark side: a dramatic mass
unemployment that may precede an unpredictable job market transformation.
People and, hence, think tanks [
28,39
] and governments [
10
] are frightened. A
pessimistic report [
28
] of the McKinsey Global Institute (MGI) foresees that AI
may globally replace the equivalent of the activities of 1.1 billion employees by
erasing $15.8 trillion in wages. By releasing people from repetitive jobs,
intelligent machines may leave a majority of citizens with the value of their labor
insu cient to pay for a socially acceptable standard of living [
39
]. The
revolution is ongoing. Chatbots are slowly replacing call center agents in some of
their tasks. Self-driving trains are already reducing the number of drivers in our
trains. Self-driving cars are ghting to replace cab drivers in our cities. Drones
are expanding automation in managing delivery of goods by drastically reducing
the number of delivery people. And, even more cognitive and artistic jobs are
challenged. Intelligent machines may produce music jingles for commercials [
5
],
write novels, produce news articles and so on. Intelligent risk predictors may
replace doctors [
2,22,12
]. Chatbots along with massive open online courses may
replace teachers and professors [
13
]. Coders risk being replaced by machines too
[
30
]. According to the White House report on AI [
10
], this overwhelming progress
of AI can initiate long-standing disruptions of local markets and, according to
the MGI report [
28
], nobody's job will be left unchanged.
Surprisingly, the rise of AI is largely supported by the knowledge of an
unaware mass of people who risk seeing a large part of their wages canceled by
machines. In fact, along with someone selling knowlegde for nuts with Amazon
Mechanical Turk, Crowd ower or SurveyMonkey and along with those aware
programmers who set up these intelligent machines, an unaware mass of people
is providing precious training data by passionately doing their job {
translating, interacting with customers, teaching { or simply performing their activities
on the net { answering an email, interacting on messaging services, leaving an
opinion on a hotel. These data are a goldmine for AI machines as learning
systems transform these interactions in knowledge. By doing their normal everyday
activity, many workers are shooting themselves in the foot and unaware
people are \donating" their knowledge to machines. This is an enormous and legal
knowledge theft taking place in our modern era.
As researchers in Arti cial Intelligence, we have a tremendous responsibility:
building intelligent machines that \support the parents of their intelligence"
[
39
] rather than intelligent machines that steal their knowledge to do their jobs.
Moreover, we need to nd ways to nancially support job seekers as they train
to catch up with these novel unpredictable jobs. We need to prepare an antidote
as we spread this poison in the job market.
This paper proposes Human-in-the-loop Arti cial Intelligence (HIT-AI) as
a novel paradigm for a responsible Arti cial Intelligence. The idea is simple:
giving the right value to the knowledge producers. Recognizing that any AI system
has humans in the loop, HIT-AI promotes interpretable learning machines and,
therefore, arti cial intelligence systems with a clear knowledge life cycle. For
HIT-AI systems, it will be clear whose knowledge has been used in a speci c
deployment or in speci c situations. This is a way to give the rightful credit
and revenue to the original knowledge producers. Hence, HIT-AI is a possible
antidote to the poisoning of the job market.
The rest of the paper is organized as follows. Section 2 describes current
trends and the enabling paradigms for Human-in-the-loop AI. Section 3 sketches
a proposal for a better future. Then, Section 4 draws some conclusions.
2
Human-In-The-Loop AI: Trends and EnablingParadigms
Nowadays, data seem to be the principal source of knowledge for machines. This
section describes: (1) how data have become more important than programmers
to \teach" machines (Sec. 2.1); (2) how explainable Arti cial Intelligence (Sec.
2.2) and distributed representations for symbols (Sec. 2.3) can be used to
understand how these machines learn from data. This description is extremely useful
for proposing the agenda of Human-in-the-loop Arti cial Intelligence.
2.1
Transferring Knowledge to Machines: from Programming to
Autonomous Learning
Since the beginning of the digital era, programming is the preferred way to
\teach" machines. Arti cial non-ambiguous programming languages have been
developed to have a clear tool to tell machines how to solve new tasks or how to
be useful. According to this paradigm, whoever wants to \teach" machines has
to master one of these programming languages. These people, called
programmers, have been teaching machines for decades and have made these machines
extremely useful. Nowadays, it is di cult to imagine passing a single day
without using the big network of machines that programmers have contributed to
building.
Not all the tasks can be solved by programming, so autonomous learning has
been reinforced as an alternative way of controlling the \behavior" of machines.
In autonomous learning, machines are asked to learn from experience. With the
paradigm of programming, we have asked machines to go to school before these
machines have learned to walk through trial and error. This is why machines
have always been good in solving very complex cognitive tasks but very poor in
working with everyday simple problems. The paradigm of autonomous learning
has been introduced to solve this problem.
In these two paradigms, who should be paid for transferring knowledge to
machines and how should they be paid? In the programming paradigm, roles are
clear: programmers are the \teachers" and machines are the \learners". Hence,
programmers could be paid for their work while they are teaching machines that
are learning. In the autonomous learning paradigm, the activity of programmers
is con ned to the selection of the most appropriate learning model and of the
examples to show to these learning machines. Nobody is paid while machines
are learning.
From the point of view of HIT-AI, the trend of shifting from programming
to autonomous learning is dangerous. In fact, programming is a fair paradigm
as it keeps humans in the loop. On the contrary, autonomous learning is an
unfair model of transferring knowledge as the real knowledge is extracted from
data produced by unaware people. Hence, little seems to be done by humans
and machines seem to do the whole job. Yet, knowledge is stolen without paying
what it is worth.
2.2
Explainable Autonomous Learning Machines
Explaining the decisions of autonomous learning machines is a very hot topic
nowadays: dedicated workshops or speci c sessions in major conferences are
ourishing [
1,21
]. In speci c areas of application, for example, medicine, trust in
intelligent machines cannot be blind as nal decisions can have a deep impact
on humans. Hence, understanding why a decision is taken becomes extremely
important. However, what is exactly an explainable machine learning model is
still an open debate [
25
].
In HIT-AI, explainable machine learning can play a crucial role. In fact,
seen from another perspective, explaining machine learning decisions can keep
humans in the loop in two ways: 1) giving the last word to humans; and, 2)
explaining what data sources are responsible for the nal decision. In the rst
case, the decision power is left in the hands of very specialized professionals
who use machines as advisers. This is a clear case of human-in-the-loop AI. Yet,
this is con ned to highly specialized knowledge workers in some speci c areas.
The second case instead is fairly more important. In fact, machines that take
decisions or work on a task are constantly using knowledge extracted from data.
Spotting which data have been used for a speci c decision or for a speci c action
of the machine is very important in order to give credit to whoever has produced
these data. In general, data are produced by anyone and everyone, not only by
knowledge workers. Hence, understanding why a machine takes a decision may
become a way to keep everybody in the loop of AI.
2.3
Symbolic Knowledge and Distributed Representations in Learning Machines
In current AI systems, knowledge is stored in tensors of real numbers called
distributed representations. These representations are pushing learning models
[
23,36
] towards amazing results in many high-level tasks such as image
recognition [
17,38
], image generation [
14
], image captioning [
46,50
], machine translation
[
3,53
], syntactic parsing [
45,48
] and even game playing at a human level [
37,29
].
Explaining AI decisions seems simple when learning machines treat images
as this knowledge is stored similarly to distributed representations. For
example, in neural networks, input images and layers of the networks are tensors of
real numbers. Interpreting these networks is generally done by visualizing how
layers represent salient subparts of target images. Hence, these networks can be
examined and understood.
However, a large part of the knowledge is not expressed in images but with
symbols, which apparently are not similar to distributed representations. Both
in natural and arti cial languages, combinations of symbols are used to convey
knowledge. In fact, for natural languages, sounds are transformed into letters
or ideograms and these symbols are combined to produce words. Words then
form sentences and sentences form texts, discourses, dialogs, which ultimately
convey knowledge, emotions, and so on. Hence, to explain decisions of learning
machines, we need to understand how symbolic knowledge is represented in
distributed representations.
For HIT-AI, there is a tremendous opportunity to track how symbolic
knowledge ows in the knowledge life cycle. Although symbols seem to fade away in
current AI systems, there is a strict link between distributed representations
and symbols, the rst being an approximation of the second [
32,33,51,11
]. In
this way, symbolic knowledge producers can also be rewarded for their unaware
work.
3
Human-In-The-Loop AI: A Simple Proposal for aBetter Future
The AI revolution is largely based on an enormous knowledge theft, which will
possibly be a never-ending source of revenues for companies. In fact, skilled and
unskilled workers do their own everyday jobs and leave important traces. These
traces are the training examples that machines can use to learn. Hence, AI is
stealing these workers' knowledge by learning from their interactions. The stolen
knowledge is going to produce never-ending revenues for companies for years.
This is a major problem since only that very small fraction of the population who
own shares of these companies can bene t from this never-ending revenue source
and the real owners of the knowledge are not participating in this redistribution
of wealth.
The model we propose with Human-in-the-loop Arti cial Intelligence
(HITAI) seeks to give back part of the revenues to the real owners { the knowledge
producers. The key idea is that any pro t-making interaction a machine does
has to constantly repay whoever has produced the original knowledge used to
do that interaction. Assigning rewards per decision is very important as it may
be an incentive to produce better services today and to have better services in
the future. In fact, people have an incentive to work better knowing that their
future revenues depend on how they treat di cult and odd cases today.
Realizing HIT-AI poses two big challenges: the \political" challenge of
convincing companies to share bene ts with producers of the data; the
\infrastructural" challenge of managing ownership of knowledge in the knowledge life cycle.
These are two di erent, yet interrelated challenges.
The \political" challenge is very tough: convincing companies to share the
bene ts of AI technologies may be impossible. But, \if a free society cannot help
the many who are poor, it cannot save the few who are rich" [
19
]. In fact, in the
long run, if there is not a way to redistribute bene ts to knowledge producers {
the poor people {, the overall market will collapse and, hence, companies { the
rich people { will also lose these bene ts. Although reasonable, this may not be
an argument for today's CEOs focused on the short term.
Hence, if convincing companies is impossible, companies should be forced to
share bene ts. A possibility to achieve this is to start by protecting personal
data with two mechanisms:
{ a legal mechanism: protecting \unaware" knowledge production by extending
the copyright law
{ a technological mechanism: promoting new \ownership-aware le systems"
which release and accept data only if owners are speci ed.
Both mechanisms should be promoted by governments. The legal mechanism is
very slow as it has to go from the national level to the international level. The
technological mechanism can be faster as it may be funded by local government
grants or by spontaneous social movements such as the nordic model of MyData1
and, then, spread all over as a novel concept of \ownership-aware le system"
strongly required by nal users.
The \infrastructural" challenge is di erent but again di cult: HIT-AI needs
technologies that produce a trusted knowledge life cycle of AI systems in order
to reward knowledge producers. Managing ownership in the knowledge life cycle
poses major technological and moral issues and it is certainly more complex
than simply using knowledge while forgetting its source. Each interaction has to
be tracked and assigned to a speci c individual.
To build a trusted knowledge life cycle of AI systems, we need to investigate
two core problems: rst, building Knowledge Life Cycle Transparent Arti cial
Intelligence systems and, second, building Trusted Technologies.
Indeed, building Knowledge Life Cycle Transparent Arti cial Intelligence is
mandatory because, in order to reward knowledge producers, systems need to
exactly know who is responsible for a speci c decision. Knowledge Life Cycle
Transparent AI may share techniques with explainable AI (Sec. 2.2) but its
focus is di erent: identifying the causal source of each inference of AI systems.
For Neural Networks [
16
] (NN), it may seem impossible to identify the causal
source of an inference. This is a research topic. Yet, the causal source of an
inference can be determined in many machine learning models. In support vector
machines [
7,8
], it is possible to determine which support vector is participating
to each single decision and which is its \role", that is, its weight, with respect
to that decision. In decision tree learning [
34
], each decision node is justi ed by
a set of examples. Hence, even if today it seems di cult to trace back causal
source of an output in NN, this can be a promising eld of research.
Designing Trusted Technologies is the second core problem as the
knowledge life cycle should be clear and knowledge items correctly tracked. This core
problem is largely linked to the new \ownership-aware le system", which has
1 https://mydata.org/
been mentioned above. However, HIT-AI needs to guarantee that data in
circulation have speci c owners without revealing who the owner is. This is also
required by the General Data Protection Regulation (GDPR) [
9
]. Hence, HIT-AI
will use Digital Identity Protocols and Mechanisms [
6
] to identify people but, at
the same time, HIT-AI will ensure the use of Privacy Preserving Protocols and
Mechanisms, which can be obtained by using data encryption and Block Chains
[
31,43
].
HIT-AI is an alternative to other solutions of wealth redistribution like
Universal Basic Income (UBI) [
40
], which may not be viable [
52
]. In fact, UBI should
be governed at the national level through taxes. But, generally, companies act
in a transnational level paying taxes on revenues where it is more convenient.
Hence, UBI may not be easy to apply.
With a small fraction of the resources needed for UBI, government grants can
instead help HIT-AI to grow a fairer society by winning those di cult \political"
and infrastructural challenges for which the AI eld does not have answers right
now.
4
Conclusions
The bright Arti cial Intelligence revolution has a dark side: a possible, dramatic
job market contraction before its unpredictable transformation. A peasant or,
even, a wise politician of the late 19th century would have never imagined that
after 100 years yoga trainer, pet caretaker and ayurveda massage therapist were
common jobs. Today, the situation is similar with a complication: the speed of
the AI revolution. It's hard to imagine what's next on the job market and, hence,
what are the skills needed for being part of the labor force of the future. We
urge a strategy for the immediate future to mitigate the dark side of the AI
revolution.
In this paper, we proposed Human-in-the-loop Arti cial Intelligence
(HITAI) as a fairer AI approach, which leverages the most gigantic knowledge theft
of the modern era. In fact, unaware skilled and unskilled knowledge workers are
shooting themselves in their feet by passionately doing their normal, everyday
work as these workers are producing the knowledge which Arti cial Intelligence
is making a pro t on. HIT-AI aims to give back a large part of this pro t to its
legitimate owners. As modern Merry Men, HIT-AI researchers should ght for
a fairer Robin Hood Arti cial Intelligence that gives back what it steals.
1.
Aha
,
D.W.
,
Darrell
,
T.
,
Pazzani
,
M.
,
Reid
,
D.
,
Sammut
,
C.
,
Stone
, P. (eds.):
Proceedings of the IJCAI Workshop on Explainable Arti cial Intelligence (XAI) (Aug2017
), http://www.intelligentrobots.org/files/IJCAI2017/IJCAI-17
_ XAI_WS_Proceedings.pdf
2.
Austin
,
P.C.
,
Tu
,
J.V.
,
Ho
,
J.E.
,
Levy
,
D.
,
Lee
,
D.S.:Using Methods from the Data-Mining and Machine-Learning Literature for Disease Classi cation and Prediction: A Case Study Examining Classi cation of Heart Failure Subtypes
.
Journal of Clinical Epidemiology66
(
4
),
398
{
407
(
2013
)
3.
Bahdanau
,
D.
,
Cho
,
K.
,
Bengio
,
Y.:Neural Machine Translation by Jointly Learning to Align and Translate
.
arXiv preprint arXiv:1409.0473
(
2014
)
4.
Beccaceci
,
R.
,
Fallucchi
,
F.
,
Giannone
,
C.
,
Spagnoulo
,
F.
,
Zanzotto
,
F.M.:Education with '\Living Artworks" in Museums
.
In: Proceedings of the International Conference on Computer Supported Education (CSEDU)
. pp.
346
{
349
(
2009
)
5.
Briot
,
J.P.
,
Hadjeres
,
G.
,
Pachet
,
F.
:
Deep Learning Techniques for Music Generation-A Survey
.
arXiv preprint arXiv:1709.01620
(
2017
)
6.
Camp
,
J.: DigitalIdentity
.
IEEE Technology and society Magazine23
(
3
),
34
{
41
(
2004
)
7.
Cortes
,
C.
,
Vapnik
,
V.
:
Support Vector Networks
.
Machine Learning20
, 1{
25
(
1995
)
8.
Cristianini
,
N.
,
Shawe-Taylor
, J.:
An Introduction to Support Vector Machines and Other Kernel-based Learning Methods
. Cambridge University Press (
March 2000
), http://www.amazon.ca/exec/obidos/redirect?tag=
citeulike09
-
20
&path=ASIN/0521780195
9.
EuropeanParliament
and
European Council: Regulation (EU) 2016/679 on the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of Such Data, andRepealing Directive95
/46/EC (
General Data Protection Regulation)
.
O cial Journal of the European Union L119
,
1
{
88
(Apr
2016
), http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=OJ:L:
2016
:119:TOC
10.
ExecutiveO ce of the President of the United States of America: Arti cial Intelligence, Automation, and the Economy (2016
), https: //obamawhitehouse.archives.gov/sites/whitehouse.gov/files/documents/ Artificial-Intelligence-Automation-Economy.PDF
11.
Ferrone
,
L.
,
Zanzotto
,
F.M.
:
Symbolic, Distributed and Distributional Representations for Natural Language Processing in the Era of Deep Learning: a Survey
.
CoRR abs/1702
.00764 (
2017
), http://arxiv.org/abs/1702.00764
12.
Ferroni
,
P.
,
Zanzotto
,
F.M.
,
Scarpato
,
N.
,
Riondino
,
S.
,
Nanni
,
U.
,
Roselli
,
M.
,
Guadagni
,
F.
:
Risk Assessment for Venous Thromboembolism in ChemotherapyTreated Ambulatory Cancer Patients: A Machine Learning Approach
.
Medical Decision Making37
(
2
),
234
{
242
(
2017
)
13.
Gohd
,
C.
:
Your Next Teacher Could Be a Robot. https://futurism.com/arerobotic-teachers-the-future/(Feb2017
), https://futurism.com
/ are-robotic-teachers-the-future/
14.
Goodfellow
,
I.
,
Pouget-Abadie
,
J.
,
Mirza
,
M.
,
Xu
,
B.
,
Warde-Farley
,
D.
,
Ozair
,
S.
,
Courville
,
A.
,
Bengio
,
Y.
:
Generative Adversarial Nets
.
In: Advances in Neural Information Processing Systems
. pp.
2672
{
2680
(
2014
)
15.
Gray
,
A.
,
Ali
,
M.
,
Gao
,
Y.
,
Hedrick
,
J.
,
Borrelli
,
F.
:
Semi-Autonomous VehicleControl for Road Departure and Obstacle Avoidance
.
IFAC Control of Transportation Systems
pp.
1
{
6
(
2012
)
16.
Haykin
,
S.
: Neural Networks:
A ComprehensiveFoundation. Prentice HallPTR
, Upper Saddle River, NJ, USA, 2nd edn. (
1998
)
17.
He
,
K.
,
Zhang
,
X.
,
Ren
,
S.
,
Sun
, J.:
Identity Mappings in Deep Residual Networks
. In: Leibe,
B.
,
Matas
,
J.
,
Sebe
,
N.
,
Welling
, M. (eds.)
Proceedings of the European Conference on Computer Vision (ECCV)
. pp.
630
{
645
. Springer International Publishing,
Cham
(
2016
)
18.
Hu man
, E.C.,
Miner
,
J.L.
:
Robotic Sweeper Cleaner with Dusting Pad (Mar 182008
),
uS Patent 7
,
346
,
428
19.
Kennedy
,
J.F.
:
Inaugural Address of the President of United States (Jan1961
), washington, D.C.
20.
Kerly
,
A.
,
Hall
,
P.
,
Bull
,
S.
:
Bringing Chatbots into Education: Towards Natural Language Negotiation of Open Learner Modelsmodels
.
Knowledge-Based Systems 20(2)
,
177
{
185
(
2007
)
21.
Kim
,
B.
,
Malioutov
,
D.M.
,
Varshney
,
K.R.
,
Weller
,
A
. (eds.):
Proceedings of the ICML Workshop on Human Interpretability in Machine Learning (WHI) (Aug2017
)
22.
Kourou
,
K.
,
Exarchos
,
T.P.
,
Exarchos
,
K.P.
,
Karamouzis
,
M.V.
,
Fotiadis
,
D.I.: Machine Learning Applications in Cancer Prognosis and Prediction
.
Computational and Structural Biotechnology Journal13
,
8
{
17
(
2015
)
23.
LeCun
, Y.,
Bengio
,
Y.
,
Hinton
, G.:
Deep Learning
.
Nature521
(
7553
),
436
{
444
(
2015
)
24.
Lipson
,
H.
,
Kurman
,
M.
:
Driverless: Intelligent Cars and the Road Ahead
. MIT Press (
2016
)
25.
Lipton
,
Z.C.
:
The Mythos of Model Interpretability
.
In: Proceedings of ICML Workshop on Human Interpretability of Machine Learning
. vol.
abs/1606
.03490 (
2016
), http://arxiv.org/abs/1606.03490
26.
Litman
, T.:
Autonomous Vehicle Implementation Predictions
.
Victoria Transport Policy Institute28
(
2014
)
27.
Lutin
,
J.M.
,
Kornhauser
,
A.L.
,
Masce
,
E.L.L.
:
The Revolutionary Development of Self-Driving Vehicles and Implications for the Transportation Engineering Profession
.
Institute of Transportation Engineers Journal83
(
7
),
28
(
2013
)
28.
Manyika
,
J.
,
Chui
,
M.
,
Miremadi
,
M.
,
Bughin
,
J.
,
George
,
K.
,
Willmott
,
P.
,
Dewhurst
,
M.
:
A FutureThatWorks: Automation, Employment, and Productivity
.
McKinsey Global Institute (Jan2017
)
29.
Mnih
,
V.
,
Kavukcuoglu
,
K.
,
Silver
,
D.
,
Graves
,
A.
,
Antonoglou
,
I.
,
Wierstra
,
D.
,
Riedmiller
,
M.
:
Playing Atari with Deep Reinforcement Learning
.
arXiv preprint arXiv:1312.5602
(
2013
)
30.
Murphy
,
M.
:
Ginni Rometty on the End of Programming
.
Bloomberg Businessweek (Sep2017
), https://www.bloomberg.com/news/features/2017-09-20/ ginni-rometty
-on-artificial-intelligence
31.
Nakamoto
,
S.
: Bitcoin:
APeer-To-Peer Electronic Cash System
(
2008
)
32.
Plate
,
T.A.
:
Holographic Reduced Representations
.
IEEE Transactions on Neural Networks6
(
3
),
623
{
641
(
1995
). https://doi.org/10.1109/72.377968
33.
Plate
,
T.A.
:
Holographic Reduced Representation: Distributed Representation for Cognitive Structures
.
No. 150 in Lecture Notes
,
Center for the Study of Language and Information
, Stanford, California (
2003
)
34.
Quinlan
,
R.J.:C4:5: Programs for Machine Learning
. Morgan Kaufmann, San Mateo (
1993
)
35.
Revathi
,
G.
,
Dhulipala
,
V.S.
:
Smart Parking Systems and Sensors: A Survey
.
In: Proceedings of International Conference on Computing, Communication and Applications (ICCCA)
. pp.
1
{
5
.
IEEE
(
2012
)
36.
Schmidhuber
,
J.
:
Deep Learning in Neural Networks: An Overview. Neural Networks61
,
85
{
117
(
2015
)
37.
Silver
,
D.
,
Huang
,
A.
,
Maddison
,
C.J.
,
Guez
,
A.
,
Sifre
,
L.
, Van Den Driessche, G.,
Schrittwieser
,
J.
,
Antonoglou
,
I.
,
Panneershelvam
,
V.
,
Lanctot
,
M.
, et al.:
Mastering the Game of Go with Deep Neural Networks and Tree Search
.
Nature529
(
7587
),
484
{
489
(
2016
)
38.
Simonyan
,
K.
,
Zisserman
,
A.
:
Very Deep Convolutional Networks for Large-Scale Image Recognition
.
In: Proceedings of the International Conference on Learning Representations (ICLR)
(
2015
)
39.
Stone
,
P.
,
Brooks
,
R.
,
Brynjolfsson
,
E.
,
Calo
,
R.
,
Etzioni
,
O.
,
Hager
,
G.
,
Hirschberg
,
J.
,
Kalyanakrishnan
,
S.
,
Kamar
,
E.
,
Kraus
,
S.
,
Leyton-Brown
,
K.
,
Parkes
,
D.
, Press,
W.
,
Saxenian
,
A.
,
Shah
,
J.
,
Tambe
,
M.
,
Teller
,
A.
:
Arti cial Intelligence and Life in 2030
.
Tech. rep., One Hundred Year Study on Arti cial Intelligence: Report of the 2015-2016 Study Panel
, Stanford University, Stanford, CA (
September 2016
), http://ai100.stanford.edu/2016-report
40.
Straubhaar
, T.:
On the Economics of a Universal Basic Income
.
Intereconomics52
(
2
),
74
{80 (Mar
2017
). https://doi.org/10.1007/s10272-017-0649-8, https:// doi.org/10.1007/s10272-017-0649-8
41.
Taylor
, C.,
Parker
,
A.
,
Lau
,
S.
,
Blair
,
E.
,
Heninger
,
A.
,
Ng
,
E.
,
DiBernardo
, E.,
Witman
,
R.
,
Stout
,
M.
:
Robot Vacuum Cleaner (Jun 302005
),
uS Patent 11
,
171
,
031
42.
Trajkovic
,
M.
,
Colmenarez
,
A.J.
,
Gutta
,
S.
,
Trovato
,
K.I.
:
Computer Vision Based Parking Assistant (Jan 272004
),
uS Patent 6
,
683
,
539
43.
Tschorsch
,
F.
,
Scheuermann
,
B.
:
Bitcoin and Beyond: A Technical Survey on Decentralized Digital Currencies
.
IEEE Communications Surveys Tutorials18
(
3
),
2084
{
2123
(thirdquarter
2016
). https://doi.org/10.1109/COMST.
2016
.2535718
44.
Ulrich
,
I.
,
Mondada
,
F.
,
Nicoud
, J.D.: Autonomous Vacuum Cleaner.
Robotics and Autonomous Systems19
(
3-4
),
233
{
245
(
1997
)
45.
Vinyals
,
O.
,
Kaiser
, L.u.,
Koo
,
T.
,
Petrov
,
S.
,
Sutskever
,
I.
,
Hinton
, G.:
Grammar as a Foreign Language
. In: Cortes,
C.
,
Lawrence
,
N.D.
,
Lee
,
D.D.
,
Sugiyama
,
M.
,
Garnett
,
R
. (eds.)
Advances in Neural Information Processing Systems28
, vol.
2
, pp.
2773
{
2781
. Curran Associates, Inc. (
2015
), http://papers.nips.cc/paper/ 5635-grammar
-as-a-foreign-language
.pdf
46.
Vinyals
,
O.
,
Toshev
,
A.
,
Bengio
,
S.
,
Erhan
,
D.
:
Show and Tell: A Neural Image Caption Generator
.
In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
. pp.
3156
{
3164
(
2015
)
47.
Wallace
,
R.S.
: The Anatomy of
A.L.I.C.E.
, pp.
181
{
210
. Springer Netherlands, Dordrecht (
2009
). https://doi.org/10.1007/978-1-
4020
-6710-5 13, http://dx.doi. org/10.1007/978-1-
4020
-6710-5_
13
48.
Weiss
,
D.
,
Alberti
,
C.
, Collins,
M.
,
Petrov
,
S.
:
Structured Training for Neural Network Transition-Based Parsing
.
In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
. pp.
323
{
333
.
Association for Computational Linguistics (2015
). https://doi.org/10.3115/v1/
P15
- 1032, http://www.aclweb.org/anthology/P15-1032
49.
Weizenbaum
, J.:
ELIZA-A Computer Program for the Study of Natural Language Communication Between Man and Machine
.
Communications of the ACM9
(
1
),
36
{
45
(
1966
)
50.
Xu
,
K.
,
Ba
,
J.
,
Kiros
,
R.
,
Cho
,
K.
,
Courville
,
A.
,
Salakhudinov
,
R.
,
Zemel
,
R.
,
Bengio
,
Y.
:
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
. In: Bach,
F.
,
Blei
,
D
. (eds.)
Proceedings of the 32nd International Conference on Machine Learning. Proceedings of Machine Learning Research
, vol.
37
, pp.
2048
{
2057
. PMLR, Lille,
France (07{09 Jul2015
), http: //proceedings.mlr.press/v37/xuc15.html
51.
Zanzotto
,
F.M.
,
Ferrone
,
L.
,
Baroni
,
M.
:
When the Whole is Not Greater Than the Combination of Its Parts: A \Decompositional" Look at Compositional Distributional Semantics
.
Comput. Linguist
.
41
(
1
),
165
{173 (Mar
2015
). http://dx. doi.org/10.1162/COLI_a_
00215
52.
Zheng
,
Y.
,
Guerriero
,
M.
,
Lopez
,
E.V.
,
Haverman
,
P.
:
Universal Basic Income (UBI): A Policy Option for China beyond 2020? Tech. rep
.,
UNDP China O ce
(
2017
), http://www.cn.undp.org/content/china/en/home/ library/poverty/universal-basic-income-
-a-working-paper
.html
53.
Zou
, W.Y.,
Socher
,
R.
,
Cer
,
D.M.
,
Manning
, C.D.:
Bilingual Word Embeddings for Phrase-Based Machine Translation
.
In: Proceedings of the Empirical Natural Language Processing Conference (EmNLP)
. pp.
1393
{
1398
(
2013
)