=Paper=
{{Paper
|id=Vol-2052/paper14
|storemode=property
|title=From Natural Language to Argumentation and Cognitive Systems
|pdfUrl=https://ceur-ws.org/Vol-2052/paper14.pdf
|volume=Vol-2052
|authors=Theodoros Mitsikas,Nikolaos Spanoudakis,Petros Stefaneas,Antonis Kakas
|dblpUrl=https://dblp.org/rec/conf/commonsense/MitsikasSSK17
}}
==From Natural Language to Argumentation and Cognitive Systems==
https://ceur-ws.org/Vol-2052/paper14.pdf
From Natural Language to Argumentation and Cognitive Systems
Theodoros Mitsikas Nikolaos Spanoudakis
School of Applied School of Production Engineering and Management,
Mathematical and Physical Sciences, Technical University of Crete,
National Technical University of Athens, nikos@science.tuc.gr
mitsikas@central.ntua.gr
Petros Stefaneas Antonis Kakas
School of Applied Mathematical and Physical Sciences, Department of Computer Science,
National Technical University of Athens, University of Cyprus,
petros@math.ntua.gr antonis@ucy.ac.cy
Abstract “The quality of food is very important for me. When
possible try to economize. I like to eat organic food. I
This paper reports on a program for the development of cogni- am not diabetic but I like to avoid sugary foods. I prefer
tive systems based on user specifications expressed in Natural not to eat red meat except for special occasions.”
Language in the form of high-level guidelines for the desired
behaviour of those systems. Language analysis from existing This would be followed by implicit evaluations of pur-
NLP tools is used to generate scenarios based on the users’ chases suggested or made by the cognitive assistant, e.g.:
description of their preferences in the provision of the service
by the system. These scenarios are combined with the SoDA “The fish last night was very good but not enough.”
Methodology to develop and generate an argumentation theory
that captures the guidelines of operation given in the natural Such advanced expressions of specification forms the ulti-
language description of the user. The intended architecture of mate final goal of our work, namely, to be able to automati-
a fully automated system that generates argumentation based cally capture these natural language specifications and their
cognitive systems from natural language is presented, together contextual meaning so that we can build automated systems
with a first analysis of the challenges posed and evaluation of whose operation is influenced or governed by these guide-
progress in addressing them. This system is called Gorgias-NL,
lines. This goal can be addressed, to a large extent, modularly
reflecting the central task of the approach to provide a natural
language interface to the Gorgias argumentation system. in two stages. In the first stage the challenge is the linguistic
extraction of (i) the predicate relations expressed in the natu-
ral language text and (ii) the preference meaning conveyed
1 Introduction explicitly or implicitly in the text.
This preference information is relative amongst different
We are today experiencing a dramatic increase in the in- entities at various levels, e.g. in the example above a pref-
terest of developing systems that have a human-like be- erence of quality over economy is expressed as well as that
haviour in providing personalized services to users. This of organic food over non-organic food. Terms like quality or
is mainly driven by the market for systems that will be able economy of food do not have a full meaning until they are
to help people in their every day life in a naturally intel- immersed into common sense knowledge in the context of
ligent way. Such systems, fall under the area of Cognitive the application discourse. Acquiring, organizing and using
Systems (Langley, Laird, and Rogers 2009; Langley 2012; the relevant common sense knowledge forms a second part
Michael et al. 2015) needing to interact with their users in of our program for the development of these systems, outside
a natural way and generally exhibit a behaviour that is cog- the scope of this paper.
nitively compatible with that of human users without any
technical knowledge, but rather relying on the common sense
knowledge that people have and utilize through their use
1.1 Argumentation for Cognitive Systems
of natural language. In general, cognitive systems are thus The basis of our approach will be that of argumentation and
envisaged to capture and learn their users’ personal require- its computational form as developed in Artificial Intelligence
ments through dialogues in natural language that are high- over the last two decades (see e.g. (Bench-Capon and Dunne
level and non-reliant on technical knowledge. The central 2007; Rahwan and Simari 2009) for an overall view). Argu-
challenge then, for building such systems, is to bridge the mentation will form the “middleware” between the natural
gap between Natural Language used by humans and Ma- language user input and the machine language on upon which
chine Languages through which these systems are built and the systems will be implemented. There are many reasons
executed on today’s computers. for choosing argumentation as the “technology basis”, such
For example, we would like a user to be able to specify as its natural link to human reasoning/thinking (Mercier and
her/his guidelines for their own Cognitive Assistant for online Sperber 2011) and the computational form that it exhibits in
shopping in the following manner: its dialectical process.
Specifically, our approach aims to produce, from the natu- Assistant. Their development is driven by their direct link
ral language user guidelines, an argumentation theory com- to the market and they use methods targeted to the specific
posed of arguments, a conflict relation between arguments needs of their application. Google Assistant1 and Alexa2 use
and preference information that would give strength of some phrase matching, which triggers the appropriate action or
arguments over others. As mentioned above, we will separate event. Siri3 registers specific vocabulary to identify user’s
the task into two phases. In the first phase we use standard intents. Cortana analyses queries using bag-of-words models
Natural Language Processing (NLP) tools, such as that of and a classifier, after eliminating noise (non-alphabetic and
Stanford Core NLP (Manning et al. 2014) to syntactically non-English words) and applying stemming (Elwany and
analyse the text sentences of the guidelines. This analysis is Shakeri 2014). Similarly, in IBM’s Watson, linguistic analysis
used to extract the symbolic predicate relations that the guide- of natural language input is carried out by two deep parsing
lines document contains. These are then used to form typical components, an English Slot Grammar (ESG) parser and
application scenarios in terms of collections of conditions un- a predicate-argument structure (PAS), followed by pattern-
der which various decisions or options are sanctioned. Hence, based relation extraction (McCord, Murdock, and Boguraev
in the example above we may form a scenario where “a food 2012; Lally et al. 2012).
with high quality sanctions the option to buy it” whereas Other notable assistants are the open-source projects Lu-
another scenario would specify that “a food with a high price cida4 and Mycroft5 . Lucida uses OpenEphyra, a Java frame-
sanctions it the decision not to buy it”. work, as an internal parser to extract textual information and
Preference information is also based on the linguistic anal- answer queries, plus a Wikipedia database stored in Lemur’s
ysis of the text and patterns of expression in the text which Indri format. Mycroft’s parsers take in a sentence and attempt
contain implicitly the preferences of the user. The aim is to to find user’s intention by searching for matching words and
capture these through refined scenarios where the options phrases from registered intents and vocabulary.
sanctioned differ from those of the more general scenario. In comparison, our approach rests on argumentation as a
For example, we would have a general scenario where “food principled foundation for capturing human reasoning. The
that contains red meat is sanctioned as not suitably to buy” emphasis is not so much on the underlying learning technol-
but a refined scenario of “food that contains red meat for ogy for analysing natural language input but on the subse-
a special occasion is sanctioned as suitably to buy”. Some- quent use of the information extracted. Our aim is to attempt
times this would be clear from the text but not always. For to give this information a symbolic logical form, using ar-
example what is the preference of the user above for food of gumentation as a basis for common sense logical reasoning,
high-quality but also of high-price? Such combined scenarios rather than a direct operational meaning. Also unlike many
could be put to the user for expressing more directly her/his methods for Natural Language Understanding our linguistic
preferences or could be kept “in mind” by the system waiting analysis does not aim to translate directly the text into some
for further information on the possible user preference. formal logical (classical or non-classical) language but rather
The consideration of combined application scenarios fol- to form typical scenarios or partial models that capture the
lows the SoDA (Software Development through Argumenta- meaning of the text.
tion) Methodology (Spanoudakis, Kakas, and Moraitis 2016; After a brief review of argumentation, the next section de-
2017) for developing applications of argumentation. This scribes the extraction of the predicate relations expressed in
methodology gives a principled and systematic way of the natural language text. It also gives the overall architecture
considering preferences in scenarios and their refinements. of the Gorgias-NL system that implements our approach. The
This methodology then enables the automatic generation functionality of the system is illustrated through an example
of an argumentation theory within the preference-based of scenario generation and argument extraction. The initial
argumentation framework of the Gorgias system (Kakas, evaluation, challenges concerning open NLP problems, and
Mancarella, and Dung 1994; Dimopoulos and Kakas 1995; the challenges emerging through our development and evalu-
Kakas and Moraitis 2003) capturing the scenario informa- ation process are presented in Section 4. Finally, in Section 5
tion. The generated argumentation theory and implemen- we conclude with a brief description of future work needed
tation code is transparent to the user but can be executed to complete our program of work.
directly under the Gorgias system to suggest choices under
the user guidelines and to provide an explanation support- 2 Argumentation and SoDA Methodology
ing these choices in terms of the argumentation process that
In this section we briefly review the basic theory of argumen-
makes them suitable choices.
tation on which our approach rests and the SoDA Method-
Our approach is implemented in a system, called Gorgias-
ology used for developing the argumentation code that cap-
NL, whose aim is to pipeline together these different stages
ture the user’s guidelines. This framework was proposed in
of analysis from the natural language text of the user to the
(Kakas, Mancarella, and Dung 1994) and further developed
Gorgias argumentation code that captures this.
later in (Kakas and Moraitis 2003) as a preference-based
1.2 Related Work 1
https://developers.google.com/actions/
2
Our approach falls within the general effort of developing https://aws.amazon.com/lex/
3
human-like AI systems that link directly with their human https://developer.apple.com/sirikit/
4
users in Natural Language. Such systems include proprietary http://www.lucida.ai/
5
personal assistants, e.g. Alexa, Cortana, Siri, and Google https://mycroft.ai/
argumentation framework with a view to support flexible 3 Gorgias-NL Design and First Version
decision making for autonomous agents. Our approach is based on extracting typical scenarios repre-
Application requirements are captured via argumentation senting the user’s preferences from the text document. A two
theories composed of arguments at different levels. Object step process is used. First, initial scenarios are extracted via a
level arguments support the possible decisions, or options, natural language analysis. Subsequently, the SoDA Methodol-
in the specific application domain, while first-level priority ogy is applied, where the combined and refined scenarios are
arguments express preferences on the object level arguments extracted. These scenarios constitute a complete argumenta-
in order to resolve possible conflicts. Higher-order priority tion software which implements the user’s preferences.
arguments are also used to resolve potential conflicts between
priority arguments of the first or subsequent levels. 3.1 Natural Language Understanding for
Cognitive Assistants
Argumentation theories are expressed in this framework in
A Cognitive Assistant must be capable of extracting the gen-
the language of rules and priorities on rules. An argumenta-
eral and specific contexts via language analysis, without the
tion theory is a pair (T , P) whose sentences are rules of the
need for detailed operational instructions. The user’s lan-
form L ← Body, where Body is a conjunction of positive
guage and vocabulary used in this domain consist of a subset
or negative literals. Rules in T capture argument schemes for
of a Natural Language concerning everyday tasks.
building object level arguments. On the other hand, rules
The user will set the general guidelines and probably place
in P represent argument schemes for building priority ar-
an emphasis on distinct cases, possibly expressing exceptions
guments. The head L of these rules has the general form, L
to the guidelines s/he sets. This interaction has not necessar-
=pref er(rule1, rule2), where rule1 and rule2 are atoms
ily an order or a time frame. New guidelines or exceptions
naming two rules and pref er refers to an (irreflexive) higher
can arise, as the user needs, the environment, and the cog-
priority relation amongst the rules of the theory.
nitive system evolve. A guideline can contain conditions,
The semantics of an argumentation theory is defined via exceptions, actions, or any combination of them.
the standard notion of admissibility within an abstract argu- The extraction of guidelines and preferences involves find-
mentation framework < Args, Att > associated to any given ing important words of a sentence and generating a knowl-
theory (T , P). In this the attack relation is defined through edge representation corresponding to the user’s guidelines.
the conflicts between literals or between opposing priorities, Combined with a representation of commonsense knowl-
prefer(r,r’) and prefer(r’,r) and a notion of relative strength edge, the Cognitive System can reason over user preferences
for the attacking argument that is build through the priority and user queries expressed using the same subset of Natural
rules that the arguments contain (see (Kakas and Moraitis Language. These would typically provide information about
2003) for the details). the user’s current needs and request a recommendation on
what decision to take. The query, considers the current en-
The simplicity of this argumentation framework facili- vironment of the user and returns options that fall under the
tates the systematic “Software Development for Argumen- personal guidelines of the user.
tation”(SoDA) methodology for developing applications of
argumentation through a process of directly mapping the 3.2 Gorgias-NL Architecture
user or system requirements to an argumentation theory. The
main feature of this methodology is the incremental consid- Gorgias-NL is based on a modular design, which is depicted
eration of scenarios in which the developer is asked to state in Fig. 1. All modules can function independently, and they
the default preferred behaviour (i.e. preferred options) of communicate using simple specifications. Firstly, the user’s
the system and subsequently to consider refinements of the text input concerning preferences and policies, is processed
scenarios where this preference changes. The methodology by the NLP module. The NLP module relies on state-of-
concentrates on scenarios where conflicting options can oc- the-art NLP software to obtain the language analysis for
cur and for each such case a process of refinement is initiated. the given text input. Language analysis includes coreference
A tool, called Gorgias-B (http://gorgiasb.tuc.gr), helps the de- resolution, i.e. determining possible expressions that refer
veloper to consider his/her application according to the SoDA to the same person, object or thing. At the current stage of
methodology and offers a high-level environment through implementation, those are performed by Stanford NLP, but
which the software code of the underlying argumentation any other software that outputs CoNNL-U data can be used
theory is automatically generated. with minor modifications.
Afterwards, the module responsible for scenario extraction
The important characteristic of the methodology that stems uses the language analysis input to generate the preliminary
from the declarative nature of the underlying argumentation scenarios. Although developed as a module for Gorgias-NL,
framework is that the interaction with the developer is car- it can be used independently as it generates conditions and op-
ried out in the high-level application language and hence no tions/actions from natural language. Within Gorgias-NL, the
technical knowledge is needed by the developer. This is very generated scenarios capture the user’s general preferences.
important in our work as it indeed allows the users to express Scenario generation is followed by application of the SoDA
their requirements in Natural Language without the need Methodology, to obtain the refined scenarios which capture
to consider how this will be mapped into an argumentation the general user preferences. They form a consistent set of
theory or rules and priority rules. arguments, to be later used at the decision level.
(e.g. ”that”, if it is introducing a “that” clause). Forms of “be”
User
are also ignored, e.g. if it is a copular verb. Determiners as
Speech possessive pronouns are taken into account. Articles, while
to text they do not appear in the final form of scenarios, are used to
Common determine if a term is a variable or a constant, together with
Sense Environment
Text Input NLP Knowledge the grammatical number of nouns.
Queries Actions The generated scenarios are of the form . The no-
Scenario SoDA Argumentation tation is as follows: scenario id is a number used to
Genaration Methodology Software
identify the scenario, sentence id is a list of numbers
corresponding to the natural language sentence(s) which the
Figure 1: Design of the prototype system scenario refers to, Conditions is a list of zero or more
predicates defining the context that constitutes the scenario,
and Options is a list of one or more predicates correspond-
The derived final argumentation software captures the pre-
ing to decision options and which are valid in the scenario.
ferred guidelines of operation of the user, while being con-
To illustrate how scenarios are extracted from the guide-
sistent. Combined with the common sense knowledge, the
lines text consider the following example concerning the
environment, and user specific knowledge, queries can be
management of discount offers. “Normally, discard coupons.
answered or actions can be taken.
If a coupon is related to my wish list, save it, unless it is ex-
3.3 Scenario generation pensive. If a coupon offers a large discount, save it. Discard
the coupons that are out-of-date.” After coreference resolu-
For the scenario generation, the prototype is using a strict tion and language analysis performed by Stanford NLP, the
grammar-to-model parser. Syntactic structures are translated scenario generation yields the following scenarios:
to predicates, along with their corresponding terms, in order <1, 1, {}, discard(Coupon)>
to extract the first level symbolic information. Predicates <2, 2, {related to(Coupon,wish list)},
derived from the root of the sentence, identify the option or save(Coupon)>
the decision of the scenario. An overview of this is as follows: <3, 2, {expensive(Coupon),
Copular Verbs The predicate is the conditional clause, related to(Coupon,wish list)},
while the subject is a term (e.g. “the coupon is out-of-date” neg(save(Coupon))>
generates the predicate out of date(coupon)). <4, 3, {large(discount),
Auxiliary Verbs At this initial stage of our work we are offer(Coupon,discount)}, save(Coupon)>
assuming that auxiliary verbs do not form part of the pred- <5, 4, {out of date(Coupon)},
icate. Such verbs can qualify the predicate and its link to discard(Coupon)>
other information in the sentence but this would only be As shown above, each sentence can generate more than
important if it makes a difference on a typical scenario. one scenario, e.g. if an exception is included. The latter
Verbs The other verbs are predicates, while their respective part of the second sentence “unless the coupon is expen-
subjects, objects, and noun modifiers are terms. In the sive” implies an additional scenario which includes the extra
domain of a “cognitive assistant”, a greater focus needs to condition expensive(Coupon) and the unique option
be given in imperative mood. neg(save(Coupon)). This option is derived from the
initial natural language sentence which does not provide an
Adjectives Adjective phrases are predicates, and the cor- alternative option, thus the negation is used. For simplic-
responding noun is the term (e.g. “Pick up some cheap ity, we will assume that we have additional information to
wine. . . ” is cheap(wine)). identify the two options of not saving and discarding.
Adverbs Adverbs and adverb phrases can be terms (e.g.
“Wake me up tomorrow” is wake me up(tomorrow)), 3.4 Argument extraction from context scenarios
or can be ignored. Given a set of primary scenarios extracted directly from the
Prepositions Prepositions are used as a variation of the pred- text as in the above example we then need to consider combi-
icate (e.g. “When I’m at work. . . ” is at work), or can be nations of these scenarios to further capture the guidelines.
ignored. The SoDA methodology helps us to consider such combi-
Lemmatisation Predicates and terms are lemmatised. Ex- nations in a structured and systematic way. For example,
ceptions to this include gerund, participles, and past tense. considering the combination of scenarios 2 and 5 we get a
new combined scenario:
Preference Several adverbs and prepositions can state ex- <6, {2,4}, {related to(C,wish list),
plicitly a user preference (e.g. “Normally allow calls”, “I out of date(C)}, {save(C), discard(C)}>
prefer fish over meat”). where initially both options are possible. At this stage, we
This analysis can ignore some words of a sentence. While consider the task of making a combined scenario more deter-
some information can be lost, it is not always essential. Sim- ministic by trying to ascertain if the user has some (implicit)
ilar to (McCord, Murdock, and Boguraev 2012), we are ig- preference in such a scenario. For example, we can use prag-
noring auxiliary verbs and words that introduce verb phrases matic information that comes from the common sense knowl-
edge that typically out of date coupons to give preference to node and target node have the same set of options. The second
the discard option. This is a heuristic process that can also be pass removes all nodes that are not the root and there is no
captured through argumentation, namely using the common edge having them as targets. The final context graph for our
sense and practical information to support arguments for the example is shown in Fig. 3.
various options in a combined scenario, as in the example
above where we have a strong argument to discard. In cases,
where it is not possible to decide in favour of a smaller set
of preferred options in a combined scenario we can either
ask the user for extra information on this or we can wait
until some information that can help with this is acquired by
the cognitive assistant. In our running example one possible
outcome of this process could give the following combined
scenarios.
<7, {3,4}, {large(discount),
offer(C,discount), out of date(C)},
discard(C)>
<8, {2,3}, {expensive(C),
related to(C,wish list),
large(discount), offer(C,discount)}, Figure 3: The refined context graph after the second step
{save(C), discard(C)}> where all edges that connect same coloured nodes have been
<9, {2,3,4}, {expensive(C), eliminated along with node 2.2 that becomes orphan
related to(C,wish list),large(discount),
offer(C,discount), out of date(C)},
discard(C)>
An algorithm is then applied that results in the generation The algorithm then transforms the context graph into a
of a Gorgias argumentation theory from such context sce- structure called threads in the SoDA methodology. This con-
narios extracted from the text. This algorithm first constructs siders any path from the root to a leaf for each pair of op-
a context graph that puts scenarios in levels from the more tions and creates a corresponding thread by defining all the
general contexts to more specific ones. The context graph is a possible contexts - combinations of conditions - from the
graph whose nodes correspond to scenarios. The nodes have most general to the more specific, under which a decision
a name (corresponding to a scenario), a set of options that are between a pair of options is reached. In general, a thread,
valid in the node’s scenario and a level integer. Its root (first T ij , is based on a pair of conflicting options Oi and Oj , and
node that has only outgoing edges) is a node named true. is a sequence, {P k } (1 ≤ k ≤ n), where each P k has the
The algorithm to transform scenarios to such a graph starts form, P k =< S k , Cij k k
, Cji >, where S k , Cij
k k
, Cji are sets of
from the “true scenario” at the root and then gradually adds k k−1 k−1 k−1
scenario conditions, such that S = S ∪ Cij ∪ Cji
scenarios from simple ones to more complex connecting with (and hence S 1 ⊆ S 2 ⊆ ... ⊆ S n ). Here Cij k
and Cji k
are
an edge those that are more specific contexts of others. Fig. 2 contextual conditions (possibly empty) such that Oi is pre-
shows this initial graph for our example. Note that each node ferred over Oj in the consistent scenario S k ∪ Cij k
and Oj
has its name on its label. For simplicity of representation, k k
and since we only have two options, the valid option at each is preferred over Oi in the consistent scenario S ∪ Cji .
node is denoted by the colour of its background, white for Hence, in a thread based on two conflicting options we
discard(C) and grey for save(C). Hence a note which is half iteratively consider (consistent) refinements of an application
white and half grey, e.g. node 4.1, designates that both options scenario and at each such level, k, of iteration new contex-
are possible. The context graph is refined in two passes. The tual information is considered, denoted by Cij k k
or Cji , under
which one of the options is preferred over the other. Con-
textual information can be given for none of the preferences
k k
(in which case both Cij and Cji are equal to the empty set),
for one or for both preferences between the two options Oi
and Oj . In the special case of the contextual information not
imposing any further refinement of the current scenario, i.e.
k k
when Cij = {true} (or Cji = {true}), we have a default
preference (at level k) of Oi over Oj (resp. Oj over Oi ) and
the thread stops.
Algorithm 1 shows the process of transforming the context
graph to threads. Figure 4 shows the four threads for our
Figure 2: The graph of the generated scenarios example. The first thread T1 is given below, where it should
be noted that the node with two valid options causes the T1
first one removes all edges that connect nodes whose source thread to fork creating T12 and T11:
for each pair of conflicting options do
for each path that contains these options do
Start a new thread for each node starting from the root
to the leaf do
add P level =< S level =
level level
{node name}, Cij = {}, Cji = {} >
if (Oi is valid in node)) then
level
Cij = {true}
if (Oj is valid in node)) then
level
Cji = {true}
level level
if Cij 6= {} ∧ Cji 6= {} then
This is the special case where both options Figure 4: The threads depicted graphically in our context
are valid in this node. Fork the thread and graph. T1 is the bold dashed line (with large dashes), T11 the
level
make the Cij = {true} in the existing forked thread from the previous one shown with the dotted
level
thread and Cji = {true} in the other line and T2 shown with the solid line. T22 forks from T2
thread (dashed line with small dashes). Note that T22 does not
for each next node starting from level towards the include node 3.2 (the arrow just passes over it in the figure)
leaf do
if
level
(Oi ∈ option(next node)) ∧ Cij = {}
then From the threads we can automatically generate a corre-
Cijlevel
= sponding Gorgias argumentation theory that captures the
{name(next node) − name(node)} arguments for each of the options involved in a thread and
node = next node their relative strength at each level of the thread. This is done
by generating argument rules that support the priority of one
argument rule over another at the previous level. The full
Algorithm 1: For generating the scenarios graph details of this are omitted due to lack of space. They can be
found in (Spanoudakis, Kakas, and Moraitis 2017).
The generated argumentation theory can be executed using
the Gorgias-B tool (http://gorgiasb.tuc.gr) to reach decisions
T 1 (O1 = discard(C), O2 = save(C)) = { under given current circumstances. For our coupons example,
P 0 =< S 0 = {true}, C12
0
= {true}, given a specific coupon c1 for which we know it offers a
0 large discount Gorgias-B will return the decision save(c1)
C21 = {related to(C, wish list)} >, together with the explanation:
P 1 =< S 1 = {related to(C, wish list)},
1 1 “When [offer(c1, discount), large(discount)] choose save(c1)”
C12 = {expensive(C)}, C21 = {true} >,
P 2 < S 2 = {related to(C, wish list),expensive(C)}, based on the arguments that support this decision. If
2
C12 = {true}, in addition we know that this coupon is out of date (e.g.
as time passes) Gorgias-B will now return the decision
3 3
P =< S = {related to(C, wish list), discard(c1) together with the new explanation:
expensive(C), large(discount), off er(C, discount)},
3 3 “When [offer(c1, discount), out of date(c1), large(discount)]
C12 = {out of date(C)}, C21 = {true} >, prefer discard(c1) over save(c1)”.
P 4 =< S 4 = {related to(C, wish list), These explanations can now be transformed into natural
expensive(C), large(discount), language to be presented to the user if and when requested.
off er(C, discount), out of date(C)},
4 4
4 Evaluation and Challenges
C12 = {true}, C21 = {} >}
Our initial evaluation sought to establish evidence for the
T 12 (O1 = discard(C), O2 = save(C)) = { feasibility of our approach, to uncover possible weaknesses
P 3 =< S 3 = {related to(C, wish list), in the implementation and to point out future challenges. We
describe here briefly our plans for evaluation of our approach.
expensive(C), large(discount),
3 3
off er(C, discount)}, C12 = {true}, C21 = {} >, 4.1 Evaluation
11 There is no widely accepted evaluation data or tests relevant
T (O1 = discard(C), O2 = save(C)) = {
1 1 to the domain of a cognitive assistant. In order to obtain a
P =< S = {related to(C, wish list)}, preliminary evaluation of the prototype system, we asked a
1 1
C12 = {out of date(C)}, C21 = {true} >, group of users to express their preferences in a very short
paragraph in natural language, on a task of their own choice.
P 2 =< S 2 = {related to(C, wish list),
2 2
out of date(C)}, C12 = {true}, C21 = {} >}
The generated scenarios captured satisfactory the overall Normally allow calls. If an advertiser calls, don’t an-
preferences of roughly 70% of the sentences expressed by swer the call. If the parser is strictly based on gram-
the users. Examples of user preferences are: (i)“Categorize mar and structures, the generated first scenario is
work emails by their code in the subject. If I’m cced, put the <1,1,{},allow(Call)>. In the second sentence, the
email in the relevant folder. If the email is personal, read the word “calls” is a verb, unlike the word “calls” in the first
subject of the email to me, except if I’m at a doctor.” and (ii) sentence which is a noun and the direct object. That will
“If I have a meeting with a client, make sure the appropriate lead to different predicates and terms comparing e.g. to
documents according to the client are on my desktop. If I the sentence “Deny calls from advertisers”. Moreover, the
have a meeting with Mr. Doe interrupt us after 10 minutes possible different predicates e.g. allow, answer and deny,
with something urgent! Remind me to pick up my kids every don’t answer, are the same, respectively, in the context of
day at 3:00 except from Tuesday, when you should remind how to handle a phone call, given this simple policy.
me at 1:00.”. Examples like the last two sentences were only • Communication between human and computer in a context
partially captured, due to duration, date and time mentions, of a Cognitive Personal Assistant requires identification
which the prototype did not support during the evaluation of the type of a sentence. A given sentence can be a query
stages. (e.g. “Where is my car?”), a command (e.g. “Turn off the
We have used this evaluation exercise to help improve our lights.”), a fact (e.g. “I parked my car at. . . ”), or part of a
natural language parser into scenarios. In future we will con- policy (e.g. “Normally discard coupons.”). Identifying the
tinue to perform such evaluation exercises, with larger groups type of the sentence is can benefit from work on question
of users, for early feedback on the system’s performance. and intent classification.
We also note that the evaluation process has reveal interest-
ing and useful information about how human users perceive Together with these challenges we also plan to explore
cognitive systems/assistants. For example, initially no user other state of the art methods and tools for NLP such as
expressed a high-level policy or an exception within a policy. neural networks with grammar-to-model training, or the use
Some users expressed simple if-then rules or simple com- of Combinatory Categorial Grammars, e.g. openCCG 6 and
mands (those are excluded from the above results). Other compare their utility.
users, initially expressed sentences containing an excessive
amount of non-essential information. More complex cases 5 Conclusions and Future Work
with exceptions, and without inessential information were We have presented the design of Gorgias-NL, an
expressed after an example was given, or after specific instruc- argumentation-based system for building cognitive personal
tions to express themselves more freely, as in a conversation assistants, together with its first implementation. Gorgias-NL
with an “intelligent” human. One example concerning a shop- is based on a modular design, which uses a NLP parser that
ping assistant, specified by a user, is the sentence “I would generates scenarios from user guidelines and a module fol-
like you to inform me or alert me when you find through lowing the SoDA methodology to generate an argumentation
Google medium dresses for the summer in green or white theory capturing the guidelines.
colour.”, which our parser failed to capture. After a request We plan to continue the process of further development of
to repeat the same guideline as it would be addressed to a the system together with an evaluation with human users so
human personal assistant, the user rephrased the sentence that we can test early the ecological validity of our design
as “Tell me if you find medium summer dresses in green or decisions. In a fully automated system we aim to allow the
white.”, which the parser captured. This raises interesting user to confirm or not its recommendations or if the user
questions about how people perceive an artificial cognitive wishes to engage into a (structured) dialogue with the system
assistant and how thus they behave when interacting with in order to be persuaded or not about the recommendation of
such a system. the cognitive assistant. This dialogue will form a new input
for the system that could enrich the argumentation theory
4.2 Future Linguistic Challenges capturing the user’s guidelines.
Challenges concerning the interaction of the user with a cog- The most important future development is that of exploring
nitive system through automatic NLP include: the use of common sense knowledge in the approach and the
• Language analysis, coreference and ellipsis resolution of design of our systems. Its use will be important at various
NLP software is not, yet, perfect. Furthermore, the analysis levels. For example, it can be used in helping to understand
of complex sentences depends on the right punctuation of implicit preferences of the user and thus resolve conflicts
user’s input. One example is the phrase “If I have a meet- when we consider combinations of scenarios under the SoDA
ing with a client, make sure the appropriate documents, Methodology.
according to the client, are on top of the desk”. Depending Common sense Knowledge will also be important when
on where the commas are, the language analysis differs, our cognitive systems, once build, are deployed in a real en-
and the respectively generated scenarios do not necessarily vironment where they will need to decide on what options
capture the user’s preferences. are best suited for their users. During their execution they
will need to link the current “sensory information” that the
• Language variability: Reasoning on the generated scenar- system will be getting from its application environment with
ios can be impossible without using the context, even for
6
relatively simple inputs. Consider the following sentences: http://openccg.sourceforge.net/
the high-level concepts used to express the personal user McCord, M. C.; Murdock, J. W.; and Boguraev, B. K. 2012.
guidelines. Common Sense Knowledge will help recognize Deep parsing in Watson. IBM Journal of Research and De-
which of these high-level concepts hold in any given current velopment 56(3.4):3:1–3:15.
situation and hence which arguments apply in the current Mercier, H., and Sperber, D. 2011. Why Do Humans Reason?
case of reasoning by the system. This recognition can be Arguments for an Argumentative Theory. Behavioral and
done as a subsidiary argumentation process on the beliefs of Brain Sciences 34(2):57–74.
the system based on common sense arguments, sanctioning
Michael, L.; Kakas, A.; Miller, R.; and Turán, G. 2015. Cog-
them from the sensory information framework (see (Diakidoy
nitive Programming. In Proceedings of the 3rd International
et al. 2014) where common sense knowledge is used in this
Workshop on Artificial Intelligence and Cognition (AIC), 3–
way for narrative text comprehension). The Gorgias frame-
18.
work supports this two-level argumentation process, one for
options and one for beliefs on which options are decided and Rahwan, I., and Simari, G. R. 2009. Argumentation in
hence this facilitates the integration of common sense knowl- Artificial Intelligence. Springer Publishing Company, Incor-
edge uniformly in terms of argumentation. Nevertheless, the porated, 1st edition.
main challenge with common sense knowledge is acquiring, Spanoudakis, N. I.; Kakas, A. C.; and Moraitis, P. 2016.
organizing, and accessing the relevant part of common sense Applications of Argumentation: The SoDA Methodology.
knowledge when this needs to be used. In ECAI 2016 - 22nd European Conference on Artificial
Intelligence, 29 August-2 September 2016, The Hague, The
References Netherlands, 1722–1723.
Bench-Capon, T. J. M., and Dunne, P. E. 2007. Argumenta- Spanoudakis, N. I.; Kakas, A. C.; and Moraitis, P. 2017. The
tion in Artificial Intelligence. Artificial Intelligence 171(10- SoDA Methodology and Gorgias-B Tool for Developing Ar-
15):619–641. gumentation Systems, AMCL-TR 01 2017. Technical report,
Applied Mathematics and Computers Laboratory, Technical
Diakidoy, I.-A.; A.; Michael, L.; and Miller, R. 2014. Story University of Crete, Chania, Greece.
Comprehension through Argumentation. In Proceedings of
the 5th International Conference on Computational Models
of Argument (COMMA), 31–42.
Dimopoulos, Y., and Kakas, A. C. 1995. Logic programming
without negation as failure. In Logic Programming, Proceed-
ings of the 1995 International Symposium, Portland, Oregon,
USA, December 4-7, 1995, 369–383.
Elwany, E., and Shakeri, S. 2014. Enhancing Cortana User
Experience Using Machine Learning. Recall 55(54.61):24–
24.
Kakas, A., and Moraitis, P. 2003. Argumentation based
decision making for autonomous agents. In Proceedings of
the Second International Joint Conference on Autonomous
Agents and Multiagent Systems, AAMAS ’03, 883–890. New
York, NY, USA: ACM.
Kakas, A. C.; Mancarella, P.; and Dung, P. M. 1994. The ac-
ceptability semantics for logic programs. In Logic Program-
ming, Proceedings of the Eleventh International Conference
on Logic Programming, Santa Marherita Ligure, Italy, June
13-18, 1994, 504–519.
Lally, A.; Prager, J. M.; McCord, M. C.; Boguraev, B. K.;
Patwardhan, S.; Fan, J.; Fodor, P.; and Chu-Carroll, J. 2012.
Question analysis: How Watson reads a clue. IBM Journal of
Research and Development 56(3.4):2:1–2:14.
Langley, P.; Laird, J. E.; and Rogers, S. 2009. Cognitive
architectures: Research issues and challenges. Cognitive
Systems Research 10(2):141 – 160.
Langley, P. 2012. The cognitive systems paradigm. In
Advances in Cognitive Systems, 3–13.
Manning, C. D.; Surdeanu, M.; Bauer, J.; Finkel, J.; Bethard,
S. J.; and McClosky, D. 2014. The Stanford CoreNLP
natural language processing toolkit. In Association for Com-
putational Linguistics (ACL) System Demonstrations, 55–60.