=Paper=
{{Paper
|id=Vol-2956/paper32
|storemode=property
|title=Commonsense Reasoning with Argumentation for Cognitive Robotics
|pdfUrl=https://ceur-ws.org/Vol-2956/paper32.pdf
|volume=Vol-2956
|authors=Alexandros Vassiliades,Nick Bassiliades,Theodore Patkos
|dblpUrl=https://dblp.org/rec/conf/ruleml/VassiliadesBP21
}}
==Commonsense Reasoning with Argumentation for Cognitive Robotics==
https://ceur-ws.org/Vol-2956/paper32.pdf
Commonsense Reasoning with Argumentation for
Cognitive Robotics?
Alexandros Vassiliades1,2[0000−0003−4569−503X] , Nick
Bassiliades1[0000−0001−6035−1038] , and Theodore Patkos2
1
Aristotle University, School of Informatics, Thessaloniki, Greece
{valexande,nbassili}@csd.auth.gr
2
Foundation for Research and Technology, Institute of Computer Science, Heraklion, Greece
patkos@ics.forth.gr
Abstract. Commonsense Reasoning is a cognitive ability which is found only in
humans, but the desire is to implement it in Artificial Intelligence when solving
tasks. Moreover, the use of arguments can reveal how and why a human individual
supports or rejects an opinion. In this thesis, our goal is to study theoretically the
problem of commonsense reasoning and develop methods for enhancing the com-
monsense reasoning capabilities of a cognitive robotic system that acts in a house-
hold environment. The commonsense reasoning mechanism is implemented with
the use of argumentation, by developing argumentation frameworks that can facil-
itate commonsense knowledge. Additionally, we use Semantic Web technologies
to add commonsense knowledge in the commonsense reasoning mechanism, and
we construct a framework that accommodates our methods.
Keywords: Commonsense Reasoning · Argumentation · Semantic Web · Cogni-
tive Robotics.
1 Introduction
Commonsense Reasoning (CR) is a cognitive characteristic that is found only in human
individuals, but is a long desired aspect that Artificial Intelligence (AI) should have,
in order to reason with methods closer to human reasoning. Unfortunately, CR cannot
be represented by a set of rules or algorithms in order to be implemented in an AI
system. For this reason, we need to find more sophisticated methods to represent CR
in an AI system. One idea is to utilize cognitive methods that humans use when they
perform CR. For instance, the use of arguments oftenly reveals why and how a human
individual uses her CR to support or reject an opinion. Moreover, it is interesting to
investigate the semantic relations between entities that allow humans to answer complex
questions with CR. For example, why the answer coffee is the most common answer to
the question “Name an entity that is related to sugar, spoon, milk, and mug” [9].
?
This project has received funding from the Hellenic Foundation for Research and Innovation
(HFRI) and the General Secretariat for Research and Technology (GSRT), under grant agree-
ment No 188.
Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons Li-
cense Attribution 4.0 International (CC BY 4.0).
The problem we are addressing is to define Argumentation Frameworks (AFs) that
can represent commonsense knowledge, in order to enhance the commonsense reason-
ing capabilities of a cognitive robotic system when arguing with a human in a negotia-
tion dialogue. The cognitive robotic system acts in a household environment. Moreover,
we embed knowledge from Semantic Web (SW) knowledge repositories such as Con-
ceptNet [15], WordNet [16], and DBpedia [4] to extend the commonsense knowledge
that exists in the knowledge graph of the cognitive robotic system. For the latter, we
develop methods that iron out the noise that can be found in crowd-source knowledge
graphs.
The contribution of this thesis until now, as this is the second year of the Ph.D.,
can be summarized as follows: At a theoretical level (a1) Two surveys were published,
one for cognitive robotics and how knowledge representation can help computer vision
[9], and one for argumentation and how it can achieve explainability [19], and (a2)
Argumentation frameworks that can represent commonsense arguments were developed
[21,17]. At a methodological level (b1) Mechanisms that find semantic similarity using
commonsense knowledge from SW knowledge graphs were developed [18,22], and
(b2) A framework which can be implemented in a cognitive robotic system that acts in
a household environment and can use CR through argumentation [20].
The reminder of this paper is organized as follows. Section 2, gives the related
work. In Section 3, we define the problem that we solve and we present our results so
far, discussing also some ideas for future work. We conclude our paper with Section 4,
that contains a discussion and some open questions in the area that we are researching.
2 Related Work
This thesis combines different research areas, such as argumentation, knowledge repre-
sentation for cognitive robotics, and knowledge retrieval from SW knowledge resources
such as ConceptNet [15], WordNet [16], and DBpedia [4] in order to achieve common-
sense reasoning. For this reason, we analyze each section separately with respect to
related work.
Argumentation: Our intention with argumentation was to define an AF that could
achieve commonsense reasoning, in order to be used in an argumentation dialogue.
Initially, we defined commonsense arguments as exceptions to regular arguments [21],
as any other study which claimed to represent commonsense arguments did not define
what a commonsense argument is and used preference rules [6], Event Calculus [1],
enthymemes [13], and web resources [11]. But soon we realized that this formalization
was not enough, and we defined an AF which relates arguments with a domain and
a scope, (i.e., a set of entities upon which it can be applied and a set of entities that
can be accepted), as a more appropriate way to represent commonsense knowledge.
The only paper we found in this area was of Búdan et. al [5], where the authors relate
arguments with topics. Búdan et. al consider that topics and arguments are semantically
interrelated and the acceptability of an argument depends on the semantic proximity of
the arguments that defend it.
Knowledge Representation: In this part of our research we develop methods
on how the ontology scheme can help facilitate commonsense knowledge, in order
to achieve CR. For instance, how a household object can be related with its charac-
teristics or with an action, in order to be easily accessible by the knowledge retrieval
mechanism of the cognitive robotic system. For this part we were inspired by studies
such as KnowRob [2] and RoboCSE [7], among others, where the authors construct a
knowledge representation for a cognitive robotic system that acts in a household en-
vironment, which can facilitate commonsense knowledge for performing human tasks
and recognize relations between objects in the environment. Moreover, we created a
mechanism that evaluates the semantic similarity between a household object and an
action, and if they are adequately related then this commonsense knowledge is added to
the knowledge graph. Similar methods are presented in [7,25].
Knowledge Retrieval from Semantic Web: The knowledge retrieval from SW
knowledge repositories, part of our research, is two-fold. Firstly, we develop methods
that find semantic similarity between two entities. Secondly, we develop sophisticated
algorithms to iron out the noise of a knowledge graph. Both are constructed for enrich-
ing the commonsense knowledge of the knowledge graph of a cognitive robotic system.
The algorithms that we have developed were mostly based on [23,24,10], among oth-
ers. The difference is that these studies extract knowledge only from a single knowledge
repository, and do not implement a sophisticated method to exploit the semantic infor-
mation in the repository.
3 Commonsense Reasoning for Cognitive Robotics
The goal of this thesis is to develop AFs that can represent commonsense knowledge, in
order to help a cognitive robotic system to use arguments with commonsense knowledge
in an argumentation dialogue. For this reason, we present our theoretical research and
the AFs that we have developed (sub Section 3.1), we describe the mechanisms for
representing and evaluating commonsense knowledge (sub Section 3.2), and we show
an open-ended knowledge retrieval framework that acts in a household environment and
can use commonsense arguments (sub Section 3.3).
3.1 Overview of frameworks and methodologies
Our theoretical approach was two-fold as we had to understand how the knowledge
representation can be developed in order to contain commonsense knowledge [9], and
how argumentation can help with CR [19]. For the former, we found that the descrip-
tive capability of languages such as Turtle and OWL, can be enough for constructing
a knowledge representation that will contain commonsense knowledge. The key is to
represent the knowledge with an easily understandable architecture for the classes, and
an intuitive understanding of properties. For the latter, we quickly understood that argu-
mentation is a tool that can offer great explainability to any AI system. But even though
Abstract AFs (AAFs) seem more intuitive in understanding they tend to lose explain-
ability compared to their structured counterparts [3]. For this reason, we consider that
new frameworks should be developed that connect these two areas.
The construction of an AF that can represent commonsense arguments [21,17], was
the result of an extensive literature review over argumentation. We managed to find
that even though AAF are more intuitive for understanding they tend to lose descriptive
capabilities, in contrast to Structured AF. For example, AAFs cannot easily explain
what are the facts that support a claim. Therefore, we created an AF that closes the gap
between an AAF and a Structured counterpart, by relating each argument from a AAF
with a domain and a scope of application. More specifically, we relate the arguments
with a set of entities upon which it can be applied and set of entities that can be accepted.
For example, the argument a = “All apples are red” has a domain of application over
all apples in our universe, but if we have the second argument b = “Granny Smiths
are green apples” and consider that these two are the only two type of apples in our
universe, then the argument b restricts the scope of a to all apples in the universe except
Granny Smiths.
Future Work: We intend to analyze the complexity of the algorithms that solve
the verification, credulously acceptance, and skeptically acceptance for the extensions
of [17], and define dialogue protocols, in order to implement the framework naturally
in dialogues. Moreover, we plan to define an AAF that contains different type of attacks
that enables the types of attacks to be part of the argument exchange process.
3.2 Representation of Commonsense Knowledge
In order to represent commonsense knowledge for a household environment we con-
structed an OWL ontology with information from VirtualHome [12,14], and a query
answering mechanism on top of the ontology [18]. The user can choose between a set
of predefined queries, where she just needs to give a keyword for a SPARQL query
to be generated, or can address her own SPARQL query. The predefined queries can
be seen in Table 1; we choose these queries as they are the most commonly addressed
queries to a cognitive robotic system that acts in a household environment [9]. At this
stage, these queries can help elderly people who have the need of a (robotic) assistant
in their household. Notice that we plan to extend the set of predefined queries, as this
was an initial batch for which we could find datasets in order to represent the queries
and perform our evaluation. Moreover, the framework has a semantic matching mech-
anism that relates entities from the knowledge base of the framework with entities that
do not exist in it, using information from ConceptNet [15], WordNet [16], and DBpe-
dia [4]. Therefore, expanding the range of queries that the framework can answer. The
framework was mostly constructed using Python and OWL.
Table 1: Table with the query categories.
Query Input Output
Q1 On what objects can I perform the actions X1,..,Xn if I am in room Y? actions X1,..,Xn & condition Y objects O1,...,Ot
Q2 On what objects can I perform the actions X1,..,Xn? actions X1,..,Xn objects O1,...,Ot
Q3 What can I do with objects O1,...,Om? objects O1,...,Om actions X1,..,Xl
Q4 What objects are related to objects O1,...,Om? objects O1,...,Om objects O1,...,Ot
Q5 Give me the category of activities for A activity A activities A1,...,An
Q6 Give me related objects to O1,...,Om objects O1,...,Om objects O1,...,Ot
Q7 Give me similar action(s) to X action X actions X1,..,Xl
Q8 Recommend an Activity based on the description A description A activity A
We also created a relation evaluation mechanism between real life objects and ac-
tions [22]. Basically the mechanism can answer the question “Can the action X be
performed on/by the object Y?”. If the answer to the question is positive, the mecha-
nism will insert this commonsense knowledge (i.e., that the action can be performed
on/by the object) in the knowledge graph of the cognitive robotic system. The mecha-
nism given an object and an action label, creates two subgraphs with information from
ConceptNet, and based on a semantics-based metric evaluates if the subgraphs are ad-
equately related. The semantics-based metric takes into consideration the topology of
the subgraph and the relations in it. The threshold that the metric uses is the result of
training over positive and negative relations (i.e., related and not related).
Positive and negative relations were collected from Something Something Dataset3 .
Something-Something consists of a large collection of short video clips containing ac-
tions performed on and with household objects. The actions involve either one type of
object (e.g., open a bottle) or two distinct types of objects (e.g., put coins in a box).
Those pairs that existed in the description of at least one video were automatically char-
acterized as positive pairs. The negative relations were manually annotated.
Future Work: We plan to evaluate further the mechanism from [22], over more
graphs and other types of relations.
3.3 Commonsense Reasoning Mechanism
The framework that we constructed and can use commonsense reasoning through argu-
mentation, is an extension of [18], after we have considered [21]. The architecture of
the framework can be seen in Figure 1 and Figure 2.
Fig. 1: Architecture of the Knowledge Retrieval Component
For Figure 1 each step in the workflow is annotated with a number in a circle that
indicates the order in the workflow path. Blue coloured circles indicate optional steps.
3
https://20bn.com/datasets/something-something
For Figure 2 each step in the workflow is annotated with a number in a circle that
indicates the order in the argumentation dialogue. Notice that there are alternative paths
in the dialogue.
Fig. 2: Architecture of the Learning through Argumentation Component
As mentioned, the framework uses the query mechanism over the household on-
tology from [18]. Additionally, a user can argue with the framework over the returned
answers, with two different scenarios. Missing where the user considers that there is
an entity missing from the answers of the framework. Wrong where the user considers
that there is an entity that should not exist in the answers of the system. According to
what keywords the user gave in her initial question the framework will access the in-
formation in its knowledge base, with SPARQL queries, or use the semantic matching
algorithm (sub Section 3.2), in order to create commonsense arguments, and answer to
the user why something is missing or wrong. If the user is still not convinced then the
framework will accept the user recommendation if she can back-up her opinion with in-
formation from an external trustworthy knowledge base. An external knowledge base is
considered trustworthy by the framework according to the trust score that it has. Also,
the trust score is not fixed as it can be reduced or increased according to who won the
argumentation dialogue (i.e., the user or the framework). The framework is mostly built
using Python and OWL.
Future Work: As for future work we plan to embed in the framework the object
action relation mechanism [22] and construct a dataset about object characteristics.
4 Open Questions and Discussion
In this section we pose some open questions that we face in our research, and any
answer to them would benefit us on how to continue our research.
Q1: The ontology about households we have built is constructed upon a dataset. But no
matter how big the dataset is, it remains restricted to a specific set of labels, so what
would be an appropriate solution in order to be able to extend the knowledge? Are
our methods for extracting knowledge from the SW enough?
Q2: What is the appropriate method to find a threshold based on training for the semantics-
based metric?
Q3: Should argumentation be combined with other reasoning techniques, to perform
more accurately commonsense reasoning?
Q4: The AF that we defined seems to capture a good representation of commonsense
knowledge, but should we stick to a theoretical approach in order to make it more
descriptive before we move to an implementation, or should we proceed with an
implementation?
Q5: The Structured AF [3] or the AAF [8], capture more accurately CR?
Q6: Is a “clever” open-ended knowledge representation enough to represent common-
sense knowledge for commonsense reasoning?
As mentioned this is the second year of the Ph.D. thesis. Therefore, it is natural that
some results have already been delivered. Nevertheless, it should be an open debate on
if these methods seem rational, and if so how could they be extended. For this reason,
we pose these open questions that argue with core ideas in our research.
References
1. Almpani, S., Stefaneas, P.S.: On proving and argumentation. In: AIC. pp. 72–84 (2017)
2. Beetz, M., Beßler, D., Haidu, A., Pomarlan, M., Bozcuoğlu, A.K., Bartels, G.: Know
rob 2.0—a 2nd generation knowledge processing framework for cognition-enabled robotic
agents. In: 2018 IEEE International Conference on Robotics and Automation (ICRA). pp.
512–519. IEEE (2018)
3. Besnard, P., Garcia, A., Hunter, A., Modgil, S., Prakken, H., Simari, G., Toni, F.: In-
troduction to structured argumentation. Argument & Computation 5(1), 1–4 (2014).
https://doi.org/10.1080/19462166.2013.869764
4. Bizer, C., Lehmann, J., Kobilarov, G., Auer, S., Becker, C., Cyganiak, R., Hellmann, S.:
Dbpedia-a crystallization point for the web of data. Web Semantics: science, services and
agents on the world wide web 7(3), 154–165 (2009)
5. Budán, M.C., Cobo, M.L., Martinez, D.C., Simari, G.R.: Proximity semantics for topic-based
abstract argumentation. Information Sciences 508, 135–153 (2020)
6. Čyras, K.: Argumentation-based reasoning with preferences. In: International Conference on
Practical Applications of Agents and Multi-Agent Systems. pp. 199–210. Springer (2016)
7. Daruna, A., Liu, W., Kira, Z., Chetnova, S.: Robocse: Robot common sense embedding. In:
2019 International Conference on Robotics and Automation (ICRA). pp. 9777–9783. IEEE
(2019)
8. Dung, P.M.: An argumentation-theoretic foundation for logic programming. The Journal of
logic programming 22(2), 151–177 (1995)
9. Gouidis, F., Vassiliades, A., Patkos, T., Argyros, A., Bassiliades, N., Plexousakis, D.: A re-
view on intelligent object perception methods combining knowledge-based reasoning and
machine learning. AAAI-MAKE 2020: Combining Machine Learning and Knowledge En-
gineering (2020)
10. Icarte, R.T., Baier, J.A., Ruz, C., Soto, A.: How a general-purpose commonsense
ontology can improve performance of learning-based image retrieval. arXiv preprint
arXiv:1705.08844 (2017)
11. Kobbe, J., Opitz, J., Becker, M., Hulpus, I., Stuckenschmidt, H., Frank, A.: Exploiting back-
ground knowledge for argumentative relation classification. In: 2nd Conference on Lan-
guage, Data and Knowledge (LDK 2019). Schloss Dagstuhl-Leibniz-Zentrum fuer Infor-
matik (2019)
12. Liao, Y.H., Puig, X., Boben, M., Torralba, A., Fidler, S.: Synthesizing environment-aware
activities via activity sketches. In: Proceedings of the IEEE Conference on Computer Vision
and Pattern Recognition. pp. 6291–6299 (2019)
13. Mailly, J.G.: Using enthymemes to fill the gap between logical argumentation and revision
of abstract argumentation frameworks. arXiv preprint arXiv:1603.08789 (2016)
14. Puig, X., Ra, K., Boben, M., Li, J., Wang, T., Fidler, S., Torralba, A.: Virtualhome: Simulat-
ing household activities via programs. In: Proceedings of the IEEE Conference on Computer
Vision and Pattern Recognition. pp. 8494–8502 (2018)
15. Speer, R., Chin, J., Havasi, C.: Conceptnet 5.5: An open multilingual graph of general knowl-
edge. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 31 (2017)
16. Strapparava, C., Valitutti, A., et al.: Wordnet affect: an affective extension of wordnet. In:
Lrec. vol. 4, p. 40. Citeseer (2004)
17. Vassiliades, A., Patkos, T., Flouris, G., Bikakis, A., Bassiliades, N., Plexousakis,
D.: Abstract argumentation frameworks with domain assignments. 30th Inter-
national Joint Conference on Artificial Intelligence (IJCAI-21), 2021, Mon-
treal, Canada (2021), https://intelligence.csd.auth.gr/publication/conference-papers/
abstract-argumentation-frameworks-with-domain-assignments/
18. Vassiliades, A., Bassiliades, N., Gouidis, F., Patkos, T.: A knowledge retrieval framework
for household objects and actions with external knowledge. In: International Conference on
Semantic Systems. pp. 36–52. Springer, Cham (2020)
19. Vassiliades, A., Bassiliades, N., Patkos, T.: Argumentation and explainable artificial intelli-
gence: a survey. The Knowledge Engineering Review 36 (2021)
20. Vassiliades, A., Bassiliades, N., Patkos, T.: An open-ended web knowledge retrieval frame-
work with explanation and learning through argumentation. Submitted: Semantic Web Jour-
nal (2021)
21. Vassiliades, A., Patkos, T., Bikakis, A., Flouris, G., Bassiliades, N., Plexousakis, D.: Prelim-
inary notions of arguments from commonsense knowledge. In: 11th Hellenic Conference on
Artificial Intelligence. pp. 211–214 (2020)
22. Vassiliades, A., Patkos, T., Efthymiou, V., Bikakis, A., Bassiliades, N., Plexousakis,
D.: Object-action association extraction from knowledge graphs. International Confer-
ence on Semantic Systems Amsterdam (2021), https://intelligence.csd.auth.gr/publication/
conference-papers/object-action-association-extraction-from-knowledge-graphs/
23. Young, J., Basile, V., Kunze, L., Cabrio, E., Hawes, N.: Towards lifelong object learning
by integrating situated robot perception and semantic web mining. In: Proceedings of the
Twenty-second European Conference on Artificial Intelligence. pp. 1458–1466. IOS Press
(2016)
24. Young, J., Basile, V., Suchi, M., Kunze, L., Hawes, N., Vincze, M., Caputo, B.: Making sense
of indoor spaces using semantic web mining and situated robot perception. In: European
Semantic Web Conference. pp. 299–313. Springer (2017)
25. Zhou, Y., Schockaert, S., Shah, J.: Predicting conceptnet path quality using crowdsourced
assessments of naturalness. In: The World Wide Web Conference. pp. 2460–2471 (2019)