Commonsense Reasoning with Argumentation for Cognitive Robotics?

Abstract

Commonsense Reasoning is a cognitive ability which is found only in humans, but the desire is to implement it in Artificial Intelligence when solving tasks. Moreover, the use of arguments can reveal how and why a human individual supports or rejects an opinion. In this thesis, our goal is to study theoretically the problem of commonsense reasoning and develop methods for enhancing the commonsense reasoning capabilities of a cognitive robotic system that acts in a household environment. The commonsense reasoning mechanism is implemented with the use of argumentation, by developing argumentation frameworks that can facilitate commonsense knowledge. Additionally, we use Semantic Web technologies to add commonsense knowledge in the commonsense reasoning mechanism, and we construct a framework that accommodates our methods.

Keywords

• Commonsense Reasoning
• Argumentation
• Semantic Web
• Cognitive Robotics

-

Commonsense Reasoning (CR) is a cognitive characteristic that is found only in human individuals, but is a long desired aspect that Artificial Intelligence (AI) should have, in order to reason with methods closer to human reasoning. Unfortunately, CR cannot be represented by a set of rules or algorithms in order to be implemented in an AI system. For this reason, we need to find more sophisticated methods to represent CR in an AI system. One idea is to utilize cognitive methods that humans use when they perform CR. For instance, the use of arguments oftenly reveals why and how a human individual uses her CR to support or reject an opinion. Moreover, it is interesting to investigate the semantic relations between entities that allow humans to answer complex questions with CR. For example, why the answer coffee is the most common answer to the question “Name an entity that is related to sugar, spoon, milk, and mug” [ 9 ].

The problem we are addressing is to define Argumentation Frameworks (AFs) that can represent commonsense knowledge, in order to enhance the commonsense reasoning capabilities of a cognitive robotic system when arguing with a human in a negotiation dialogue. The cognitive robotic system acts in a household environment. Moreover, we embed knowledge from Semantic Web (SW) knowledge repositories such as ConceptNet [ 15 ], WordNet [ 16 ], and DBpedia [ 4 ] to extend the commonsense knowledge that exists in the knowledge graph of the cognitive robotic system. For the latter, we develop methods that iron out the noise that can be found in crowd-source knowledge graphs.

The contribution of this thesis until now, as this is the second year of the Ph.D., can be summarized as follows: At a theoretical level (a1) Two surveys were published, one for cognitive robotics and how knowledge representation can help computer vision [ 9 ], and one for argumentation and how it can achieve explainability [ 19 ], and (a2) Argumentation frameworks that can represent commonsense arguments were developed [ 21,17 ]. At a methodological level (b1) Mechanisms that find semantic similarity using commonsense knowledge from SW knowledge graphs were developed [ 18,22 ], and (b2) A framework which can be implemented in a cognitive robotic system that acts in a household environment and can use CR through argumentation [ 20 ].

The reminder of this paper is organized as follows. Section 2, gives the related work. In Section 3, we define the problem that we solve and we present our results so far, discussing also some ideas for future work. We conclude our paper with Section 4, that contains a discussion and some open questions in the area that we are researching. 2

Related Work

This thesis combines different research areas, such as argumentation, knowledge representation for cognitive robotics, and knowledge retrieval from SW knowledge resources such as ConceptNet [ 15 ], WordNet [ 16 ], and DBpedia [ 4 ] in order to achieve commonsense reasoning. For this reason, we analyze each section separately with respect to related work.

Argumentation: Our intention with argumentation was to define an AF that could achieve commonsense reasoning, in order to be used in an argumentation dialogue. Initially, we defined commonsense arguments as exceptions to regular arguments [ 21 ], as any other study which claimed to represent commonsense arguments did not define what a commonsense argument is and used preference rules [ 6 ], Event Calculus [ 1 ], enthymemes [ 13 ], and web resources [ 11 ]. But soon we realized that this formalization was not enough, and we defined an AF which relates arguments with a domain and a scope, (i.e., a set of entities upon which it can be applied and a set of entities that can be accepted), as a more appropriate way to represent commonsense knowledge. The only paper we found in this area was of Bu´dan et. al [ 5 ], where the authors relate arguments with topics. Bu´dan et. al consider that topics and arguments are semantically interrelated and the acceptability of an argument depends on the semantic proximity of the arguments that defend it.

Knowledge Representation: In this part of our research we develop methods on how the ontology scheme can help facilitate commonsense knowledge, in order to achieve CR. For instance, how a household object can be related with its characteristics or with an action, in order to be easily accessible by the knowledge retrieval mechanism of the cognitive robotic system. For this part we were inspired by studies such as KnowRob [ 2 ] and RoboCSE [ 7 ], among others, where the authors construct a knowledge representation for a cognitive robotic system that acts in a household environment, which can facilitate commonsense knowledge for performing human tasks and recognize relations between objects in the environment. Moreover, we created a mechanism that evaluates the semantic similarity between a household object and an action, and if they are adequately related then this commonsense knowledge is added to the knowledge graph. Similar methods are presented in [ 7,25 ].

Knowledge Retrieval from Semantic Web: The knowledge retrieval from SW

knowledge repositories, part of our research, is two-fold. Firstly, we develop methods that find semantic similarity between two entities. Secondly, we develop sophisticated algorithms to iron out the noise of a knowledge graph. Both are constructed for enriching the commonsense knowledge of the knowledge graph of a cognitive robotic system. The algorithms that we have developed were mostly based on [ 23,24,10 ], among others. The difference is that these studies extract knowledge only from a single knowledge repository, and do not implement a sophisticated method to exploit the semantic information in the repository. 3

Commonsense Reasoning for Cognitive Robotics

The goal of this thesis is to develop AFs that can represent commonsense knowledge, in order to help a cognitive robotic system to use arguments with commonsense knowledge in an argumentation dialogue. For this reason, we present our theoretical research and the AFs that we have developed (sub Section 3.1), we describe the mechanisms for representing and evaluating commonsense knowledge (sub Section 3.2), and we show an open-ended knowledge retrieval framework that acts in a household environment and can use commonsense arguments (sub Section 3.3). 3.1

Overview of frameworks and methodologies

Our theoretical approach was two-fold as we had to understand how the knowledge representation can be developed in order to contain commonsense knowledge [ 9 ], and how argumentation can help with CR [ 19 ]. For the former, we found that the descriptive capability of languages such as Turtle and OWL, can be enough for constructing a knowledge representation that will contain commonsense knowledge. The key is to represent the knowledge with an easily understandable architecture for the classes, and an intuitive understanding of properties. For the latter, we quickly understood that argumentation is a tool that can offer great explainability to any AI system. But even though Abstract AFs (AAFs) seem more intuitive in understanding they tend to lose explainability compared to their structured counterparts [ 3 ]. For this reason, we consider that new frameworks should be developed that connect these two areas.

The construction of an AF that can represent commonsense arguments [ 21,17 ], was the result of an extensive literature review over argumentation. We managed to find that even though AAF are more intuitive for understanding they tend to lose descriptive capabilities, in contrast to Structured AF. For example, AAFs cannot easily explain what are the facts that support a claim. Therefore, we created an AF that closes the gap between an AAF and a Structured counterpart, by relating each argument from a AAF with a domain and a scope of application. More specifically, we relate the arguments with a set of entities upon which it can be applied and set of entities that can be accepted. For example, the argument a = “All apples are red” has a domain of application over all apples in our universe, but if we have the second argument b = “Granny Smiths are green apples” and consider that these two are the only two type of apples in our universe, then the argument b restricts the scope of a to all apples in the universe except Granny Smiths.

Future Work: We intend to analyze the complexity of the algorithms that solve the verification, credulously acceptance, and skeptically acceptance for the extensions of [ 17 ], and define dialogue protocols, in order to implement the framework naturally in dialogues. Moreover, we plan to define an AAF that contains different type of attacks that enables the types of attacks to be part of the argument exchange process. 3.2

Representation of Commonsense Knowledge

In order to represent commonsense knowledge for a household environment we constructed an OWL ontology with information from VirtualHome [ 12,14 ], and a query answering mechanism on top of the ontology [ 18 ]. The user can choose between a set of predefined queries, where she just needs to give a keyword for a SPARQL query to be generated, or can address her own SPARQL query. The predefined queries can be seen in Table 1; we choose these queries as they are the most commonly addressed queries to a cognitive robotic system that acts in a household environment [ 9 ]. At this stage, these queries can help elderly people who have the need of a (robotic) assistant in their household. Notice that we plan to extend the set of predefined queries, as this was an initial batch for which we could find datasets in order to represent the queries and perform our evaluation. Moreover, the framework has a semantic matching mechanism that relates entities from the knowledge base of the framework with entities that do not exist in it, using information from ConceptNet [ 15 ], WordNet [ 16 ], and DBpedia [ 4 ]. Therefore, expanding the range of queries that the framework can answer. The framework was mostly constructed using Python and OWL.

We also created a relation evaluation mechanism between real life objects and actions [ 22 ]. Basically the mechanism can answer the question “Can the action X be performed on/by the object Y?”. If the answer to the question is positive, the mechanism will insert this commonsense knowledge (i.e., that the action can be performed on/by the object) in the knowledge graph of the cognitive robotic system. The mechanism given an object and an action label, creates two subgraphs with information from ConceptNet, and based on a semantics-based metric evaluates if the subgraphs are adequately related. The semantics-based metric takes into consideration the topology of the subgraph and the relations in it. The threshold that the metric uses is the result of training over positive and negative relations (i.e., related and not related).

Positive and negative relations were collected from Something Something Dataset3. Something-Something consists of a large collection of short video clips containing actions performed on and with household objects. The actions involve either one type of object (e.g., open a bottle) or two distinct types of objects (e.g., put coins in a box). Those pairs that existed in the description of at least one video were automatically characterized as positive pairs. The negative relations were manually annotated.

Future Work: We plan to evaluate further the mechanism from [ 22 ], over more graphs and other types of relations. 3.3

Commonsense Reasoning Mechanism

The framework that we constructed and can use commonsense reasoning through argumentation, is an extension of [ 18 ], after we have considered [ 21 ]. The architecture of the framework can be seen in Figure 1 and Figure 2.

For Figure 1 each step in the workflow is annotated with a number in a circle that indicates the order in the workflow path. Blue coloured circles indicate optional steps. 3 https://20bn.com/datasets/something-something For Figure 2 each step in the workflow is annotated with a number in a circle that indicates the order in the argumentation dialogue. Notice that there are alternative paths in the dialogue. As mentioned, the framework uses the query mechanism over the household ontology from [ 18 ]. Additionally, a user can argue with the framework over the returned answers, with two different scenarios. Missing where the user considers that there is an entity missing from the answers of the framework. Wrong where the user considers that there is an entity that should not exist in the answers of the system. According to what keywords the user gave in her initial question the framework will access the information in its knowledge base, with SPARQL queries, or use the semantic matching algorithm (sub Section 3.2), in order to create commonsense arguments, and answer to the user why something is missing or wrong. If the user is still not convinced then the framework will accept the user recommendation if she can back-up her opinion with information from an external trustworthy knowledge base. An external knowledge base is considered trustworthy by the framework according to the trust score that it has. Also, the trust score is not fixed as it can be reduced or increased according to who won the argumentation dialogue (i.e., the user or the framework). The framework is mostly built using Python and OWL.

Future Work: As for future work we plan to embed in the framework the object action relation mechanism [ 22 ] and construct a dataset about object characteristics. 4

Open Questions and Discussion

In this section we pose some open questions that we face in our research, and any answer to them would benefit us on how to continue our research.

Q1: The ontology about households we have built is constructed upon a dataset. But no matter how big the dataset is, it remains restricted to a specific set of labels, so what would be an appropriate solution in order to be able to extend the knowledge? Are our methods for extracting knowledge from the SW enough? Q2: What is the appropriate method to find a threshold based on training for the semanticsbased metric? Q3: Should argumentation be combined with other reasoning techniques, to perform more accurately commonsense reasoning? Q4: The AF that we defined seems to capture a good representation of commonsense knowledge, but should we stick to a theoretical approach in order to make it more descriptive before we move to an implementation, or should we proceed with an implementation? Q5: The Structured AF [ 3 ] or the AAF [ 8 ], capture more accurately CR? Q6: Is a “clever” open-ended knowledge representation enough to represent commonsense knowledge for commonsense reasoning?

As mentioned this is the second year of the Ph.D. thesis. Therefore, it is natural that some results have already been delivered. Nevertheless, it should be an open debate on if these methods seem rational, and if so how could they be extended. For this reason, we pose these open questions that argue with core ideas in our research.

References

1. Almpani , S. , Stefaneas , P.S.: On proving and argumentation . In: AIC . pp. 72 - 84 ( 2017 )

2. Beetz , M. , Beßler , D. , Haidu , A. , Pomarlan , M. , Bozcuog˘lu, A.K. , Bartels , G.: Know rob 2.0-a 2nd generation knowledge processing framework for cognition-enabled robotic agents . In: 2018 IEEE International Conference on Robotics and Automation (ICRA) . pp. 512 - 519 . IEEE ( 2018 )

3. Besnard , P. , Garcia , A. , Hunter , A. , Modgil , S. , Prakken , H. , Simari , G. , Toni , F. : Introduction to structured argumentation . Argument & Computation 5 ( 1 ), 1 - 4 ( 2014 ). https://doi.org/10.1080/19462166. 2013 .869764

4. Bizer , C. , Lehmann , J. , Kobilarov , G. , Auer , S. , Becker , C. , Cyganiak , R. , Hellmann , S. : Dbpedia-a crystallization point for the web of data . Web Semantics: science, services and agents on the world wide web 7 ( 3 ), 154 - 165 ( 2009 )

5. Buda´n, M.C. , Cobo , M.L. , Martinez , D.C. , Simari , G.R. : Proximity semantics for topic-based abstract argumentation . Information Sciences 508 , 135 - 153 ( 2020 )

6. Cˇyras , K.: Argumentation-based reasoning with preferences . In: International Conference on Practical Applications of Agents and Multi-Agent Systems . pp. 199 - 210 . Springer ( 2016 )

7. Daruna , A. , Liu , W. , Kira , Z. , Chetnova , S. : Robocse: Robot common sense embedding . In: 2019 International Conference on Robotics and Automation (ICRA) . pp. 9777 - 9783 . IEEE ( 2019 )

8. Dung , P.M.: An argumentation-theoretic foundation for logic programming . The Journal of logic programming 22(2) , 151 - 177 ( 1995 )

9. Gouidis , F. , Vassiliades , A. , Patkos , T. , Argyros , A. , Bassiliades , N. , Plexousakis , D.: A review on intelligent object perception methods combining knowledge-based reasoning and machine learning . AAAI-MAKE 2020: Combining Machine Learning and Knowledge Engineering ( 2020 )

10. Icarte , R.T., Baier , J.A. , Ruz , C. , Soto , A. : How a general-purpose commonsense ontology can improve performance of learning-based image retrieval . arXiv preprint arXiv:1705.08844 ( 2017 )

11. Kobbe , J. , Opitz , J. , Becker , M. , Hulpus , I. , Stuckenschmidt , H. , Frank , A. : Exploiting background knowledge for argumentative relation classification . In: 2nd Conference on Language, Data and Knowledge (LDK 2019 ). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik ( 2019 )

12. Liao , Y.H. , Puig , X. , Boben , M. , Torralba , A. , Fidler , S. : Synthesizing environment-aware activities via activity sketches . In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . pp. 6291 - 6299 ( 2019 )

13. Mailly , J.G. : Using enthymemes to fill the gap between logical argumentation and revision of abstract argumentation frameworks . arXiv preprint arXiv:1603.08789 ( 2016 )

14. Puig , X. , Ra , K. , Boben , M. , Li , J. , Wang , T. , Fidler , S. , Torralba , A. : Virtualhome: Simulating household activities via programs . In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . pp. 8494 - 8502 ( 2018 )

15. Speer , R. , Chin , J. , Havasi , C. : Conceptnet 5.5: An open multilingual graph of general knowledge . In: Proceedings of the AAAI Conference on Artificial Intelligence . vol. 31 ( 2017 )

16. Strapparava , C. , Valitutti , A. , et al.: Wordnet affect: an affective extension of wordnet . In: Lrec . vol. 4 , p. 40 . Citeseer ( 2004 )

17. Vassiliades , A. , Patkos , T. , Flouris , G. , Bikakis , A. , Bassiliades , N. , Plexousakis , D. : Abstract argumentation frameworks with domain assignments . 30th International Joint Conference on Artificial Intelligence (IJCAI-21) , 2021 , Montreal, Canada ( 2021 ), https://intelligence.csd.auth.gr/publication/conference-papers/ abstract-argumentation-frameworks-with-domain-assignments/

18. Vassiliades , A. , Bassiliades , N. , Gouidis , F. , Patkos , T. : A knowledge retrieval framework for household objects and actions with external knowledge . In: International Conference on Semantic Systems . pp. 36 - 52 . Springer, Cham ( 2020 )

19. Vassiliades , A. , Bassiliades , N. , Patkos , T. : Argumentation and explainable artificial intelligence: a survey . The Knowledge Engineering Review 36 ( 2021 )

20. Vassiliades , A. , Bassiliades , N. , Patkos , T. : An open-ended web knowledge retrieval framework with explanation and learning through argumentation . Submitted: Semantic Web Journal ( 2021 )

21. Vassiliades , A. , Patkos , T. , Bikakis , A. , Flouris , G. , Bassiliades , N. , Plexousakis , D. : Preliminary notions of arguments from commonsense knowledge . In: 11th Hellenic Conference on Artificial Intelligence . pp. 211 - 214 ( 2020 )

22. Vassiliades , A. , Patkos , T. , Efthymiou , V. , Bikakis , A. , Bassiliades , N. , Plexousakis , D. : Object-action association extraction from knowledge graphs . International Conference on Semantic Systems Amsterdam ( 2021 ), https://intelligence.csd.auth.gr/publication/ conference-papers/ object-action-association-extraction-from-knowledge-graphs/

23. Young , J. , Basile , V. , Kunze , L. , Cabrio , E. , Hawes , N.: Towards lifelong object learning by integrating situated robot perception and semantic web mining . In: Proceedings of the Twenty-second European Conference on Artificial Intelligence . pp. 1458 - 1466 . IOS Press ( 2016 )

24. Young , J. , Basile , V. , Suchi , M. , Kunze , L. , Hawes , N. , Vincze , M. , Caputo , B. : Making sense of indoor spaces using semantic web mining and situated robot perception . In: European Semantic Web Conference . pp. 299 - 313 . Springer ( 2017 )

25. Zhou , Y. , Schockaert , S. , Shah , J.: Predicting conceptnet path quality using crowdsourced assessments of naturalness . In: The World Wide Web Conference . pp. 2460 - 2471 ( 2019 )