Consider a typical task-solving scenario where a robot is performing a task involving tool use. When a robot is operating in a dynamic environment, it can not be assumed that a tool required in the task will always be available. Our research work concerns the development of a knowledge-based computation system to determine a substitute for the unavailable tool. During the development, we identi ed the requirements regarding the knowledge base for our scenario and selected 9 existing knowledge bases for review. In this article, we review existing knowledge bases developed for the service robotics and investigate their suitability for this speci c application. The knowledge bases are reviewed with respect to various criteria corresponding to the categories knowledge acquisition, knowledge representation, and knowledge processing. Our main contribution is to facilitate the selection of a knowledge base according to one's requirements of a target application for service robots involving household-objects.
It is not uncommon to nd a tool needed for a certain task unavailable. However,
humans tend to circumvent such hurdle by improvising the usability of a suitable
existing object in the environment. For a robot who is expected to work alongside
humans in the real word is bound to face such obstacles and an e ective way to
carry on with the task for it in such situations would be to nd a substitute. A
selection of an appropriate substitute requires a knowledge driven non-invasive
deliberation to determine its suitability. Baber in [
For instance, consider a scenario in which a robot has to choose between a plate and a mouse pad as an alternative for a tray. A tray, in general, can be de ned as a rigid, rectangular, at, wooden, brown colored object while a plate can be de ned as a rigid, circular, semi- at, white colored object and a mouse pad as soft, rectangular, at, leather-based object. Bear in mind, however, that some properties are more relevant than the others with respect to the primary purpose of the tool. For a tray whose primary purpose is to carry, rigid and at are more relevant to carry than a material or a color of a tray. Consequently, to nd the most appropriate substitute, the relevant properties of the unavailable tool needs to correspond to as large a degree as possible to the properties of the possible choices for a substitute.
The proposed approach performs a knowledge-driven reasoning to identify the relevant properties of the unavailable tool and determines the most similar substitute on the basis of those properties. Since the computation requires an access to the ontological knowledge about properties of a missing tool and of existing objects in the environment, we set out to explore the existing knowledge bases. The primary objective of this exploration was aimed at determining whether the knowledge about objects from the existing knowledge bases can be exploited in our approach.
The demand for such ontological knowledge about objects has been increasing (see Table 2). Especially, for the developers of the reasoning systems such as tool selection, task planning or an action selection aimed at a service robot who is expected to perform household tasks, an unhindered access to a stack of knowledge about objects or the environment is a primary concern. Since there are many knowledge bases developed for service robots, it can be cumbersome to scrutinize each one of them to examine the usefulness to the intended system. The objective of this review article is to provide an overview of the existing knowledge bases which can facilitate the selection of a knowledge base according to one's requirements of a target computation system involving household-objects for service robots. 2
There has been an increasing interest in the knowledge-based systems aimed
at various applications in robotics such as human-robot interaction [12], action
recognition [
As a result, we selected 39 papers covering 9 knowledge bases for evaluation 1. The involved knowledge bases are summarized in Table 1 along with their acronyms by which they are identi ed. The plot in Figure 1 illustrates the life span of each knowledge base. d) Final Literature Selection: In the last step we revised the nal list and extracted the most important papers according to: 1. Content - we looked for papers providing detailed descriptions of con gurations, content, performance, interfaces, etc. of the knowledge base. This information is necessary to assess the knowledge bases with respect to different criteria. 2. Impact - we examined the impact of each paper on the basis of the number of citations the selected papers have received and how those numbers have evolved over the years as illustrated in 'Impact of the paper' column of table 2. In terms of the number of citations, KNOWROB is so far the most in uential knowledge base since its inception while individual papers referencing OMICS and OMRKF are continuously cited. 1 This complete list is available at https://essfiles.ivs.cs.ovgu.de/index.php/ s/PgdgM19V6GSQ5IW
For KNOWROB, however, we have isolated 4 from over 40 papers. For the comprehensive list of the paper, please visit the web page of KnowRob (see Appendix A) As a result, the original list was ltered and eventually 20 research papers were selected covering the 9 knowledge bases (see Table 2). 3
For reviewing, we focus on three primary components to characterize a knowledge base: how is the knowledge acquired, how is it represented and how is it processed. This section provides an aerial view of the knowledge bases along the same characteristics. For each characteristic: we have selected the following criteria (see Table 3)
The real-world is di erently perceived by a robot than its human counterpart due to the limited perception capabilities of the robot. It is, therefore, necessary to distinguish between what knowledge was acquired from the sensory data or from the robotic perspective and what knowledge was acquired from non-sensory sources such as web pages, manually encoded, or from a human perspective. For the proposed approach, the acquisition of knowledge from a robotic perspective is prudent. Note that a substitute selection is a decision relative to an agent's (human or robot) understanding of an object. We believe that a substitute should be chosen based on a robot's understanding of an object instead of human's understanding since it is a robot that is supposed to manipulate the substitute. Thus, when reviewing the 'knowledge acquisition' of the knowledge bases, we focused on the source of the knowledge and what kind of knowledge was acquired from this source. These two aspects inform what knowledge is from human perspective and what knowledge is from robot perspective. For instance, some knowledge bases have extracted common sense knowledge about objects from WordNet or OpenCyc and geometric data about an object, spatial relation between objects or metric map of an environment are acquired using vision sensors such as a camera or laser scanner.
The knowledge acquired from various resources needs to be accumulated and encoded in a formal language such that it provides meaningful description of the world and can be processed smoothly. The 'representation formalism' criteria examines the di erent formalisms used to represent the knowledge in the knowledge bases (see Table 5). Since the proposed approach deliberates on a possible substitute non-invasively, we are interested in a logic-based representation of the world that will allow a reasoning-based computation.
On one hand our proposed approach requires the understanding of the environment and the objects in it from the robotic perspective; on the other hand, the knowledge about the objects need to be represented in a logic-based formalism. These two requirements can co-exist when knowledge is grounded in the sensory data. The grounding of knowledge is popularly known as symbol grounding or symbol anchoring. For a developer who wishes to use knowledge representation and reasoning techniques for a robotic application, it is recommended that knowledge is grounded into the robot's reality of the world. Therefore, the selected knowledge bases are reviewed to examine what knowledge in the knowledge base is grounded (see Table 6).
Understanding the environment or objects in it from a robot perspective has its own share of di culties. For instance, the knowledge that is acquired from the sensors carries a baggage of uncertainty. The uncertainty can be due to the Knowledge Base KNOWROB MLN-KB NMKB OMICS OMRKF ORO OUR-K PEIS RoboBrain
Validation by Users Mechanism
Knowledge Content Validation by Users Probabilistic Model Noisy sensor information Statistical Relational Mod- Relations between objects, types of objects els Median-based Principle of Speci city Validation by Users Bayesian Inference
Noise in the web data Incomplete Knowledge Uncertainty not considered Uncertainty not separately modeled Unknown objects and its properties Unknown objects, action selection, context recognition Disambiguate multiple groundings of a symbol Inconsistencies due to knowledge coming from di erent resources, Disambiguate due to the same word having di erent meaning noisy data or partial observability and can manifest into various forms such as incompleteness, inconsistency, ambiguities that can a ect the overall quality of the knowledge. One way to deal with the uncertainty is, for instance, by representing the uncertain knowledge probabilistically. In order to do that, one needs to identify what knowledge is uncertain. The 'modeling of uncertainty' criteria focuses on these two issues: what kind of knowledge is modeled for uncertainty and what mechanism is used for modeling (see Table 7).
Not all knowledge can be perceived using the sensory sources, for instance, typical topological relations between objects and places such as cups are usually in the kitchen or the similarity relations between objects which can not perceived visually in its entirety. The possible source for such type of knowledge would be by reasoning about the existing knowledge and drawing inferences from it or by querying the knowledge base. The knowledge processing criteria looks into both the aspects of processing: what knowledge in the knowledge base is inferred or can be queried and what inference or query mechanism is used to achieve that (see Table 8).
So far, we have discussed the characteristics of the knowledge bases with respect to knowledge acquisition, representation and processing. For a knowledge base to be useful, size of the knowledge base is a critical piece of information. The size of the knowledge base can be measured in terms of quantities in which di erent kind of knowledge is available, for instance, number of objects, properties, relations etc. In table 9, we have provided the information on the size of each knowledge base as reported in the respective literature.
Accessibility is the quality of being easily available to use. The knowledge bases should be developed such that they can be used by the developers around Around 8000 classes that describe events, actions, objects, mathematical concepts and so on 40 objects comprise 100 images and on average 4.25 a ordance for each objects Not available As of 2004, 400 users with 26,000 accepted submissions, 400 images of indoor objects (current number of images unknown) comprising a total of 100000 entries in the form of objects, actions, senses.
Knowledge about approximately 300 objects as per 2005 56 object classes and 60 predicates that states relation with objects Knowledge about approximately 300 objects as per 2005 15 objects that comprise 2 to 5 images for each object 44347 concepts and 98465 relations Knowledge Download? Base KnowRob the world in various applications.The knowledge base accessibility criteria examines the ways in which each knowledge base is made accessible to the developers.In the accessibility, we have examined, if the knowledge bases are available to download or install, if there are tutorials or any other documentation available to get the user started and if there is information on API available. Additionally, we also check what kind of licensing is made available. The Table 10 summarizes the accessibility of each knowledge base. Since for OMICS, OMRKF, OUR-K and PEIS, we were not able to nd the required information, we have indicated NA (Not Applicable) in the table. Additionally, we have provided the available web pages for the knowledge bases in appendix A. In this article, we have investigated the existing knowledge base approaches for service robot applications and evaluated their capabilities related to a speci c research project. For this purpose we searched for knowledge bases that are developed for household robots and contain knowledge about household objects and we identi ed 9 existing knowledge bases. In addition to the life span of each knowledge base, the information concerning which knowledge bases have made impact in the community was measured in the form of a number of citations the literature related to each knowledge base has received. The knowledge bases were reviewed with respect to the amount of knowledge it holds. Each knowledge base was further examined with respect to the following criteria: acquisition of knowledge (what knowledge is acquired and what is the source), representation formalism, symbol grounding, modeling of uncertainty (what knowledge is modeled and what mechanism is used) and lastly, inference mechanism (what knowledge was inferred or queried and what mechanism was used?). We concluded our review by evaluating the accessibility of each knowledge base to the users.
For our approach, we are interested in the knowledge base that has ontological knowledge about household objects, especially knowledge about the properties and uses of the objects. It is also prudent that the required knowledge is grounded and the uncertainty caused by the partial observability of the environment due to the noisy sensor is modeled using probability or fuzzy logic. Based on our investigation, we have concluded that the knowledge bases KnowRob and MLNKB seems suitable to our purpose. In the future work, we will conduct the experiments where the knowledge extracted from them will be used to evaluate the performance of our approach. In: Fuzzy Systems and Knowledge Discovery. FSKD 2006. Lecture Notes in Computer Science. vol. 4223, pp. 596{606. Springer Berlin Heidelberg (2006). https://doi.org/10.1007/11881599 10. Lallee, S., Lemaignan, S., Lenz, A., Melhuish, C., Natale, L., Skachek, S., Van Der Zant, T., Warneken, F., Dominey, P.F.: Towards a platformindependent cooperative human-robot interaction system: I. Perception. IEEE/RSJ 2010 International Conference on Intelligent Robots and Systems, IROS 2010 - Conference Proceedings (October 2010), 4444{4451 (2010). https://doi.org/10.1109/IROS.2010.5652697 11. Lemaignan, S., Ros, R., Mosenlechner, L., Alami, R., Beetz, M.: ORO, a knowledge management platform for cognitive architectures in robotics. IEEE/RSJ 2010 International Conference on Intelligent Robots and Systems, IROS 2010 - Conference Proceedings (April), 3548{3553 (2010). https://doi.org/10.1109/IROS.2010.5649547 12. Lemaignan, S., Warnier, M., Sisbot, E.A., Clodic, A., Alami, R.: Arti cial cognition for social humanrobot interaction: An implementation. Arti cial Intelligence 247, 45{69 (2017). https://doi.org/10.1016/j.artint.2016.07.002, http://dx.doi.org/ 10.1016/j.artint.2016.07.002 13. Lim, G.H., Suh, I.H., Suh, H.: Ontology-based uni ed robot knowledge for service robots in indoor environments. IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans 41(3), 492{509 (2011). https://doi.org/10.1109/TSMCA.2010.2076404 14. Pineda, L.A., Meza, I.V., Aviles, H., Gershenson, C., Rascon, C., Alvarado, M., Salinas, L.: IOCA: An Interaction-Oriented Cognitive Architecture. Journal Research in Computing Science 54, 273{284 (2010) 15. Pineda, L.A., Rodr guez, A., Fuentes, G., Rascon, C., Meza, I.: A light nonmonotonic knowledge-base for service robots. Intelligent Service Robotics 10(3), 159{171 (2017). https://doi.org/10.1007/s11370-017-0216-y 16. Ros, R., Lemaignan, S., Sisbot, E.A., Alami, R., Steinwender, J., Hamann, K., Warneken, F.: Which one? Grounding the referent based on efcient human-robot interaction. Proceedings - IEEE International Workshop on Robot and Human Interactive Communication pp. 570{575 (2010). https://doi.org/10.1109/ROMAN.2010.5598719 17. Saxena, A., Jain, A., Sener, O., Jami, A., Misra, D.K., Koppula, H.S.: RoboBrain:
Large-Scale Knowledge Engine for Robots. arXiv pp. 1 { 11 (2014) 18. Suh, I.H., Lim, G.H., Hwang, W., Suh, H., Choi, J.H., Park, Y.T.: Ontology-based multi-layered robot knowledge framework (OMRKF) for robot intelligence. IEEE International Conference on Intelligent Robots and Systems (October), 429{436 (2007). https://doi.org/10.1109/IROS.2007.4399082 19. Tenorth, M., Beetz, M.: KNOWROB- Knowledge Processing for Autonomous Personal Robots. In: IEEE/RSJ International Conference on Intelligent Robots and Systems. pp. 4261{4266 (2009) 20. Tenorth, M., Beetz, M.: KnowRob: A knowledge processing infrastructure for cognition-enabled robots. International Journal of Robotics Research (2013). https://doi.org/10.1177/0278364913481635 21. Tenorth, M., Beetz, M.: Representations for robot knowledge in the KNOWROB framework. Arti cial Intelligence 247, 151{169 (2015). https://doi.org/10.1016/j.artint.2015.05.010, http://dx.doi.org/10.1016/j. artint.2015.05.010 A
Appendix 1. http://www.aass.oru.se/Research/Robots/projects.html (PEIS) 2. https://www.openrobots.org/wiki/oro-server (ORO) 3. https://web.stanford.edu/~yukez/eccv2014.html (MLNKB) 4. http://robobrain.me/about.html (Robobrain) 5. https://github.com/RoboBrainCode (Robobrain) 6. https://tinyurl.com/y9uboh62 (NMKB)