=Paper=
{{Paper
|id=Vol-2728/short6
|storemode=property
|title=Use of Ontology to Model the Perception of the Environment by a Humanoid Robot
|pdfUrl=https://ceur-ws.org/Vol-2728/short6.pdf
|volume=Vol-2728
|authors=Helio Azevedo,José Pedro Ribeiro Belo,Isaque Souza, Roseli Romero
|dblpUrl=https://dblp.org/rec/conf/ontobras/AzevedoBSR20
}}
==Use of Ontology to Model the Perception of the Environment by a Humanoid Robot==
https://ceur-ws.org/Vol-2728/short6.pdf
Use of Ontology to Model the
Perception of the Environment by a Humanoid Robot
Helio Azevedo1 , José Pedro Ribeiro Belo2 , Isaque E. de Souza1 , Roseli A. F. Romero2
1
Center for Information Technology Renato Archer (CTI)
Campinas – SP – Brazil
2
University of São Paulo (USP)
São Carlos – SP – Brazil
{helio.azevedo,isaque.souza}@cti.gov.br
rafrance@icmc.usp.br, josepedroribeirobelo@usp.br
Abstract. Social robotics is a research area that aims to develop models that
allow robots and humans to interact naturally. One of the factors that compro-
mises the evolution of social robotics is the difficulty in integrating cognitive and
robotic systems, mainly due to the volume and complexity of the information ex-
isting in a dynamic environment. In this work, we are proposing an ontology,
named OntPercept, for formalize the communication between the cognitive and
robotic systems. Thanks to this ontology is possible to model the environment
perception using the information captured by the sensors present in the robotic
system. It will be shown that OntPercept simplifies the development, reproduc-
tion and comparison of experiments associated with social robotics.
1. Introduction
Robots are capable of millimeter-precision movements by performing repetitive tasks op-
erating in structured environments where objects are in known and predictable locations.
Thus, it is not surprising that robots are used more in manufacturing operations, such as
painting and welding, instead of operations where, diversity of actions, direct contact with
humans and changes in the environment are part of the system requirements.
Despite its complexity, the demand for the use of robotic agents in environments
other than manufacturing is a necessity of modern society. The World Robotics 2017
report [IFR 2017] indicates a cumulative volume of around 27 billion U.S. dollars in the
service robots sales forecast for the 2018-2020 biennium. This upward trend in the use of
service robots requires a growth in research involving Human-Robot Interaction (HRI),
in particular in the sub-area defined by Fong [Fong et al. 2003] as Socially Interactive
Robots (SIR) or “Social Robotics”, the term used in this article.
This work contributes in this process by developing an environment directed to
social robotics involving the areas of ontology, HRI and cognitive science. For this,
in our previous work, an architecture, named “Cognitive Model Development Environ-
ment (CMDE)” was proposed [Azevedo et al. 2017b] [Azevedo et al. 2017a]. The aim of
CMDE was to accelerate the development of social robotic systems establishing a clear
communication between the cognitive system and the robotic system responsible for gen-
erating the environment perception (in Fig. 1).
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
Figure 1. The CMDE architecture.
An important component of CMDE is the OntPercept ontology, that is responsible
for formalizing the communication of the environment perception data from Cognitive to
Robotic System. Since its first version in 2017 1 , two were the focus of evolution: opti-
mization of objects access (bypassing performance issues of the producer-consumer infor-
mation exchange model used with the triple store) and inclusion of the memory concept
as a resource for robot long term interaction.
The organization of this article follows the steps used to create an ontology. The
hierarchy, concepts and properties of the OntPercept are detailed in Section 2. In Section
3 is presented the test environment. In Section 4, the conclusion of this work is presented.
2. The OntPercept ontology
The ontology presented in this article emphasizes the need to formalize the information
that transits between the processing nodes defined by CMDE architecture (Fig. 1). The
idea is to enable the cognitive system to receive perception information from the environ-
ment with a high level of cognition.
The development of an ontology is accelerated by the use of appropriate modeling
methodologies and tools. This work uses the Ontology Development 101 methodology
[Noy and Mcguinness 2001], which proposes seven steps for the construction of an on-
tology. In Table 1, the methodology steps instantiated in the context of OntPercept are
shown. The Protégé editor [Protégé 2020] was selected to model OntPercept. After mod-
eling, it is possible to identify anomalies using the Pellet reasoner [Stardog-Union 2017]
tool, available as a plugin in the Protégé editor.
Table 1. Steps of the “Ontology Development 101” methodology
Step Comments
1. Domain and Scope Domain: social robotics, Scope: environment perception.
2. Reuse SUMO and IEEE 1872-2015.
3. Concepts Model for perception info.
4. Hierarchy Combines top-down with bottom-up strategies.
5. Properties Directly derived from robotic sensor data.
6. Facets Refines the properties by defining: type, cardinality, and domain.
7. Instances The instances are created by the environment perception.
1
Initially the ontology was called OntSense, the name was changed to ontPercept to distance itself from
the sensor notion and reflect the essence of this ontology: environment perception.
2.1. The OntPercept Development
An important feature of the OntPercept is adherence to the standards of international or-
ganizations with the aim of minimizing effort and providing a better dissemination of the
results. In Fig. 2, the relationship with top-level ontologies is presented as well as the
main concepts of the new ontology. The upper block of this figure presents the top on-
tologies, namely: SUMO [Pease 2019]and IEEE 1872/2015 [IEEE, R&A Society 2015].
namely: SUMO [Pease 2019], which provides a high-level view of existing objects in the
world, and IEEE 1872/2015 [IEEE, R&A Society 2015], which provides specific objects
associated with robotics and automation.
Figure 2. The OntPercept and two top level ontologies: SUMO and IEEE-1872.
In the lower block, the key concepts of OntPercept are presented. The concept
OntPercept:RobotPerceptionEvent represents the super class used as a basis for defining
the environment perception. All other concepts derive or maintain a relationship with this
super class. Next, we will explore the OntPercept:RobotMemory concept. Details of the
other concepts can be obtained at [Azevedo 2018].
OntPercept:RobotMemory concept: Although not directly associated with environment
perception, the OntPercept:RobotMemory2 concept represents the initial step in the im-
plementation of robotic long-term interaction systems. In this work, information retrieval
is associated with the use of retrieval cues3 stored together with the information con-
tent. The idea is to use association mechanisms similar to those used in the memorization
process of humans [McLeod 2018]. To achieve this goal, the memory register is stored
with information from the environment perception. The memory register is modeled by
the SUMO:Object and SUMO:Process classes. The SUMO:Object class roughly corre-
sponds to the class of ordinary objects and the SUMO:Process class models the class of
things that happen and have temporal parts or stages [Pease 2019] (Fig. 3).
2
In this work, the term “memory” represents the structures and processes associated with the storage
and retrieval of information or experiences.
3
A retrieval cue is a hint or clue that can help retrieval.
Figure 3. Relationship of OntPercept:RobotMemory concept.
Table 2. Properties of OntPercept:RobotMemory concept
Property Description Values
hasCaptureTime The instant of memory capture. Capture instant
hasDuration The time during which the event exists. Rational number
hasPerception A set of perception objects. RobotPerceptionEvent
hasContents A Object or Process to be stored. Entity class
In Table 2, it can be found the properties of the OntPercept:RobotMemory concept.
Basically, the core is represented by a relationship with an instance of the SUMO:Entity
class, which is super class of SUMO:Object and SUMO:Process. The other relationships
represent cues that allow the retrieval of the memory register: capture time, event duration
and the sensory environment perception at capture time.
An important memory concept feature is associated with the forgetting process.
In humans the action of forgetting information from the Long Term Memory (LTM) can
be explained using the following theories: interference, retrieval failure and lack of con-
solidation [McLeod 2018]. The Interference Theory states that forgetting occurs because
of interference with other records, either by inhibition caused by an earlier record with
similar characteristics or by overlapping the old record by the new one. The Retrieval
Failure Theory states that a given record cannot be recovered because cues (e.g. smell,
local, emotion and mood) are not present. In the Lack of Consolidation Theory, forgetting
occurs due to biological failures in the record storage.
Unfortunately, robot memory is finite, and judicious use of memory is advisable.
Two theories mentioned above may assist in the memory management process. The Inter-
ference Theory can be used to replace one robot’s experience with another similar, where
the robot has succeeded in its operation. On the other hand, the absence of pointers to the
memory register can be used as a catalyst for garbage collector operations in the records,
similarly to Retrieval Failure Theory. Note that by using ontology to formalize mem-
ory operations, axioms can be created to identify these obsolete records. Subsequently,
reasoning tools4 identify these records and triggers actions for their removal.
2.2. Building the OntPercept ontology
In Fig. 4, the relationships established between OntPercept classes and top-level ontolo-
gies are presented. Classes from SUMO are flagged in green and IEEE 1872-2105 in
4
By reasoning we mean deriving facts that are not explicitly expressed in ontology or knowledge base
[Obitko 2007].
blue. As an example, the properties generateBy, hasCaptureTime and isSenseOf asso-
ciated with the class OntPercept:RobotPerceptionEvent represent relationships with top-
level ontologies. Finally, it is also worth to mention the class SUMO:Object that models
the elements present in the environment.
Figure 4. Example of OntPercept and top-level ontologies relationships.
2.3. Providing interfaces for OntPercept
The final step established by the “Ontology Development 101” methodology involves the
generation, storage and retrieval of object instances. The strategy adopted to enable the
handling of instances is presented in Fig. 5, with the definition of three processing nodes:
Cognition Node, responsible for the implementation of the cognitive system, SPARQL
Server Node, which implements a database with sensory information and Robotic Agent
Node, which directly controls all sensors and actuators of the robotic agent.
Basically, the robotic agent obtains information from the environment and routes it
to the cognitive system using a predefined communication protocol such as REST/HTTP.
The cognitive system (“Cognition Node”) processes the environment perception informa-
tion, makes decisions and sends actions to the robotic agent via socket communication.
In order to reduce end-user effort, it was created two APIs to access the “SPARQL
Server Node”. These APIs provide a simpler interface for using the technologies men-
tioned here (see Fig. 5). The APIs implement insertion and retrieval operations of per-
ception events captured by the robotic agent.
The use of a triple store for sending the environment perception represents a bot-
tleneck in this deployment. In one cycle, dozens of information are stored in the triple
store by the Robotic Node and then, they are read and immediately removed by the Cog-
nition Node. In order to optimize this process only information associated with long-term
Figure 5. CMDE architecture processing nodes presented as a white box.
memory will be kept in the triple store, the other triples will be serialized and sent directly
to the Cognition Node.
3. Using the Ontology
The test is accomplished by the exercise of the ontology in a controlled environment
adhering to CMDE architecture. More specifically, given a usage scenario, the OntPercept
must offers resources to represent the exchange of perception information between the
cognitive and robotic systems.
Robot House Simulator (RHS) [Belo et al. 2017, Belo et al. 2018] was designed
to test CMDE architecture and the OntPercept ontology. Another element involves the
cognitive system that must control the robotic agent present in the simulator. In this
instantiation, this control is performed using the Soar cognitive architecture [Laird 2012].
The steps to create an experiment involve: defining the desired behavior for the
robotic agent, defining Soar production rules implementing the behavior, and finally, ex-
ercising the results in RHS simulator. The execution of a experiment can be visualized in
the video available at https://youtu.be/rPqfQvReXDo.
4. Conclusion and Future Works
This article proposed an ontology to establish the communication between cognitive and
robotic systems. This ontology was incorporated into CMDE architecture and a simu-
lator for social robotics, called “Robot House Simulator” (RHS). All these elements are
used together to design, develop and validate systems geared towards social robotics.
All configuration items generated in this development are released into GitHub repos-
itory (https://github.com/helioaz/OntPercept) with the GNU GPL v3.0
license.
The results achieved represent a important step in the development of an integrated
environment for the development of cognitive systems with application in social robotics.
This framework is constantly evolving and, as such, a catalyst for future proposals, e.g.:
inclusion of new elements in OntPercept; interface with others cognitive architectures
[ACT 2015, Gudwin et al. 2018, Kotseruba et al. 2018]; improve the RHS simulator with
new environments, buildings and streets and validation in a real environment.
Acknowledgements
This research is partially funded by the CAPES/MEC, the CNPq, the FAPESP under the grant
2017/01687-0 and the INCT-INSAC (National Institute of Science and Technology) under the
grant CNPq 465755/2014-3, FAPESP 2014/50851-0 and FAPESP 2018/25782-5.
References
ACT (2015). ACT-R Home. http://act-r.psy.cmu.edu/. [Online; accessed 08
feb. 2019].
Azevedo, H. (2018). Integration of cognitive and robotic systems through an ontology
to model the perception of the environment. PhD thesis, University of São Paulo
(USP), São Carlos, Brazil. https://teses.usp.br/teses/disponiveis/
55/55134/tde-18102018-103203/en.php.
Azevedo, H., Belo, J. P. R., and Romero, R. A. F. (2017a). Cognitive and Robotic Sys-
tems: Speeding up Integration and Results. In 2017 Latin American Robotics Sympo-
sium (LARS) and 2017 Brazilian Symposium on Robotics (SBR), pages 1–6. IEEE.
Azevedo, H., Belo, J. P. R., and Romero, R. A. F. (2017b). Reducing the gap between
cognitive and robotic systems. In 26th IEEE International Symposium on Robot and
Human Interactive Communication - RO-MAN2017, pages 1049–1054, Lisbon, Portu-
gal. IEEE. ISBN: 978-1-5386-3517-9.
Belo, J. P. R., Azevedo, H., and Romero, R. A. F. (2018). Enhancements in a social robotic
simulator for indoor environments. In 2018 Latin American Robotic Symposium, 2018
Brazilian Symposium on Robotics (SBR) and 2018 Workshop on Robotics in Education
(WRE), pages 457–463. IEEE.
Belo, J. P. R., Romero, R. A. F., and Azevedo, H. (2017). RHS simulator for robotic
cognitive systems. In 2017 Latin American Robotics Symposium (LARS) and 2017
Brazilian Symposium on Robotics (SBR), pages 1–6, Curitiba, Brazil. IEEE. ISBN:
978-1-5090-3656-1.
Fong, T., Nourbakhsh, I., and Dautenhahn, K. (2003). A survey of socially interactive
robots. Robotics and Autonomous Systems, 42(3-4):143–166.
Gudwin, R., Paraense, A., de Paula, S., Fróes, E., Gibaut, W., Castro, E., Figueiredo,
V., and Raizer, K. (2018). An Overview of the Multipurpose Enhanced Cognitive
Architecture (MECA). Procedia Computer Science, 123:155–160.
IEEE, R&A Society (2015). IEEE 1872 2015, Standard Ontologies for Robotics and
Automation.
IFR (2017). Why service robots are booming world-
wide. ifr.org/ifr-press-releases/news/
why-service-robots-are-booming-worldwide. [Online; accessed
30 jun. 2020].
Kotseruba, I., Gonzalez, O. J. A., and Tsotsos, J. K. (2018). A review of 40 years of cog-
nitive architecture research: Focus on perception, attention, learning and applications.
Computing Research Repository - CoRR, abs/1610.08602v3.
Laird, J. E. (2012). The Soar Cognitive Architecture. AISB Quarterly, 171(134):224–235.
McLeod, S. (2018). Forgetting. https://www.simplypsychology.org/
forgetting.html. [Online; accessed 13 aug. 2019].
Noy, N. F. and Mcguinness, D. L. (2001). Ontology Development 101: A Guide to
Creating Your First Ontology. Technical report, Stanford University.
Obitko, M. (2007). Translations between Ontologies in Multi-Agent Systems. PhD thesis,
Czech Technical University in Prague.
Pease, A. (2019). Suggested upper merged ontology (sumo). http://www.
adampease.org/OP/. [Online; accessed 15 aug. 2019].
Protégé (2020). Welcome to the protege ontology library. https://protegewiki.
stanford.edu/wiki/Protege4UserDocs. [Online; accessed 23 jun. 2020].
Stardog-Union (2017). Faq - pellet. https://github.com/stardog-union/
pellet/wiki/FAQ. [Online; accessed 08 feb. 2019].