This paper introduces a Fuzzy Logic framework for scene learning, recognition and similarity detection, where scenes are taught via human examples. The framework allows a robot to: (i) deal with the intrinsic vagueness associated with determining spatial relations among objects; (ii) infer similarities and dissimilarities in a set of scenes, and represent them in a hierarchical structure represented in a Fuzzy ontology. In this paper, we briefly formalize our approach and we provide a few use cases by way of illustration. Nevertheless, we discuss how the framework can be used in real-world scenarios.
In order to achieve a natural interaction between humans and robots, it is crucial robots be able not only to learn by human examples, but also to organize learned knowledge for a long-term interaction, as well as for communicating it to humans. On the one hand, such learning techniques as Learning by Example [4][5] are efficient ways to teach robots well-defined skills, but the resulting representation typically does not take interaction aspects into account. On the other hand, as any form of inductive reasoning, learning can deal with generalisation and the conceptualisation of knowledge only to a limited extent [3], and therefore methods to support robot-to-human communication and vagueness in scene descriptions must be considered.
In this paper, we present a perception framework3 structurally requiring human examples and representing the environment using a formalism based on Fuzzy Logic. The approach allows a robot to acquire scenes of the robot’s workspace via human examples, represent spatial relations among objects therein using fuzzy concepts, and hierarchically classify scenes on the basis of their similarities. We focus on scenes related to a tabletop scenario, which are sequentially shown by a human teacher. Our approach populates a fuzzy ontology, which can be used to encode vague representations of spatial relations and reason upon them to perform classification, ground human-robot communication and, in perspective, perform task planning. 3 A first implementation is available at: https://github.com/EmaroLab/fuzzy_sit.
In the following Sections, we briefly introduce our framework and we show a few simple examples to describe the system’s behaviour and discuss its main features. Our framework is based on a fuzzy OWL ontology [8], managed by the fuzzyDL reasoner [1], which (i) represents scenes in terms of fuzzy objects and fuzzy spatial relations among them, and (ii) determines the similarity between any pair of scenes using their fuzzy descriptions.
We assume as an input (i) the classification of perceived objects according to a number of predefined fuzzy sets (e.g., Book or Cup), i.e., the object type, and (ii) the knowledge of the spatial relations objects are involved in (e.g., right or left), both with the associated degree of membership, as shown in Table 1. Two mapping procedures are used to manipulate the ontology at runtime: M1 maps percepts to a fuzzy scene individual S in the ontology, whereas M2 creates a new fuzzy scene class Scene from a single individual S would the latter not be classified by any class in the ontology.
In order to understand how M1 works, let us assume that the robot perceives a scene where two books, namely B and D, are detected, with D at the right hand side of B (first scenario in Section 1). A fuzzy set Book is defined to assess the degree of membership of the two books B and D, for instance Book(B,1) and Book(D,.8). A fuzzy spatial relation right is used to represent the degree of membership of the assertion D at the right hand side of B, for instance right(B,D,.9). Then, for all objects which are in a right relationship with other objects, we sum up the degrees of membership of the corresponding fuzzy relation, in this case .9, to determine a reified description relating object types and relations [6], e.g., hasBookRight(.8), which we use to define S. In general terms:
S ,
where:
e jk = åL j
9lk:L j
å max min L¯ (I);Y:lk(I)
Y 2L¯ I2L j
:
In (
We assess the similarity between two scenes by comparing the number e jk of
relations involving a certain object type (e.g., Book and right) in scene descriptions in
the form of (
M2 determines Sigma Counter restrictions for all the reified spatial relations in the set D¯ . When all restrictions are computed, a fuzzy scene class Scene is created as a fuzzy set whose purpose is to classify S. Considering the example introduced above, a description is created such that hasBookRight.atLeast(.8), and it is assigned to Scene. In formulas:
Scene
O
j2L¯ ;k2l¯
9hasD jk:atLeast(Le f tShoulder(e jk)):
(
Three remarks can be made: (i) a class is inductively derived from a single example,
when the fuzzy ontology does not contain any fuzzy set whose description is suitable
to classify a scene individual; (ii) human supervision may be needed to corroborate
or modify the class description; (iii) the scene formulation in (
Experiments have been performed in an incremental manner. At bootstrap, the ontology does not contain assertions. Then, all scenes are shown sequentially to the robot, which applies the mappings M1 and M2 described above. We assume a fuzzy scene individual as being recognised when the associated degree of membership is higher than :97. When all scenes are classified, a hierarchy is induced, which is shown in Figure 1. If we focus on the two last columns of Table 1, it is possible to see how both learning and recognition performance scale with the complexity of descriptions, as well as with the number of assertions in the ontology. All experiments have been performed using a graphical user interface (available open source3), which simulates the perception of objects and spatial relations. Results have been collected using an Intel Core i5-460M (2:53 Ghz) processor with 4 GB of DDR3 memory.
Table 1 shows a few examples of scenes presented to the robot, where only two
object types and one spatial relation is considered. The first column of the table depicts
object arrangements, whereas the second and the third column represent the fuzzy
predicates specifying the degrees of memberships for object types and spatial relations.
The fourth column shows the description of the resulting fuzzy scene individual,
obtained applying (
O o i r a n e c S
D
D C
C
In the recognition phase, a new fuzzy scene individual S can be classified as being an instance of one of three fuzzy scene classes (i.e., Scene1, Scene2 and Scene3), whereas in the learning phase the hierarchy must be updated to create a new fuzzy scene class, which increases the overall computational time. It is noteworthy that learning and classification performance depend also on the number of objects in the scene and the related spatial relations.
It is possible to observe that if Scene2 or Scene3 are classified, also the objects arrange- Scene1 ment in Scene1 holds with a full degree, given the 1.0 1.0 qualitative nature of spatial relations. Furthermore, 0.32 Scene2 and Scene3 share some similarity, even if Scene2 Scene3 not as clearly as in the previous case, which is rep- Figure 1. The s0c.e2n4e hierarchy obresented by the reasoner with a low fuzzy implica- tained as a result of the learning protion value between them. Such a behavior is due to cess: circles are fuzzy scene classes, the Sigma Count approach, specifically to how the while arrows are fuzzy implications left-shoulder function changes between 0 and e jk. with the related degree of membership.
Table 1 shows also how the fuzzyDL reasoner
can deal with vague spatial relations. It is noteworthy that, just for the sake of argument,
degrees of membership in spatial relations are negatively correlated with the actual
distance between objects. Finally, the framework can also deal with inaccurate scene
recognitions. It could be the case that an object B is classified as a Book(B; :8) and also
as a Cup(B; :2), which is slightly located on the left hand side of D, i.e., right(D,B,.3).
In this case, applying (
We introduce a framework based on Fuzzy Logic to learn, recognise and hierarchically classify tabletop scenes presented to the robot by a human teacher. The framework can serve as a basis to ground human-robot communication processes: on the one hand, it can deal with the intrinsic vagueness associated with human perception of spatial relationships; on the other hand, its performance is sufficiently good for a natural interaction in human-robot interaction scenarios.
The framework is under test to benchmark its scalability and representation capabilities. Currently, research activities focus on: (i) determining possible singularities in the scene representation to avoid degenerate cases; (ii) integration between natural spatial representations and speech-based interaction; (iii) evaluation of action planning methods to perform object manipulation with the aim of recreating a previously perceived scene.
This work has been partly supported by a grant of the Fondazione/Stiftelsen C.M. Lerici awarded to the first author.