Description Logics (DLs) are nowadays used to face a variety of problems. When dealing with numerical data coming from the real world, however, the use of traditional logics results in a loss of useful information that can be otherwise exploited using more expressive logics. Fuzzy extensions of traditional DLs, being able to represent vague concepts, are well suited to reason on such objects. In this paper we present an architecture for the automatic building and querying of a fuzzy ontology related to the representation of objects in terms of their composing parts. Our approach mainly aims to face the problem of visual grasping, which is of wide interest in the robotics eld.
The decomposition of an object in parts has been recognized as an important
problem in arti cial intelligence: it is considered both as a human-like way of
reasoning on objects [
Fuzzy DLs extend crisp DLs by adding imprecision and vagueness in the reasoning process, thus giving some degrees of truth in place of binary answers as yes or no. Although the available fuzzy reasoners are not yet as powerful as their crisp counterparts, some interesting applications can be found. One of them lies in the robotics eld, in which a symbolic representation of objects can improve the grasping capabilities of a robot by the use of some semantic information, regarding both the type of grasp itself and the structure of the object to be grasped.
To the best of our knowledge, the problem of semantic part decomposition is
still an open problem and there are no tools available to automatically create a
fuzzy ontology from raw concepts. The use of ontologies for object recognition has
been investigated in some works as [
There are some recent works in which fuzzy DLs are thoroughly used to
reason on multimedia information (see [
In this paper, we show why we use f-SHIN [
The f-S HI N logic
The f-SHIN logic is the fuzzy extension of the SHIN logic [
For the f-SHIN logic there exists a reasoner called FiRE 1, while there exist other reasoners like fuzzyDL2 which is based on the fuzzy extension of the SHIF logic. The reason why we chose to use FiRE as reasoner is, independently from the supported reasoning services, the high expressivity of the underlying f-SHIN logic as it supports cardinality restrictions; on the other hand, such a choice needs some functional blocks to be added to carry out operations like the de nition of concepts in terms of membership functions. 3
As anticipated in the previous section, due to the limitations of the reasoner, the whole architecture is complex and requires some functional elements to be split among di erent modules (e.g. the reasoner used on the de nitions ontology is di erent from the one used on the objects ontology). The whole architecture of the system is depicted in Fig. 1. 1 http://www.image.ece.ntua.gr/~nsimou/FiRE/ 2 http://gaia.isti.cnr.it/~straccia/software/fuzzyDL/fuzzyDL.html Types of features to extract from the image Types and parameters of membership functions Names of fuzzy concepts (e.g. LongObject) External ontology Creation of component axioms in the TBox Creation of roles related to component concepts Creation of object axioms
in the TBox Objects ontology building
Image segmentation and part decomposition Feature extraction for each part
Calculation of relationships
among the parts Selection of interesting quantitative measures Calculation and selection of truth values
Image analysis Fuzzy concepts with truth degrees
Creation of component assertions in the ABox Calculation of the GLB
for each component Creation of role assertions
in the ABox Calculation of the GLB for the whole object
Object recognition
Fuzzy reasoner
The \high level" information, which re ects the kind of knowledge that is to be extracted from the image, is encoded in the external ontology; the image analysis and the numerical calculations are performed with MATLABTM, while the intermediate steps are performed either in MATLABTMor in JavaTM. The FiRE reasoner is standalone, thus some steps are still to be carried out by hand.
As an example, we will model a fork in terms of its parts; thus, we will use the images shown in Fig. 2. 3.1
The external ontology, also called the \de nitions ontology", is used to specify the kinds of membership functions to be used as well as the kinds of features to be extracted from the objects found in the images (e.g. elongation, eccentricity, parallelism with respect to other objects and so on) and the meanings of concepts like LongObj and SmallObj in terms of membership functions.
Taking the ontology described in [
In our ontology, a concept like \a long object" is modeled as an individual longObject of meta-class Length which has, as its membership function, another individual longMF of a subclass of MembFunc with the function parameters given as datatype properties. By means of the Jena Ontology API3 and the Pellet reasoner4, information like the kind and the parameters of a membership function representing a concept related to the image is extracted to feed the image analysis module; thus, a SPARQL query like: SELECT * WHERE { ?x rdfs:subClassOf :GeometryConcept . ?y rdf:type ?x . ?y :hasMembershipFunction ?z . ?z rdf:type ?w . ?w rdf:subClassOf :MembershipFunction .
FILTER (?w != :MembershipFunction) . ?z :hasParameter1 ?k1 . ?z :hasParameter2 ?k2 .
OPTIONAL {?z :hasParameter3 ?k3} .
OPTIONAL {?z :hasParameter4 ?k4} } is used to extract the individuals representing the actual fuzzy geometry concepts (e.g. LongObject) used in the objects ontology and their related membership functions data (e.g. a sigmoidal function with two parameters).
The ontology is built by a domain expert to re ect the physical characteristics
of the robot, so that for example an object can be considered \long" with respect
to the maximum aperture of the robot hand. Although a system of measurement
has to be established, we now use only pixel measures.
3 http://jena.sourceforge.net/
4 http://clarkparsia.com/pellet/
(a) Original image
(b) Image after segmentation
(c) Image after edge dilation and part decomposition (with three parts
out of six put in evidence)
In this phase, the original image is converted in a binary image after thresholding
and edge recognition performed by Canny method [
After the rst phase, some features like the area, the length of the major axis of the ellipse having the same normalized second central moments as the selected region, and so on, are extracted from each found part (see Tab. 1 for some examples of extracted values); then, some quantitative characteristics are computed: for example, the measure of parallelness , given and as the angles between the major axes of the two objects and the x axis of the image, is de ned as = j cos ( ) j, while the distance between two parts, instead, is de ned as the minimum distance between their convex hulls.
Using the de nitions from the external ontology, for every part we calculate the degree of membership of each feature to its related membership functions. For example, for the feature \length" (i.e. the length of its major axis), the truth values for the functions \LongObj", \MediumLengthObj" and \ShortObj" are calculated; if a MediumLengthObj is associated to a generalized bell curve membership function with parameters a = 240, b = 2:5, c = 600 and the length of the major axis of the considered object is 456.61 pixels, the object will belong to the class MediumLengthObj with a truth degree = 0:93. Using the results from the previous phase, and taking as a working hypothesis that all the found parts belong to the same object (i.e. there is just one object in the scene), for each part only the membership functions which give the highest truth value for each feature are selected; for example, if a part has a truth degree over a threshold for the membership function \MediumLengthObj", the concept MediumLengthObj is added to the concept representing that part in the fuzzy ontology. At the end, we obtain a concept like (for the sake of simplicity we list only some concepts and roles):
MediumLengthObj u SmallObj u 5 parall u 1 near u . . . where ObjClass1 is the newly created concept related to the part which has been considered. A new fuzzy concept is created only if the current analyzed part does not belong to any existing concept, i.e. there is no concept that fully describes the part (it can be veri ed via the fuzzy reasoner). Since FiRE does not let us write fuzzy TBox axioms, the degrees of truth are discarded in this phase.
When there are no parts left, a role for each concept is created. For example, from the class ObjClass1 the role hasObjClass1 is created, so that the class Fork can be created using the previously found number of objects per class:
1 hasObjClass1 u 4 hasObjClass2 u
This is due to the fact that the f-SHIN logic lacks of the quali ed cardinality
restrictions, so a general hasPart role cannot be used. We use a \typographical"
operation, yet the problem of role creation has been faced in [
Once the objects ontology TBox has been built, it is possible to nd whether an object, after it has been decomposed in parts, belongs to a class or not (i.e. how much it can be considered to belong to the considered class with respect to a certain threshold); the image analysis steps are the same for the ontology building phase.
When for every part all the pertaining concepts and roles can be written in the ABox, the fuzzy reasoning is performed to nd the GLB of that part belonging to a certain class; then, roles like hasObjClass1 are created with the same value of the found GLBs and, at the end, the GLB of the main object is calculated.
This procedure can be applied to determine whether a speci c kind of grasp can be performed or not on the selected object. For example, given the concept de ned as (for the sake of simplicity using no roles):
MediumLengthObj u HighlyEccentricalObj representing objects that are graspable by a pinch grip, we can nd which part of the object (if any) can be grasped this way via a subsumption check. 4
In this paper we have presented a possible architecture for the generation and the use of a fuzzy ontology for object recognition by means of objects decomposition in parts. We take advantage of the use of fuzzy cardinality restrictions which, to the best of our knowledge, have not been fully exploited in the current fuzzy DLs applications (e.g. multimedia retrieval). Our results are preliminar and prone to errors, partly due to limitations in the modules in use (e.g. the fuzzy reasoner is still experimental), partly due to the approximations induced by the use of a SHIN logic, while at least quali ed cardinality restrictions would be needed.
As future work, we plan to take advantage of a more powerful fuzzy DL as it seems to be needed for object modeling purposes, so we will work on a more powerful reasoner and on a better integration between classical and fuzzy knowledge bases; furthermore, as we plan to use the system as an aid to the grasping task, we will add physical information (that can obtained via di erent sensors, e.g. haptic devices) and further information on the grasping types along with their quality measurements.