This paper describes the participation of the CPPP/UFMS group in the robot vision task. We have applied the spatial pyramid matching proposed by Lazebnik et al. This method extends bag-of-visualwords to spatial pyramids by concatenating histograms of local features found in increasingly ne sub-regions. To form the visual vocabulary, kmeans clustering was applied in a random subset of images from training dataset. After that the images are classi ed using a pyramid match kernel and the k-nearest neighbors. The system has shown promising results, particularly for object recognition.
In recent years, robotics has achieved important advances, such as the intelligent
industrial devices that are increasingly accurate and e cient. Despite the recent
advances, most robots still represent the surrounding environment by means of a
map with information about obstacles and free spaces. To increase the
complexity of autonomous tasks, robots should be able to get a better understanding
of images. In particular, the ability to identify scenes such as o ce, kitchen,
as well as objects, is an important step to perform complex tasks [
This paper presents the participation of our group in the 6th edition of the
Robot Vision challenge1 [
Experimental results have shown that the image recognition system provides promising results, in particular to the object recognition task. Among four systems, the proposed system ranked second using the number of cluster k = 400 and number of images M = 150 for training the vocabulary.
This paper is described as follows. Section 2 presents the image recognition system used by our group in the robot vision challenge. The experiments and results of the proposed system are described in Section 3. Finally, conclusions and future works are discussed in Section 4. 2
In this section, we describe the image recognition system used in the challenge. This system can be described into 3 steps: i) feature extraction; ii) spatial pyramid matching; iii) classi cation. The following sections describe each step in details. 2.1
In the feature extraction step, the system extracts SIFT descriptors from 16 16 patches computed over a grid with spacing of 8 pixels. For each patch i, 128 descriptors are calculated, i.e., it is calculated a vector 'i 2 <128. To train the visual vocabulary, we perform k-means clustering of a random subset of descriptors D = f'g from the training set according to Equation 1. Throughout the paper, the number of clusters will be referred to as k and the size of the random subset of images will be referred to as M .
C = k-means(D) (1) where C 2 Rek 128 represents the clusters.
Then, each vector descriptor 'i is associated to the closest cluster according to the Euclidean distance (Equation 2). The index associated is usually called visual word in the bag-of-visual-word approach.
k i = arg min j'i; Cij j=1 (2) where j:j is the Euclidean distance. 2.2
The pyramid matching was proposed to nd an approximate correspondence between two sets, such as histograms. It works placing a sequence of grids over the space and calculating a weighted sum of the number of matches. Consider a sequence of grids at resolution l = 0; : : : ; L. A grid at level l has 2dl cells, where d is the space dimension which in our case is d = 2. In each cell i, it calculates the histogram Hl(i) of visual words . The number of matches at level l for two image X e Y is given by:
In order to penalize matches at larger cells, the pyramid match kernel between images X e Y , considering all levels l, is given by:
2dl Il = X min (HXl (i); HYl (i))
i=1 L(X; Y ) = 1 2L
L I0 + X l=1
1 2L l+1Il k KL(X; Y ) = X
L(Xj ; Yj ) j=1
The kernel above is calculated to each visual word, such that: where Xj indicates that the kernel will consider only the visual word j. 2.3
The multi-class classi cation is done with the k-nearest neighbor using the kernel KL described above. Given a test image, it calculates the kernel value for all training images and assigns the room/category of the closest training image. The same procedure is done for object recognition, i.e., it is detected the presence of the objects of the closest training image. 3
In this section we describe the experiments and results of the proposed system. To train the system, we have used 5000 visual images divided into 10 rooms/categories: Corridor, Hall, ProfessorO ce, StudentO ce, TechnicalRoom, Toilet, Secretary, VisioConference, Warehouse, ElevatorArea. An example of each category can be seen in Figure 1. The train dataset also provides 8 objects: Extinguisher, Phone, Chair, Printer, Urinal, Bookself, Trash, Fridge. Examples of images containing each of the objects can be seen in Figure 2. The number of images for each room/category and object is summarized in Table 1.
To test the proposed system, we have used the validation dataset composed by 1500 images. The results for di erent values of number of cluster k and images (3) (4) (5) (a) dor
Corri(b) Hall (c) Professor O ce (f) Toilet (g) Secretary (h) Visio Conference (i) house
Ware(j) Elevator Area used to obtain the vocabulary M can be seen in Table 2. The nal size of the descriptor is given by k PlL=0 22l. For L = 2 and k = 300, the nal size is 6300. Despite high number of descriptors, the system takes on average 0.9545 seconds to process an image. For each image, our system provided the room category and the presence of objects. The scores shown in the table are the sum of all the scores obtained for the images. The rules shown in Table 3 are used when calculating the nal score for an image.
Finally, the Table 4 shows the results for all participations in the robot vision challenge. Three groups have submitted solutions to this challenge and the baseline indicates the results obtained by the dataset provided script (Color & Depth histogram + SVM). For the competition we submitted four runs with di erent values for the parameters k and M (see Section 2.1). Our system ranked second, achieving 1738.75 points on this task for k = 400 and M = 150. This paper described the participation of our group in the Robot vision challenge. In this challenge, the proposed system ranked second among four others systems. Thus, the image recognition system has shown promising results, particularly for object recognition.
As future work, we intend to extend the approach for color images and use other classi ers, such as Support Vector Machine, which it is known to provide better results than k-nearest neighbors. In addition, the system will be applied in the depth images.
RCG and LCR were supported by the CNPq and PET-Fronteira, respectively.