In this paper, we describe an approach to the automatic plant identi cation task of the LifeCLEF 2014 lab. The image descriptors for all submitted runs were obtained using the bag-of-visual-words method. For the leaf scans, we use multiscale triangular shape descriptor and for the other plant organs Opponent SIFT extracted around points of interest obtained using Harris-Laplace detector. We then use approximate k-means (AKM) algorithm to cluster these descriptors in large number of clusters/visual words (approximately 200K). Each image in the training and test dataset is represented as a sparse high-dimensional histogram of term (visual word) occurrences. The similarity between two images is de ned as a L2 distance over the obtained histograms. We use the standard tf-idf weighting scheme, which reduces the contribution that commonly occurring, and therefore less discriminative, words make to the similarity. To obtain the predictions, we employ a late fusion scheme for combining the similarities/ranks from multiple ranked image lists build for each type of view. Overall the proposed methods performed well, we ranked fth, out of 10 competing groups.
The ImageCLEF plant identication competition is organized every year since
2011 and aims to benchmark the progress in the area of plant identication from
images [
Contrary to previous plant identi cation benchmarks, queries are not de ned
as single images but as plant observations, meaning a set of 1 to 5 images
depicting the same individual plant observed by the same person the same day.
Each image of a query observation is associated with a single view type (entire
plant, branch, leaf, fruit, ower, stem or leaf scan) and with contextual
metadata (data, location, author). The motivation of the task is to t better with a
real scenario where one user tries to identify a plant by observing its di erent
organs. The details of this competition are described in [
In this paper, we describe our approach and runs submitted to the LifeCLEF
2014 Plant Task. The approach is based on bag-of-visual-words representation.
We are using Harris-Laplace detector to detect points of interest. From these
points, local invariant descriptors are then extracted. We used Opponent SIFT
as local descriptors [
The remainder of this paper is organized as follows. Section 2 brie y presents the training and test dataset. The image processing and feature extraction algorithms are described in Section 3. Section 4 presents the information fusion and classi cation algorithms that we used to obtain the predictions. Section 5 presents the results from the experimental evaluation. Finally, the conclusions and a summary are given in Section 6. 2
The Plant Identi cation task is based on the Pl@ntView dataset which focuses
on 500 herb, tree and fern species centered on France (some plants observations
are from neighboring countries) [
The distribution of training and test data of the Pl@ntView dataset is depicted in Table 1. As can be seen from the presented data, most of the images in the training and the test dataset are from the Flower view. 3
For image description, we used the bag-of-visual-words approach [
similarity de nition. The creation of the visual vocabulary starts with detection of interesting points in the images, and then proceeds with extracting local invariant descriptors from them. Finally, the visual codebook is obtained by clustering the large set of descriptors obtained from all of the images. The resulting clusters represent the visual words, while all the visual words comprise the visual codebook. The image description phase assigns all of the local image descriptors to the visual words from the visual codebook. Each image is then described with a high-dimensional histogram and each component from the histogram is the number of descriptors that are assigned to a given visual word. Finally, the images are ranked using term frequency inverse document frequency (tf-idf) scores which reduce the in uence of visual words which occur in many images. In the reminder of this section, we explain the phases in more details. 3.1
Image Processing and Feature Extraction The images can be categorized in two groups. The rst group is represented by scan and scan-like images, and in the second group are images from plants organs in natural surroundings, like branch, leaf, fruit, stem, ower and images from the entire plant. Having this in mind, we used two di erent feature extraction algorithms for the given images.
For the rst group of images (scans and scan-like images of leaf) we used
the triangle side lengths and angle (TSLA) descriptor from [
For the second group of images, we used Opponent SIFTs as local descriptors
extracted over the area around points of interest [
Secondly, over the area around points of interest, Opponent SIFT descriptors
were extracted. Opponent SIFT describes all the channels in the opponent color
space (eq. 1) using SIFT descriptors [
Approximate k-means (AKM)
The construction of a visual codebook is an essential part of the
bag-of-visualwords approach to image representation. For example, in our case, we are
clustering more than 10M local descriptors into more than 200K clusters. Generating
clusters from such a large quantity of data presents challenges to traditionally
used algorithms like k-means [
In typical k-means, the vast majority of computation time is spent on
calculating nearest neighbours between the points and cluster centers. We replace
this exact computation by an approximate nearest neighbor method, and use a
forest of 8 randomized k-d trees built over the cluster centers at the beginning
of each iteration to increase speed. We use the implementation from Philbin
et al. [
The conjunction of these trees creates an overlapping partition of the feature
space and helps to mitigate quantization e ects, where features which fall close
to a partition boundary are assigned to an incorrect nearest neighbour. This
robustness is especially important in high-dimensions, where due to the "curse of
dimensionality" [
Flower
Fruit
Stem
Leaf
Branch The complete pipeline for extracting the visual descriptors, creating the visual codebook and obtaining the image descriptors is presented in Figure 2. The proposed pipeline for obtaining the image descriptors is as follows. First, we apply Harris-Laplace detector on the images (training and test) that belong to the second group of images (leaf, ower, fruit, stem, entire and branch) and generate opponent SIFT local descriptors around the detected points. For the rst group of images (scans of leaves), we generate TSLA descriptors. Next, we use the generated descriptors to construct the visual codebooks. Note that, di erent visual codebooks were created for the di erent views of the plants (seven in total, one for each view). We randomly select a subset of the local descriptors (TSLA and opponent SIFT) from the training images for each view separately. The number of the local descriptors is varying from 6M for the scans to 10M for the owers. We use these descriptors as a input to the approximate k-means algorithm to obtain the clusters/visual words that will constitute the visual codebooks. Finally, each image in the training and test dataset is represented as a sparse high-dimensional histogram of term (visual word) occurrences.
The similarity between two images is de ned as a L2 distance over the
obtained histograms. We use the standard tf-idf weighting scheme [
For each run, we used the fact that images in the test dataset are associated with
plant observations to perform multiple image queries for all image organs and
scans having the same ObservationID value [
Lk corresponding to the query images I1, ..., Ik. { Finally, the 300 rst image results were kept for each list and were merged into a nal list L using a late fusion scheme.
We used three di erent late fusion schemes to obtain the nal predictions.
1. Min. rank fusion: For this fusion scheme, we used the Leave Out
algorithm (LO) [
...
List of Images I1 I2 . . .
Ik Same ObservationID
Img.D Img.D
Img.D We submitted three runs for the LifeCLEF 2014 Plant Task. As we stated previously, the three runs rely on the same visual descriptors but we used di erent fusion schemes to obtain the nal predictions. The results from the runs are presented in Tabale 2. The table contains the scores by image and observation. In our submitted runs, these two values for each run are the same. First we obtained the predictions for the observations and later on, we just apply these predictions for the images that are part of the corresponding observation.
The best performing run is the run named FINKI Run 1. This run is a combination of the other two runs. Namely, the predictions for the test images denoted with Leaf Scans were taken from the FINKI Run 2 and the predictions for the other images were taken from the run with name FINKI Run 3. We made this combination because in the validation phase, when we apply the algorithm on the ImageCLEF 2013 Plant Task, we obtained better results for the images denoted with Leaf Scan using the technique implemented in FINKI Run 1. . . .
L ..1.
L ..2.
By comparing the second and third run in Table 2 we can conclude that taking into consideration the distribution of the images across the di erent species does help in boosting the predictive performance. The run named FINKI Run 3 has better score compared to the run with name FINKI Run 2.
In Table 3, we present the detailed scores obtained for each type of plant organs. The best results are obtained for the Leaf Scan images. This is to be expected because these images contain only leaves and are taken in very controlled environment, in most of the cases on a white sheet as a background. The second best score is obtained for the images with owers. The lowest score is obtained for the images with branches and images that contain the entire plant. These images are most challenging in respect to the variant background and lightening conditions under which these images are taken.
Our best performing run was ranked fth from 10 di erent participants/research group. 6
We submitted three runs on LifeCLEF 2014 Plant Task. The image descriptors for all three runs are obtained using the bag-of-visual-words approach. For the leaf scans we are using multiscale triangular shape descriptor and for the other plant organs we are using Opponent SIFT extracted around points of interest obtained using Harris-Laplace detector. We are using approximate k-means (AKM) algorithm to cluster these descriptors in large number of clusters/visual words (approximately 200K). Each image in the training and test dataset is represented as a sparse high-dimensional histogram of term (visual word) occurrences. The similarity between two images is de ned as a L2 distance over the obtained histograms. We use the standard tf-idf weighting scheme, which reduces the contribution that commonly occurring, and therefore less discriminative, words make to the similarity.
Applied on the LifeCLEF 2014 Plant Task our approach was ranked fth, out of 10 competing groups. The approach we presented is general. We are planning to extend it with di erent image descriptors in order to tackle the di erent aspects of each plant organ/view. The inclusion of more image descriptors requires development of di erent and more complex weighting/fusion schemes.
We would like to acknowledge the support of the European Commission through the project MAESTRA - Learning from Massive, Incompletely annotated, and Structured Data (Grant number ICT-2013-612944).