We describe our participation in the plant identi cation task of ImageClef 2013. We submitted one fully automatic run that uses di erent features for the uniform background (isolated leaves) and natural background (unconstrained photos) categories. Besides the category information, meta-data was only used in the natural background category. Our approach employs a variety of shape, texture and color descriptors. As in the previous years, we used shape and texture only for isolated leaves and observed them to be very e ective. Our system obtained the best results in this category with a score of 0.607 which is the inverse rank of the retrieved class, averaged over all queried photos and users. As for the natural background category, we used a limited approach using a restricted set of features that were extracted globally due to lack of time, and obtained a score of 0.181.
The ImageCLEF plant identi cation competition is organized every year since
2011 and aims to benchmark progress in the area of plant identi cation from
photographs [
The organizers collected a large set of data from 250 di erent plant species over the course of several years. Part of this data formed the training set that was distributed to the participants along with the corresponding groundtruth. The remaining data was shared with the participants in order to collect their systems' responses, while the corresponding groundtruth was kept sequestered.
Submitted systems were scored in terms of the inverse average rank of
the correct class for each submitted query. The details of this competition are
described in [
As a collaboration from two universities in Istanbul, we submitted a single fully automatic run (Sabanci-Okan-Run1) that uses di erent features for the uniform background (isolated leaves) category and natural background (unconstrained photos) sub-categories. The category information was obtained from the metadata of the query image. This handling of queries in di erent categories was done to select the appropriate feature set for each group, but it also helped with the handling of this large task.
As in the previous years, we used shape and texture only for isolated
leaves and observed them to be very e ective. We had the best average score
overall last year in both the automatic and manual categories [
For the natural background category, we used texture and color features for the ower, fruit and entire sub-categories; shape and texture for the leaf category; and only texture features for the stem category. The feature group selection was done based on our previous experiences in this problem and in order to increase generalization performance; it also helped reduce the time spent in feature extraction.
Meta-data was used only in the natural background category; speci cally the month information was used to narrow down successfully the alternatives for fruit and ower categories. 3
Although segmentation is of crucial signi cance for content description, it
has been used in our system only for isolated leaves and stems. In contrast,
segmentation of photographs with a natural background is either not meaningful
(i.e. the whole picture contains some part of the plant) or not an easy problem
even though the background is well-de ned (e.g. a plant photographed with the
forest ground). In ImageCLEF 2012, we had used an approach where photos
were aggressively segmented to leave only a single leaf in the image, in order to
channel photographs to our successful isolated leaf recognition system [
Isolated leaves usually possess an uniform background, often with uneven
illumination and sometimes shadow. Their segmentation has been conducted as
in the past, using edge preserving morphological simpli cation by means of area
attribute lters, followed by an adaptive threshold [
Preprocessing stages were present only for the isolated leaves, in the form of size and orientation normalization. Speci cally, we align the leaves' major axis with the vertical and normalize their height to 600 pixels, preserving the aspect ratio. Orientation normalization is realized through principal component analysis, with additional correction coming from the leaf petiole's location. 5
Given the high visual variability of this year's dataset categories as well as
the number of classes, feature extraction has become more challenging than
ever before. Consequently, a large spectrum of descriptors has been evaluated,
including shape, texture, color and local invariants. Moreover, considering the
strong relation between seasons and image categories such as fruit and owers,
meta-data have also been exploited with great success. Here we summarize only
the new descriptors, while the others have been explained in detail at the previous
working notes [
In particular, following the success of our past systems with scan and scan-like data (isolated leaves), it has been chosen not to greatly modify their descriptor set; instead we mainly optimized their parameters in order to cope with the higher class count. In addition, only one new descriptor was included in the feature extraction set: the edge background/foreground histogram. It is computed on the binary mask of its input and it consists in calculating the ratio of background to foreground pixels in a subwindow centered on each edge pixel. The normalized histogram of the said ratios constitutes the end feature vector.
As far as photographs are concerned, given the extreme variation of
viewpoint and scale (especially w.r.t. the category \entire"), we resorted to using
rather traditional, yet still reliable color descriptors. In particular, we employed
the color autocorrelogram [
We further employed the saturation-weighted hue histogram [
W = X Sx Hx
x
(1)
where Hx and Sx are the hue and saturation values at position x and ij the
Kronecker delta function. As far as the color space is concerned, we have used
LSH [
And last, in order to exploit the e ect of seasons on fruit and owers, it has been decided to use the meta-data accompanying the visual samples, and speci cally the month of acquisition.
Data
The competition data consisted of a training set which was made available to all the participants, along with the corresponding groundtruth les, and a test set whose groundtruth was kept sequestered. The distribution of the data in each category and in each of these sets is shown in Table 1.
We split the available training data shown in Table 1 into train and
validation subsets. The training set was used in training the corresponding
classi er and the validation set was used as our internal test data for evaluating
di erent features and algorithms. In order to help with the generalization
capability, we tried to avoid having very similar images in the train and validation
splits. Speci cally, pictures from an individual plant were put in either the train
or validation subset. The selection of the samples was done as described in [
Feature extraction was done from the whole picture, except for the case of leaf images in the SheetAsBackground and the NaturalBackground categories, where segmentation step preceded feature extraction. The approach of using global features or using only color features is clearly not su cient for unconstrained photos (e.g. ower, fruit, entire categories), however we did not have time to incorporate other methods based on local features.
The classi ers used for di erent categories were all trained with the training portion of the available data shown in Table 2, except for the leaf sub-category of NaturalBackground photographs. For this group, we used the same system developed for recognizing the SheetAsBackground category, after a simple segmentation of the image.
As classi er, we used a Support Vector Machine (SVM) classi er based on their good performance in many object recognition problem and used the SMO classi er inside the WEKA toolbox. The parameters for the SVM was set asC = 10 and a polynomial kernel of degree 2 after some limited tests with the validation set.
In Table 3, we give the cross-validation accuracy obtained while training
a classi er using 10-fold cross-validation, as well as the accuracy of the same
classi er on the validation subset. In the last column of this table, we also include
the average inverse rank results published by the competition organizers for each
category [
Participation into the ImageCLEF Plant Identi cation competition is an arduous task, especially when done in collaboration, with di erent people working in di erent parts of the problem. Last year we had to transfer partial results back and forth, since alternating steps of segmentation, preprocessing, feature extraction, and classi cation were done by di erent people in our small group. This year we streamlined this process a little better and concentrated on what we could accomplish the best. For that reason, we worked on isolated leaves the most, while some categories received minimal attention (e.g. leaves under the NaturalBackground category).
As the o cial results indicate, we obtained the best results in recognizing
isolated leaves (SheetAsBackground category), with an average inverse rank of
0.607. This score roughly indicates that that the correct class was returned as
top-1 or top-2 alternative for the majority of queries, which is a promising result
for the plant retrieval problem. In recognizing the unconstrained photographs in
the NaturalBackground category, we started working on a system based on SIFT
features [