We describe our system in 2014 LifeCLEF [1] Plant Identication Competition. The sub-system for isolated leaf category (LeafScans) was basically the same as last year [2], while plant photographs in all the remaining categories were classi ed using either local descriptors or deep learning techniques. However, due to large amount of data, large number of classes and shortage of time, our system was not very successful in the plant photograph sub-categories; but we obtained better results in isolated leaf images. As announced by the organizers, we obtained an inverse rank score of 0.127 overall and 0.449 for isolated leaves.
Plant identi cation campaign within LifeCLEF 2014 was similar to previous
years, but it was in larger scale with twice the number of classes and images
as of last year [
Our submission was very similar to that submitted in 2013 for the case of isolated leaves, while we started to build a new system for plant photographs using local descriptors and deep learning techniques. We split the task to share the work load: for Flower, Fruit and Entire categories, we used dense-SIFT descriptors, while for Branch and Leaf categories we used convolutional neural networks (CNN). The stem category was also recognized using globally extracted texture and color features, similar to LeafScans.
In both of these two approaches, the main problem was the large number of classes, which resulted in long training times. As a result, we exploited the meta-data wherever applicable: namely to split the ower/fruit categories according to owering/fruit bearing periods and trained a separate classi er for each time period. In this way, we aimed to reduce the number of classes dealt by each classi er. In the other categories (Stem, Branch, Entire and Leaf) time information did not seem very useful and was not used; in fact we did not use meta-data anywhere else in the system.
Preprocessing stages were present only for the isolated leaf images. Speci cally, we align the leaf's major axis with the vertical through principal component analysis, with additional correction coming from the leaf petiole's location. Then size normalization is achieved by normalizing the leaf height to 600 pixels, preserving the aspect ratio. Orientation normalization done this way is quite successful, however is not error-proof. Furthermore, errors in orientation normalization typically cause overall errors because most of the features are sensitive to orientation. There was no preprocessing for plant photographs. 3 3.1
The descriptors used for characterizing the samples of the LeafScan category are
identical to those used during the plant identi cation track of ImageCLEF 2013
[
BSS = fmaximum; minimum; median; varianceg
(1)
Area Width Factor (AWF) is computed on grayscale and constitutes a slight
variation of the leaf width factor introduced in Ref. [
Angle Code Histogram (ACH) has been used in Ref. [
Orientation Histogram (OH) is computed on grayscale data. After computing the orientation map using a 11x11 edge detection operator for determining the dominant orientation at each pixel, the feature vector is computed as the normalized histogram of n bins of dominant orientations.
Circular Covariance Histogram (CCH) and Rotation Invariant Point Triplets
(RIT) are both texture descriptors [
Color Auto-correlogram (AC) was used for color description [
Saturation-weighted Hue Histogram (SWHH) was also used as a color descriptor,
where the total value of each bin W ; 2 [0; 360] is calculated as:
W = X Sx Hx
x
(3)
where Hx and Sx are the hue and saturation values at position x and ij the
Kronecker delta function [
We used a single Support Vector Machine (SVM) classi er, as described in
[
We used local descriptors and deep learning techniques for the task of recognizing photographs of plants, where shape descriptors are not useful and global color and texture information is of limited use. In Stem category only, we used texture and color information globally, as the task seemed somewhat easier than others and due to lack of time.
Local Descriptors - Dense SIFT This approach is applied to Flower, Fruit and Entire categories. Due to the large number of classes that increase training times and decrease accuracy, we rst divided each category into sub-categories, using the date information from meta-data. Then we trained a separate classi er for each sub-category (15 of them in total), in order to reduce the problems and increase accuracy.
In the dense-SIFT approach, 16-by-16 patches are collected from images and
SIFT descriptors are calculated for each patch [
We used the Weka toolbox [
Deep Learning - Convolutional Neural Network: We trained a convolutional neural network (CNN) for the Leaf and Branch categories, that were seen as the most challenging sub-categories. The CNN we employed contains 8 layers where the output of the last fully connected softmax layer produces a distribution over the class labels. The rst and the fourth layers are convolutional layers with 5x5 kernels; the second and the fth layers are local contrast normalization layers; the third and the sixth layers are max-pooling layers; the seventh layer is a locally connected convolutional layer. The Recti ed Linear Units (ReLU) non-linearity is applied to the output of every convolutional and fully-connected layer [18{20]
In order to process each color image, the largest square region of the image is cropped and down-scaled to 32x32x3 for further processing. The rst convolutional layer lters the 32x32x3 input image with 32 kernels of size 5x5x3; the second convolutional layer takes as input the (response-normalized and pooled) output of the rst convolutional layer and lters it with 64 kernels of size 5x5x32; the third convolutional layer, a locally-connected layer with unshared weights, has 64 kernels of size 3x3x64; the nal layer is a fully connected soft-max layer.
In order to reduce over- tting we used two techniques: data augmentation and drop-out. For data augmentation we applied label preserving transformations on the image data such as re ections, small rotations and translations whereas dropout is employed in the third convolutional layer, just before the soft-max layer.
Global Features - Stem Category: For the characterization of the samples of the Stem category, we employed a combination of texture and color descriptors obtained from the whole image. The reason was two-fold: lack of time and the impression that this sub-category was relatively simpler compared to other subcategories.
We used as descriptors those used with isolated leaf images (i.e. OH, CCH, RIT), as well as the Morphological Covariance (MC). M C is the morphological equivalent of the usual covariance operator. It is based on successive erosions by means of point couples at various distances and orientations:
M C(f ) = V ol "P2;v (f ) =V ol (f )
(4)
where " denotes the erosion operator, P2;v a point couple separated by a vector
v and V ol the image volume, i.e. the sum of pixel values. As such it can capture
the directionality as well as periodicity of its input [
For training classi ers and validating our methods, we split the provided training set into two (Train and Validation sets), separately for each sub-category. In doing the split of the available development data, we kept all images of an individual plant in the Train or Validation splits, in order to reduce over tting.
We report the top-1 recognition accuracy of our system on this Validation data, in the fourth column of Table 1, since the labels for the Test set are not yet available. We also report the inverse-rank scores obtained from Test data, announced by the competition organizers, in the fth column. Note that the top-1 accuracies and the inverse-rank scores are not directly comparable, due to data set change, di erent metrics, and also because the o cial score uses a user-based averaging, while the accuracy results use an average over images.
The way we split the training data resulted in having fewer number of classes in the Validation set, as seen in Table 1, but we did not want to split images of the same individual plant across the two sets (as some of them could be very similar). As a result, the validation performance was not fully informative, as some classes were not in the set; but this was considered preferrable to alternatives.
Our results are not very good except for the LeafScan category where the
task is easier and we have obtained very good results in the last years [