We present the plant classi cation system submitted by the QUT RV team to the LifeCLEF 2015 plant task. Our system learns a content speci c feature for various plant parts such as branch, leaf, fruit, ower and stem. These features are learned using a deep convolutional neural network. Experiments on the LifeCLEF 2015 plant dataset show that the proposed method achieves good performance with a score of 0:633 on the test set.
Fine-grained image classi cation has received considerable attention recently
with a particular emphasis on classifying various species of birds, dogs and
plants [
To evaluate the current performance of the state-of-the-art vision
technology for plant recognition, the Plant Identi cation Task of the LifeCLEF
challenge [
Inspired by [
Test Sample GCNN Our proposed system consists of two main parts. First, we perform transfer learning to learn a domain-generic feature termed as GCNN from all plants images (regardless of content type). Second, we manually cluster the dataset into subsets based on content type and learn a feature speci c to each subset ( SCNN ). For each image we extract both domain-generic ( GCNN ) and subsetspeci c ( SCNN ) features, these features are obtained from layer 20, l20, of the deep network. The two feature vectors are then concatenated to form a single feature vector as shown in Figure 1. These features are then used to learn a multi-class linear SVM. Power and l2 norm are applied independently for domain-generic feature and content speci c feature prior to combining the feature vectors. There are 7 pre-de ned content types consisting of images from the entire plant, branch, leaf, fruit, ower, stem or a leaf scan. In both the training and testing phases all participants are allowed to use the indicated content.
We make use of the content type to learn a DCNN that is ne-tuned
(specialised) for a subset of the content types. However, because there is a limited
number of images for each content type, we rst group the most visually similar
content types toghether. In particular, we de ne four subsets. The rst subset
conists of the the entire plant and branch content types, the second subset
consists of the leaf and leaf scan content types, the third subset contains fruit and
ower content types, and the fourth subset consists of the stem only.
Krizhevsky et al. [
In this work we ne-tune a general model for the task of plant classi
cation. The base model that we ne-tune is the best performing model from
ILSVRC [
Transfer learning has usually been applied by ne-tuning a general network,
such as the network of Krizhevsky et al. [
Inspired by the ndings of Zhang et al. [
A separate DCNN is learned for each of the K = 4 pre-de ned subsets by netuning the domain-speci c model, described in Section 2.3. The aim is to learn features for each subset that will allow us to more easily di erentiate visually similar content of plant species. As such, for each subset, we apply ne-tuning to the pre-trained GoogLeNet model. To train the k-th subset (Subsetk) we use the Nk images assigned to this subset Xk = [x1; :::; xNk ], with their corresponding class labels.
The only di erence between these models and the pre-trained GoogLeNet model is that the number of outputs for the last fully connected layer, of each model, is set to the number of training classes in each subset. Transfer learning is then applied separately to each network using backpropogation and stochastic gradient descent (SGD). For each image belonging to the k-th subset a subset feature vector SCNNk is obtained by taking the output of the last convolutional layer l20.
In this section we present a comparative performance evaluation of our proposed method on a validation set and the de ned test sets. The provided training dataset is split into two sets: roughly 10% of the total training data was used as a validation set and the rest is used for training the models. The split is based on observation id because nal testing is also observation-based.
This results in 82,033 training images, including 21,746 for the branch and entire subset, 32,186 for fruit and ower subset, 23,234 for the leaf and leaf scan subset and 4,867 for the stem subset. The validation set consists of 9,725 images.
We use Ca e [
First we assess our proposed method on the validation set. We conducted three sets of experiments which examine the e ectives of the domain-speci c feature vector, the subset feature vector and the combination of these two feature vectors.
The results on the validation set, shown in Table 1, demonstrate that the combination of these two feature vectors provides a considerable performance improvement. The combination of these two feature vectors achieves a mean accuracy of 66:6%. This is an absolute improvement of 6.5 percentage points over the domain-speci c feature vector GCNN which achieves a mean accuracy of 60:1%. By comparison, the subset feature vector SCNNk achieves a mean accuracy of only 58:0%. We believe that the subset feature vector performs worse than the domain-speci c feature vector because of the limited number of training images for each subset. In this section, we present our submitted results for the LifeCLEF2015 plant challenge. We submitted three runs: { RUN1 is the result of using proposed system for classi cation purpose. Only the rank 1 score is submitted for each observation. { RUN2 is the image retrieval task where we take the rst 5 predictions. { RUN3 is based on RUN 2 but we perform an additional softmax normalization for the rst ve predictions.
In Figure 2 we present the overall performance for all of the competitors using the de ned score metric. It can be seen that our best performing system is RUN 2 which achieved a score of 0:633. This is slightly worse than SNUMED INFO systems (RUN 4 and RUN 3).
In Figure 3 we present results for the image-based run. It can be seen that our proposed method provides competitive performance for both the imagebased and observation-based metrics. However, we do have a minor performance loss for the image-based result compared to the observation-based result. 4
In this paper we presented a domain-speci c feature learning and subset-speci c feature learning system applied to the plant identi cation task of LifeCLEF 2015. For domain-speci c feature learning, we have shown that it is possible to perform transfer learning from a DCNN pre-trained on the larger-scale ImangNet dataset. Furthermore, we have presented a subset feature learning system that is able to learn content speci c features. This approach yields highly competitive performance with a score of 0:633 for this year's task.
The Australian Centre for Robotic Vision is supported by the Australian Research Council via the Centre of Excellence program. NICTA is funded by the Australian Government through the Department of Communications, as well as the Australian Research Council through the ICT Centre of Excellence program. We would also like to thank Professor Chunhua Shen and Dr. Lingqiao Liu for the fruitful conversations of this work.