This paper describes our proposal in the multi-organ plant identi cation task (LifeClef2017 challenge [8]). The objective of the challenge is to evaluate to what extent machine learning and computer vision can learn from noisy data compared to trusted data. To address the challenge, we employ our recent proposed hybrid generic-organ convolutional neural network, abbreviated HGO-CNN [11] to train on di erent composition of plant datasets. Overall, all the submitted runs obtained comparable results in the LifeClef2017 plant classi cation task.
Plant classi cation has received particular attention in the computer vision eld
[
In 2013, LifeClef challenge [
The rest of the working note is organized as follows. In Section 2, we present the methodology of our proposed architecture. Section 3 illustrates its training scheme. Section 4 shows the experiments and results for both the validation and testing set. Lastly, Section 5 presents conclusion and future work. 2
Unlike previous approaches [
The proposed HGO-CNN comprises of four layers or components: (i) a shared
layer, (ii) an organ layer, (iii) generic layer, and (iv) a species layer. We
introduce shared layer for both the generic and organ components. The reasons are
threefold. First, [
In this challenge, we re ne some of the con gurations in the original
HGOCNN architecture: (1) the data layer normalization technique, called Batch
Normalization (BN) [
Pre-Training Two-Path CNN We design a two-path CNN as shown in Fig.
2 for the purpose of training two di erent components: the generic and organ
speci c features. This two-path CNN is initially pre-trained using the ImageNet
challenge dataset [
Organ layer After we obtained the pre-trained two-path CNN, one of the CNN path is repurposed for the organ task. This organ layer is trained together with the shared layer, using seven kinds of prede ned organ labels. We obtain organbased feature maps, xorg 2 RH W Z where H; W and Z are the height, width and number of channels of the respective feature maps. Since PlantClef2017 dataset does not provide organ information for every plant image, we train the organ layer based on the previous PlantClef2015 training set.
Generic layer After training the organ layer, another CNN path is repurposed for the generic task. This generic layer is trained using the species labels, regardless of organ information. We obtain generic-based feature maps, xgen 2 RH W Z . To allow both the organ and generic layers to share the common proceeding layer, we keep the shared layer's weights to be consistent. This is achieved by setting their learning rate to zero.
Species layer To introduce correlation between both the organ and generic components, a fusion function y = g(xorg; xgen) is employed at stage L (after the last convolutional layer for both components as shown in Fig. 1) to produce the organ and generic correlation feature maps, y 2 RH W Z . In our model, g performs summation of these two sets of features: yi;j;k = xorgi;j;k + xgeni;j;k (2) where 1 i H, 1 j W , 1 k Z. The feature maps, y will then go through two convolution layers to learn the combined representation of generic and organ features. Since these two convolution layers are new randomlyinitialised, we set their learning rate to be 10 times higher than the other layers during training. 4
Our architecture is trained using the Ca e [
For the trusted training set, we rst downloaded all 256287 images. We then randomly selected 208878 images for training and 47409 images for validation. To increase the robustness of the system in recognising multi-organ plant images, a multi-scale training was adopted. We isotropically rescaled the training images into three di erent sizes: 256, 385 and 512, then randomly cropped 224*224 pixels from the rescaled images to feed into the network for training. By doing this, the crop from the larger scaled images will correspond to a small part of the image or particularly subpart of the organ; while the crop from the smaller scaled images will correspond to the global structure of a plant. Besides that, we also increased the data size by mirroring the input image during training. After the data augmentation, we obtained 626634 training images and a validation set of 142227 images.
However, for the noisy dataset, we only managed to crawl up to 918216 number of images which is about 60% of the total number of images from the web due to resource limitations. We then separated it into 738716 images for training and 179500 images for validation. We performed the same data augmentation to produce another training set that contains 2216148 images and a validation set of 538500 images. For the testing set, all 25170 images are downloaded and similarly augmented. 4.2
For the evaluation of our validation set, the softmax output from our CNN
model for each image was rst collected. An averaging fusion was then used to
combine the softmax scores of the augmented validation set. In this experiment,
we computed the top-1 classi cation result (Acc) to infer the robustness of the
system. We compared our method to the generic network, VGG-16 net [
Table 1 shows the comparison of the performance results. We can observe that the VGG-16 net performed better in the noisy dataset while our proposed HGOCNN performed better in the trusted dataset. There are two possible reasons: (1) the organ layer in HGO-CNN that was trained on previous PlantClef2015 dataset might not be robust enough to model such a huge and diverse data, (2) noisy dataset in this case is better modeled using generic features regardless of the organ information as the generic features might include many independent plant features that help in the classi cation performance. 4.3
We submitted four runs to the LifeClef 2017 challenge. We netuned all models using both the training and validation set to increase the robustness of the models. To obtain the observation level predictions, an averaging fusion method was employed to combine the results of the testing images that have the same observation id. Their performance were evaluated based on Mean Reciprocal Rank (MRR). The characteristics of each run are stated below: { UM Run 1: Proposed HGO-CNN that trained with the trusted set only. { UM Run 2: VGG-16 net that trained with the noisy set only. { UM Run 3: Combined results of UM Run 1 and UM Run 2 based on averaging fusion at image level. { UM Run 4: Combined results of UM Run 1 and UM Run 2 based on max voting at image level. Fig. 3 shows the overall results of the LifeClef2017 multi-organ plant classication task. We observed that Run 2 which is ranked at 12th out of a total of 28 runs is the best among the submitted runs while Run 1 which is ranked at 19th shows the lowest result. Henceforth, we make a deduction that the fusion model HGO-CNN that we currently trained is not generalized enough to predict unseen testing images. Run 3 (ranked at 13th) and Run 4 (ranked at 15th), both are the combined results of Run 1 and Run 2 respectively, are ranked lower compared to Run 2. This is clearly due to the poor performance of the Run 1 model. Furthermore, Run 3 is ranked higher than Run 4, an indication that the averaging fusion method performs better than the max voting method.
Next, we compared the results of our submitted runs based on di erent composition of the dataset. Table 2 shows the results of the trusted training set(EOL). We observe that our UM Run 1 provides a comparable result, where it is ranked at 5th out of a total of 11 runs. We believe that the performance could have been better if the organ layer in the HGO-CNN is trained with the latest Plantclef2017 dataset. However, it was restricted as the organ information is not provided for most of the images. In Table 3, we have the lowest rank but we used only 60% of the noisy WEB dataset. Furthermore, there are only two participants in this category which is hardly a thorough comparison. Lastly, in Table 4, we observed comparable results for our submitted runs. However, we believe that our performance can be improved. In the current experiments, we separately trained the CNN models using two di erent datasets and inferred the results using average fusion. It is possible that the performance could have been better if both of the datasets are trained in one single end-to-end CNN model without the prerequisite of external fusion to infer the species. Moreover, training on 100% of noisy images might be able to boost up the classi cation performance. 5
This working note explains the implementation of the HGO-CNN for the Plantclef2017 challenge. We described the methodology of our proposed architecture and analyzed the results based on both validation and testing set. We observed that our current HGO-CNN model is not generalized enough to predict unseen testing images. This might due to the lack of robustness at the organ layer trained using the previous PlantClef2015 dataset. In the future, we will revise our proposed model to increase its robustness by re-training all layers using the latest datasets that incorporate both trusted and noisy images.
This research is supported by the Postgraduate (PPP) Grant PG007-2016A, from University of Malaya; and the used K40 GPU was donated by NVIDIA Corporation.