Recent advances in computer vision have made possible the use of neural networks in large scale image retrieval tasks. An example application is the automated plant classification. However, training a network from scratch takes a lot of computational effort and turns out to be very time consuming. In this paper, we investigate a transfer learning approach in the context of the 2017 PlantCLEF task, for automatic plant image classification. The proposed approach is based on the well-known AlexNet Convolutional Neural Network (CNN) model. The network was fine-tuned using the 2017 PlantCLEF Encyclopedia of Life (EOL) training data, which consists of approximately 260,000 plant images belonging to 10,000 species. The learning process was sped up in the upper layers leaving original features almost untouched. Our best proposed official run scored 0,361 in terms of the Mean Reciprocal Rank (MRR) when evaluated on the test dataset.
Plants are one of the most vital life forms on Earth, having a tremendous contribution
to the well-functioning of the ecosystems. Hence plant taxonomy plays a key role in
preservation of plant species across the globe. Nevertheless, plant taxonomy is
problematic and often results in duplicate identifications, given the difficulty of the
classification task and the error of the human operator performing the manual classification.
Therefore, the plant identification challenge of the Conference and Labs of the
Evaluation Forum (CLEF) [
Convolutional neural networks came to the attention of the research groups when a
team led by Geoffrey Hinton and Alex Krizhevsky won the ImageNet Large Scale
Visual Recognition Competition (ILSVRC) [
Lately, a trend has been observed that industry together with research groups are
using more and more in their work deep convolutional networks architectures, due to
the high performance outputs. The same trend was also observed among the research
groups participating in the LifeCLEF plant identification challenge with outstanding
performance [
PlantCLEF 2016 campaign has brought together 94 research groups. Among these
only 8 research groups succeeded in submitting their runs [
PlantCLEF 2017 campaign, continues the challenge and aims to automatically
detect in the Pl@ntNet raw query images, specimens of plants belonging to the provided
training data. Another objective for this year is to evaluate the performance of a
system built with noisy data against one built using trusted data. Therefore, two main
training sets are provided, each being part of the same list of 10,000 plant species: a
“trusted” training set based on the online collaborative Encyclopedia of Life (EoL)
and a “noisy” training set built using web crawlers [
In this paper, we present the participation of the UPB HES SO team to the task.
Our proposal is a transfer learning approach, adapted to plant image classification.
We use a pre-trained model of the AlexNet CNN [
The rest of this paper is organized as follows. Section 2 describes the proposed method based on the fine-tuning of the AlexNet model for plant identification. Section 3 describes the training process, experiments and the results. Conclusions and future challenges are presented in Section 4. 2
In order to build our plant identification system we make use of transfer learning. For
that we considered a pre-trained deep neural network model which is part of the
Matlab Neural Network Toolbox [
AlexNet can be split into two distinct parts, based on the role they accomplish in the neural network. The first part of AlexNet is responsible with feature learning, being comprised of five convolutional layers from which the first, the second and the fifth are followed by max-pooling layers; while the second part of AlexNet encloses three fully-connected layers with an output layer of 1000 neurons for classification, representing each class in the neural network (see Fig. 1).
For fine-tuning of AlexNet we used the PlantCLEF 2017 Encyclopedia of Life (EOL) training set, having a base learning rate of 0.001 and a batch size of 512 without weight decay. In order to prevent the overfitting, two strategies were used: firstly, we set a threshold so that the training stops if the mean accuracy of the previous 50 iterations is greater than 99%; and secondly we imposed a L2 regularization factor of 0.001. The training process was sped up in the last fully connected layer, by multiplying the base learning rate with a factor of 10.
Also, the last fully connected layer was modified to fit our needs: its output size was increased to 10,000, representing our classes. Therefore, the network will be able to distinguish between 10,000 different species of plants. This section gives an overview on experiments we have conducted for both validation of the method and for final training itself. We firstly sought for the optimal hyper parameters in order to fine-tune the neural network model by conducting short experiments on a selected group of 1,000 categories of plants. Then, we took another validation step towards final training phase by using all the 10,000 categories of plants together with the optimal set of hyper parameters. Thereafter, we have fine-tuned the network in order to obtain our final runs. These aspects will be depicted in the following sub-sections. We have considered the PlantCLEF 2017 Encyclopedia of Life (EOL) training dataset containing about 260,000 images from which some were in grayscale and others in CMYK. We have removed all the grayscale images since most of them were representing sketches of plants, low quality images, or even maps (see Fig. 2). In this way a total of 65 images from 45 categories were removed from the database. We also found 19 CMYK images which have been reduced to RGB, i.e., removing the 4th dimension, and further used in training. All the images were resized to 227 x 227 pixels due to the AlexNet input layer requirements.
In our experiments we have used the Matlab Neural Network Toolbox [
To find the optimal hyper parameters to use for fine-tuning the network, we have selected the first 1,000 categories, in ascending order, from the training dataset. The main reason for choosing only a small sub-set from the training dataset was that in this way we could get a quick estimate of the final results by varying several hyper parameters.
Experiments thus consisted in choosing the first 1,000 categories from the training set, representing 25,094 images from which 80% (per class) were used for training and the rest of 20% (per class) were used in evaluation.
We split our preliminary experiments in two phases. In the first phase we conducted a coarse exploration of data by varying several hyper parameters: weight learn rate along with the bias learn rate multiplying factors in the last fully connected layer, batch size and threshold used in the early stopping strategy. The purpose of this step was to find the best approach which could score the highest accuracy when evaluated on validation data. In the second phase we took the winner from the previous one and started to vary the L2 regularization factor to see what contribution could bring to the performance of the network. Throughout the experiments the base learning rate was constant.
Experiments have shown that weight and bias learn rates from the last fully connected layer as well as the strategies used to prevent the overfitting have a major impact on the performance of the network. Therefore the best approach having a multiplying factor of 10 for weight and bias learn rates and a batch size of 512 has reached the early stopping threshold of 98% in 2,066 iterations, i.e., about 53 epochs, scoring 45.6% accuracy on the validation data (see Table 1).
Then, in the second phase the best approach having the same hyper parameters as the winner from the first one but with a L2 regularization factor of 0.001 has reached the early stopping threshold of 99% in 3,117 iterations, i.e., about 80 epochs, scoring 47% accuracy on the validation data (see Table 2).
We have carried out yet another preliminary experiment before we trained the network for our submission: we validated the optimal hyper parameters on the entire training set from which 80% of images (per class) were used for training and the rest of 20% of images (per class) were used for validation. The main objective of this experiment was to give us an estimate about the network performance when trained on the entire dataset. We therefore trained the network with the optimal hyper parameters having a base learning rate of 0.001, weight and bias learning rate multiplying factor in the last fully connected layer of 10, batch size of 512 along with an early stopping threshold of 99% and L2 regularization factor of 0.001. The model has reached the threshold in 16,668 iterations, i.e., about 42 epochs, and scored 30.55% accuracy on the validation set.
No. L2 Regularization Number of Epochs Accuracy 3.2
Results on the testing data Following the previous experiments, we have used the above mentioned hyper parameters in order to train four models which have been submitted for evaluation. Each of these runs is detailed below as follows: ─ UPB HES-SO Run 1: This is our primary submission hence is the starting point for the other runs. In this run we fine-tuned the AlexNet neural networks model using the optimal hyper parameters from the validation step without weight decay: • Base learning rate • Weight learn rate factor • Bias learn rate factor
This model has reached the early stopping threshold in 51,943 iterations, i.e., about
104 epochs, and scored 0.326 in terms of the Mean Reciprocal Rank (MRR) when
evaluated on the test dataset. It took the 11th place amongst the models which were
trained only with trusted dataset [
We have considered the PlantCLEF 2017 EOL training dataset, mainly because we wanted to validate our method aiming for the highest performance and we thought that this couldn’t be achieved on the noisy dataset alone.
All the models were trained using a NVIDIA Quadro M4000 GPU. Training for the first run took about 7 days to complete, while for the other runs took about 27 hours, the latter being derived from the first run.
The results for these four runs are depicted within Table 3.
In this paper, we presented the participation of UPB HES SO team to the 2017 PlantCLEF challenge. Choosing the optimal hyper parameters when fine-tuning a pretrained neural network model is a sensitive topic. Although transfer learning might be very convenient when comes to deep convolutional neural networks, it could lead to poor system performance if hyper parameters are wrongly selected. Another important aspect is the choice of the pre-trained model which should satisfy both the requirements of the task and the hardware limitations.
We explored several hyper parameters in context of transfer learning to find to what extent each one would impact the performance of the network. Thereby we found that weight and bias learn rates from the last fully connected layer as well as the strategies used to prevent the overfitting influence the network's ability to adapt to new tasks. Besides early stopping threshold and L2 regularization factor, a learn rate schedule has to be considered in order to improve the network performance. Thus our system scored 0,326 MRR when trained without weight decay, respectively 0.361 MRR when the base learning rate was dropped to half every epoch.
As previously mentioned, the chosen pre-trained model is a particularly important
aspect. Therefore we believe that our method could be improved if we choose for
instance one of the VGG Neural Network models [
We deeply understand the tremendous impact of plant taxonomy for environmental conservation. We intend to continue our research activity over the plant classification problem, complementing our approach with more advanced techniques such as data augmentation or even make use of Probabilistic Neural Networks (PNN). In this way, we aim to provide a sustainable approach when comes to ecological monitoring studies and environmental conservation. 21. Joly, A., Goeau, H., Glotin, H., et al.: LifeCLEF 2017 Lab Overview: multimedia species identification challenges, CLEF 2017 Proceedings, Springer Lecture Notes in Computer Science (LNCS). (2017)