Deep learning techniques have signi cantly improved plant species classi cation in recent years. The goal of the 2018 ExpertLifeCLEF challenge was to compare the performance of human experts to machines trained on the PlantCLEF 2017 dataset containing 10.000 classes. We used the Inception, ResNet and DenseNet architectures to solve this complex task. In our experiments, complex neural net layouts yield strong results, comparable to human performance. We further push the overall accuracy through iterative adjustment of class weights. An ensemble consisting of a ResNet50 and two DenseNet201 with ne-tuned class weights reached a top1-accuracy of 77% on the test set.
The ExpertLifeCLEF 2018 challenge [
This year's training set is exactly the same as for the PlantCLEF 2017 task. The clean dataset holds 256.288 samples and the noisy set 1.432.162. These samples are for 10.000 di erent plants. Both datasets show massive class imbalance. 2.1
No changes have been made regarding the clean set. Some images from the noisy set had to be removed since they were either not compatible with image processing tools or/and had corrupt EXIF data, which led to complications. This a ected 4.398 total images which is a very small part of the whole dataset and therefore neglectable. No further ltering has been done to either of the datasets. 3
We tested and used three network architectures, namely Inception v3[
The mentioned three CNNs have been tested with various learning rates, batch sizes and optimizers to nd the best hyperparameter setup. All experiments described have been done using the clean dataset.
Batch Size We used a NVIDIA P6000 graphics card for training. Since it was not possible to utilize all of the 24GB of VRAM for the training of a single model, the GPU memory was split into parts of 8GB. This way, we were able to train multiple models at the same time and nish more experiments. The restriction of VRAM per model led to a maximum batch size of 64 for the Inception v3 which outperformed models trained with a lower batch size. The ResNet50 had similar results, but with a maximum batch size of 32. The DenseNet201 used even more of the GPU memory and could only be used with a batch size of 16. Optimizer Inception, ResNet and DenseNet were each tested with Stochastic Gradient Descent (SGD) and Adam. A static learning rate of 0.01 was used for the SGD. All architectures performed much better with the SGD than with then the adaptive optimizer.
The experiments showed that the decay schedule was best for the Inception v3 and the ResNet50. The DenseNet201 did not bene t from this schedule and performed better with a static learning rate and Nesterov momentum. 3.2
We used a variety of augmentation operations to further increase the dataset diversity. Selected experiments showed that the validation error can be greatly (a) Original.
(b) Zoomed in. reduced by the augmentations. Finally, we decided to implement following augmentation methods: Horizontal and vertical ip, zooming, rotating, shearing and shifting.
The range of every augmentation method was determined by empirical testing. Augmentation artifacts, for example as result of rotation, were lled in with a re ection of the original image content. 4
We trained the ResNet and Inception models according to best practices. However, we tried to ne-tune the DenseNet201 with adjusted class weights over multiple iterations and we will describe this speci c process in more detail. For all experiments, ten percent of the trusted image set were used for validation. 4.1
In an attempt to balance the classes, the initial weights were generated from the initial distribution of the samples. Therefore, the weight of each class was initially calculated using the average samples per class divided by the number of samples of that particular class. This redistribution of class weights already improved the accuracy by 9% over our local test set. Over the course of six iterations, the class weights were further modi ed. With each iteration a small test set was predicted and the error rates per class were calculated. The error rate is the percentage of samples which were classi ed correctly.
We modi ed each class weight using the following equation: weight = weight (1:85 error rate) (2) We reduced weights for classes with an accuracy of 85% or higher by a small percentage; classes with a lower accuracy got higher weights in the next iteration. 4.2
Each model was trained with 10.000 samples per epoch, for up to 1.000 epochs. Even with a very powerful GPU, the training of one model took three to ve days. After the training, new class weights were generated using the error rates of the model on the test set and another model was trained with the modi ed class weights. We repeated this process six times until we submitted our runs. 5
This year's ExpertLifeCLEF challenge was themed as human experts vs machines. The results show that some of the experts did have a remarkably high accuracy, which was rather surprising - but not all experts did beat the machine performance. Our system was able to beat one expert and scores were within a small margin with other experts.
Most of our models were trained on the noisy data set, because the last year's
PlantCLEF results showed that the highest scores were achieved when training
mostly on the noisy data [
TUC MI run1 The rst submission was based on a weighted voting between ve models which were all trained on the noisy dataset. These ve models consisted of one ResNet50, an Inception v3 and three DenseNet201. The weight of the vote was calculated using the validation accuracy of each model. We trained each of the DenseNets with a di erent class weight con guration.
TUC MI run3 This submission consisted of the predictions of one DenseNet201. This model had the best single net validation accuracy of all our trained models. It was only trained on the noisy set and is the last iteration of the ne-tuning process described in 4.1.
TUC MI run4 Another single DenseNet201 was used for the fourth run, taken from the last iteration of the class weight ne-tuning. Unlike run3, this model was mainly trained on the clean data set.
TUC MI run5 Weighted voting of three models -two DenseNet201 and a single
ResNet50 - resulted in our best scores. The two DenseNets were taken from run3
and run4. The ResNet50 model was trained on the noisy set, but with no further
weight con gurations.
In our experiments, the DenseNet outperformed Inception v3 and the ResNet50
architectures. The adjustment of the class weights used during training did
improve the performance of the DenseNet gradually. A single ne-tuned DenseNet
already scored a Top1-accuracy of 71.8% which should increase further with more
training time. Considering the strong results of our experiments and the
submissions of other participants, we can conclude that machine plant classi cation is
within reach of human-like performance. Some experts are able to identify plants
based on images with very high accuracy. Most likely, further improvements of
deep learning techniques will close this gap in the next few years.
Our system was implemented with Keras [