A robust snake species classi er could aid in the treatment of snake bites. In this report, the technique of transfer learning is revisited to understand the signi cance of the underlying pre-trained network and the supervised datasets used for pre-training. In low data regime, the methodology of transfer learning has been instrumental in building reliable image classi ers. Comparisons are made between the pre-trained networks trained on datasets of di erent sizes and classes. Performance improves signi cantly when the pre-trained network is trained on a much larger supervised dataset. Using country metadata improves the performance considerably. In SnakeCLEF2020 challenge, an F1-score of 0.625 was achieved.
Snakebite is the second most deadly neglected tropical disease [
With the goal of developing biodiversity monitoring systems, LifeCLEF [
Recently, an extensive study has been performed [
4.1
Vanilla ResNet50-v2 [
The following strategies were adopted during training: { Trained for 10,000 steps. { The batch size of each step was 512. { Mixup augmentation was used. { Staircase based Learning rate scheduler. { Optimizer: Schocastic gradient descent with momentum 0.9.
{ Cross Entropy Loss. 4.2
Preprocessing The given images are of varied sizes. During the training process, the images are rst resized to 512x512x3 dimensions using bilinear interpolation method and a random crop of 456x456x3 was taken. The images were also horizontally ipped with a probability of 0.5. During the validation and the testing process, the images were only resized to 456x456x3 dimensions using bilinear interpolation. The images were also normalized with a standard deviation and mean of 0.5 and 0.5 respectively for training, validation, and testing process. Batch Accumulation Training can be very ine cient if the mini-batch size is small due to noisy gradients. To accommodate large mini-batch size into GPU memory, batch accumulation is generally used. Gradients are accumulated over 16 steps without updating the model and then updated. Although the size of each mini-batch is 32, the e ective mini-batch size is 512.
Learning Rate Learning rate is the crucial hyperparameter to the task of
netuning. After linear warmup, a stair-case based learning rate scheduler was used
following the hyperrule provided by [
Augmentation Deep neural networks are prone to undesirable behaviors such as memorization, sensitivity to adversarial examples, and sampling bias. To combat the issues of over tting, mixup augmentation [15] was used. Mixup strategy trains the network on convex combinations of pairs of examples and their labels. The combination can be controlled by a factor, . With = 0:1 , a sample training image is shown in Figure 3. This favours the network to discriminate between various classes better.
Fine-tuning is done on two open source models of ResNet50-v2 architecture pre-trained on ImageNet-1k (Model-A) and ImageNet-21k (Model-B) datasets respectively under same training conditions. Results are summarized below: The network pre-trained on ImageNet-21k signi cantly outperforms its ImageNet-1k counterpart. Especially, classes with fewer data points are discriminated better re ected by the signi cant improvement in the F1-score.
Model-A disagrees with Model-B for 1,989 images, where Model-B is correct. Also, Model-B disagrees with Model-A for 433 images, where Model-A is correct. By analyzing images with the highest discrepancies (i.e) the images for which Model-B is correct and the probability of the correct species inferred from ModelA is very small, further insights could be gained. An attribution technique [13] to understand which pixels(features) are considered important by the model was performed. The generated saliency maps for top 3 images where models disagree the maximum was chosen. A single gradient step with respect to the target class for the given image was calculated. As shown in Figure 4, by ranking the pixels with respect to the gradients, the saliency maps generated from ModelB tend to be much more concentrated in the area of interest indicating better generalization. Several snake species are constrained by their geographical location. Metadata about where the image was taken was given in the form of Country and Continent. The distribution of images per country follows a long tail distribution and is concentrated mostly in the United States Of America (61.42%). Images were taken from 187 countries. In the absence of such information, \UNKNOWN" is marked. The probability of a species given a country is precomputed from the training dataset distribution as follows: p(s j c) = tsc=tc (1) p(s j c) = probability of a species, given a country. tsc = total number of images belonging to the species, s found in the country, c. tc = total number of images found in the country, c.
The generated, p(s j c), is provided at 1 along with the source code used for training. It is worth noting that the images marked with "UNKNOWN" data is considered as a country for the purposes of pre-computation. The nal probabilities are then adjusted as follows: ps = p s p(s j c) (2) ps = probability of the species, given the image and country. p s = probability of the species inferred from the model.
The probabilites are normalized to ensure a sum of 1. Using this technique on Model-B, the scores improved from 0.5813 to 0.6019.
The test dataset on which the nal scores were calculated, follows a data distribution similar to validation dataset and the model achieves an F1-score of 0.625 when tested on AICrowd platform. 7
Although the models were trained with the same hyper-parameters, Model-B
performs better than Model-A. These results signify the importance of having
generalized visual representations before ne-tuning is done on a domain-speci c
dataset. Label smoothing[
The compute resources required to perform the experiments was provided by hostkey.com 2 1 https://github:com/GokulEpiphany/snakes-round-4-train/tree/master/ metadata 2 https://www:hostkey:com/ 13. Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classi cation models and saliency maps, 2013. arXiv:1312:6034. 14. Yuxin Wu and Kaiming He. Group normalization, 2018. arXiv:1803:08494. 15. Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization, 2017. arXiv:1710:09412.