This paper presents methods that were utilized in the ImageCLEFcoral 2020 challenge. The challenge contains two following subtasks: automatic coral reef annotation and localization, and automatic coral reef image pixel-wise parsing. In the rst subtask, we tested two methods - SSD, and Mask R-CNN. In the second subtask, we tested only Mask R-CNN. Performance improvements were achieved by careful cleaning of the dataset and by both o ine and online data augmentations.
With changes in world climate in recent years, the danger of losing coral reefs and the ecosystem they support is increasing. Therefore, detailed monitoring of these ecosystems can be critical for their future. However, because of the complexity of coral images, they are very di cult for people to annotate, which opens possibilities for automatic detection.
Within the ImageCLEFcoral 2020 challenge [
For the rst subtask, we tested two detection methods, one single-shot detector - SSD [6] and one two-shot detector - Mask R-CNN [4]. In the second subtask, we utilized Mask R-CNN, because it also provides semantic parsing information.
The data for this task originates from a growing, large-scale collection of images taken from coral reefs around the world as part of a coral reef monitoring project with the Marine Technology Research Unit at the University of Essex. The dataset contains 440 images in total with ground-truth annotations for 13 coral classes.
The provided dataset is challenging from many di erent perspectives. First, each image contains a large number of di erent corals, to be more speci c 28 on average. Second, the large imbalance in the total number of instances among coral classes occurs. Third, there exists big intra-class variability in appearance and size, see Fig. 1.
Third, the quality of images is highly inconsistent and some images are very blurry. Last but not least, during a hand-made inspection, we revealed that approximately 120 images are rotated by 180 degrees with respect to the groundtruth annotations. In the rst step, we address the rotation problem mentioned above and rotate all the images to the correct orientation. In the next step, the database was split into two subsets - train, and validation set. Due to the big imbalance between classes, during the splitting, we focused to preserve class distribution across both of the sets. Our goal was split to split the dataset in the way, that the train set contains approximately 85% of the images and validation set the rest. The nal split with respect to the classes, after which the train set contains 371 images, and the validation set 69 images, can be found in Table 1. To enrich and expand our train set, we utilize data augmentations. Primary, standard augmentations common for computer vision tasks were used, to be more speci c, random horizontal ip, random vertical ip, random crop with resize, and Gaussian blur. Moreover, during the data analysis, we noticed, that two di erent light 'themes' are common - the blue one, and the green one. We simulate this e ect by using a color lter, see Fig 2.
We tested both o ine (before the training) and also online (during the training) augmentations, however, we reached better results using online augmentations during the experiments. A comparison of the in uence of distinct online data augmentation on mean average precision (mAP) on our validation set can be found in Table 2.
We tested two detection models - SSD [6], and Mask R-CNN [4]. Both models were pretrained on Pascal VOC 2007 dataset [3]. For both models, we used standard implementation in Keras [2].
All the images were resized to the resolution 1024 1024 pixels during the Mask R-CNN training and to 512 512 pixels during the SSD training. Both tested methods were trained with a batch size of 1 during 200 epochs using SGD optimizer with an initial learning rate l = 0:0001 and step decay d = 0:1 after 100 epochs. For the purpose of challenge, we choose the model with best mAP on the validation set. 4
We evaluate both trained models on the validation set. Exemplary results on the validation set can be found in Fig. 3. Mask R-CNN detects much more bounding boxes than SSD, however, the majority of them are false positives. To be more speci c, on the validation set, Mask R-CNN detects 2148 bounding boxes, but only 44.7% are true positives. On the other hand, SSD detects only 1029, but 71.3% are true positives.
Detailed comparison of average precision in the localization subtask for individual classes can be found in Table 3. It should be noted that neither of the models were able to learn to detect ve of the low frequent class. On the other hand, both models reach very good results for class Sponge barrel despite the fact its fth least frequent class. We argue this occurs because of its big dissimilarity from other classes.
Mean average precision (mAP) on our validation set and the challenge test set can be found in Table 4. In the localization subtask, SSD overcomes Mask RCNN. Both models surprisingly over-performed on the test set by a large margin. Unfortunately, due to the test set access limitation, we can only guess, why this phenomenon happened. In this paper, we present working notes from ImageCLEFcoral 2020 challenge. We employed two detection methods, both of them based on neural networks, SSD, and Mask R-CNN. In the localization subtask, SSD reached better mAP by a large margin. Despite the fact reached results were mediocre, we believe that with more advanced state-of-the-art detection methods, it is possible to reach satisfactory results. Other future improvements we see in an expansion of the training set, or utilizing knowledge distillation approach.
Acknowledgement: The work has been supported by the grant of the University of West Bohemia, project No. SGS-2019-027. Moreover, access to computing and storage facilities owned by parties and projects contributing to the National Grid Infrastructure MetaCentrum provided under the programme "Projects of Large Research, Development, and Innovations Infrastructures" (CESNET LM2015042), is greatly appreciated. image annotation. In: CLEF2020 Working Notes. CEUR Workshop Proceedings, CEUR-WS.org (2020) 2. Chollet, F., et al.: Keras. https://keras.io (2015) 3. Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results. http://www.pascalnetwork.org/challenges/VOC/voc2007/workshop/index.html 4. He, K., Gkioxari, G., Dollar, P., Girshick, R.: Mask r-cnn. In: Proceedings of the
IEEE international conference on computer vision. pp. 2961{2969 (2017) 5. Ionescu, B., Muller, H., Peteri, R., Abacha, A.B., Datla, V., Hasan, S.A., DemnerFushman, D., Kozlovski, S., Liauchuk, V., Cid, Y.D., Kovalev, V., Pelka, O., Friedrich, C.M., de Herrera, A.G.S., Ninh, V.T., Le, T.K., Zhou, L., Piras, L., Riegler, M., l Halvorsen, P., Tran, M.T., Lux, M., Gurrin, C., Dang-Nguyen, D.T., Chamberlain, J., Clark, A., Campello, A., Fichou, D., Berari, R., Brie, P., Dogariu, M., Stefan, L.D., Constantin, M.G.: Overview of the ImageCLEF 2020: Multimedia retrieval in lifelogging, medical, nature, and internet applications. In: Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings of the 11th International Conference of the CLEF Association (CLEF 2020), vol. 12260. LNCS Lecture Notes in Computer Science, Springer, Thessaloniki, Greece (September 2225 2020) 6. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.C.: Ssd: Single shot multibox detector. In: European conference on computer vision. pp. 21{37. Springer (2016)