SEMANTIC SEGMENTATION, DETECTION AND LOCALIZATION OF MUCOSAL LESIONS FROM GASTROINTESTINAL ENDOSCOPIC IMAGES USING SUMNET Velmurugan Balasubramanian 4 5 Rajiv Kumar 2 5 Sarasa Jyothsna Kamireddi 3 5 Rachana Sathish 1 5 Debdoot Sheet 0 5 Debdoot Sheet, Centre of excellence in AI, Department of Electrical Engineering , IIT, Kharagpur , India Rachana Sathish, Department of Electrical Engineering , IIT, Kharagpur , India Rajiv Kumar, Department of Chemical Engineering , IIT, Kharagpur , India Sarasa Jyothsna Kamireddi, Department of Electrical Engineering , IIT, Kharagpur , India Velmurugan Balasubramanian, School of medical science and Technology , IIT, Kharagpur , India [2] Sharib Ali, Noha Ghatwary , Barbara Braden, Dominique Lamarque, Adam Bailey, Stefano Realdon, Renato Can- 2019 1. METHOD

We trained a fully convolutional neural network based on SUMNet [1] architecture, described in Fig.1, using Pytorch, for segmentation, detection and localization of lesions in Gastrointestinal endoscopic images using 386 images from EDD 2020 dataset [2]. An 80:20 training-validation split was followed with additional weights given to the under-represented classes depending upon their overall frequency of occurrence. We augmented the dataset with rotation, affine, scaling, projective and multi-crop transformations to accommodate for the variations caused due to scope positioning and augmented with variable brightness and HSV values to accommodate for images enhanced with narrow-band imaging and variable lighting conditions [ 3 ] [ 4 ]. We used the ADAM learning rate optimizer and binary cross-entropy loss function for training. SUMNet features (i) an encoder-decoder architecture with the pooling indices of encoder being passed to the corresponding decoder upsampling layers, (ii) encoder having a VGG11 like architecture pre-initialized with ImageNet pretrained weights and (iii) concatenation of activations of the encoder with that of the decoder, combining the features of segmentation networks for natural and biomedical images [1].

2. RESULT

Our model was able to obtain dice coefficients of 0.977, 0.974, 0.986, 0.987, 0.961 and 0.545, 0.219, 0.172, 0.339, 0.573 on the training and validation sets for Barretts oesophagus, suspicious, high-grade dysplasia, cancer and polyp classes respectively. A class-wise distribution of the abnormalities detected in the test dataset is shown in Table 1 and

Copyright c 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

Fig. 1: SUMNet architecture.

 Class Name

Barrett’s Esophagus Suspicious High grade dysplasia Cancer Polyp

 No of instances

21 6 9 1 30 Fig. 2 shows examples of the semantic masks and the bounding boxes we obtained for each of the five classes. We were able to obtain a semantic segmentation score of 0.538 with a standard deviation of 0.35 in our test submission and a mean detection score of 0.16 with a standard deviation of 0.074.

3. REFERENCES

nizzaro, Jens Rittscher, Christian Daul, and James East. Endoscopy disease detection challenge 2020. arXiv preprint arXiv:2003.03376, February 2020.

 [3] Georg Wimmer , Andreas Uhl, and Andreas Vecsei . Evaluation of domain specific data augmentation techniques for the classification of celiac disease using endoscopic imagery . In 2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP) , pages 1 - 6 . IEEE, 2017 . [4] Andrea Asperti and Claudio Mastronardo . The effectiveness of data augmentation for detection of gastrointestinal diseases from endoscopical images . arXiv preprint arXiv:1712.03689 , 2017 .