Automated Polyp Segmentation in Colonoscopy using MSRFNet Saurab Rauniyar1 , Abhishek Srivastava2 , Vabesh Kumar Jha3 , Ritika Kumari Jha4 , Debesh Jha5 , & Ashish Rauniyar6 1 Vyobotics, Singapore 2 Indian Statistical Institute, India 3 Khwopa College of Engineering, Nepal 4 Care Medical Center, Nepal 5 UiT - The Arctic University of Norway, Norway 6 SINTEF Digital, Norway ashish.rauniyar@sintef.no ABSTRACT (1) The “MediaEval 2021" challenge entails the chance to study Colorectal cancer is one of the major cause of cancer-related death polyp segmentation and develop accurate automated polyp around the world. High-quality colonoscopy is considered manda- segmentation algorithms. Therefore, in this paper, we present tory for resecting and preventing colorectal cancers. In the recent our approach based on “MSRFNet" for automatic polyp seg- past, various technological advances have been made towards im- mentation. proving the quality of colonoscopy. Despite the technical advance- (2) In addition to the challenge, we evaluated our method on ment, some polyps are frequently missed during colonoscopy exam- the Endotect Challenge Dataset1 and 2020 Medico Auto- inations. Polyp detection ( for example, adenomas) rates are largely matic Polyp Segmentation Challenge Dataset2 to demon- influenced by inter-endoscopist variability. Therefore, it is very strate the efficiency of our approach. challenging to standardize a high-quality colonoscopy. A computer- aided detection system could solve the problem with miss-detection. 2 RELATED WORKS The “MediaEval 2021” challenge entails the chance to study and Automated polyp segmentation is a well-established topic. There develop accurate automated polyp segmentation algorithms [6]. has been several works proposed on the automated polyp seg- In this paper, we propose our approach based on MSRFNet. Our mentation [1, 4, 9, 16]. In addition to the individual work, there are experimental findings show that the model trained on the Kvasir- several competitions and challenges held in order to solve the polyp SEG dataset and evaluated on a competition test dataset obtains a segmentation problem [1, 2, 5, 7, 8]. Paudel et al.[10] provided a dice coefficient of 0.7055, Jaccard of 0.6176, a recall of 0.7293, and winning solution for the 2020 “Medico automatic polyp segmen- a precision of 0.7769. In addition to the MediaEval 2021 challenge, tation challenge”. They used efficientNet [14] as an encoder and we evaluated our approach on the Endotect Challenge Dataset and UNet [11] as decoder. Additionally, they also used a channel-spatial “2020 Medico Automatic Polyp Segmentation Challenge Dataset". attention module and deep supervision to improve the performance The results further demonstrate the efficiency of our approach. of the network. Similarly, Thambawita et al. [15] provided another solution for the segmentation task at the 3rd International En- 1 INTRODUCTION doscopy Computer Vision Challenge and Workshop (EndoCV2021). The proposed architecture, DivergentNets, combined TriUNet with Colorectal cancer is the third leading cause of cancer-related death UNet++, FPN, DeepLabv3, and DeepLabv3+, into a single model to globally [12]. Although colonoscopy has improved the detection of achieve generalizable performance. “Medico: Transparency in Medi- colorectal polyps, computer-aided detection could better indicate cal Image Segmentation" challenge aims to develop transparent and the presence and location of polyps in the colon. A CAD system explainable automated methods. We participate in this challenge to could assist endoscopists by finding out the missed polyps. One of provide our effective solution and benchmark our solutions against the other significant advantages of the CAD system is that they other participants on the same test dataset. are not influenced human bias or inter and intraobserver variabil- ity. Therefore, such systems could improve clinical performance irrespective of gastroenterologists expertise. In this respect, we 3 DATASET propose our approach based on MSRFNet [13], which was specially HyperKvasir [3] was provided by the challenge organizers as the designed for the segmentation of medical images. development dataset. Kvasir-SEG consists of 1000 images, their In summary, the main contribution of the paper are as follows: corresponding ground truth and the bounding box information. As the test dataset, the organizers provided us with 200 unique images Copyright 2021 for this paper by its authors. Use permitted under Creative Commons consisting of at least one polyp image. Similarly, we performed License Attribution 4.0 International (CC BY 4.0). MediaEval’21, 13-15 December 2021, Online further experiments on the Endotect Challenge Dataset and Medico Automatic Polyp Segmentation Challenge Dataset. 1 https://endotect.com/ 2 https://multimediaeval.github.io/editions/2020/tasks/medico/ MediaEval’21, December 13-15 2021, Online S. Rauniyar et. al. Figure 2: Qualitative results of the proposed solution Table 1: Results of our polyp segmentation method on the test set provided by MediaEval 2021 challenge Dataset Dice Jaccard Recall Prec. Development 0.9217 0.8914 0.9198 0.9666 Test 0.7055 0.6176 0.7293 0.7769 Table 2: Results of our polyp segmentation method on the test set provided by Endotect & 2020 Medico Automatic Polyp Segmentation Challenge Training Data Testing data Dice Jaccard Recall Prec. Endotect Kvasir-SEG Challenge 0.8131 0.6967 0.8581 0.7874 2020 Medico Kvasir-SEG Challenge 0.7575 0.6337 0.7197 0.8414 qualitative results, it can be observed that our approach is able to detect the obvious polyps, including small polyps, however, it fails Figure 1: Components of MSRFNet, a) Dual-scale dense fu- in challenging cases. Due to the unavailability of the ground truth, sion(DSDF) block and b) multi-scale residual fusion (MSRF). we can only present the original images and the predicted masks. Dotted rectangle block in (b) represents multi-scale feature From the Table 2, we can observe that our model obtains de- exchange in MSRFNet [13]. scent performance for both datasets that further demonstrates the efficiency of our approach. 4 METHODOLOGY 6 CONCLUSION & FUTURE WORK MSRFNet architecture and its components are shown in Figure. 1. We competed on the organizer’s dataset using MSRFNet architec- MSRFNet has a dual-scale dense fusion (DSDF) block that consists of ture and achieved a dice coefficient of 0.7055, Jaccard index of 0.6176, residual dense connections and is capable of transferring data across a recall of 0.7293, and a precision of 0.7769. In the polyp segmenta- different scales. It is a fully convolutional network that computes tion challenge task, the MSRFNet performed well, as depicted by the multi-scale features and fuses them effectively using a DSDF different performance metrics. We further evaluated our results on block. The residual nature of the DSDF block improves gradient flow the Endotect Challenge Dataset and 2020 Medico Automatic Polyp which improves the training efficiency. Due to page limitations, for Segmentation Challenge Dataset that demonstrated the efficiency more details on the working, components and architecture details of our approach. of MSRFNet, readers are requested to refer [13]. In the future, we want to enhance the MSRFNet performance design by assessing the best hyperparameter settings for the auto- 5 RESULTS AND ANALYSIS matic polyp segmentation. It can be observed from Table 1 that our trained MSRFNet model achieved a dice coefficient of 0.7055, Jaccard index of 0.6167, and ACKNOWLEDGMENTS a precision of 0.7769 on the MediaEval organiser’s test dataset, The computations in this paper were performed on the equipment manifesting MSRFNet generalization capabilities. Similarly, the provided by the Experimental Infrastructure for Exploration of qualitative results are demonstrated in Figure2. The qualitative Exascale Computing (eX3), which is financially supported by the results show both obvious polyps and difficult polyps. From the Research Council of Norway under the contract 270053. Medico: Transparency in Medical Image Segmentation MediaEval’21, December 13-15 2021, Online REFERENCES evaluation and comparison of classification, segmentation and in- [1] Sharib Ali, Felix Zhou, Barbara Braden, Adam Bailey, Suhui Yang, ference time for endoscopy. In International Conference on Pattern Guanju Cheng, Pengyi Zhang, Xiaoqiong Li, Maxime Kayser, Roger D Recognition. 263–274. Soberanis-Mukul, and others. 2020. An objective comparison of detec- [8] Debesh Jha, Steven A Hicks, Krister Emanuelsen, Håvard Johansen, tion and segmentation algorithms for artefacts in clinical endoscopy. Dag Johansen, Thomas de Lange, Michael A Riegler, and Pål Halvorsen. Scientific reports 10, 1 (2020), 1–15. 2020. Medico multimedia task at mediaeval 2020: Automatic polyp [2] Jorge Bernal, Nima Tajkbaksh, Francisco Javier Sanchez, Bogdan J segmentation. arXiv preprint arXiv:2012.15244 (2020). Matuszewski, Hao Chen, Lequan Yu, Quentin Angermann, Olivier [9] Ge-Peng Ji, Yu-Cheng Chou, Deng-Ping Fan, Geng Chen, Huazhu Romain, Bjørn Rustad, Ilangko Balasingham, and others. 2017. Com- Fu, Debesh Jha, and Ling Shao. 2021. Progressively Normalized Self- parative validation of polyp detection methods in video colonoscopy: Attention Network for Video Polyp Segmentation. In In proceedings of results from the MICCAI 2015 endoscopic vision challenge. IEEE Medical image computing and computer assisted intervention. transactions on medical imaging 36, 6 (2017), 1231–1249. [10] Sahadev Poudel and Sang-Woong Lee. 2020. Automatic Polyp Seg- [3] Hanna Borgli, Vajira Thambawita, Pia H Smedsrud, Steven Hicks, mentation Using Channel-Spatial Attention with Deep Supervision. Debesh Jha, Sigrun L Eskeland, Kristin Ranheim Randel, Konstantin (2020). Pogorelov, Mathias Lux, Duc Tien Dang Nguyen, Dag Johansen, [11] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Carsten Griwodz, Håkon K Stensland, Enrique Garcia-Ceja, Peter T Convolutional networks for biomedical image segmentation. In Inter- Schmidt, Hugo L Hammer, Michael A Riegler, Pål Halvorsen, and national Conference on Medical image computing and computer-assisted Thomas de Lange. 2020. HyperKvasir, a comprehensive multi-class intervention. 234–241. image and video dataset for gastrointestinal endoscopy. Scientific Data [12] Juan Silva, Aymeric Histace, Olivier Romain, Xavier Dray, and 7, 1 (2020), 283. https://doi.org/10.1038/s41597-020-00622-y Bertrand Granado. 2014. Toward embedded detection of polyps in wce [4] Yun Bo Guo and Bogdan Matuszewski. 2019. Giana polyp segmenta- images for early diagnosis of colorectal cancer. International journal tion with fully convolutional dilation neural networks. In Proceedings of computer assisted radiology and surgery 9, 2 (2014), 283–293. of the 14th International Joint Conference on Computer Vision, Imaging [13] Abhishek Srivastava, Debesh Jha, Sukalpa Chanda, Umapada Pal, Hå- and Computer Graphics Theory and Applications. 632–641. vard D Johansen, Dag Johansen, Michael A Riegler, Sharib Ali, and Pål [5] Steven Hicks, Debesh Jha, Vajira Thambawita, Pål Halvorsen, Bjørn- Halvorsen. 2021. MSRF-Net: A Multi-Scale Residual Fusion Network Jostein Singstad, Sachin Gaur, Klas Pettersen, Morten Goodwin, Sra- for Biomedical Image Segmentation. arXiv preprint arXiv:2105.07451 (2021). vanthi Parasa, Thomas de Lange, and others. 2021. MedAI: Trans- [14] Mingxing Tan and Quoc Le. 2019. Efficientnet: Rethinking model parency in Medical Image Segmentation. Nordic Machine Intelligence scaling for convolutional neural networks. In International Conference 1, 1 (2021), 1–4. on Machine Learning. 6105–6114. [6] Steven Hicks, Debesh Jha, Vajira Thambawita, Hugo Hammer, Thomas [15] Vajira Thambawita, Steven A Hicks, Pål Halvorsen, and Michael A de Lange, Sravanthi Parasa, Michael Riegler, and Pål Halvorsen. 2021. Riegler. 2021. Divergentnets: Medical image segmentation by network Medico Multimedia Task at MediaEval 2021: Transparency in Medical ensemble. arXiv preprint arXiv:2107.00283 (2021). Image Segmentation. In Proceedings of MediaEval 2021 CEUR Work- [16] Zongwei Zhou, Md Mahfuzur Rahman Siddiquee, Nima Tajbakhsh, shop. and Jianming Liang. 2019. Unet++: Redesigning skip connections to [7] Steven A Hicks, Debesh Jha, Vajira Thambawita, Pål Halvorsen, Hugo L exploit multiscale features in image segmentation. IEEE transactions Hammer, and Michael A Riegler. 2021. The EndoTect 2020 challenge: on medical imaging 39, 6 (2019), 1856–1867.