Identification and removal of polyps during colonoscopy have become a standard procedure. It is often challenging to detect polyps, as they are often hard to differentiate from surrounding normal tissue. These polyps are usually covered with stool, mucosa, and other materials that can obscure the correct diagnosis. This is especially true for the small, flat, and sessile polyps that are typically not visible during colonoscopy. Moreover, this increases the miss-rate of polyps up-to 25% [8] and increases the risk of colorectal cancer in the affected patient. An increase in the 1% adenoma detection rate leads to a 3% decrease in the risk of colorectal cancer [3]. Recently, deep learning techniques have been developed to overcome these challenges and improve polyp detection accuracy during colonoscopy. Polyp segmentation based deep learning methods has been successfully applied for automatic polyp detection in a real-time.
The automatic polyp segmentation plays an important role in the identification and localization of the polyps in the affected regions. It helps in analyzing the images or even video frames and classify each pixel into polyp or non-polyp class instances. This allows the clinician in easy, fast, and more accurate identification of the polyp in the affected region. The automated polyp segmentation can help in the development of a Computer-Aided Diagnosis (CADx) system, which is specially designed for colonoscopy procedures.
The "Medico Automatic Polyp Segmentation Challenge" [6] consists of two tasks. The first task is "Polyp segmentation task" and the second is "Algorithm efficiency task". We have submitted our model in task 1 only.
For semantic segmentation task, encoder-decoder networks like FCN [9], U-Net [10], etc are mostly preferred over other approaches. U-Net and its variants are used for both natural image segmentation and biomedical image segmentation. In general, the encoder uses multiple convolutions to learn and capture the essential semantic features ranging from low-level to high-level. These upscaled features are then concatenated with the features from the encoder using the skip connections and then followed by convolution layers to generate the final output in the form of a binary mask.
The encoder acts as a feature extractor, where the decoder uses features extracted from the input to produce to desired segmentation mask. The encoder can be replaced by a pre-trained network such as VGG16 [12], VGG19 [12], etc. These pre-trained networks are already trained on the ImageNet [11] dataset and have the necessary feature extraction capabilities. Architectures like SegNet [2] and TernausNet [5] use pre-trained VGG16 and VGG11 respectively for segmentation task.
With the success of the residual network [4], ResNet50 is one of the commonly used architecture for any transfer learning task. The residual network uses two 3 × 3 convolutional layers and an identity mapping. Each convolution layer is followed by a batch normalization layer and a Rectified Linear Unit (ReLU) activation function. The identity mapping is the shortcut connection connecting the input and output of the convolutional layer. The identity mapping helps in building a deeper neural network by eliminating the problem of vanishing gradients and exploding gradients. [11] is used . The use of a pre-trained encoder helps the model to converge easily. The input image is fed into the pre-trained ResNet50 encoder, consisting of a series of residual blocks as their main component. These residual blocks help the encoder extract the important features from the input image, which are then passed to the decoder. The decoder starts a transpose convolution that upscales the incoming feature maps into the desired shape. Next, these upscaled feature maps are concatenated with the specific shape feature maps from the pre-trained encoder via skip connections. These skip connections help the model to get all the low-level semantic information from the encoder, which allows the decoder to generate the desired feature maps. After that, it is followed by the two 3 × 3 convolution layer, where each layer is followed by a batch normalization layer and a ReLU non-linearity. The last decoder block's output is passed to a 1×1 convolution layer, which is further passed to a sigmoid activation function, finally generating the desired binary mask.
The FastAI (version 2.0) library [1] is used to train and evaluate our model. We have employed resizing, flipping, rotating, zooming, lightning, warping, and normalizing intensity based on the ImageNet dataset to augment the input images for training. The model uses Adam optimizer with an initial learning rate of 10 −2 , and cross-entropy loss as its loss function. We have employed the one-cycle policy where the learning rate changes during training and achieves super-convergence [13]. We have run just 50 epochs for training, and the model has converged.
The Medico Automatic Polyp Segmentation challenge [6] provides an opportunity to study the potential and challenges of automated polyp segmentation. This study aims at building a model that performs well on the organizer's dataset while training on a separate Kvasir-SEG dataset [7].
Table 1 shows the overall results of the U-Net-ResNet50 architecture on the Kvasir-SEG test dataset and the organizer's test dataset provided for the final evaluation of the model. For the evaluation of the model, the Jaccard index, Sørensen-Dice coefficient (DSC), recall, precision (Prec.), accuracy (Acc.), and the F2 are used as the evaluation metrics. Our trained U-Net-ResNet50 model achieved
With our U-Net-ResNet50, we achieved competitive performance on the organizer's dataset with a dice coefficient of 0.8154. By replacing the U-Net encoder with a pre-trained ResNet50 and employing a one-cycle policy during training, we are able to converge the model in a short time. Thus, it helps in reducing the training time as the encoder weights are not initialized from scratch. This is an important step towards faster convergence, which would be useful when the availability of high-performance computing resources is limited.
In the future, we would like to experiment with more than one pre-trained encoder by fusing their feature maps and using them for training our model.
The computations in this paper were performed on the equipment provided by the Experimental Infrastructure for Exploration of Exascale Computing (eX3), which is financially supported by the Research Council of Norway under the contract 270053.
The authors would also like to thank the machine learning group of Mohn Medical Imaging and Visualization (MMIV) Centre, Norway, for providing the computing infrastructure for the experiments.