Endoscopic artefact detection challenge consists of 1) Artefact detection, 2) Semantic segmentation, and 3) Out-ofsample generalisation. For Semantic segmentation task, we propose a multi-plateau ensemble of FPN[1] (Feature Pyramid Network) with EfficientNet[2] as feature extractor/encoder. For Object detection task, we used a three model ensemble of RetinaNet[3] with Resnet50[4] Backbone and FasterRCNN[5] (FPN + DC5[6]) with Resnext101 Backbone[7, 8]. A PyTorch implementation to our approach to the problem is available at github.com/ubamba98/EAD2020. Index Terms- Endoscopy, FPN, EfficientNet, RetinaNet, Faster RCNN, Artefact detection.
The given dataset of EndoCV-2020 [
Due to the variable aspect ratios and sizes in the training data, we adopted a stage-dependent rescaling policy. During the training stage, we cropped the images to a fixed size of 512x512 without resizing. This made sure that the spatial information was not lost and at the same time, the input to models was within trainable limits. During validation and testing time, we padded the images such that both dimensions are a multiple of 128 which is required for the EfficientNet backbone (to handle max-pooling in deeper models). As
Copyright c 2020 for this paper by its authors. Use permitted under
Creative Commons License Attribution 4.0 International (CC BY 4.0).
the number of samples in the dataset were relatively low and
unbalanced, various augmentation techniques were adopted
to prevent overfitting and achieve generalization. Horizontal
and vertical flip, cutout (random holes)[
Algorithm 1 CutMix for Segmentation for each iteration do input, target = get minibatch(dataset) if mode == training then input s, target s = shuffle minibatch(input, target) lambda = Unif(0,1) r x = Unif(0,W) r y = Unif(0,H) r w = Sqrt(1 - lambda) r h = Sqrt(1 - lambda) x1 = Round(Clip(r x - r w / 2, min=0)) x2 = Round(Clip(r x + r w / 2, max=W)) y1 = Round(Clip(r y - r h / 2, min=0)) y2 = Round(Clip(r y + r h / 2, min=H)) input[:, :, x1:x2, y1:y2] = input s[:, :, x1:x2, y1:y2] target[:, :, x1:x2, y1:y2] = target s[:, :, x1:x2, y1:y2] end if output = model forward(input) loss = compute loss(output, target) model update() end for
For Object detection spatial transformations - flip and random scaling and rotating were used.
Due to high variability and early overfitting nature in the
dataset, the main focus was on making a strong ensemble by
training models on different optimisation plateaus. A total
of 8 different plateaus with permutations of two different
optimisers and four different loss functions were optimised
with EfficientNet backbone increasing the depth, width and
resolution three times, going from B3 to B5 (Table 1). For
optimisers, Ranger and Over9000 were used. Ranger is a
synergistic optimiser combining RAdam (rectified Adam)[
Complete segmentation training pipeline was divided into four stages Stage 1 - CutMix was disabled to reduce regularization effect, encoder was loaded with ImageNet weights and freezed for the decoder to learn spatial features without being stuck into saddle point, crops were taken with at least one pixel having a positive mask.
Stage 2 - CutMix was enabled for strong regularization, and encoder was unfreezed to learn spatial features of endoscopic images.
Stage 3 - Random crops were trained instead of non-empty crops for the model to learn negative samples.
Stage 4 - Very few epochs with CutMix disabled and encoder freezed for generalization on original data.
For every consecutive stage best checkpoint of the previous stage was loaded.
After analysing predictions on holdout, it was found that the number of false positives was quite high. To counter this, we implemented a novel post-processing algorithm which specifically reduced the number of false positives in the predictions (Algorithm 2). Three sets of thresholds - max prob thresh, min prob thresh, min area thresh were tuned for this given task.
max prob thresh and min prob thresh were tuned using grid search on holdout dataset, whereas min area thresh was calculated by sorting sum of positive pixels of every class and taking the 2:5th percent respectively. min area thresh used after calculation were 2000, 128, 256, 256 and 1024 respectively for every class. The results for triple threshold on single best model are compiled in Table 3 and comparison of our best performing models in Table 4.
iou thresh score thresh weights
Algorithm 2 Triple Threshold for each sample do output masks = model(each sample) final masks = [] i = 0 for each output mask in output masks do max mask = output mask > max prob thresh if max mask.sum() < min area thresh[i] then output mask = zeros(output mask.shape) else
output mask = output mask > min prob thresh end if i = i + 1 final masks.append(output mask) end for end for
For Object Detection, individual models were trained with SGD as optimizer and confidence threshold of 0.5. To counter the variance and improve the performance of our model predictions, general ensembling was performed. Retinanet with backbones of FPN and Resnet50 and Faster RCNN with backbones of FPN and Resnext 101 32xd were trained (Table 5). Our ensemble strategy involves finding overlapping boxes of the same class and average their positions while adding their confidences. For finding the best parameters for ensembling the three models predictions, we ran a grid search with all possible combinations of the given range of values (Table 2).
We achieved a Segmentation score which was a weighted linear combination of dice, IOU and F2 of 0.5675 on the final leader board and for object detection task, an mAP of 0.2061 was obtained.
Gastric cancer accounts for around 1 million deaths each year which can be prevented by early diagnosis. In this paper we explored multi-plateau ensemble to generalize pixel level segmentation and localization of artefacts in endoscopic images. We developed novel augmentation and post-processing algorithms for better and robust model convergence.
[2] Mingxing Tan and Quoc V. Le. Efficientnet: Rethinking model scaling for convolutional neural networks, 2019. [3] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollr. Focal loss for dense object detection, 2017. [4] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015. Seiryo Watanabe, Ilkay Oksuz, Qingtian Ning, Shufan Yang, Mohammad Azam Khan, Xiaohong W. Gao, Stefano Realdon, Maxim Loshchenov, Julia A. Schnabel, James E. East, Geroges Wagnieres, Victor B. Loschenov, Enrico Grisan, Christian Daul, Walter Blondel, and Jens Rittscher. An objective comparison of detection and segmentation algorithms for artefacts in clinical endoscopy. Scientific Reports, 10, 2020.