-5pt Endoscopic imaging is largely used as the diagnostic tool for Colon polyps-induced GI tract cancer. This diagnosis via image identification requires expertise that may be lacking in inexperienced physicians. Hence, using a software-aided approach to detect those anomalies may better identify the tissue abnormalities. In this paper, a novel deep learning network 'XP-Net' with Efective Pyramidal Squeeze Attention (EPSA) module using hierarchical adversarial knowledge distillation by a combination of two teacher networks is proposed. It adds 'complementary knowledge' to the student network- thus aiding in the improvement of network performance. The lightweight EPSA block enhances the current network architecture by capturing multi-scale spatial information of objects at a granular level with long-range channel dependency. The XP-Net compiled into the NVIDIA TensorRT engine gave a better real-time performance in terms of throughput. The proposed network has achieved a dice score of 0.839 and IoU of 0.805 in the validation data set, and it was able to attain an average throughput of 60 fps in mobile GPU. This proposed deep learning-based segmentation approach is expected to aid clinicians in addressing the complications involved in the identification and removal of precancerous anomalies more competently.
Colorectal polyps are one of the early indicators of lower
Gastro-Intestinal (GI) tract cancer. These polyps are extra
growth lumps of tissues, having no particular function
in the bodily processes [
In summary, the contributions in this paper are as and student is denoted by T(1,2)y and Syˆ. In
follows: hierarchical knowledge distillation, the student loss is
denoted by Ls which consists of weighted combination
of two terms, (a) the sum of dice [
The overall student loss is given in equation 2.
• A hierarchical dual teacher knowledge distillation
network to transfer the complementary
knowledge of both networks to a student.
• A student network with a lower computational
cost for real-time performance without
significantly reducing accuracy.
• Experiments: By evaluating our model’s
generality in the external Kvasir-Seg dataset [
In our methodology, a student network is derived from
two teacher networks through a hierarchical
knowledge distillation process. The two teacher networks that
are highly computational, transfer their complementary
knowledge to a lightweight student network. The
baseline U-Net architecture has the ability to capture features
at multiple scales. To enhance this visual perception of
the U-Net network, we implemented an Efective
Pyramidal Squeeze Attention (EPSA) block at the first encoder
of the U-Net.This attention mechanism boosts the
allocation of the most informative feature expressions while
suppressing the less useful ones, allowing the model to
focus on clinically crucial areas [
* [(, ˆ, ˆ), 1]
LS-GAN loss denoted as L. The L is made up
of two mean square error term. one term is between
the HD output after being passed a "fake" datasample
from the teacher, and a tensor of all zeros [
+ [(, (1,2)ˆ, (1,2)ˆ, 1]
2.2.1. Teacher Network CABR32-P-CBR64-P-CBR128-P-CBR256-PCBR512-UPCONV256-CBR256-UPCONV128(1) (2) (3) (a) Teacher and Student Discriminator Network (b) Teacher Network Blocks (c) Student Network Blocks
2.2.2. Student Network CAPDBR32-P-CPDBR64-P-CPDBR128-PCPDBR256-P-CPDBR512-UPCONV256CPDBR256-UPCONV128-CPDBR128-UPCONV64CPRBR64-UPCONV32-CPDBR32-CE1 • CPDBRK
CPDBRK represents stack of (A) point wise convolution of kernel size (1,5) and depth wise convolution of kernel size (1,1) followed by Batch norm and Relu and (B) point wise convolution of kernel size (5,1) and depth wise convolution of kernel size (1,1) followed by Batch norm and Relu. All the convolution layers consists of K number of feature outputs. • CAPDBRK 2.2.3. Discriminator Network CAT-DC32-CAT-DC128-CAT-DC128-CAT-DC32CAT-DC32-ENCONV • CAT • DCK
CAT is the concatenation of two diferent layers either from teacher or student network.
inator blocks (DC) and an End Convolution (ENCONV). In our proposed model, the feature map from encoder 1, encoder 3, decoder 1, decoder 3 from the teacher or student network are used for hierarchical knowledge distillation. The full network architecture is described in Fig.2.
the availability of big datasets of polyp images or videos
along with high-quality, manual annotations provided
by experts. These annotations provide the ground truth
necessary to train the supervised deep learning models.
EndoCV2022 challenge provided us with series of
sequence dataset of 2631 images with their corresponding
ground truth masks [
Both the teacher network is trained using Adam optimizer with initial learning rate of 3e− 4 with step learning rate scheduler of gamma 0.1 and step size of 30. The networks were trained for 450 epoch with batch size of 8. The student network was trained using Adam optimizer with 1 0.5 and 2 0.999 with a initial learning rate of 1e− 4 with step learning rate scheduler of gamma 0.1 and step size of 30. After multiple experiments of initializing weights with uniform, xavier-uniform and kaiming-uniform given in pytorch weight initialization, it showed that kaiming uniform weight initialization have helped for better convergence of model. We also implemented our model in Nvidia TensorRT inference library for efective realtime model throughput. All the models were trained using Nvidia RTX 3090 GPU.
The networks were evaluated and the computed metrics are reported in Table.1. In the validation data of EndoCV dataset, the Teacher 1 model was able to achieve 0.893 and 0.889 for Dice score and IoU score, respectively. Similarly, the teacher 2 model was able to achieve 0.871 and 0.884 for the same metrics. The student network has achieved a commendable dice and IoU score of 0.839 and 0.805 even with the reduced number of learnable parameters. The trade-of here is the larger sized teacher network for a minimal loss in the accuracy of the light weight student network. Similarly, these metrics were calculated for Kvasir-Seg dataset and is reported in the Table 1.
Results have shown that the teacher 2 perform better for region with higher amount of specular reflection than teacher 1 for those regions. The student network thus obtains the complimentary knowledge from the two teacher networks. With reference to the ground truth, it is observed that the student network had proper segmentation even though one of the teachers had missed areas in its segmentation masks as shown in Fig 3. These results show that multiple teacher knowledge helps to generalize better segmentation.
As a part of benchmarking the network in terms of inference time, the model was converted into TensorRT engine for faster throughput. The model was able to attain an average throughput of 60 fps on GeForce RTX 3070 mobile GPU and 120 fps in Nvidia RTX 3090 GPU. From the results, we believe that constructing multiple teacher models which focuses on various aspects of the input data can distill a superior student network.
The proposed network is light weight and does faster computation when compared with traditional networks that are used for segmentation. Since this uses dual teachers for knowledge distillation, by increasing the number of teacher networks, there is room for further improvement in performance. Moreover, the sample size of data also plays a crucial role in the accuracy of the network. Further studies can be done to design a much more intelligent network for polyps and other varieties of early cancer tissues.