Cancer is one of the leading cause of death across the globe and projection of radiation towards tumor is the standard treatment of cancer. The first step of irradiation is to delineate tumor from organs near to the tumor. Unlike previous methods of CT segmentation, this paper proposed a procedure for segmentation of organs individually in CT images of the thoracic region such as Heart, Aorta, Trachea and Esophagus and merging them to form multi organ Segmented image. The aim of this method is to avoid coarse output from dilated UNets. The overlapping issues has addressed by calculating the mode of eight neighborhood of pixels. The performance of the proposed technique tested on 60 CT scans collected from SegTHOR Challenge. Dice ratio and Hausdorff distance have used in evaluation paradigm.
Index Terms— Hausdorff distance, dilated U-nets, eight neighborhoods, Coarse output, CT Segmentation.
1. INTRODUCTION
Cancer is the second leading cause of the death across the globe and projection of radiation is a crucial step for treatment of esophageal and Lung cancer. Radiation dose has to be given with great precision because it is a way of delivering packets of high energy (in general X-rays) to kill cancer and reduce side effects. Success rate of radiation therapy, determined by the cell growth of tumor-effected organs and the organs near to the tumor-effected organs (called as organs at risk) before and after the treatment. Radiation does not kill a cell, it destroys the connection between DNA and cell. This leads to abruption in the process of tumor cell division, called as abortive mitosis and sometimes this can happen to organs near the tumor. Therefore, delineation of organs need to done carefully and accurately. Radiation therapy is a treatment of choice for Lung and Esophageal cancer. The irradiation on organs begins with segmentation of target tumor and the organs near the tumor in computed tomography (CT-scans) images.
In general, experts do segmentation manually by intensity levels and anatomical knowledge e.g. Esophagus is located behind Heart, Trachea is above the Spinal cord, etc. The manual process is costly, time consuming and tedious. This leads to evolution of techniques for automatic segmentation of organs to assist the doctor.
Automatic Segmentation of organs is quite challenging and achieving higher accuracy is very difficult due to several factors such as acquiring volumetric data, low contrast images, variable size of organs from patient to patient, similarity between the shapes of organs and overfitting towards organs with high intensity or betterstructured organs.
Recent trends towards development of deep learning
architectures is performing quite well as compared to the
traditional methods, especially working with large volumes
of data and on variety of data such as audio, video, medical,
social, sensor, etc. The development of parallel GPU’s,
publically labeled datasets, powerful frame works like
Tensor flow, and Theano became quite accessible in
addition to speeding up the training of deep learning models.
Deep learning became a fuel for many computer vision
problems such as moving object detection, segmentation,
motion tracking [
However, several works have addressed for automatic
segmentation of organs at risk on CT/MRI scans at different
parts of body and using deep learning techniques. In a
review paper [
To segment organs (Esophagus, Trachea, Heart and Aorta) accurately from the raw CT images, we are using dilated/ Atrous convolution and the schematic diagram is shown in Fig 1. Proposed architecture is selected based on two reasons 1) the size of organs varies from patient to patient and slice to slice, 2) the output of convolutional neural network in multi segmentation is a coarse output. The detailed explanation of the Dilated U-Net is given in section 2.1. Thereafter, the final segmented output is formed by summation of the individual output coming from each model.
The use of deep convolutional neural network for fully
connection fashioned segmentation has been addressed
successfully in [
Let x[i] be the 1-D input signal, y[i] be the output of dilated convolution of kernel w[k] of length k is
k y i x i r.k w k i1 (1)
The parameter corresponds as stride through which we alter the image. Standard convolution is a special case in dilated convolution, if = 1. Dilated convolution helps in enlarging the filed view of kernels at any layer of the Dilated U-Net. Dilated U-Net uses a small kernel (typically 3 × 3 kernel) in order to control computational time and number of parameters. Dilated convolution with r as rate introduces − 1 zeros in successive values in kernel i.e. enlarging of × kernel into = + ( − 1)( − 1) without increasing the computational amount and parameters. It offers a Mechanism to find trade-off between small field of view and large field of view.
We used a Dilated U-Net architecture with 14 layers, in which first six layers involves operation of convolution, dilation, ReLU and batch normalization followed by average pooling after every consecutive two layers. The last six layers are up sampled using bilinear interpolation and concatenation with previous layers. Further, it involves convolution, ReLU and batch normalization. Convolution, ReLU and batch normalization applied on seventh layer and passed to the dilation box individually. The detailed architecture is shown in Figure 2.
The feature maps of seventh layer followed convolution, ReLU and batch normalization passed separately to each layer in dilation box. In dilation box the output of seventh layer convoluted with different dilation rate from = 20 25 and summation of this six layers is given as input to the seventh layer of Dilated U-Net architecture and it is shown in Figure 3.
The outputs from the Heart, Esophagus, Aorta and Trachea are combined to form an overall segmented result. The drawback of this process is overlapping of individual organ regions. To overcome the overlapping regions problem, the mode of eight-neighborhood intensities is used for location as shown in equation 2 i 1, j 1 i, j 1 i 1, j 1 i 1, j i, j i 1, j i 1, j 1 i, j 1 i 1, j 1 (2)
From this eight neighborhood locations, the mode is calculated and replaces the value at location , . By this approach, the overlapping issue is addressed.
We performed the experiments on standard dilated U-Net architecture. Further, post processing has applied on Dilated U-Net output using mode as filter. Hausdroff and Dice ratio has used as quantitative evaluation measures. The proposed algorithm has implemented in Python 3.5, 64-bit Windows8 platform with Intel Xenon CPU@2.80 GHz, 64 GB of RAM and 8 GB of GPU.
The performance of the proposed method is evaluated on
SegTHOR challenge dataset in CodaLab [
Each scan is normalized to zero mean and unit standard deviation. Train test split is performed with test size as 0.2, resulting 32 scans for training and 8 for validation and remaining 20 CT scans for testing.
Data augmentation need to apply when training data is less and it is necessary for network to learn desired properties at microscopic level. In images like CT scans and MRI scans, shift, rotation and deformation invariance is needed and it’s implemented in U-Net. We implemented rotation, shift variance-using Keras ImageDataGenerator class focusing on width, height, rotation, and zoom parameters.
The data is different for each organ i.e. the number of active voxels for each organ is different, so to avoid overfitting towards a dominant organ we trained each organ separately and summation has done over the intensities of four models. We fine-tune the weights using binary cross entropy loss. Adam gradient descent has used with a learning rate 0.0001 until 75 epochs with 300 epochs per step.
The results of the proposed method are compared with UNet architecture and are shown in Figure 4. The U-Net output is shown in Figure 4(b), it is noticed that the output is coarse in nature. Figure 4(c) shows the results of segmentation of Heart, Esophagus and Aorta whereas Figure 4(d) presents the output of overlapping regions separated by applying post processing.
The Table1 gives comparative quantitative evalaution of the proposed U-Net in terms of the dice ratio with U- Net, and proposed U-Net postprocessing. The proposed U-net outperformed basic
U-Net in
Aorta, Esophagus and Trachea. The proposed U-Net run parllel with Heart. The postproceesing helped to achieve high results in Trachea than proposed U-Net. the proposed U-Net in terms of the Hausdorff distance with
U- Net, and proposed U-Net postprocessing. The proposed U-net outperformed basic U-Net in Esophagus. The post proceesing helped to achieve high results than basic U-Net and the proposed U-Net in Trachea.
dilated U-Net d) Output after Post processing 60 CT scans dataset from SegTHOR challenge. Individual segmentation of organs from background and augmentation helped to train models on low level and high level features. The results were further improved with a deliberated post processing. For performance evaluation, Dice ratio and
Distance metrics were used wherein the segmentation of Esophagus and Trachea shows significant improvement.