In this work, we propose a 16-layer 3D convolutional neural network with a slice selection technique employed in the task of 3D Computed Tomography Image data of Tuberculosis(TB) patients which attained 10 th place in the ImageCLEF 2019 Tuberculosis - Severity scoring challenge. The goal is aimed at estimating TB severity based on the CT image. The best result reported in this work is Area Under the ROC Curve (AUC) of 0.61 and a binary accuracy of 61.5%. Codes for this work can be found at URL.1
Tuberculosis is a potentially serious infectious disease that a ects lungs, and
sometimes other parts of the body. Mycobacterium tuberculosis, is the
bacteria that causes the infection and can spread from one person to another via
cough, spit or sneezes. Most infections do not cause any symptoms, which is
refered as latent tuberculosis and 10% of them turns to potentially fatal
active diseases. People with HIV/AIDS and smokers are more vulnerable to active
TB. The symptoms of the infections are cough with blood-containing mucus,
night sweats, fever, chills, loss of appetite and severe weight-loss. About
onequarter of the world population has been infected with the disease. In 2010,
1.2 -1.45 million deaths have occurred due to the disease, mostly in developing
countries which makes it the second most common cause of death from
infectious disease [
Copyright c 2019 for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0). CLEF 2019, 9-12
September 2019, Lugano, Switzerland.
more detailed information about the infection than X-ray images [
Training 3D Convolutional Nets on volumetric data is computationally
expensive due to depth information which requires additional feature learning and
hence results in an increased number of learnable paramters. Moreover, all the
data samples do not have same depth size which complicates training. To
address this problem, we introduce a novel data partitioning technique which makes
the training method not only e ective but most importantly feasible. We show,
using partial depth information from the each volumetric data point, it is
possible to achieve good AUC and accuracy values. We show our approach which
achieves 10 th place among a total of 100 participants in the ImageCLEF 2019
Tuberculosis - Severity scoring challenge [
This section provides details of the setup used in the di erent experiments. A brief description of the data set used for the experimentation will be given, followed by description of the network architecture. Moreover, details of the training regiment with the metrics are provided.
For replication of this work, it is relevant to mention that all of the experiments were performed on a machine with Windows system with Intel Core(TM) i7-7700 CPU @3.60GHz processor, 32 GB RAM, a single CUDA-enabled NVIDIA GTX 1050 4GB graphical processing unit (GPU), Python 3.6.7, Keras 2.2.4 with Tensor ow 1.12.0 backend, and CUDA compilation tools, release 10.0, V10.0.130 dependencies for GPU acceleration. 2.1
The dataset for the severity scoring(SVR) task is provided by ImageCLEF
Tuberculosis 2019 [
The original severity score which is assigned by a medical doctor is included as
training meta-data which are annotated in a scale from 1("critical/very bad") to
5("very good). This grade is converted to "LOW" (scores 4 and 5) and "HIGH"
(scores 1, 2 and 3) thereby reducing the task to a binary classi cation problem
as per task descriptions provided by ImageCLEF Tuberculosis 2019[
In order to prepare the data for training, we rst resize individual slices of the 3D input volume to 128 128 with bicubic interpolation.
This is followed by a slice selection technique that we introduce in this work, where a depth size of 32 and 16 - which results in two experimental settings discussed in 1 - are chosen for our nal submissions.
In Figure 2, we show a visual representation of the slice selection technique. For a given input volume the rst 4 slices, the middle 8 slices and the last 4 slices are extracted. The middle slice of the input volume is obtained by taking the half of the input volume depth. These three sub components are then stacked to reconstruct the desired input volume where, in this case, is a depth size of 16. In other words, the input volume consists of a total of 16 slices. The main motivation behind the proposed technique eliminates the problem of GPU exhaustion during optimization. Since the default input volume consist of large number of slices, it was entire impossible to allocate tensors for computation in our experimental setup discussed at the end of Section 1.
The network used in this work was inspired by the architecture used for real
time object recognition by integrating a volumetric occupancy grid
representation with a supervised convolutional net named VoxNet [
The convolutional blocks in Figure 3a is then followed by a batch normalization layer. The deep learning community has quickly adopted the use of batch normalization as it introduces a form of regularization which restrains the network form simply memorizing the training dataset, which means the network is expected to generalize better on unseen data with use of batch normalization. The output from the batch normalization layer is attened and passed to a series of fully connected layers with two dense layers having 1028 neurons, one with 512 neurons. Each of the dense layers were connected to a dropout layer which drops the neuron connection with a probability of 40%. The output from the nal two dropout layers was followed by a dense layer of 2 neurons. The network architecture is shown in Figure 3.
Softmax activation shown in equation 2 was applied on the last layer to get the probability results for the binary classi cation problem. The output of the softmax function is equivalent to a categorical probability distribution, it tells you the probability that any of the classes are true. This enables better performance of the model.
(zj ) =
e(zj) PK k=1 e(zk) : (2) where z is a vector of the inputs to the output layer (if you have 10 output units, then there are 10 elements in z). And again, j indexes the output units, so j = 1, 2, ..., K. 2.4
Stochastic Gradient descent was used to optimize the weights of the network via backpropagation with a learning rate of 10 4 with a momentum of 0.9. Cross-entropy error between the predicted and ground truth was used as the loss function show in Equation 3 and the weights were updated using minibatches for every iteration. The initialization of the weights was done at random and the biases were initialized as zero.
L(y; y^) =
i=1 c=1 1 XN XC 1yi Cc log pmodel[yi Cc ] N (3)
In equation 3, the double sum is over the observations i, whose number is N , and the categories c, whose number is C. The term 1yi in Cc is the indicator function of the ith observation belonging to the cth category. The pmodel is the probability predicted by the model for the ith observation to belong to the cth category. When there are more than two categories, the neural network outputs a vector of C probabilities, each giving the probability that the network input should be classi ed as belonging to the respective category. When the number of categories is just two, the neural network outputs a single probability with the other one being 1 minus the output.
Training was continued for 300 epochs on layers L1 to L16 in Figure 3 with a validation split of 0.1 and the nal evaluation was done on the test set. For the nal submissions, we only make change in input volume depth size which also required us to change the batch size which results two di erent settings shown in Table 1. All the other remaining parameters were kept same in the two settings. It is noteworthy that, in both the con gurations, the learnable parameters are 23,808,378.
The task is evaluated as binary classi cation problem, including measures such as Area Under the ROC Curve (AUC) and accuracy. An AUC of 1 represents a perfect classi cation system where True positive rate is 1 and False positive rate is 0. Since the ranking of the techniques will be rst based on the AUC and then by the accuracy, AUC is the optimizing metric and the binary accuracy is the satisfying metric. 3
From Table 2 it can be seen that CFG-A achieves the highest AUC and accuracy values in the experiments conducted in this work. CFG-A, where the network is trained with an input volume of 128 128 32 , this achieves an average test AUC of 0.611 and test accuracy of 61.5%. Also to note, the batch size for this con guration was set to 4, since any larger values resulted in GPU memory exhaustion. This still led to achieve the best result in the set of experiments conducted which is surprising. CFG-B which is trained and evaluated with an input volume of 128 128 16. From the experiments it can be said that the degradation in CFG-B is due to lower number of slices in the input volume which results in information loss. Even though the batch size was 16 in this setting, the information loss outweighs the impact on the overall performance.
Name CFG-A CFG-B
In Figures 4 and 5 we show the training logs for both the con gurations. Both CFG-A and CFG-B are trained for 300 epochs. In Figure 4, which portrays the training log for CFG-A, it can be seen that the network starts to over t only after 25 epochs. CFG-A achieves a highest validation accuracy of 68%, where in the test set an accuracy of 61.5% is achieved which results in test-val accuracy margin of 6.5% between validation and testing set even when the batch size is set to 4.
In the case of CFG-B which is shown in Figure 5 the network starts over tting after 60 epochs and achieves a highest validation accuracy of 82.5%. It is surprising that this con guration achieves only a test accuracy of 53.8% which results in a test-val accuracy of 28.7%. From this behaviour we can say that the preprocessing technique employed in CFG-B with depth size of 16 results the test set to not be representative of the validation set. This setup also causes information loss which results in signi cantly lower performance than CFG-A.
We demonstrate a 3D convolutional neural network with a newly proposed
preprocessing technique, slice selection from volumetric data, used in the task
to estimate severity based on CT Image of Tuberculosis patients. This work
achieved 10 th place with a test AUC of 0.611 and test accuracy of 61.5% in the
ImageCLEF 2019 Tuberculosis - Severity Scoring challenge [
In future works, the results will be further analyzed to gain a better understanding of the reasons behind the results. In addition, various networks architectures will be experimented and further improvements in the proposed slice selection technique will be made, in an attempt to build a robust deep learning model to estimate severity of TB patients.