Convolutional neural networks have achieved state-of-theart results in general image classi cation tasks and have shown success in several applications within the medical imaging domain. In this paper, we apply a 3D convolutional neural network (CNN) to a dataset of tuberculosis-positive computed tomography (CT) scans to solve the task of automatically categorizing each tuberculosis (TB) case into one of ve possible TB types in the context of the ImageCLEFtuberculosis 2018 challenge. The size of the volumetric scans poses unique constraints on the network and the training process. The CT volumes are segmented with the provided masks, which are further pre-processed prior to training our model. Our best run ranked 2nd with an unweighted Cohen's Kappa of 0.1736 and an accuracy of 35.33%.
For the second year, ImageCLEF [
Deep learning approaches have been shown to be successful on a large variety
of computer vision and image analysis tasks [
The training set provided by the ImageCLEF organizers consisted of patient chest CT scans of ve di erent types of TB along with their labels. Often patients had multiple scans and all scans of the same patient were of the same type. There were 228, 210, 100, 79, and 60 patients belonging to In ltrative, Focal, Tuberculoma, Miliary, and Fibro-cavernous types, respectively. The dataset totaled 677 patients with 1008 scans (Table 1). Each scan consists of approximately 100 512 512 slices. The depth of each scan varies and was changed to a constant number of slices.
The pre-processing stage consisted of 7 steps (Figure 1). The supplied masks [
The two datasets each had a respective train/validation split of 80/20.
Initially this split was done by scan and validation accuracy was relatively high.
However, when submitting results on the test set the accuracy was much lower
and close to random. This was thought to be at least partially due to the method
for splitting. Because some patients had multiple scans, there were scans in the
train and validation set from the same patient. Upon visual inspection, scans
from the same patient were indeed similar (see Figure 3).
The trained models were 3D convolutional neural networks using the software
library Keras [
Each network (Figure 4) had ve convolution layers with recti ed linear unit
(ReLU) activations, each with a following batch normalization and max pooling
with dropout layer. These led to two fully connected layers, each with batch
normalization and dropout. Finally, these activations went through a softmax
layer, which output a tensor of size ve, for each category. Categorical
crossentropy was used as the loss function, and Adam [
One of the restrictive parts of training this model was the batch size. Such small batches make it harder to converge.
Our most successful model was the combination of the two best models, which had inputs of di erent size image volumes. The outcome of this ensemble were 5 probabilities, one for each class. The probabilities were summed across the two models, and then this vector was iteratively scaled by a weight vector which was calculated from the class distribution. This resulted in output labels which more closely matched the data distribution. This combination of networks shown in Figure 4 was used for predicting the test labels. 4
Only runs for the tuberculosis type subtask 2 were submitted. Our initial submissions accuracies were barely better than random chance ( 28%). After combining models and weighting probabilities, the accuracy and kappa score did improve. Our best run (indicated in bold in Table 2) ranked second in unweighted kappa coe cient, but tenth in accuracy. 5
This paper applies a 3D CNN to pre-processed CT scans of the lungs. The question of whether a CNN can extract the information necessary for labeling types of TB remains open. Making predictions on image data alone has proved a challenging problem. The large size of the images and small size and class imbalance of the datasets are characteristic of medical imaging tasks. In this analysis, batch sizes were restricted to sizes of ve and fourteen samples for the two networks used. A feasible way of e ectively training with a much larger batch size is to accumulate the gradients of each batch and only update the weights of the network after storing a su cient number of batches gradients. The average of the gradients for each batch can be used to update the weights. This allows for e ectively training on larger batch sizes, circumventing memory problems.