The diagnosis of Multi Drug Resistant (MDR) tuberculosis is challenging. We present our method for classifying whether a patient has MDR tuberculosis or drug sensitive (DS) tuberculosis based on a CT scan of that person's chest, which achieved the best accuracy and the second-best AUC at the ImageCLEF 2018 Tuberculosis - MDR detection task. Our approach consists of reformatting the images in the coronal plane, converting them to png format and using transfer learning to train a ResNext 50 convolutional neural network to classify images as MDR or DS tuberculosis.
Tuberculosis is still a common disease and the diagnosis of Multi Drug Resistant
(MDR) tuberculosis is challenging. It is difficult for radiologists to distinguish between
MDR and Drug Sensitive (DS) tuberculosis and there is inconsistency in the literature
on which radiographic features are useful. For instance, presence of lymph node
calcifications is associated with MDR in some papers and with DS in other [
Neural Network Training For training of the CNN, we rented from Paperport a cloud virtual machine with 8 CPUs, Quadro P5000 GPU, 30 GB RAM, and 500 GB solid state drive created using the fast.ai public template. We took advantage of the fastai library to perform transfer training of ResNext 50 [10] convolutional neural network.
For training the CNN an image size of 64 x 64 was utilized. The learning rate was determined after running the learning rate finder function and plotting the learning rate vs. loss. See Figure 2.
Fig. 2. LEARNING RATE VS. LOSS After reviewing this curve, a learning rate of 0.002 was selected for the last layers. The last layers were trained for 2 epochs without data augmentation, then were trained for 2 additional epochs using data augmentation. For data augmentation, we used random rotations of up to 10 degrees in each direction, random changes of intensity of up to 5%, and random horizontal flipping (but no vertical flipping) based on the assumption that right and left lung are similar, but upper and lower lobes are different. Subsequently all layers were unfrozen and trained for an additional 3 epochs using a different learning rate for different layers. The final layer learning rate was kept at 0.002, but the learning rate for the middle layers was one third of the last layers and the initial layers learning rate was one ninth of the last layers. Same augmentation used at training time was also used at test time, and the average of 4 augmented images was used for each test image.
As we had analyzed each image separately, we had 200 different results for each patient, so we averaged the results of the 200 images of each patient. As expected, using the average decreased the probability of MDR tuberculosis as some of the images were including only normal or less abnormal lungs. As the number of patients with MDR was known, the probability was manually rescaled in Microsoft Excel before submission to provide the correct number of positive and negative MDR cases and to use the entire probability range from 0 to 1. 3.
When each image is scored individually, patients with MDR tuberculosis have a significant number of images scored as not MDR tuberculosis. This can be explained by the fact that significant pathology necessary to make the diagnosis of MDR tuberculosis may not be present in all images.
In the final table of results, the submitted run for MDR detection task was ranked first for accuracy among the 39 submitted runs with a prediction accuracy of 0.6144 and second for area under ROC-curve (AUC) equal 0.6114 on the test image dataset[7]. The best result in terms of AUC value was achieved by VISTA@UEvora team and resulted in AUC = 0.6178. Run MDR-Run-04-Mix-Vote-L-RT-RF.txt testflowI.csv
Although we achieved the best accuracy and second-best AUC, to be clinically useful automatic detection of MDR need to further improve. Accuracy and AUC in the 0.61 range cannot be relied upon by the treating physician. 5.
Due to the competition’s time contrains, several shortcuts were implemented: arbitrary selection of coronal images 150 to 350, conversion of images to png format, averaging results of single slices of each patient. A better selection of images containing the lungs or even better, the abnormal portion of the lungs/mediastinum, may improve results. Using Hounsfield units from the original images, instead of values in the png files may also be more accurate. Instead of averaging the results of single images and rescaling the results, utilizing a more robust approach to combining results from multiple images from the same patient may also help — possibilities to consider include using an SVM[11] or an RNN [12]. 6.
In this paper, we presented the use of transfer learning to quickly train a CNN to achieve the best accuracy and second-best AUC at the ImageCLEF 2018 Tuberculosis - MDR detection task[7]. It also achieved better results than all submission at the ImageCLEF 2017 Tuberculosis - MDR detection task. 6. Bogdan, I., et al., Overview of ImageCLEF 2018 : Challenges, Datasets and Evaluation.
Proceedings of the Ninth International Conference of the CLEF Association (CLEF 2018), 2018. 7. Yashin Dicente Cid, V.L., Vassili Kovalev, Henning Müller, Overview of ImageCLEFtuberculosis 2018 - Detecting multi-drug resistance, classifying tuberculosis type, and assessing severity score. CLEF2018 Working Notes, 2018. 8. Chung, M.J., et al., Drug-sensitive tuberculosis, multidrug-resistant tuberculosis, and nontuberculous mycobacterial pulmonary disease in nonAIDS adults: comparisons of thinsection CT findings. Eur Radiol, 2006. 16(9): p. 1934-41. 9. Howard, J.a.o., fastai. GitHub, 2018. 10. Saining Xie, R.B.G., Piotr Doll, Kaiming He, Aggregated Residual Transformations for
Deep Neural Networks. CoRR, 2016. abs/1611.05431. 11. Gao, X.W. and Y. Qian, Prediction of Multidrug-Resistant TB from CT Pulmonary Images
Based on Deep Learning Techniques. Mol Pharm, 2018. 12. Sun, J., Chong, P., Tan, Y.X.M., Binder, A., : ImageCLEF 2017: ImageCLEF tuberculosis task - the SGEast submission. CLEF2017 Working Notes. CEUR Workshop Proceedings, 2017.