Automated TB and disease classification is a dire need in these times, as traditional diagnostic procedures are ineficient. Existing literature is focused on TB identification and classification using 2D images. As 3D images contain extra depth information which helps in more accurate modelling of the disease cavern, an approach using the 3D-CNN model has been proposed to classify the type of caverns present in lung CT scans in order to ensure prompt treatment.
can be made that a model using a 3D dataset is more reliable. Therefore it is crucial to come up with new deep learning solutions that can detect TB based on 3D images and provide higher accuracy.
The dataset used for this task is from ImageCLEF 2022[
A single 3D image is provided for each of the 60 patients. Each 3D image contains around 100 slices of 2D images of 512x512 pixels. All the CT images are stored in NIFTI file format with .nii.gz file extension (g-zipped .nii files). This file format stores raw voxel intensities in Hounsfield units (HU) as well the corresponding image metadata such as image dimensions, voxel size in physical units and slice thickness.
Two versions of automatically extracted masks of the lungs were provided for each CT image. This data is available along with the CT images of the patients. The first version of segmentation is able to provide masks in an accurate manner,but tends to miss features in severe TB cases where large abnormal regions of lungs are present. On the contrary, the second segmentation provides rough bounds, but includes lesion areas.
The 3D-CNN architecture is frequently used for a stack of 2D images, particularly for medical images, as it can assess the positions of defects in the time domain. During the convolution step, the 3D-CNN generates a 3D activation map. This is required for time and volumetric context. To calculate the representation of elements at a low level, a three-dimensional filter is employed for the 3D convolution of the dataset.
Convolution and pooling layers connect only to local regions around each input in the CNN, which is a version of the classic neural network. CNNs are sparsely connected to hierarchical representation of the input, allowing them to process images from general forms to edge details. The 3D-CNN is a development of 2D-CNNs that captures discriminating features in both the spatial and temporal dimensions by dividing hierarchical 3D visual information into small cubes rather than 2D patches.
Figure 1 illustrates the notion of a 3D convolution. The input 3D image is split into 2D slices. The process of 3D convolution begins with the 2D slices, with xij denoting the jth receptive ifeld on the i th slice. Local receptive fields are constructed on 2D slices to generate 2D features yij, which are convolved along the temporal direction i to generate feature vector yj at the convolution layer. A 3D pooling layer follows the same steps as a 2D pooling layer. However, it takes the maximum or average values at each step. The CNN is ready for hierarchical learning from motifs to edges after repeated convolution and pooling operations until the picture information is adequately compressed.
The sequences of signals will be fed into the final hidden layer, where a fully connected neural network will learn the image’s attributes and generate the output as data sequences. Figure 2: Proposed 3D-CNN Architecture
The model was trained on an Intel i7 core CPU, Samsung 1TB SSD and an Nvidia GeForce GTX 2060 Super GPU System. Google Colaboratory was also used which uses a 12GB Nvidia Tesla K80 GPU. The Deep Learning framework Tensorflow was used along with tools such as Anaconda, Jupyter Notebook Environment, CUDA, Numpy, Pandas and Sci-Kit Learn.
A TB classification system was built which has been implemented as multiple binary classifiers for each of the three classes; namely Has thick walls, Has foci around, and Has calcification . The procedure for the implementation of the system is given as follows: • The labels were separated by class in order to implement multiple binary classifiers for the three classes. The separated labels were stored in a Numpy array. • The given dataset for the TB Task consists of 3D images. Each 3D image was split into a set of 2D image slices. Each slice was resized to 128x128 pixels using Spline Interpolated Zoom and Inter Cubic Interpolation. • The training images were loaded into Numpy arrays in order to train the model. • The training data was normalized as it calibrates the diferent pixel intensities into a normal distribution which makes computation eficient and helps the model converge faster. • The normalized data was split into training and testing sets. • During the learning process, the learning rate is automatically reduced if the loss remains the same for a predefined number of epochs. This callback monitors the validation loss and helps prevent stagnation. • If the model shows no improvement in the reduction of validation loss after a predefined number of epochs even after reduction of the learning rate, the training is automatically stopped. • The architecture of the model in as shown in Figure 2. • K-fold cross validation was performed to estimate the prediction of the model on unseen data and is a preventive measure for overfitting. This ensures that the proportion of the feature of interest is the same across the original data, training and the test set, which gives a more accurate estimate of the performance of the model.
• After training, the model was made to predict values for the test data.
The model was trained on a small scale dataset of five patients and then upscaled to sixty patient records. The model has attained an accuracy of 75% on training data and 60% on validation data. The area under the ROC curve for the validation data is 0.78 for the class Has Calcification , 0.74 for the class Has Foci Around, and 0.65 for the class Has Thick Walls.
For the classes Has Foci Around and Has Calcification , the most of the ROC curve is present towards the left of the line denoting the random classifier but it intersects the line. The validation accuracy is very less compared to the training accuracy and the validation loss is much greater than the training loss. Therefore it can be inferred from the figures 3, 4 and 5 that the model might be overfitting the data. This is due to the imbalance between the positive and negative binary classes present in the separated datasets for the classes Has Foci Around and Has Calcification . However, for the class Has Thick Walls, the ROC curve is present towards the left side of the line denoting the random classifier as well as does not intersect it. The training and validation accuracy plot lines are close to each other and the same is true for loss values as well. This shows that the model has correctly fitted the data. Thus, it can be inferred from this graph that the model performs well on this class as the positive and negative samples in the dataset for this class are balanced.
The following table describes the probability of the three classes predicted by the model.
In summary, a Deep 3D Convolutional Neural Network model structure has been proposed to accurately predict the classification of the cavern present in CT Scans of Tuberculosis in the Lung region. In order to increase the predictions of the model, the parameters and the hyperparameters of the 3D-CNN model were tuned. The optimal model resulted in a training accuracy of 75%. The model was tested for the test data of 16 patients and achieved a mean AUC of 0.536.
We thank the CSE department of SSN College of Engineering for letting us utilize the GPU machine extensively to implement this task. H. Schindler, J. Chamberlain, A. Campello, A. Clark, Overview of the ImageCLEF 2022: Multimedia retrieval in medical, social media and nature applications, in: Experimental IR Meets Multi- linguality, Multimodality, and Interaction, Proceedings of the 13th International Conference of the CLEF Association (CLEF 2022), LNCS Lecture Notes in Computer Science, Springer, Bologna, Italy, 2022.