Tuberculosis (TB) is a dreaded bacterial infection that affects human lungs. It has been known to mankind since ancient ages. Tuberculosis ImageCLEF 2018 proposes a set of tasks based on Computed Tomography (CT) scan images of patients' lungs. They are: multi-drug resistance (MDR) detection, tuberculosis type (TBT) classi cation and severity scoring (SVR). In this work, two di erent methods are presented to solve these problems. Texture analysis based methods (3D Modeling and Slice extraction approach) were used to generate feature values from CT scans and di erent classi ers were tested. 3D Modeling approach calculates seven statistical features of Mean, Skewness, Kurtosis, Homogeneity, Energy, Entropy and Fractal Dimension. And Slice extraction approach calculates 96 dimensional feature vector based on Contrast, Correlation, Energy, Homogeneity, Entropy and Mean. In accordance with the ranking given by the organizers, this approach was ranked 1st for multi-drug resistance detection, 5th for tuberculosis type classi cation and 3rd tuberculosis severity scoring.
Tuberculosis (TB) is an infection caused by a bacteria which named Mycobacterium tuberculosis. This bacteria generally attacks the lungs but sometimes it can damage other parts of the body. TB spreads through the air when an infected person coughs, sneezes or talks. From World Health Organization (WHO) report, tuberculosis is one of the top ten courses of death worldwide [12].
The greatest problem that can happen to a TB patient is that the organisms become resistant to two or more of the standard drugs. In contrast to drug sensitive (DS) tuberculosis, its multi-drug resistant (MDR) form is much more di cult and costly to recover from. Thus, early identi cation of the drug resistance (DR) status is of great importance for an e ective treatment. The most frequently used methods for DR detection are either costly or take a long time (up to several months), hence, there is an urgent requirement for fast, precise and cheap techniques. One possible technique is the automatic analysis of CT scan images of patient's lungs to help characterize tuberculosis.
ImageCLEF (the Image Retrieval and Analysis Evaluation Campaign of the Cross Language-Evaluation Forum) has organized challenges on image classi cation and retrieval since 2003 [11]. The 2018 edition [8] proposes 3 main tasks (and a pilot task), one being related to the analysis of tuberculosis from lung CT images [4]. This task includes three independent subtasks: multi-drug resistance (MDR) tuberculosis detection, tuberculosis type (TBT) classi cation and severity scoring (SVR). This work presents the University of Evora approach to tackle these subtasks.
The rest of the paper is organized as follows: Section 2 describes the Methodology (theory of methods, explanation of 3D modeling approach and slice extraction approach) and Section 3 introduces the Experiments and Submitted Runs (dataset description, evaluation metrics, system con guration, top-5 submitted runs for each subtask and results on test dataset). Finally, Section 4 concludes the paper. 2
To tackle ImageCLEF 2018 tuberculosis task [4], two di erent approaches were tested: one based on 3D modeling and another based on slice extraction information. Next subsections present the underlying theory and techniques of the proposed approach. 2.1
The present work is based on mean, higher order moments (skewness and kurtosis) and texture analysis to classify tuberculosis images. We have proposed two basic models for the present ImageCLEF 2018 [8] competition. In this section, we present the theoretical basis for the estimation of mean and higher order moments, fractal dimension and GLCM-based texture analysis (energy and homogeneity) techniques that used in both models.
Mean and Higher Order Moments. The mean (1st moment), skewness (3rd moment) and kurtosis (4th moment) are measures of asymmetry and normalized form of the fourth central moment of the whole bronchi ROI respectively. The mean; found by adding the pixel values and dividing by the number of pixels. If the skewness is negative, the pixel values are spread out more to the left of the mean than to the right; if skewness is positive, the pixel values are spread out more to the right. Kurtosis indicates the degree of peakedness of the values; it is based on the size of the tail of the pixel value distribution.
The skewness and kurtosis of the pixel value within the whole bronchi can
be determined using Equation (
nthmoment =
X (pi
i
p)nf (pi)
(
A(") = F "2 D
where F is a constant and D is the fractal dimension of the surface. When
the log(A(")) is plotted against log(") a straight line is obtained with a slope
equal to 2 D, which gives an estimation of the fractal dimension (Equation(
D)log(")
Gray Level Co-occurrence Matrix. Energy, homogeneity, contrast,
correlation and entropy are statistical texture features. These features are based on the
normalized gray-level co-occurrence matrix (GLCM); for a 2D bronchi map f ,
the co-occurrence matrix Mf; (k; l) represents the joint probability occurrence
of pixel pairs (with a de ned spatial relationship) having gray level values k and
l , for a given spatial o set = ( x; y) between the pair [7,14]. Mf; (k; l) is
de ned by Equation (
Mf; (k; l) = 8 X > > ><x=1n y=1n >> X X 0; otherwise >:x=1n y=1n
X 1; if f (x; y) = k and f (x + x; y + y) = l
The co-occurrence matrix Mf; (k; l) has dimension n n, where n is the number of gray levels in f . The GLCM accounts for the spatial inter-dependency or co-occurrence of two pixels at speci c relative positions. Co-occurrence matrices are calculated for the directions of 0 , 45 , 90 , and 135 and the average matrix over all o sets can be used [1,10]. In this study, the 2D formulation with 8-connexity (computed with 8 di erent o sets) was used. On the basis of this matrix, the second-order statistical features of energy and homogeneity [7] for each image slice were derived. Then, the average of energy and homogeneity over all slices was estimated.
Energy is de ned as the measure of the extent of pixel pair repetitions in
the matrix M and can be estimated using Equation (
Energy = X X Mf; (k; l)2
Homogeneity is the statistical measure of the similarity of pixels in the matrix
M and can be estimated using Equation (
Homogeneity = X X k=1n l=1n 1 + jk
Mf; (k; l) lj
Contrast returns a value after measuring the intensity contrast between a
pixel and its neighbors over the entire image. Contrast is also known as variance
or inertia and is given by Equation (
Contrast = X X jk
lj2Mf; (k; l) k=1n l=1n
Correlation returns a value after measuring how correlated a pixel is to its
neighbors over the entire image. Correlation ranges between -1 and 1; it's 1 or
-1 for a perfectly positively or negatively correlated image and is N aN for a
constant image. It can be calculated through the Equation (
X (k k)(l
l)Mf; (k; l) k=1n l=1n k l
Entropy returns a scalar value representing the entropy of a grayscale image.
Entropy is a statistical measure of randomness that can be used to characterize
the texture of an image and is de ned by Equation (
Entropy = X X
ln(Mf; (k; l)):Mf; (k; l) k=1n l=1n 2.2
The rst method is based on 3D modeling and further extraction of texture patterns. The block diagram is presented in Figure 1. First, the input data set is pre-processed: slices are extracted and 2D data is converted to 3D data by selecting lung ROIs (using masks [3] given). Then, bronchi are extracted using a pre-de ned threshold value. The texture analysis is performed within the bronchi. Finally, a feature vector is computed, followed by the application of multiple classi ers and their performance analysis. The main parts of the system are described below.
roimaging Informatics Technology Initiative) format and 3D images were extracted. The masks of each subject were also provided in NIFTI format. The masks were applied to extract the 3D lung images of each subject. Then, a threshold value was de ned in order to extract only the bronchi within the lung. Based on the threshold, ROIs were selected (see Figure 1). Then, the statistical features of mean, higher order moments (skewness and kurtosis), homogeneity, energy, and entropy of the whole bronchi pixel value distribution of each individual subject were calculated.
Estimation of Mean. For quantitative data analysis, the ROIs on multiple image slices were extracted within the bronchi. Then the whole 3D bronchi ROI mean of signal intensity pixel values was computed by weighted (by pixel number) average of slice values (using all pixel values within the multi-slice ROIs encompassing the bronchi). The histograms of signal intensity were plotted with the mean values.
bronchi pixel value distribution higher order moments (skewness and kurtosis)
and fractal dimension of pixel values were calculated within the bronchi ROI
using the Equations (
In this method, a focus was taken on individual slices. Using a threshold and performing averages over attributes computed on each slice the nal feature values for the particular image was obtained. First the input dataset is preprocessed: slice extraction, ROI generation using a mask and ROIs selection based on threshold values are done; then, using Texture analysis on each ROI features are extracted by averaging slice-wise values; nally, a feature vector is computed, followed by the application of multiple classi ers and their performance analysis. The block diagram is presented in Figure 2. Individual Slice Consideration. Since the images were in NIFTI format (where a single image has several slices), so in this approach, we have taken individual slices for further processing and nally performed a weighted average among all slices. Figure 3 shows a slice of Tuberculosis CT scan image.
After slice extraction, an ROI for each slice was de ned based on the given provided mask [3]. Figure 4 shows a mask for image depicted in Figure 3.
Observing each slice pattern, a threshold value of 15000 (pixel area) was chosen to ensure that no slices with meaningful information would be missed. Here, meaningful information corresponds to some dots being present in the ROI.
Feature Extraction. There are two parts for the feature extraction task: Texture Analysis on each ROI and Averaging Attribute Value.
For texture analysis, the Gray Level Co-occurrence Matrix (GLCM) [7] based on texture features from the selected slices was computed. For each ROI, 96dimensional features were extracted.
For present work, 16 o sets are considered for generation of GLCM as shown below. The value 1, 2, 3, 4 represent the pixel distance from the point of interest in each of the four directions: 0, 45, 90 and 135 degrees. So altogether 16 o sets are available which eventually generates 16 co-occurrence matrices. Now, from each of this co-occurrence matrix, 06 features are computed namely: Contrast, Correlation, Energy, Homogeneity, Entropy and Mean. Finally, 16 6 = 96 dimensional feature vector is generated. (a) (0 1); (0 2); (0 3); (0 4); ! at 0 degree direction (b) (-1 1); (-2 2); (-3 3); (-4 4); ! at 45 degree direction (c) (-1 0); (-2 0); (-3 0); (-4 0); ! at 90 degree direction (d) (-1 -1); (-2 -2); (-3 -3); (-4 -4) ! at 135 degree direction
Averaging attribute values for all slices was done to generate the nal feature vector, obtaining the correspondent 96 average feature values (for contrast, correlation, energy, homogeneity, entropy and mean). 2.4
For each of the previous approaches, several machine learning algorithms were used to build classi cation models. The tested classi ers were: Linear Discriminant Analysis (LDA), Logistic Regression (L), Multilayer Perceptron (MLP), Simple Logistic (SL), Sequential Minimal Optimization (SMO), Logistic Model Trees (LMT), Random Forest (RF) and Random Tree (RT). A simple voting scheme (Vote) also experimented. 3
As already mentioned, there are 3 di erent sub-tasks: multi-drug resistant (MDR) detection, tuberculosis type (TBT) classi cation and severity scoring (SVR). This section describes the datasets and introduces the evaluation measures and system con guration. 3.1
The second sub-task is a multi-class classi cation problem with ve tuberculosis types: in ltrative, focal, tuberculoma, miliary, and bro-cavernous. No information about the relation between this the classes are given. Table 2 presents the details for training and test sets. For training set, the total number of patients 677, but 1008 chest CT scans of TB patients along with the TB type. And some patients include more than one scan. Similar to the test set, patients 317 but chest CT scans image 505.
The third sub-task consisted of chest CT scans of TB patients with the corresponding severity score (1 to 5) and the severity level (\low" and \high"). Table 3 presents the details for training and test sets.
Moreover, lung segmentation extracted automatically [3] were also provided. In our work, we used this segmentation to restrict the region of interest of the lungs. Figure 5 shows sample slices of the Computerized Tomography (CT) images with segmented lungs.
For MDR task the performance of the system was measured using the Area Under the Curve (AUC) of the Receiver Operator Characteristic (ROC). The ROC curve is created by plotting the true positive rate against the false positive rate. Cohen's Kappa coe cient (Kappa) is used for measuring the system performance of TBT task. Kappa statistic is measure inter-rater agreement for qualitative (categorical) items. The performance of the system of SVR task was used Root Means Square Error (RMSE). RMSE presents the sample standard deviation of the di erences between observed values and predicted values.
To evaluate the system, a strati ed k-fold cross-validation approach was used. The value of k was chosen experimentally. Regarding resources, all experiments were carried out using MATLAB 2017b software and Weka 3.8.1 toolkit [6] in a system with 3.5 GHz CPU, 8 GB RAM. 3.3
From the total 30 runs allowed for submission for the ImageCLEF 2018 Tuberculosis classi cation challenge (10 runs for each task), 23 runs were uploaded. The runs were selected based on best accuracy, AUC, Kappa coe cient and RMSE measures (calculated with a cross-validation procedure over the training dataset), with the machine learning algorithms ne-tuned experimentally. For TBT subtask, it uses the 3D modeling approach (Section 2.2) with the machine learning Random Forest (RF) algorithm (with the following parameters: numFeatures=20, numIterations=1500 and seed=20) and for the SVR subtask, the 3D modeling approach with the multi-layer perception algorithm (MLP) with parameter trainingTime=100. This work presents texture analysis based approaches to categorize di erent TB images for 3 di erent problems: multi-drug resistance (MDR) detection, tuberculosis type (TBT) classi cation and severity scoring (SVR). Two di erent texture analyses, one based on 3D modeling and another based on slice extraction were proposed. Their individual and combined performances were tested using di erent machine learning classi ers.
Though in terms of the accuracy both approaches are very competitive, using the TB tasks ImageCLEF 2018 performance measures (AUC for MDR, Kappa for TBT and RMSE for SVR subtasks), using 3D modeling features seems more promising.
In future, we will use the patient clinical information to improve the overall performance of three tasks. 5
The authors thank the LEADER and gLINK Erasmus Mundus projects for
supporting this research.
11. H. Muller, P. Clough, T. Deselaers, B. Caputo, and I. CLEF. Experimental
evaluation in visual information retrieval. The Information Retrieval Series, 32:1{554,
2010.
12. W. H. Organization et al. Global tuberculosis report 2016. World Health
Organization, 2016.
13. S. Peleg, J. Naor, R. Hartley, and D. Avnir. Multiple resolution texture analysis and
classi cation. IEEE Transactions on Pattern Analysis and Machine Intelligence,
PAMI-6(