Nowadays tuberculosis is still a widespread disease that causes worldwide more than one million deaths and ten million new infections every year. As part of ImageCLEF 2018, we investigated whether the severity of the disease can be determined from CT scans, only. We therefore extracted features from the images which we then tested with several classi ers. Afterwards we chose the best combinations of di erent feature sets and classi cation models. Our best approach is based on three features, namely cavitation, cavity tissue, and infection ratio. Combined with random forests we achieved rank 10 regarding the RMSE measure.
In 2018 tuberculosis was listed as one of the top ten causes of death worldwide [
In this paper we present a feature-based approach for severity scoring of
lung tuberculosis exclusively based on CT scans. This work is a contribution
to the severity score tuberculosis task of ImageCLEF 2018 [
We extracted features from CT images assuming that medical experts look for
irregularities in the lungs that are typical for tuberculosis while analyzing CT
scans in the context of severity score determination. In the medical literature,
di erent kinds of irregularities and lesions in the lung associated with pulmonary
tuberculosis are described. Our feature choice is for the most part based on this
description. In this section, feature extraction methods are described that we
used in our approach. Since the most of our feature extraction methods worked
on binary images, we binarized all CT images using IsoData method [
Calci cation is signi cant to the disease pattern of tuberculosis[
boundary[i] = mask[i] erode(mask[i]); with i 2 f0; slices(mask)g 2.2 Boundary extraction from CT scans:
bscan[i] = boundary[i] scan[i] 2.3 Adaption of masks: while (max(bscan[i] ) 700) do mask[i] = erode(mask[i] )
Step 2.2 end while (3) Summation of pixels with value
700 in mask[i] scan[i]
With erode [
Water accumulation is a potential concomitant of an existing tuberculosis
disease. Nevertheless we assumed that an existing tuberculosis infection weakens
the immune system of the patient and may lead to trace-diseases. There are
several indispositions that are associated with uid retention within the lungs or
in the pleural space between the lungs and the ribs, such as pleural e usion[
One of the classic indicators of lung tuberculosis are the pulmonary cavities
which occur in 50 percent of patients [
We extracted the pulmonary cavities from single CT image slices as dark
spots completely surrounded by light tissue. Since we wanted to avoid nding
(a)
(b)
similar structures in other parts of CT images than lungs, we used the lung masks
that were provided by the organizers of the task using the approach described
in [
After processing the lung masks we performed the pulmonary cavitation search in binarized CT images as follows: First, we removed all objects smaller than 20 pixels because it is unlikely for a cavitation to be of such small size and analyzing such objects would unnecessarily require processing time. Since bronchioles scanned across and shadows caused by breathing and body movements could be falsely recognized as cavities, in the second step, we closed all holes that were smaller than two pixels. Obviously, the internal parts of undesired objects could be larger than 2 pixels, but, on the other side, we had to prevent erroneously discarding parts of real cavities. Because cavity walls are usually thicker than bronchiole walls, in the third step, we performed morphological opening with a 2 2 square to discard undesired objects remained after the second step. We considered all holes that were completely surrounded by walls as pulmonary cavities after performing these three preprocessing steps. For performance reasons we estimated the volumes of pulmonary cavities by simply summing up the pixels of found cavities and cavity walls, respectively, over all CT scan slices in the le. 2.4
Pulmonary tuberculosis is an infectious lung disease whose bacilli spread through
the lungs and cause lung tissue damage. Depending on the type of tuberculosis
di erent types of lesions occur in the lungs (see Figure 2). That makes it di cult
to estimate the amount of the a ected part of the lungs automatically. Since the
infection of the most tuberculosis types cause a thickening of the lung tissue
which can be recognized in CT scans, we simply estimated the ratio of the lung
tissue to the entire lung volume. In our approach we did not di erentiate between
healthy and a ected lung tissue which is a di cult task, our approach is based
on the assumption that the lung tissue ratio compared to the lung volume is
smaller in healthy persons than in persons su ering from tuberculosis. In order
to highlight the lung lesions, we rst binarized the CT images as described above.
After the binarization we simply counted the number of white pixels and related
it to the number of pixels in the lung mask [
In order to overcome this problem we decided to compare intervals of Hounseld Units with the help of histograms. As the intervals of Houns eld Units of di erent tissues overlap, it is di cult to determine reasonable bins. Therefore we divided the interval of [ 1024; 3000] into 20 equal sized bins. In the classi cation task every bin has been regarded as a single feature. Since we assume that the degree of pulmonary tuberculosis can correlate with the overall health of a patient, we considered the shape of the lungs, as well. We also assume that the di erence between the shapes of the two lungs can provide information about the patient's health. To obtain a comparison measure, it is su cient to look at the masks.
Note, that the following procedure needs all slices to be processed separately.
In order to compare the di erent lungs, all masks have to be divided into two
separate lung masks. Afterwards the contours of the relevant regions are
calculated. In [
7
I(A; B) = X jmiA
miBj
(1)
with
miA = sign(hiA) log(hiA) and
miB = sign(hiB) log(hiB) ;
where hiA and hiB are the Hu Moments of the contours A and B [
Now there are comparison values for each slice of the CT Scan but it is desired to receive one representative measure for the match. So nally, the average value over all slices has to be calculated. 3
We used di erent classi ers for predicting the severity score and the severity level of tuberculosis using the feature set obtained from the feature extraction step described in Section 2. In the training phase of classi cation models, we performed feature selection based on the cross-validation mean square error for severity score on the training set. We tried di erent classi cation methods including the multi-class support vector machine (SVM) with RBF kernel, the k-nearest-neighbor (kNN) algorithm, and the multi-layer perceptron classi er with di erent parameter settings. Below we describe the best classi cation models with respect to the mean square error on the training set and the way of predicting the severity score and the severity level of tuberculosis. 3.1
Using the Chi Square method it turned out that 13 of the 17 features are the most meaningful. Apparently the histograms of the higher Houns eld Units are not very informative. Therefore, all features have been considered for the decision tree, except for the histograms of ranges with values greater than 50.
In Figure 3 the structure of the resulting decision tree is demonstrated. The numbers followed by an H stand for the ranges of the histograms. The assigned classes of the leaf nodes are highlighted in blue. Arrows to the left indicate that the condition of the parent node applies. Arrows to the right represent the non-applicability. The classi er was tted using the Gini Impurity, a minimum fraction of 3% per leaf and a minimum quality gain of 0:01. The structure shows that most scores of the same severity class ("LOW"/"HIGH") share similar features. But the scores 3 and 4 are often separated by only one property, as well. According to the paths of the decision tree, the distinction between score 3 and 4 is the most di cult task and probably represents the biggest source of errors regarding the AUC score, since both degrees belong to di erent severity classes. 3.2
Using random forests and linear regression as classi ers we interpreted the prediction of the severity score and the severity level of tuberculosis as a regression problem. In our rst attempt, we converted the severity level into the numbers 0 Inf Ratio < 0.7631 3 4 Inner Cav <
156 2
1 Outer Cav <
6897 1 2
Outer Cav <
9481 1 and 1, where 1 means "HIGH" and 0 means "LOW" severity level. We used the random forest classi er with the maximum depth value of 2 because the larger values led to over tting of the classi cation model. We calculated the severity scores from the predicted severity levels by dividing the severity level values into intervals.
In the second test, we trained two separate classi cation models for the severity score and the severity level prediction. Since the severity score and the severity level values mismatched for some data items, we adjusted the severity score values depending on the corresponding severity level values at the extreme boundaries of the severity level. In particular, we set the severity score values to 1 if the corresponding severity level values were higher than 0.95. If the severity level values were below 0.22, we set the corresponding severity score values to 5 regardless the values that were predicted for the severity score before. Here, we also used the random forest classi er with the maximum depth value of 3. Additionally, we submitted a linear regression model which we trained on a subset of the training set for which we achieved the best results with respect to the cross-validation mean square error for the severity score. We used only a subset of the training set because linear regression is sensitive to outliers that we assumed in the training set due to variations in the mean square errors in di erent cross-validation runs.
Due to performance variations for the severity level in di erent cross-validation runs for the random forest model trained in the second test, we assumed that the classi cation model for prediction of the severity level over tted on the training set. Therefore, in the third test, we reduced the maximum depth value to 2 for the random forest model. Furthermore, we refrained from the adjustment of the severity score values based on the severity level. 4
A maximum of 10 runs could be submitted by each group per subtask in the ImageCLEF 2018 TB task. This section shows the nal performance results of our feature-based approach in the severity scoring challenge (subtask 3). The nal ranking was based on the root mean square error (RMSE) for the severity score. Table 1 summarizes the results for di erent runs of our approach ordered by the ranking provided by the subtask organizers. Additionally, we listed the results of the best runs with respect to the root mean square error (RMSE) and the Area Under the ROC Curve (AUC) submitted in the competition.
The best run of our approach was obtained by the random forest classi cation model with severity score adjustment described in the second test in Section 3.2 (indicated as Rnd Frst depth 3 in the table) using only three features: size of cavity, size of cavity tissue, and the infection ratio. Although our best run achieved the tenth rank regarding the RMSE measure, it was ranked sixteenth according to the AUC measure. In order to improve the results regarding the AUC measure too, we performed feature selection based on the cross-validation AUC value for the severity level on the training set using the same classi cation method. The so selected features were calci cation, infection ratio, size of cavity, and the third, the sixth and the tenth bins of the histogram. Although this run was only ranked on the 25th placed regarding the RMSE, it achieved the eight place regarding the AUC measure.
Our second best run was achieved by the random forest classi cation model with the maximum depth value of 2 without severity score adjustment and the linear regression model trained on the subset of the training set (indicated as Rnd Frst depth 2 and Lin Reg part in the table) on the same feature subset as our best run. The performance results of our approach that calculated the severity score from the predicted severity level values by the random forest classication model (indicated as Rnd Frst score by level in the table) were the worst among the regression based approaches described in Section 3.2. Although the AUC value for the severity level was the same as for our best run, the RMSE value for the severity score calculated based on the severity level was much worse than for the separate severity scores prediction model.
The decision trees unfortunately performed worst. They only achieved the ranks 32-34 regarding the RMSE measure. The severity class was determined on the basis of the received scores. For this, two methods were used. In the rst approach, the values 1, 2 and 3 represent the class "HIGH", and 4 and 5 belong to class "LOW". In the other method the probability p of a high severity was calculated by the formula p = 5 y^
4 , where y^ stands for the predicted severity score of the decision tree. The results showed that the rst method scored signi cantly better AUC values. Our best decision tree even reached the ninth rank in regard to the AUC measure. { { 0.7840 1 { { 0.8934 5 Rnd Frst depth 3 cav., cav. tissue, inf. ratio 0.9626 10 Rnd Frst depth 2 cav., cav. tissue, inf. ratio 0.9768 13 Lin Reg part cav., cav. tissue, inf. ratio 0.9768 14 Rnd Frst depth 3 calc., inf. ratio, cav., 1.1046 25
hist. bins 3,6,10
Rnd Frst score by level cav., cav. tissue, inf. ratio 1.2040 29
In this paper we have shown that our feature-based approach is competitive to
other participants of the ImageCLEF 2018 challenge [
Finally, the feature choice in our approach was for the most part based on own observations that could be approved by medical studies published in the literature in recent years. We believe that we could improve the feature extraction and consequently the nal results of our approach by consulting medical experts specialized in treatment of pulmonary tuberculosis.