=Paper=
{{Paper
|id=Vol-1866/paper_146
|storemode=property
|title=ImageCLEF 2017: Supervoxels and Co-occurrence for Tuberculosis CT Image Classification
|pdfUrl=https://ceur-ws.org/Vol-1866/paper_146.pdf
|volume=Vol-1866
|authors=Vitali Liauchuk,Vassili Kovalev
|dblpUrl=https://dblp.org/rec/conf/clef/LiauchukK17
}}
==ImageCLEF 2017: Supervoxels and Co-occurrence for Tuberculosis CT Image Classification==
https://ceur-ws.org/Vol-1866/paper_146.pdf
ImageCLEF 2017: Supervoxels and
Co-occurrence for Tuberculosis CT Image
Classification
Vitali Liauchuk and Vassili Kovalev
United Institute of Informatics Problems, Minsk, Belarus
vitali.liauchuk@gmail.com
Abstract. The paper presents image description and classification meth-
ods which were used by United Institute of Informatics Problems (UIIP)
group for tuberculosis image classification task. A method based on co-
occurrence of adjacent supervoxels in 3D computed tomography (CT)
images was used for subtask #1 which was dedicated to image-based
recognition of multi-drug resistant tuberculosis. For subtask #2 which
is dedicated to automated categorization of tuberculosis patients into
one of five types of tuberculosis, extended multidimensional multi-sort
co-occurrence matrices were used for describing the CT scans. Both two
submitted runs were ranked 7th in both subtasks.
Keywords: Supervoxels, Dictionary, Co-occurrence,
Image Classification
1 Introduction
The tuberculosis task [1] of ImageCLEF 2017 Challenge [2] considers two sub-
tasks both dealing with 3D CT images. The subtask #1 is dedicated to the
problem of single image-based distinguishing between multi-drug resistant tu-
berculosis (MDR TB) cases and drug sensitive (DS) ones. The task itself is very
challenging and so far there are no techniques reported which allow robust and
accurate prediction of tuberculosis drug resistance based solely on lung CT im-
ages. Several studies reported the statistically significant links between presence
of visually detected radiological findings and drug resistance status [3, 4]. Also
some research was carried out to detect statistically significant links between
the drug resistance and structural features of radiological images of lung [5] and
some trials were made to assess the possible prediction accuracy [6]. However
in those studies the datasets used were not large enough and contained relapse
tuberculosis cases which have much higher probability of begin drug-resistant.
Instead, the dataset collected for ImageCLEF 2017 tuberculosis subtask #1 con-
tained only DS and MDR cases without transitional single-drug resistant and
poly-drug resistant ones and no relapses in order to make the dataset as unbi-
ased as possible. Training set in this subtask included 230 CT images: 134 drug
sensitive and 96 drug resistant cases. Test set consisted of 214 CT images and
was slightly biased towards MDR tuberculosis: 101 drug sensitive and 113 drug
resistant cases.
The subtask #2 of ImageCLEF 2017 tuberculosis task is aimed at auto-
matic categorization of CT images into one of five types of tuberculosis types:
Infiltrative, Focal, Tuberculoma, Miliary and Fibro-cavernous. Having 500 CT
scans in training set and 300 images in tests set, the subtask provides a valu-
able benchmark for computerized methods of CT image content description and
classification.
2 Data Preparation
2.1 Segmentation of lung regions
For extraction of lung regions in both subtasks, a domestic implementation of
a conventional approach of segmentation using non-rigid image registration was
used instead of the one proposed by the organizers. In our case the method
utilized 130 reference CT scans with manually segmented lungs. These 130 CT
scans represent completely separate dataset and have no respect to any of Im-
ageCLEF tuberculosis datasets. For each target CT scan, a similarity measure
was calculated between the target image and the reference images and top-5
most similar reference images were selected. The selected images along with the
corresponding lung masks were registered using ’elastix’ software tool [7], the
final segmentation mask was obtained by means of averaging. The implemented
method demonstrated high robustness to the presence of large lesion in lungs
(see Fig. 1).
Fig. 1. Example slices of CT images with segmented lungs.
2.2 Supplementary dataset for supervoxels dictionaries
For subtask #1 we used an image description method which operates with so-
called image supervoxels which are basically a 3D-version of conventional 2D
image superpixels [8]. The method considers categorization of image supervoxels
into classes according to a precalculated supervoxel dictionary similarly to the
well known bag-of-words approach [9]. For composing a meaningful supervoxels
dictionary, an auxiliary image dataset was composed. The dataset included 229
small 3D CT image regions of size 128×128×128 voxels extracted from CT
scans. Before extraction of regions, the original CT images were re-sampled using
nearest-neighbor interpolation in order to equalize sizes of voxel along all the
three axes, i.e. make them cubic-shaped (see examples in Fig. 2).
Fig. 2. Example slices of CT image regions with extracted supervoxels.
3 Subtask #1: Drug Resistance prediction
For subtask #1, a method for quantitative description of biomedical images
based on supervoxel representation and utilizing the co-occurrence concept was
used. To our best knowledge the potential of superpixel/supervoxel-based image
description has not been extensively researched yet [10].
3.1 General scheme of image description method
The proposed image description method includes two major stages: (a) gener-
ating a supervoxel dictionary and (b) describing the images using the obtained
dictionary. With this study, superpixel dictionaries were represented by the sets
of features of the most typical supervoxels occurring on the images of a given
type. The generating superpixel dictionary stage included the following steps:
– selection of a certain number of representative images of given type;
– extraction of supervoxels from the selected images;
– extraction of supervoxels features;
– splitting the supervoxels feature-space into N clusters;
– calculating cluster (class) centroids;
– composing the supervoxel dictionary (set of centroids).
The image description stage was based on calculation of histograms and co-
occurrence matrices of image supervoxels categorized into N classes according
to the previously obtained dictionary. This included the following:
– extraction of supervoxels from the target image;
– extraction of supervoxels features;
– categorization of each supervoxel into one of N classes according to the pre-
calculated dictionary;
– calculating a co-occurrence matrix [11] of the categorized supervoxels.
3.2 Composing supervoxel dictionary
In [12] the authors used superpixel dictionaries for semantic segmentation of
street images. A set of 1708 superpixel features including color, texture, shape
and location features was used for the task of scene description. In our study, we
used a set of 6 major supervoxel features which basically describe texture and
shape of a single supervoxel:
– mean intensity of internal pixels;
– standard deviation of intensity;
– entropy of intensity;
– mean gradient magnitude;
– sphericity;
– “cubeness”.
Sphericity was calculated using formula:
π 1/3 (6V )2/3
Sphericity = , (1)
A
where V is the total number of voxels in supervoxel, and A is the number of
border voxels in supervoxel. Sphericity value is close to 1 if the supervoxels
shape is similar to sphere. “Cubeness” feature expressed the extent of how much
the supervoxel shape is similar to cube and was calculated as:
V
Cubeness = , (2)
Vbb
where Vbb corresponds to number of voxels in bounding box of the supervoxel
considered. Maximum value of “cubeness” feature is 1 in case of ideally cubic-
shaped supervoxel. The meaning of this feature is dictated by the algorithm of
generation of supervoxels. At the initialization step, the shape of all supervoxels
is cubic. And if the underlying image substrate is enough homogeneous the resul-
tant supervoxels remain cubic-shaped. A 3D version of the superpixel generation
algorithm [8] used with this study has two control parameters: superpixel size
Fig. 3. Example supervoxels generated using different sets of parameters:
Sz = 32, Reg = 0.3 (left); Sz = 32, Reg = 1.0 (middle); Sz = 24, Reg = 0.3 (right).
Sz and a regularization parameter Reg. The examples of generated supervoxels
are shown in Fig. 3.
Supervoxel dictionaries were generated for several combinations of parame-
ters Sz and Reg. Supervoxel clustering was performed using k-means algorithm,
number of clusters being set to N = 8, 16, 32 and 64.
3.3 Image classification
The proposed method was finally applied to the training set images of Image-
CLEF tuberculosis subtask #1. The following combinations of control parame-
ters of supervoxel extraction algorithm were used: Sz = 32, Reg = 0.1; Sz = 32,
Reg = 0.3; Sz = 32, Reg = 1; Sz = 48, Reg = 0.1; Sz = 48, Reg = 0.3;
Sz = 48, Reg = 1. The supervoxel dictionary size N was set to 8, 16, 32 and
64. Supervoxel maps were calculated for all of the 230 training CT scans, class
numbers were assigned to supervoxels and for each image a supervoxels class
co-occurrence matrix was calculated which was further used as a feature vector
for prediction.
Only feature vector elements which were correlated with drug resistance at
significance level p < 0.05 were selected for further prediction. Finally, assess-
ment of patients drug resistance probability was performed with use of Logistic
Regression classifier within 5-fold cross-validation procedure. Area under ROC-
curve (AUC) was used as a measure of prediction performance. The results are
shown in Table 3.3. According to the table of results, the best prediction per-
formance was achieved for the following combination of parameters: Sz = 32,
Reg = 1 and size of supervoxel dictionary N = 64. However, it should be notices
that even the highest achieved performance is pretty low in general and with
230 study cases corresponds to statistical significance level of roughly p ∼ 0.01.
4 Subtask #2: tuberculosis type detection
For this subtask, an extended multi-sort, multi-dimensional co-occurrence ma-
trix approach was used which is proven to be powerful and flexible enough to
Table 1. Area under ROC-curve (AUC) for drug resistance prediction.
Supervoxel parameters N =8 N = 16 N = 32 N = 64
Sz = 32, Reg = 0.1 0.48 0.49 0.52 0.46
Sz = 32, Reg = 0.3 0.53 0.47 0.45 0.53
Sz = 32, Reg = 1.0 0.56 0.51 0.54 0.59
Sz = 48, Reg = 0.1 0.44 0.48 0.53 0.53
Sz = 48, Reg = 0.3 0.45 0.53 0.58 0.53
Sz = 48, Reg = 1.0 0.46 0.47 0.44 0.52
capture a broad range of structural properties of both 2D and 3D medical im-
ages [13]. For describing lung CT image structure we employed the most general,
six-dimensional matrices of IIGGAD type counting voxel pairs with certain in-
tensities (I), gradient magnitudes (G), and mutual angles (A) between the gra-
dient vectors. CT image intensity range from -1000 to +500 Hounsfield Units
(HU) was quantized to 12 bins. Gradient values and angles between gradients
were both quantized to 6 bins. The matrices consider co-occurrence of voxels in
3D on distances from 1 to 5 measured in axial voxel size. The values of algorithm
parameters were selected empirically to maximize classification performance on
training set of images. The resultant multidimensional co-occurrence matrices
contained in total 122 × 62 × 6 × 5 = 155 520 elements, most of which were
however zeros.
To reduce the dimensionality of the feature space, a Principal Component
Analysis method (PCA) was applied and the first 50 PC’s were considered for
further analysis. The prediction of patient’s tuberculosis type was performed
with use of Random Forests classifier.
The evaluation of the proposed image classification method within 5-fold
cross-validation procedure demonstrated classification accuracy of 57.0% and
the un-weighted Cohens Kappa coefficient value was 0.442.
5 Submission and results
Since subtask #1 represent a very challenging and important problem, most of
the efforts of our team was focused on drug resistance prediction subtask. Several
different approaches were tested and various descriptor types were examined. Al-
gorithms of automated detection of lesions [14] were used to derive additional
information on the affected level of lungs. Some trials were made to utilize Deep
Learning classification methods to distinguish between DS and MDR tubercu-
losis lesion appearance but no success was achieved there. Finally, since none
of the additional approaches provided any significant increase of classification
performance, only one run was submitted for subtask #1. In case of subtask #2,
one single run was submitted as well.
In the final table of results the submitted run for subtask #1 was ranked
as 7th among the 28 submitted runs with area under ROC-curve (AUC) equal
to 0.5415 and prediction accuracy of 49.3% on the test image dataset. The best
result in terms of AUC value was achieved by MedGIFT team [15] and resulted in
AUC = 0.5825. Best drug resistance prediction accuracy of 56.8% was achieved
by HHU DBS team [16].
For subtask #2, the submitted run was ranked 7th among the total number
of 23 runs with recognition accuracy of 39.0% and Cohen’s Kappa equal to
0.196. The best results in this subtask were obtained by SGEast team [17] which
corresponded to Cohen’s Kappa value of 0.2438 and recognition accuracy of
40.3%.
6 Conclusions
In this paper, image classification methods employed by UIIP group for Im-
ageCLEF 2017 tuberculosis task were described. Being tested within the two
subtasks, image description methods based on co-occurrence of voxels and su-
pervoxels proved to be efficient for description of textural appearance of 3D
medical images. It should be noticed that even the top-performing run submit-
ted for subtask #1 resulted in AUC = 0.59 which is pretty low in terms of
classification task. The conclusion which can be drawn is that the task of predic-
tion of tuberculosis drug resistance based on a single CT image probably cannot
be solved with a reasonable accuracy.
Acknowledgements
This study was supported by the National Institute of Allergy and Infectious
Diseases, National Institutes of Health, U.S. Department of Health and Human
Services, USA through the CRDF project OISE-16-62631-1.
References
1. Dicente Cid, Y., Kalinovsky, A., Liauchuk, V., Kovalev, V., , Müller, H.: Overview
of ImageCLEFtuberculosis 2017 - predicting tuberculosis type and drug resistances.
In: CLEF2017 Working Notes. CEUR Workshop Proceedings, Dublin, Ireland,
CEUR-WS.org (September 11-14 2017)
2. Ionescu, B., Müller, H., Villegas, M., Arenas, H., Boato, G., Dang-Nguyen, D.T.,
Dicente Cid, Y., Eickhoff, C., Garcia Seco de Herrera, A., Gurrin, C., Islam, B.,
Kovalev, V., Liauchuk, V., Mothe, J., Piras, L., Riegler, M., Schwall, I.: Overview
of ImageCLEF 2017: Information extraction from images. In: Experimental IR
Meets Multilinguality, Multimodality, and Interaction 8th International Conference
of the CLEF Association, CLEF 2017. Volume 10456 of Lecture Notes in Computer
Science., Dublin, Ireland, Springer (September 11-14 2017)
3. Cha, J., Lee, H.Y., Lee, K.S., Koh, W.J., Kwon, O.J., Yi, C.A., Kim, T.S., Chung,
M.J.: Radiological findings of extensively drug-resistant pulmonary tuberculosis
in non-AIDS adults: comparisons with findings of multidrug-resistant and drug-
sensitive tuberculosis. Korean journal of radiology 10(3) (2009) 207–216
4. Lee, E.S., Park, C.M., Goo, J.M., Yim, J.J., Kim, H.R., Lee, H.J., Lee, I.S., Im,
J.G.: Computed tomography features of extensively drug-resistant pulmonary tu-
berculosis in non-HIV-infected patients. Journal of computer assisted tomography
34(4) (2010) 559–563
5. Kovalev, V., Liauchuk, V., Safonau, I., Astrauko, A., Skrahina, A., Tarasau, A.:
Is there any correlation between the drug resistance and structural features of
radiological images of lung tuberculosis patients? In: Computer Assisted Radiology
- 27th International Congress and Exhibition (CARS-2013). Volume 8., Springer,
Heidelberg (2013) 18–20
6. Kovalev, V., Liauchuk, V., Kalinouski, A., Rosenthal, A., Gabrielian, A., Skrahina,
A., Astrauko, A., Tarasau: Utilizing radiological images for predicting drug resis-
tance of lung tuberculosis. In: Computer Assisted Radiology - 27th International
Congress and Exhibition (CARS-2015). Volume 10., Springer, Barcelona (2015)
129–130
7. Klein, S., Staring, M., Murphy, K., Viergever, M.A., Pluim, J.P.: Elastix: a toolbox
for intensity–based medical image registration. IEEE Transactions on medical
imaging 29(1) (2010) 196–205
8. Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: SLIC super-
pixels compared to state–of–the–art superpixel methods. IEEE Transactions on
Pattern Analysis and Machine Intelligence 34(11) (November 2012) 2274–2282
9. Yang, J., Jiang, Y.G., Hauptmann, A.G., Ngo, C.W.: Evaluating bag–of–visual–
words representations in scene classification. In: Proceedings of the international
workshop on Workshop on multimedia information retrieval, ACM (2007) 197–206
10. Sicre, R., Tasli, E., Gevers, T.: Superpixel based angular differences as a mid-
level image descriptor. In: 22nd International Conference on Pattern Recognition
(ICPR-2014), Stockholm, Sweden (2014) 3732–3737
11. Micusik, B., Kosecka, J.: Semantic segmentation of street scenes by superpixel
co-occurrence and 3d geometry. In: IEEE Workshop on Video-Oriented Object
and Event Classification, Japan (2009) 625–632
12. Tighe, J., Lazebnik, S.: Scalable nonparametric image parsing with superpixels.
In: Proceedings of the 11th European conference on Computer vision (ECCV’10),
Germany, Heidelberg (2010) 352–365
13. Kovalev, V.A., Kruggel, F., Gertz, H.J., von Cramon, D.Y.: Three–dimensional
texture analysis of MRI brain datasets. IEEE Transactions on Medical Imaging
20(5) (May 2001) 424–433
14. Kalinovsky, A., Liauchuk, V., Tarasau, A.: Lesion detection in ct images using deep
learning semantic segmentation technique. ISPRS - International Archives of the
Photogrammetry, Remote Sensing and Spatial Information Sciences XLII-2/W4
(2017) 13–17
15. Dicente Cid, Y., Batmanghelich, K., Müller, H.: Textured graph-model of the lungs
for tuberculosis type classification and drug resistance prediction: participation in
ImageCLEF 2017. In: CLEF2017 Working Notes. CEUR Workshop Proceedings,
Dublin, Ireland, CEUR-WS.org (September 11-14 2017)
16. Braun, D., Singhof, M., Tatusch, M., Conrad, S.: Convolutional neural networks
for multidrug-resistant and drug-sensitive tuberculosis dinstinction. In: CLEF2017
Working Notes. CEUR Workshop Proceedings, Dublin, Ireland, CEUR-WS.org
(September 11-14 2017)
17. Sun, J., Chong, P., Tan, Y.X.M., Binder, A.: ImageCLEF 2017: ImageCLEF tuber-
culosis task - the SGEast submission. In: CLEF2017 Working Notes. CEUR Work-
shop Proceedings, Dublin, Ireland, CEUR-WS.org (Septem-
ber 11-14 2017)