The paper presents image description and classi cation methods which were used by United Institute of Informatics Problems (UIIP) group for tuberculosis image classi cation task. A method based on cooccurrence of adjacent supervoxels in 3D computed tomography (CT) images was used for subtask #1 which was dedicated to image-based recognition of multi-drug resistant tuberculosis. For subtask #2 which is dedicated to automated categorization of tuberculosis patients into one of ve types of tuberculosis, extended multidimensional multi-sort co-occurrence matrices were used for describing the CT scans. Both two submitted runs were ranked 7th in both subtasks.
The tuberculosis task [
The subtask #2 of ImageCLEF 2017 tuberculosis task is aimed at automatic categorization of CT images into one of ve types of tuberculosis types: In ltrative, Focal, Tuberculoma, Miliary and Fibro-cavernous. Having 500 CT scans in training set and 300 images in tests set, the subtask provides a valuable benchmark for computerized methods of CT image content description and classi cation. 2 2.1
For extraction of lung regions in both subtasks, a domestic implementation of
a conventional approach of segmentation using non-rigid image registration was
used instead of the one proposed by the organizers. In our case the method
utilized 130 reference CT scans with manually segmented lungs. These 130 CT
scans represent completely separate dataset and have no respect to any of
ImageCLEF tuberculosis datasets. For each target CT scan, a similarity measure
was calculated between the target image and the reference images and top-5
most similar reference images were selected. The selected images along with the
corresponding lung masks were registered using 'elastix' software tool [
For subtask #1, a method for quantitative description of biomedical images
based on supervoxel representation and utilizing the co-occurrence concept was
used. To our best knowledge the potential of superpixel/supervoxel-based image
description has not been extensively researched yet [
The proposed image description method includes two major stages: (a) generating a supervoxel dictionary and (b) describing the images using the obtained dictionary. With this study, superpixel dictionaries were represented by the sets of features of the most typical supervoxels occurring on the images of a given type. The generating superpixel dictionary stage included the following steps: { selection of a certain number of representative images of given type; { extraction of supervoxels from the selected images; { extraction of supervoxels features; { splitting the supervoxels feature-space into N clusters; { calculating cluster (class) centroids; { composing the supervoxel dictionary (set of centroids).
The image description stage was based on calculation of histograms and
cooccurrence matrices of image supervoxels categorized into N classes according
to the previously obtained dictionary. This included the following:
{ extraction of supervoxels from the target image;
{ extraction of supervoxels features;
{ categorization of each supervoxel into one of N classes according to the
precalculated dictionary;
{ calculating a co-occurrence matrix [
In [
Sphericity was calculated using formula: (1) (2) Sphericity = 1=3(6V )2=3
A ; where V is the total number of voxels in supervoxel, and A is the number of border voxels in supervoxel. Sphericity value is close to 1 if the supervoxels shape is similar to sphere. \Cubeness" feature expressed the extent of how much the supervoxel shape is similar to cube and was calculated as:
Cubeness =
V
Vbb
;
where Vbb corresponds to number of voxels in bounding box of the supervoxel
considered. Maximum value of \cubeness" feature is 1 in case of ideally
cubicshaped supervoxel. The meaning of this feature is dictated by the algorithm of
generation of supervoxels. At the initialization step, the shape of all supervoxels
is cubic. And if the underlying image substrate is enough homogeneous the
resultant supervoxels remain cubic-shaped. A 3D version of the superpixel generation
algorithm [
Supervoxel dictionaries were generated for several combinations of parameters Sz and Reg. Supervoxel clustering was performed using k-means algorithm, number of clusters being set to N = 8; 16; 32 and 64. The proposed method was nally applied to the training set images of ImageCLEF tuberculosis subtask #1. The following combinations of control parameters of supervoxel extraction algorithm were used: Sz = 32, Reg = 0:1; Sz = 32, Reg = 0:3; Sz = 32, Reg = 1; Sz = 48, Reg = 0:1; Sz = 48, Reg = 0:3; Sz = 48, Reg = 1. The supervoxel dictionary size N was set to 8, 16, 32 and 64. Supervoxel maps were calculated for all of the 230 training CT scans, class numbers were assigned to supervoxels and for each image a supervoxels class co-occurrence matrix was calculated which was further used as a feature vector for prediction.
Only feature vector elements which were correlated with drug resistance at signi cance level p < 0:05 were selected for further prediction. Finally, assessment of patients drug resistance probability was performed with use of Logistic Regression classi er within 5-fold cross-validation procedure. Area under ROCcurve (AUC) was used as a measure of prediction performance. The results are shown in Table 3.3. According to the table of results, the best prediction performance was achieved for the following combination of parameters: Sz = 32, Reg = 1 and size of supervoxel dictionary N = 64. However, it should be notices that even the highest achieved performance is pretty low in general and with 230 study cases corresponds to statistical signi cance level of roughly p 0:01. 4
For this subtask, an extended multi-sort, multi-dimensional co-occurrence
matrix approach was used which is proven to be powerful and exible enough to
capture a broad range of structural properties of both 2D and 3D medical
images [
To reduce the dimensionality of the feature space, a Principal Component Analysis method (PCA) was applied and the rst 50 PC's were considered for further analysis. The prediction of patient's tuberculosis type was performed with use of Random Forests classi er.
The evaluation of the proposed image classi cation method within 5-fold cross-validation procedure demonstrated classi cation accuracy of 57.0% and the un-weighted Cohens Kappa coe cient value was 0.442. 5
Since subtask #1 represent a very challenging and important problem, most of
the e orts of our team was focused on drug resistance prediction subtask. Several
di erent approaches were tested and various descriptor types were examined.
Algorithms of automated detection of lesions [
In the nal table of results the submitted run for subtask #1 was ranked
as 7th among the 28 submitted runs with area under ROC-curve (AUC) equal
to 0.5415 and prediction accuracy of 49.3% on the test image dataset. The best
result in terms of AUC value was achieved by MedGIFT team [
For subtask #2, the submitted run was ranked 7th among the total number
of 23 runs with recognition accuracy of 39.0% and Cohen's Kappa equal to
0.196. The best results in this subtask were obtained by SGEast team [
In this paper, image classi cation methods employed by UIIP group for ImageCLEF 2017 tuberculosis task were described. Being tested within the two subtasks, image description methods based on co-occurrence of voxels and supervoxels proved to be e cient for description of textural appearance of 3D medical images. It should be noticed that even the top-performing run submitted for subtask #1 resulted in AUC = 0.59 which is pretty low in terms of classi cation task. The conclusion which can be drawn is that the task of prediction of tuberculosis drug resistance based on a single CT image probably cannot be solved with a reasonable accuracy.
This study was supported by the National Institute of Allergy and Infectious Diseases, National Institutes of Health, U.S. Department of Health and Human Services, USA through the CRDF project OISE-16-62631-1.