National Library of Medicine (NLM) at ImageCLEF2015: Medical Clustering Task

Abstract

Besides recognizing medical image modalities, such as Xrays, MRIs, histology images, uorescence microscopy images, endoscopy images, photos, illustrations, etc., the detection of visual content is equally important. Once the main modality class is detected, a modality such as Xray can be broken to di erent sub-classes representing di erent body parts such as arms, legs, neck, torso, etc. Such a classi cation can further help the current image-based search engines to return appropriate results based on visual content similarity. For our participation in the ImageCLEF2015 Medical image clustering task, we implemented a classi cation scheme based on a neural network using two di erent feature collections { which proved their value in object recognition and chest X-ray analysis.

Keywords

• body parts x-ray
• classi cation
• shape features
• texture features
• modality detection
• ImageCLEF

-

Medical image retrieval in the context of large collections is a challenging and demanding task [16,17]. Increased research interest has resulted in years long systematic evaluation e orts1. One outcome of the evaluation was that knowing the modality of an image, i.e., whether it is an X-ray, CT, MRI or a photograph, radically improves the performance of image retrieval [11,19]. This section describes the motivation for the particular feature sets in use, provides a brief description of the di erent features, and nally has a short section describing the classi er. 1 http://www.imageclef.org 2.1

Description of features

To characterize the di erent body parts in the X-ray images, we considered two di erent feature sets. Feature Set A is inspired from object detection [7,12], and was used with success in a previous work [9] to detect pulmonary abnormalities in frontal chest X-ray images. Feature Set B has been utilized with success in [14] for a medical CBIR system. These features cover a large number of properties, such as color distribution, edginess, texture, curvatures, pixel densities, shape, and other measures necessary to describe images such as in Figure 1. We note that the body parts in the X-ray images appear in in di erent size, rotation, shape, etc. Therefore, features invariant to size and rotation or shape are most appropriate.

Set A: Is a versatile and compact feature set combining shape, edge and texture descriptors. The nal feature representation is built by concatenating the different descriptors (histograms) extracted from the segmented lung regions. In particular, in Set A, the following shape and texture descriptors were considered: Intensity Histogram (IH), Gradient Magnitude Histogram (GM), Shape Descriptor Histogram (SD), Curvature Descriptor Histogram (CD), Histogram of Oriented Gradient (HOG) [ 5 ], Local Binary Pattern (LBP) [13]. A modi ed multiscale approach proposed by Frangi et al. [ 6 ] is considered to compute the eigenvalues of Hessian matrix needed for the shape and curvature descriptors. The Hessian describes the second-order surface curvature properties of the local image intensity surface. The normalization makes these descriptors intensity invariant. In [9] we determined that quantizing these features into 32 bins provides good discrimination performance. The size of the feature descriptor is 192. Set B : Is a diversi ed, low-level feature collection involving intensity, edge, texture, color and shape moment features. The feature representation is built by concatenating the di erent descriptors (histograms) extracted from the segmented lung regions. In particular, the following descriptors were considered: Color Layout Descriptor (CLD), Edge Histogram Descriptor (EHD) from MPEG-7 standard [10], Color and Edge Direction Descriptor (CEDD) [ 3 ], Fuzzy Color and Texture Histogram (FCTH) [ 4 ] Tamura texture descriptor, Gabor texture feature [8], and other texture features such as primitive length, edge frequency, and autocorrelation [15]. This feature set comprises 595 dimensions. Set C : Is a union of set A and set B. Even though some of the features are similar or similar characteristics, this extended feature collection can be a powerful descriptor for such particular type of X-ray images of di erent body parts, as discussed in [9]. Practically the features were stitched together to form a larger feature descriptor. The dimension of this feature descriptor is 787. 2.2

Feature classi cation

For classi cation a neural network-based classi er was used. Neural networks in particular are known for their capability of estimating complex decision surfaces [ 2 ] and handling multi-class problems. Due to the large numbers of features to be handled (up to 787 dimensions), and the lack of information about the possible correlations among the di erent feature components, a fully connected multi-layer perceptron network was utilized. The number of neurons in the input layer was selected based on the dimensionality of the input feature vector. The number of output neurons was also set based on the possible outcomes: head-neck, upper-limb, body, and lower-limb, while the number of neurons in the hidden layer was estimated based on several trial runs. Finally, for the experiments 15 neurons were considered as being optimal in the hidden layer. For training error-backpropagation strategy was considered, while for learning rate = 0.004, and momentum = 0.3 were used. The di erent parameters were established based on several trial runs. For the number of hidden neurons in the hidden layer, we considered the criteria to have as less possible neuron to keep the complexity low. Therefore, the recognition time became faster. 3

Experiments

This section gives a brief description of the data followed by the description of the evaluation protocols, and the di erent results obtained by the neural classi er utilizing the di erent feature collections. 3.1

Data description

The data provided by the ImageCLEF2015 [18] Medical image clustering task organizers contains 500 X-ray images of variable sizes [ 1 ]. The content of the image is equally distributed among X-rays containing head-neck, upper-limb, body, and lower-limb images. An equal number of 100 images of true negatives - containing completely di erent images, were also provided to help the researchers providing negative examples to their systems. Some images from the data collection are shown in Figure 1. For more details about the data, please refer to [ 1 ]. 3.2

Evaluation protocols

The accuracy (ACC) was measured to properly evaluate the performance of the method. Each of our experiments follows a 10x cross-validation protocol, and the reported results are the average scores of the di erent folds. However, the competition organizers used three di erent measures, namely the exact match, any match and the Hamming distance [ 1 ]. 3.3

Results

Using the previously mentioned feature set A, B and the C, three di erent experiments were conducted. The rst results are presented in Table 1.

One can observe the superiority of the Set C, which contains both features from set A and Set B, respectively. It is quite important to mention the fact, that color features such as CEDD, FCTH also contribute to the higher accuracy of the system, whilst, features such as intensity histogram, LBP and HOG, which (a) Body (b) Head Neck (c) Upper Limb (d) Lower Limb (f) Body (g) Head Neck (h) Upper Limb (i) Lower Limb (j) Other describe the edginess and texture of the image, perform the worse; even though they are rotation invariant, except HOG. The results shown in Table 1. are reported using the 10x cross-validation protocol on the training samples (labels available) provided originally by the organizers.

The results reported in Table 2. were generated based on the test set (no labels available) provided by the competition organizers. This image collection contains 250 samples, similar to the training material, equally distributed among the 5 classes. To train the neural network all data (500 samples) available in the training set were considered. Despite the usage of di erent metrics for evaluating the performance of this experiment, the rank among the features is preserved, as expected.

Conclusion

In this paper we introduced three di erent feature sets (A, B and C) applied with sucess in pulmonary disease detection in chest X-ray images, and image retrieval and modality detection. It is interesting to note that features such as intensity histogram or histogram of oriented gradient or localy binary patterns among others belonging to set A were less e ective than generic features such as Fuzzy Color and Texture histogram, Tamura texture feature, Gabor texture features, Edge histogram descriptor and Color Layout descriptor coming from feature set B. The combination of these features into a larger set C allowed us to classify quite e ectively the di erent body parts from the X-ray images.

Among the 7 participants in the contest (see detailed description in [ 1 ]) we ended up on 4th place, while among the 29 runs submitted by the di erent research groups, our runs nished at 11th,17th, and 25th, respectively.

To further improve the classi cation scores, we envision an exhaustive feature selection mechanism applied to the set C, to eliminate those features which are rotation variant such as HOG, and other feature components which do not contribute much to the nal scores.

Acknowledgment

This research is supported by the Intramural Research Program of the National Institutes of Health (NIH), National Library of Medicine, and Lister Hill National Center for Biomedical Communications (LHNCBC). 7. Gonzalez, R.C., Woods, R.E.: Digital Image Processing (3rd Edition). Prentice-Hall,

Inc., Upper Saddle River, NJ, USA (2006) 8. Howarth, P., Yavlinsky, A., Heesch, D., Ruger, S.: Medical image retrieval using texture, locality and colour. In: Peters, C., Clough, P., Gonzalo, J., Jones, G., Kluck, M., Magnini, B. (eds.) Multilingual Information Access for Text, Speech and Images. Lecture Notes in Computer Science, vol. 3491, pp. 740{749. Springer Berlin Heidelberg (2005) 9. Jaeger, S., Karargyris, A., Candemir, S., Folio, L., Siegelman, J., Callaghan, F.M., Xue, Z., Palaniappan, K., Singh, R.K., Antani, S., Thoma, G.R., Wang, Y., Lu, P., McDonald, C.J.: Automatic tuberculosis screening using chest radiographs. IEEE Trans. Med. Imaging 33(2), 233{245 (2014) 10. Lux, M.: Caliph & emir: Mpeg-7 photo annotation and retrieval. In: Proceedings of the 17th ACM International Conference on Multimedia. pp. 925{926. MM '09, ACM, New York, NY, USA (2009) 11. Muller, H., Kalpathy-Cramer, J., Demner-Fushman, D., Antani, S.: Creating a classi cation of image types in the medical literature for visual categorization. In: SPIE medical imaging (2012) 12. Murphy, K.P., Torralba, A., Eaton, D., Freeman, W.T.: Object detection and localization using local and global features. In: Toward Category-Level Object Recognition. pp. 382{400 (2006) 13. Ojala, T., Pietikainen, M., Harwood, D.: A comparative study of texture measures with classi cation based on featured distributions. Pattern Recognition 29(1), 51{59 (1996) 14. Rahman, M.M., You, D., Simpson, M.S., Antani, S., Demner-Fushman, D., Thoma, G.R.: Interactive cross and multimodal biomedical image retrieval based on automatic region-of-interest (ROI) identi cation and classi cation. IJMIR 3(3), 131{146 (2014) 15. Singh, S., Sharma, M.: Texture analysis experiments with meastex and vistex benchmarks. In: Singh, S., Murshed, N., Kropatsch, W. (eds.) Advances in Pattern Recognition ICAPR 2001. Lecture Notes in Computer Science, vol. 2013, pp. 419{426. Springer Berlin Heidelberg (2001) 16. Vajda, S., You, D., Antani, S., Thoma, G.R.: Label the many with a few: Semiautomatic medical image modality discovery in a large image collection. In: 2014 IEEE Symposium on Computational Intelligence in Healthcare and e-health, CICARE 2014, Orlando, FL, USA, December 9-12, 2014. pp. 167{173 (2014) 17. Vajda, S., You, D., Antani, S., Thoma, G.: Large image modality labeling initiative using semi-supervised and optimized clustering. International Journal of Multimedia Information Retrieval 4(2), 143{151 (2015) 18. Villegas, M., Muller, H., Gilbert, A., Piras, L., Wang, J., Mikolajczyk, K., de Herrera, A.G.S., Bromuri, S., Amin, M.A., Mohammed, M.K., Acar, B., Uskudarli, S., Marvasti, N.B., Aldana, J.F., del Mar Roldan Garc a, M.: General Overview of ImageCLEF at CLEF2015 Labs. Lecture Notes in Computer Science, Springer International Publishing (2015) 19. You, D., Rahman, M.M., Antani, S., Demner-Fushman, D., Thoma, G.R.: Textand content-based biomedical image modality classi cation. In: Proc. SPIE Medical Imaging. pp. 86740L{86740L{8 (2013)

References

1. Amin , M.A. , Mohammed , M.K. : Overview of the ImageCLEF 2015 medical clustering task . In: CLEF2015 Working Notes. CEUR Workshop Proceedings , CEURWS.org, Toulouse, France (September 8-11 2015 )

2. Bishop , C.M. : Neural Networks for Pattern Recognition . Oxford University Press, Inc., New York, NY, USA ( 1995 )

3. Chatzichristo s, S.A. , Boutalis , Y.S.: Cedd: Color and edge directivity descriptor: A compact descriptor for image indexing and retrieval . In: Proceedings of the 6th International Conference on Computer Vision Systems . pp. 312 { 322 . ICVS' 08 , Springer-Verlag, Berlin, Heidelberg ( 2008 )

4. Chatzichristo s, S.A. , Boutalis , Y.S. : Fcth: Fuzzy color and texture histogram - a low level feature for accurate image retrieval . In: Proceedings of the 2008 Ninth International Workshop on Image Analysis for Multimedia Interactive Services . pp. 191 { 196 . WIAMIS '08, IEEE Computer Society, Washington, DC, USA ( 2008 )

5. Dalal , N. , Triggs , B. : Histograms of oriented gradients for human detection . In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005 ), 20 - 26 June 2005, San Diego, CA, USA. pp. 886 { 893 ( 2005 )

6. Frangi , A.F. , Niessen , W.J. , Vincken , K.L. , Viergever , M.A. : Muliscale vessel enhancement ltering . In: Medical Image Computing and Computer-Assisted Intervention - MICCAI'98 , First International Conference, Cambridge, MA, USA, October 11 - 13 , 1998 . pp. 130 { 137 ( 1998 )