The goal of this work is to investigate the performance of classical methods for feature description and classi cation, and to identify the di culties of the ImageCLEF 2010 modality classi cation subtask. In this paper, we describe di erent approaches based on visual information for classifying medical images into 8 di erent modality classes. Since within the same class, images depict very di erent objects, we focus on global descriptors such as histograms extracted from scale-space, log-Gabor and phase congruency feature images. We also investigated di erent classi cation approaches based on support vector machines and random forests. A grid-search associated to a 10 folds cross-validation has been performed on a balanced set of 2390 images to nd the best hyperparameters for the di erent models we propose. All experiments have been conducted with MATLAB on a Workstation with Intel Duo Core 3.16 Ghz and 4Gb of RAM. Our approach based on simple SVM and random forests give best performance and achieve respectively an overall f-measure of 74:13% and 73:59%.
In this paper, we investigate di erent classi cation approaches for the new
ImageCLEF subtask modality classi cation [
The modality classi cation problem is an instance of a multi-class classi cation problem. We denote I : R2 ! R an image we want to classify into one of the 8 di erent classes fCkgk2f1; ;8g, namely computerized tompography, graphics, magnetic resonance imaging, nuclear medicine, positron emission tomography, optical imaging, ultrasound and X-ray. I is described by a feature vector we denote x, and y its corresponding class label. The goal of multi-class classi cation is then to train a decision function such that: where fxn; yngn2f1; ;Ng is the training set composed of image descriptors and their corresponding class labels. In the following section, we will show how to compute the di erent components of x.
Feature Representation As mentioned in the introduction, we will focus on statistical texture features to describe our images. We propose to use 3 di erent types of descriptors based on scale-space, log-Gabor and phase-congruency feature images.
Scale-space statistical descriptors A basic approach to characterize the
image variations at di erent band of the frequency spectrum is to perform its
multi-scale analysis [
exp 2 1 (i2 + j2) 2 where (i; j) 2 [ s; s]2, (2s + 1) (2s + 1) being the size of the lter mask. By convolving the input image I with the kernel in Eq.2, we obtain following feature images:
L = I
G
By doing this for a set of scales f mgm2f1; ;Mg, we obtain feature images
fL m gm2f1; ;Mg. For each of these feature images, we compute its
corresponding intensity histograms which gives us a set of descriptors fHmgaussgm2f1; ;Mg.
log-Gabor statistical descriptors A common approach to characterize
textures is to use a bank of Gabor lters which permit to analyze di erent part
of the frequency spectrum for di erent orientations [
G(f ) = exp (log(f =f0))2 (log( =f0))2 where f0 is the center frequency of the sinusoid and is a scaling factor of the bandwidth. To cover the whole frequency spectrum, a range of di erent scales and orientations must be considered while designing the lter bank. After convolving the input image I with the corresponding even-symmetric and odd-symmetric lters for a certain scale and orientation we obtain pairs if response images Ie; and Io; . From these responses, we compute the corresponding amplitude and phase:
A ; = q
(Ie; )2 + (Io; )2 ; = arctan Ie; ; Io; (2) (3) (4) (5) By doing this for a set of scales p2f1; ;P g and orientations q2f1; ;Qg, we obtain a set of feature images A p; q ; p; q p;q. For each of them, we compute the corresponding intensity histogram which nally gives us the descriptors
Hpg;aqbor p;q.
Phase congruency statistical descriptors Phase congruency is an absolute measure of feature signi cance which is invariant to changes in illumination or contrast [5]. It is closely related to the concept of local energy model which postulates that signi cant features are perceived at points of maximum phase congruency. The phase congruency function is de ned as:
E P C = P A (7) (8) (9) where E is the local energy and A the Fourier amplitudes. It is possible to approximate the function above by using quadrature lters such as the log-Gabor lters described in the previous part. Indeed, by using the response images Ie; and Io; , we can approximate the phase congruency for a certain orientation as follows:
P C = r (P P
Ie; )2 + (P
Io; )2 q(Ie; )2 + (Io; )2 For di erent orientations t2f1; ;T g, we obtain the feature images fP C t gt2f1; ;T g. For each of them, we nally compute the corresponding intensity histogram which gives us the descriptors fHtpcgt2f1; ;T g.
Global descriptors After computing all these descriptors for an input image I, a global descriptor x is created by concatenating all histograms: 2 x = 6 4| 3T Hmgauss scale{-zspace ; } |
Hpg;aqbor log-{Gzabor ; Htpc } |phase-congruenc}y {z We investigated several approaches to classify images according to the descriptors we described in the previous section.
Simple Multi-Class SVM From a set of images and their corresponding labels, we compute their descriptors and generate a training set fxn; yngn2f1; ;Ng. Recall the classifcation problem statement, we are looking for a function such that: ; N g (10) (x) = hw ; '(x)i + b where ' is a non-linear mapping from the space spanned by our descriptors into a hidden feature space with higher dimensionality, w is a weighting vector and b a bias. w, ' and b de ne a hyperplane separating the di erent classes in the high-dimensional space. To nd the optimal hyperplane, we maximize the margin between the classes and this, with a minimum of classi cation errors [6]. This can be written as the dual convex optimization problem:
1 minimize 2 kwk2 (12) subject to: yn(hw ; '(xn)i + b) 1 Since the problem may not be separable, we allow misclassi cation errors by introducing slack variables that induce a soft margin: Let us rst consider a 2 class classi cation problem where yn 2 f 1; 1g. We model the function as: (11) (13) (14) (15) (16) 1 N minimize 2 kwk2 + C X n
n=1 subject to: yn(hw ; '(xn)i + b) 1 n where C is a tradeo parameter weighting the impact of the errors and thus the exibility of the model. After solving the dual, it can be shown that a solution wopt of this minimization is always a linear combination of the training vectors fxngn with weights f ngn2f1;:::;Ng:
N wopt = X
n '(xn) n=1 which leads to the following model for :
N N (x) = X i h'(xn) '(x)i + b = X i K(xn; x) + b;
n=1 n=1 where K is the kernel associated to ' in the higher dimensional space. To handle complex non-linear relations between the descriptors and their labels, K is chosen as a RBF kernel, leading to the following decision function:
N (x) = X n=1 n exp kxn xk2 + b: Now that we have a decision function for a binary classi cation problem, we need to extend it to multi-class. Two common methods are to combine several binary SVM into a multi-class model:
One-vs-one: binary classi ers are built for each combination of classes One-vs-rest: a binary classi er is built for each class against all the others Since the one-vs-rest combination is less suitable in terms of class balancing, we choose a one-vs-one approach as suggested by [7].
Multi-Kernel Multi-Class SVM In the previous section, the descriptors x are a concatenation of di erent feature histograms. Each of these feature types spans its own space and may live in a di erent subspace. For this reason, it is advantageous to use a kernel for each feature type that is especially adapted to the features. This leads to a set of P decision functions where P is the number of feature types considered:
N (p)(x(p)) = X n=1 (p) exp n (p) x(p) n x(p) 2 + b(p): where x(p) is the descriptors of type p. The global decision function would be then a combination of each "specialized" decision function. In this paper, we investigate two combination approaches: linear multi-SVM: (x) =
(p)(x(p)) product multi-SVM:
P 1 X P p=1
P (x) = Y p=1 (p)(x(p)) which we respectively call linear and product Multi-SVM in the remaining of this paper.
Random Forests Random forests are basically a ensemble of T independent random trees we denote (t) which vote for the most popular class t2f1; ;T g [8]. Each tree (t) can be seen as an ensemble of random split functions which partition the feature space and give an estimate of the posterior probability distribution P (Ckjx; (t)) for each class Ck. The forest F = (1); ; (T ) combines the trees estimates as follows:
T P (Ckjx) = T1 X P (Ckjx; (t)) t=1 (17) (18) (19) (20) The class of an unseen point is then determined by using a maximum a posteriori criterion: y = argmax(P (Ckjx)) (21)
k A clear advantage of random forests over the SVM approaches is their inherent multi-class characteristic. 4 4.1
Experimental setup In this section, we compare the di erent approaches we decribed above by performing a 10-folds cross-validation on a set of 2390 labeled images distributed over the classes as follows:
CT: Computerized tomography (314 images) GX: Graphics, typically drawing and graphs, (355 images) MR: Magnetic resonance imaging (299 images) NM: Nuclear Medicine (204 images) PET: Positron emission tomography including PET/CT (285 images) PX: optical imaging including photographs, micrographs, gross pathology etc (330 images) US: ultrasound including (color) Doppler (307 images) XR: x-ray including x-ray angiography (296 images)
For each test, classes are rebalanced by using random subsampling. A gridsearch has been performed to nd the best hyperparameters as shown on Fig.2. For the SVM-based classi ers, the tradeo parameter C and are optimized. Concerning the random forests, we are looking for the best number of trees and the best number of random tests at each node. Note that features are scaled so that each dimension is in the range [0; 1]. To evaluate the di erent approaches, we compute several quality measures from the confusion matrix: overall accuracy, precision, recall, f-measure and error rate. Overall results summarized in Tab.1 show that the simple SVM and random forests perform better than the multi-SVM approaches. This is contrary to what could be expected from these methods designed to capture di erent aspects of the features. Note that in the multi-SVM approaches, each of the SVM was trained independently. A joint training of all kernels, called Multiple-Kernel Learning (MKL) [9, 10], may perform better.
As feature representation, we relied on global statistical descriptors extracted from the textural information contained in the images from the 8 di erent modalities. To extract this textural information we used classical lters which are not
Simple SVM Multi SVM linear Multi SVM product
Random Forest especially adapted to the high variability we can observe in the di erent classes. Thus, the resulting texture information may not be discriminative enough. Indeed, some images such as charts or drawing present almost no textural information. Moreover, by computing global statistics on these features, a lot of information which might be usefull for classi cation is discarded. In the present work, we did not make use of any RGB information, this would have de nitely helped to discriminate images from classes which do not contain any color images. 5
In the present work, our goal was to get a rst insight into the ImageCLEF challenge by taking part to its new subtask modality classi cation. We presented di erent approaches for the classi cation of medical images into 8 imaging modalities. As a feature representation, we extracted global statistics computed from the textural information contained in the images. We then investigated four classi cation methods based on SVMs and random forests. Approaches based on simple SVM and random forests give the best performance and achieve respectively an overall f-measure of 74:13% and 73:59%. After analyzing our preliminary results, there is clearly a lot of room for improvement concerning our feature representation. In future work, we will focus on de ning new sparse and more discriminative representations of the data which should lead to better classi cation results. 5. P. Kovesi, Image Features From Phase Congruency, Videre: A Journal of Computer
Vision Research. MIT Press, 1999 6. C. Cortes and V. Vapnik, Support Vector Networks, Mach. Learn.,1995 7. C.W. Hsu and C.J. Lin, A comparison of methods for multi-class support vector machines, IEEE Transactions on Neural Networks, 2002. 8. L. Breiman, Random Forests, Machine Learning, 2001 9. F.R. Bach, G.R.G Lanckriet and M.I. Jordan, Multiple kernel learning, conic duality, and the SMO algorithm, International Conference on Machine Learning, 2004 10. G.R.G. Lanckriet, N. Cristianini, L.E. Ghaoui, P. Bartlett and M.I. Jordan, Learning the kernel matrix with semide nite programming, J.Machine Learning Research, 2004