This paper describes our participation at the ImageClef 2009 medical annotation task. In this task we have used the bag-of-words approach for image representation. We submitted one run, using support-vector-machines trained on the visual word histograms in multiple scales. In this task our result ranked first, with error score of 852.8. In the last several years, "patch-based" representations and "bag-of-features" classification techniques have been proposed for general object recognition tasks [1 - 6]. In these approaches, a shift is made from the pixel entity to a "patch" - a small window centered on the pixel. In its most simplified form, raw pixel values (intensities) within the window are used as the components of the feature vector. It is possible to take the patch information as a collection of pixel values, or to shift the representation to a different set of features based on the pixels, such as SIFT features [7], and reduce the dimensionality of the representation via dimensionality reduction techniques, such as principlecomponent analysis (PCA) [8].
Patch-based methods have evolved from texton methods in texture analysis [
In this task we are presented with 12,729 classified x-ray images, labeled according to four label sets, labels are based on labeling standards from the last four years. Label sets from 2005 and 2006 contain 57 and 116 labels, respectively. Label sets from 2007 and 2008 contain 116 and 196 IRMA codes. The goal is to classify about 2000 unseen images according the four label sets. Error evaluation scheme is described in the task website.
We model an image as a collection of local patches, where a patch is a small rectangular sub region of the image. Each patch is represented as a codeword index out of a finite vocabulary of visual codewords. Images are compared and classified based on this discrete and compact representation.
We built a dictionary from a random subset of 400 images from the database. The dictionary building process extracts patches of a fixed size of 9x9 pixels with a grid of 6 pixels spacing, patches are normalized to have 0 mean and 1 variance. We then compute a covariance matrix of a set of roughly 2,000,000 patches, and apply PCA to find its eigenvectors. The 6 vectors with the highest energy are shown in Figure 1.
These eigenvectors are later used as a base for the rest of the patches in the database. Patch center coordinates are added to the feature set, in order to include information about the visual words layout. Running k-means algorithm on this set produces 1000 dictionary visual words. A sample dictionary is displayed in Figure 2.
The dictionary building process is repeated in 3 image scales: full resolution, 1/2 scale and 1/8 scale. The resulting dictionary is a concatenation of the 3 dictionaries from the 3 scales. In the image representation step, patches are extracted from each image using a dense grid- around every pixel. An image is represented as a word histogram over the multi-scale dictionary. Image classification is performed on the word histograms by an SVM classifier with kernel. Multi-class classification is implemented using one-vs-one heuristic. In the training step each IRMA code is treated as a separate label, without using the hierarchical nature of the code. The classifier output is returned without using wildcards, except for replacing trailing ‘0’s with ‘*’. This does not damage the error score if the last digit in the true code is ‘0’, and can reduce the error score if the last digit is non-zero.
Kernel parameter and SVM tradeoff parameters C were exhaustively searched to minimize cross-validation average error over 5 experiments, where in each experiment 2000 random images served as test data. Parameter tweaking was done using the 2007 code labels. Figure 3 displays the error landscape of the scanned parameters space. The optimal parameters set was used in all four classification tasks.
We presented a classification system for large medical databases, based on compact bag-of-features image representation. The system achieves comparatively good results in the ImageClef 2009 medical annotation challenge, while maintaining efficient computation times.