The ImageCLEF 2006 medical automatic annotation task encompasses 11,000 images from 116 categories, compared to 57 categories for 10,000 images of the similar task in 2005. As a baseline for comparison, a run using the same classifiers with the identical parameterization as in 2005 is submitted. In addition, the parameterization of the classifier was optimized according to the 9,000/1,000 split of the 2006 training data. In particular, texture-based classifiers are parallel combined with classifiers, which use spatial intensity information to model common variabilities among medical images. However, all individual classifiers are based on global features, i.e. one feature vector describes the entire image. The parameterization from 2005 yields an error rate of 21.7%, which ranks 13th among the 28 submissions. The optimized classifier yields 21.4% error rate (rank 12), which is insignificantly better.
The content of a medical image is represented by texture features proposed by Tamura et al. [
The single classifiers are combined within a parallel scheme, which performs a weighting of the normalized distances obtained from the single classifiers Ci, and applies the nearest-neighbordecision function C to the resulting distances: dc(q, r) = X λi · di(q, r) i (1) where λi, 0 ≤ λi ≤ 1, Pi λi = 1 denotes the weight for the normalized distance di(q, r) obtained from classifier Ci for a sample q and a reference r. 2.3
Based on a combination of classifiers used for the annotation task in 2005 [
All results are obtained non-interactively, i.e. without relevance feedback by a human user. Table 1 shows the error rates in percent obtained for the 1,000 unknown images using single k-nearest neighbor classifiers and their combination for both k = 1 and k = 5. The error rate of 21.4% ranks 12th among 28 submitted runs for this task. The weights that were optimized on the ImageCLEF 2005 medical automatic annotation task (10,000 images from 57 categories) yield an error rate of 21.7% and rank 13th. 4
The comparability of similar experiments such as ImageCLEF 2005 and ImageCLEF 2006 is difficult since several parameters such as the images and the class definitions were changed. In general, the difficulty of a classification problem is proportional to the error rate, increases with the number of classes, and decreases with the number of references per class. Since the number of references varies for the classes, this relation is rather complex than linear. However, linearity can be used as a rough but first approximation.
Since the error rates are not substantially improved by a training on the 2006 data, the classifier that has been optimized for 2005 is also suitable for the ImageCLEF 2006 medical automatic annotation task. The parameterization optimized for ImageCLEF 2005 and 2006 yields E2005 = 13.3% and E2006 = 21.7%, respectively, and the quotient E2005/E2006 can be used to relate the results from the 2005 and 2006 campaigns. In other words, the best E = 16.2% from 2006 would approximately have resulted in E = 10, 0%, which is better than the best rate obtained in 2005. This is in accordance with the rank of submission, that drops from the 2nd to the 13th place. In conclusion, general improvements for automatic image annotation have been made during the last year. 5
This work is part of the IRMA project, which is funded by the German Research Foundation, grant Le 1108/4.