Image classi¯cation or annotation is proved di±cult for the computer algorithms. The Naive-Bayes Nearest Neighbor method is proposed to tackle the problem, and achieved the state of the art results on Caltech-101 and Caltech-256 image databases. Although the method is simple and fast, for the real applications, it su®er from the imbalance of the training datasets. In this paper, we extend the image to class distance which is more general, and use the random sampling technique to alleviate the situation of the imbalance of the training datasets. We perform our method on the ImageCLEF 2010 Photo Annotation task, and the results(INSUNHIT) showing that the algorithm is fast and stable. Although it does not achieving the state of the art performance, more image features can be used to improve the performance and dimension reduction techniques can be adopted to reduce the complexity of space and time.
Although human beings recognize the scenes or objects in an image easily,
automatic image classi¯cation and annotation is a challenging task for computer
programs. According to [
In recent thirty years researchers have proposed many image descriptors and
learning algorithms to recognize objects and scenes. The image descriptors in
the literature is roughly classi¯ed into ¯ve categories: global descriptors[
The learning algorithms have been proposed with the development of
image descriptors. Various kinds of learning algorithms are proposed or adopted in
the literature, including Non-parametric methods[
Although these image recognition systems succeed in several tasks or benchmark databases, they remain naive for the real world applications. In the real world image databases, the visual concepts shares large intra-class and interclass variability, and the relation of the concepts is complicated. In addition, the images used for training is imbalanced, resulting in the failure of some learning algorithms. Finally, the databases contain large scale images, and the simplicity and the running time need to be considered when design image categorization systems.
In this paper we describe our system(INSUNHIT) for ImageCLEF2010 Photo
Annotation task. We use the dense SIFT[
Overview of the ImageCLEF2010 Photo Annotation
Dataset
In this section we brie°y discuss some di±culties of the ImageCLEF Photo
Annotation dataset. For a detail overview of the dataset please refer to [
First, compared to the benchmark datasets used in the literature, the visual concepts(annotation keywords) is more abstract and shares more variability. There exist 93 visual concepts, including objects(such as \trees" and \°owers"), scenes(such as \desert", \sky" and so on), abstract concepts(such as \macro", \spring" and \summer"). Some of the concepts have visual similar properties| for example, \child" and \baby"|, and the image features can not discriminate them perfectly. Image descriptors is di±cult to choose.
Second, the image dataset is very imbalanced. The quantity of training images of each concept ranges from 12 training images(\skateboard"), to 7484 images(\Neutral Illumination"). This makes the classi¯cation system more di±cult to train the concepts.
In this section we introduce our learning algorithm in detail. Our algorithm extends the NBNN method, and the RS-ICD is stable for measuring the distance of the image to a query concept. 3.1
The NBNN method de¯nes the distance of image to class to cope with the large intra-class variance of the class. The method is based on the Naive-Bayes and Kernel Density Estimation framework, and the image to class distance is the near optimal distance when training images is very large.
Here we only review the method of NBNN. The NBNN methods operate on the local patch based image features. For a given class Cj; j 2 (1; 2; :::; L), and query image Q and its corresponding image descriptors d1; d2; :::; dn, the distance of Q and class Cj is de¯ned as follows:
n d(Q; Cj) = X N NCj (di)
1 where N NCj (di) is the nearest distance of the descriptor di to class Cj:
N NCj (di) = min(distance(di; dCj )); dCj 2 Cj The classi¯cation process is proceed:
Copt = argminC (d(Q; Cj)); Cj 2 C (1) (2) (3) (4) (5)
Although the NBNN is e®ective for image classi¯cation that output a single label when decision, it can not be extend to image annotation(multi label) because the image to class distance of each image is not comparable. In order to deal with this problem, the distance should be normalized: When K = 1, both the distance function and the decision function are the same as the NBNN method. Therefore the distance de¯ned by NBNN is a special case of our Image to Class Distance(ICD). The most important is that ICD is comparable between images, and the distance can be used to do multi label classi¯cation. 3.3 End 4 4.1
The Image to Class Distance su®ers from the imbalanced training datasets. We use random sampling technique to tackle the problem. Therefore the our algorithm described as follows: The Random Sampling Image to Class Distance Based Annoation Begin
For t = 1,2,...T
Random sampling L images from each class Ci denoted as Ci(t); End Extract the descriptors of query image Q For t = 1,2,...T
compute dK(Q, Ci(t)) of each class End the image to class distance dK(Q, Ci) = average(dK(Q, Ci)) compute the probability p(Q, Ci)=exp(-a*dK(Q, Ci))
Here we describe the experimental setup of our algorithm. The images are transformed into gray image, and resized to 300 pixels while keep the aspect ratio if the image's length or width are larger than 300 pixels. 128 bin Dense SIFT image features with the step of 8 pixels are extracted. The parameter T of Random Sampling process is set to 10.
We run the algorithm on a computer with Intel Core 2 Duo Q9400 CPU, 4 GB
Memory, 32 bit linux operating system. The SIFT extraction matlab program is
provided by Lazbnik[
To evaluate the performance of the system, Mean Average Precision results is performed as the main evaluation measure. 4.2
In this section we discuss the results of our system(INSUNHIT). All the results
is showed on the website[
Our results achieved the centered of the overall results. We use di®erent setup, and achieve similar results. This indicate that our algorithm is very stable. Classifying the whole test image dataset takes about 12 hours. Although the performance of our algorithm is far behind the state of art performance, further improvements can be easily obtained. First, the 128-bin SIFT descriptor is too large, while for the image classi¯cation, we can try other image descriptors or dimension reduction techniques. Second, multiple image descriptors should be used to improve the annotation results. In recent years, most of the promising results on benchmark image datasets are performed by combining multiple image features or multiple classi¯ers that computed on these image features. Acknowledgments. This investigation was supported by the project of the National Natural Science Foundation of China (grants No. 60973076), Special Fund Projects for Harbin Science and Technology Innovation Talents(grants No. 2010RFXXG003) and Microsoft Fund Projects HIT.KLOF.2009021.