This paper describes our participation in photo Flickr retrieval task at the ImageCLEF 2012 Campaign. Our aim is to evaluate the performance of topic models, such as Latent Dirichlet Allocation (LDA), in image retrieval based on the textual information surrounding the images. To do this, we propose to extract topics from Flickr user tags1 using the LDA topic model. Then, we use the Jensen-Shannon Divergence measure to compute the similarity between queries and user tags representing images.
Many works in the image retrieval literature have shown that, in Web case,
textual retrieval is more efficient than contenet retrieval [
The common method of searching images by context is to use directly the
text surrounding the images by applying the well known tf-idf scheme [
To resolve those problems , latent dimension can be used to reduce the termdocument matrix to a much lower dimension subspace that captures most of the variance in the corpus. The main idea of this technique consists in modeling documents as a distribution of topics where each topic is a distribution of words.
In this paper, we choose to use Latent Dirichlet Allocation (LDA) [
Topic models are widely used in textual information processing and have shown their interest in many tasks. Recently, this technique was used in image representation and processing.
In the image retrieval domain, LDA is mainly used in visual level. Hoster
et. al. [
Elango et. al. [
In this work, we are interested in applying LDA to the textual information related to the images. Resulted topics are then used to find images similar to a textual user query.
Our paper is organized as follows. Section 2 presents a review of LDA topic model and similarity measure that we use in image retrieval. In section 3, we present experimental results on photo Flickr retrieval task and conclude the paper in section 4. 2
The main idea behind the use of topic model in our work is that the image is probably an illustration of the overall subject (topic) in the document. User tags are likely to be motivating feature to represent the image since they normally describe the image content. For this, we use user tags to extract textual topics of the images. Figure 1 shows an example of a set of topics extracted from Flickr user tags.
ocean beach sea coast pacific water waves rocks surf sand space stars chandra galaxy smithsonian institution telescope
star universe ray
art painting museum gallery artist modern contemporary media mixed collage phone mobile
cell camera sony cellphone ericsson nokia telephone blackberry sunset evening sky dusk clouds silhouette landscape
sun twilight sundown Fig. 1. Top 10 words of 5 topics extracted from the Flickr user tags
In a large collection, the main problem is that many documents are about the same idea. Topic models are used to connect documents that share similar patterns (meaning) by discovering patterns of words.
The idea behind LDA is to model documents as a distribution of topics where each topic defines a distribution over words. Specifically, we assume that K topics are associated with a collection, and that each document defines a distribution over (hidden) topics. The posterior probability of these latent variables determines a hidden decomposition of the collection into topics.
We have D documents using a vocabulary of V word types. Each documents contains M word tokens. We assume K topics. Each document has a K-dimensional multinomial θ over topics with a common Dirichlet prior Dir(α). Each topic has a V-dimensional multinomial ϕ over words with a common symmetric Dirichlet prior Dir(β).
Figure 2 shows the various components of this model.
α θ z w
M
D φ
K β
The generative process of LDA is described as follow: (1) For each topic,
(a) Draw a distribution over words ϕ s Dir(β) (2) For each document, (a) Determine topic distribution θd s Dir(α) (b) For each word, (i) Generate topic z s M ult(θ) (ii) Generate word w s M ult(ϕ). 2.2
After running LDA on a corpus, it is possible to use its output to compare
documents to each other. In our case, each tag has a distribution over topics.
Many works use the KL-divergence [
DKL(P ||Q) =
i ∑ P (i)ln P (i)
Q(i) .
(1) where P and Q are two probability distributions over topics of two documents p and q.
But the problem is that the KL-divergence is not symmetric i.e. DKL(P ||Q) ̸=
DKL(Q||P ). An example of symmetric divergence measure named Jensen-Shannon
divergence [
According to the obtained results, our method does not perform very well compared to the best run in ImageCLEF2012 competition. A possible explication of this result is that the user query is generally composed of a few words. Consequently, we do not knew about its topic. 2 http://mirflickr.liacs.nl 4
This work studies the impact of textual topics in image retrieval. We have applied the LDA topic model to the user tags representing images. Results show that this approach does not perform very well. In future works, we plan to improve results by using the query expansion technique to well know about the possible topic of the query.