This paper presents the fourth participation of the SINAI group, University of Ja¶en, in the Photo Retrieval task at Image CLEF 2009. Our system uses only the text of the queries, and a clustering system (based on kmeans) that combines di®erent approaches based on a di®erent use of the cluster data of the queries. The o±cial results shown that the combination between the title of each query and the other titles of the clusters obtain our best performance and that our clustering system did not work well.
In this paper we describe our approach to the ImagePhoto 2009 evaluation campaign at CLEF, over
the new collection, which contains 498,920 images from Belga News Agency. Given a monolingual
English query the goal of the ImagePhoto task is to ¯nd as many relevant images as possible from
an image collection[
In 2008 this task took a di®erent approach to evaluate the image clustering. This year the organizers give special value to the diversity of results. Given a query the goal is to retrieve a relevant set of images at the top of a ranked list. Text and visual information can be used to improve the retrieval methods, and the main evaluation points are the use of pseudo-relevant feedback (PRF), query expansion, IR systems with di®erent weighting functions and clustering or ¯ltering methods applied over the cluster terms. Our system makes use of text information, not visual information, to improve the retrieval methods. This year, a new method has been implemented to cluster images.
The following section describes the queries built and the new algorithm of clustering. In Section 3 we explain the experiment carried out. Finally, the obtained results and conclusions are presented in Section 4.
In our system we have used an automatic modality, without user interaction, with English text
information (not visual information). The English collection documents have been preprocessed
as usual (English stopwords removal and the Porter's stemmer[
Past campaigns our adhoc system worked with di®erent IR systems, and test di®erent
approaches, such as combination of retrieved lists or fusion of a ¯ltering method that used some
clusters terms. The precision results obtained were very similar to di®erent languages[
The weighting function of the IR system is a parameter changed to test previous results and
based on them we have used Okapi. The use of Psedo-Relevance Feedback (PRF) to improve the
retrieval process is not conclusive, but in general the precision is increased in past experiments,
so it is always used with Lemur. The blind feedback algorithm is based on the probabilistic term
relevance weighting formula developed by Robertson and Sparck Jones[
The evolution of our system introduces a clustering module, based on the algorithm kmeans, and the creation of the ¯nal topics using the information of the title and the clusters terms. 2.1
The ¯rst module of our system uses the data of the title and the cluster title, in some cases, and combines them to obtain the ¯nal topics. These topics, in a second step, are run over the IR system, and a ¯nal list of relevant documents is obtained.
The following four sets of topics have been built for these experimentation: 1. In the baseline case only the title of the query is used against the index of the IR system. 2. Each ¯nal topic is a combination between the query title and the title of the last cluster. 3. Each ¯nal topic is a combination between the query title and the other words of each cluster.
The ¯gure 1 shows a general scheme of the system developed. 2.2
It has been found that the variability of top results in a list of documents retrieved as answer to a
query, the performance of the retrieval systems increases too, being in some cases more desirable
to have less but more varied items in this list [
We have applied kmeans on each list returned by the Lemur IR system. For this, Rapid Miner tool was used2. The clustering algorithm has tried to group these results, without any concern on ranking, into 4 di®erent groups. The number of groups has been stablished at this value as documents in the training set have this average number of clusters speci¯ed in their metadata. 1Available at http://www.lemurproject.org/ 2Available at http://rapid-i.com
Once each of the documents in the list has been labeled to its resulting cluster index, the list has been reordered according to the described principle: we ¯ll the list by alternating documents from di®erent clusters. 3
The data set contains 498,920 images from Belga News Agency, an image search engine for news photographs. Each photograph will be up to a maximum of 512 pixels in either width or height, accompanied by a caption composed of English text up to a few sentences in length. Di®erent to the data last year, captions are provided without a speci¯c format to increase the challenges to participants. Caption might contain date and place where image was captured.
The topics statements include a query title with few words, and some cluster titles and descriptions, to promote the diversity of the ¯nal results, as appear in the Figure 2. In our system descriptions are not used. In our system we have proved the following con¯gurations: 2. (2) SINAI2 - title and ¯nal cluster. This experiment combines the query title with the title of the ¯nal cluster that appear in the topics ¯le. Lemur also uses Okapi as weighting function and PRF. 3. (3) SINAI3 - title and all clusters. This experiment combines the query title with all the words that appear in the titles of all the clusters. Lemur also uses Okapi as weighting function and PRF. 4. (4) SINAI4 - clustering. The query title and each cluster title (except the last one that combines all) are run against the index generated by the IR system. Several lists of relevant documents are retrieved, and the clustering module combines them to obtain the ¯nal list of relevant documents. The aim of this experiment is to increment the diversity of the retrieved results using a clustering algorithm. 4
The data set of the collection has been indexed using Lemur3 IR system, by applying Okapi weighing function and using Pseudo-Relevance Feedback (PRF). We have used only textual information in English. Table 1 shows the results obtained in our four experiments submitted this year.
Experiment sinai1 T TXT sinai2 TCT TXT sinai3 TCT TXT sinai4 TCT TXT
We have experimented with di®erent kinds of cluster combination. However, as we can see in Table 1, the application of clustering does not improve the results greatly. In fact, only in the run used SINAI3 the query the original title and the titles of all the clusters overcomes the baseline case SINAI1 that only uses the original title. Unfortunately, the experiment SINAI4 that applies our clustering and fusion approach has achieved the worst results. Thus, the obtained results show that it is necessary to continue investigating the clustering methodology. In addition, the use of visual information could improve the ¯nal system. 5
This work has been supported by the Regional Government of Andaluca (Spain) under excellence project GeOasis (P08-41999), the Spanish Government under project Text-Mess TIMOM (TIN2006-15265-C06-03) and the local project RFC/PP2008/UJA-08- 16-14.