This paper describes our participation at ImageCLEF 2009. We participated in the photographic retrieval task (ImageCLEFPhoto). Our method is based on intermedia pseudo-relevance feedback. We have enhanced the pseudo-relevance feedback mechanism by using semantic selectional restrictions. We use Terrier for text retrieval and our own simple block-based visual retrieval engine. The results obtained at imageCLEF 2009 show that our method is robust and promising. However, there is room for improvement on the visual retrieval as well as the topics without cluster descriptions.
This paper describes our participation at ImageCLEF 2009. We participated in the photographic
retrieval task (ImageCLEFPhoto). This year's task targeted promoting diversity in image search.
It involved an annotated image collection of approximately half a million images, and fty queries
divided into two sets: one with a subject and provided speci c subtopics (clusters), while the other
with only a topic. A full description of the task can be found in [
We submitted four runs, aiming at evaluating our method as well as the resources used. Similar to our participation last year, our method is based on intermedia pseudo-relevance feedback. However, to in order to account for the much larger data set, we have introduced some modi cations to our visual component. We have also enhanced the textual retrieval component, as well as to the pseudo-relevance feedback mechanism by using semantic selectional restrictions.
The results obtained at imageCLEF 2009 show that our method is robust and promising. However there is room for improvement on the visual retrieval as well as the topics without cluster descriptions.
The rest of this paper is organized as follows: Section 2 describes the visual retrieval component, Section 3 the text processing of the query, Section 4 the enhanced pseudo-relevance feedback and Section 5 the results we obtained at ImageCLEF 2009, then we conclude the paper.
The text is tokenized and preprocessed by removing stop words (grammatical words which do not
contribute to the meaning) and punctuation. The rest of the terms are converted to lower case
and stemmed using the Snowball stemmer [
Queries are tokenized and preprocessed similarly; stop words and punctuation are removed and the rest of the terms are stemmed. The queries consist of a topic and a cluster description when available, in addition to the expansion terms from the top visual results. Named-entities are given more weight and multiple-token named-entities are chunked into one term by adding quotes around them.
For text retrieval, we use the Terrier Information Retrieval platform, a Java-based Information
Retrieval platform available from the University of Glasgow [
In this phase, the query is expanded with terms potentially related to the query. Common ways for text query expansion include adding synonyms and other related terms to the query. However, according to our experiments, this approach leads to the introduction of many noisy terms. Instead, we opted for the extraction of related terms from the ve highest-ranked results retrieved by the content-based system described in Section 2. For the data set in our experiments, all the terms associated with the image are extracted except for stop words. In order to expand the query without introducing noise, the candidate text is compared to the query topic. If the image is found to be potentially related to the topic, the text query is expanded with the relevant terms. To compute the relatedness of the image annotaion to the topic, we use the minimum threshold of one common non-grammatical word, due to data sparseness.
The purpose of the query expansion module is not only to augment the query by adding
new candidate related terms to it, but also to enhance it by adding weights to its key terms
and ltering out potentially noisy terms from expansion. We also avoid expanding the query
with named entities that do not have a semantic relationship with the query. This is crucial in
photographic collections, since by their nature, photographs and image queries are often bound
by geographical constraints. In order to ensure that potential expansion images do not introduce
con icting geographical terms in the query, we rst build a lter from the location speci ed in the
query. We make use of WordNet [
Visual Text With PRF Combined
Visual Text With PRF Combined P10 0.072 0.732 0.548 0.604 P10 0.12 0.136 0.552 0.568
0.1128 0.7368 0.6327 0.6371
0.1768 0.1124 0.6171 0.6049 6
We experimented at ImageCLEF 2009 with applying semantic selectional restrictions to enhance intermedia pseudo-relevance feedback and di erent methods of query formulation for clustered queries. We will further analyze the results in order to understand the signi cance of the chosen measures, given that the precision varies signi cantly at di erent levels of recall.