Our participation in the ImageCLEF Wikipedia retrieval task aims to study the e±ciency of using two contextual factors in image retrieval: metadata which contains speci¯c information about images, and textual content which contains general information about images. For this aim, the Lucene library is used for indexing and searching. We propose also to combine both factors using two di®erent methods: one based on simple linear function and one based on scores comparison. In addition, a comparison between monolingual and multilingual image retrieval using queries in a single language (English) and queries in di®erent language is done. Results show that the use of textual content is more useful then the use of metadata and the combination of both factors further improves results. In addition, the use of all provided languages exceeds over the use of only English langage.
This is our ¯rst participation in imageClef Wikipedia retrieval task. We aim in this work to evaluate the impact of some contextual factors in image retrieval using the Wikipedia collection. More precisely, we used the Lucene Search engine1 to calculate an image relevance score for each contextual factor: textual content and metadata. The di®erence between these two factors is that the metadata contains the description of the image, so the most speci¯c information. On the contrary, the textual content can be the same for two or more images, so it contains general information about the image. 1 http://lucene.apache.org/
Consequently, comparing the use of speci¯c information and general information to represent images returns to compare the use of metadata and textual content to compute the image score.
To well evaluate these two factors, a combination between them should be done. We propose so to use either a classical linear combination, or a comparisonbased approach.
In addition, we would like to study the impact of the monolingual and multilingual image retrieval by using queries in a single language (English) and queries in di®erent language. When using an English query, only the English documents are used, and when using a query in di®erent languages (English, French and Dutch), the three translations of the query will be used in a single query, and the whole collection (English and/or French and/or Dutch documents) will be used.
The remainder of this paper is organized as follows. Section 2 describes our retrieval model. Section 3 details the runs and discuss the results. Finally, a conclusion and future work are done in section 4. 2 2.1
To calculate a relevance score for images using the metadata or the textual content, we used the Lucene library for indexing and searching.
Let q be a query, t a term of q , dj a given document and imi;j is an image
belonging to the document dj. The relevance score of an image equals the
relevance score of the document containing the image. It is calculated according to
the following formula provided by the Lucene search engine [
S(imi;j; q) = coord(q; dj) ¤ queryN orm(q)¤ X (tf (t 2 dj) ¤ idf (t)2 ¤ boost(t:f ield 2 dj) ¤ lengthN orm(t:f ield 2 dj)) (1) where : { coord(q; d): Coordination factor, based on the number of query terms the document contains. The coordination factor gives a boost to documents that contain more of the search terms than other documents. { queryN orm(q): Normalization value for a query, given the sum of the squared weights of each of the query terms. { tf (t 2 dj): Term frequency factor for the term t in the document di: how many times the term t occurs in the document. { idf (t): Inverse document frequency of the term: a measure of how "unique" the term is. Very common terms have a low idf ; very rare terms have a high idf . { boost(t:f ield 2 d): Field and document boost, as set during indexing. It can be used to statically boost certain ¯elds and certain documents over others. { lengthN orm(t:f ield 2 d): Normalization value of a ¯eld, given the number of terms within the ¯eld. This value is computed during indexing and stored in the index norms. Shorter ¯elds (fewer tokens) get a bigger boost from this factor. 2.2
In order to combine both scores of textual content and metadata, we propose to use either a simple linear combination or a comparison-based combination.
To calculate a linear combination score of each image SLC (im), we used the following equation:
SLC (im) = ® £ Stxt(im) + (1 ¡ ®) £ Smd(im) where Stxt(im) is the image score according to the textual content and Smd(im) is image score according to the metadata.
Since both scores of metadata (speci¯c information) and textual content (generic information) are computed by the same equation and the same way, we propose here to use for each image only the best factor, which gives the best representation for the image. To achieve this idea, we propose to use the maximum score of both factors after normalization, as follows:
SCC (im) = M ax(Stxt(im); Smd(im)) Where SCC (im) is the ¯nal score of the image after combination. (2) (3) 3 3.1
In this section, we present the results obtained by our approach when using the metadata and textual content. Table 1 presents our o±cial runs. { redcad01tx: This run used only the English version of textual content and queries. In fact, a score was calculated for each document using the formula 1, and then, this score was attributed to all images in this document. { redcad02tx: This run used all provided textual content and all query languages. More precisely, we have concatenated the queries provided in three language in a single query, and then performed the search in the textual content in di®erent languages. { redcad01md: This run used only the English version of the metadata and the queries. All extracted ¯elds are used with the same importance i.e. the boost factor in formula 1 was ¯xed to 1. { redcad02md: This run used the metadata with all queries language as in the case of redcad02tx approach. All ¯elds are used with the same importance.
According to the di®erent metrics, results show that the use of a query composed of three languages (redcad02tx and redcad02md) is more e±cient than the use of a single language query (redcad01tx and redcad01md). This result is expected given that the images are described or belonging on documents in di®erent languages. In fact, it is possible that some relevant images are described in a language and the query is in another language, then the image cannot be retrieved.
By comparing the use of textual content and the metadata, we note that results obtained by using textual content outperform results obtained by using the metadata. A possible reason is that some relevant images have a short or no associated metadata. 3.2
In our additional runs, we combined the best results obtained by using the textual content and the metadata. In these experiments, the whole collection and only one query composed of the three queries were used.
Table 2 presents obtained results. Our o±cial runs are in grayed boxes.
According to the di®erent metrics, the combination of both scores of textual content and metadata improves the retrieval accuracy. More precisely, best results are obtained with ® 2 [0.4..0.6]. Consequently, we can conclude that both factors are complementary and support the image retrieval.
Concerning the comparison-based combination, table 2 presents obtained results. We note here that using a comparison-based combination improves slightly results of a classical linear combination. 3.3
Thanks to previous experiments, we can conclude that using generic information (contextual content) is more signi¯cant than using speci¯c information (metadata) about images. However, combining both factors improves results, so they are both important to determine image relevance. In fact, both factors are complementary.
To compare fairly our runs to o±cial ones in the Wikipedia retrieval, we take into account only o±cial runs using textual information without relevance feedback or query expansion. Table 4 shows o±cial results ranked by MAP:
According to this table, we have obtained third rank, but if we include our best run which is obtained by comparison-based combination (MAP=0.2537), our rank becomes the second. 4
In this paper, we addressed the context-based image retrieval approach. We studied and evaluated the impact of using two contextual factors: textual content and metadata in image retrieval. Results in the Wikipedia Clef 2011 collection showed that the textual content which contains general information about images is more signi¯cant then the metadata which contains speci¯c information, but the combination of both factors is encouraged and consequently the two factors are complementary.
In addition, comparing the use of monolingual and multilingual retrieval according to the query language, we can conclude that the use of a query composed of di®erent language is more interesting than the use of queries in a single language.
In future work, we plan to extract and use semantics of image contextual factors in image retrieval and to add a content based image retrieval in our system.