The goal of our participation in the WikipediaMM task of CLEF 2008 was to study the use of the name of images in a context-based retrieval approach. We evaluated this factor in three manners. The first one consists of using image names explicitly: we computed a similarity score between the query and the name of images using the vector space model. The second one consists of combining results obtained using the textual content of documents and results obtained using the first method. Finally, in our last approach, image names are used less explicitly. We proposed to use all the textual content of documents, but we increased the weight of terms in the image name. Results show that the ”image name” is an intersting factor. Even if the image names are used as an additional source of evidence, they allow to better performance. Moreover, we conclude thet using the image name implicitly performs better results than using it explicitly.
Image retrieval is divided into two main approaches: content-based and context-based image retrieval, each one has some advantages and disadvantages.
Our approach belongs to context-based image retrieval. In this paper, we study the impact of using the name of images in the retrieval process.
We evaluated three algorithms that take into account the name of images in retrieval. In a first approach, we calculate a score for each image using its name. This score in combined in a second approach with the score of the document based on the textual content. In a third approach, we evaluate the impact of the image name implicitly: in fact, we increase the weight of terms in the image name. Consequently, documents having a relevant image name will be top ranked.
The rest of the paper is organized as follows: section 2 describes the goal of our participation in the WikipediaMM task of Clef 2008. Section 3 presents our approaches. In section 4, we detail our empirical evaluation of the proposed approaches and discuss our results. Finally, section 5 concludes with possible directions for future work. 2
In previous work on the WikipediaMM collection, we evaluated the use of ancestors, brothers and
child nodes to estimate the relevance scores of images [
This year, the aim of our participation in WikipediaClef 2008 is to study the impact of the
image names in image retrieval. Many works [
As the element type where we can find image names can change from a collection to another, we identify the names of images in documents using their extension. Any token that ends by an image extension as ”.jpeg” and ”.gif” for example is considered as an image name (see figure 1). In the indexing phase, we thus index image names in a separate index. 3.1
Baseline model: the XFIRM model
We use the XFIRM XML search engine [
The model is based on a relevance propagation method. During query processing, relevance scores are computed at leaf nodes level and then at inner nodes level thanks to a propagation of leaf nodes scores through the document tree. An ordered list of subtrees is then returned to the user.
Let q = t1, . . . , tn be a query composed of n terms. Relevance values of leaf nodes are computed thanks to a similarity function RSV (q, ln).
n RSV (q, ln) = X wiq ∗ wiln, where wiq = tfiq ∗ idfi i=1 and wiln = tfiln ∗ idfi (1) Where wiq and wiln are the weights of term i in query q and leaf node ln respectively. tfiq and tfiln are the frequency of i in q and ln respectively, idfi = log(|D|/(|di| + 1)) + 1, with |D| the total number of documents in the collection, and |di| the number of documents containing i.
Each node in the document tree is then assigned a relevance score which is function of the relevance scores of the leaf nodes it contains.
rn = |Lrn|. X
αdist(n,lnk)−1 lnk∈Ln (2) (3) (4) dist(n, lnk) is the distance between node n and leaf node lnk in the document tree, i.e. the number of arcs that are necessary to join n and lnk, and α ∈]0..1] allows to adapt the importance of the dist parameter. |Lrn| is the number of leaf nodes being descendant of n and having a non-zero relevance value (according to equation 1).
As the aim of the task is to return the whole documents (more precisely the file names of documents containing relevant images), we take α = 1 to ensure that best ranked elements will be article elements. 3.2 3.2.1
Algorithms used to evaluate the impact of image names
To explicitly use the terms composing the image name and study their importance, we first propose to only use the image name terms to retrieve relevant images. We compute a score for each image using the vector space model. We evaluated three similarity measures.
Let us consider a vector space of N dimensions, with N the number of terms in the collection, a query q, and an image name ImN amei.
The first similarity measure used is the Cosine Similarity (Equation 3).
The second one is the Dice Coefficient presented in Equation 4.
WImName(Im) = cos(q, ImN amei) = WImName(Im) = sim(q, ImN amei) =
PjN=1 wjqwjImNamei qPjN=1 (wjq)2qPjN=1 (wjImNamei )2
2 PjN=1 wjqwjImNamei
PjN=1 (wjq)2 + PjN=1 (wjImNamei )2 The last one is the Inner Product measure, expressed in Equation 5.
N WImName(Im) = sim(q, ImN amei) = X wjqwjImNamei j=1 (5) In the three formula, wqj ∈ [0..1] is the weight of term tj in the query, and wImNamei ∈ [0..1] j is the weight of tj in the image name. These weights are evaluated using a tf*idf formula.
As elements to return must be documents and not images, final results are the documents containing the relevant images. We thus have: WImName(Doc) = WImName(Im) There is no modification in ranking, as each document contains only one image. 3.2.2
Combining image name scores to document scores To evaluate the textual content of documents, we used the XFIRM system. As results returned by this system are XML elements and not documents, we rerank results according to documents: in fact, for each relevant element, we use its document in the final results. If two elements from the same document are returned, we rank the document according to the element having the best score.
The score of the document obtained thanks to the image name WImName(doc) can be combined with the score obtained by the XFIRM system WXF IRM (doc). We used the combination function presented in equation 6.
W (doc) = λWXF IRM (doc) + (1 − λ)WImName(doc) (6) where λ is a pivot parameter ∈ [0..1], WXF IRM (doc) represents the document score obtained using the XFIRM system, and WImName(doc) is the score of the same image obtained by the image name.
We propose in this section to modify the term weighting formula (equation 1) used in the XFIRM model, by increasing the score of terms belonging to the image name. The new formula is as follows: n RSV (q, ln) = X wiq ∗ wiln,
i=1 where wiq = tfiq ∗ idfi and wiln =
K ∗ tfiln ∗ idfi if ti is an image name term tfiln ∗ idfi otherwise (7) K > 1 is a constant used to increase the weight of image name terms.
In the example of figure 2, without the implicit specification of the image name, the scores of document 1 and document 2 are equal. When applying our algorithm, document 1 becomes more relevant than document 2. This seems more logic as the image of document 1 concerns the query more than the image of document 2. 4
In the following, our official runs are in grayed boxes. Results obtained using our first algorithm (equation 3, 4 and 5) are presented in table 1.
The Cosine (SigRunImageName) and Inner Product (SigRunImageNameInner) measures give approximately the same results, which are better than those using the Dice coefficient.
Table 2 shows results obtained when combining image scores evaluated using the cosine measure (equation 3) with whole document score (equation 6). Run SigRunText only uses the document textual content evaluated with the XFIRM system. The other runs, obtained by combining image score evaluated with the cosine measure and image score evaluated with all textual content of documents, shows that this combination allows to better performance. the best run obtained is the one evaluated with λ = 0.9. This means that the image name can improve results as an additionnel source of evidence but not as a main one: the importance factor of textual content is 0.9 comperatively to 0.1 for the image name.
Runs obtained by implicit use of the image name (equation 7) are detailed in table 3.
Runs K RunImage11 1.1 RunImage12 1.2 RunImage13 1.3 RunImage14 1.4 RunImage15 1.5 RunImage16 1.6 RunImage17 1.7 RunImage18 1.8 RunImage19 1.9 SigRunImage 2 SigRunImage5 5 SigRunImage10 10
Comparing the best run obtained by implicit use of image name (K = 1.1, RunImage11) to the run obtained using document textual content (SigRunText), we notice that the MAP measure of the first one is more better.
As a main conclusion about the use of image names in contextual image retrieval, we notice that the image name is a relevant contextual element of image retrieval whatever the way it is introduced (implicitly or explicitly). Despite these good results, we think that results could be more improved if we resolve the following problem: in the used collection, many image names are not well indexed as they concatenate several terms. Let us consider the example of the image name ”Alonewitheverybody.jpg”, this name is considered as a single term while in reality it is composed of 4 terms ”alone”, ”with”, ”every” and ”body”.
More work is thus needed to resolve the image name indexing problems and strongly confirm our conclusions.
Obtained results confirm our intuition that the image name is an important factor in the image retrieval when it is significant. In fact, combining the image name score and the document textual score improves results comperatively to ones obtained by these sources of evidence separately. Whereas, the more interesting result is that using the image name implicitly allows obtaining higher retrieval accuracy in term of MAP measure than using the image name explicitly. 5
In this paper, we presented a study about the impact of the use of image name terms in contextual image retrieval. We evaluated the explicit and the implicit use of these terms. Results in Wikipedia Clef 2008 showed that this factor leads to performance improvement.
Results can be more improved as we have a problem of the indew process of image names (these latter are not well indexed as terms are concatenated).
In future work, we will try to find a solution of the image name indexing problem. We also plan to evaluate our methods on other collections having significant image names.
Our implemented system can only process queries which specify exact images, then our system cannot process concept clauses. Therefore, we should consider how to calculate retrieval status values using concept clauses.