This paper describes our participation in the Medical Image Retrieval task of Image CLEF 2008. Our aim was to evaluate different combination methods for purely textual and visual approaches. Our most interesting conclusion is that combining results provided by both methods using classical combination function allows to obtain higher retrieval accuracy in terms of MAP . MAP values than combination according to query type. Moreover, it is more reliable than using only textual retrieval or using only visual retrieval.
• Content Based Image Retrieval (CBIR) systems use low-level image features to return images similar to an image used as example. The main problem of this approach is that visual similarity may not correspond to semantic similarity (for example a CBIR system can return a picture of blue sky when the example image is a blue car).
In order to take advantages of both techniques, we propose in this paper an evaluation of
combination methods in the Medical retrieval task of CLEF 2008. Our aim is to compare classical
combination using a linear combination function and a combination method that takes into
account the query type: visual, textual, mixed. The two systems we used for content-based image
retrieval and text-based image retrieval are respectively GIFT [
The rest of the paper is organized as follows. Section 2 describes our approach to evaluate Medical Retrieval queries. In fact, we describe both used systems and our combination techniques. In section 3, we present an empirical evaluation of the proposed methods carried out using the Medical Retrieval Task in Image CLEF 2008. We conclude in section 4 with a discussion on our finding and suggestions for future work. 2 2.1
We use the XFIRM XML search engine [
The model is based on a relevance propagation method. During query processing, relevance scores are computed at leaf nodes level and then at inner nodes level thanks to a propagation of leaf nodes scores through the document tree. An ordered list of subtrees is then returned to the user.
Let q = t1, . . . , tn be a query composed of n terms. Relevance values of leaf nodes are computed using a similarity function RSV (q, ln).
n
q RSV (q, ln) = X wi ∗ wiln, i=1 where wiq = tfiq ∗ idfi and wiln = tfiln ∗ idfi (1) Where wiq and wiln are the weights of term i in query q and leaf node ln respectively. tfiq and tfiln are the frequency of i in q and ln respectively, idfi = log(|D|/(|di| + 1)) + 1, with |D| the total number of documents in the collection, and |di| the number of documents containing i.
Each node in the document tree is then assigned a relevance score which is function of the relevance scores of the leaf nodes it contains.
rn = |Lrn|. X
αdist(n,lnk)−1 lnk∈Ln (2) dist(n, lnk) is the distance between node n and leaf node lnk in the document tree, i.e. the number of edges that are necessary to join n and lnk, and α ∈]0..1] allows to adapt the importance of the dist parameter. |Lrn| is the number of leaf nodes being descendant of n and having a non-zero relevance value (according to equation 1).
In the indexing phase, only two fields of documents of the medical textual collection are indexed (”caption” and ”title”) as they are the only ones which contain significant textual information. As the aim of the task is to return image identifier (document), we take α = 1 in equation 2 to ensure that best ranked elements will be documents. As each document corresponds to a single image, we indexed documents using the image name and not the document name, in order to return directly the required element (image). In the rest of the paper, document relevance is thus equivalent to image relevance in the retrieval process. 2.2
GIFT or GNU Image Finding Tool [
It uses common techniques from content-based information retrieval [
In the experiments presented here, we directly used GIFT results kindly provided by organizers, with no further processing. 2.3
In ImageCLEFmed 2008, there are 30 topics composed of both textual and images example queries, with 10 topics classified in the following three categories: • Visual : topics where visual system alone is expected to reach good performance (a visual query example is given in figure 1)
Show me images of a knee x-ray.
Zeige mir Röntgenbilder des Knies.
Montre-moi des radiographies du genou. • Mixed : topics with a visual influence but where text can improve results strongly (a mixed query example is showed in figure 2)
Show me all x-ray images containing one or more fractures.
Zeige mir alle Brüche auf Röntgenbildern.
Montre-moi toutes les fractures sur des radiographies. microscopic giant cell
Mikroskopie Riesenzellen
Image microscopique de cellules géantes
We evaluated two combination methods between the XFIRM and GIFT systems. 2.3.1
In this approach, we used the two aforementioned systems on the whole set of queries and merged their results to obtain a single result list.
The overall structure of this approach is depicted in figure 4.
Example Image Textual corpus
Example Image
Example
Image XFIRM system
GIFT system
To merge the two results lists into a single list of ranked results we first normalize scores obtained by the two systems, and then we use a simple and classic linear combination of evidences: F S(image) = α · SXF IRM (image) + (1 − α) · SGIF T (image) (3) when α is a pivot parameter ∈ [0..1], SXF IRM (image) represents the image score obtained using the XFIRM system, and SGIF T (image) is the score of the same image obtained by the GIFT system.
When α is set to 1, only the image score from the XFIRM system is used. On the other hand, only the image score by the GIFT system is used when the value of α is set to 0. Our official runs in Medical Image Retrieval task were submitted with α equal to 0.1, 0.5, 0.9 and 1.0. The fusion scores of the image and textual retrieval methods are then ranked in a descending order. 2.3.2
We also evaluated the processing of each category of queries with a different system. We thus used the GIFT system to evaluate visual topics, the XFIRM system to evaluate textual (semantic) topics, and a classic combination function (Equation 3) of the two systems to evaluate mixed topics. Figure 5 illustrates our approach.
Example Image
Example Image
Example
Image Images corpus
Textual corpus Classical combination approach The MAP of this run showed that performance increases comparatively to textual processing and classical combination function.
More work and experimentations are needed to show the difference and the effectiveness of each approach. We plan in future work to evaluate our approaches using other systems as results are also depending of the systems used for retrieval.
Table 2 shows results for each query category using respectively the XFIRM system, the GIFT system and the best combination of both (with α = 0.9). For all query categories, best MAP is obtained using classical combination with α = 0.9. So the intuition that visual queries must be processed with a CBIR system, textual queries must be processed with textual based retrieval system and mixed queries must be processed with a combination of both is not validated in the results. This shows that the classification of queries into visual, mixed and textual categories is not adequate in improving results, and consequently, using classical combination remains the best way to have best results.
Our proposed approach described in figure 5 is very simple as it used queries classification
provided by organizers. We think that it can be improved and give best results by studying
techniques of automatic query classification using other features. In fact, query classification is
an active research field where Good [
In this paper, we evaluated a classical combination approach and an approach based on queries classification using two systems (GIFT and XFIRM). Best results for the MAP metric are obtained using the classical combination method, more precisely with giving more importance for textual information than visual ones. However, our proposed method of query classification processing leads to a significant performance degradation. This can be explained by the fact that classify queries into visual, textual and mixed queries is not an adequate way of classification. In addition, results are also depending of the used systems.
In future work, we plan to confirm our conclusions by evaluating this approach using systems other than GIFT and XFIRM and using other collection (as Medical Image Retrieval in Image CLEF 2007). In addition, one of the most enticing directions of future work is to study the automatic classification of queries in order to more study the impact of this technique in image retrieval.