The second participation of the MIRACLE (Multilingual Information RetrievAl for the CLEf campaign) research group in the ImageCLEF task is described in this paper. New techniques, devoted to the combination of linguistic and statistical language processing methods, have been tested, continuing with the experiments carried out in last year This is the second time for the MIRACLE (Multilingual Information RetrievAl for the CLEf campaign) research group as a participant in the Image CLEF task. The work presented in this paper is the continuation of the experiments carried out in CLEF 2003. Some new techniques, like the inclusion of linguistic information for monolingual English tasks or the application of EuroWordnet as a translation and query expansion tool, have been developed and tested.
As can be seen in the figure, different tools have been used to process english queries:
BRILL: A tagger, based on Brill's work [
SETA Module: In a final step, the text can be divided in sentences and their constituent phrases can be extracted using this module. Thus, this module is a linguistic parser for english, implemented using prolog.
EWN synonym: This box represents a subsystem used to extract the corresponding WordNet synonyms for a given word. So, a query expansion is implemented based on semantic information contained in WordNet database. Optionally, the linguistic category of a given word is used when the semantic expansion is performed. For example, if a word acting as a name is going to be expanded, only synonyms of the given word that can act as a name are considered.
The rest of languages have been treated using EuroWordNet, where available, or translation tools, like Systran
[
For the rest of languages, only the baseline index database has been used to search translated queries.
Tokenize
Stopwords
Tagger
Nouns
ming
√ × √ × √ × √ × × √ × √ × √ × √ × × √ √ × × √ √ Query Process
DB 2 DB 2 DB 2 DB 3 DB 3 DB 4 DB 4 DB 4 DB 3 DB 3
Run Name mirobaseen mirosbaseen mironounen mirosnounen miroscnounen miroppbaseen mirosppbaseen miroppnounen mirosppnounen miroscppnounen mirorppbaseen mirorscppbaseen In this table, 'Topic Words' means that all simple words (excluding stopwords) are used to search the corresponding index database. 'Synonyms' means that all synonyms for a word found in WordNet are used to expand the query, without any refinement. 'Nouns' stands for the situation where the query text is tagged and only words acting as nouns are selected as part of the final query. 'Proper Names' is used to mark that only recognized proper names in the text are used as part of the query. 'Synonyms with category' is used to distinguish the process in which not all the synonyms of a words are taken into account, but only those synonyms that can act with the same category than the initial word are included in the query. Finally, in the last two experiments included in the table, the narrative of the query (only available for the english queries) is used as the input to the SETA module, in charge of parsing the text and getting a more precise category for the word.
Regarding these results, it is important to highlight some points: first of all, the basic experiment (taken as the baseline) produces the best results. Very different results are obtained from the fourth result in advance and again a gap in the average precision can be seen from the eighth result in advance. These differences in precision show that, when all words are used in the characterization of the textual captions results are better and the inclusion of more linguistic information (like proper nouns or synonyms) does not lead to an improvement. On the other side, if only common or proper nouns are used to represent the documents there is a loss in precision, perhaps due to the fewer number of words used for document characterization. Also, its worth mentioning that the experiment using all linguistic information that available tools can extract is among the worse ones
For the bilingual experiments two different approaches, depending on available information, have been applied. These two approaches are:
A EuroWordNet based approach, where information contained in the ILI index provided by EuroWordNet is used to translate the original query. This approach has been applied for the following languages: Spanish, German, French and Italian.
A translator based approach, where online translation tools, in particular Systran and Translation Experts tools, have been applied to translate the queries from the initial language to the target language (English in Image CLEF tasks).
In both approaches, the index database used corresponds to the baseline, i.e., the one where all words (excluding stopwords) are considered as indexes. Table 4 summarizes the features of the experiments defined for this multilingual task.
Tokenize
Remove
Stopwords
EWN Languages
web translator: BabelFish (http://babelfish.altavista.com)
web translator: TransExp (http://www.tranexp.com)
DB 1 DB 1 DB 1 Run Names mirowbaseit mirowbasefr mirowbasees mirowbaseesc mirowbasege mirobaseru mirobaseja mirobasezh mirobasedu mirobasesw mirobaseda
According to these results, one important fact to mention is the loss of precision, taking into account the best
monolingual experiment. As can be seen, a decrease of 34,07% in precision, marking again the importance of the
quality of the translators used in multilingual environments. Cases where EuroWordNet has been used as a
translation tool can be compared with CLEF 2003 obtained results [
Mixing text based retrieval with content based image retrieval (CBIR) for the Adhoc task This year, the MIRACLE team has made a first step in image content retrieval. This first step has led to the definition of experiments where content based image retrieval (CBIR) is applied. This is the case of the adhoc retrieval task, where some runs mixing results obtained using textual search and CBIR search have been submitted. The CBIR subsystem used for this experiment is based on GIFT 0.1.9 and will be described in the next section. The text retrieval subsystem is the one used in text based experiments, although for initial test and tuning of the overall system, last year data and text search systems have been used.
The process of mixing textual and image results begins taking the list with the images returned by the text search subsystem and their relevance figures and building a query for the CBIR subsystem. The content search is performed and a new search is performed considering the 5 first elements returned. Finally, results obtained with this last relevance feedback approach are combined with the original results list returned by the textual search subsystem. The expression used to combine these partial lists is: k REL _ VIS weight _ vis × REL _ TXT weight _ txt , factor_vis, factor_txt, In this expression, REL_VIS and REL_TXT are the relevance value returned by the CBIR subsystem and the text search subsystem respectively. factor_vis, factor_txt, weight_vis and weight_txt are parameters to be defined and can be used to adjust the overall system according to obtained results, for example, giving more importance to textual results or CBIR results.
Results of the text and CBIR mixing experiments Two sets of experiments have been done. Results for the first set are included in Table 6, where the initial set of text search results have been some of the experiments defined in Table 2. Comparing to Table 3, these results are very close (and always below) of the ones where only textual search is applied. This can be due to the chosen configuration of the combination algorithm. More tests should be made to extract a valid conclusion.
Some other experiments where executed using a different textual search subsystem. Obtained results for these experiments (Table 7 and Table 8) have been always worse than the previously mentioned ones. One of these sets of experiments, Table 8, is a bilingual one with English as a target language and Spanish as the initial language.
This year ImageCLEF organizers have defined a new task where the main focus is image content based retrieval. For this purpose a set of medical images, including scans, x-ray images and photographs of different illness has been made available to ImageCLEF participants.
The CBIR system used has been GIFT 0.1.9 [
Although the first step in the search process for this task must involve an image, textual descriptions of the medical cases have been used to try to improve retrieval results.
The search process can be divided in the following steps: 1. The initial query, formed by one image, is introduced in the CBIR system to obtain a set of images to define the query. 2. The CBIR system returns a list of images along with the corresponding relevance values. The number of images used in the search process is called relevance threshold and constitutes a system configuration parameter. 3. Previous steps have produced a valid query which is introduced in the overall system. The complete system is formed by a textual subsystem and a CBIR subsystem. In a first step both subsystemas are used to perform the search process. 4. Partial results lists are combined using an intersection operator: images not appearing in both partial lists are dropped. Two special parameters make it possible to consider textual results more important than CBIR ones or vice versa. 5. The previous step produces a unique results list that is again introduced in the CBIR subsystem. The new results list obtained is again combined, applying the intersection operator, with the output of the textual subsystem.
The overall process is depicted in Figure 2. The expression used to obtain a unique relevance value according to the partial results lists produced by textual and CBIR subsystems is:
k REL _ VIS weight _ vis × REL _ TXT weight _ txt , Rel = for elements appearing only in one partial list Some other models, based on different ways to combine the output of the textual and CBIR subsystems, have been tested but the one described here has produced the best results. Four different runs have been defined according to different values for the configuration parameters defined for the overall system. These parameters include: the minimum threshold to build the initial query, the number of results used for relevance feedback and the weights given to textual and image results.
Average precision figures obtained for the submitted experiments are included in Table 9. As can be seen, the difference in precision among the first and the last run is around 2%, not enough to extract some conclusions about which method (or configuration parameters set) is the best. On the other hand, according to the obtained rank for these runs, there is still room for improvement in this task, perhaps testing new configurations or new values for defined parameters or taking the most of textual descriptions related to each medical case.
This is the second year for the MIRACLE team taking part in the CLEF campaign and in the ImageCLEF track in particular. The main goal pursued this year was to continue with the research in finding a right combination of linguistic and statistical methods to improve the Information Retrieval process. MIRACLE group is also very interested in the field of multimedia retrieval so, the content based image retrieval task defined this year as part of the ImageCLEF track was a great opportunity to take a first step in the field. From out point of view, obtained results for the adhoc retrieval task are very good. Average precision values for the monolingual english task are a little bit better than the ones obtained last year, pointing that is difficult to improve results for this task. Perhaps the best performance figures that can be obtained with actual technology have been reached. On the contrary, bilingual tasks, in the way we have developed them, can be improved.
A mention apart must be made for the content based image retrieval task, where obtained results are not as good as for the textual task. This fact drives us to increase efforts devoted to this kind of retrieval for the following campaigns.
This work has been partially supported by the projects OmniPaper (European Union, 5th Framework Programme for Research and Technological Development, IST-2001-32174) and MIRACLE (Regional Government of