This paper describes the participation of the University of Alicante (UA) in CLEF 2005 image retrieval task. For this purpose we used an image retrieval system based on probabilistic information combined with ontological information and a feedback technique. Several information streams are created using different sources: stems, words and bigrams; the final result is obtained combining them. Also a voting-based strategy has been developed joining three different systems of participant Universities.
¤This work has been developed in the framework of the project CICYT R2D2 (TIC2003-07158-C04) and it has been partially funded by the Spanish Government through the grant BES-2004-3935. techniques based on the text that describes the image. Moreover, there are hybrid ones that combine both approaches.
There are two main techniques for text-based systems: probabilistic and natural language
processing (NLP) [
Our system combines probabilistic and automatic extracted knowledge from text that describes
the image. We have initially used a probabilistic information retrieval system: Xapian [
Furthermore, an ontology has been created using St. Andrews Corpus [
In order to deal with the multilingual view we have made an analysis with the ImageCLEF 2004 queries set and several automatic translators. Topics in different languages are first translated into English and then retrieval is performed.
This paper is structured as follows. Section 2 presents the system in depth. Section 3 explains the development of the image ontology. Section 4 provides the result of our participation in ImageCLEF 2005. Section 5 describes the joint participation. Some conclusions and future work close the document. 2
The system implements an approach based on probabilistic information and knowledge extracted
from the corpus. The system uses Xapian [
With the information of the fields, and depending on the upper or lowercase in the first letter of the token (word, stem or bigram), a weight to each of this tokens is assigned1(see Table 1).
Headline
Shtitle Description
Data
Photo Location
Notes Categories ½ 100 ¤ f ield weight if 1st letter is uppercase
50 ¤ f ield weight if 1st letter is lowercase 1The weights have been adjusted to have the best performance over Image CLEF 2004 query set.
Using this weighting scheme, we create three indexes and the indexing phase finishes. 2.2
In this phase the query topics are processed and a list of relevant documents to each query is given. The main idea is to perform three independent retrievals, one in each index and then combine them to get a unique list of retrieved images.
The first step prepares the query to be posed to the retrieval system. Given a topic, stop words are removed and then each word is transformed to have three queries, one for each index.
Once we have the three queries, a first retrieval in each index is performed. The second step is applies the relevance feedback. Xapian allows us to use feedback in a simple way: we must only select those documents which we consider relevant. Some experiments over ImageCLEF 2004 query set reveal that the number of documents to get the better results is twenty three.
In the next step the ontology is used. With this ontology we know both the categories related to the query and the categories of each retrieved document. Using this information the relevance of the documents having any category in common with the query can be increased. This process is done separately in the three retrieved lists.
The last step combines the three lists to get a global result. We believe that each stream gives a different kind of information, and thus, each stream must have a different weight. We made an analysis to make the best weight tuning considering the contribution of each information flow. ² Stem flow: It can increase the recall because with the stemming, same words but morphologically different can be matched. ² Word flow: It can increase the precision. Only words exactly equal are matched. ² Bigram flow: It can increase the precision. Word pairs, such as compound noun phrases. So, the weights selected to do the combination are2: - Stem flow: 0.5 - Word flow: 0.1 - Bigram flow: 0.3
In the combination, each document has as score the sum of its flow scores multiplied by their corresponding weight.
WDoc = 0:5 ¤ WF low + 0:1 ¤ WW ord + 0:3 ¤ WBigram 2.3
In order to deal with multilingual features we have decided to make an analysis of several online
translators. Among different well-known translators such as Babel [
In this section we are going to describe how the ontology is created and how the system uses this information in the retrieval process. This ontology represents a knowledge base, and it is an approach to use knowledge resources in information retrieval. The idea is to extract the categories related with a query and then retrieve documents with the same categories.
2These weights have been adjusted doing an analysis over ImageCLEF 2004 topics to get the best performance. 3.1
The ontology has been automatically built using the St. Andrews corpus. Each image caption has a field called <CATEGORIES> (see figure 2). For each image in the corpus the categories and the words of the <TITLE> field are extracted. The ontology is used to create an “ontology-document” database that relate each category with the corresponding words. The process is shown in figure 3. Queries are posed to the ontology index in order to retrieve a ranked list of “ontology-documents”. The score of the category document ’C’ indicates the probability of a query to require a document of this category ’C’. The category ranking results will be used for document re-ranking using the lists of relevant documents (one for each stream) obtained from first step image retrieval process. When we make the retrieval in each of the index created (stem, word and bigram), we have a list of documents. Each of these documents has several categories, so if we know the categories related to the query, we can increase the weight of those documents having some category in common with the query. Figure 4 shows this process, where weights extracted from the category database (wc1, wc2 and wc3) are added to the total weight of the image according to their common categories.
Four experiments have been proposed for the evaluation. In each one we combine different techniques in order to discover how these techniques help in the retrieval process. The features that have been merged in different experiments are: ² Uses of stems, words or bigrams(tokens) adjusting their weights. ² Use of correct fields adjusting their weights. ² Selection of different flow weights when combining the results. ² Use of categories (ontology). ² Use of automatic feedback.
Combining all these features, over 100 experiments have been performed, but we have only selected the best four in order to participate in ImageCLEF 2005. Experiments selected and their features are shown in next table:
X
X X X
X
X X
The results of the monolingual system over ImageCLEF05 query set are:
As we can see in the previous table, Experiment3 provides the best performance. Experiment3 uses stems, words and bigrams, and also implements feedback and categories knowledge.
According to Baseline, Experiment1 and Experiment2 comparison it seems that word and bigram adding do not improive the results3.
The improvement related to he use of the knowledge ontology can observed when comparing Experiment2 and Experiment3. This increase could be greater with a more sophisticated ontology, for example, use a more general ontology that combines more general concepts and specific domain terms.
3During the experiments over ImageCLEF 2004 dataset, word and bigram adding provided a slight improvement.
Apart from the system described, we have made a joint participation within the R2D2 project4 framework integrating our system and the ones belonging to UNED group from Madrid and SINAI group from Ja´en. We have developed a voting among them. A combination between UA and SINAI has been done for English, Dutch, French, German, Italian, Russian and Spanish. The three systems have been only combined for Spanish. The systems selected for the combination implement feedback and use query titles and automatic translation.
The voting has been developed using the weights of each document in each retrieved list. The first step makes a normalization of the weights for each document list dividing each weight by the maximum weight of all the documents in the list. Then the different weights of each document in each list are summed. In this way, documents that appear in the three lists are more relevant than those appearing in just two lists. Finally the documents are sorted according to the calculated weight and a final document list is generated.
MAP results obtained from the collaborative experiments are shown in the Table 4. Ranking positions are shown in brackets.
German
Italian
Russian Spanish (eur) Spanish (lat)
This paper has presented the approach used by the University of Alicante in the ImageCLEF 2005 adhoc retrieval task. This image retrieval system implements a combination of probabilistic and knowledge-based approaches. The implementation and tuning of the method have been shown and results have been explained.
To continue improving the system there are several ways that can be taken in account. One
of them is to consider the use of NLP to improve the information retrieval [
Another work to be developed is the creation and management of the ontology, that is, the use
of knowledge in the retrieval process [
4R2D2 project: Question Answering in Digital Documents. Reference TIC2003-07158-C04 financed by the Science Technology Department 2003-2006 of the Spanish Government. The main goal of the project is the evaluation and development of Question Answering and Document Retrieval systems in multilingual scenarios. See http://gplsi.dlsi.ua.es/r2d2/indexEn for more details.