In this paper we present the results of BUAP/UPV universities in WebCLEF, a particular task of CLEF 2005. Particularly, we evaluate our information retrieval system in the bilingual English to Spanish track. Our system uses a term reduction process based on the Transition Point technique. Our results show that it is possible to reduce the number of terms to index, thereby improving the performance of our system. We evaluate di erent percentages of reduction over a subset of EuroGOV, in order to determine the best one. We observed that after reducing the 82.55% of the corpus, a Mean Reciprocal Rank of 0.0844 was obtained, compared with 0.0465 of such evaluation with full documents.
High volume of information in Internet leds to developed novel techniques for managing of data, specially when we deal with information in multiple languages. There are su cient example scenarios in which users may be interested in information which is in a language other than their own native language. A common language scenario is where a user has some comprehension ability for a given language but s/he is not su ciently pro cient to con dently specify a search request in that language. Thus, a search system that can deal with this problem should be of a high bene t. The World Wide Web (WWW) is a natural setting for cross-lingual information retrieval; the European Union is a typical example of a multilingual scenario, where multiple users have to deal with information published in at least 20 languages.
In order to reinforce research in this area, CLEF (Cross-Language Evaluation Forum) has been
compiling a set of multi-lingual corpora and promoting the evaluation of multiple multi-lingual
information retrieval systems for diverse kinds of data [
This paper reports results on the evaluation of a Cross-Language Information Retrieval System
(CLIRS) for the bilingual English to Spanish subtask of WebCLEF 2005. A document indexing
reduction is proposed, in order to improve precision of CLIRS and to diminish the storing space
on such systems. Our proposal is based on the use of the Transition Point (TP) technique, which
is somehow a method that obtains important terms from a document. We evaluate di erent
percentages of TP over a subset of EuroGOV corpus [
The next section describes our information retrieval system in detail. Section 3 brie y introduces the corpus used in our experiments, and the results obtained after evaluation. Finally, a discussion of our experiments is presented. 2
We used a boolean model with Jaccard similarity formula for our CLIRS. Our goal was to determine the behaviour of document indexing reduction in an information retrieval environment. In order to reduce the terms from every document treated, we applied a technique named Transition Point, which is described as follows. 2.1
The transition point is a frequency value that splits the vocabulary of a document in two sets of
terms (low and high frequency). This technique is based on Zipf Law of Word Ocurrences [
Alternatively, TP can be localized by identifying the lowest frequency (from the highest
frequencies) that it is not repeated; this characteristic comes from properties of Booth's law of low
frequency words [
Let us consider a frequency-sorted vocabulary of a document; i.e., VT P = [(t1; f1); :::; (tn; fn)], with fi fi 1, then T P = fi 1, iif fi = fi+1. The most important words are those that obtain the closest frequency values to TP, i.e.,
T PSET = ftij(ti; fi) 2 VT P ; U1 fi
U2g; where U1 is a lower threshold obtained by a given neighbourhood percentage of TP (NTP), thus, U1 = (1 N T P ) T P . U2 is the upper threshold and it is calculated in a similar way (U2 = (1 + N T P ) T P ).
We have used TP technique in diverse areas of natural language processing (NLP), like:
clustering of short texts [
Our information retrieval is based on the Boolean Model, and, in order to rank the documents retrieved, we used the Jaccard similarity function, applied to the query and every document of the corpus used. Previously, each document was preprocessed and its index terms were selected (the preprocessing phase is described in section 3.1). For this purpose, several values of a neighbourhood of TP were used as thresholds, as equation 2 indicates. 3 3.1
We used a subset of EurGov corpus for our evaluation. This subset was composed by a set of Spanish Internet pages, originally obtained from European government-related sites. We named this corpus BiEnEs.
In order to construct this corpus, for every page compiled in the EuroGOV corpus, we determine
its language by using TexCat [
The preprocessing of the BiEnEs corpus consisted of elimination of punctuation symbols, Spanish stopwords, numbers, html tags, script codes and style cascade sheets codes.
For the evaluation of BiEnEs, a set of 134 queries was composed and re ned, in order to provide gramatically correct \English" queries. Queries and assessments were created by the participants in the WebCLEF track, and the particular case of the queries were later reviewed and in some cases corrected in their English translation by the NLP Group at UNED. Queries were distributed in the following way: 67 homepages and 67 named page ndings.
We applied a preprocessing phase to this set of queries. First, we used an online translation system 1 in order to translate every query from English to Spanish. After that, an elimination of punctuation symbols, spanish stopwords and numbers was done.
We did not apply a rigorous method of translation, due to the fact that our main goal in our rst participation on WebCLEF was to determine the quality of terms reduction in our CLIRS. 3.2
In order to determine the behaviour of document indexing reduction on CLIRS, we submit to the contest, a set of ve runs, which are described as follows.
First Run: This run used \Full documents" as evaluation corpus, and conformed the baseline for our experiments. We named it the \Full" evaluation.
Second Run: This run used an evaluation corpus composed of the reduction of every document, using the TP technique with a neighbourhood of 10% around TP. We named it the \TP10" evaluation.
Third Run: This run used an evaluation corpus composed of the reduction of every document, using the TP technique with a neighbourhood of 20% around TP. We named it the \TP20" evaluation.
Fourth Run: This run used an evaluation corpus composed of the reduction of every document, using the TP technique with a neighbourhood of 40% around TP. We named it the \TP40" evaluation. Fifth Run: This run used an evaluation corpus composed of the reduction of every document, using the TP technique with a neighbourhood of 60% around TP. We named it the \TP60" evaluaton. We proposed an index reduction method for a cross-lingual information retrieval system. Our proposal is based on the transition point technique.
After submitting ve runs on the bilingual English to Spanish subtrack from WebCLEF, we observed that it is possible to reduce terms in the documents that conform the corpus of a CLIRS, not only by reducing the time needed for indexing but also by improving the precision of the results obtained by CLIRS.
Our method is linear in computational time, and therefore it can be used in practical tasks. Until now, results obtained in terms of MRR are very low, but ndings show that by applying better techniques of English to Spanish translation of queries, results can be dramatically improved.
We were concerned with the impact of indexing reduction on CLIRS, and in the future we hope to improve other components of our CLIRS, for instance, the use of vector space model, in order to improve the MRR.
The TP technique has shown an e ective use on diverse areas of NLP, and its best features for NLP, are mainly two: a high content of semantic information and the sparseness that can be obtained on vectors for document representation on models based on the vector space model. On the other hand, its language independence allows to use this technique in CLIRS, that is the matter of WebCLEF.