Mercure at CLEF-2 N.NASSR, M.BOUGHANEM IRIT/SIG Campus Univ. Toulouse III 118, Route de Narbonne F-31062 Toulouse Cedex 4 Email : fnassr, boughaneg@irit.fr Tel : 05-61-55-63-22 Fax: 05-61-55-63-23 1 Summary This paper presents the experiments undertaken by our team (IRIT team) in multilingual, bilin- gual and monolingual tasks at CLEF programme. Our approach to CLIR is based on query translation. In bilingual experiment a dictionary is used to translate the queries from French to English and two techniques for desambiguiation were tested: aligned corpus and dictionary strat- egy. Desambiguiation technique is applied to select the best terms from the (translated) targed queries. All these experiments were done using Mercure system [2] which is presented in section 2 of this paper. The section 3 describes our general CLIR methodology, and nally, section 4 describes experiments and results performed at CLEF programme. 2 Mercure model 2.1 Model description Mercure is an information retrieval system based on a connectionist approach and modelled by a multi-layered network. The network is composed of a query layer (set of query terms), a term layer representing the indexing terms and a document layer [1],[2]. Mercure includes the implementation of a retrieval process based on spreading activation forward and backward through the weighted links. Queries and documents can be either inputs or outputs of the network.The links between two layers are symmetric and their weights are based on the tf  idf measure inspired from the OKAPI[3] term weighting formula.  the term-document link weights are expressed by: ij  (h1 + h2  log( nNi )) tf dij = dlj (1) h3 + h4  d + h5  tfij  the query-term (at stage s) links are weighted as follows:  nq qtf (s) nqu ;qtfui si (nqu > qtfui ) u ui qui = qtfui otherwise (2) 1 2.2 Query evaluation A query is evaluated using the spreading activation process described as follows : 1. The query u is the input of the network. Each node from the term layer computes an input Q value from this initial query: ( i ) = ui and then an activation value : In t q ( i) = ( ( i )) where g is the identity function. Out t g In t 2. These signals are propagated forwards through the network from the term layer to the documentPlayer. Each document node computes an input : ( j ) = Ti=1 ( i)  ij and then an activation , In d Out t w ( j) = Out d RSV( u j ) = ( ( j )) Q ;d g In d : Notations : T : the total number of indexing terms, N : the total number of documents, qui: the weight of the term i in the query , t u i : the term i , t t dj : the document j , d w ij : the weight of the link between the term i and the document j , t d dlj : document length in words (without stop words),  : average document length, ij : the term frequency of i in the document j , d tf t d ni : the number of documents containing term i , t nq u: the query length, (number of unique terms) qtf ui : query term frequency. 3 General Clir Methodology Our CLIR approach is based on query translation. It is illustrated by three main steps: Indexing, Translation and Dismabiguation described as follows:  Indexing : a separate index is built for the documents in each language. English words are stemmed using Porter algorithm, French words are stemmed using a truncature (7 rst characters), no stemming for the German, Italian and Spanish words. The German, Italian and Spanish stoplists were downloaded from Internet.  Translation : is based on \dictionaries". For the CLEF2 experiments, ve bilingual dictionaries were used all of which were actually simply a list of terms in language 1 that l were paired with some equivalent terms in language 2. Table 1, shows the source and the l number of entries in each dictionary. Type Source nb. entries E2F http://www.freedict.com 42443 E2G http://www.freedict.com 87951 E2I http://www.freedict.com 13478 E2S http://www.freedict.com 20700 F2E http://www.freedict.com 35200 Table 1: Dictionaries characteristics  Desambiguiation: when multiple translations exist for a given term, desambiguiation was performed by selecting the bests target query terms equivalent for each source query term. 2 Two strategies of desambiguiation were tested. The rst one is based on aligned corpus and the second one is based on dictionary. The rst desambiguiation based on aligned corpus consist of: 1. Retrieving the top documents (X=20) for each source query term i in aligned corpus. t 2. Retrieving the top documents (X'=20) for each translation ij for i in the same aligned t t corpus. ij is one translation for i among another. t t 3. Desambiguiation of the translated query consist of matching the retrieval documents (pro les) from the di erent translation against the source query pro le. The best terms are the terms which have the best matchnig. The second desambiguiation based on dictionary is described as follows : 1. Each source query term i is translated in target language using bilingual dictionary. t 2. Each translation ij from i is transalted in source language using bilingual dictionary. t t tij is the one translation for 1 among another. t 3. The desambiguiation of the transalted query consist of retaining only target terms that return the source query term. However if a speci c term has an unique substitution this term is retained in all cases. 4 Experiment and Results 4.1 Multilingual experiment One run iritmuEn2A using English topics and retrieving documents from the pool of documents in all four languages (German, French, Italian, Spanish and English), was submitted. The queries were translated using the downloaded dictionaries. No desambiguiation, all the translated words were retained in the target queries. The run was performed by doing individual runs for pair languages and merging the results to form the nal ranked list. Run-Id P5 P10 P15 P30 Exact Avg. Prec. iritmuEn2A(50 queries) 0.4040 0.3520 0.3173 0.2760 0.1509 0.1039 Pair language P5 P10 P15 P30 Exact Avg. Prec. E2F (49 queries) 0.2204 0.2102 0.1823 0.1415 0.2005 0.2044 E2S (49 queries) 0.3633 0.3265 0.3116 0.2537 0.2589 0.2281 E2I (47 queries) 0.1872 0.1596 0.1475 0.1255 0.1320 0.1321 E2E (47 queries) 0.5149 0.4085 0.3518 0.2716 0.4564 0.4863 Table 2: Comparison results of pair search and multilingual list Table 2 shows the results of pair languages (example, E2F means English queries translated to French and compared to French documents, etc.). We can easily notice that the monolingual (E2E) search performs much more better than all the pair (E2F, E2G, E2I, E2S) searches. Moreover, all the pair searches have their average precision better than the multilingual search. The merging strategy caused the loss of relevant documents. 4.2 Bilingual experiment Two runs using French topics and retrieving documents from the pool of document in English language, were submitted. The bilingual experiment was carried on using French to English free dictionary + desambiguiation. Two desambiguiation strategies were tested : 3  Aligned corpus strategy : desambiguiation based on aligned corpus was performed using WAC (Word-wide-web Aligned Corpus) parallel corpus built by RALI Lab (http://www- rali.iro.umontreal.ca/wac/).  (English-French) dictionary strategy: desmabiguiation based on dictionary was performed using free (English-French) dictionary. Table 1, shows the source and the number of entries in (English-French) dictionary. Two runs were submitted: irit1bFr2En where desambiguiation based on dictionary and irit2bFr2En where desmabiguiation based on aligned corpus Ocial results Run-Id P5 P10 P15 P30 Excat Avg.Prec. irit1bFr2En 0.3660 0.2979 0.2468 0.1844 0.3258 0.3294. irit2bFr2En 0.3787 0.2957 0.2440 0.1794 0.3250 0.3398. Table 3: Comparison between the desmabiguiation strategies Table 3 compares the desambiguiation strategies. It can be seen that the desambiguiation based on aligned corpus is slightly better than the desambiguation based on dictionary at average pre- cison but no di erence at excat precision. Non ocial results Run-id (33 queries) P5 P10 P15 P30 Exact Avg.Prec irit1bFr2En 0.2638 0.1915 0.1660 0.1312 0.2304 0.2375 Dico 0.3660 0.2936 0.2397 0.1809 0.3161 0.3305 Impr (%) -27,92 -34.77 -30,74 -27.47 -27.11 -28.13 irit2bFr2En 0.3787 0.3043 0.2496 0.1851 0.3249 0.3436 Dico 0.3660 0.2936 0.2397 0.1809 0.3161 0.3305 Impr (%) 3.46 3.64 4.13 2.32 2.78 4 Table 4: Impact of the desambiguiation Table 4 compares the results between the runs irit1bFr2En and irit2bFr2En (Dictionary+desambiguiation) and Dictionary only. It can be seen that the desambiguiation based on aligned corpus is better than the dictionary and the desambiguiation based on dictionary. The desambiguiation based on aligned corpus is e ective the average precision improves of 4%. 4.3 Monolingual experiments Four runs were submitted in monolingualtasks : iritmonoFR, iritmonoIT,iritmonoGE, iritmonoSP Table 5 shows that French monolingual results seem to be better than both Italian, Spanish and the German. Italian results are better than Spanish and German. Spanish results are better than German. These runs were done using exactly the same procedures the only di erence concerns the stemming which was used only for French. We notice clearly that the monolingual search is much better than both the multilingual and the bilingual searches. 4 Run-id (33 queries) P5 P10 P15 P30 Exact Avg. Prec. iritmonoFR FR (49 queries) 0.4286 0.3898 0.3483 0.2830 0.3565 0.3700 iritmonoIT IT (47 queries) 0.4723 0.3894 0.3574 0.2730 0.3568 0.3491 iritmonoGE GE (49 queries) 0.4327 0.3816 0.3442 0.2884 0.2736 0.2632 iritmonoSP SP (49 queries) 0.4694 0.4347 0.4082 0.3626 0.3356 0.3459 Table 5: Comparison between monolingual search 5 Conclusion In this paper we have presented, our experiments for CLIR at CLEF programme. In multilingual IR, we showed that the merging strategy caused the loss of relevant documents, In bilingual IR, we showed that the desambiguiation technique based on aligned corpus for trans- lated queries is e ective. Results of experiments have also showed that using free dictionaries are fesaible, and desambiguiation based on aligned corpus give the good results even though the documents of aligned corpus are independent from those of database. References [1] M.Boughanem, C.Chrisment, C.Soule-Dupuy, Query modi cation based on relevance back- propagation in Adhoc environment, Information Processing and Managment. April 1999. [2] M.Boughanem,T.Dkaki,J.Mothe,C.Soule-Dupuy: Mercure at trec7. Proceedings of the 7th International Conference on Text REtrieval TREC7, E. M. Voorhees and Harman D.K. (Ed.),NIST SP 500-236, Nov. 1997. [3] S.Robertson and al Okapi at TREC-6, Proceedings of the 6th International Conference on Text REtrieval TREC6, Harman D.K. (Ed.), NIST SP 500-236, Nov. 1997. 5