Appendix A Results of the TEL@CLEF Track Giorgio Maria Di Nunzio dinunzio@dei.unipd.it 0 Nicola Ferro ferro@dei.unipd.it 0 Department of Information Engineering University of Padua Italy 591 632 -

Introduction 3

Results for CLEF 2009 Ad-hoc TEL@CLEF Track 3. Individual Experiment Results and Graphs This section provides the individual results for each official experiment. For each experiment the following tables and graphs are shown: - Overall statistics and information - Interpolated recall vs precision averages plot - Average precision statistics and box plot - Average precision comparison to median plot - Document cutoff levels vs precision at DCL plot - R-Precision statistics and box plot - R-Precision comparison to median plot Topics are identified with DOIs, as well as the experiments. The prefix for the DOI of a topic is 10.2452. The following example shows how to build the DOI for a topic given its number: for topic 200-AH, the corresponding DOI is 10.2452/200-AH List of Submitted Experiments 7 en en en en en en en en en en en en en en en en en en en en en en en en en en en en en en en en en en en en en en en en

TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD T TD TD TD TD TD TD TD TD TD TD T TD TD TD TD TD TD TD TD TD TD TD TD TD en en en en en en de de de de de de de de de de de de de de de de de de de de de de de de de de de de de de de de de de

TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD T TD TD TD TD TD TD TD TD TD TD TD TD TD TD de fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr el el el el

TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD T TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD T T T T aeb celi celi celi celi celi AH-TEL-BILI-X2EN-CLEF2009 el de de fr fr it de fr de fr de de fr fr zh zh zh de de de fr fr de de de de de de fr fr de de fr fr fr de fr de fr en

T TD TD TD TD TD TD TD TD TD TD TD TD TD T TD TD TD TD TD TD TD TD TD T TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD chemnitz en fr fr it en fr en fr zh en fr fr en en en fr fr en en fr fr en fr en fr de en en it it en de de en de de en en de de

TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD Track Overview Results and Graphs 15

 Topic Identifier

Ad−Hoc TEL Monolingual English Task Top 5 Participants − Comparison to Median Average Precision by Topic (Topics 726−AH to 750−AH) 1

 Topic Identifier

s t n e m i r e p x E 10.2455/TUKEY_T_TEST.A08D93626F6F439C96C3FC146DDD2BBB s t n e m i r e p x E Precision averages (%) for individual queries

Ad−HocTELBilingualFrenchTask−ComparisontoMedianAveragePrecisionbyTopic(Topics701−AHto725−AH) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 15 t n e m ir e p xE10 e h ft so cp i o fT 5 o r e b m u N

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 Average Precision DE_INDEXBL DE_INDEXBL

Precision averages (%) for individual queries

Ad−HocTELBilingualFrenchTask−ComparisontoMedianR−PrecisionbyTopic(Topics701−AHto725−AH) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 10 t n e irm 8 e xp E the 6 f so c iop 4 T f o rbe 2 m u N

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 R−Precision DE_INDEXBL DE_INDEXBL

0.0747 IR with multiple indexes, ESA and aggregated scores 0.2223 Precision averages (%) for individual queries

Ad−HocTELBilingualFrenchTask−ComparisontoMedianAveragePrecisionbyTopic(Topics701−AHto725−AH) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 0%5 Precision averages (%) for individual queries

Ad−HocTELBilingualFrenchTask−ComparisontoMedianR−PrecisionbyTopic(Topics701−AHto725−AH) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 R−Precision EN_INDEXALLESA EN_INDEXALLESA

0%0% Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment Precision averages (%) for individual queries

Ad−HocTELBilingualFrenchTask−ComparisontoMedianAveragePrecisionbyTopic(Topics701−AHto725−AH) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 Average Precision EN_INDEXBL EN_INDEXBL

0%5 Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment R_PRECISION Precision averages (%) for individual queries

Ad−HocTELBilingualFrenchTask−ComparisontoMedianR−PrecisionbyTopic(Topics701−AHto725−AH) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 R−Precision EN_INDEXBL EN_INDEXBL

0%0% Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment Precision averages (%) for individual queries

Ad−HocTELBilingualFrenchTask−ComparisontoMedianAveragePrecisionbyTopic(Topics701−AHto725−AH) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 Average Precision TCDDEFRRUN2 TCDDEFRRUN2

0%5 R_PRECISION Precision averages (%) for individual queries

Ad−HocTELBilingualFrenchTask−ComparisontoMedianR−PrecisionbyTopic(Topics701−AHto725−AH) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 R−Precision TCDDEFRRUN2 TCDDEFRRUN2

0%0% Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment Precision averages (%) for individual queries

Ad−HocTELBilingualFrenchTask−ComparisontoMedianAveragePrecisionbyTopic(Topics701−AHto725−AH) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 Average Precision TCDDEFRRUN3 TCDDEFRRUN3

0%5 Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment R_PRECISION Precision averages (%) for individual queries

Ad−HocTELBilingualFrenchTask−ComparisontoMedianR−PrecisionbyTopic(Topics701−AHto725−AH) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 25 t n e irm20 e xp E the15 f so c iop10 T f o rbe 5 m u N

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 R−Precision TCDDEFRRUN3 TCDDEFRRUN3

0%0% Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment Precision averages (%) for individual queries

Ad−HocTELBilingualFrenchTask−ComparisontoMedianAveragePrecisionbyTopic(Topics701−AHto725−AH) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 25 t n e irm20 e xp E the15 f so c iop10 T f o rbe 5 m u N

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 Average Precision TCDENFRRUN2 TCDENFRRUN2

0%5 Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment R_PRECISION Precision averages (%) for individual queries

Ad−HocTELBilingualFrenchTask−ComparisontoMedianR−PrecisionbyTopic(Topics701−AHto725−AH) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 25 t n e irm20 e xp E the15 f so c iop10 T f o rbe 5 m u N

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 R−Precision TCDENFRRUN2 TCDENFRRUN2

0%0% Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment Precision averages (%) for individual queries

Ad−HocTELBilingualFrenchTask−ComparisontoMedianAveragePrecisionbyTopic(Topics701−AHto725−AH) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 25 t n e irm20 e xp E the15 f so c iop10 T f o rbe 5 m u N

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 Average Precision TCDENFRRUN3 TCDENFRRUN3

20% 10% 0%5 Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment R_PRECISION Precision averages (%) for individual queries

Ad−HocTELBilingualFrenchTask−ComparisontoMedianR−PrecisionbyTopic(Topics701−AHto725−AH) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 25 t n e irm20 e xp E the15 f so c iop10 T f o rbe 5 m u N

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 R−Precision TCDENFRRUN3 TCDENFRRUN3

Overall statistics for 50 queries : Total number of documents over all queries Retrieved Relevant Relevant retrieved Priority

Query Construction 50,000 Source Language 1,853 Topic Fields 995 Pooled 0.0131 CL-ESA with 169283 dimensions, querying the french 0.1370 collection with german topics 0%0% Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment Precision averages (%) for individual queries

Ad−HocTELBilingualFrenchTask−ComparisontoMedianAveragePrecisionbyTopic(Topics701−AHto725−AH) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 30 t n iem25 r e p xE20 e h ftso15 cp i o fT10 o r e b m 5 u N 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 Average Precision

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 Average Precision CLESA169283DEINFR CLESA169283DEINFR

20% 10% 0%5 Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment R_PRECISION Precision averages (%) for individual queries

Ad−HocTELBilingualFrenchTask−ComparisontoMedianR−PrecisionbyTopic(Topics701−AHto725−AH) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 25 t n e irm20 e xp E the15 f so c iop10 T f o rbe 5 m u N 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 R−Precision

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 R−Precision CLESA169283DEINFR CLESA169283DEINFR

Overall statistics for 50 queries : Total number of documents over all queries Retrieved Relevant Relevant retrieved Priority

Query Construction 50,000 Source Language 1,853 Topic Fields 1,152 Pooled 0.0442 CL-ESA with 169283 dimensions, querying the french 0.1552 collection with english topics 0%0% Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment Precision averages (%) for individual queries

Ad−HocTELBilingualFrenchTask−ComparisontoMedianAveragePrecisionbyTopic(Topics701−AHto725−AH) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 25 t n e irm20 e xp E the15 f so c iop10 T f o rbe 5 m u N 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 Average Precision

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 Average Precision CLESA169283ENINFR CLESA169283ENINFR

Docs Cutoff Levels Precision at DCL (%) 5 docs 22.80 10 docs 20.20 15 docs 20.00 20 docs 19.80 30 docs 18.33 100 docs 11.38 200 docs 7.25 500 docs 3.86 1000 docs 2.30 R-Precision (precision after R document retrieved, where R = Relevant retrieved) 18.61 100% 90% 80% 70% 60% 40% 30% 20% 10% 0%5 Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment R_PRECISION Precision averages (%) for individual queries

Ad−HocTELBilingualFrenchTask−ComparisontoMedianR−PrecisionbyTopic(Topics701−AHto725−AH) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 R−Precision CLESA169283ENINFR CLESA169283ENINFR

Overall statistics for 50 queries : Total number of documents over all queries Retrieved Relevant Relevant retrieved Priority

Query Construction 50,000 Source Language 1,853 Topic Fields 973 Pooled 0%0% Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment −1 701−AH702−AH703−AH704−AH705−AH706−AH707−AH708−AH709−AH710−AH711−AH712−AH713−AH714−AH715−AH716−AH717−AH718−AH719−AH720−AH721−AH722−AH723−AH724−AH725−AH

 TopicIdentifier

Precision averages (%) for individual queries

Ad−HocTELBilingualFrenchTask−ComparisontoMedianAveragePrecisionbyTopic(Topics701−AHto725−AH) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 25 t n e irm20 e xp E the15 f so c iop10 T f o rbe 5 m u N 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 Average Precision

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 Average Precision COMBINEDDEINFR COMBINEDDEINFR

20% 10% 0%5 Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment Combined rankings, each computed with a different topic field, querying the french collection with german topics 5.88 0.00 0.00 0.00 12.96 21.74 20.59 16.67 15.00 13.73 3.45 10.98 52.50 45.00 20.37 8.33 45.45 0.00 12.90 1.59 31.67 0.00 33.33 23.73 12.36 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 20 t n e m i rpe15 x E e h ftso10 cp i o T f o re 5 b m u N 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 R−Precision

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 R−Precision COMBINEDDEINFR COMBINEDDEINFR

−1 701−AH702−AH703−AH704−AH705−AH706−AH707−AH708−AH709−AH710−AH711−AH712−AH713−AH714−AH715−AH716−AH717−AH718−AH719−AH720−AH721−AH722−AH723−AH724−AH725−AH

 TopicIdentifier

Ad−HocTELBilingualFrenchTask−ComparisontoMedianR−PrecisionbyTopic(Topics726−AHto750−AH) −1 726−AH727−AH728−AH729−AH730−AH731−AH732−AH733−AH734−AH735−AH736−AH737−AH738−AH739−AH740−AH741−AH742−AH743−AH744−AH745−AH746−AH747−AH748−AH749−AH750−AH

 TopicIdentifier

Overall statistics for 50 queries : Total number of documents over all queries Retrieved Relevant Relevant retrieved Priority

Query Construction 50,000 Source Language 1,853 Topic Fields 611 Pooled 20% 10% 0%0% Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment 6.61 0.00 0.01 2.80 8.03 0.40 7.93 0.02 10.00 3.26 0.48 0.98 18.75 18.93 17.18 2.22 35.58 0.36 4.72 3.77 20.44 0.25 0.00 10.42 1.83 −1 701−AH702−AH703−AH704−AH705−AH706−AH707−AH708−AH709−AH710−AH711−AH712−AH713−AH714−AH715−AH716−AH717−AH718−AH719−AH720−AH721−AH722−AH723−AH724−AH725−AH

 TopicIdentifier

Ad−HocTELBilingualFrenchTask−ComparisontoMedianAveragePrecisionbyTopic(Topics726−AHto750−AH) −1 726−AH727−AH728−AH729−AH730−AH731−AH732−AH733−AH734−AH735−AH736−AH737−AH738−AH739−AH740−AH741−AH742−AH743−AH744−AH745−AH746−AH747−AH748−AH749−AH750−AH

 TopicIdentifier

Precision averages (%) for individual queries

Ad−HocTELBilingualFrenchTask−ComparisontoMedianAveragePrecisionbyTopic(Topics701−AHto725−AH) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 40 ten35 m i rpe30 x E ftsoe2205 h cp i foT15 o re10 b m u 5 N 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 Average Precision

Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

 Average Precision COMBINEDENINFR COMBINEDENINFR

80% 70% 60% 500 1000 Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment Combined rankings, each computed with a different topic field, querying the french collection with english topics Interquartile range Mean Mean With No Outliers Std With No Outliers 701-AH 702-AH 703-AH 704-AH 705-AH 706-AH 707-AH 708-AH 709-AH 710-AH 711-AH 712-AH 713-AH 714-AH 715-AH 716-AH 717-AH 718-AH 719-AH 720-AH 721-AH 722-AH 723-AH 724-AH 725-AH

 COMBINEDENINFR COMBINEDENINFR