Introduction 3
Results for CLEF 2008 Ad-hoc TEL@CLEF Track 3. Individual Experiment Results and Graphs This section provides the individual results for each official experiment. For each experiment the following tables and graphs are shown: - Overall statistics and information - Interpolated recall vs precision averages plot - Average precision statistics and box plot - Average precision comparison to median plot - Document cutoff levels vs precision at DCL plot - R-Precision statistics and box plot - R-Precision comparison to median plot Topics are identified with DOIs, as well as the experiments. The prefix for the DOI of a topic is 10.2452. The following example shows how to build the DOI for a topic given its number: for topic 200-AH, the corresponding DOI is 10.2452/200-AH List of Submitted Experiments 7 10.2415/AH-TEL-MONO-ENCLEF2008.CHEMNITZ.CUT_MULTI10_W1MINUSX_PL USPLUS 10.2415/AH-TEL-MONO-ENCLEF2008.CHEMNITZ.CUT_MULTI10_W1_PLUSPLUS 10.2415/AH-TEL-MONO-ENCLEF2008.CHEMNITZ.CUT_MULTI10_WX_PLUSPLUS 10.2415/AH-TEL-MONO-EN
CLEF2008.CHEMNITZ.CUT_SIMPLE United States 10.2415/AH-TEL-MONO-EN
CLEF2008.CHESHIRE.BKAHTELMENTDT2F United States 10.2415/AH-TEL-MONO-EN
CLEF2008.CHESHIRE.BKAHTELMENTT2FB 10.2415/AH-TEL-MONO-ENCLEF2008.IMAG.IPAL01 10.2415/AH-TEL-MONO-ENCLEF2008.IMAG.IPAL02 10.2415/AH-TEL-MONO-ENCLEF2008.IMAG.IPAL03 10.2415/AH-TEL-MONO-ENCLEF2008.IMAG.IPAL04 10.2415/AH-TEL-MONO-ENCLEF2008.INESC.RUN1 10.2415/AH-TEL-MONO-ENCLEF2008.INESC.RUN2 10.2415/AH-TEL-MONO-ENCLEF2008.INESC.RUN3 10.2415/AH-TEL-MONO-ENCLEF2008.INESC.RUN4 10.2415/AH-TEL-MONO-ENCLEF2008.ISI.IN_EXPC2C10 10.2415/AH-TEL-MONO-ENCLEF2008.KARLSRUHE.AIFB_BL_EN 10.2415/AH-TEL-MONO-ENCLEF2008.OPENTEXT.OTEN08T 10.2415/AH-TEL-MONO-ENCLEF2008.OPENTEXT.OTEN08TD 10.2415/AH-TEL-MONO-ENCLEF2008.OPENTEXT.OTEN08TDE 10.2415/AH-TEL-MONO-ENCLEF2008.OPENTEXT.OTEN08TDZ 10.2415/AH-TEL-BILI-X2ENCLEF2008.TWENTE.M 10.2415/AH-TEL-MONO-ENCLEF2008.UFRGS.UFRGS_MONO_EN1 10.2415/AH-TEL-MONO-ENCLEF2008.UFRGS.UFRGS_MONO_EN2 10.2415/AH-TEL-MONO-ENCLEF2008.UNINE.UNINEEN1 10.2415/AH-TEL-MONO-ENCLEF2008.UNINE.UNINEEN2 10.2415/AH-TEL-MONO-ENCLEF2008.UNINE.UNINEEN3 10.2415/AH-TEL-MONO-EN
CLEF2008.UNINE.UNINEEN4 AH-TEL-MONO-EN-CLEF2008 en en en en en en en en en en en en en en en en en en en en en en en en en en en en en en en en en en en en en de de de
TD TD TD TD TD T T T T T TD TD TD TD TD TD TD TD TD TD T TD TD TD TD TD TD TD TD TD T TD TD TD TD TD TD TD TD TD AUTOMATIC AUTOMATIC AUTOMATIC no no yes 10.2415/AH-TEL-MONO-DECLEF2008.CHEMNITZ.CUT_MULTI10_WX_PLUSPLUS de de de de de de de de de de de de de de de de de de de de de es de de de de de fr fr fr fr fr fr fr fr fr fr fr fr fr
TD TD T TD TD TD TD TD TD TD TD TD TD T TD TD TD TD TD TD T TD TD TD TD TD TD TD TD TD TD TD T TD TD TD TD TD TD TD 10.2415/AH-TEL-MONO-FR-CLEF2008.XEROX.AF
AH-TEL-MONO-FR-CLEF2008 10.2415/AH-TEL-MONO-FR-CLEF2008.XEROX.F fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr fr de de de es fr de es fr nl de fr nl fr es es es fr fr de de fr fr de de
TD TD T TD TD TD TD TD TD T TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD 10.2415/AH-TEL-BILI-X2DE-CLEF2008.XEROX.F AH-TEL-BILI-X2DE-CLEF2008
AH-TEL-BILI-X2DE-CLEF2008 AH-TEL-BILI-X2DE-CLEF2008 AH-TEL-BILI-X2DE-CLEF2008 AH-TEL-BILI-X2DE-CLEF2008 AH-TEL-BILI-X2DE-CLEF2008 AH-TEL-BILI-X2DE-CLEF2008 AH-TEL-BILI-X2DE-CLEF2008 AH-TEL-BILI-X2DE-CLEF2008 AH-TEL-BILI-X2DE-CLEF2008 AH-TEL-BILI-X2DE-CLEF2008 AH-TEL-BILI-X2DE-CLEF2008 AH-TEL-BILI-X2DE-CLEF2008 AH-TEL-BILI-X2DE-CLEF2008 AH-TEL-BILI-X2DE-CLEF2008 AH-TEL-BILI-X2DE-CLEF2008 AH-TEL-BILI-X2DE-CLEF2008 en en es es fr en fr en fr en en fr fr en en fr fr en en de en es de en nl en de en de en en de de
TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD Track Overview Results and Graphs 13 100% 90% 80% 0%
0% 1 0.8 0.6 0.4 0.2 −0.2 −0.4 −0.6 −0.8 −1 1 0.8 0.6 0.4 0.2 −0.2 −0.4 −0.6 −0.8 −1
Recall Levels vs Mean Interpolated Precision unine [Experiment UNINEEN3; MAP 37.54%; Not Pooled] inesc [Experiment RUN3; MAP 36.23%; Not Pooled] chemnitz [Experiment CUT_SIMPLE; MAP 35.62%; Pooled] jhu−apl [Experiment JHUMOEN4RF; MAP 35.31%; Not Pooled] cheshire [Experiment BKAHTELMENTDT2F; MAP 34.66%; Pooled] unine [Experiment UNINEEN3; MAP 37.54%; Not Pooled] inesc [Experiment RUN3; MAP 36.23%; Not Pooled] chemnitz [Experiment CUT_SIMPLE; MAP 35.62%; Pooled] jhu−apl [Experiment JHUMOEN4RF; MAP 35.31%; Not Pooled] cheshire [Experiment BKAHTELMENTDT2F; MAP 34.66%; Pooled] chemnitz [Experiment CUT_MULTI10_W1MINUSX_PLUSPLUS; MAP 16.20%; Not Pooled] chemnitz [Experiment CUT_MULTI10_W1_PLUSPLUS; MAP 14.53%; Not Pooled] xerox−sas [Experiment CACAOENGENGEXPANDED; MAP 13.30%; Not Pooled] xerox [Experiment AF; MAP 8.74%; Not Pooled] 0% 10% 20% 30% 70% 80%
Ad−Hoc TEL Monolingual English Task − Tukey T test with "top group" highlighted s t n e m i r e p x E 100% 90% 80% 70% 60% 50% 40% 30% 20% 0%
5 1 0.8 0.6 0.4 0.2 −0.2 −0.4 −0.6 −0.8 −1 1 0.8 0.6 0.4 0.2 −0.2 −0.4 −0.6 −0.8 −1
Precision unine [Experiment UNINEEN3; R−Prec 38.80%; Not Pooled] inesc [Experiment RUN3; R−Prec 36.49%; Not Pooled] chemnitz [Experiment CUT_SIMPLE; R−Prec 37.16%; Pooled] jhu−apl [Experiment JHUMOEN4RF; R−Prec 35.12%; Not Pooled] cheshire [Experiment BKAHTELMENTDT2F; R−Prec 35.15%; Pooled] chemnitz [Experiment CUT_MULTI10_W1MINUSX_PLUSPLUS; R−Prec 16.87%; Not Pooled] chemnitz [Experiment CUT_MULTI10_W1_PLUSPLUS; R−Prec 14.75%; Not Pooled] xerox [Experiment AF; R−Prec 10.42%; Not Pooled] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
R−Precision 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 25 t n e irm20 e xp E the15 f so c iop10 T f o rbe 5 m u
N
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 20 t n e m i rpe15 x E e h ftso10 cp i o T f o re 5 b m u
N
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8
Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 0%5 Precision averages (%) for individual queries 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 15 t n e m ir e p xE10 e h tf so cp i o fT 5 o r e b m u N
Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 100% 90% 80% 70% 60% 0%0% Precision averages (%) for individual queries 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8
Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
0%5 Precision averages (%) for individual queries 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 15 t n e m ir e p xE10 e h ft so cp i o fT 5 o r e b m u N
Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 0%0% Precision averages (%) for individual queries 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 xerox
Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
0%5 Precision averages (%) for individual queries 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 30 xerox
ENG_FRE_S ENG_FRE_S
Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
0%0% Precision averages (%) for individual queries 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 40 xerox Priority
Query Construction 49,770 Source Language 1,339 Topic Fields 664 Pooled
Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
0%5 Precision averages (%) for individual queries 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 40 ten35 m i rpe30 x E ftsoe2205 h cp i oT15 f o re10 b m u 5 N xerox
GER_FRE_J GER_FRE_J
Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
100% 90% 80% 70% 60% 0%0% Precision averages (%) for individual queries 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 xerox
Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
0%5 Precision averages (%) for individual queries 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 25 t n e irm20 e xp E the15 f so c iop10 T f o rbe 5 m u N xerox PRF_ENG_FRE
Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
100% 90% 80% 70% 60% 0%0% Precision averages (%) for individual queries 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 xerox 0.0011 DicoAdapt+PRF 0.1144
Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
20% 10% 0%5 Precision averages (%) for individual queries 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 50 xerox PRF_GER_FRE
Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
100% 90% 80% 70% 60% 40% 30% 20% 10% 0%0% Precision averages (%) for individual queries 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 40 ten35 m i rpe30 x E tfseo2250 h cp i foT15 o re10 b m u 5 N Overall statistics for 50 queries : Total number of documents over all queries Retrieved Relevant Relevant retrieved 0.0025 CacaoEngFreExpanded 0.0734
Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Precision averages (%) for individual queries Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment 20% 10% 0%5
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
R−Precision 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 40 ten35 m i rpe30 x E tfsoe2205 h cp i oT15 f o re10 b m u 5
N −1 476−AH 47 −AH 478−AH 479−AH 480−AH 481−AH 482−AH 483−AH 484−AH 485−AH 486−AH 487−AH Topi4c8Id−eAnHtifier 489−AH 490−AH 491−AH 492−AH 493−AH 494−AH 495−AH 496−AH 497−AH 498−AH 49 −AH 50 −AH −1 451−AH 452−AH 453−AH 454−AH 45 −AH 456−AH 457−AH 458−AH 459−AH 460−AH 461−AH 462−AH Topi4c63Id−eAnHtifier 464−AH 465−AH 46 −AH 467−AH 468−AH 469−AH 470−AH 471−AH 472−AH 473−AH 474−AH 475−AH Ad−HocTELBilngualFrenchTask−ComparisontoMedianR−PrecisionbyTopic(Topics476−AHto500−AH)
0%0% Precision averages (%) for individual queries 0.6667 0.0000 0.0025 0.0125 0.0600 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 40 Overall statistics for 50 queries : Total number of documents over all queries Retrieved Relevant Relevant retrieved Priority
Query Construction 50,000 Source Language 1,339 Topic Fields
451 Pooled 0.0069 CacaoEngFrePlain 0.0763
Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
−1 451−AH 452−AH 453−AH 454−AH 45 −AH 456−AH 457−AH 458−AH 459−AH 460−AH 461−AH 462−AH Topi4c63Id−eAnHtifier 464−AH 465−AH 46 −AH 467−AH 468−AH 469−AH 470−AH 471−AH 472−AH 473−AH 474−AH 475−AH Precision averages (%) for individual queries Ad−HocTELBilngualFrenchTask−ComparisontoMedianR−PrecisionbyTopic(Topics451−AHto475−AH)
Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment 20% 10% 0%5 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
R−Precision 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 30 −1 476−AH 47 −AH 478−AH 479−AH 480−AH 481−AH 482−AH 483−AH 484−AH 485−AH 486−AH 487−AH Topi4c8Id−eAnHtifier 489−AH 490−AH 491−AH 492−AH 493−AH 494−AH 495−AH 496−AH 497−AH 498−AH 49 −AH 50 −AH −1 451−AH 452−AH 453−AH 454−AH 45 −AH 456−AH 457−AH 458−AH 459−AH 460−AH 461−AH 462−AH Topi4c63Id−eAnHtifier 464−AH 465−AH 46 −AH 467−AH 468−AH 469−AH 470−AH 471−AH 472−AH 473−AH 474−AH 475−AH Ad−HocTELBilngualFrenchTask−ComparisontoMedianR−PrecisionbyTopic(Topics476−AHto500−AH)
0%0% Precision averages (%) for individual queries 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 50 t n e irm40 e xp E the30 f so c iop20 T f o rbe10 m u N Overall statistics for 50 queries : Total number of documents over all queries Retrieved Relevant Relevant retrieved Priority
Query Construction 34,074 Source Language 1,339 Topic Fields
82 Pooled 0.0001 CacaoGerFreExpanded 0.0289
Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
−1 451−AH 452−AH 453−AH 454−AH 45 −AH 456−AH 457−AH 458−AH 459−AH 460−AH 461−AH 462−AH Topi4c63Id−eAnHtifier 464−AH 465−AH 46 −AH 467−AH 468−AH 469−AH 470−AH 471−AH 472−AH 473−AH 474−AH 475−AH −1 476−AH 47 −AH 478−AH 479−AH 480−AH 481−AH 482−AH 483−AH 484−AH 485−AH 486−AH 487−AH Topi4c8Id−eAnHtifier 489−AH 490−AH 491−AH 492−AH 493−AH 494−AH 495−AH 496−AH 497−AH 498−AH 49 −AH 50 −AH Precision averages (%) for individual queries Ad−HocTELBilngualFrenchTask−ComparisontoMedianR−PrecisionbyTopic(Topics451−AHto475−AH)
Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment 20% 10% 0%5 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
R−Precision 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 50 t n e irm40 e xp E the30 f so c iop20 T f o r be10 m u
N −1 476−AH 47 −AH 478−AH 479−AH 480−AH 481−AH 482−AH 483−AH 484−AH 485−AH 486−AH 487−AH Topi4c8Id−eAnHtifier 489−AH 490−AH 491−AH 492−AH 493−AH 494−AH 495−AH 496−AH 497−AH 498−AH 49 −AH 50 −AH −1 451−AH 452−AH 453−AH 454−AH 45 −AH 456−AH 457−AH 458−AH 459−AH 460−AH 461−AH 462−AH Topi4c63Id−eAnHtifier 464−AH 465−AH 46 −AH 467−AH 468−AH 469−AH 470−AH 471−AH 472−AH 473−AH 474−AH 475−AH Ad−HocTELBilngualFrenchTask−ComparisontoMedianR−PrecisionbyTopic(Topics476−AHto500−AH)
100% 90% 80% 70% 60% 40% 30% 20% 10% 0%0% Precision averages (%) for individual queries 0.00 2.24 0.00 4.81 1.15 0.00 0.50 0.00 0.00 0.00 0.00 0.02 0.00 0.02 21.06 0.00 0.00 0.00 0.00 0.00 0.06 0.00 0.00 0.00 46.04 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 0.2 ce reen 0 if D−0.2 −0.4 −0.6 −0.8 50 t n e irm40 e xp E the30 f so c iop20 T f o rbe10 m u N Overall statistics for 50 queries : Total number of documents over all queries Retrieved Relevant Relevant retrieved Priority
Query Construction 32,036 Source Language 1,339 Topic Fields
86 Pooled 0.0001 CacaoGerFrePlain 0.0307
Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Ad−Hoc TEL Bilingual French Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
−1 451−AH 452−AH 453−AH 454−AH 45 −AH 456−AH 457−AH 458−AH 459−AH 460−AH 461−AH 462−AH Topi4c63Id−eAnHtifier 464−AH 465−AH 46 −AH 467−AH 468−AH 469−AH 470−AH 471−AH 472−AH 473−AH 474−AH 475−AH −1 476−AH 47 −AH 478−AH 479−AH 480−AH 481−AH 482−AH 483−AH 484−AH 485−AH 486−AH 487−AH Topi4c8Id−eAnHtifier 489−AH 490−AH 491−AH 492−AH 493−AH 494−AH 495−AH 496−AH 497−AH 498−AH 49 −AH 50 −AH
Ad−Hoc TEL Bilingual French Task − Box plot of the Topics of the Experiment 100% 90% 80% 70% 60% Interquartile range Mean Mean With No Outliers Std With No Outliers 451-AH 452-AH 453-AH 454-AH 455-AH 456-AH 457-AH 458-AH 459-AH 460-AH 461-AH 462-AH 463-AH 464-AH 465-AH 466-AH 467-AH 468-AH 469-AH 470-AH 471-AH 472-AH 473-AH 474-AH 475-AH