Introduction 3
Results for CLEF 2009 Ad-hoc Persian@CLEF Track 3. Individual Experiment Results and Graphs This section provides the individual results for each official experiment. For each experiment the following tables and graphs are shown: - Overall statistics and information - Interpolated recall vs precision averages plot - Average precision statistics and box plot - Average precision comparison to median plot - Document cutoff levels vs precision at DCL plot - R-Precision statistics and box plot - R-Precision comparison to median plot Topics are identified with DOIs, as well as the experiments. The prefix for the DOI of a topic is 10.2452. The following example shows how to build the DOI for a topic given its number: for topic 200-AH, the corresponding DOI is 10.2452/200-AH List of Submitted Experiments 7 jhu-apl jhu-apl jhu-apl jhu-apl jhu-apl opentext opentext opentext opentext AH-PERSIAN-MONO-FA-CLEF2009
TDN Track Overview Results and Graphs 10 0% 0% 1 0.8 0.6 0.4 0.2 −0.4 −0.6 −0.8 −1 1 0.8 0.6 0.4 0.2 −0.4 −0.6 −0.8 −1 601−AH602−AH603−AH604−AH605−AH606−AH607−AH608−AH609−AH610−AH611−AH612−AH613−AH614−AH615−AH616−AH617−AH618−AH619−AH620−AH621−AH622−AH623−AH624−AH625−AH 626−AH627−AH628−AH629−AH630−AH631−AH632−AH633−AH634−AH635−AH636−AH637−AH638−AH639−AH640−AH641−AH642−AH643−AH644−AH645−AH646−AH647−AH648−AH649−AH650−AH
Ad−Hoc TEL Monolingual Persian Task − Box Plot of the Topics
unine [Experiment UNINEPE1; MAP 43.80%; Pooled] opentext [Experiment OTFA09TDE; MAP 39.53%; Pooled]
opentext [Experiment OTFA09T; MAP 38.23%; Pooled] qazviniau [Experiment IAUPERFA3; MAP 37.62%; Pooled]
opentext [Experiment OTFA09TD; MAP 37.43%; Pooled] qazviniau [Experiment IAUPERFA1; MAP 34.59%; Pooled] opentext [Experiment OTFA09TDZ; MAP 27.12%; Pooled] qazviniau [Experiment IAUPERFA4; MAP 18.46%; Pooled] qazviniau [Experiment IAUPERFA2; MAP 17.02%; Pooled] 0% 10% 20% 30% 70% 80% 90%
100% 40% 50% 60%
Average Precision
Track Overview Results and Graphs 14 0%
5 1 0.8 0.6 0.4 0.2 −0.4 −0.6 −0.8 −1 1 0.8 0.6 0.4 0.2 −0.4 −0.6 −0.8 −1 601−AH602−AH603−AH604−AH605−AH606−AH607−AH608−AH609−AH610−AH611−AH612−AH613−AH614−AH615−AH616−AH617−AH618−AH619−AH620−AH621−AH622−AH623−AH624−AH625−AH
Ad−Hoc TEL Monolingual Persian Task Top 5 Participants − Comparison to Median R−Precision by Topic (Topics 626−AH to 650−AH) 626−AH627−AH628−AH629−AH630−AH631−AH632−AH633−AH634−AH635−AH636−AH637−AH638−AH639−AH640−AH641−AH642−AH643−AH644−AH645−AH646−AH647−AH648−AH649−AH650−AH jhu−apl [Experiment JHUFASK41R400TD; R−Prec 48.62%; Pooled] unine [Experiment UNINEPE4; R−Prec 49.59%; Pooled] opentext [Experiment OTFA09TDE; R−Prec 42.43%; Pooled] qazviniau [Experiment IAUPERFA3; R−Prec 40.33%; Pooled] jhu−apl [Experiment JHUFASK41R400TD; R−Prec 48.62%; Pooled] unine [Experiment UNINEPE4; R−Prec 49.59%; Pooled] opentext [Experiment OTFA09TDE; R−Prec 42.43%; Pooled] qazviniau [Experiment IAUPERFA3; R−Prec 40.33%; Pooled]
Ad−Hoc TEL Monolingual Persian Task − Box Plot of the Topics unine [Experiment UNINEPE2; R−Prec 46.06%; Pooled] unine [Experiment UNINEPE1; R−Prec 44.95%; Pooled] opentext [Experiment OTFA09TDE; R−Prec 42.43%; Pooled] opentext [Experiment OTFA09T; R−Prec 41.31%; Pooled] opentext [Experiment OTFA09TD; R−Prec 40.82%; Pooled] qazviniau [Experiment IAUPERFA3; R−Prec 40.33%; Pooled] qazviniau [Experiment IAUPERFA1; R−Prec 37.50%; Pooled] opentext [Experiment OTFA09TDZ; R−Prec 26.24%; Pooled] qazviniau [Experiment IAUPERFA4; R−Prec 22.04%; Pooled] qazviniau [Experiment IAUPERFA2; R−Prec 19.58%; Pooled] 0% 10% 20% 30% 40% 60% 70% 80% 90%
100% 50%
R−Precision
Track Overview Results and Graphs 18 Topic 601-AH 602-AH 603-AH 604-AH 605-AH 606-AH 607-AH 608-AH 609-AH 610-AH 611-AH 612-AH 613-AH 614-AH 615-AH 616-AH 617-AH 618-AH 619-AH 620-AH 621-AH 622-AH 623-AH 624-AH 625-AH 626-AH 627-AH 628-AH 629-AH 630-AH 631-AH 632-AH 633-AH 634-AH 635-AH 636-AH 637-AH 638-AH 639-AH 640-AH 641-AH 642-AH 643-AH 644-AH 645-AH 646-AH 647-AH 648-AH 649-AH 650-AH
SNO 5.37 0.41 0.63 3.15 0.00 0.62 0.00 5.94 4.70 0.00 0.74 0.00 3.45 2.72 0.98 0.90 1.49 38.49 29.34 0.00 18.69 1.28 1.46 0.58 3.06 0.00 2.34 17.89 16.17 31.94 7.18 0.00 0.43 1.59 8.23 1.76 0.51 12.61 0.41 0.36 1.79 7.92 0.46 1.76 8.79 7.75 0.73 15.26 47.09 0.49 Average precision (non-interpolated) for all relevant documents (averaged over queries) 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 100% 90% 80% 70% 60%
Ad−Hoc TEL Monolingual Persian Task − Distribution of the Topics of the Experiment 6 t iren5 m e xp E4 e h tfscop3 i o f 2 T o r e ub1 m N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Average Precision 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 100% 90% 80% 70% 60% 0%5
Ad−Hoc TEL Monolingual Persian Task − Box plot of the Topics of the Experiment
Ad−Hoc TEL Monolingual Persian Task − Distribution of the Topics of the Experiment 8 tn7 e m ire6 xp E ftcspoe45 h i o3 T f o r2 e b m u1 N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
R−Precision 0%0%
Ad−Hoc TEL Monolingual Persian Task − Box plot of the Topics of the Experiment 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 25 t n e irm20 e xp E the15 f so c iop10 T f o rbe 5 m u N Precision averages (%) for individual queries
Ad−Hoc TEL Monolingual Persian Task − Distribution of the Topics of the Experiment 100% 90% 80% 70% 60% 0%5 Precision averages (%) for individual queries 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 20 t n e m i rpe15 x E e h ftso10 cp i o T f o re 5 b m u N
Ad−Hoc TEL Monolingual Persian Task − Box plot of the Topics of the Experiment
Ad−Hoc TEL Monolingual Persian Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% R−Precision 100% 90% 80% 70% 60% 0%0% Ad−Hoc TEL Monolingual Persian Task − Box plot of the Topics of the Experiment 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 10 t n e irm 8 e xp E the 6 f so c iop 4 T f o rbe 2 m u N
Ad−Hoc TEL Monolingual Persian Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Average Precision Precision averages (%) for individual queries 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 100% 90% 80% 70% 60% 0%5
Ad−Hoc TEL Monolingual Persian Task − Box plot of the Topics of the Experiment
Ad−Hoc TEL Monolingual Persian Task − Distribution of the Topics of the Experiment 6 t iren5 m e xp E4 e h ftscop3 i o f 2 T o r e ub1 m N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
R−Precision 0%0% Ad−Hoc TEL Monolingual Persian Task − Box plot of the Topics of the Experiment 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 20 t n e m i rpe15 x E e h ftso10 cp i o T f o re 5 b m u N
Ad−Hoc TEL Monolingual Persian Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Average Precision 100% 90% 80% 70% 60% 0%5 Precision averages (%) for individual queries 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 15 t n e m ir e p xE10 e h ft so cp i o fT 5 o r e b m u N
Ad−Hoc TEL Monolingual Persian Task − Box plot of the Topics of the Experiment
Ad−Hoc TEL Monolingual Persian Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% R−Precision unine Priority
Query Construction 50,000 Source Language 4,464 Topic Fields 4,041 Pooled 2 AUTOMATIC Persian title true 0.3686 Fusion of : 1.no stemming, DFRPL2;2.Light Stemming, 0.4555 okapi;3. 5-gram, Language model 0%0% Ad−Hoc TEL Monolingual Persian Task − Box plot of the Topics of the Experiment 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 Precision averages (%) for individual queries
Ad−Hoc TEL Monolingual Persian Task − Distribution of the Topics of the Experiment 8 tn7 e m ire6 xp E tfscoep45 h i o3 T f o r2 e b m u1 N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Average Precision
Precision averages (%) for individual queries 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 unine 100% 90% 80% 70% 60% 0%5
Ad−Hoc TEL Monolingual Persian Task − Box plot of the Topics of the Experiment
Ad−Hoc TEL Monolingual Persian Task − Distribution of the Topics of the Experiment 8 tn7 e m ire6 xp E ftcspoe45 h i o3 T f o r2 e b m u1 N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
R−Precision unine Priority
Query Construction 50,000 Source Language 4,464 Topic Fields 4,168 Pooled 0%0% Ad−Hoc TEL Monolingual Persian Task − Box plot of the Topics of the Experiment 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 Precision averages (%) for individual queries
Ad−Hoc TEL Monolingual Persian Task − Distribution of the Topics of the Experiment 6 t irne5 m e xp E4 e h ftscop3 i o f 2 T o r e bu1 m N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Average Precision
unine Fusion of four different runs:5-grams with DFRIneC2,light stemming with Okapi and Rocchio query expansion, nostemming with DFR-PL2 and nostemming with DFR-PL2 and query expansion 0%5 Ad−Hoc TEL Monolingual Persian Task − Box plot of the Topics of the Experiment 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 15 t n e m ir e p xE10 e h tf so cp i o fT 5 o r e b m u N
Ad−Hoc TEL Monolingual Persian Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% R−Precision unine Priority
Query Construction 50,000 Source Language 4,464 Topic Fields 4,190 Pooled 0%0% Ad−Hoc TEL Monolingual Persian Task − Box plot of the Topics of the Experiment 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 Precision averages (%) for individual queries
Ad−Hoc TEL Monolingual Persian Task − Distribution of the Topics of the Experiment 8 tn7 e m ire6 xp E ftscope45 h i o3 T f o r2 e b m u1 N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Average Precision
unine Fusion of three runs: nostemming with Okapi and Rocchio query expansion, light stemming with Okapi and Rocchio query expansion, perstem withDFR-PB2 and IDF query expansion 20% 10% 0%5
Ad−Hoc TEL Monolingual Persian Task − Box plot of the Topics of the Experiment 0.9030 0.1000 0.3448 0.4531 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
R−Precision
Ad−Hoc TEL Monolingual Persian Task − Distribution of the Topics of the Experiment 6 t irne5 m e xp E4 e h ftscop3 i o f 2 T o r e bu1 m N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
R−Precision unine Overall statistics for 50 queries : Total number of documents over all queries Retrieved Relevant Relevant retrieved Priority
Query Construction 50,000 Source Language 4,464 Topic Fields 4,215 Pooled 4 AUTOMATIC Persian title, description, narrative true 0%0% Ad−Hoc TEL Monolingual Persian Task − Box plot of the Topics of the Experiment Precision averages (%) for individual queries 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Average Precision
Ad−Hoc TEL Monolingual Persian Task − Distribution of the Topics of the Experiment 8 tn7 e m ire6 xp E ftcspoe45 h i o3 T f o r2 e b m u1 N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Average Precision unine Fusin of three runs: light stemming with Okapi and Rocchio query expansion, perstem with DFR-PL2 and Rocchio query expansion and no stemming with language model and Rochio query expansion 20% 10% 0%5
Ad−Hoc TEL Monolingual Persian Task − Box plot of the Topics of the Experiment 0.9179 0.1750 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
R−Precision
Ad−Hoc TEL Monolingual Persian Task − Distribution of the Topics of the Experiment 8 tn7 e m ire6 xp E ftcsepo54 h i o3 T f o r2 e b m u1 N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
R−Precision Overall statistics for 50 queries : Total number of documents over all queries Retrieved Relevant Relevant retrieved Priority
Query Construction 49,000 Source Language 4,464 Topic Fields 650 Pooled 1 AUTOMATIC English title, description true 0.0007 (Title+Desc) , Wikipedia Concepts (WM-Wikifier), 0.0336 Simple Structure Query, PRF(5,10) 0%0% Ad−Hoc TEL Bilingual Persian Task − Box plot of the Topics of the Experiment 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8
Ad−Hoc TEL Bilingual Persian Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Average Precision Docs Cutoff Levels Precision at DCL (%) 5 docs 4.00 10 docs 4.00 15 docs 3.87 20 docs 4.60 30 docs 4.13 100 docs 3.72 200 docs 2.99 500 docs 1.97 1000 docs 1.30 R-Precision (precision after R document retrieved, where R = Relevant retrieved) 4.24 100% 90% 80% 70% 60% 40% 30% 20% 10% 0%5 Ad−Hoc TEL Bilingual Persian Task − Box plot of the Topics of the Experiment R_PRECISION 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 40
Ad−Hoc TEL Bilingual Persian Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% R−Precision Overall statistics for 50 queries : Total number of documents over all queries Retrieved Relevant Relevant retrieved Priority
Query Construction 49,000 Source Language 4,464 Topic Fields 659 Pooled 2 AUTOMATIC English title, description true 0.0006 (Title+Desc) , Wikipedia Concepts (WM-Wikifier), 0.0344 Simple Structure Query, PRF(5,20) 100% 90% 80% 70% 60% 40% 30% 20% 10% 0%0% Ad−Hoc TEL Bilingual Persian Task − Box plot of the Topics of the Experiment 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 50 t n e irm40 e xp E the30 f so c iop20 T f o rbe10 m u N 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Average Precision
Ad−Hoc TEL Bilingual Persian Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Average Precision Docs Cutoff Levels Precision at DCL (%) 5 docs 4.00 10 docs 3.80 15 docs 3.73 20 docs 4.40 30 docs 4.33 100 docs 3.82 200 docs 3.11 500 docs 2.04 1000 docs 1.32 R-Precision (precision after R document retrieved, where R = Relevant retrieved) 4.19 100% 90% 80% 70% 60% 40% 30% 20% 10% 0%5 Ad−Hoc TEL Bilingual Persian Task − Box plot of the Topics of the Experiment R_PRECISION 0.00 3.15 26.51 0.43 0.00 2.11 8.33 0.00 0.00 25.58 0.86 1.00 1.68 3.03 0.00 0.00 10.87 0.00 5.15 0.00 3.70 0.00 1.22 0.00 0.00 Precision averages (%) for individual queries 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 40 ten35 m i rpe30 x E ftsoe2205 h cp i foT15 o re10 b m u 5 N 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
R−Precision
Ad−Hoc TEL Bilingual Persian Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% R−Precision Overall statistics for 50 queries : Total number of documents over all queries Retrieved Relevant Relevant retrieved Priority
Query Construction 32,246 Source Language 4,464 Topic Fields 773 Pooled 3 AUTOMATIC English title, description true 0.0010 (Title+Desc) , Wikipedia Concepts (WM-Wikifier), 0.0422 Simple Structure Query 100% 90% 80% 70% 60% 40% 30% 20% 10% 0%0% Ad−Hoc TEL Bilingual Persian Task − Box plot of the Topics of the Experiment 0.03 2.79 25.21 0.06 0.00 1.06 12.38 0.00 0.00 12.69 0.29 0.59 0.00 1.52 0.00 0.04 11.15 0.00 1.37 1.74 0.91 0.00 0.44 0.00 0.00 Precision averages (%) for individual queries IAUPEREN3 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 ce 0.2 reen 0 ifD−0.2 −0.4 −0.6 −0.8 50 t n e irm40 e xp E the30 f so c iop20 T f o rbe10 m u N 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Average Precision
Ad−Hoc TEL Bilingual Persian Task − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Average Precision 100% 90% 80% 70% 60% Ad−Hoc TEL Bilingual Persian Task − Box plot of the Topics of the Experiment Interquartile range Mean Mean With No Outliers Std With No Outliers 601-AH 602-AH 603-AH 604-AH 605-AH 606-AH 607-AH 608-AH 609-AH 610-AH 611-AH 612-AH 613-AH 614-AH 615-AH 616-AH 617-AH 618-AH 619-AH 620-AH 621-AH 622-AH 623-AH 624-AH 625-AH
IAUPEREN3 40 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
R−Precision
Ad−Hoc TEL Bilingual Persian Task − Distribution of the Topics of the Experiment