Introduction 3
Results for CLEF 2006 GeoCLEF Tracks List of Submitted Experiments 7 berkeley berkeley daedalus daedalus daedalus daedalus daedalus hagen hagen hagen hagen hagen hildesheim hildesheim hildesheim hildesheim alicante alicante alicante alicante berkeley berkeley berkeley berkeley daedalus daedalus daedalus daedalus daedalus hildesheim hildesheim hildesheim hildesheim hildesheim imp-coll imp-coll jaen jaen jaen jaen jaen ms-china ms-china ms-china ms-china ms-china nicta nicta nicta nicta nicta rfia-upv rfia-upv rfia-upv rfia-upv sanmarcos
TDN TD TD TDN TDN TD TDN TD TD TDN TDN TDN TDN TD TD TDN TD TDN TDN TDN TDN TDN TDN TD TDN TD TD TDN TDN TDN TDN TD TD TD TDN TD TD TDN TD TD TD TD T TD T TDN TDN TDN TD TD TD TD TDN TD TDN TD sanmarcos sanmarcos sanmarcos sanmarcos talp talp talp talp talp u.buffalo u.buffalo u.buffalo u.buffalo u.groningen u.groningen u.groningen u.groningen u.groningen u.twente u.twente u.twente u.twente u.twente unsw unsw unsw unsw unsw xldb xldb xldb xldb xldb alicante alicante alicante berkeley berkeley daedalus daedalus daedalus daedalus daedalus sanmarcos sanmarcos sanmarcos sanmarcos sanmarcos berkeley berkeley berkeley berkeley sanmarcos sanmarcos sanmarcos sanmarcos
AUTOMATIC AUTOMATIC AUTOMATIC MANUAL AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC MANUAL MANUAL AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC MANUAL MANUAL AUTOMATIC MANUAL MANUAL AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC MANUAL AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC MANUAL AUTOMATIC AUTOMATIC MANUAL MANUAL MANUAL MANUAL MANUAL AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC MANUAL AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC no yes no yes no yes yes no no no yes no yes yes yes no no no yes yes no yes no yes yes no no no no no no yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes
MANUAL AUTOMATIC MANUAL AUTOMATIC AUTOMATIC MANUAL AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC MANUAL AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC MANUAL AUTOMATIC yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes no no no yes no no no yes yes yes yes yes yes yes yes yes yes yes yes Track Overview Results and Graphs 13 v 40% A 30% 20% 10% 0%
0% 1 0.8 0.6 0.4 0.2 −0.2 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier FUHddGYYYTD [MAP 22.29%; Pooled]
FUHddGYYYTDN [MAP 21.41%; Pooled] FUHddGYYYMTDN [MAP 19.99%; Pooled]
BKGeoD2 [MAP 18.22%; Pooled] FUHddGNNNTD [MAP 16.94%; Pooled]
HIGeodederun4n [MAP 16.01%; Pooled] tsn HIGeodederun4 [MAP 15.58%; Pooled] e m i r e xpE FUHddGNNNTDN [MAP 12.23%; Pooled]
HIGeodederun6 [MAP 12.14%; Pooled] HIGeodederun6n [MAP 11.34%; Pooled]
GCdeNtLg [MAP 10.01%; Pooled]
GCdeNA [MAP 9.28%; Pooled] GCdeAtLg [MAP 7.36%; Pooled]
GCdeAA [MAP 7.15%; Pooled] GCdeAO [MAP 5.48%; Pooled] 0% 10% 20% 30% 40% 50% 60% 70%
Mean Average Precision 80%
90% 100%
GeoCLEF Monolingual German track − Tukey T test with "top group" highlighted BKGeoD1
FUHddGNNNTD HIGeodederun4n 40% 30% 20% 10% 0%
5 1 0.8 0.6 0.4 0.2 −0.2 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier
FUHddGYYYTD [R−Prec 21.53%; Pooled]
FUHddGYYYTDN [R−Prec 20.56%; Pooled] FUHddGYYYMTDN [R−Prec 20.39%; Pooled]
BKGeoD1 [R−Prec 19.99%; Pooled]
BKGeoD2 [R−Prec 18.57%; Pooled] HIGeodederun4 [R−Prec 18.15%; Pooled]
FUHddGNNNTD [R−Prec 18.00%; Pooled]
HIGeodederun6 [R−Prec 13.45%; Pooled] FUHddGNNNTDN [R−Prec 13.40%; Pooled]
GCdeNtLg [R−Prec 11.53%; Pooled]
GCdeNA [R−Prec 10.19%; Pooled]
GCdeAA [R−Prec 8.93%; Pooled] GCdeAtLg [R−Prec 8.78%; Pooled]
GCdeAO [R−Prec 6.62%; Pooled] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
R−Precision
GeoCLEF Monolingual German track − Tukey T test with "top group" highlighted FUHddGYYYTD
BKGeoD2 FUHddGNNNTDN HIGeodederun6n HIGeodederun6
GCdeNtLg
GCdeNA GCdeAtLg
GCdeAA GCdeAO 0.05 0.0806 Spanish-English using all topic fields and other 0.2164 sources
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 8 tn7 e m ire6 xp E itfcsoep45 h fo3 T o r2 e b m u1 N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 8 tn7 e m ire6 xp E iftcsope45 h fo3 T o r2 e b m u1 N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision Precision averages (%) for individual queries Topic 026 6.35 Topic 027 1.18 Topic 028 11.57 Topic 029 21.08 Topic 030 100.00 Topic 031 13.41 Topic 032 91.32 Topic 033 0.46 Topic 034 42.42 Topic 035 2.10 Topic 036 0.00 Topic 037 1.60 Topic 038 0.56
Topic 039 Topic 040 Topic 041 Topic 042 Topic 043 Topic 044 Topic 045 Topic 046 Topic 047 Topic 048 Topic 049 Topic 050 16.01 26.20 0.23 0.82 1.67 22.48 1.67 69.30 3.44 70.96 33.33 23.43 1 0.8 0.6 0.4 0.2
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision 0%5 Precision averages (%) for individual queries Topic 026 11.11 Topic 027 5.26 Topic 028 10.53 Topic 029 22.22 Topic 030 100.00 Topic 031 13.56 Topic 032 80.65 Topic 033 0.00 Topic 034 33.33 Topic 035 0.00 Topic 036 0.00 Topic 037 6.25 Topic 038 0.00
Topic 039 Topic 040 Topic 041 Topic 042 Topic 043 Topic 044 Topic 045 Topic 046 Topic 047 Topic 048 Topic 049 Topic 050 12.50 21.43 0.00 0.00 0.00 34.21 0.00 66.67 8.33 72.92 50.00 33.33 1 0.8 0.6 0.4 0.2
GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 8 tn7 e m ire6 xp E iftscope45 h fo3 T o r2 e b m u1 N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision Precision averages (%) for individual queries Topic 026 Topic 027 Topic 028 Topic 029 Topic 030 Topic 031 Topic 032 Topic 033 Topic 034 Topic 035 Topic 036 Topic 037 Topic 038 0.01 0.84 11.51 25.94 0.00 65.65 97.82 2.12 13.61 0.71 2.96 6.28 6.63
Topic 039 Topic 040 Topic 041 Topic 042 Topic 043 Topic 044 Topic 045 Topic 046 Topic 047 Topic 048 Topic 049 Topic 050 0.98 56.39 22.43 35.55 0.13 18.03 5.67 61.44 28.60 82.80 78.74 17.96 1 0.8 0.6 0.4 0.2
GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment
GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment 8 tn7 e m ire6 xp E iftcspoe45 h fo3 T o r2 e b m u1 N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision 0%5 GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment Exact R-Precision Maximum Minimum First Quartile Second Quartile Third Quartile Interquartile range Mean Standard Deviation Lower Outlier Threshold Upper Outlier Threshold Mean With No Outliers Std With No Outliers Precision averages (%) for individual queries Topic 026 Topic 027 Topic 028 Topic 029 Topic 030 Topic 031 Topic 032 Topic 033 Topic 034 Topic 035 Topic 036 Topic 037 Topic 038 0.00 0.00 5.56 30.30 0.00 67.45 90.00 3.00 13.51 0.00 6.36 0.00 0.00
Topic 039 Topic 040 Topic 041 Topic 042 Topic 043 Topic 044 Topic 045 Topic 046 Topic 047 Topic 048 Topic 049 Topic 050 2.99 61.87 34.67 39.62 0.00 28.16 16.67 60.71 33.90 73.96 70.51 22.00 1 0.8 0.6 0.4 0.2
GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision 0.0421 EN->ES using L&H query translation. title, desc 0.2882 and narrative GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment Precision averages (%) for individual queries Topic 026 Topic 027 Topic 028 Topic 029 Topic 030 Topic 031 Topic 032 Topic 033 Topic 034 Topic 035 Topic 036 Topic 037 Topic 038 0.11 38.67 10.67 47.52 0.00 62.41 97.40 1.53 9.83 2.03 0.02 0.00 0.93
Topic 039 Topic 040 Topic 041 Topic 042 Topic 043 Topic 044 Topic 045 Topic 046 Topic 047 Topic 048 Topic 049 Topic 050 53.58 67.60 16.07 47.96 3.18 2.83 1.38 63.69 0.97 73.23 74.35 10.18 1 0.8 0.6 0.4 0.2
GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision 60% 0%5 Exact R-Precision Maximum Minimum First Quartile Second Quartile Third Quartile Interquartile range Mean Standard Deviation Lower Outlier Threshold Upper Outlier Threshold Mean With No Outliers Std With No Outliers Precision averages (%) for individual queries Topic 026 Topic 027 Topic 028 Topic 029 Topic 030 Topic 031 Topic 032 Topic 033 Topic 034 Topic 035 Topic 036 Topic 037 Topic 038 0.00 41.03 13.89 54.55 0.00 60.39 90.77 4.00 13.51 5.26 0.00 0.00 0.00
Topic 039 Topic 040 Topic 041 Topic 042 Topic 043 Topic 044 Topic 045 Topic 046 Topic 047 Topic 048 Topic 049 Topic 050 50.75 68.35 18.67 50.94 4.17 7.77 0.00 53.57 1.69 63.02 69.59 4.00 1 0.8 0.6 0.4 0.2 10 t n e irm8 xep E the 6 f so c iop 4 T f o r e b 2 m u N
GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision Priority
Query Construction 25,000 Source Language 2,054 Topic Fields 729 Pooled 0.0291 Automatic English Spanish title + desc 0.1505 0%0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment Precision averages (%) for individual queries Topic 026 Topic 027 Topic 028 Topic 029 Topic 030 Topic 031 Topic 032 Topic 033 Topic 034 Topic 035 Topic 036 Topic 037 Topic 038 2.12 2.04 24.55 0.98 13.11 1.02 19.71 0.01 15.87 9.91 11.02 11.45 0.00
Topic 039 Topic 040 Topic 041 Topic 042 Topic 043 Topic 044 Topic 045 Topic 046 Topic 047 Topic 048 Topic 049 Topic 050 16.72 54.13 0.86 20.06 0.00 29.54 2.21 11.07 6.94 5.81 57.45 4.01 1 0.8 0.6 0.4 0.2 −0.2 −0.4 −0.6 −0.8 10 t n e irm8 xep E the 6 f so c iop 4 T f o r e b 2 m u N
GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision 60% 0%5 Precision averages (%) for individual queries Topic 026 Topic 027 Topic 028 Topic 029 Topic 030 Topic 031 Topic 032 Topic 033 Topic 034 Topic 035 Topic 036 Topic 037 Topic 038 0.00 2.56 27.78 3.03 17.88 5.10 20.77 0.00 18.92 15.79 22.73 17.24 0.00
Topic 039 Topic 040 Topic 041 Topic 042 Topic 043 Topic 044 Topic 045 Topic 046 Topic 047 Topic 048 Topic 049 Topic 050 23.88 61.15 6.67 28.30 0.00 37.86 0.00 14.29 11.86 7.55 66.82 12.00 1 0.8 0.6 0.4 0.2 −0.2 −0.4 −0.6 −0.8
GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment
GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment 8 tn7 e m ire6 xp E iftscope45 h fo3 T o r2 e b m u1 N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision Priority
Query Construction 25,000 Source Language 2,054 Topic Fields 659 Pooled 2 AUTOMATIC Portuguese title, description true 0.0221 Automatic Portuguese Spanish title+desc 0.1261 0%0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment Precision averages (%) for individual queries Topic 026 Topic 027 Topic 028 Topic 029 Topic 030 Topic 031 Topic 032 Topic 033 Topic 034 Topic 035 Topic 036 Topic 037 Topic 038 1.72 3.17 7.13 28.54 0.29 0.87 19.83 0.09 21.30 2.99 0.68 0.05 0.00
Topic 039 Topic 040 Topic 041 Topic 042 Topic 043 Topic 044 Topic 045 Topic 046 Topic 047 Topic 048 Topic 049 Topic 050 27.01 53.35 1.00 31.64 0.03 5.52 2.29 5.19 3.17 5.65 45.66 5.21 1 0.8 0.6 0.4 0.2 −0.2 −0.4 −0.6 −0.8 15 t n e m ir xep E10 e h ft so cp i o fT 5 o r e b m u N
GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision 60% 0%5 Precision averages (%) for individual queries Topic 026 Topic 027 Topic 028 Topic 029 Topic 030 Topic 031 Topic 032 Topic 033 Topic 034 Topic 035 Topic 036 Topic 037 Topic 038 5.56 12.82 13.89 30.30 2.65 5.10 20.77 1.00 29.73 5.26 3.64 0.00 0.00
Topic 039 Topic 040 Topic 041 Topic 042 Topic 043 Topic 044 Topic 045 Topic 046 Topic 047 Topic 048 Topic 049 Topic 050 23.88 60.43 9.33 37.74 0.00 14.56 8.33 7.14 3.39 7.55 55.76 8.00 1 0.8 0.6 0.4 0.2 −0.2 −0.4 −0.6 −0.8
GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment
GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment 8 tn7 e m ire6 xp E itfscoep45 h fo3 T o r2 e b m u1 N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision Priority
Query Construction 25,000 Source Language 2,054 Topic Fields 655 Pooled 3 AUTOMATIC Portuguese title, description, narrative true 0.0230 Automatic Portuguese Spanish title-desc-narr 0.1310 0%0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment Precision averages (%) for individual queries Topic 026 Topic 027 Topic 028 Topic 029 Topic 030 Topic 031 Topic 032 Topic 033 Topic 034 Topic 035 Topic 036 Topic 037 Topic 038 1.78 2.67 8.07 32.55 0.28 1.43 20.01 0.06 23.16 4.01 0.72 0.05 0.00
Topic 039 Topic 040 Topic 041 Topic 042 Topic 043 Topic 044 Topic 045 Topic 046 Topic 047 Topic 048 Topic 049 Topic 050 27.15 53.78 0.99 31.54 0.03 5.84 2.15 5.76 3.37 5.68 51.21 5.28 1 0.8 0.6 0.4 0.2 −0.2 −0.4 −0.6 −0.8 15 t n e m ir xpe E10 e h tf so cp i o fT 5 o r e b m u N 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision 60% 30% 20% 10% 0%5 Precision averages (%) for individual queries Topic 026 Topic 027 Topic 028 Topic 029 Topic 030 Topic 031 Topic 032 Topic 033 Topic 034 Topic 035 Topic 036 Topic 037 Topic 038 5.56 12.82 16.67 30.30 2.65 6.67 20.77 1.00 40.54 5.26 3.64 0.00 0.00
Topic 039 Topic 040 Topic 041 Topic 042 Topic 043 Topic 044 Topic 045 Topic 046 Topic 047 Topic 048 Topic 049 Topic 050 25.37 60.43 9.33 37.74 0.00 15.53 0.00 7.14 3.39 7.55 59.45 10.00 1 0.8 0.6 0.4 0.2 −0.2 −0.4 −0.6 −0.8
GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment 8 tn7 e m ire6 xp E iftsceop54 h fo3 T o r2 e b m u1 N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision Overall statistics for 25 queries : Total number of documents over all queries Retrieved Relevant Relevant retrieved Priority
Query Construction 25,000 Source Language 1,060 Topic Fields 591 Pooled 0.0098 EN->PT using L&H query translation. title and 0.1291 description used 0%0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment Precision averages (%) for individual queries Topic 026 Topic 027 Topic 028 Topic 029 Topic 030 Topic 031 Topic 032 Topic 033 Topic 034 Topic 035 Topic 036 Topic 037 Topic 038 1.22 0.03 12.27 27.03 0.81 43.74 14.19 0.00 0.14 0.55 0.00 1.29 14.35
Topic 039 Topic 040 Topic 041 Topic 042 Topic 043 Topic 044 Topic 045 Topic 046 Topic 047 Topic 048 Topic 049 Topic 050 5.45 0.01 0.01 10.65 2.46 0.00 27.39 33.97 0.13 79.32 24.37 15.70 1 0.8 0.6 0.4 0.2 −0.2 −0.4 −0.6 −0.8 15 t n e m ir xep E10 e h ft so cp i o fT 5 o r e b m u N 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision 60% 30% 20% 10% 0%5 GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment Exact R-Precision Maximum Minimum First Quartile Second Quartile Third Quartile Interquartile range Mean Standard Deviation Lower Outlier Threshold Upper Outlier Threshold Mean With No Outliers Std With No Outliers Precision averages (%) for individual queries Topic 026 Topic 027 Topic 028 Topic 029 Topic 030 Topic 031 Topic 032 Topic 033 Topic 034 Topic 035 Topic 036 Topic 037 Topic 038 0.00 0.00 21.88 33.33 0.00 39.34 26.42 0.00 0.00 0.00 0.00 0.00 12.50
Topic 039 Topic 040 Topic 041 Topic 042 Topic 043 Topic 044 Topic 045 Topic 046 Topic 047 Topic 048 Topic 049 Topic 050 8.70 0.00 0.00 20.00 4.17 0.00 36.59 37.88 0.00 73.43 27.78 27.27 1 0.8 0.6 0.4 0.2 ce reen 0 if D −0.2 −0.4 −0.6 −0.8 15 t n e m ir xep E10 e h tf so cp i o fT 5 o r e b m u N 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision Overall statistics for 25 queries : Total number of documents over all queries Retrieved Relevant Relevant retrieved Priority
Query Construction 25,000 Source Language 1,060 Topic Fields 630 Pooled 0.0114 EN->PT using L&H query translation. title, desc and 0.1422 narrative 0%0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment Precision averages (%) for individual queries Topic 026 Topic 027 Topic 028 Topic 029 Topic 030 Topic 031 Topic 032 Topic 033 Topic 034 Topic 035 Topic 036 Topic 037 Topic 038 1.27 8.97 24.05 33.34 0.69 21.61 7.76 0.00 0.15 0.14 0.00 0.07 18.63
Topic 039 Topic 040 Topic 041 Topic 042 Topic 043 Topic 044 Topic 045 Topic 046 Topic 047 Topic 048 Topic 049 Topic 050 8.98 0.01 0.00 43.84 6.26 0.00 24.53 51.65 2.96 69.74 8.80 24.03 1 0.8 0.6 0.4 0.2 ce reen 0 if D −0.2 −0.4 −0.6 −0.8 15 t n e m ir xpe E10 e h ft so cp i o fT 5 o r e b m u N 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision Docs Cutoff Levels Precision at DCL (%) 5 docs 24.80 10 docs 22.80 15 docs 22.67 20 docs 20.80 30 docs 19.73 100 docs 13.36 200 docs 9.14 500 docs 4.55 1000 docs 2.52 R-Precision (precision after R document retrieved, where R = Relevant retrieved) 16.34 100% 90% 80% 70% 60% 30% 20% 10% 0%5 GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment Exact R-Precision Maximum Minimum First Quartile Second Quartile Third Quartile Interquartile range Mean Standard Deviation Lower Outlier Threshold Upper Outlier Threshold Mean With No Outliers Std With No Outliers Precision averages (%) for individual queries Topic 026 Topic 027 Topic 028 Topic 029 Topic 030 Topic 031 Topic 032 Topic 033 Topic 034 Topic 035 Topic 036 Topic 037 Topic 038 0.00 20.59 34.38 38.46 0.00 21.31 20.75 0.00 0.00 0.00 0.00 0.00 12.50
Topic 039 Topic 040 Topic 041 Topic 042 Topic 043 Topic 044 Topic 045 Topic 046 Topic 047 Topic 048 Topic 049 Topic 050 13.04 0.00 0.00 42.86 8.33 0.00 35.37 54.55 0.00 64.34 5.56 36.36 1 0.8 0.6 0.4 0.2 ce reen 0 if D −0.2 −0.4 −0.6 −0.8 15 t n e m ir xpe E10 e h ft so cp i o fT 5 o r e b m u N 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision Overall statistics for 25 queries : Total number of documents over all queries Retrieved Relevant Relevant retrieved
Priority
Query Construction 25,000 Source Language 1,060 Topic Fields
608 Pooled 0.0511 Spanish-Portuguese 0.1174 1 MANUAL Spanish; Castilian title, description true 0%0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment Precision averages (%) for individual queries Topic 026 Topic 027 Topic 028 Topic 029 Topic 030 Topic 031 Topic 032 Topic 033 Topic 034 Topic 035 Topic 036 Topic 037 Topic 038 1.17 6.97 8.54 6.86 38.66 14.38 49.84 1.16 2.24 2.47 2.88 0.14 30.10
Topic 039 Topic 040 Topic 041 Topic 042 Topic 043 Topic 044 Topic 045 Topic 046 Topic 047 Topic 048 Topic 049 Topic 050 30.28 1.41 1.15 49.61 0.11 7.75 2.95 25.94 8.84 2.29 14.11 10.34 1 0.8 0.6 0.4 0.2 ce reen 0 if D −0.2 −0.4 −0.6 −0.8 15 t n e m ir xep E10 e h tf so cp i o fT 5 o r e b m u N 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision Docs Cutoff Levels Precision at DCL (%) 5 docs 23.20 10 docs 21.60 15 docs 18.67 20 docs 17.40 30 docs 16.00 100 docs 9.88 200 docs 6.52 500 docs 3.97 1000 docs 2.43 R-Precision (precision after R document retrieved, where R = Relevant retrieved) 14.88 100% 90% 80% 70% 60% 30% 20% 10% 0%5
GeoCLEF Bilingual Portuguese track − Retrieved documents vs Precision GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment Exact R-Precision Maximum Minimum First Quartile Second Quartile Third Quartile Interquartile range Mean Standard Deviation Lower Outlier Threshold Upper Outlier Threshold Mean With No Outliers Std With No Outliers Precision averages (%) for individual queries Topic 026 Topic 027 Topic 028 Topic 029 Topic 030 Topic 031 Topic 032 Topic 033 Topic 034 Topic 035 Topic 036 Topic 037 Topic 038 0.00 12.75 15.62 7.69 42.86 13.11 58.49 0.00 0.00 0.00 0.00 0.00 25.00
Topic 039 Topic 040 Topic 041 Topic 042 Topic 043 Topic 044 Topic 045 Topic 046 Topic 047 Topic 048 Topic 049 Topic 050 26.09 4.17 2.88 48.57 0.00 10.53 8.54 33.33 8.82 9.79 27.78 15.91 1 0.8 0.6 0.4 0.2 ce reen 0 if D −0.2 −0.4 −0.6 −0.8 10 t n e irm8 xep E the 6 f so c iop 4 T f o r e b 2 m u N 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision Overall statistics for 25 queries : Total number of documents over all queries Retrieved Relevant Relevant retrieved Priority
Query Construction 25,000 Source Language 1,060 Topic Fields 655 Pooled 0.0497 Automatic Spanish Portuguese title + desc 0.1415 100% 90% 80% 70% 30% 20% 10%
GeoCLEF Bilingual Portuguese track − Interpolated Recall vs Average Precision 0%0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment Precision averages (%) for individual queries Topic 026 Topic 027 Topic 028 Topic 029 Topic 030 Topic 031 Topic 032 Topic 033 Topic 034 Topic 035 Topic 036 Topic 037 Topic 038 7.79 0.16 13.15 23.65 1.64 16.14 75.79 0.17 3.21 2.76 1.92 1.72 16.26
Topic 039 Topic 040 Topic 041 Topic 042 Topic 043 Topic 044 Topic 045 Topic 046 Topic 047 Topic 048 Topic 049 Topic 050 8.25 2.84 1.12 25.88 0.02 7.97 23.82 9.22 5.42 51.93 39.97 13.12 1 0.8 0.6 0.4 0.2 ce reen 0 if D −0.2 −0.4 −0.6 −0.8 10 t n e irm8 xpe E the 6 f so c iop 4 T f o r e b 2 m u N 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision SMGeoESPT2 100% 90% 80% 70% 60% Precision averages (%) for individual queries Topic 026 Topic 027 Topic 028 Topic 029 Topic 030 Topic 031 Topic 032 Topic 033 Topic 034 Topic 035 Topic 036 Topic 037 Topic 038
GeoCLEF Bilingual Portuguese track − Retrieved documents vs Precision
GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment
GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment 8 tn7 e m ire6 xp E iftcspoe45 h fo3 T o r2 e b m u1 N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision