Appendix D Results of the GeoCLEF Track Prepared by: Giorgio Maria Di Nunzio and Nicola Ferro {dinunzio, ferro}@dei.unipd.it Department of Information Engineering University of Padua Italy 1 2 Introduction 3 4 Results for CLEF 2006 GeoCLEF Tracks The following pages contain the results and graphs for all the experiments that have been officially submitted to the CLEF 2006 campaign for the GeoCLEF track. This document is divided in three main parts: 1. List of submitted experiments 2. Track overview results and graphs 3. Individual experiment results and graphs 1. List of Submitted Experiments This section gives a listing of all experiments and their characteristics: Participant: the name of the participant who submitted the experiment. Country: country of the participant. Identifier: unique identifier for each experiment. Task: track/task to which the experiment belongs. Topic language: language of the topics used to create the experiment (ISO identifiers for language). Topic fields: identifies the parts of the topics used to create the experiment (T = title, D = Description, N = Narrative). Query constr.: identifies how the query has been constructed from topic fields (manual/automatic). Pool: specifies if experiment was used for relevance assessment pooling. 2. Track Overview Results and Graphs For each track/task graphs and tables are shown in order to compare the experiments. The graphs and tables contain the following information: - Mandatory experiments title + description (TD) of at most top five participants - Interpolated recall vs precision averages plot - Average precision comparison to median plot - All experiments - Average precision box plot - Average precision Tukey t-test plot - Mandatory experiments title + description (TD) of at most top five participants - Document cutoff levels (DCL) vs precision at DCL plot - R-Precision comparison to median plot - All experiments - R-Precision box plot - R-Precision Tukey t-test plot - A table with descriptive statistics of performance figures for each topic 3. Individual Experiment Results and Graphs This section provides the individual results for each official experiment. For each experiment the following tables and graphs are shown: - Overall statistics and information - Interpolated recall vs precision averages plot - Average precision statistics and box plot - Average precision comparison to median plot - Document cutoff levels vs precision at DCL plot - R-Precision statistics and box plot - R-Precision comparison to median plot 5 6 List of Submitted Experiments 7 8 Participant Country Experiment ID Task Topic Topic Query Pool Lang. Fields Construction berkeley United States BKGeoD2 GC-MONO-DE-CLEF2006 de TDN AUTOMATIC yes berkeley United States BKGeoD1 GC-MONO-DE-CLEF2006 de TD AUTOMATIC yes daedalus Spain GCdeNtLg GC-MONO-DE-CLEF2006 de TD AUTOMATIC yes daedalus Spain GCdeAA GC-MONO-DE-CLEF2006 de TDN MANUAL yes daedalus Spain GCdeAtLg GC-MONO-DE-CLEF2006 de TDN MANUAL yes daedalus Spain GCdeNA GC-MONO-DE-CLEF2006 de TD MANUAL yes daedalus Spain GCdeAO GC-MONO-DE-CLEF2006 de TDN MANUAL yes hagen Germany FUHddGYYYTD GC-MONO-DE-CLEF2006 de TD AUTOMATIC yes hagen Germany FUHddGNNNTD GC-MONO-DE-CLEF2006 de TD AUTOMATIC yes hagen Germany FUHddGNNNTDN GC-MONO-DE-CLEF2006 de TDN AUTOMATIC yes hagen Germany FUHddGYYYTDN GC-MONO-DE-CLEF2006 de TDN AUTOMATIC yes hagen Germany FUHddGYYYMTDN GC-MONO-DE-CLEF2006 de TDN AUTOMATIC yes hildesheim Germany HIGeodederun4n GC-MONO-DE-CLEF2006 de TDN AUTOMATIC yes hildesheim Germany HIGeodederun4 GC-MONO-DE-CLEF2006 de TD AUTOMATIC yes hildesheim Germany HIGeodederun6 GC-MONO-DE-CLEF2006 de TD AUTOMATIC yes hildesheim Germany HIGeodederun6n GC-MONO-DE-CLEF2006 de TDN AUTOMATIC yes alicante Spain enTD GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes alicante Spain enTDN GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no alicante Spain enTDNGeoNames GC-MONO-EN-CLEF2006 en TDN MANUAL yes alicante Spain UAUJAUPVenenExp1 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no berkeley United States BKGeoE4 GC-MONO-EN-CLEF2006 en TDN MANUAL yes berkeley United States BKGeoE2 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no berkeley United States BKGeoE3 GC-MONO-EN-CLEF2006 en TDN MANUAL no berkeley United States BKGeoE1 GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes daedalus Spain GCenAtLg GC-MONO-EN-CLEF2006 en TDN MANUAL no daedalus Spain GCenNtLg GC-MONO-EN-CLEF2006 en TD MANUAL no daedalus Spain GCenNA GC-MONO-EN-CLEF2006 en TD MANUAL yes daedalus Spain GCenAA GC-MONO-EN-CLEF2006 en TDN MANUAL yes daedalus Spain GCenAO GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no hildesheim Germany HIGeoenenrun1n GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no hildesheim Germany HIGeoenenrun2n GC-MONO-EN-CLEF2006 en TDN AUTOMATIC yes hildesheim Germany HIGeoenenrun3 GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes hildesheim Germany HIGeoenenrun1 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no hildesheim Germany HIGeoenenrun2 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no imp-coll United ICgeoMLtdn GC-MONO-EN-CLEF2006 en TDN MANUAL yes Kingdom imp-coll United ICgeoMLtd GC-MONO-EN-CLEF2006 en TD MANUAL yes Kingdom jaen Spain sinaiEnEnExp3 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no jaen Spain sinaiEnEnExp1 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC yes jaen Spain sinaiEnEnExp2 GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes jaen Spain sinaiEnEnExp4 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no jaen Spain sinaiEnEnExp5 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no ms-china China msramanual GC-MONO-EN-CLEF2006 en TD MANUAL yes ms-china China msrawhitelist GC-MONO-EN-CLEF2006 en T AUTOMATIC yes ms-china China msraexpansion GC-MONO-EN-CLEF2006 en TD AUTOMATIC no ms-china China msralocal GC-MONO-EN-CLEF2006 en T AUTOMATIC no ms-china China msratext GC-MONO-EN-CLEF2006 en TDN AUTOMATIC yes nicta Australia MuTdnManQexpGeo GC-MONO-EN-CLEF2006 en TDN MANUAL no nicta Australia MuTdnTxt GC-MONO-EN-CLEF2006 en TDN AUTOMATIC yes nicta Australia MuTdQexpPrb GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes nicta Australia MuTdRedn GC-MONO-EN-CLEF2006 en TD AUTOMATIC no nicta Australia MuTdTxt GC-MONO-EN-CLEF2006 en TD AUTOMATIC no rfia-upv Spain rfiaUPV01 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no rfia-upv Spain rfiaUPV02 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no rfia-upv Spain rfiaUPV03 GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes rfia-upv Spain rfiaUPV04 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC yes sanmarcos United States SMGeoEN4 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no 9 Participant Country Experiment ID Task Topic Topic Query Pool Lang. Fields Construction sanmarcos United States SMGeoEN5 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no sanmarcos United States SMGeoEN1 GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes sanmarcos United States SMGeoEN3 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no sanmarcos United States SMGeoEN5 GC-MONO-EN-CLEF2006 en TDN MANUAL yes talp Spain TALPGeoIRTDN2 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no talp Spain TALPGeoIRTD1 GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes talp Spain TALPGeoIRTDN1 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC yes talp Spain TALPGeoIRTD2 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no talp Spain TALPGeoIRTDN3 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no u.buffalo United States UBGTDrf1 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no u.buffalo United States UBGTDrf2 GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes u.buffalo United States UBManual2 GC-MONO-EN-CLEF2006 en TDN MANUAL no u.buffalo United States UBGManual1 GC-MONO-EN-CLEF2006 en TDN MANUAL yes u.groningen Netherlands CLCGGeoEE1 GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes u.groningen Netherlands CLCGGeoEE2 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC yes u.groningen Netherlands CLCGGeoEE5 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no u.groningen Netherlands CLCGGeoEE10 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no u.groningen Netherlands CLCGGeoEE11 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no u.twente Netherlands utGeoTIB GC-MONO-EN-CLEF2006 en T MANUAL yes u.twente Netherlands utGeoTdIB GC-MONO-EN-CLEF2006 en TD MANUAL yes u.twente Netherlands utGeoTIBm GC-MONO-EN-CLEF2006 en TD AUTOMATIC no u.twente Netherlands utGeoTdnIB GC-MONO-EN-CLEF2006 en TDN MANUAL yes u.twente Netherlands utGeoTdnIBm GC-MONO-EN-CLEF2006 en TDN MANUAL no unsw Australia unswTitleBaseline GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes unsw Australia unswNarrBaseline GC-MONO-EN-CLEF2006 en TDN AUTOMATIC yes unsw Australia unswNarrMap GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no unsw Australia unswTitleF46 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no unsw Australia unswNarrF41 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no xldb Portugal XLDBGeoENAut02 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no xldb Portugal XLDBGeoENAut05 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no xldb Portugal XLDBGeoManualEN GC-MONO-EN-CLEF2006 en TD MANUAL no xldb Portugal XLDBGeoENAut03_2 GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes xldb Portugal XLDBGeoENAut03 GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes alicante Spain esTD GC-MONO-ES-CLEF2006 es TD AUTOMATIC yes alicante Spain esTDN GC-MONO-ES-CLEF2006 es TD AUTOMATIC yes alicante Spain esTDNGeoNames GC-MONO-ES-CLEF2006 es TDN MANUAL yes berkeley United States BKGeoS1 GC-MONO-ES-CLEF2006 es TD AUTOMATIC yes berkeley United States BKGeoS2 GC-MONO-ES-CLEF2006 es TDN AUTOMATIC yes daedalus Spain GCesNA GC-MONO-ES-CLEF2006 es TD MANUAL yes daedalus Spain GCesAtLg GC-MONO-ES-CLEF2006 es TDN MANUAL yes daedalus Spain GCesAO GC-MONO-ES-CLEF2006 es TDN MANUAL yes daedalus Spain GCesAA GC-MONO-ES-CLEF2006 es TDN MANUAL yes daedalus Spain GCesNtLg GC-MONO-ES-CLEF2006 es TD MANUAL yes sanmarcos United States SMGeoES4 GC-MONO-ES-CLEF2006 es TD AUTOMATIC yes sanmarcos United States SMGeoES5 GC-MONO-ES-CLEF2006 es TDN AUTOMATIC yes sanmarcos United States SMGeoES1 GC-MONO-ES-CLEF2006 es TD AUTOMATIC yes sanmarcos United States SMGeoES2 GC-MONO-ES-CLEF2006 es TDN AUTOMATIC yes sanmarcos United States SMGeoES3 GC-MONO-ES-CLEF2006 es TD MANUAL yes berkeley United States BKGeoP2 GC-MONO-PT-CLEF2006 pt TDN AUTOMATIC yes berkeley United States BKGeoP1 GC-MONO-PT-CLEF2006 pt TD AUTOMATIC yes berkeley United States BKGeoP4 GC-MONO-PT-CLEF2006 pt TDN AUTOMATIC yes berkeley United States BKGeoP3 GC-MONO-PT-CLEF2006 pt TD AUTOMATIC yes sanmarcos United States SMGeoPT4 GC-MONO-PT-CLEF2006 pt TD AUTOMATIC yes sanmarcos United States SMGeoPT2 GC-MONO-PT-CLEF2006 pt TD AUTOMATIC yes sanmarcos United States SMGeoPT1 GC-MONO-PT-CLEF2006 pt TDN AUTOMATIC yes sanmarcos United States SMGeoPT3 GC-MONO-PT-CLEF2006 pt TDN AUTOMATIC yes 10 Participant Country Experiment ID Task Topic Topic Query Pool Lang. Fields Construction xldb Portugal XLDBGeoPTAut02 GC-MONO-PT-CLEF2006 pt TD MANUAL yes xldb Portugal XLDBGeoPTAut05 GC-MONO-PT-CLEF2006 pt TD AUTOMATIC yes xldb Portugal XLDBGeoManualPT GC-MONO-PT-CLEF2006 pt TD MANUAL yes xldb Portugal XLDBGeoPTAut03 GC-MONO-PT-CLEF2006 pt TD AUTOMATIC yes xldb Portugal XLDBGeoPTAut03_2 GC-MONO-PT-CLEF2006 pt TD AUTOMATIC yes berkeley United States BKGeoED2 GC-BILI-X2DE-CLEF2006 en TDN MANUAL yes berkeley United States BKGeoED1 GC-BILI-X2DE-CLEF2006 en TD AUTOMATIC yes hagen Germany FUHedGNNNTDN GC-BILI-X2DE-CLEF2006 en TDN AUTOMATIC yes hagen Germany FUHedGYYYTDN GC-BILI-X2DE-CLEF2006 en TDN AUTOMATIC yes hagen Germany FUHedGNNNTD GC-BILI-X2DE-CLEF2006 en TD AUTOMATIC yes hagen Germany FUHedGYYYTD GC-BILI-X2DE-CLEF2006 en TD AUTOMATIC yes hagen Germany FUHedGYYYMTDN GC-BILI-X2DE-CLEF2006 en TDN AUTOMATIC yes hildesheim Germany HIGeoenderun21 GC-BILI-X2DE-CLEF2006 en TD AUTOMATIC yes hildesheim Germany HIGeoenderun22 GC-BILI-X2DE-CLEF2006 en TD AUTOMATIC yes hildesheim Germany HIGeoenderun21n GC-BILI-X2DE-CLEF2006 en TDN AUTOMATIC yes hildesheim Germany HIGeoenderun22n GC-BILI-X2DE-CLEF2006 en TDN AUTOMATIC yes hildesheim Germany HIGeodeenrun12 GC-BILI-X2EN-CLEF2006 de TD AUTOMATIC yes hildesheim Germany HIGeodeenrun13n GC-BILI-X2EN-CLEF2006 de TDN AUTOMATIC yes hildesheim Germany HIGeodeenrun11n GC-BILI-X2EN-CLEF2006 de TDN AUTOMATIC no hildesheim Germany HIGeodeenrun11 GC-BILI-X2EN-CLEF2006 de TD AUTOMATIC no hildesheim Germany HIGeodeenrun13 GC-BILI-X2EN-CLEF2006 de TD AUTOMATIC no jaen Spain sinaiEsEnExp1 GC-BILI-X2EN-CLEF2006 es TDN AUTOMATIC yes jaen Spain sinaiDeEnExp2 GC-BILI-X2EN-CLEF2006 de TD AUTOMATIC no jaen Spain sinaiEsEnExp3 GC-BILI-X2EN-CLEF2006 es TD AUTOMATIC no jaen Spain sinaiDeEnExp1 GC-BILI-X2EN-CLEF2006 de TDN AUTOMATIC no jaen Spain sinaiEsEnExp2 GC-BILI-X2EN-CLEF2006 es TD AUTOMATIC yes sanmarcos United States SMGeoESEN1 GC-BILI-X2EN-CLEF2006 es TDN MANUAL yes sanmarcos United States SMGeoESEN2 GC-BILI-X2EN-CLEF2006 es TD AUTOMATIC yes berkeley United States BKGeoES1 GC-BILI-X2ES-CLEF2006 en TD AUTOMATIC yes berkeley United States BKGeoES2 GC-BILI-X2ES-CLEF2006 en TDN AUTOMATIC yes sanmarcos United States SMGeoENES1 GC-BILI-X2ES-CLEF2006 en TD AUTOMATIC yes sanmarcos United States SMGeoPTES2 GC-BILI-X2ES-CLEF2006 pt TD AUTOMATIC yes sanmarcos United States SMGeoPTES3 GC-BILI-X2ES-CLEF2006 pt TDN AUTOMATIC yes berkeley United States BKGeoEP1 GC-BILI-X2PT-CLEF2006 en TD AUTOMATIC yes berkeley United States BKGeoEP2 GC-BILI-X2PT-CLEF2006 en TDN AUTOMATIC yes sanmarcos United States SMGeoESPT1 GC-BILI-X2PT-CLEF2006 es TD MANUAL yes sanmarcos United States SMGeoESPT2 GC-BILI-X2PT-CLEF2006 es TD AUTOMATIC yes 11 12 Track Overview Results and Graphs 13 14 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-DE-CLEF2006 GeoCLEF Monolingual German track Top 5 Participants − Interpolated Recall vs Average Precision 100% hagen [FUHddGYYYTD; MAP 22.29%; Pooled] berkeley [BKGeoD1; MAP 21.51%; Pooled] 90% hildesheim [HIGeodederun4; MAP 15.58%; Pooled] daedalus [GCdeNtLg; MAP 10.01%; Pooled] 80% 70% Average Precision 60% 50% 40% 30% 20% 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall GeoCLEF Monolingual German track Top 5 Participants − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050 ) 1 hagen [FUHddGYYYTD; MAP 22.29%; Pooled] berkeley [BKGeoD1; MAP 21.51%; Pooled] hildesheim [HIGeodederun4; MAP 15.58%; Pooled] daedalus [GCdeNtLg; MAP 10.01%; Pooled] 0.8 0.6 0.4 0.2 Difference 0 −0.2 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 15 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-DE-CLEF2006 GeoCLEF Monolingual German track − Box Plot of the Topics FUHddGYYYTD [MAP 22.29%; Pooled] BKGeoD1 [MAP 21.51%; Pooled] FUHddGYYYTDN [MAP 21.41%; Pooled] FUHddGYYYMTDN [MAP 19.99%; Pooled] BKGeoD2 [MAP 18.22%; Pooled] FUHddGNNNTD [MAP 16.94%; Pooled] HIGeodederun4n [MAP 16.01%; Pooled] Experiments HIGeodederun4 [MAP 15.58%; Pooled] FUHddGNNNTDN [MAP 12.23%; Pooled] HIGeodederun6 [MAP 12.14%; Pooled] HIGeodederun6n [MAP 11.34%; Pooled] GCdeNtLg [MAP 10.01%; Pooled] GCdeNA [MAP 9.28%; Pooled] GCdeAtLg [MAP 7.36%; Pooled] GCdeAA [MAP 7.15%; Pooled] GCdeAO [MAP 5.48%; Pooled] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Mean Average Precision 16 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-DE-CLEF2006 GeoCLEF Monolingual German track − Tukey T test with "top group" highlighted BKGeoD1 FUHddGYYYTD FUHddGYYYTDN FUHddGYYYMTDN BKGeoD2 FUHddGNNNTD HIGeodederun4n Experiments HIGeodederun4 FUHddGNNNTDN GCdeNtLg HIGeodederun6 HIGeodederun6n GCdeNA GCdeAtLg GCdeAA GCdeAO 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 arcsin(sqrt(Mean average precision)) 17 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-DE-CLEF2006 GeoCLEF Monolingual German track Top 5 Participants − Retrieved documents vs Precision 100% hagen [FUHddGYYYTD; R−Prec 21.53%; Pooled] berkeley [BKGeoD1; R−Prec 19.99%; Pooled] 90% hildesheim [HIGeodederun4; R−Prec 18.15%; Pooled] daedalus [GCdeNtLg; R−Prec 11.53%; Pooled] 80% 70% 60% R−Precision 50% 40% 30% 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) GeoCLEF Monolingual German track Top 5 Participants − Comparison to Median R−Precision by Topic (Topics 026 to 050) 1 hagen [FUHddGYYYTD; R−Prec 21.53%; Pooled] berkeley [BKGeoD1; R−Prec 19.99%; Pooled] hildesheim [HIGeodederun4; R−Prec 18.15%; Pooled] daedalus [GCdeNtLg; R−Prec 11.53%; Pooled] 0.8 0.6 0.4 0.2 Difference 0 −0.2 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 18 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-DE-CLEF2006 GeoCLEF Monolingual German track − Box Plot of the Topics FUHddGYYYTD [R−Prec 21.53%; Pooled] FUHddGYYYTDN [R−Prec 20.56%; Pooled] FUHddGYYYMTDN [R−Prec 20.39%; Pooled] BKGeoD1 [R−Prec 19.99%; Pooled] BKGeoD2 [R−Prec 18.57%; Pooled] HIGeodederun4 [R−Prec 18.15%; Pooled] FUHddGNNNTD [R−Prec 18.00%; Pooled] Experiments HIGeodederun4n [R−Prec 17.68%; Pooled] HIGeodederun6n [R−Prec 13.72%; Pooled] HIGeodederun6 [R−Prec 13.45%; Pooled] FUHddGNNNTDN [R−Prec 13.40%; Pooled] GCdeNtLg [R−Prec 11.53%; Pooled] GCdeNA [R−Prec 10.19%; Pooled] GCdeAA [R−Prec 8.93%; Pooled] GCdeAtLg [R−Prec 8.78%; Pooled] GCdeAO [R−Prec 6.62%; Pooled] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% R−Precision 19 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-DE-CLEF2006 GeoCLEF Monolingual German track − Tukey T test with "top group" highlighted FUHddGYYYTD FUHddGYYYMTDN FUHddGYYYTDN HIGeodederun4n FUHddGNNNTD HIGeodederun4 BKGeoD1 Experiments BKGeoD2 FUHddGNNNTDN HIGeodederun6n HIGeodederun6 GCdeNtLg GCdeNA GCdeAtLg GCdeAA GCdeAO 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 arcsin(sqrt(R Precision)) 20 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-DE-CLEF2006 Average Precision R-Precision Topic Minimum 1st Q. Median 3rd Q. Maximum Mean Std Minimum 1st Q. Median 3rd Q. Maximum Mean Std 026 0.0000 0.0004 0.0012 0.0040 0.0172 0.0034 0.0050 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 027 0.0000 0.0020 0.0135 0.0209 0.0359 0.0132 0.0106 0.0000 0.0231 0.0692 0.1154 0.1385 0.0683 0.0512 028 0.0000 0.0084 0.0745 0.2362 0.3916 0.1238 0.1293 0.0000 0.0156 0.1562 0.3438 0.4375 0.1855 0.1736 029 0.0011 0.0193 0.0716 0.3494 0.4667 0.1629 0.1739 0.0000 0.0000 0.0000 0.3333 0.3333 0.1458 0.1708 030 0.1256 0.4398 0.5873 0.6712 0.7862 0.5300 0.1906 0.1000 0.5000 0.6000 0.6333 0.7667 0.5354 0.1832 031 0.0023 0.0230 0.0507 0.1284 0.8249 0.1912 0.3124 0.0000 0.0329 0.0658 0.2171 0.8158 0.2097 0.2943 032 0.0023 0.2005 0.6070 0.6648 0.7861 0.4625 0.2885 0.0185 0.2407 0.5833 0.6667 0.8148 0.4711 0.2834 033 0.0018 0.0221 0.0573 0.0909 0.1176 0.0560 0.0413 0.0000 0.0000 0.0588 0.1176 0.1765 0.0772 0.0670 034 0.0000 0.0738 0.4100 0.4744 0.6704 0.3121 0.2253 0.0000 0.0882 0.4265 0.4706 0.6176 0.3143 0.2239 035 0.0000 0.0048 0.0305 0.0675 0.1231 0.0377 0.0349 0.0000 0.0000 0.0278 0.0556 0.1667 0.0417 0.0517 036 0.0000 0.0018 0.0079 0.0495 0.4167 0.0873 0.1627 0.0000 0.0000 0.0000 0.0000 0.3333 0.0625 0.1344 037 0.0000 0.0001 0.0065 0.0878 0.2303 0.0531 0.0851 0.0000 0.0000 0.0000 0.0909 0.2727 0.0511 0.0876 038 0.0000 0.0072 0.0221 0.0962 0.2381 0.0635 0.0765 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 039 0.0004 0.0133 0.0690 0.2032 0.3491 0.1119 0.1171 0.0000 0.0370 0.1296 0.2037 0.4074 0.1319 0.1194 040 0.0000 0.1421 0.2841 0.3259 0.4016 0.2285 0.1237 0.0000 0.2073 0.3659 0.4268 0.4878 0.3110 0.1507 041 0.0000 0.0091 0.0180 0.0491 0.2184 0.0441 0.0649 0.0000 0.0000 0.0526 0.1053 0.2105 0.0592 0.0716 042 0.0062 0.0167 0.0409 0.0706 0.1273 0.0468 0.0339 0.0000 0.0319 0.0532 0.1383 0.2128 0.0785 0.0635 043 0.0016 0.0090 0.0178 0.0329 0.0617 0.0239 0.0194 0.0000 0.0000 0.0357 0.0714 0.1429 0.0446 0.0513 044 0.0023 0.0079 0.0132 0.0233 0.3340 0.0463 0.0902 0.0000 0.0000 0.0000 0.0000 0.3333 0.0417 0.0962 045 0.0000 0.0003 0.0166 0.0635 0.6250 0.0652 0.1528 0.0000 0.0000 0.0000 0.0000 0.5000 0.0312 0.1250 046 0.0000 0.0000 0.1612 0.2803 0.5108 0.1782 0.1561 0.0000 0.0000 0.2500 0.2500 0.5000 0.1875 0.1443 047 0.0000 0.0060 0.0299 0.0622 0.3911 0.0515 0.0946 0.0000 0.0000 0.0000 0.0833 0.5000 0.0625 0.1344 048 0.0950 0.2137 0.5931 0.8834 0.9161 0.5646 0.3107 0.1594 0.2464 0.6232 0.8406 0.8841 0.5634 0.2775 049 0.0000 0.0095 0.0432 0.0807 0.1763 0.0529 0.0546 0.0000 0.0000 0.0000 0.0833 0.1667 0.0417 0.0745 050 0.0000 0.0053 0.0373 0.0599 0.0755 0.0352 0.0280 0.0000 0.0000 0.0833 0.0833 0.1667 0.0573 0.0587 ALL 0.0548 0.0965 0.1391 0.1910 0.2229 0.1418 0.0556 0.0662 0.1086 0.1570 0.1928 0.2153 0.1509 0.0486 21 22 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-EN-CLEF2006 GeoCLEF Monolingual English track Top 5 Participants − Interpolated Recall vs Average Precision 100% xldb [XLDBGeoManualEN; MAP 30.34%; Not Pooled] alicante [enTD; MAP 27.23%; Pooled] 90% sanmarcos [SMGeoEN4; MAP 26.37%; Not Pooled] unsw [unswTitleBaseline; MAP 26.22%; Pooled] jaen [sinaiEnEnExp4; MAP 26.11%; Not Pooled] 80% 70% Average Precision 60% 50% 40% 30% 20% 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall GeoCLEF Monolingual English track Top 5 Participants − Comparison to Median Mean Average Precision by Topic (Topics 026 to 05 0) 1 xldb [XLDBGeoManualEN; MAP 30.34%; Not Pooled] alicante [enTD; MAP 27.23%; Pooled] sanmarcos [SMGeoEN4; MAP 26.37%; Not Pooled] unsw [unswTitleBaseline; MAP 26.22%; Pooled] 0.8 jaen [sinaiEnEnExp4; MAP 26.11%; Not Pooled] 0.6 0.4 0.2 Difference 0 −0.2 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 23 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-EN-CLEF2006 GeoCLEF Monolingual English track − Box Plot of the Topics sinaiEnEnExp1 [MAP 32.24%; Pooled] XLDBGeoManualEN [MAP 30.34%; Not Pooled] enTDN [MAP 29.85%; Not Pooled] BKGeoE4 [MAP 28.87%; Pooled] SMGeoEN3 [MAP 28.57%; Not Pooled] BKGeoE3 [MAP 28.27%; Not Pooled] unswNarrBaseline [MAP 27.58%; Pooled] rfiaUPV02 [MAP 27.35%; Not Pooled] enTD [MAP 27.23%; Pooled] rfiaUPV04 [MAP 26.60%; Pooled] BKGeoE2 [MAP 26.56%; Not Pooled] SMGeoEN4 [MAP 26.37%; Not Pooled] SMGeoEN1 [MAP 26.37%; Pooled] unswTitleBaseline [MAP 26.22%; Pooled] sinaiEnEnExp4 [MAP 26.11%; Not Pooled] rfiaUPV01 [MAP 25.07%; Not Pooled] sinaiEnEnExp2 [MAP 25.04%; Pooled] BKGeoE1 [MAP 24.99%; Pooled] UBManual2 [MAP 24.46%; Not Pooled] MuTdnTxt [MAP 24.44%; Pooled] sinaiEnEnExp5 [MAP 24.07%; Not Pooled] UAUJAUPVenenExp1 [MAP 24.03%; Not Pooled] MuTdnManQexpGeo [MAP 24.00%; Not Pooled] msramanual [MAP 23.95%; Pooled] SMGeoEN5 [MAP 23.77%; Pooled] SMGeoEN5 [MAP 23.77%; Not Pooled] UBGTDrf1 [MAP 23.44%; Not Pooled] MuTdRedn [MAP 23.41%; Not Pooled] rfiaUPV03 [MAP 23.35%; Pooled] UBGTDrf2 [MAP 23.30%; Pooled] MuTdTxt [MAP 23.12%; Not Pooled] UBGManual1 [MAP 23.07%; Pooled] sinaiEnEnExp3 [MAP 22.95%; Not Pooled] MuTdQexpPrb [MAP 22.18%; Pooled] unswTitleF46 [MAP 22.15%; Not Pooled] Experiments CLCGGeoEE11 [MAP 21.94%; Not Pooled] CLCGGeoEE2 [MAP 21.63%; Pooled] XLDBGeoENAut05 [MAP 21.45%; Not Pooled] XLDBGeoENAut03_2 [MAP 20.79%; Pooled] msrawhitelist [MAP 20.00%; Pooled] ICgeoMLtdn [MAP 19.53%; Pooled] HIGeoenenrun3 [MAP 18.75%; Pooled] XLDBGeoENAut03 [MAP 18.67%; Pooled] msralocal [MAP 18.37%; Not Pooled] msratext [MAP 18.35%; Pooled] CLCGGeoEE5 [MAP 17.57%; Not Pooled] HIGeoenenrun1n [MAP 17.47%; Not Pooled] CLCGGeoEE1 [MAP 17.30%; Pooled] utGeoTIBm [MAP 17.18%; Not Pooled] CLCGGeoEE10 [MAP 16.90%; Not Pooled] utGeoTdnIBm [MAP 16.77%; Not Pooled] HIGeoenenrun1 [MAP 16.76%; Not Pooled] ICgeoMLtd [MAP 16.49%; Pooled] utGeoTIB [MAP 16.23%; Pooled] XLDBGeoENAut02 [MAP 15.79%; Not Pooled] msraexpansion [MAP 15.21%; Not Pooled] GCenAA [MAP 13.60%; Pooled] TALPGeoIRTD1 [MAP 13.42%; Pooled] GCenAtLg [MAP 13.05%; Not Pooled] HIGeoenenrun2n [MAP 12.13%; Pooled] enTDNGeoNames [MAP 12.01%; Pooled] TALPGeoIRTDN1 [MAP 11.79%; Pooled] HIGeoenenrun2 [MAP 11.66%; Not Pooled] utGeoTdnIB [MAP 11.34%; Pooled] TALPGeoIRTDN3 [MAP 9.97%; Not Pooled] GCenNtLg [MAP 9.37%; Not Pooled] GCenNA [MAP 8.93%; Pooled] GCenAO [MAP 8.91%; Not Pooled] TALPGeoIRTD2 [MAP 7.66%; Not Pooled] utGeoTdIB [MAP 7.32%; Pooled] TALPGeoIRTDN2 [MAP 6.38%; Not Pooled] unswNarrF41 [MAP 4.01%; Not Pooled] unswNarrMap [MAP 4.00%; Not Pooled] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Mean Average Precision 24 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-EN-CLEF2006 GeoCLEF Monolingual English track − Tukey T test with "top group" highlighted XLDBGeoManualEN sinaiEnEnExp1 enTDN SMGeoEN3 unswNarrBaseline BKGeoE4 BKGeoE3 SMGeoEN1 SMGeoEN4 unswTitleBaseline sinaiEnEnExp4 enTD rfiaUPV02 BKGeoE2 rfiaUPV04 sinaiEnEnExp2 BKGeoE1 MuTdnTxt sinaiEnEnExp5 rfiaUPV01 UBManual2 SMGeoEN5 SMGeoEN5 MuTdTxt UBGTDrf1 UBGTDrf2 msramanual MuTdnManQexpGeo sinaiEnEnExp3 UBGManual1 MuTdRedn rfiaUPV03 Experiments unswTitleF46 UAUJAUPVenenExp1 XLDBGeoENAut05 MuTdQexpPrb CLCGGeoEE11 CLCGGeoEE2 XLDBGeoENAut03_2 msrawhitelist ICgeoMLtdn msralocal msratext CLCGGeoEE1 utGeoTIBm utGeoTdnIBm XLDBGeoENAut03 CLCGGeoEE5 utGeoTIB CLCGGeoEE10 XLDBGeoENAut02 HIGeoenenrun3 ICgeoMLtd HIGeoenenrun1n msraexpansion HIGeoenenrun1 GCenAtLg TALPGeoIRTD1 TALPGeoIRTDN1 GCenAA HIGeoenenrun2n utGeoTdnIB enTDNGeoNames GCenNtLg TALPGeoIRTDN3 HIGeoenenrun2 GCenAO utGeoTdIB GCenNA TALPGeoIRTDN2 unswNarrF41 TALPGeoIRTD2 unswNarrMap 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 arcsin(sqrt(Mean average precision)) 25 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-EN-CLEF2006 GeoCLEF Monolingual English track Top 5 Participants − Retrieved documents vs Precision 100% xldb [XLDBGeoManualEN; R−Prec 33.60%; Not Pooled] alicante [enTD; R−Prec 28.01%; Pooled] 90% sanmarcos [SMGeoEN4; R−Prec 28.57%; Not Pooled] unsw [unswTitleBaseline; R−Prec 28.21%; Pooled] jaen [sinaiEnEnExp4; R−Prec 22.61%; Not Pooled] 80% 70% 60% R−Precision 50% 40% 30% 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) GeoCLEF Monolingual English track Top 5 Participants − Comparison to Median R−Precision by Topic (Topics 026 to 050) 1 xldb [XLDBGeoManualEN; R−Prec 33.60%; Not Pooled] alicante [enTD; R−Prec 28.01%; Pooled] sanmarcos [SMGeoEN4; R−Prec 28.57%; Not Pooled] unsw [unswTitleBaseline; R−Prec 28.21%; Pooled] 0.8 jaen [sinaiEnEnExp4; R−Prec 22.61%; Not Pooled] 0.6 0.4 0.2 Difference 0 −0.2 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 26 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-EN-CLEF2006 GeoCLEF Monolingual English track − Box Plot of the Topics XLDBGeoManualEN [R−Prec 33.60%; Not Pooled] sinaiEnEnExp1 [R−Prec 29.34%; Pooled] SMGeoEN4 [R−Prec 28.57%; Not Pooled] SMGeoEN1 [R−Prec 28.57%; Pooled] enTDN [R−Prec 28.51%; Not Pooled] SMGeoEN3 [R−Prec 28.36%; Not Pooled] unswTitleBaseline [R−Prec 28.21%; Pooled] enTD [R−Prec 28.01%; Pooled] BKGeoE4 [R−Prec 27.11%; Pooled] unswTitleF46 [R−Prec 26.87%; Not Pooled] rfiaUPV04 [R−Prec 26.67%; Pooled] BKGeoE3 [R−Prec 26.58%; Not Pooled] rfiaUPV02 [R−Prec 26.50%; Not Pooled] unswNarrBaseline [R−Prec 25.88%; Pooled] SMGeoEN5 [R−Prec 25.81%; Pooled] SMGeoEN5 [R−Prec 25.81%; Not Pooled] msramanual [R−Prec 25.45%; Pooled] UBGTDrf1 [R−Prec 25.16%; Not Pooled] BKGeoE2 [R−Prec 24.84%; Not Pooled] UBGManual1 [R−Prec 24.73%; Pooled] UBManual2 [R−Prec 24.59%; Not Pooled] rfiaUPV01 [R−Prec 24.18%; Not Pooled] ICgeoMLtdn [R−Prec 23.55%; Pooled] msrawhitelist [R−Prec 23.52%; Pooled] UAUJAUPVenenExp1 [R−Prec 23.19%; Not Pooled] MuTdnManQexpGeo [R−Prec 23.00%; Not Pooled] sinaiEnEnExp4 [R−Prec 22.61%; Not Pooled] msralocal [R−Prec 22.45%; Not Pooled] MuTdQexpPrb [R−Prec 22.40%; Pooled] UBGTDrf2 [R−Prec 22.19%; Pooled] XLDBGeoENAut05 [R−Prec 21.97%; Not Pooled] BKGeoE1 [R−Prec 21.95%; Pooled] sinaiEnEnExp2 [R−Prec 21.94%; Pooled] CLCGGeoEE2 [R−Prec 21.94%; Pooled] rfiaUPV03 [R−Prec 21.93%; Pooled] Experiments MuTdRedn [R−Prec 21.92%; Not Pooled] MuTdnTxt [R−Prec 21.84%; Pooled] MuTdTxt [R−Prec 21.55%; Not Pooled] XLDBGeoENAut03_2 [R−Prec 21.53%; Pooled] CLCGGeoEE11 [R−Prec 21.44%; Not Pooled] msratext [R−Prec 21.23%; Pooled] sinaiEnEnExp5 [R−Prec 20.95%; Not Pooled] sinaiEnEnExp3 [R−Prec 20.28%; Not Pooled] CLCGGeoEE1 [R−Prec 19.83%; Pooled] ICgeoMLtd [R−Prec 19.69%; Pooled] XLDBGeoENAut03 [R−Prec 19.47%; Pooled] msraexpansion [R−Prec 18.53%; Not Pooled] utGeoTdnIBm [R−Prec 18.12%; Not Pooled] HIGeoenenrun3 [R−Prec 17.85%; Pooled] CLCGGeoEE5 [R−Prec 17.77%; Not Pooled] CLCGGeoEE10 [R−Prec 17.62%; Not Pooled] utGeoTIBm [R−Prec 17.38%; Not Pooled] utGeoTIB [R−Prec 17.38%; Pooled] HIGeoenenrun1n [R−Prec 16.33%; Not Pooled] HIGeoenenrun1 [R−Prec 15.95%; Not Pooled] GCenAA [R−Prec 15.70%; Pooled] XLDBGeoENAut02 [R−Prec 15.28%; Not Pooled] TALPGeoIRTD1 [R−Prec 13.70%; Pooled] utGeoTdnIB [R−Prec 13.66%; Pooled] GCenAtLg [R−Prec 13.57%; Not Pooled] TALPGeoIRTDN1 [R−Prec 13.16%; Pooled] HIGeoenenrun2 [R−Prec 13.05%; Not Pooled] HIGeoenenrun2n [R−Prec 13.04%; Pooled] GCenNtLg [R−Prec 10.87%; Not Pooled] enTDNGeoNames [R−Prec 10.30%; Pooled] TALPGeoIRTDN3 [R−Prec 9.85%; Not Pooled] GCenNA [R−Prec 9.70%; Pooled] GCenAO [R−Prec 9.52%; Not Pooled] TALPGeoIRTD2 [R−Prec 8.84%; Not Pooled] TALPGeoIRTDN2 [R−Prec 8.13%; Not Pooled] utGeoTdIB [R−Prec 7.62%; Pooled] unswNarrF41 [R−Prec 4.06%; Not Pooled] unswNarrMap [R−Prec 4.06%; Not Pooled] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% R−Precision 27 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-EN-CLEF2006 GeoCLEF Monolingual English track − Tukey T test with "top group" highlighted XLDBGeoManualEN SMGeoEN1 SMGeoEN4 unswTitleBaseline unswTitleF46 sinaiEnEnExp1 SMGeoEN3 enTDN enTD BKGeoE4 unswNarrBaseline rfiaUPV02 BKGeoE3 UBGManual1 rfiaUPV04 UBGTDrf1 SMGeoEN5 SMGeoEN5 msramanual UBManual2 rfiaUPV01 ICgeoMLtdn BKGeoE2 msrawhitelist msralocal XLDBGeoENAut05 MuTdnManQexpGeo sinaiEnEnExp4 MuTdnTxt MuTdTxt MuTdQexpPrb XLDBGeoENAut03_2 Experiments CLCGGeoEE2 UBGTDrf2 MuTdRedn BKGeoE1 CLCGGeoEE11 UAUJAUPVenenExp1 sinaiEnEnExp2 sinaiEnEnExp5 rfiaUPV03 msratext sinaiEnEnExp3 ICgeoMLtd XLDBGeoENAut03 CLCGGeoEE1 msraexpansion utGeoTdnIBm utGeoTIB utGeoTIBm CLCGGeoEE10 CLCGGeoEE5 HIGeoenenrun3 XLDBGeoENAut02 HIGeoenenrun1n GCenAA HIGeoenenrun1 utGeoTdnIB TALPGeoIRTD1 GCenAtLg TALPGeoIRTDN1 HIGeoenenrun2n HIGeoenenrun2 GCenNtLg TALPGeoIRTDN3 GCenNA utGeoTdIB GCenAO enTDNGeoNames TALPGeoIRTDN2 TALPGeoIRTD2 unswNarrMap unswNarrF41 −0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 arcsin(sqrt(R Precision)) 28 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-EN-CLEF2006 Average Precision R-Precision Topic Minimum 1st Q. Median 3rd Q. Maximum Mean Std Minimum 1st Q. Median 3rd Q. Maximum Mean Std 026 0.0000 0.0096 0.0997 0.1511 0.5009 0.1191 0.1294 0.0000 0.0000 0.1111 0.2500 0.5556 0.1553 0.1612 027 0.0000 0.0071 0.0272 0.0605 0.1257 0.0373 0.0367 0.0000 0.0000 0.0526 0.1053 0.2105 0.0620 0.0521 028 0.0000 0.0016 0.0509 0.0948 0.3017 0.0694 0.0789 0.0000 0.0000 0.1053 0.2105 0.4211 0.1211 0.1218 029 0.0000 0.0563 0.1046 0.1983 0.5485 0.1386 0.1084 0.0000 0.1111 0.1111 0.2222 0.6667 0.1598 0.1446 030 0.0000 0.2022 0.5984 0.8356 1.0000 0.5443 0.3362 0.0000 0.2917 0.6667 0.8333 1.0000 0.5137 0.3090 031 0.0105 0.0634 0.2611 0.3582 0.6879 0.2423 0.1662 0.0339 0.1525 0.2542 0.3771 0.6610 0.2730 0.1446 032 0.0000 0.5633 0.8145 0.9047 0.9631 0.6987 0.2725 0.0000 0.6452 0.7419 0.8387 0.9032 0.6757 0.2383 033 0.0000 0.0019 0.0035 0.0142 0.4713 0.0479 0.1163 0.0000 0.0000 0.0000 0.0500 0.5500 0.0644 0.1383 034 0.0000 0.0489 0.3693 0.4167 0.8056 0.2916 0.2039 0.0000 0.0000 0.3333 0.4167 0.6667 0.3242 0.2420 035 0.0000 0.0097 0.0276 0.0781 0.5397 0.0640 0.0994 0.0000 0.0000 0.0000 0.0000 0.5000 0.0479 0.0981 036 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 037 0.0000 0.0028 0.0183 0.1002 0.2731 0.0553 0.0668 0.0000 0.0000 0.0625 0.1875 0.3750 0.0933 0.1027 038 0.0000 0.0017 0.0128 0.0331 1.0000 0.0541 0.1384 0.0000 0.0000 0.0000 0.0000 1.0000 0.0137 0.1170 039 0.0000 0.0491 0.1120 0.3437 0.5778 0.1872 0.1558 0.0000 0.0625 0.1875 0.3750 0.5000 0.2149 0.1620 040 0.0000 0.0539 0.2175 0.3213 0.8560 0.2112 0.1861 0.0000 0.0536 0.2143 0.2857 0.7857 0.2084 0.1661 041 0.0000 0.0000 0.0024 0.0086 0.2500 0.0123 0.0412 0.0000 0.0000 0.0000 0.0000 0.2500 0.0068 0.0411 042 0.0000 0.0109 0.0970 0.5016 1.0000 0.2534 0.2866 0.0000 0.0000 0.0000 0.5000 1.0000 0.1918 0.2717 043 0.0000 0.0026 0.0082 0.0259 0.3115 0.0290 0.0583 0.0000 0.0000 0.0000 0.0000 0.3750 0.0342 0.0759 044 0.0000 0.0468 0.1071 0.1595 0.2895 0.1143 0.0731 0.0000 0.1053 0.1579 0.2105 0.3684 0.1583 0.0898 045 0.0000 0.0069 0.0823 0.2265 0.8256 0.1498 0.1894 0.0000 0.0000 0.0000 0.1667 0.8333 0.1301 0.2065 046 0.0000 0.1542 0.6686 0.7083 1.0000 0.5205 0.3059 0.0000 0.2500 0.6667 0.6667 1.0000 0.4840 0.2994 047 0.0000 0.0116 0.0364 0.0647 0.1914 0.0460 0.0418 0.0000 0.0000 0.0417 0.0833 0.2917 0.0559 0.0645 048 0.0625 0.5158 0.6973 0.7856 0.9086 0.6182 0.2347 0.0208 0.5573 0.6667 0.7292 0.8542 0.6084 0.2152 049 0.0000 0.1624 0.2667 0.5000 0.6429 0.2953 0.1969 0.0000 0.0000 0.5000 0.5000 0.5000 0.2534 0.2517 050 0.0000 0.0477 0.1323 0.2303 0.3143 0.1378 0.0974 0.0000 0.0667 0.1333 0.3333 0.4000 0.1726 0.1266 ALL 0.0400 0.1565 0.2163 0.2459 0.3224 0.1975 0.0682 0.0406 0.1589 0.2184 0.2492 0.3360 0.2009 0.0652 29 30 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-ES-CLEF2006 GeoCLEF Monolingual Spanish track Top 5 Participants − Interpolated Recall vs Average Precision 100% alicante [esTD; MAP 35.08%; Pooled] berkeley [BKGeoS1; MAP 31.82%; Pooled] 90% daedalus [GCesNtLg; MAP 16.12%; Pooled] sanmarcos [SMGeoES1; MAP 14.71%; Pooled] 80% 70% Average Precision 60% 50% 40% 30% 20% 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall GeoCLEF Monolingual Spanish track Top 5 Participants − Comparison to Median Mean Average Precision by Topic (Topics 026 to 05 0) 1 alicante [esTD; MAP 35.08%; Pooled] berkeley [BKGeoS1; MAP 31.82%; Pooled] daedalus [GCesNtLg; MAP 16.12%; Pooled] sanmarcos [SMGeoES1; MAP 14.71%; Pooled] 0.8 0.6 0.4 0.2 Difference 0 −0.2 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 31 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-ES-CLEF2006 GeoCLEF Monolingual Spanish track − Box Plot of the Topics esTD [MAP 35.08%; Pooled] esTDN [MAP 32.37%; Pooled] BKGeoS1 [MAP 31.82%; Pooled] BKGeoS2 [MAP 30.03%; Pooled] GCesNtLg [MAP 16.12%; Pooled] SMGeoES2 [MAP 15.33%; Pooled] esTDNGeoNames [MAP 15.25%; Pooled] Experiments SMGeoES3 [MAP 14.71%; Pooled] SMGeoES1 [MAP 14.71%; Pooled] SMGeoES5 [MAP 14.71%; Pooled] GCesAtLg [MAP 14.13%; Pooled] SMGeoES4 [MAP 13.78%; Pooled] GCesAA [MAP 13.48%; Pooled] GCesNA [MAP 12.73%; Pooled] GCesAO [MAP 12.21%; Pooled] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Mean Average Precision 32 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-ES-CLEF2006 GeoCLEF Monolingual Spanish track − Tukey T test with "top group" highlighted esTD esTDN BKGeoS1 BKGeoS2 GCesNtLg SMGeoES2 SMGeoES5 Experiments SMGeoES1 SMGeoES3 GCesAtLg SMGeoES4 GCesAA GCesAO esTDNGeoNames GCesNA 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 arcsin(sqrt(Mean average precision)) 33 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-ES-CLEF2006 GeoCLEF Monolingual Spanish track Top 5 Participants − Retrieved documents vs Precision 100% alicante [esTD; R−Prec 35.83%; Pooled] berkeley [BKGeoS1; R−Prec 32.11%; Pooled] 90% daedalus [GCesNtLg; R−Prec 18.59%; Pooled] sanmarcos [SMGeoES1; R−Prec 20.44%; Pooled] 80% 70% 60% R−Precision 50% 40% 30% 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) GeoCLEF Monolingual Spanish track Top 5 Participants − Comparison to Median R−Precision by Topic (Topics 026 to 050) 1 alicante [esTD; R−Prec 35.83%; Pooled] berkeley [BKGeoS1; R−Prec 32.11%; Pooled] daedalus [GCesNtLg; R−Prec 18.59%; Pooled] sanmarcos [SMGeoES1; R−Prec 20.44%; Pooled] 0.8 0.6 0.4 0.2 Difference 0 −0.2 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 34 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-ES-CLEF2006 GeoCLEF Monolingual Spanish track − Box Plot of the Topics esTD [R−Prec 35.83%; Pooled] esTDN [R−Prec 33.77%; Pooled] BKGeoS1 [R−Prec 32.11%; Pooled] BKGeoS2 [R−Prec 29.94%; Pooled] SMGeoES5 [R−Prec 20.44%; Pooled] SMGeoES3 [R−Prec 20.44%; Pooled] SMGeoES1 [R−Prec 20.44%; Pooled] Experiments SMGeoES2 [R−Prec 20.29%; Pooled] SMGeoES4 [R−Prec 18.63%; Pooled] GCesNtLg [R−Prec 18.59%; Pooled] GCesNA [R−Prec 17.18%; Pooled] GCesAA [R−Prec 17.01%; Pooled] GCesAtLg [R−Prec 16.58%; Pooled] esTDNGeoNames [R−Prec 16.23%; Pooled] GCesAO [R−Prec 13.82%; Pooled] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% R−Precision 35 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-ES-CLEF2006 GeoCLEF Monolingual Spanish track − Tukey T test with "top group" highlighted esTD esTDN BKGeoS1 BKGeoS2 SMGeoES1 SMGeoES3 SMGeoES5 Experiments SMGeoES2 SMGeoES4 GCesNtLg GCesNA GCesAtLg GCesAA GCesAO esTDNGeoNames 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 arcsin(sqrt(R Precision)) 36 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-ES-CLEF2006 Average Precision R-Precision Topic Minimum 1st Q. Median 3rd Q. Maximum Mean Std Minimum 1st Q. Median 3rd Q. Maximum Mean Std 026 0.0000 0.0000 0.0171 0.0236 0.1518 0.0372 0.0592 0.0000 0.0000 0.1111 0.1111 0.1667 0.0704 0.0711 027 0.0000 0.0000 0.0099 0.0171 0.1035 0.0162 0.0268 0.0000 0.0000 0.0000 0.0256 0.1026 0.0205 0.0352 028 0.0000 0.0067 0.2370 0.2582 0.3937 0.1676 0.1405 0.0000 0.0000 0.2778 0.2986 0.3333 0.1741 0.1449 029 0.0007 0.2228 0.2759 0.4774 0.6863 0.3195 0.1918 0.0000 0.2879 0.4242 0.5379 0.6061 0.3818 0.1744 030 0.0000 0.1352 0.1489 0.3549 0.6869 0.2430 0.2010 0.0000 0.2003 0.2119 0.3891 0.6689 0.2786 0.1839 031 0.0085 0.0099 0.0107 0.5776 0.7252 0.2147 0.3057 0.0314 0.0510 0.0706 0.5422 0.7333 0.2387 0.2731 032 0.1984 0.2000 0.8612 0.9447 0.9782 0.6720 0.3488 0.2077 0.2077 0.8462 0.8885 0.9154 0.6477 0.3232 033 0.0000 0.0001 0.0003 0.0161 0.5464 0.0527 0.1452 0.0000 0.0100 0.0100 0.0475 0.7100 0.0867 0.1927 034 0.0000 0.0908 0.1834 0.3013 0.3533 0.1848 0.1142 0.0000 0.0338 0.2432 0.4257 0.5135 0.2414 0.1849 035 0.0015 0.0858 0.1217 0.1507 0.1645 0.1093 0.0501 0.0000 0.1053 0.2105 0.2105 0.2632 0.1614 0.0855 036 0.0000 0.0029 0.1093 0.1531 0.5137 0.1200 0.1554 0.0000 0.0273 0.2091 0.2341 0.6091 0.1915 0.1959 037 0.0000 0.0001 0.0746 0.1105 0.2206 0.0744 0.0790 0.0000 0.0000 0.1379 0.1724 0.2759 0.1103 0.1011 038 0.0000 0.0000 0.0333 0.0667 0.2000 0.0473 0.0580 0.0000 0.0000 0.0000 0.0000 0.2000 0.0400 0.0828 039 0.0148 0.0783 0.2211 0.2572 0.3246 0.1746 0.1049 0.0149 0.1530 0.2090 0.2239 0.3881 0.1990 0.1030 040 0.0000 0.3429 0.5408 0.6722 0.7764 0.4881 0.2507 0.0000 0.3525 0.6043 0.6853 0.7338 0.4950 0.2550 041 0.0084 0.0090 0.2018 0.2814 0.3878 0.1703 0.1417 0.0533 0.0700 0.2400 0.3200 0.4000 0.2080 0.1267 042 0.0104 0.1179 0.1835 0.3087 0.5082 0.2177 0.1446 0.0000 0.2689 0.2830 0.3915 0.5283 0.2906 0.1588 043 0.0000 0.0000 0.0133 0.0288 0.4845 0.0486 0.1227 0.0000 0.0000 0.0000 0.1146 0.5833 0.0778 0.1582 044 0.0001 0.0459 0.1462 0.2915 0.4134 0.1760 0.1423 0.0000 0.1019 0.2136 0.3786 0.4660 0.2421 0.1513 045 0.0000 0.0044 0.0096 0.0153 0.0456 0.0123 0.0139 0.0000 0.0000 0.0000 0.0000 0.0833 0.0167 0.0345 046 0.0357 0.1065 0.1310 0.5951 0.8330 0.3167 0.3032 0.1071 0.1429 0.1429 0.5625 0.7500 0.3262 0.2514 047 0.0000 0.0001 0.0407 0.1044 0.1203 0.0500 0.0470 0.0000 0.0000 0.0339 0.1653 0.2203 0.0746 0.0797 048 0.0540 0.0589 0.4031 0.7027 0.8118 0.4110 0.2973 0.0755 0.0755 0.4226 0.6377 0.7208 0.4111 0.2648 049 0.0006 0.0987 0.5089 0.5745 0.7874 0.3817 0.2746 0.0184 0.1175 0.5115 0.6682 0.7051 0.4267 0.2677 050 0.0000 0.0347 0.0415 0.0807 0.2721 0.0685 0.0710 0.0000 0.0450 0.1200 0.1400 0.3600 0.1107 0.0919 ALL 0.1221 0.1386 0.1471 0.2655 0.3508 0.1910 0.0837 0.1382 0.1705 0.2029 0.2756 0.3583 0.2209 0.0710 37 38 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-PT-CLEF2006 GeoCLEF Monolingual Portuguese track Top 5 Participants − Interpolated Recall vs Average Precision 100% xldb [XLDBGeoManualPT; MAP 30.12%; Pooled] berkeley [BKGeoP3; MAP 16.92%; Pooled] 90% sanmarcos [SMGeoPT2; MAP 13.44%; Pooled] 80% 70% Average Precision 60% 50% 40% 30% 20% 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall GeoCLEF Monolingual Portuguese track Top 5 Participants − Comparison to Median Mean Average Precision by Topic (Topics 026 to050) 1 xldb [XLDBGeoManualPT; MAP 30.12%; Pooled] berkeley [BKGeoP3; MAP 16.92%; Pooled] sanmarcos [SMGeoPT2; MAP 13.44%; Pooled] 0.8 0.6 0.4 0.2 Difference 0 −0.2 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 39 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-PT-CLEF2006 GeoCLEF Monolingual Portuguese track − Box Plot of the Topics XLDBGeoManualPT [MAP 30.12%; Pooled] XLDBGeoPTAut05 [MAP 29.32%; Pooled] XLDBGeoPTAut02 [MAP 25.70%; Pooled] XLDBGeoPTAut03 [MAP 19.29%; Pooled] BKGeoP4 [MAP 17.36%; Pooled] BKGeoP3 [MAP 16.92%; Pooled] Experiments BKGeoP2 [MAP 16.31%; Pooled] BKGeoP1 [MAP 16.22%; Pooled] XLDBGeoPTAut03_2 [MAP 15.13%; Pooled] SMGeoPT2 [MAP 13.44%; Pooled] SMGeoPT1 [MAP 10.98%; Pooled] SMGeoPT3 [MAP 10.98%; Pooled] SMGeoPT4 [MAP 10.63%; Pooled] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Mean Average Precision 40 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-PT-CLEF2006 GeoCLEF Monolingual Portuguese track − Tukey T test with "top group" highlighted XLDBGeoManualPT XLDBGeoPTAut05 XLDBGeoPTAut02 XLDBGeoPTAut03 BKGeoP3 BKGeoP4 Experiments BKGeoP1 SMGeoPT2 BKGeoP2 XLDBGeoPTAut03_2 SMGeoPT3 SMGeoPT1 SMGeoPT4 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 arcsin(sqrt(Mean average precision)) 41 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-PT-CLEF2006 GeoCLEF Monolingual Portuguese track Top 5 Participants − Retrieved documents vs Precision 100% xldb [XLDBGeoManualPT; R−Prec 35.89%; Pooled] berkeley [BKGeoP3; R−Prec 16.51%; Pooled] 90% sanmarcos [SMGeoPT2; R−Prec 15.02%; Pooled] 80% 70% 60% R−Precision 50% 40% 30% 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) GeoCLEF Monolingual Portuguese track Top 5 Participants − Comparison to Median R−Precision by Topic (Topics 026 to 050) 1 xldb [XLDBGeoManualPT; R−Prec 35.89%; Pooled] berkeley [BKGeoP3; R−Prec 16.51%; Pooled] sanmarcos [SMGeoPT2; R−Prec 15.02%; Pooled] 0.8 0.6 0.4 0.2 Difference 0 −0.2 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 42 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-PT-CLEF2006 GeoCLEF Monolingual Portuguese track − Box Plot of the Topics XLDBGeoManualPT [R−Prec 35.89%; Pooled] XLDBGeoPTAut05 [R−Prec 34.57%; Pooled] XLDBGeoPTAut02 [R−Prec 28.09%; Pooled] XLDBGeoPTAut03 [R−Prec 23.91%; Pooled] XLDBGeoPTAut03_2 [R−Prec 17.29%; Pooled] BKGeoP4 [R−Prec 17.22%; Pooled] Experiments BKGeoP3 [R−Prec 16.51%; Pooled] BKGeoP2 [R−Prec 16.46%; Pooled] BKGeoP1 [R−Prec 16.43%; Pooled] SMGeoPT2 [R−Prec 15.02%; Pooled] SMGeoPT1 [R−Prec 13.91%; Pooled] SMGeoPT3 [R−Prec 13.91%; Pooled] SMGeoPT4 [R−Prec 13.57%; Pooled] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% R−Precision 43 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-PT-CLEF2006 GeoCLEF Monolingual Portuguese track − Tukey T test with "top group" highlighted XLDBGeoManualPT XLDBGeoPTAut05 XLDBGeoPTAut02 XLDBGeoPTAut03 SMGeoPT2 XLDBGeoPTAut03_2 Experiments BKGeoP4 BKGeoP3 BKGeoP1 SMGeoPT3 SMGeoPT1 SMGeoPT4 BKGeoP2 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 arcsin(sqrt(R Precision)) 44 GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-PT-CLEF2006 Average Precision R-Precision Topic Minimum 1st Q. Median 3rd Q. Maximum Mean Std Minimum 1st Q. Median 3rd Q. Maximum Mean Std 026 0.0000 0.0017 0.0040 0.0159 0.1959 0.0239 0.0538 0.0000 0.0000 0.0000 0.0000 0.2667 0.0308 0.0799 027 0.0004 0.0005 0.0012 0.0310 0.6435 0.0663 0.1771 0.0000 0.0098 0.0098 0.0784 0.7353 0.0913 0.1993 028 0.0000 0.0297 0.1309 0.1940 0.4252 0.1592 0.1436 0.0000 0.0938 0.1562 0.2344 0.5000 0.2043 0.1576 029 0.0638 0.1708 0.2773 0.3452 0.5215 0.2619 0.1421 0.1538 0.1795 0.3333 0.4038 0.4872 0.3057 0.1258 030 0.0112 0.3349 0.4555 0.5110 0.6566 0.3965 0.2318 0.0714 0.3393 0.4286 0.5714 0.6429 0.4066 0.2028 031 0.0000 0.1601 0.1842 0.2724 0.3334 0.1946 0.0936 0.0000 0.1311 0.1475 0.2418 0.4098 0.1942 0.1208 032 0.2995 0.4778 0.5345 0.6936 0.8587 0.5878 0.1539 0.3774 0.5566 0.5849 0.6792 0.8302 0.6096 0.1156 033 0.0000 0.0010 0.0019 0.0362 0.0988 0.0232 0.0373 0.0000 0.0000 0.0000 0.0000 0.2500 0.0192 0.0693 034 0.0003 0.0016 0.0457 0.1608 0.2519 0.0815 0.0931 0.0000 0.0000 0.1250 0.1250 0.3750 0.1058 0.1123 035 0.0000 0.0048 0.0094 0.0155 0.1505 0.0202 0.0397 0.0000 0.0000 0.0000 0.0000 0.1111 0.0085 0.0308 036 0.0000 0.0000 0.0097 0.0247 0.0952 0.0177 0.0257 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 037 0.0000 0.0005 0.0390 0.1791 0.3321 0.0966 0.1272 0.0000 0.0000 0.0000 0.2847 0.4444 0.1261 0.1788 038 0.0174 0.0544 0.0978 0.1432 0.7500 0.1790 0.2241 0.0000 0.0000 0.0000 0.2500 0.7500 0.1731 0.2774 039 0.0426 0.0450 0.0896 0.1481 0.3276 0.1257 0.0966 0.0435 0.0870 0.1304 0.1739 0.3478 0.1505 0.0966 040 0.0003 0.0008 0.0073 0.2421 0.5025 0.1204 0.1658 0.0000 0.0000 0.0833 0.3021 0.5000 0.1571 0.1694 041 0.0000 0.0001 0.0056 0.3792 0.6930 0.1594 0.2346 0.0000 0.0000 0.0192 0.4471 0.6731 0.1864 0.2594 042 0.1065 0.1883 0.3088 0.4082 0.5240 0.3060 0.1323 0.2000 0.2571 0.3714 0.4857 0.5429 0.3736 0.1175 043 0.0002 0.0008 0.0011 0.0187 0.0809 0.0174 0.0312 0.0000 0.0000 0.0000 0.0104 0.1667 0.0256 0.0552 044 0.0037 0.0065 0.0317 0.0819 0.3660 0.0830 0.1174 0.0000 0.0000 0.1053 0.1809 0.4605 0.1407 0.1512 045 0.0334 0.1661 0.2139 0.3015 0.3538 0.2268 0.0941 0.0854 0.2591 0.3415 0.3689 0.4756 0.3039 0.1011 046 0.0000 0.2242 0.2866 0.5128 0.5987 0.3561 0.1850 0.0000 0.3295 0.3333 0.5455 0.6515 0.4079 0.1747 047 0.0000 0.0000 0.0084 0.0686 0.1916 0.0404 0.0613 0.0000 0.0000 0.0000 0.0441 0.2353 0.0385 0.0744 048 0.0000 0.4794 0.5924 0.8680 0.9241 0.5979 0.3096 0.0000 0.5175 0.6014 0.7832 0.8112 0.5756 0.2604 049 0.0000 0.1378 0.2267 0.2731 0.3344 0.2035 0.0964 0.0000 0.1389 0.2778 0.3472 0.3889 0.2350 0.1401 050 0.0000 0.0395 0.1226 0.2166 0.2360 0.1240 0.0925 0.0000 0.0909 0.2273 0.2727 0.3409 0.1836 0.1168 ALL 0.1063 0.1283 0.1631 0.2089 0.3012 0.1788 0.0662 0.1357 0.1475 0.1651 0.2495 0.3589 0.2021 0.0784 45 46 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2DE-CLEF2006 GeoCLEF Bilingual German track Top 5 Participants − Interpolated Recall vs Average Precision 100% berkeley [BKGeoED1; MAP 15.61%; Pooled] hagen [FUHedGYYYTD; MAP 12.80%; Pooled] 90% hildesheim [HIGeoenderun21; MAP 11.86%; Pooled] 80% 70% Average Precision 60% 50% 40% 30% 20% 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall GeoCLEF Bilingual German track Top 5 Participants − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) 1 berkeley [BKGeoED1; MAP 15.61%; Pooled] hagen [FUHedGYYYTD; MAP 12.80%; Pooled] hildesheim [HIGeoenderun21; MAP 11.86%; Pooled] 0.8 0.6 0.4 0.2 Difference 0 −0.2 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 47 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2DE-CLEF2006 GeoCLEF Bilingual German track − Box Plot of the Topics BKGeoED2 [MAP 16.82%; Pooled] BKGeoED1 [MAP 15.61%; Pooled] HIGeoenderun21n [MAP 13.15%; Pooled] FUHedGYYYTD [MAP 12.80%; Pooled] FUHedGYYYTDN [MAP 12.34%; Pooled] Experiments FUHedGNNNTD [MAP 12.11%; Pooled] HIGeoenderun21 [MAP 11.86%; Pooled] FUHedGYYYMTDN [MAP 11.48%; Pooled] HIGeoenderun22n [MAP 10.46%; Pooled] HIGeoenderun22 [MAP 9.69%; Pooled] FUHedGNNNTDN [MAP 5.48%; Pooled] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Mean Average Precision 48 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2DE-CLEF2006 GeoCLEF Bilingual German track − Tukey T test with "top group" highlighted BKGeoED2 BKGeoED1 FUHedGNNNTD HIGeoenderun21n FUHedGYYYTD Experiments HIGeoenderun21 FUHedGYYYTDN FUHedGYYYMTDN HIGeoenderun22n HIGeoenderun22 FUHedGNNNTDN 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 arcsin(sqrt(Mean average precision)) 49 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2DE-CLEF2006 GeoCLEF Bilingual German track Top 5 Participants − Retrieved documents vs Precision 100% berkeley [BKGeoED1; R−Prec 15.44%; Pooled] hagen [FUHedGYYYTD; R−Prec 11.94%; Pooled] 90% hildesheim [HIGeoenderun21; R−Prec 15.18%; Pooled] 80% 70% 60% R−Precision 50% 40% 30% 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) GeoCLEF Bilingual German track Top 5 Participants − Comparison to Median R−Precision by Topic (Topics 026 to 050) 1 berkeley [BKGeoED1; R−Prec 15.44%; Pooled] hagen [FUHedGYYYTD; R−Prec 11.94%; Pooled] hildesheim [HIGeoenderun21; R−Prec 15.18%; Pooled] 0.8 0.6 0.4 0.2 Difference 0 −0.2 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 50 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2DE-CLEF2006 GeoCLEF Bilingual German track − Box Plot of the Topics BKGeoED2 [R−Prec 18.56%; Pooled] BKGeoED1 [R−Prec 15.44%; Pooled] FUHedGNNNTD [R−Prec 15.34%; Pooled] HIGeoenderun21n [R−Prec 15.21%; Pooled] HIGeoenderun21 [R−Prec 15.18%; Pooled] Experiments FUHedGYYYTDN [R−Prec 12.45%; Pooled] FUHedGYYYTD [R−Prec 11.94%; Pooled] HIGeoenderun22n [R−Prec 11.77%; Pooled] HIGeoenderun22 [R−Prec 11.72%; Pooled] FUHedGYYYMTDN [R−Prec 11.57%; Pooled] FUHedGNNNTDN [R−Prec 6.24%; Pooled] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% R−Precision 51 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2DE-CLEF2006 GeoCLEF Bilingual German track − Tukey T test with "top group" highlighted FUHedGNNNTD BKGeoED2 HIGeoenderun21n HIGeoenderun21 BKGeoED1 Experiments HIGeoenderun22n HIGeoenderun22 FUHedGYYYTDN FUHedGYYYTD FUHedGYYYMTDN FUHedGNNNTDN 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 arcsin(sqrt(R Precision)) 52 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2DE-CLEF2006 Average Precision R-Precision Topic Minimum 1st Q. Median 3rd Q. Maximum Mean Std Minimum 1st Q. Median 3rd Q. Maximum Mean Std 026 0.0000 0.0001 0.0006 0.0022 0.1000 0.0102 0.0298 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 027 0.0008 0.0068 0.0114 0.0167 0.0271 0.0125 0.0080 0.0154 0.0308 0.0615 0.1192 0.1385 0.0713 0.0447 028 0.0030 0.0199 0.2800 0.3404 0.4485 0.2100 0.1657 0.0000 0.0625 0.3750 0.4297 0.4375 0.2614 0.1871 029 0.0061 0.0122 0.0645 0.4400 0.5040 0.2033 0.2187 0.0000 0.0000 0.0000 0.3333 0.3333 0.1515 0.1741 030 0.0183 0.0419 0.0728 0.1025 0.1984 0.0818 0.0558 0.0333 0.0667 0.1000 0.1583 0.2667 0.1182 0.0765 031 0.0051 0.0265 0.0531 0.5173 0.7984 0.2305 0.3106 0.0000 0.0033 0.0921 0.5493 0.7632 0.2428 0.3031 032 0.3107 0.4858 0.5628 0.5719 0.8326 0.5559 0.1549 0.4259 0.4907 0.5370 0.5556 0.8519 0.5673 0.1313 033 0.0000 0.0000 0.0001 0.0119 0.0272 0.0062 0.0103 0.0000 0.0000 0.0000 0.0000 0.1176 0.0107 0.0355 034 0.0000 0.0693 0.0886 0.5085 0.6819 0.2526 0.2495 0.0000 0.0588 0.1176 0.5515 0.6765 0.2754 0.2588 035 0.0008 0.0108 0.0205 0.0427 0.0575 0.0260 0.0201 0.0000 0.0000 0.0000 0.0972 0.1667 0.0505 0.0678 036 0.0000 0.0006 0.0047 0.0261 0.0660 0.0155 0.0213 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 037 0.0000 0.0014 0.0034 0.0061 0.0795 0.0113 0.0231 0.0000 0.0000 0.0000 0.0000 0.1818 0.0165 0.0548 038 0.0000 0.0000 0.0027 0.0054 0.1354 0.0150 0.0401 0.0000 0.0000 0.0000 0.0000 0.2500 0.0227 0.0754 039 0.0004 0.0136 0.0357 0.1466 0.2652 0.0781 0.0883 0.0000 0.0463 0.0741 0.1667 0.2593 0.1077 0.0819 040 0.0072 0.2165 0.2901 0.3833 0.3912 0.2860 0.1193 0.0244 0.3110 0.4146 0.4390 0.4634 0.3659 0.1277 041 0.0110 0.0273 0.0526 0.0630 0.1102 0.0525 0.0304 0.0000 0.0000 0.0000 0.0921 0.2105 0.0478 0.0684 042 0.0004 0.0027 0.0071 0.0314 0.2404 0.0377 0.0718 0.0000 0.0000 0.0213 0.0798 0.3617 0.0754 0.1171 043 0.0016 0.0026 0.0063 0.0203 0.2569 0.0333 0.0751 0.0000 0.0000 0.0000 0.0714 0.5000 0.0779 0.1480 044 0.0017 0.0044 0.0098 0.0143 0.1111 0.0187 0.0314 0.0000 0.0000 0.0000 0.0000 0.3333 0.0303 0.1005 045 0.0000 0.0000 0.0059 0.0151 0.0417 0.0107 0.0135 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 046 0.0000 0.0015 0.0136 0.0530 0.1680 0.0333 0.0503 0.0000 0.0000 0.0000 0.0000 0.2500 0.0227 0.0754 047 0.0000 0.0000 0.0000 0.0002 0.0059 0.0006 0.0018 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 048 0.3399 0.5821 0.8120 0.8631 0.9178 0.7193 0.1841 0.3623 0.5978 0.7681 0.7790 0.8841 0.6904 0.1545 049 0.0000 0.0000 0.0286 0.1405 0.1957 0.0644 0.0800 0.0000 0.0000 0.0000 0.1667 0.1667 0.0606 0.0841 050 0.0011 0.0107 0.0355 0.0470 0.0548 0.0301 0.0200 0.0000 0.0000 0.0000 0.0833 0.0833 0.0379 0.0435 ALL 0.0548 0.1072 0.1211 0.1306 0.1682 0.1198 0.0298 0.0624 0.1173 0.1245 0.1531 0.1856 0.1322 0.0322 53 54 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2EN-CLEF2006 GeoCLEF Bilingual English track Top 5 Participants − Interpolated Recall vs Average Precision 100% jaen [sinaiEsEnExp2; MAP 22.56%; Pooled] sanmarcos [SMGeoESEN2; MAP 22.46%; Pooled] 90% hildesheim [HIGeodeenrun12; MAP 16.03%; Pooled] 80% 70% Average Precision 60% 50% 40% 30% 20% 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall GeoCLEF Bilingual English track Top 5 Participants − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) 1 jaen [sinaiEsEnExp2; MAP 22.56%; Pooled] sanmarcos [SMGeoESEN2; MAP 22.46%; Pooled] hildesheim [HIGeodeenrun12; MAP 16.03%; Pooled] 0.8 0.6 0.4 0.2 Difference 0 −0.2 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 55 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2EN-CLEF2006 GeoCLEF Bilingual English track − Box Plot of the Topics sinaiEsEnExp1 [MAP 27.07%; Pooled] SMGeoESEN1 [MAP 25.52%; Pooled] sinaiEsEnExp2 [MAP 22.56%; Pooled] SMGeoESEN2 [MAP 22.46%; Pooled] sinaiEsEnExp3 [MAP 22.09%; Not Pooled] sinaiDeEnExp2 [MAP 21.64%; Not Pooled] Experiments HIGeodeenrun11n [MAP 19.03%; Not Pooled] sinaiDeEnExp1 [MAP 18.68%; Not Pooled] HIGeodeenrun12 [MAP 16.03%; Pooled] HIGeodeenrun13n [MAP 15.65%; Pooled] HIGeodeenrun11 [MAP 15.04%; Not Pooled] HIGeodeenrun13 [MAP 14.56%; Not Pooled] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Mean Average Precision 56 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2EN-CLEF2006 GeoCLEF Bilingual English track − Tukey T test with "top group" highlighted SMGeoESEN1 sinaiEsEnExp1 sinaiEsEnExp2 sinaiEsEnExp3 sinaiDeEnExp2 Experiments SMGeoESEN2 sinaiDeEnExp1 HIGeodeenrun11n HIGeodeenrun12 HIGeodeenrun13n HIGeodeenrun11 HIGeodeenrun13 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 arcsin(sqrt(Mean average precision)) 57 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2EN-CLEF2006 GeoCLEF Bilingual English track Top 5 Participants − Retrieved documents vs Precision 100% jaen [sinaiEsEnExp2; R−Prec 20.63%; Pooled] sanmarcos [SMGeoESEN2; R−Prec 23.29%; Pooled] 90% hildesheim [HIGeodeenrun12; R−Prec 17.52%; Pooled] 80% 70% 60% R−Precision 50% 40% 30% 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) GeoCLEF Bilingual English track Top 5 Participants − Comparison to Median R−Precision by Topic (Topics 026 to 050) 1 jaen [sinaiEsEnExp2; R−Prec 20.63%; Pooled] sanmarcos [SMGeoESEN2; R−Prec 23.29%; Pooled] hildesheim [HIGeodeenrun12; R−Prec 17.52%; Pooled] 0.8 0.6 0.4 0.2 Difference 0 −0.2 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 58 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2EN-CLEF2006 GeoCLEF Bilingual English track − Box Plot of the Topics SMGeoESEN1 [R−Prec 24.80%; Pooled] sinaiEsEnExp1 [R−Prec 24.27%; Pooled] SMGeoESEN2 [R−Prec 23.29%; Pooled] sinaiEsEnExp2 [R−Prec 20.63%; Pooled] sinaiEsEnExp3 [R−Prec 20.42%; Not Pooled] sinaiDeEnExp2 [R−Prec 19.55%; Not Pooled] Experiments HIGeodeenrun11n [R−Prec 19.01%; Not Pooled] HIGeodeenrun12 [R−Prec 17.52%; Pooled] sinaiDeEnExp1 [R−Prec 16.49%; Not Pooled] HIGeodeenrun11 [R−Prec 15.18%; Not Pooled] HIGeodeenrun13n [R−Prec 14.83%; Pooled] HIGeodeenrun13 [R−Prec 14.65%; Not Pooled] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% R−Precision 59 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2EN-CLEF2006 GeoCLEF Bilingual English track − Tukey T test with "top group" highlighted SMGeoESEN1 sinaiEsEnExp1 SMGeoESEN2 sinaiEsEnExp3 sinaiEsEnExp2 Experiments sinaiDeEnExp2 HIGeodeenrun11n sinaiDeEnExp1 HIGeodeenrun12 HIGeodeenrun11 HIGeodeenrun13 HIGeodeenrun13n 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 arcsin(sqrt(R Precision)) 60 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2EN-CLEF2006 Average Precision R-Precision Topic Minimum 1st Q. Median 3rd Q. Maximum Mean Std Minimum 1st Q. Median 3rd Q. Maximum Mean Std 026 0.0000 0.0090 0.0319 0.1993 0.2257 0.0831 0.0944 0.0000 0.0000 0.0556 0.2222 0.2222 0.0926 0.1042 027 0.0000 0.0000 0.0004 0.0281 0.0730 0.0166 0.0281 0.0000 0.0000 0.0000 0.0526 0.1053 0.0263 0.0420 028 0.0000 0.0008 0.1172 0.2284 0.3183 0.1204 0.1196 0.0000 0.0000 0.1316 0.2632 0.3684 0.1404 0.1425 029 0.0347 0.0486 0.0886 0.1245 0.2108 0.0948 0.0533 0.1111 0.1111 0.1111 0.2222 0.2222 0.1574 0.0572 030 0.0349 0.3559 0.6747 0.9623 1.0000 0.6142 0.3575 0.0000 0.4167 0.6667 0.8333 1.0000 0.5972 0.3368 031 0.1256 0.1606 0.1903 0.2383 0.4912 0.2272 0.1119 0.0678 0.1525 0.2203 0.3051 0.4746 0.2387 0.1208 032 0.4530 0.5124 0.8519 0.9554 0.9713 0.7503 0.2249 0.5806 0.5968 0.7742 0.8871 0.9355 0.7527 0.1445 033 0.0000 0.0000 0.0000 0.0035 0.0552 0.0065 0.0158 0.0000 0.0000 0.0000 0.0000 0.1500 0.0167 0.0444 034 0.0000 0.0883 0.2868 0.4205 0.4514 0.2609 0.1705 0.0000 0.0000 0.3333 0.3333 0.6667 0.2778 0.2392 035 0.0000 0.0005 0.0107 0.0229 0.0390 0.0127 0.0130 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 036 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 037 0.0000 0.0003 0.0015 0.0557 0.0966 0.0260 0.0425 0.0000 0.0000 0.0000 0.0938 0.1250 0.0365 0.0563 038 0.0000 0.0144 0.0577 0.1625 1.0000 0.1769 0.2945 0.0000 0.0000 0.0000 0.0000 1.0000 0.0833 0.2887 039 0.0046 0.0071 0.0494 0.1246 0.3686 0.0800 0.1070 0.0000 0.0000 0.0000 0.0938 0.3750 0.0573 0.1114 040 0.2523 0.2680 0.3227 0.7960 0.8837 0.5001 0.2738 0.1429 0.2143 0.3214 0.7143 0.7857 0.4345 0.2663 041 0.0000 0.0004 0.0027 0.0116 0.0225 0.0068 0.0084 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 042 0.0000 0.0009 0.0349 0.2763 0.5833 0.1352 0.1919 0.0000 0.0000 0.0000 0.2500 0.5000 0.1250 0.2261 043 0.0000 0.0001 0.0063 0.0194 0.0349 0.0102 0.0119 0.0000 0.0000 0.0000 0.0000 0.1250 0.0208 0.0487 044 0.0063 0.0300 0.0763 0.1244 0.2248 0.0858 0.0713 0.0000 0.0658 0.1053 0.1579 0.3421 0.1272 0.1015 045 0.0023 0.0032 0.0219 0.1364 0.3550 0.0814 0.1113 0.0000 0.0000 0.0000 0.1667 0.3333 0.0694 0.1114 046 0.0591 0.3195 0.3990 0.5678 0.9167 0.4226 0.2447 0.0000 0.3333 0.3333 0.6667 0.6667 0.3889 0.2392 047 0.0001 0.0204 0.0317 0.0423 0.0571 0.0305 0.0182 0.0000 0.0000 0.0417 0.0833 0.0833 0.0382 0.0375 048 0.6689 0.7143 0.8082 0.9132 0.9219 0.8110 0.1018 0.6250 0.6875 0.7396 0.8438 0.8542 0.7552 0.0871 049 0.0052 0.0595 0.2538 0.3667 0.6429 0.2625 0.2241 0.0000 0.0000 0.0000 0.2500 0.5000 0.1250 0.2261 050 0.0638 0.1723 0.1966 0.2315 0.2343 0.1910 0.0500 0.1333 0.2000 0.2667 0.2667 0.3333 0.2444 0.0519 ALL 0.1456 0.1584 0.2033 0.2251 0.2707 0.2003 0.0418 0.1465 0.1584 0.1928 0.2196 0.2480 0.1922 0.0361 61 62 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2ES-CLEF2006 GeoCLEF Bilingual Spanish track Top 5 Participants − Interpolated Recall vs Average Precision 100% berkeley [BKGeoES1; MAP 25.71%; Pooled] sanmarcos [SMGeoENES1; MAP 12.82%; Pooled] 90% 80% 70% Average Precision 60% 50% 40% 30% 20% 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall GeoCLEF Bilingual Spanish track Top 5 Participants − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) 1 berkeley [BKGeoES1; MAP 25.71%; Pooled] sanmarcos [SMGeoENES1; MAP 12.82%; Pooled] 0.8 0.6 0.4 0.2 Difference 0 −0.2 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 63 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2ES-CLEF2006 GeoCLEF Bilingual Spanish track − Box Plot of the Topics BKGeoES2 [MAP 27.45%; Pooled] BKGeoES1 [MAP 25.71%; Pooled] Experiments SMGeoENES1 [MAP 12.82%; Pooled] SMGeoPTES3 [MAP 11.50%; Pooled] SMGeoPTES2 [MAP 10.89%; Pooled] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Mean Average Precision 64 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2ES-CLEF2006 GeoCLEF Bilingual Spanish track − Tukey T test with "top group" highlighted BKGeoES2 BKGeoES1 Experiments SMGeoENES1 SMGeoPTES3 SMGeoPTES2 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 arcsin(sqrt(Mean average precision)) 65 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2ES-CLEF2006 GeoCLEF Bilingual Spanish track Top 5 Participants − Retrieved documents vs Precision 100% berkeley [BKGeoES1; R−Prec 26.45%; Pooled] sanmarcos [SMGeoENES1; R−Prec 16.89%; Pooled] 90% 80% 70% 60% R−Precision 50% 40% 30% 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) GeoCLEF Bilingual Spanish track Top 5 Participants − Comparison to Median R−Precision by Topic (Topics 026 to 050) 1 berkeley [BKGeoES1; R−Prec 26.45%; Pooled] sanmarcos [SMGeoENES1; R−Prec 16.89%; Pooled] 0.8 0.6 0.4 0.2 Difference 0 −0.2 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 66 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2ES-CLEF2006 GeoCLEF Bilingual Spanish track − Box Plot of the Topics BKGeoES2 [R−Prec 27.04%; Pooled] BKGeoES1 [R−Prec 26.45%; Pooled] Experiments SMGeoENES1 [R−Prec 16.89%; Pooled] SMGeoPTES3 [R−Prec 15.27%; Pooled] SMGeoPTES2 [R−Prec 14.67%; Pooled] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% R−Precision 67 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2ES-CLEF2006 GeoCLEF Bilingual Spanish track − Tukey T test with "top group" highlighted BKGeoES2 BKGeoES1 Experiments SMGeoENES1 SMGeoPTES2 SMGeoPTES3 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 arcsin(sqrt(R Precision)) 68 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2ES-CLEF2006 Average Precision R-Precision Topic Minimum 1st Q. Median 3rd Q. Maximum Mean Std Minimum 1st Q. Median 3rd Q. Maximum Mean Std 026 0.0001 0.0009 0.0172 0.0186 0.0212 0.0115 0.0100 0.0000 0.0000 0.0000 0.0556 0.0556 0.0222 0.0304 027 0.0084 0.0174 0.0267 0.1205 0.3867 0.0948 0.1634 0.0000 0.0192 0.1282 0.1987 0.4103 0.1385 0.1628 028 0.0713 0.0784 0.1067 0.1477 0.2455 0.1239 0.0703 0.0556 0.1181 0.1389 0.1944 0.2778 0.1556 0.0800 029 0.0098 0.1970 0.2854 0.3630 0.4752 0.2711 0.1683 0.0303 0.2348 0.3030 0.3636 0.5455 0.2970 0.1823 030 0.0000 0.0000 0.0028 0.0350 0.1311 0.0274 0.0580 0.0000 0.0000 0.0265 0.0646 0.1788 0.0464 0.0752 031 0.0087 0.0098 0.0143 0.6322 0.6565 0.2628 0.3449 0.0510 0.0510 0.0667 0.6216 0.6745 0.2894 0.3204 032 0.1971 0.1980 0.2001 0.9751 0.9782 0.5096 0.4259 0.2077 0.2077 0.2077 0.9019 0.9077 0.4862 0.3813 033 0.0001 0.0005 0.0009 0.0168 0.0212 0.0076 0.0099 0.0000 0.0075 0.0100 0.0325 0.0400 0.0180 0.0164 034 0.0983 0.1267 0.1587 0.2176 0.2316 0.1676 0.0548 0.1351 0.1351 0.1892 0.3243 0.4054 0.2324 0.1172 035 0.0071 0.0170 0.0299 0.0549 0.0991 0.0393 0.0356 0.0000 0.0395 0.0526 0.0789 0.1579 0.0632 0.0577 036 0.0002 0.0051 0.0072 0.0497 0.1102 0.0308 0.0458 0.0000 0.0273 0.0364 0.1045 0.2273 0.0727 0.0893 037 0.0000 0.0004 0.0005 0.0757 0.1145 0.0357 0.0517 0.0000 0.0000 0.0000 0.0431 0.1724 0.0345 0.0771 038 0.0000 0.0000 0.0000 0.0236 0.0663 0.0151 0.0289 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 039 0.0098 0.1278 0.2701 0.3376 0.5358 0.2509 0.1918 0.0299 0.1866 0.2388 0.3172 0.5075 0.2537 0.1695 040 0.5335 0.5367 0.5413 0.5919 0.6760 0.5705 0.0601 0.6043 0.6043 0.6115 0.6349 0.6835 0.6245 0.0335 041 0.0086 0.0096 0.0100 0.1766 0.2243 0.0827 0.1027 0.0667 0.0867 0.0933 0.2267 0.3467 0.1573 0.1152 042 0.2006 0.2867 0.3164 0.3865 0.4796 0.3335 0.1001 0.2830 0.3538 0.3774 0.4245 0.5094 0.3887 0.0807 043 0.0000 0.0002 0.0003 0.0089 0.0318 0.0068 0.0140 0.0000 0.0000 0.0000 0.0104 0.0417 0.0083 0.0186 044 0.0283 0.0485 0.0584 0.2091 0.2954 0.1235 0.1126 0.0777 0.1286 0.1553 0.3058 0.3786 0.2078 0.1206 045 0.0138 0.0196 0.0221 0.0314 0.0567 0.0274 0.0168 0.0000 0.0000 0.0000 0.1042 0.1667 0.0500 0.0745 046 0.0519 0.0562 0.1107 0.6201 0.6369 0.2943 0.3035 0.0714 0.0714 0.1429 0.5536 0.6071 0.2857 0.2637 047 0.0097 0.0262 0.0337 0.1236 0.2860 0.0861 0.1138 0.0169 0.0297 0.0339 0.1737 0.3390 0.1085 0.1348 048 0.0565 0.0567 0.0581 0.7562 0.8280 0.3463 0.3975 0.0755 0.0755 0.0755 0.6575 0.7396 0.3192 0.3360 049 0.4566 0.4982 0.5745 0.7545 0.7874 0.6148 0.1445 0.5576 0.5853 0.6682 0.6982 0.7051 0.6442 0.0650 050 0.0401 0.0491 0.0528 0.1212 0.1796 0.0853 0.0578 0.0400 0.0700 0.1000 0.1450 0.2200 0.1120 0.0672 ALL 0.1089 0.1135 0.1282 0.2615 0.2745 0.1768 0.0818 0.1467 0.1512 0.1689 0.2660 0.2704 0.2006 0.0615 69 70 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2PT-CLEF2006 GeoCLEF Bilingual Portuguese track Top 5 Participants − Interpolated Recall vs Average Precision 100% sanmarcos [SMGeoESPT2; MAP 14.16%; Pooled] berkeley [BKGeoEP1; MAP 12.60%; Pooled] 90% 80% 70% Average Precision 60% 50% 40% 30% 20% 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall GeoCLEF Bilingual Portuguese track Top 5 Participants − Comparison to Median Mean Average Precision by Topic (Topics 026 to 5 00) 1 sanmarcos [SMGeoESPT2; MAP 14.16%; Pooled] berkeley [BKGeoEP1; MAP 12.60%; Pooled] 0.8 0.6 0.4 0.2 Difference 0 −0.2 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 71 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2PT-CLEF2006 GeoCLEF Bilingual Portuguese track − Box Plot of the Topics BKGeoEP2 [MAP 14.30%; Pooled] SMGeoESPT2 [MAP 14.16%; Pooled] Experiments SMGeoESPT1 [MAP 12.81%; Pooled] BKGeoEP1 [MAP 12.60%; Pooled] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Mean Average Precision 72 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2PT-CLEF2006 GeoCLEF Bilingual Portuguese track − Tukey T test with "top group" highlighted SMGeoESPT2 SMGeoESPT1 Experiments BKGeoEP2 BKGeoEP1 0.2 0.22 0.24 0.26 0.28 0.3 0.32 0.34 0.36 0.38 0.4 arcsin(sqrt(Mean average precision)) 73 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2PT-CLEF2006 GeoCLEF Bilingual Portuguese track Top 5 Participants − Retrieved documents vs Precision 100% sanmarcos [SMGeoESPT2; R−Prec 17.42%; Pooled] berkeley [BKGeoEP1; R−Prec 14.77%; Pooled] 90% 80% 70% 60% R−Precision 50% 40% 30% 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) GeoCLEF Bilingual Portuguese track Top 5 Participants − Comparison to Median R−Precision by Topic (Topics 026 to 050) 1 sanmarcos [SMGeoESPT2; R−Prec 17.42%; Pooled] berkeley [BKGeoEP1; R−Prec 14.77%; Pooled] 0.8 0.6 0.4 0.2 Difference 0 −0.2 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 74 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2PT-CLEF2006 GeoCLEF Bilingual Portuguese track − Box Plot of the Topics SMGeoESPT2 [R−Prec 17.42%; Pooled] BKGeoEP2 [R−Prec 16.34%; Pooled] Experiments SMGeoESPT1 [R−Prec 14.88%; Pooled] BKGeoEP1 [R−Prec 14.77%; Pooled] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% R−Precision 75 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2PT-CLEF2006 GeoCLEF Bilingual Portuguese track − Tukey T test with "top group" highlighted SMGeoESPT2 SMGeoESPT1 Experiments BKGeoEP2 BKGeoEP1 0.2 0.25 0.3 0.35 0.4 0.45 arcsin(sqrt(R Precision)) 76 GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2PT-CLEF2006 Average Precision R-Precision Topic Minimum 1st Q. Median 3rd Q. Maximum Mean Std Minimum 1st Q. Median 3rd Q. Maximum Mean Std 026 0.0117 0.0120 0.0125 0.0453 0.0779 0.0286 0.0328 0.0000 0.0000 0.0000 0.0333 0.0667 0.0167 0.0333 027 0.0003 0.0009 0.0357 0.0797 0.0897 0.0403 0.0462 0.0000 0.0196 0.0833 0.1667 0.2059 0.0931 0.0921 028 0.0854 0.1041 0.1271 0.1860 0.2405 0.1450 0.0667 0.1562 0.1875 0.2188 0.2812 0.3438 0.2344 0.0786 029 0.0686 0.1525 0.2534 0.3019 0.3334 0.2272 0.1131 0.0769 0.1923 0.3205 0.3590 0.3846 0.2756 0.1363 030 0.0069 0.0075 0.0122 0.2015 0.3866 0.1045 0.1881 0.0000 0.0000 0.0357 0.2500 0.4286 0.1250 0.2052 031 0.1438 0.1526 0.1887 0.3268 0.4374 0.2397 0.1354 0.1311 0.1475 0.1885 0.3033 0.3934 0.2254 0.1170 032 0.0776 0.1098 0.3201 0.6281 0.7579 0.3689 0.3186 0.2075 0.2358 0.4245 0.6698 0.7547 0.4528 0.2610 033 0.0000 0.0000 0.0009 0.0067 0.0116 0.0033 0.0056 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 034 0.0014 0.0015 0.0120 0.0272 0.0321 0.0143 0.0154 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 035 0.0014 0.0035 0.0151 0.0262 0.0276 0.0148 0.0133 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 036 0.0000 0.0000 0.0096 0.0240 0.0288 0.0120 0.0144 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 037 0.0007 0.0011 0.0072 0.0150 0.0172 0.0081 0.0082 0.0000 0.0000 0.0000 0.0278 0.0556 0.0139 0.0278 038 0.1435 0.1530 0.1745 0.2437 0.3010 0.1984 0.0707 0.1250 0.1250 0.1875 0.2500 0.2500 0.1875 0.0722 039 0.0545 0.0685 0.0861 0.1963 0.3028 0.1324 0.1146 0.0870 0.1087 0.1304 0.1957 0.2609 0.1522 0.0753 040 0.0001 0.0001 0.0071 0.0213 0.0284 0.0107 0.0135 0.0000 0.0000 0.0208 0.0833 0.1250 0.0417 0.0589 041 0.0000 0.0001 0.0057 0.0113 0.0115 0.0057 0.0065 0.0000 0.0000 0.0144 0.0288 0.0288 0.0144 0.0167 042 0.1065 0.1826 0.3486 0.4673 0.4961 0.3250 0.1773 0.2000 0.2571 0.3714 0.4571 0.4857 0.3571 0.1267 043 0.0002 0.0007 0.0129 0.0436 0.0626 0.0221 0.0292 0.0000 0.0000 0.0208 0.0625 0.0833 0.0313 0.0399 044 0.0000 0.0000 0.0387 0.0786 0.0797 0.0393 0.0454 0.0000 0.0000 0.0461 0.0987 0.1053 0.0493 0.0572 045 0.0295 0.1339 0.2418 0.2596 0.2739 0.1967 0.1125 0.0854 0.2195 0.3537 0.3598 0.3659 0.2896 0.1363 046 0.0922 0.1758 0.2996 0.4281 0.5165 0.3020 0.1763 0.1667 0.2500 0.3561 0.4621 0.5455 0.3561 0.1557 047 0.0013 0.0155 0.0419 0.0713 0.0884 0.0434 0.0370 0.0000 0.0000 0.0294 0.0735 0.0882 0.0368 0.0441 048 0.0229 0.2711 0.6083 0.7453 0.7932 0.5082 0.3428 0.0979 0.3147 0.5874 0.6888 0.7343 0.5017 0.2817 049 0.0880 0.1145 0.1924 0.3217 0.3997 0.2181 0.1372 0.0556 0.1667 0.2778 0.3611 0.4444 0.2639 0.1596 050 0.1034 0.1173 0.1441 0.1986 0.2403 0.1580 0.0591 0.1591 0.1705 0.2273 0.3182 0.3636 0.2443 0.0935 ALL 0.1260 0.1271 0.1348 0.1423 0.1430 0.1347 0.0088 0.1477 0.1482 0.1561 0.1688 0.1742 0.1585 0.0127 77 78 Individual Experiment Results and Graphs 79 80 berkeley BKGeoD2 GC-MONO-DE-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language German Relevant 602 Topic Fields title, description, narrative Relevant retrieved 490 Pooled true Geometric Mean Average Precision 0.0434 German topics TDN with blind feedback Binary Preference (BPREF) 0.1665 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision 100% 0 34.92 BKGeoD2 10 32.21 90% 20 28.25 80% 30 22.43 40 18.05 70% 50 17.57 Average Precision 60% 60 17.00 70 15.88 50% 80 13.29 40% 90 9.95 30% 100 2.16 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 18.22 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.9069 Minimum 0.0000 First Quartile 0.0160 Second Quartile 0.0682 Third Quartile 0.2145 Interquartile range 0.1986 Mean 0.1822 Standard Deviation 0.2623 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4016 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0965 Std With No Outliers 0.1178 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries BKGeoD2 Topic 026 0.36 Topic 039 16.21 0.8 Topic 027 2.34 Topic 040 40.16 Topic 028 12.02 Topic 041 18.63 0.6 Topic 029 34.86 Topic 042 12.73 Topic 030 78.49 Topic 043 3.34 Topic 031 6.82 Topic 044 0.23 0.4 Topic 032 73.90 Topic 045 4.61 Topic 033 0.37 Topic 046 11.88 0.2 Topic 034 29.92 Topic 047 8.61 Difference Topic 035 1.76 Topic 048 90.69 0 Topic 036 0.80 Topic 049 3.74 Topic 037 0.00 Topic 050 1.11 −0.2 Topic 038 1.85 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 81 berkeley BKGeoD2 GC-MONO-DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual German track − Retrieved documents vs Precision 100% 5 docs 21.60 BKGeoD2 10 docs 20.00 90% 15 docs 20.53 80% 20 docs 19.40 30 docs 17.60 70% 100 docs 11.60 60% 200 docs 6.66 R−Precision 500 docs 3.33 50% 1000 docs 1.96 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 18.57 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.8841 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0556 Third Quartile 0.2500 Interquartile range 0.2500 Mean 0.1857 Standard Deviation 0.2650 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4390 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1014 Std With No Outliers 0.1337 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries BKGeoD2 Topic 026 0.00 Topic 039 11.11 0.8 Topic 027 10.77 Topic 040 43.90 Topic 028 25.00 Topic 041 21.05 0.6 Topic 029 33.33 Topic 042 21.28 Topic 030 76.67 Topic 043 0.00 Topic 031 2.63 Topic 044 0.00 0.4 Topic 032 75.93 Topic 045 0.00 Topic 033 0.00 Topic 046 25.00 0.2 Topic 034 23.53 Topic 047 0.00 Difference Topic 035 5.56 Topic 048 88.41 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 82 berkeley BKGeoD1 GC-MONO-DE-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language German Relevant 602 Topic Fields title, description Relevant retrieved 435 Pooled true Geometric Mean Average Precision 0.0466 German automatic TD with blind feedback Binary Preference (BPREF) 0.1936 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision 100% 0 44.24 BKGeoD1 10 34.84 90% 20 33.46 80% 30 27.55 40 22.05 70% 50 21.30 Average Precision 60% 60 18.54 70 16.72 50% 80 14.63 40% 90 12.13 30% 100 4.05 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 21.51 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.8944 Minimum 0.0001 First Quartile 0.0128 Second Quartile 0.0610 Third Quartile 0.3398 Interquartile range 0.3270 Mean 0.2151 Standard Deviation 0.2827 Lower Outlier Threshold 0.0001 Upper Outlier Threshold 0.7862 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1868 Std With No Outliers 0.2500 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries BKGeoD1 Topic 026 0.08 Topic 039 0.04 0.8 Topic 027 1.37 Topic 040 33.63 Topic 028 39.16 Topic 041 21.84 0.6 Topic 029 35.02 Topic 042 1.68 Topic 030 78.62 Topic 043 0.93 Topic 031 2.13 Topic 044 0.50 0.4 Topic 032 78.61 Topic 045 2.69 Topic 033 8.34 Topic 046 27.94 0.2 Topic 034 67.04 Topic 047 6.10 Difference Topic 035 12.31 Topic 048 89.44 0 Topic 036 1.03 Topic 049 5.17 Topic 037 0.01 Topic 050 3.33 −0.2 Topic 038 20.81 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 83 berkeley BKGeoD1 GC-MONO-DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual German track − Retrieved documents vs Precision 100% 5 docs 25.60 BKGeoD1 10 docs 24.40 90% 15 docs 22.13 80% 20 docs 20.80 30 docs 19.07 70% 100 docs 11.36 60% 200 docs 6.52 R−Precision 500 docs 3.09 50% 1000 docs 1.74 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 19.99 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.8696 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0462 Third Quartile 0.3476 Interquartile range 0.3476 Mean 0.1999 Standard Deviation 0.2812 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8148 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1720 Std With No Outliers 0.2494 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries BKGeoD1 Topic 026 0.00 Topic 039 0.00 0.8 Topic 027 4.62 Topic 040 39.02 Topic 028 43.75 Topic 041 21.05 0.6 Topic 029 33.33 Topic 042 0.00 Topic 030 70.00 Topic 043 0.00 Topic 031 1.32 Topic 044 0.00 0.4 Topic 032 81.48 Topic 045 0.00 Topic 033 17.65 Topic 046 25.00 0.2 Topic 034 61.76 Topic 047 0.00 Difference Topic 035 5.56 Topic 048 86.96 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 8.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 84 daedalus GCdeNtLg GC-MONO-DE-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 13,055 Source Language German Relevant 523 Topic Fields title, description Relevant retrieved 256 Pooled true Geometric Mean Average Precision 0.0152 Normal text Left geo run Binary Preference (BPREF) 0.0876 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision 100% 0 34.44 GCdeNtLg 10 20.62 90% 20 18.05 80% 30 12.74 40 11.81 70% 50 9.92 Average Precision 60% 60 5.87 70 4.39 50% 80 3.59 40% 90 2.11 30% 100 2.07 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 10.01 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.5258 Minimum 0.0000 First Quartile 0.0150 Second Quartile 0.0411 Third Quartile 0.1501 Interquartile range 0.1351 Mean 0.1001 Standard Deviation 0.1364 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.2404 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0667 Std With No Outliers 0.0748 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries GCdeNtLg Topic 026 0.00 Topic 039 1.73 0.8 Topic 027 0.00 Topic 040 13.83 Topic 028 24.04 Topic 041 2.14 0.6 Topic 029 4.17 Topic 042 4.11 Topic 030 44.21 Topic 043 0.55 Topic 031 3.80 Topic 044 4.96 0.4 Topic 032 1.85 Topic 045 0.00 Topic 033 2.96 Topic 046 18.29 0.2 Topic 034 52.58 Topic 047 3.16 Difference Topic 035 6.74 Topic 048 18.82 0 Topic 036 0.79 Topic 049 7.86 Topic 037 19.80 Topic 050 0.00 −0.2 Topic 038 13.91 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 85 daedalus GCdeNtLg GC-MONO-DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual German track − Retrieved documents vs Precision 100% 5 docs 13.60 GCdeNtLg 10 docs 15.20 90% 15 docs 15.20 80% 20 docs 15.40 30 docs 13.20 70% 100 docs 6.60 60% 200 docs 4.10 R−Precision 500 docs 1.91 50% 1000 docs 1.02 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 11.53 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.5588 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0395 Third Quartile 0.1866 Interquartile range 0.1866 Mean 0.1153 Standard Deviation 0.1621 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3125 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0778 Std With No Outliers 0.1016 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries GCdeNtLg Topic 026 0.00 Topic 039 3.70 0.8 Topic 027 0.00 Topic 040 26.83 Topic 028 31.25 Topic 041 10.53 0.6 Topic 029 0.00 Topic 042 8.51 Topic 030 53.33 Topic 043 0.00 Topic 031 3.95 Topic 044 16.67 0.4 Topic 032 1.85 Topic 045 0.00 Topic 033 5.88 Topic 046 25.00 0.2 Topic 034 55.88 Topic 047 0.00 Difference Topic 035 11.11 Topic 048 24.64 0 Topic 036 0.00 Topic 049 0.00 Topic 037 9.09 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 86 daedalus GCdeAA GC-MONO-DE-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction MANUAL Retrieved 793 Source Language German Relevant 535 Topic Fields title, description, narrative Relevant retrieved 80 Pooled true Geometric Mean Average Precision 0.0026 All text And geo run Binary Preference (BPREF) 0.0734 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision 100% 0 32.83 GCdeAA 10 24.61 90% 20 9.71 80% 30 9.01 40 6.13 70% 50 5.08 Average Precision 60% 60 3.79 70 0.19 50% 80 0.19 40% 90 0.19 30% 100 0.19 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 7.15 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.5862 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0323 Third Quartile 0.1003 Interquartile range 0.1003 Mean 0.0715 Standard Deviation 0.1222 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1850 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0501 Std With No Outliers 0.0600 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries GCdeAA Topic 026 0.00 Topic 039 11.55 0.8 Topic 027 0.00 Topic 040 0.00 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 16.67 Topic 042 7.97 Topic 030 58.62 Topic 043 3.23 Topic 031 3.20 Topic 044 0.54 0.4 Topic 032 0.23 Topic 045 9.52 Topic 033 11.76 Topic 046 0.00 0.2 Topic 034 0.00 Topic 047 3.33 Difference Topic 035 0.00 Topic 048 18.50 0 Topic 036 6.67 Topic 049 0.00 Topic 037 15.15 Topic 050 4.14 −0.2 Topic 038 7.67 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 87 daedalus GCdeAA GC-MONO-DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual German track − Retrieved documents vs Precision 100% 5 docs 16.80 GCdeAA 10 docs 16.80 90% 15 docs 14.40 80% 20 docs 12.20 30 docs 8.53 70% 100 docs 3.16 60% 200 docs 1.60 R−Precision 500 docs 0.64 50% 1000 docs 0.32 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 8.93 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.6000 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.1534 Interquartile range 0.1534 Mean 0.0893 Standard Deviation 0.1400 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0680 Std With No Outliers 0.0930 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries GCdeAA Topic 026 0.00 Topic 039 18.52 0.8 Topic 027 0.00 Topic 040 0.00 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 33.33 Topic 042 14.89 Topic 030 60.00 Topic 043 14.29 Topic 031 6.58 Topic 044 0.00 0.4 Topic 032 1.85 Topic 045 0.00 Topic 033 11.76 Topic 046 0.00 0.2 Topic 034 0.00 Topic 047 16.67 Difference Topic 035 0.00 Topic 048 18.84 0 Topic 036 0.00 Topic 049 0.00 Topic 037 18.18 Topic 050 8.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 88 daedalus GCdeAtLg GC-MONO-DE-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction MANUAL Retrieved 20,707 Source Language German Relevant 535 Topic Fields title, description, narrative Relevant retrieved 208 Pooled true Geometric Mean Average Precision 0.0122 All text Left geo run Binary Preference (BPREF) 0.0657 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision 100% 0 30.53 GCdeAtLg 10 17.52 90% 20 10.02 80% 30 9.06 40 8.46 70% 50 7.83 Average Precision 60% 60 6.07 70 2.25 50% 80 1.43 40% 90 1.11 30% 100 0.52 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 7.36 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.5909 Minimum 0.0000 First Quartile 0.0030 Second Quartile 0.0306 Third Quartile 0.0838 Interquartile range 0.0809 Mean 0.0736 Standard Deviation 0.1227 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1658 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0443 Std With No Outliers 0.0473 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries GCdeAtLg Topic 026 0.00 Topic 039 6.04 0.8 Topic 027 0.00 Topic 040 16.58 Topic 028 14.08 Topic 041 2.14 0.6 Topic 029 0.11 Topic 042 8.66 Topic 030 59.09 Topic 043 3.06 Topic 031 2.30 Topic 044 1.06 0.4 Topic 032 1.85 Topic 045 7.92 Topic 033 0.32 Topic 046 0.00 0.2 Topic 034 1.53 Topic 047 6.33 Difference Topic 035 0.21 Topic 048 9.50 0 Topic 036 0.21 Topic 049 4.90 Topic 037 23.03 Topic 050 6.73 −0.2 Topic 038 8.29 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 89 daedalus GCdeAtLg GC-MONO-DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual German track − Retrieved documents vs Precision 100% 5 docs 16.80 GCdeAtLg 10 docs 12.40 90% 15 docs 11.73 80% 20 docs 11.60 30 docs 9.47 70% 100 docs 5.00 60% 200 docs 3.36 R−Precision 500 docs 1.56 50% 1000 docs 0.83 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 8.78 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.6000 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0185 Third Quartile 0.1516 Interquartile range 0.1516 Mean 0.0878 Standard Deviation 0.1332 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.2500 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0664 Std With No Outliers 0.0815 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries GCdeAtLg Topic 026 0.00 Topic 039 14.81 0.8 Topic 027 0.00 Topic 040 19.51 Topic 028 25.00 Topic 041 10.53 0.6 Topic 029 0.00 Topic 042 14.89 Topic 030 60.00 Topic 043 7.14 Topic 031 6.58 Topic 044 0.00 0.4 Topic 032 1.85 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 0.00 Topic 047 16.67 Difference Topic 035 0.00 Topic 048 15.94 0 Topic 036 0.00 Topic 049 0.00 Topic 037 18.18 Topic 050 8.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 90 daedalus GCdeNA GC-MONO-DE-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction MANUAL Retrieved 4,128 Source Language German Relevant 523 Topic Fields title, description Relevant retrieved 152 Pooled true Geometric Mean Average Precision 0.0032 run mandatory Binary Preference (BPREF) 0.0851 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision 100% 0 34.22 GCdeNA 10 24.65 90% 20 17.18 80% 30 10.70 40 8.81 70% 50 8.53 Average Precision 60% 60 4.58 70 3.81 50% 80 2.07 40% 90 1.80 30% 100 1.76 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 9.28 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.5258 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0106 Third Quartile 0.1438 Interquartile range 0.1438 Mean 0.0928 Standard Deviation 0.1442 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.2392 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0590 Std With No Outliers 0.0870 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries GCdeNA Topic 026 0.00 Topic 039 0.93 0.8 Topic 027 0.00 Topic 040 0.41 Topic 028 23.20 Topic 041 0.00 0.6 Topic 029 3.33 Topic 042 1.06 Topic 030 43.75 Topic 043 0.16 Topic 031 4.83 Topic 044 20.00 0.4 Topic 032 0.31 Topic 045 0.00 Topic 033 11.76 Topic 046 12.50 0.2 Topic 034 52.58 Topic 047 0.00 Difference Topic 035 7.20 Topic 048 23.92 0 Topic 036 0.00 Topic 049 0.00 Topic 037 2.27 Topic 050 0.00 −0.2 Topic 038 23.81 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 91 daedalus GCdeNA GC-MONO-DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual German track − Retrieved documents vs Precision 100% 5 docs 16.80 GCdeNA 10 docs 16.80 90% 15 docs 14.93 80% 20 docs 13.60 30 docs 10.80 70% 100 docs 4.52 60% 200 docs 2.78 R−Precision 500 docs 1.21 50% 1000 docs 0.61 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 10.19 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.5588 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0213 Third Quartile 0.1299 Interquartile range 0.1299 Mean 0.1019 Standard Deviation 0.1657 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.2500 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0491 Std With No Outliers 0.0783 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries GCdeNA Topic 026 0.00 Topic 039 3.70 0.8 Topic 027 0.00 Topic 040 2.44 Topic 028 37.50 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 2.13 Topic 030 53.33 Topic 043 0.00 Topic 031 5.26 Topic 044 16.67 0.4 Topic 032 1.85 Topic 045 0.00 Topic 033 11.76 Topic 046 25.00 0.2 Topic 034 55.88 Topic 047 0.00 Difference Topic 035 5.56 Topic 048 24.64 0 Topic 036 0.00 Topic 049 0.00 Topic 037 9.09 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 92 daedalus GCdeAO GC-MONO-DE-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction MANUAL Retrieved 24,328 Source Language German Relevant 602 Topic Fields title, description, narrative Relevant retrieved 259 Pooled true Geometric Mean Average Precision 0.0125 All text Or geo run Binary Preference (BPREF) 0.0442 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision 100% 0 23.17 GCdeAO 10 11.94 90% 20 8.26 80% 30 7.21 40 6.02 70% 50 5.35 Average Precision 60% 60 3.78 70 3.26 50% 80 2.67 40% 90 1.59 30% 100 0.40 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 5.48 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.3825 Minimum 0.0000 First Quartile 0.0055 Second Quartile 0.0238 Third Quartile 0.0703 Interquartile range 0.0647 Mean 0.0548 Standard Deviation 0.0809 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1658 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0411 Std With No Outliers 0.0443 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries GCdeAO Topic 026 1.35 Topic 039 6.04 0.8 Topic 027 1.65 Topic 040 16.58 Topic 028 0.00 Topic 041 1.12 0.6 Topic 029 0.11 Topic 042 5.22 Topic 030 12.56 Topic 043 3.06 Topic 031 2.30 Topic 044 0.66 0.4 Topic 032 38.25 Topic 045 7.92 Topic 033 2.38 Topic 046 0.00 0.2 Topic 034 1.53 Topic 047 6.33 Difference Topic 035 0.06 Topic 048 9.50 0 Topic 036 0.13 Topic 049 4.90 Topic 037 0.23 Topic 050 6.73 −0.2 Topic 038 8.29 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 93 daedalus GCdeAO GC-MONO-DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual German track − Retrieved documents vs Precision 100% 5 docs 9.60 GCdeAO 10 docs 7.60 90% 15 docs 6.40 80% 20 docs 6.80 30 docs 7.33 70% 100 docs 6.04 60% 200 docs 4.10 R−Precision 500 docs 1.92 50% 1000 docs 1.04 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 6.62 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.4630 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.1096 Interquartile range 0.1096 Mean 0.0662 Standard Deviation 0.1056 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1951 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0497 Std With No Outliers 0.0671 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries GCdeAO Topic 026 0.00 Topic 039 14.81 0.8 Topic 027 13.85 Topic 040 19.51 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 6.38 Topic 030 10.00 Topic 043 7.14 Topic 031 6.58 Topic 044 0.00 0.4 Topic 032 46.30 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 0.00 Topic 047 16.67 Difference Topic 035 0.00 Topic 048 15.94 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 8.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 94 hagen FUHddGYYYTD GC-MONO-DE-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 24,118 Source Language German Relevant 602 Topic Fields title, description Relevant retrieved 449 Pooled true Geometric Mean Average Precision 0.0413 second run Binary Preference (BPREF) 0.2011 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision 100% 0 41.56 FUHddGYYYTD 10 37.05 90% 20 34.25 80% 30 30.28 40 25.63 70% 50 24.85 Average Precision 60% 60 18.58 70 14.71 50% 80 12.24 40% 90 9.42 30% 100 1.79 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 22.29 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.9085 Minimum 0.0000 First Quartile 0.0095 Second Quartile 0.0307 Third Quartile 0.4244 Interquartile range 0.4149 Mean 0.2229 Standard Deviation 0.2992 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.9085 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2229 Std With No Outliers 0.2992 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries FUHddGYYYTD Topic 026 0.12 Topic 039 7.76 0.8 Topic 027 1.14 Topic 040 32.59 Topic 028 0.78 Topic 041 6.26 0.6 Topic 029 1.93 Topic 042 0.62 Topic 030 69.84 Topic 043 0.97 Topic 031 82.49 Topic 044 2.55 0.4 Topic 032 63.16 Topic 045 62.50 Topic 033 4.02 Topic 046 33.63 0.2 Topic 034 44.77 Topic 047 0.54 Difference Topic 035 3.07 Topic 048 90.85 0 Topic 036 41.67 Topic 049 0.00 Topic 037 2.41 Topic 050 2.82 −0.2 Topic 038 0.87 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 95 hagen FUHddGYYYTD GC-MONO-DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual German track − Retrieved documents vs Precision 100% 5 docs 24.00 FUHddGYYYTD 10 docs 21.60 90% 15 docs 21.33 80% 20 docs 21.00 30 docs 19.60 70% 100 docs 12.04 60% 200 docs 6.92 R−Precision 500 docs 3.22 50% 1000 docs 1.80 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 21.53 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.8841 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0741 Third Quartile 0.4467 Interquartile range 0.4467 Mean 0.2153 Standard Deviation 0.2865 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8841 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2153 Std With No Outliers 0.2865 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries FUHddGYYYTD Topic 026 0.00 Topic 039 7.41 0.8 Topic 027 7.69 Topic 040 46.34 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 0.00 Topic 030 66.67 Topic 043 0.00 Topic 031 81.58 Topic 044 0.00 0.4 Topic 032 59.26 Topic 045 50.00 Topic 033 11.76 Topic 046 25.00 0.2 Topic 034 44.12 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 88.41 0 Topic 036 33.33 Topic 049 0.00 Topic 037 0.00 Topic 050 16.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 96 hagen FUHddGNNNTD GC-MONO-DE-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 23,756 Source Language German Relevant 602 Topic Fields title, description Relevant retrieved 439 Pooled true Geometric Mean Average Precision 0.0510 fourth run Binary Preference (BPREF) 0.1574 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision 100% 0 41.78 FUHddGNNNTD 10 35.24 90% 20 30.43 80% 30 24.74 40 18.09 70% 50 16.62 Average Precision 60% 60 13.78 70 8.27 50% 80 5.16 40% 90 3.98 30% 100 0.18 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 16.94 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.7469 Minimum 0.0000 First Quartile 0.0176 Second Quartile 0.0528 Third Quartile 0.2886 Interquartile range 0.2710 Mean 0.1694 Standard Deviation 0.2161 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5884 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1453 Std With No Outliers 0.1834 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries FUHddGNNNTD Topic 026 1.72 Topic 039 2.60 0.8 Topic 027 3.59 Topic 040 31.10 Topic 028 2.89 Topic 041 1.10 0.6 Topic 029 9.86 Topic 042 1.39 Topic 030 58.84 Topic 043 1.15 Topic 031 18.85 Topic 044 33.40 0.4 Topic 032 58.23 Topic 045 0.63 Topic 033 9.85 Topic 046 28.11 0.2 Topic 034 44.77 Topic 047 3.53 Difference Topic 035 5.28 Topic 048 74.69 0 Topic 036 3.23 Topic 049 1.77 Topic 037 19.30 Topic 050 7.55 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 97 hagen FUHddGNNNTD GC-MONO-DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual German track − Retrieved documents vs Precision 100% 5 docs 24.00 FUHddGNNNTD 10 docs 20.80 90% 15 docs 21.07 80% 20 docs 19.20 30 docs 18.40 70% 100 docs 9.96 60% 200 docs 6.20 R−Precision 500 docs 3.14 50% 1000 docs 1.76 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 18.00 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.6333 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0714 Third Quartile 0.2906 Interquartile range 0.2906 Mean 0.1800 Standard Deviation 0.2161 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1800 Std With No Outliers 0.2161 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries FUHddGNNNTD Topic 026 0.00 Topic 039 3.70 0.8 Topic 027 13.85 Topic 040 48.78 Topic 028 6.25 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 4.26 Topic 030 63.33 Topic 043 7.14 Topic 031 27.63 Topic 044 33.33 0.4 Topic 032 57.41 Topic 045 0.00 Topic 033 11.76 Topic 046 25.00 0.2 Topic 034 44.12 Topic 047 0.00 Difference Topic 035 5.56 Topic 048 62.32 0 Topic 036 0.00 Topic 049 0.00 Topic 037 27.27 Topic 050 8.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 98 hagen FUHddGNNNTDN GC-MONO-DE-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language German Relevant 602 Topic Fields title, description, narrative Relevant retrieved 426 Pooled true Geometric Mean Average Precision 0.0385 third run Binary Preference (BPREF) 0.1134 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision 100% 0 32.25 FUHddGNNNTDN 10 24.35 90% 20 18.96 80% 30 16.61 40 15.05 70% 50 13.18 Average Precision 60% 60 10.26 70 7.12 50% 80 5.24 40% 90 3.75 30% 100 0.57 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 12.23 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.6954 Minimum 0.0034 First Quartile 0.0123 Second Quartile 0.0298 Third Quartile 0.1342 Interquartile range 0.1218 Mean 0.1223 Standard Deviation 0.2016 Lower Outlier Threshold 0.0034 Upper Outlier Threshold 0.2754 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0558 Std With No Outliers 0.0780 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries FUHddGNNNTDN Topic 026 0.36 Topic 039 26.16 0.8 Topic 027 2.47 Topic 040 27.54 Topic 028 0.91 Topic 041 3.52 0.6 Topic 029 4.45 Topic 042 2.59 Topic 030 45.60 Topic 043 6.17 Topic 031 5.58 Topic 044 1.38 0.4 Topic 032 67.99 Topic 045 0.34 Topic 033 2.03 Topic 046 13.95 0.2 Topic 034 13.24 Topic 047 0.66 Difference Topic 035 2.98 Topic 048 69.54 0 Topic 036 1.65 Topic 049 3.41 Topic 037 1.28 Topic 050 0.95 −0.2 Topic 038 1.11 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 99 hagen FUHddGNNNTDN GC-MONO-DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual German track − Retrieved documents vs Precision 100% 5 docs 19.20 FUHddGNNNTDN 10 docs 19.60 90% 15 docs 17.60 80% 20 docs 15.60 30 docs 14.00 70% 100 docs 8.84 60% 200 docs 5.58 R−Precision 500 docs 2.86 50% 1000 docs 1.70 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 13.40 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.6852 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0312 Third Quartile 0.2100 Interquartile range 0.2100 Mean 0.1340 Standard Deviation 0.2037 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0875 Std With No Outliers 0.1304 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries FUHddGNNNTDN Topic 026 0.00 Topic 039 22.22 0.8 Topic 027 4.62 Topic 040 34.15 Topic 028 3.12 Topic 041 10.53 0.6 Topic 029 0.00 Topic 042 4.26 Topic 030 46.67 Topic 043 14.29 Topic 031 15.79 Topic 044 0.00 0.4 Topic 032 68.52 Topic 045 0.00 Topic 033 0.00 Topic 046 25.00 0.2 Topic 034 20.59 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 65.22 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 100 hagen FUHddGYYYTDN GC-MONO-DE-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 24,363 Source Language German Relevant 602 Topic Fields title, description, narrative Relevant retrieved 462 Pooled true Geometric Mean Average Precision 0.0337 first run Binary Preference (BPREF) 0.1929 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision 100% 0 44.57 FUHddGYYYTDN 10 35.11 90% 20 32.13 80% 30 27.93 40 23.36 70% 50 22.49 Average Precision 60% 60 19.04 70 15.51 50% 80 12.08 40% 90 9.05 30% 100 1.16 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 21.41 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.9161 Minimum 0.0000 First Quartile 0.0086 Second Quartile 0.0478 Third Quartile 0.3660 Interquartile range 0.3574 Mean 0.2141 Standard Deviation 0.2865 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8249 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1849 Std With No Outliers 0.2517 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries FUHddGYYYTDN Topic 026 0.13 Topic 039 34.91 0.8 Topic 027 1.14 Topic 040 32.59 Topic 028 1.74 Topic 041 6.26 0.6 Topic 029 1.93 Topic 042 4.07 Topic 030 67.56 Topic 043 5.87 Topic 031 82.49 Topic 044 0.91 0.4 Topic 032 63.16 Topic 045 4.78 Topic 033 11.66 Topic 046 33.63 0.2 Topic 034 44.77 Topic 047 0.00 Difference Topic 035 3.07 Topic 048 91.61 0 Topic 036 41.67 Topic 049 0.58 Topic 037 0.02 Topic 050 0.09 −0.2 Topic 038 0.72 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 101 hagen FUHddGYYYTDN GC-MONO-DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual German track − Retrieved documents vs Precision 100% 5 docs 27.20 FUHddGYYYTDN 10 docs 23.60 90% 15 docs 22.40 80% 20 docs 22.20 30 docs 20.67 70% 100 docs 12.52 60% 200 docs 7.30 R−Precision 500 docs 3.41 50% 1000 docs 1.85 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 20.56 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.8841 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0625 Third Quartile 0.3603 Interquartile range 0.3603 Mean 0.2056 Standard Deviation 0.2785 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8841 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2056 Std With No Outliers 0.2785 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries FUHddGYYYTDN Topic 026 0.00 Topic 039 29.63 0.8 Topic 027 7.69 Topic 040 46.34 Topic 028 6.25 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 4.26 Topic 030 63.33 Topic 043 7.14 Topic 031 81.58 Topic 044 0.00 0.4 Topic 032 59.26 Topic 045 0.00 Topic 033 17.65 Topic 046 25.00 0.2 Topic 034 44.12 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 88.41 0 Topic 036 33.33 Topic 049 0.00 Topic 037 0.00 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 102 hagen FUHddGYYYMTDN GC-MONO-DE-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 24,361 Source Language German Relevant 602 Topic Fields title, description, narrative Relevant retrieved 442 Pooled true Geometric Mean Average Precision 0.0318 fifth run Binary Preference (BPREF) 0.1905 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision 100% 0 39.98 FUHddGYYYMTDN 10 37.09 90% 20 32.92 80% 30 27.08 40 21.92 70% 50 20.54 Average Precision 60% 60 16.82 70 13.13 50% 80 9.93 40% 90 7.45 30% 100 0.73 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 19.99 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.8723 Minimum 0.0000 First Quartile 0.0103 Second Quartile 0.0356 Third Quartile 0.3296 Interquartile range 0.3193 Mean 0.1999 Standard Deviation 0.2771 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7952 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1719 Std With No Outliers 0.2442 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries FUHddGYYYMTDN Topic 026 0.26 Topic 039 30.63 0.8 Topic 027 0.40 Topic 040 29.27 Topic 028 2.74 Topic 041 3.56 0.6 Topic 029 1.64 Topic 042 6.16 Topic 030 66.68 Topic 043 4.89 Topic 031 79.52 Topic 044 1.13 0.4 Topic 032 64.31 Topic 045 3.20 Topic 033 5.51 Topic 046 26.74 0.2 Topic 034 39.97 Topic 047 0.00 Difference Topic 035 3.03 Topic 048 87.23 0 Topic 036 40.74 Topic 049 1.32 Topic 037 0.01 Topic 050 0.11 −0.2 Topic 038 0.72 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 103 hagen FUHddGYYYMTDN GC-MONO-DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual German track − Retrieved documents vs Precision 100% 5 docs 25.60 FUHddGYYYMTDN 10 docs 25.60 90% 15 docs 23.20 80% 20 docs 23.00 30 docs 20.93 70% 100 docs 11.68 60% 200 docs 7.34 R−Precision 500 docs 3.33 50% 1000 docs 1.77 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 20.39 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.8116 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0625 Third Quartile 0.4085 Interquartile range 0.4085 Mean 0.2039 Standard Deviation 0.2609 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8116 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2039 Std With No Outliers 0.2609 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries FUHddGYYYMTDN Topic 026 0.00 Topic 039 40.74 0.8 Topic 027 4.62 Topic 040 41.46 Topic 028 6.25 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 10.64 Topic 030 63.33 Topic 043 7.14 Topic 031 72.37 Topic 044 0.00 0.4 Topic 032 59.26 Topic 045 0.00 Topic 033 17.65 Topic 046 25.00 0.2 Topic 034 41.18 Topic 047 0.00 Difference Topic 035 5.56 Topic 048 81.16 0 Topic 036 33.33 Topic 049 0.00 Topic 037 0.00 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 104 hildesheim HIGeodederun4n GC-MONO-DE-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language German Relevant 602 Topic Fields title, description, narrative Relevant retrieved 419 Pooled true Geometric Mean Average Precision 0.0562 Experiment with BRF(5docs,25terms) stem lucene Binary Preference (BPREF) 0.1517 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision 100% 0 45.71 HIGeodederun4n 10 35.46 90% 20 27.03 80% 30 21.27 40 16.62 70% 50 14.56 Average Precision 60% 60 11.24 70 8.62 50% 80 5.50 40% 90 3.04 30% 100 0.46 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 16.01 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.7026 Minimum 0.0020 First Quartile 0.0170 Second Quartile 0.0611 Third Quartile 0.2795 Interquartile range 0.2625 Mean 0.1601 Standard Deviation 0.1994 Lower Outlier Threshold 0.0020 Upper Outlier Threshold 0.5947 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1375 Std With No Outliers 0.1678 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries HIGeodederun4n Topic 026 0.43 Topic 039 8.87 0.8 Topic 027 1.90 Topic 040 33.91 Topic 028 29.49 Topic 041 0.73 0.6 Topic 029 41.39 Topic 042 8.44 Topic 030 24.39 Topic 043 2.42 Topic 031 5.75 Topic 044 2.10 0.4 Topic 032 70.26 Topic 045 0.20 Topic 033 6.11 Topic 046 27.44 0.2 Topic 034 42.04 Topic 047 1.11 Difference Topic 035 6.85 Topic 048 59.47 0 Topic 036 0.71 Topic 049 17.63 Topic 037 0.54 Topic 050 5.49 −0.2 Topic 038 2.57 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 105 hildesheim HIGeodederun4n GC-MONO-DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual German track − Retrieved documents vs Precision 100% 5 docs 20.00 HIGeodederun4n 10 docs 19.20 90% 15 docs 18.67 80% 20 docs 18.00 30 docs 17.07 70% 100 docs 10.04 60% 200 docs 6.32 R−Precision 500 docs 2.97 50% 1000 docs 1.68 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 17.68 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.6852 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1231 Third Quartile 0.2708 Interquartile range 0.2708 Mean 0.1768 Standard Deviation 0.2002 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6087 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1557 Std With No Outliers 0.1735 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries HIGeodederun4n Topic 026 0.00 Topic 039 14.81 0.8 Topic 027 12.31 Topic 040 39.02 Topic 028 43.75 Topic 041 5.26 0.6 Topic 029 33.33 Topic 042 14.89 Topic 030 23.33 Topic 043 7.14 Topic 031 10.53 Topic 044 0.00 0.4 Topic 032 68.52 Topic 045 0.00 Topic 033 5.88 Topic 046 25.00 0.2 Topic 034 44.12 Topic 047 0.00 Difference Topic 035 16.67 Topic 048 60.87 0 Topic 036 0.00 Topic 049 16.67 Topic 037 0.00 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 106 hildesheim HIGeodederun4 GC-MONO-DE-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language German Relevant 602 Topic Fields title, description Relevant retrieved 390 Pooled true Geometric Mean Average Precision 0.0455 Experiment with BRF(5docs,25terms) stem lucene Binary Preference (BPREF) 0.1511 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision 100% 0 41.76 HIGeodederun4 10 34.02 90% 20 24.33 80% 30 23.05 40 20.46 70% 50 18.26 Average Precision 60% 60 11.03 70 6.94 50% 80 3.93 40% 90 2.04 30% 100 0.20 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 15.58 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.6498 Minimum 0.0005 First Quartile 0.0080 Second Quartile 0.0534 Third Quartile 0.2489 Interquartile range 0.2410 Mean 0.1558 Standard Deviation 0.2032 Lower Outlier Threshold 0.0005 Upper Outlier Threshold 0.5915 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1352 Std With No Outliers 0.1790 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries HIGeodederun4 Topic 026 0.44 Topic 039 0.81 0.8 Topic 027 2.28 Topic 040 30.80 Topic 028 24.04 Topic 041 0.54 0.6 Topic 029 16.90 Topic 042 4.83 Topic 030 27.44 Topic 043 1.12 Topic 031 5.31 Topic 044 2.08 0.4 Topic 032 64.98 Topic 045 0.05 Topic 033 5.95 Topic 046 51.08 0.2 Topic 034 50.11 Topic 047 0.74 Difference Topic 035 6.76 Topic 048 59.15 0 Topic 036 0.26 Topic 049 16.67 Topic 037 0.75 Topic 050 5.34 −0.2 Topic 038 10.96 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 107 hildesheim HIGeodederun4 GC-MONO-DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual German track − Retrieved documents vs Precision 100% 5 docs 21.60 HIGeodederun4 10 docs 19.20 90% 15 docs 18.13 80% 20 docs 17.60 30 docs 16.80 70% 100 docs 9.48 60% 200 docs 5.70 R−Precision 500 docs 2.77 50% 1000 docs 1.56 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 18.15 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.6481 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1111 Third Quartile 0.3476 Interquartile range 0.3476 Mean 0.1815 Standard Deviation 0.2146 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6481 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1815 Std With No Outliers 0.2146 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries HIGeodederun4 Topic 026 0.00 Topic 039 0.00 0.8 Topic 027 12.31 Topic 040 39.02 Topic 028 40.62 Topic 041 5.26 0.6 Topic 029 33.33 Topic 042 12.77 Topic 030 26.67 Topic 043 0.00 Topic 031 13.16 Topic 044 0.00 0.4 Topic 032 64.81 Topic 045 0.00 Topic 033 5.88 Topic 046 50.00 0.2 Topic 034 50.00 Topic 047 0.00 Difference Topic 035 11.11 Topic 048 63.77 0 Topic 036 0.00 Topic 049 16.67 Topic 037 0.00 Topic 050 8.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 108 hildesheim HIGeodederun6 GC-MONO-DE-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 21,416 Source Language German Relevant 602 Topic Fields title, description Relevant retrieved 301 Pooled true Geometric Mean Average Precision 0.0076 Experiment with BRF(5docs,25terms) with NE- Binary Preference (BPREF) 0.1210 recognition and weighting, also within the BRF Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision 100% 0 34.12 HIGeodederun6 10 25.23 90% 20 18.54 80% 30 17.40 40 14.32 70% 50 13.07 Average Precision 60% 60 9.86 70 5.72 50% 80 3.72 40% 90 1.84 30% 100 0.49 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 12.14 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.5888 Minimum 0.0000 First Quartile 0.0021 Second Quartile 0.0133 Third Quartile 0.1458 Interquartile range 0.1437 Mean 0.1214 Standard Deviation 0.1944 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1459 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0296 Std With No Outliers 0.0466 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries HIGeodederun6 Topic 026 0.10 Topic 039 0.25 0.8 Topic 027 1.33 Topic 040 14.58 Topic 028 0.62 Topic 041 1.32 0.6 Topic 029 46.67 Topic 042 3.73 Topic 030 58.88 Topic 043 0.52 Topic 031 0.25 Topic 044 1.33 0.4 Topic 032 45.82 Topic 045 0.00 Topic 033 6.40 Topic 046 0.00 0.2 Topic 034 14.59 Topic 047 39.11 Difference Topic 035 0.28 Topic 048 53.78 0 Topic 036 0.00 Topic 049 8.28 Topic 037 0.01 Topic 050 5.71 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 109 hildesheim HIGeodederun6 GC-MONO-DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual German track − Retrieved documents vs Precision 100% 5 docs 19.20 HIGeodederun6 10 docs 16.00 90% 15 docs 13.87 80% 20 docs 12.80 30 docs 11.60 70% 100 docs 6.88 60% 200 docs 4.30 R−Precision 500 docs 2.14 50% 1000 docs 1.20 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 13.45 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.6333 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0213 Third Quartile 0.1872 Interquartile range 0.1872 Mean 0.1345 Standard Deviation 0.2032 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0559 Std With No Outliers 0.0922 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries HIGeodederun6 Topic 026 0.00 Topic 039 0.00 0.8 Topic 027 6.15 Topic 040 21.95 Topic 028 0.00 Topic 041 5.26 0.6 Topic 029 33.33 Topic 042 2.13 Topic 030 63.33 Topic 043 0.00 Topic 031 0.00 Topic 044 0.00 0.4 Topic 032 51.85 Topic 045 0.00 Topic 033 5.88 Topic 046 0.00 0.2 Topic 034 17.65 Topic 047 50.00 Difference Topic 035 0.00 Topic 048 53.62 0 Topic 036 0.00 Topic 049 16.67 Topic 037 0.00 Topic 050 8.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 110 hildesheim HIGeodederun6n GC-MONO-DE-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 22,151 Source Language German Relevant 568 Topic Fields title, description, narrative Relevant retrieved 296 Pooled true Geometric Mean Average Precision 0.0077 Experiment with BRF(5docs,25terms) with NE- Binary Preference (BPREF) 0.1164 recognition and weighting, also within the BRF Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision 100% 0 29.91 HIGeodederun6n 10 23.90 90% 20 18.87 80% 30 16.83 40 12.56 70% 50 11.13 Average Precision 60% 60 9.42 70 5.94 50% 80 3.46 40% 90 1.51 30% 100 0.38 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 11.34 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.5669 Minimum 0.0000 First Quartile 0.0015 Second Quartile 0.0143 Third Quartile 0.1462 Interquartile range 0.1447 Mean 0.1134 Standard Deviation 0.1847 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.2443 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0402 Std With No Outliers 0.0717 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries HIGeodederun6n Topic 026 0.12 Topic 039 24.43 0.8 Topic 027 1.43 Topic 040 12.04 Topic 028 22.37 Topic 041 1.46 0.6 Topic 029 41.64 Topic 042 1.66 Topic 030 53.48 Topic 043 0.86 Topic 031 0.23 Topic 044 1.31 0.4 Topic 032 47.14 Topic 045 0.00 Topic 033 0.18 Topic 046 0.00 0.2 Topic 034 0.00 Topic 047 2.83 Difference Topic 035 0.67 Topic 048 56.69 0 Topic 036 0.16 Topic 049 8.34 Topic 037 0.12 Topic 050 6.27 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 111 hildesheim HIGeodederun6n GC-MONO-DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual German track − Retrieved documents vs Precision 100% 5 docs 16.00 HIGeodederun6n 10 docs 16.40 90% 15 docs 15.73 80% 20 docs 14.20 30 docs 12.80 70% 100 docs 7.60 60% 200 docs 4.48 R−Precision 500 docs 2.08 50% 1000 docs 1.18 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 13.72 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment Maximum 0.6232 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.2648 Interquartile range 0.2648 Mean 0.1372 Standard Deviation 0.1981 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6232 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1372 Std With No Outliers 0.1981 GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries HIGeodederun6n Topic 026 0.00 Topic 039 25.93 0.8 Topic 027 10.77 Topic 040 29.27 Topic 028 28.12 Topic 041 5.26 0.6 Topic 029 33.33 Topic 042 4.26 Topic 030 56.67 Topic 043 0.00 Topic 031 0.00 Topic 044 0.00 0.4 Topic 032 53.70 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 62.32 0 Topic 036 0.00 Topic 049 16.67 Topic 037 0.00 Topic 050 16.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 112 alicante enTD GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 325 Pooled true Geometric Mean Average Precision 0.0700 Title and Description Binary Preference (BPREF) 0.2415 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 47.13 enTD 10 39.05 90% 20 36.01 80% 30 34.33 40 32.37 70% 50 31.03 Average Precision 60% 60 26.86 70 19.84 50% 80 17.01 40% 90 13.36 30% 100 10.10 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 27.23 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9167 Minimum 0.0000 First Quartile 0.0324 Second Quartile 0.1134 Third Quartile 0.4357 Interquartile range 0.4034 Mean 0.2723 Standard Deviation 0.2969 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.9167 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2723 Std With No Outliers 0.2969 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries enTD Topic 026 49.07 Topic 039 3.94 0.8 Topic 027 0.22 Topic 040 36.93 Topic 028 16.85 Topic 041 0.17 0.6 Topic 029 9.07 Topic 042 45.00 Topic 030 91.67 Topic 043 1.13 Topic 031 43.10 Topic 044 11.34 0.4 Topic 032 88.41 Topic 045 10.04 Topic 033 0.30 Topic 046 71.43 0.2 Topic 034 37.68 Topic 047 5.48 Difference Topic 035 5.07 Topic 048 80.82 0 Topic 036 0.00 Topic 049 36.11 Topic 037 9.16 Topic 050 26.79 −0.2 Topic 038 1.03 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 113 alicante enTD GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 28.80 enTD 10 docs 24.00 90% 15 docs 21.60 80% 20 docs 19.60 30 docs 16.93 70% 100 docs 8.84 60% 200 docs 5.32 R−Precision 500 docs 2.38 50% 1000 docs 1.30 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 28.01 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8387 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.2105 Third Quartile 0.5000 Interquartile range 0.5000 Mean 0.2801 Standard Deviation 0.2895 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8387 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2801 Std With No Outliers 0.2895 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries enTD Topic 026 55.56 Topic 039 0.00 0.8 Topic 027 0.00 Topic 040 35.71 Topic 028 26.32 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 50.00 Topic 030 83.33 Topic 043 0.00 Topic 031 45.76 Topic 044 21.05 0.4 Topic 032 83.87 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 8.33 Difference Topic 035 0.00 Topic 048 77.08 0 Topic 036 0.00 Topic 049 50.00 Topic 037 18.75 Topic 050 33.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 114 alicante enTDN GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 310 Pooled false Geometric Mean Average Precision 0.0592 Title, Description and Narrative Binary Preference (BPREF) 0.2672 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 44.88 enTDN 10 43.52 90% 20 40.20 80% 30 39.30 40 34.54 70% 50 33.86 Average Precision 60% 60 29.44 70 22.40 50% 80 20.75 40% 90 15.50 30% 100 12.75 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 29.85 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0088 Second Quartile 0.1585 Third Quartile 0.5044 Interquartile range 0.4957 Mean 0.2985 Standard Deviation 0.3381 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 1.0000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2985 Std With No Outliers 0.3381 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries enTDN Topic 026 50.09 Topic 039 36.53 0.8 Topic 027 0.66 Topic 040 32.11 Topic 028 3.93 Topic 041 0.18 0.6 Topic 029 12.84 Topic 042 100.00 Topic 030 95.83 Topic 043 0.95 Topic 031 35.57 Topic 044 8.38 0.4 Topic 032 90.05 Topic 045 29.89 Topic 033 0.50 Topic 046 69.05 0.2 Topic 034 51.52 Topic 047 1.69 Difference Topic 035 2.40 Topic 048 82.66 0 Topic 036 0.00 Topic 049 25.00 Topic 037 0.06 Topic 050 15.85 −0.2 Topic 038 0.57 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 115 alicante enTDN GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 26.40 enTDN 10 docs 22.40 90% 15 docs 20.53 80% 20 docs 19.00 30 docs 16.67 70% 100 docs 8.28 60% 200 docs 4.90 R−Precision 500 docs 2.23 50% 1000 docs 1.24 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 28.51 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1316 Third Quartile 0.5139 Interquartile range 0.5139 Mean 0.2851 Standard Deviation 0.3290 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 1.0000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2851 Std With No Outliers 0.3290 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries enTDN Topic 026 55.56 Topic 039 50.00 0.8 Topic 027 5.26 Topic 040 28.57 Topic 028 5.26 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 100.00 Topic 030 83.33 Topic 043 0.00 Topic 031 32.20 Topic 044 13.16 0.4 Topic 032 87.10 Topic 045 33.33 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 81.25 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 26.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 116 alicante enTDNGeoNames GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction MANUAL Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 195 Pooled true Geometric Mean Average Precision 0.0041 Title, Description and Narrative with GeoNames Binary Preference (BPREF) 0.0907 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 21.96 enTDNGeoNames 10 20.14 90% 20 16.45 80% 30 15.41 40 14.43 70% 50 13.74 Average Precision 60% 60 11.98 70 9.98 50% 80 8.39 40% 90 3.96 30% 100 1.87 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 12.01 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8169 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0144 Third Quartile 0.0822 Interquartile range 0.0821 Mean 0.1201 Standard Deviation 0.2220 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1323 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0219 Std With No Outliers 0.0339 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries enTDNGeoNames Topic 026 48.34 Topic 039 6.28 0.8 Topic 027 0.04 Topic 040 26.93 Topic 028 0.59 Topic 041 0.67 0.6 Topic 029 4.91 Topic 042 6.55 Topic 030 0.00 Topic 043 0.05 Topic 031 2.03 Topic 044 0.00 0.4 Topic 032 64.59 Topic 045 34.89 Topic 033 2.59 Topic 046 4.08 0.2 Topic 034 1.44 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 81.69 0 Topic 036 0.00 Topic 049 0.00 Topic 037 1.30 Topic 050 13.23 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 117 alicante enTDNGeoNames GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 11.20 enTDNGeoNames 10 docs 11.60 90% 15 docs 12.00 80% 20 docs 11.60 30 docs 10.53 70% 100 docs 5.52 60% 200 docs 3.22 R−Precision 500 docs 1.39 50% 1000 docs 0.78 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 10.30 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7917 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.0638 Interquartile range 0.0638 Mean 0.1030 Standard Deviation 0.2240 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.0678 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0065 Std With No Outliers 0.0201 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries enTDNGeoNames Topic 026 55.56 Topic 039 0.00 0.8 Topic 027 0.00 Topic 040 28.57 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 0.00 Topic 030 0.00 Topic 043 0.00 Topic 031 6.78 Topic 044 0.00 0.4 Topic 032 64.52 Topic 045 16.67 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 79.17 0 Topic 036 0.00 Topic 049 0.00 Topic 037 6.25 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 118 alicante UAUJAUPVenenExp1 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 312 Pooled false Geometric Mean Average Precision 0.0552 Voting UA, UJA and UPV Binary Preference (BPREF) 0.2172 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 43.89 UAUJAUPVenenExp1 10 36.13 90% 20 34.85 80% 30 33.53 40 33.09 70% 50 32.25 Average Precision 60% 60 24.06 70 14.30 50% 80 11.60 40% 90 7.30 30% 100 6.08 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 24.03 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8374 Minimum 0.0000 First Quartile 0.0174 Second Quartile 0.0550 Third Quartile 0.4547 Interquartile range 0.4373 Mean 0.2403 Standard Deviation 0.2870 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8374 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2403 Std With No Outliers 0.2870 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries UAUJAUPVenenExp1 Topic 026 1.93 Topic 039 21.79 0.8 Topic 027 1.00 Topic 040 32.18 Topic 028 0.64 Topic 041 0.86 0.6 Topic 029 5.50 Topic 042 64.29 Topic 030 73.47 Topic 043 1.87 Topic 031 30.41 Topic 044 3.81 0.4 Topic 032 83.74 Topic 045 11.05 Topic 033 0.22 Topic 046 66.84 0.2 Topic 034 42.11 Topic 047 1.84 Difference Topic 035 3.23 Topic 048 71.84 0 Topic 036 0.00 Topic 049 55.56 Topic 037 1.83 Topic 050 23.27 −0.2 Topic 038 1.47 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 119 alicante UAUJAUPVenenExp1 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 21.60 UAUJAUPVenenExp1 10 docs 17.60 90% 15 docs 16.80 80% 20 docs 16.60 30 docs 13.73 70% 100 docs 7.40 60% 200 docs 4.42 R−Precision 500 docs 2.09 50% 1000 docs 1.25 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 23.19 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8065 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0526 Third Quartile 0.5000 Interquartile range 0.5000 Mean 0.2319 Standard Deviation 0.2847 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8065 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2319 Std With No Outliers 0.2847 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries UAUJAUPVenenExp1 Topic 026 0.00 Topic 039 25.00 0.8 Topic 027 5.26 Topic 040 21.43 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 50.00 Topic 030 66.67 Topic 043 0.00 Topic 031 32.20 Topic 044 2.63 0.4 Topic 032 80.65 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 66.67 Topic 047 4.17 Difference Topic 035 0.00 Topic 048 68.75 0 Topic 036 0.00 Topic 049 50.00 Topic 037 6.25 Topic 050 33.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 120 berkeley BKGeoE4 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction MANUAL Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 320 Pooled true Geometric Mean Average Precision 0.0612 Manual expansion of topics 27,43,50 with deletion Binary Preference (BPREF) 0.2554 of country names from 50, blind feedback Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 49.77 BKGeoE4 10 43.96 90% 20 40.16 80% 30 38.47 40 29.84 70% 50 29.36 Average Precision 60% 60 26.20 70 21.49 50% 80 19.97 40% 90 15.51 30% 100 12.44 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 28.87 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9762 Minimum 0.0000 First Quartile 0.0204 Second Quartile 0.1581 Third Quartile 0.3803 Interquartile range 0.3600 Mean 0.2887 Standard Deviation 0.3232 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8898 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2295 Std With No Outliers 0.2610 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries BKGeoE4 Topic 026 11.00 Topic 039 35.56 0.8 Topic 027 9.30 Topic 040 33.60 Topic 028 0.16 Topic 041 1.16 0.6 Topic 029 5.86 Topic 042 75.00 Topic 030 97.62 Topic 043 31.15 Topic 031 41.33 Topic 044 9.91 0.4 Topic 032 96.31 Topic 045 15.81 Topic 033 0.06 Topic 046 70.24 0.2 Topic 034 36.93 Topic 047 2.96 Difference Topic 035 2.33 Topic 048 88.98 0 Topic 036 0.00 Topic 049 25.00 Topic 037 0.07 Topic 050 30.91 −0.2 Topic 038 0.49 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 121 berkeley BKGeoE4 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 28.80 BKGeoE4 10 docs 25.60 90% 15 docs 21.07 80% 20 docs 19.80 30 docs 17.73 70% 100 docs 8.28 60% 200 docs 4.84 R−Precision 500 docs 2.28 50% 1000 docs 1.28 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 27.11 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9032 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1667 Third Quartile 0.3957 Interquartile range 0.3957 Mean 0.2711 Standard Deviation 0.2936 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.9032 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2711 Std With No Outliers 0.2936 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries BKGeoE4 Topic 026 22.22 Topic 039 37.50 0.8 Topic 027 15.79 Topic 040 35.71 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 50.00 Topic 030 83.33 Topic 043 37.50 Topic 031 45.76 Topic 044 13.16 0.4 Topic 032 90.32 Topic 045 16.67 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 85.42 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 33.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 122 berkeley BKGeoE2 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 313 Pooled false Geometric Mean Average Precision 0.0503 English topics TDN with blind feedback, baseline Binary Preference (BPREF) 0.2326 run Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 41.75 BKGeoE2 10 38.92 90% 20 36.18 80% 30 34.92 40 28.00 70% 50 27.65 Average Precision 60% 60 24.86 70 20.70 50% 80 19.36 40% 90 15.24 30% 100 12.16 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 26.56 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9762 Minimum 0.0000 First Quartile 0.0178 Second Quartile 0.0991 Third Quartile 0.3803 Interquartile range 0.3625 Mean 0.2656 Standard Deviation 0.3312 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8898 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2044 Std With No Outliers 0.2659 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries BKGeoE2 Topic 026 11.00 Topic 039 35.56 0.8 Topic 027 1.99 Topic 040 33.60 Topic 028 0.16 Topic 041 1.16 0.6 Topic 029 5.86 Topic 042 75.00 Topic 030 97.62 Topic 043 6.53 Topic 031 41.33 Topic 044 9.91 0.4 Topic 032 96.31 Topic 045 15.81 Topic 033 0.06 Topic 046 70.24 0.2 Topic 034 36.93 Topic 047 2.96 Difference Topic 035 2.33 Topic 048 88.98 0 Topic 036 0.00 Topic 049 25.00 Topic 037 0.07 Topic 050 5.06 −0.2 Topic 038 0.49 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 123 berkeley BKGeoE2 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 26.40 BKGeoE2 10 docs 22.80 90% 15 docs 19.47 80% 20 docs 18.20 30 docs 16.27 70% 100 docs 7.84 60% 200 docs 4.70 R−Precision 500 docs 2.25 50% 1000 docs 1.25 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 24.84 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9032 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1250 Third Quartile 0.3957 Interquartile range 0.3957 Mean 0.2484 Standard Deviation 0.2971 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.9032 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2484 Std With No Outliers 0.2971 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries BKGeoE2 Topic 026 22.22 Topic 039 37.50 0.8 Topic 027 10.53 Topic 040 35.71 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 50.00 Topic 030 83.33 Topic 043 12.50 Topic 031 45.76 Topic 044 13.16 0.4 Topic 032 90.32 Topic 045 16.67 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 85.42 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 6.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 124 berkeley BKGeoE3 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction MANUAL Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 320 Pooled false Geometric Mean Average Precision 0.0596 Manual expansion of topics 27, 43,50 blind feedback Binary Preference (BPREF) 0.2493 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 48.77 BKGeoE3 10 43.31 90% 20 39.52 80% 30 37.69 40 29.00 70% 50 28.59 Average Precision 60% 60 25.42 70 20.94 50% 80 19.52 40% 90 15.34 30% 100 12.28 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 28.27 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9762 Minimum 0.0000 First Quartile 0.0204 Second Quartile 0.1581 Third Quartile 0.3803 Interquartile range 0.3600 Mean 0.2827 Standard Deviation 0.3242 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8898 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2229 Std With No Outliers 0.2608 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries BKGeoE3 Topic 026 11.00 Topic 039 35.56 0.8 Topic 027 9.30 Topic 040 33.60 Topic 028 0.16 Topic 041 1.16 0.6 Topic 029 5.86 Topic 042 75.00 Topic 030 97.62 Topic 043 31.15 Topic 031 41.33 Topic 044 9.91 0.4 Topic 032 96.31 Topic 045 15.81 Topic 033 0.06 Topic 046 70.24 0.2 Topic 034 36.93 Topic 047 2.96 Difference Topic 035 2.33 Topic 048 88.98 0 Topic 036 0.00 Topic 049 25.00 Topic 037 0.07 Topic 050 15.86 −0.2 Topic 038 0.49 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 125 berkeley BKGeoE3 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 28.80 BKGeoE3 10 docs 24.80 90% 15 docs 20.53 80% 20 docs 19.20 30 docs 17.20 70% 100 docs 8.20 60% 200 docs 4.82 R−Precision 500 docs 2.27 50% 1000 docs 1.28 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 26.58 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9032 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1667 Third Quartile 0.3957 Interquartile range 0.3957 Mean 0.2658 Standard Deviation 0.2936 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.9032 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2658 Std With No Outliers 0.2936 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries BKGeoE3 Topic 026 22.22 Topic 039 37.50 0.8 Topic 027 15.79 Topic 040 35.71 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 50.00 Topic 030 83.33 Topic 043 37.50 Topic 031 45.76 Topic 044 13.16 0.4 Topic 032 90.32 Topic 045 16.67 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 85.42 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 20.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 126 berkeley BKGeoE1 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 332 Pooled true Geometric Mean Average Precision 0.0743 Automatic with Blind Feedback TD Binary Preference (BPREF) 0.2044 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 46.31 BKGeoE1 10 37.74 90% 20 31.60 80% 30 28.99 40 27.55 70% 50 27.23 Average Precision 60% 60 25.04 70 18.52 50% 80 16.78 40% 90 13.44 30% 100 10.72 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 24.99 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9565 Minimum 0.0000 First Quartile 0.0388 Second Quartile 0.1402 Third Quartile 0.3776 Interquartile range 0.3387 Mean 0.2499 Standard Deviation 0.3045 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6952 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1586 Std With No Outliers 0.1819 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries BKGeoE1 Topic 026 6.02 Topic 039 4.51 0.8 Topic 027 3.96 Topic 040 38.53 Topic 028 3.65 Topic 041 0.56 0.6 Topic 029 14.02 Topic 042 14.35 Topic 030 91.07 Topic 043 2.48 Topic 031 48.24 Topic 044 24.20 0.4 Topic 032 95.65 Topic 045 14.14 Topic 033 0.23 Topic 046 69.52 0.2 Topic 034 25.34 Topic 047 9.80 Difference Topic 035 4.41 Topic 048 89.08 0 Topic 036 0.00 Topic 049 37.50 Topic 037 17.91 Topic 050 8.49 −0.2 Topic 038 1.11 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 127 berkeley BKGeoE1 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 24.80 BKGeoE1 10 docs 21.20 90% 15 docs 20.53 80% 20 docs 19.40 30 docs 17.07 70% 100 docs 9.20 60% 200 docs 5.54 R−Precision 500 docs 2.52 50% 1000 docs 1.33 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 21.95 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8710 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1053 Third Quartile 0.3559 Interquartile range 0.3559 Mean 0.2195 Standard Deviation 0.2780 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8710 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2195 Std With No Outliers 0.2780 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries BKGeoE1 Topic 026 11.11 Topic 039 0.00 0.8 Topic 027 10.53 Topic 040 28.57 Topic 028 5.26 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 66.67 Topic 043 0.00 Topic 031 42.37 Topic 044 21.05 0.4 Topic 032 87.10 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 8.33 Difference Topic 035 0.00 Topic 048 81.25 0 Topic 036 0.00 Topic 049 50.00 Topic 037 18.75 Topic 050 6.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 128 daedalus GCenAtLg GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction MANUAL Retrieved 21,339 Source Language English Relevant 359 Topic Fields title, description, narrative Relevant retrieved 196 Pooled false Geometric Mean Average Precision 0.0131 All text Left geo run Binary Preference (BPREF) 0.1142 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 25.07 GCenAtLg 10 24.25 90% 20 21.27 80% 30 20.73 40 17.72 70% 50 15.12 Average Precision 60% 60 10.07 70 8.72 50% 80 6.70 40% 90 3.19 30% 100 2.90 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 13.05 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6795 Minimum 0.0000 First Quartile 0.0053 Second Quartile 0.0233 Third Quartile 0.1854 Interquartile range 0.1801 Mean 0.1305 Standard Deviation 0.1919 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.2444 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0681 Std With No Outliers 0.0860 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries GCenAtLg Topic 026 11.98 Topic 039 24.44 0.8 Topic 027 0.00 Topic 040 23.19 Topic 028 1.46 Topic 041 1.04 0.6 Topic 029 0.23 Topic 042 16.86 Topic 030 56.94 Topic 043 6.49 Topic 031 1.40 Topic 044 0.62 0.4 Topic 032 67.95 Topic 045 0.00 Topic 033 0.25 Topic 046 13.10 0.2 Topic 034 2.33 Topic 047 5.03 Difference Topic 035 0.81 Topic 048 51.55 0 Topic 036 0.00 Topic 049 21.11 Topic 037 1.81 Topic 050 17.68 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 129 daedalus GCenAtLg GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 16.80 GCenAtLg 10 docs 15.20 90% 15 docs 15.20 80% 20 docs 14.20 30 docs 12.27 70% 100 docs 5.16 60% 200 docs 3.10 R−Precision 500 docs 1.42 50% 1000 docs 0.78 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 13.57 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7097 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.2333 Interquartile range 0.2333 Mean 0.1357 Standard Deviation 0.2126 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5417 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1118 Std With No Outliers 0.1796 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries GCenAtLg Topic 026 33.33 Topic 039 43.75 0.8 Topic 027 0.00 Topic 040 35.71 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 0.00 Topic 030 50.00 Topic 043 12.50 Topic 031 8.47 Topic 044 0.00 0.4 Topic 032 70.97 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 0.00 Topic 047 4.17 Difference Topic 035 0.00 Topic 048 54.17 0 Topic 036 0.00 Topic 049 0.00 Topic 037 6.25 Topic 050 20.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 130 daedalus GCenNtLg GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction MANUAL Retrieved 19,789 Source Language English Relevant 344 Topic Fields title, description Relevant retrieved 171 Pooled false Geometric Mean Average Precision 0.0095 Normal text Left geo run Binary Preference (BPREF) 0.0885 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 23.71 GCenNtLg 10 22.17 90% 20 21.00 80% 30 19.08 40 8.05 70% 50 7.24 Average Precision 60% 60 6.05 70 3.39 50% 80 2.55 40% 90 1.66 30% 100 1.50 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 9.37 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6399 Minimum 0.0000 First Quartile 0.0083 Second Quartile 0.0323 Third Quartile 0.1189 Interquartile range 0.1106 Mean 0.0937 Standard Deviation 0.1469 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1957 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0483 Std With No Outliers 0.0574 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries GCenNtLg Topic 026 11.98 Topic 039 11.86 0.8 Topic 027 0.00 Topic 040 19.57 Topic 028 7.53 Topic 041 1.04 0.6 Topic 029 4.52 Topic 042 0.00 Topic 030 9.38 Topic 043 4.58 Topic 031 1.95 Topic 044 1.36 0.4 Topic 032 29.09 Topic 045 0.00 Topic 033 0.20 Topic 046 2.11 0.2 Topic 034 35.09 Topic 047 3.23 Difference Topic 035 7.81 Topic 048 63.99 0 Topic 036 0.00 Topic 049 16.25 Topic 037 1.24 Topic 050 0.00 −0.2 Topic 038 1.59 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 131 daedalus GCenNtLg GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 11.20 GCenNtLg 10 docs 15.20 90% 15 docs 13.87 80% 20 docs 12.20 30 docs 10.27 70% 100 docs 4.56 60% 200 docs 2.76 R−Precision 500 docs 1.24 50% 1000 docs 0.68 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 10.87 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6250 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.1875 Interquartile range 0.1875 Mean 0.1087 Standard Deviation 0.1678 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3871 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0872 Std With No Outliers 0.1316 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries GCenNtLg Topic 026 33.33 Topic 039 25.00 0.8 Topic 027 0.00 Topic 040 28.57 Topic 028 15.79 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 0.00 Topic 030 0.00 Topic 043 0.00 Topic 031 8.47 Topic 044 5.26 0.4 Topic 032 38.71 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 33.33 Topic 047 4.17 Difference Topic 035 16.67 Topic 048 62.50 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 132 daedalus GCenNA GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction MANUAL Retrieved 4,772 Source Language English Relevant 344 Topic Fields title, description Relevant retrieved 95 Pooled true Geometric Mean Average Precision 0.0020 Mandatory run Binary Preference (BPREF) 0.0830 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 28.20 GCenNA 10 25.28 90% 20 16.49 80% 30 14.80 40 9.93 70% 50 9.93 Average Precision 60% 60 5.70 70 0.80 50% 80 0.80 40% 90 0.80 30% 100 0.80 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 8.93 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6128 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0147 Third Quartile 0.1148 Interquartile range 0.1148 Mean 0.0893 Standard Deviation 0.1520 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.2000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0410 Std With No Outliers 0.0628 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries GCenNA Topic 026 11.11 Topic 039 19.25 0.8 Topic 027 0.00 Topic 040 7.01 Topic 028 0.00 Topic 041 3.12 0.6 Topic 029 12.59 Topic 042 0.00 Topic 030 0.00 Topic 043 2.92 Topic 031 1.47 Topic 044 3.36 0.4 Topic 032 36.86 Topic 045 0.00 Topic 033 0.20 Topic 046 0.00 0.2 Topic 034 35.09 Topic 047 0.00 Difference Topic 035 7.81 Topic 048 61.28 0 Topic 036 0.00 Topic 049 0.00 Topic 037 1.25 Topic 050 0.00 −0.2 Topic 038 20.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 133 daedalus GCenNA GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 12.80 GCenNA 10 docs 12.00 90% 15 docs 12.27 80% 20 docs 11.60 30 docs 10.13 70% 100 docs 3.60 60% 200 docs 1.82 R−Precision 500 docs 0.74 50% 1000 docs 0.38 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 9.70 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6667 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.1111 Interquartile range 0.1111 Mean 0.0970 Standard Deviation 0.1741 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.2500 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0413 Std With No Outliers 0.0700 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries GCenNA Topic 026 11.11 Topic 039 25.00 0.8 Topic 027 0.00 Topic 040 0.00 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 0.00 Topic 043 0.00 Topic 031 10.17 Topic 044 10.53 0.4 Topic 032 51.61 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 33.33 Topic 047 0.00 Difference Topic 035 16.67 Topic 048 66.67 0 Topic 036 0.00 Topic 049 0.00 Topic 037 6.25 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 134 daedalus GCenAA GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction MANUAL Retrieved 2,209 Source Language English Relevant 359 Topic Fields title, description, narrative Relevant retrieved 117 Pooled true Geometric Mean Average Precision 0.0036 All text And geo run Binary Preference (BPREF) 0.1343 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 28.80 GCenAA 10 27.19 90% 20 18.59 80% 30 17.41 40 17.14 70% 50 16.35 Average Precision 60% 60 14.05 70 7.31 50% 80 7.16 40% 90 5.01 30% 100 4.74 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 13.60 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8607 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0221 Third Quartile 0.1250 Interquartile range 0.1250 Mean 0.1360 Standard Deviation 0.2474 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3033 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0388 Std With No Outliers 0.0739 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries GCenAA Topic 026 11.11 Topic 039 30.33 0.8 Topic 027 0.00 Topic 040 6.32 Topic 028 0.00 Topic 041 3.12 0.6 Topic 029 2.41 Topic 042 16.67 Topic 030 72.66 Topic 043 3.79 Topic 031 1.05 Topic 044 0.80 0.4 Topic 032 86.07 Topic 045 0.00 Topic 033 0.31 Topic 046 38.89 0.2 Topic 034 0.00 Topic 047 0.30 Difference Topic 035 0.00 Topic 048 60.91 0 Topic 036 0.00 Topic 049 0.00 Topic 037 2.21 Topic 050 3.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 135 daedalus GCenAA GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 17.60 GCenAA 10 docs 14.80 90% 15 docs 14.67 80% 20 docs 13.00 30 docs 11.73 70% 100 docs 4.44 60% 200 docs 2.22 R−Precision 500 docs 0.90 50% 1000 docs 0.47 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 15.70 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8387 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.1271 Interquartile range 0.1271 Mean 0.1570 Standard Deviation 0.2610 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0345 Std With No Outliers 0.0525 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries GCenAA Topic 026 11.11 Topic 039 43.75 0.8 Topic 027 0.00 Topic 040 0.00 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 0.00 Topic 030 66.67 Topic 043 12.50 Topic 031 10.17 Topic 044 5.26 0.4 Topic 032 83.87 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 0.00 Topic 047 4.17 Difference Topic 035 0.00 Topic 048 62.50 0 Topic 036 0.00 Topic 049 0.00 Topic 037 12.50 Topic 050 13.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 136 daedalus GCenAO GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 24,149 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 180 Pooled false Geometric Mean Average Precision 0.0063 All text Or geo run Binary Preference (BPREF) 0.0871 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 18.95 GCenAO 10 17.28 90% 20 14.55 80% 30 14.30 40 11.22 70% 50 9.56 Average Precision 60% 60 6.96 70 5.81 50% 80 4.19 40% 90 1.22 30% 100 1.00 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 8.91 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6795 Minimum 0.0000 First Quartile 0.0018 Second Quartile 0.0140 Third Quartile 0.0846 Interquartile range 0.0828 Mean 0.0891 Standard Deviation 0.1679 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1310 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0281 Std With No Outliers 0.0404 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries GCenAO Topic 026 11.98 Topic 039 7.29 0.8 Topic 027 0.03 Topic 040 23.19 Topic 028 0.00 Topic 041 0.33 0.6 Topic 029 0.23 Topic 042 0.99 Topic 030 5.46 Topic 043 6.49 Topic 031 1.40 Topic 044 0.62 0.4 Topic 032 67.95 Topic 045 0.00 Topic 033 0.25 Topic 046 13.10 0.2 Topic 034 2.33 Topic 047 6.33 Difference Topic 035 0.03 Topic 048 51.55 0 Topic 036 0.00 Topic 049 21.11 Topic 037 1.81 Topic 050 0.31 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 137 daedalus GCenAO GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 12.80 GCenAO 10 docs 12.00 90% 15 docs 12.00 80% 20 docs 11.20 30 docs 9.33 70% 100 docs 4.44 60% 200 docs 2.62 R−Precision 500 docs 1.28 50% 1000 docs 0.72 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 9.52 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7097 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.0948 Interquartile range 0.0948 Mean 0.0952 Standard Deviation 0.1885 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1250 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0209 Std With No Outliers 0.0418 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries GCenAO Topic 026 33.33 Topic 039 12.50 0.8 Topic 027 0.00 Topic 040 35.71 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 0.00 Topic 030 0.00 Topic 043 12.50 Topic 031 8.47 Topic 044 0.00 0.4 Topic 032 70.97 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 0.00 Topic 047 4.17 Difference Topic 035 0.00 Topic 048 54.17 0 Topic 036 0.00 Topic 049 0.00 Topic 037 6.25 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 138 hildesheim HIGeoenenrun1n GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 211 Pooled false Geometric Mean Average Precision 0.0109 no BRF base run, stem snowball Binary Preference (BPREF) 0.1532 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 34.92 HIGeoenenrun1n 10 28.72 90% 20 26.71 80% 30 23.54 40 21.02 70% 50 19.70 Average Precision 60% 60 16.56 70 8.67 50% 80 6.50 40% 90 5.51 30% 100 5.05 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 17.47 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7729 Minimum 0.0000 First Quartile 0.0015 Second Quartile 0.0178 Third Quartile 0.2501 Interquartile range 0.2486 Mean 0.1747 Standard Deviation 0.2683 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5755 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0987 Std With No Outliers 0.1777 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries HIGeoenenrun1n Topic 026 0.38 Topic 039 0.90 0.8 Topic 027 0.00 Topic 040 77.29 Topic 028 0.12 Topic 041 0.37 0.6 Topic 029 6.72 Topic 042 0.24 Topic 030 57.55 Topic 043 0.03 Topic 031 17.77 Topic 044 5.03 0.4 Topic 032 46.75 Topic 045 0.65 Topic 033 0.00 Topic 046 68.94 0.2 Topic 034 17.12 Topic 047 1.87 Difference Topic 035 0.05 Topic 048 73.51 0 Topic 036 0.00 Topic 049 1.78 Topic 037 0.16 Topic 050 9.58 −0.2 Topic 038 50.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 139 hildesheim HIGeoenenrun1n GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 21.60 HIGeoenenrun1n 10 docs 18.00 90% 15 docs 15.47 80% 20 docs 15.00 30 docs 11.87 70% 100 docs 4.68 60% 200 docs 2.80 R−Precision 500 docs 1.50 50% 1000 docs 0.84 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 16.33 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7083 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.2359 Interquartile range 0.2359 Mean 0.1633 Standard Deviation 0.2515 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5714 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0928 Std With No Outliers 0.1697 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries HIGeoenenrun1n Topic 026 0.00 Topic 039 0.00 0.8 Topic 027 0.00 Topic 040 57.14 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 66.67 Topic 043 0.00 Topic 031 20.34 Topic 044 13.16 0.4 Topic 032 51.61 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 4.17 Difference Topic 035 0.00 Topic 048 70.83 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 13.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 140 hildesheim HIGeoenenrun2n GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 23,116 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 179 Pooled true Geometric Mean Average Precision 0.0037 Experiment with BRF(5docs,25terms) with NE- Binary Preference (BPREF) 0.1174 recognition and weighting, also within the BRF Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 24.44 HIGeoenenrun2n 10 21.79 90% 20 17.90 80% 30 16.62 40 15.59 70% 50 14.86 Average Precision 60% 60 12.33 70 5.82 50% 80 3.35 40% 90 1.68 30% 100 0.84 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 12.13 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7971 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0238 Third Quartile 0.0577 Interquartile range 0.0577 Mean 0.1213 Standard Deviation 0.2278 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.0595 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0192 Std With No Outliers 0.0223 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries HIGeoenenrun2n Topic 026 0.98 Topic 039 1.14 0.8 Topic 027 0.01 Topic 040 5.95 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 2.82 Topic 042 2.38 Topic 030 70.63 Topic 043 0.00 Topic 031 27.10 Topic 044 5.71 0.4 Topic 032 49.66 Topic 045 1.26 Topic 033 0.00 Topic 046 37.68 0.2 Topic 034 0.00 Topic 047 5.19 Difference Topic 035 0.00 Topic 048 79.71 0 Topic 036 0.00 Topic 049 4.06 Topic 037 0.05 Topic 050 4.48 −0.2 Topic 038 4.35 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 141 hildesheim HIGeoenenrun2n GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 18.40 HIGeoenenrun2n 10 docs 15.20 90% 15 docs 12.53 80% 20 docs 11.60 30 docs 11.60 70% 100 docs 5.32 60% 200 docs 3.10 R−Precision 500 docs 1.39 50% 1000 docs 0.72 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 13.04 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7292 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.1466 Interquartile range 0.1466 Mean 0.1304 Standard Deviation 0.2222 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0584 Std With No Outliers 0.1029 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries HIGeoenenrun2n Topic 026 0.00 Topic 039 0.00 0.8 Topic 027 0.00 Topic 040 14.29 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 66.67 Topic 043 0.00 Topic 031 32.20 Topic 044 15.79 0.4 Topic 032 58.06 Topic 045 0.00 Topic 033 0.00 Topic 046 33.33 0.2 Topic 034 0.00 Topic 047 8.33 Difference Topic 035 0.00 Topic 048 72.92 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 13.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 142 hildesheim HIGeoenenrun3 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 214 Pooled true Geometric Mean Average Precision 0.0070 Experiment with BRF(5docs,20terms) with Binary Preference (BPREF) 0.1641 GeoNEweighting within the BRF-algorithm Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 31.16 HIGeoenenrun3 10 27.90 90% 20 27.18 80% 30 25.60 40 22.05 70% 50 20.71 Average Precision 60% 60 19.23 70 11.95 50% 80 9.30 40% 90 6.56 30% 100 4.04 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 18.75 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8680 Minimum 0.0000 First Quartile 0.0004 Second Quartile 0.0231 Third Quartile 0.3109 Interquartile range 0.3105 Mean 0.1875 Standard Deviation 0.2904 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6457 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1288 Std With No Outliers 0.2169 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries HIGeoenenrun3 Topic 026 0.70 Topic 039 0.61 0.8 Topic 027 0.00 Topic 040 85.60 Topic 028 0.01 Topic 041 0.05 0.6 Topic 029 9.28 Topic 042 0.00 Topic 030 64.57 Topic 043 0.00 Topic 031 30.35 Topic 044 4.13 0.4 Topic 032 60.36 Topic 045 0.30 Topic 033 0.00 Topic 046 62.22 0.2 Topic 034 4.07 Topic 047 6.94 Difference Topic 035 0.13 Topic 048 86.80 0 Topic 036 0.00 Topic 049 16.67 Topic 037 0.23 Topic 050 2.31 −0.2 Topic 038 33.33 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 143 hildesheim HIGeoenenrun3 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 24.00 HIGeoenenrun3 10 docs 19.60 90% 15 docs 18.13 80% 20 docs 16.40 30 docs 12.93 70% 100 docs 5.68 60% 200 docs 3.30 R−Precision 500 docs 1.55 50% 1000 docs 0.86 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 17.85 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7917 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.2599 Interquartile range 0.2599 Mean 0.1785 Standard Deviation 0.2872 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6452 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0739 Std With No Outliers 0.1625 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries HIGeoenenrun3 Topic 026 0.00 Topic 039 0.00 0.8 Topic 027 0.00 Topic 040 78.57 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 22.22 Topic 042 0.00 Topic 030 66.67 Topic 043 0.00 Topic 031 37.29 Topic 044 7.89 0.4 Topic 032 64.52 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 0.00 Topic 047 16.67 Difference Topic 035 0.00 Topic 048 79.17 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 6.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 144 hildesheim HIGeoenenrun1 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 210 Pooled false Geometric Mean Average Precision 0.0080 no BRF base run, stem snowball Binary Preference (BPREF) 0.1544 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 31.94 HIGeoenenrun1 10 27.76 90% 20 25.08 80% 30 23.05 40 20.67 70% 50 19.02 Average Precision 60% 60 17.63 70 7.65 50% 80 5.76 40% 90 5.35 30% 100 4.50 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 16.76 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7728 Minimum 0.0000 First Quartile 0.0013 Second Quartile 0.0118 Third Quartile 0.2704 Interquartile range 0.2692 Mean 0.1676 Standard Deviation 0.2634 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5799 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0909 Std With No Outliers 0.1669 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries HIGeoenenrun1 Topic 026 0.62 Topic 039 0.83 0.8 Topic 027 0.00 Topic 040 77.28 Topic 028 0.16 Topic 041 0.38 0.6 Topic 029 5.66 Topic 042 0.00 Topic 030 57.99 Topic 043 0.02 Topic 031 18.88 Topic 044 3.69 0.4 Topic 032 46.48 Topic 045 0.30 Topic 033 0.00 Topic 046 69.17 0.2 Topic 034 33.16 Topic 047 1.18 Difference Topic 035 0.03 Topic 048 72.40 0 Topic 036 0.00 Topic 049 1.20 Topic 037 0.32 Topic 050 4.16 −0.2 Topic 038 25.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 145 hildesheim HIGeoenenrun1 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 20.80 HIGeoenenrun1 10 docs 17.60 90% 15 docs 16.00 80% 20 docs 14.60 30 docs 11.87 70% 100 docs 4.56 60% 200 docs 2.70 R−Precision 500 docs 1.38 50% 1000 docs 0.84 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 15.95 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6667 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.2359 Interquartile range 0.2359 Mean 0.1595 Standard Deviation 0.2522 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4839 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0641 Std With No Outliers 0.1285 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries HIGeoenenrun1 Topic 026 0.00 Topic 039 0.00 0.8 Topic 027 0.00 Topic 040 64.29 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 66.67 Topic 043 0.00 Topic 031 20.34 Topic 044 10.53 0.4 Topic 032 48.39 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 4.17 Difference Topic 035 0.00 Topic 048 66.67 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 6.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 146 hildesheim HIGeoenenrun2 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 21,673 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 164 Pooled false Geometric Mean Average Precision 0.0018 Experiment with BRF(5docs,25terms) with NE- Binary Preference (BPREF) 0.1140 recognition and weighting, also within the BRF Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 22.44 HIGeoenenrun2 10 19.60 90% 20 18.23 80% 30 16.75 40 15.53 70% 50 15.04 Average Precision 60% 60 12.73 70 5.54 50% 80 2.57 40% 90 1.73 30% 100 0.72 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 11.66 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8007 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0024 Third Quartile 0.0413 Interquartile range 0.0413 Mean 0.1166 Standard Deviation 0.2331 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.0500 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0103 Std With No Outliers 0.0165 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries HIGeoenenrun2 Topic 026 0.89 Topic 039 0.35 0.8 Topic 027 0.03 Topic 040 0.00 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 3.19 Topic 042 0.00 Topic 030 69.12 Topic 043 0.00 Topic 031 28.44 Topic 044 3.57 0.4 Topic 032 50.00 Topic 045 0.23 Topic 033 0.00 Topic 046 43.24 0.2 Topic 034 0.24 Topic 047 2.95 Difference Topic 035 0.00 Topic 048 80.07 0 Topic 036 0.00 Topic 049 0.22 Topic 037 0.01 Topic 050 3.84 −0.2 Topic 038 5.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 147 hildesheim HIGeoenenrun2 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 17.60 HIGeoenenrun2 10 docs 13.60 90% 15 docs 11.47 80% 20 docs 11.00 30 docs 10.67 70% 100 docs 4.64 60% 200 docs 2.78 R−Precision 500 docs 1.26 50% 1000 docs 0.66 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 13.05 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7292 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.0870 Interquartile range 0.0870 Mean 0.1305 Standard Deviation 0.2467 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1111 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0149 Std With No Outliers 0.0327 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries HIGeoenenrun2 Topic 026 0.00 Topic 039 0.00 0.8 Topic 027 0.00 Topic 040 0.00 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 66.67 Topic 043 0.00 Topic 031 32.20 Topic 044 7.89 0.4 Topic 032 58.06 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 0.00 Topic 047 4.17 Difference Topic 035 0.00 Topic 048 72.92 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 6.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 148 imp-coll ICgeoMLtdn GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction MANUAL Retrieved 7,030 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 182 Pooled true Geometric Mean Average Precision 0.0166 String and Geographic terms were taken from the Binary Preference (BPREF) 0.1984 Title, Discription and Narrative and manually parsed into queries with no extra world knowledge Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 42.69 ICgeoMLtdn 10 34.66 90% 20 31.13 80% 30 29.65 40 24.96 70% 50 24.57 Average Precision 60% 60 18.10 70 14.22 50% 80 8.66 40% 90 5.37 30% 100 4.50 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 19.53 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9167 Minimum 0.0000 First Quartile 0.0063 Second Quartile 0.0578 Third Quartile 0.3506 Interquartile range 0.3443 Mean 0.1953 Standard Deviation 0.2353 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5397 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1653 Std With No Outliers 0.1850 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries ICgeoMLtdn Topic 026 0.84 Topic 039 40.21 0.8 Topic 027 2.53 Topic 040 5.78 Topic 028 25.06 Topic 041 0.00 0.6 Topic 029 22.77 Topic 042 0.00 Topic 030 29.64 Topic 043 0.00 Topic 031 2.43 Topic 044 2.63 0.4 Topic 032 42.52 Topic 045 0.69 Topic 033 0.42 Topic 046 91.67 0.2 Topic 034 38.46 Topic 047 18.25 Difference Topic 035 53.97 Topic 048 33.92 0 Topic 036 0.00 Topic 049 50.00 Topic 037 23.09 Topic 050 3.38 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 149 imp-coll ICgeoMLtdn GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 21.60 ICgeoMLtdn 10 docs 18.00 90% 15 docs 17.60 80% 20 docs 16.20 30 docs 12.93 70% 100 docs 6.16 60% 200 docs 3.38 R−Precision 500 docs 1.43 50% 1000 docs 0.73 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 23.55 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6667 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1525 Third Quartile 0.4080 Interquartile range 0.4080 Mean 0.2355 Standard Deviation 0.2213 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2355 Std With No Outliers 0.2213 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries ICgeoMLtdn Topic 026 0.00 Topic 039 50.00 0.8 Topic 027 10.53 Topic 040 14.29 Topic 028 36.84 Topic 041 0.00 0.6 Topic 029 44.44 Topic 042 0.00 Topic 030 33.33 Topic 043 0.00 Topic 031 15.25 Topic 044 5.26 0.4 Topic 032 64.52 Topic 045 0.00 Topic 033 5.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 25.00 Difference Topic 035 50.00 Topic 048 39.58 0 Topic 036 0.00 Topic 049 50.00 Topic 037 31.25 Topic 050 13.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 150 imp-coll ICgeoMLtd GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction MANUAL Retrieved 8,863 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 165 Pooled true Geometric Mean Average Precision 0.0123 String and Geographic terms were taken from the Binary Preference (BPREF) 0.1658 Title and Discription and manually parsed into queries with no extra world knowledge Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 39.84 ICgeoMLtd 10 28.03 90% 20 24.39 80% 30 23.08 40 22.54 70% 50 22.23 Average Precision 60% 60 15.89 70 10.81 50% 80 6.91 40% 90 4.56 30% 100 3.53 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 16.49 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9167 Minimum 0.0000 First Quartile 0.0063 Second Quartile 0.0263 Third Quartile 0.2621 Interquartile range 0.2559 Mean 0.1649 Standard Deviation 0.2456 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1087 Std With No Outliers 0.1531 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries ICgeoMLtd Topic 026 0.84 Topic 039 0.00 0.8 Topic 027 2.53 Topic 040 5.78 Topic 028 25.07 Topic 041 0.00 0.6 Topic 029 22.77 Topic 042 0.86 Topic 030 29.64 Topic 043 0.00 Topic 031 2.43 Topic 044 2.63 0.4 Topic 032 42.52 Topic 045 0.69 Topic 033 0.42 Topic 046 91.67 0.2 Topic 034 70.67 Topic 047 16.04 Difference Topic 035 2.61 Topic 048 33.92 0 Topic 036 0.00 Topic 049 50.00 Topic 037 7.81 Topic 050 3.38 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 151 imp-coll ICgeoMLtd GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 16.00 ICgeoMLtd 10 docs 13.20 90% 15 docs 13.33 80% 20 docs 11.80 30 docs 9.73 70% 100 docs 5.24 60% 200 docs 2.98 R−Precision 500 docs 1.28 50% 1000 docs 0.66 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 19.69 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6667 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1250 Third Quartile 0.3753 Interquartile range 0.3753 Mean 0.1969 Standard Deviation 0.2333 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1969 Std With No Outliers 0.2333 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries ICgeoMLtd Topic 026 0.00 Topic 039 0.00 0.8 Topic 027 10.53 Topic 040 14.29 Topic 028 36.84 Topic 041 0.00 0.6 Topic 029 44.44 Topic 042 0.00 Topic 030 33.33 Topic 043 0.00 Topic 031 15.25 Topic 044 2.63 0.4 Topic 032 64.52 Topic 045 0.00 Topic 033 5.00 Topic 046 66.67 0.2 Topic 034 66.67 Topic 047 16.67 Difference Topic 035 0.00 Topic 048 39.58 0 Topic 036 0.00 Topic 049 50.00 Topic 037 12.50 Topic 050 13.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 152 jaen sinaiEnEnExp3 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 317 Pooled false Geometric Mean Average Precision 0.0465 Expansión con geonames Binary Preference (BPREF) 0.1879 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 40.37 sinaiEnEnExp3 10 32.13 90% 20 27.43 80% 30 26.36 40 25.47 70% 50 24.85 Average Precision 60% 60 24.22 70 18.88 50% 80 18.34 40% 90 14.28 30% 100 11.75 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 22.95 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0234 Second Quartile 0.0823 Third Quartile 0.3049 Interquartile range 0.2815 Mean 0.2295 Standard Deviation 0.3175 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7067 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1311 Std With No Outliers 0.1743 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries sinaiEnEnExp3 Topic 026 0.00 Topic 039 4.83 0.8 Topic 027 3.39 Topic 040 28.43 Topic 028 8.27 Topic 041 2.22 0.6 Topic 029 7.53 Topic 042 3.14 Topic 030 100.00 Topic 043 1.34 Topic 031 13.09 Topic 044 16.59 0.4 Topic 032 94.60 Topic 045 8.23 Topic 033 0.24 Topic 046 70.67 0.2 Topic 034 40.48 Topic 047 6.47 Difference Topic 035 2.38 Topic 048 90.86 0 Topic 036 0.00 Topic 049 36.67 Topic 037 10.02 Topic 050 23.13 −0.2 Topic 038 1.27 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 153 jaen sinaiEnEnExp3 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 24.00 sinaiEnEnExp3 10 docs 19.60 90% 15 docs 17.33 80% 20 docs 16.20 30 docs 15.73 70% 100 docs 7.08 60% 200 docs 4.70 R−Precision 500 docs 2.38 50% 1000 docs 1.27 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 20.28 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0847 Third Quartile 0.2265 Interquartile range 0.2265 Mean 0.2028 Standard Deviation 0.3061 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0799 Std With No Outliers 0.1025 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries sinaiEnEnExp3 Topic 026 0.00 Topic 039 0.00 0.8 Topic 027 10.53 Topic 040 21.43 Topic 028 26.32 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 100.00 Topic 043 0.00 Topic 031 8.47 Topic 044 15.79 0.4 Topic 032 87.10 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 8.33 Difference Topic 035 0.00 Topic 048 85.42 0 Topic 036 0.00 Topic 049 0.00 Topic 037 12.50 Topic 050 20.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 154 jaen sinaiEnEnExp1 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 291 Pooled true Geometric Mean Average Precision 0.0482 Caso base Binary Preference (BPREF) 0.2907 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 55.37 sinaiEnEnExp1 10 50.26 90% 20 39.83 80% 30 39.10 40 36.40 70% 50 35.78 Average Precision 60% 60 28.73 70 24.12 50% 80 23.30 40% 90 19.80 30% 100 17.44 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 32.24 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0111 Second Quartile 0.1533 Third Quartile 0.6417 Interquartile range 0.6306 Mean 0.3224 Standard Deviation 0.3685 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 1.0000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.3224 Std With No Outliers 0.3685 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries sinaiEnEnExp1 Topic 026 15.33 Topic 039 37.12 0.8 Topic 027 4.32 Topic 040 31.81 Topic 028 0.18 Topic 041 2.05 0.6 Topic 029 11.49 Topic 042 100.00 Topic 030 94.84 Topic 043 2.39 Topic 031 25.61 Topic 044 8.85 0.4 Topic 032 96.05 Topic 045 64.13 Topic 033 0.15 Topic 046 86.67 0.2 Topic 034 50.62 Topic 047 0.05 Difference Topic 035 1.34 Topic 048 90.47 0 Topic 036 0.00 Topic 049 64.29 Topic 037 0.01 Topic 050 17.70 −0.2 Topic 038 0.40 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 155 jaen sinaiEnEnExp1 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 29.60 sinaiEnEnExp1 10 docs 22.40 90% 15 docs 20.27 80% 20 docs 18.80 30 docs 16.00 70% 100 docs 6.92 60% 200 docs 4.40 R−Precision 500 docs 2.14 50% 1000 docs 1.16 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 29.34 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1316 Third Quartile 0.5000 Interquartile range 0.5000 Mean 0.2934 Standard Deviation 0.3329 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 1.0000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2934 Std With No Outliers 0.3329 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries sinaiEnEnExp1 Topic 026 11.11 Topic 039 43.75 0.8 Topic 027 10.53 Topic 040 42.86 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 100.00 Topic 030 83.33 Topic 043 0.00 Topic 031 22.03 Topic 044 13.16 0.4 Topic 032 90.32 Topic 045 50.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 85.42 0 Topic 036 0.00 Topic 049 50.00 Topic 037 0.00 Topic 050 20.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 156 jaen sinaiEnEnExp2 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 323 Pooled true Geometric Mean Average Precision 0.0594 Caso base Binary Preference (BPREF) 0.2039 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 44.41 sinaiEnEnExp2 10 37.60 90% 20 33.38 80% 30 29.48 40 27.16 70% 50 26.25 Average Precision 60% 60 25.50 70 19.70 50% 80 18.99 40% 90 14.22 30% 100 11.77 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 25.04 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9762 Minimum 0.0000 First Quartile 0.0295 Second Quartile 0.1002 Third Quartile 0.3065 Interquartile range 0.2770 Mean 0.2504 Standard Deviation 0.3105 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7067 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1554 Std With No Outliers 0.1770 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries sinaiEnEnExp2 Topic 026 28.64 Topic 039 4.83 0.8 Topic 027 4.42 Topic 040 28.43 Topic 028 8.27 Topic 041 2.22 0.6 Topic 029 3.99 Topic 042 3.14 Topic 030 97.62 Topic 043 1.34 Topic 031 25.33 Topic 044 21.97 0.4 Topic 032 95.56 Topic 045 18.33 Topic 033 0.00 Topic 046 70.67 0.2 Topic 034 40.48 Topic 047 6.47 Difference Topic 035 2.38 Topic 048 90.86 0 Topic 036 0.00 Topic 049 36.67 Topic 037 10.02 Topic 050 23.13 −0.2 Topic 038 1.27 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 157 jaen sinaiEnEnExp2 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 26.40 sinaiEnEnExp2 10 docs 22.80 90% 15 docs 19.73 80% 20 docs 18.80 30 docs 17.20 70% 100 docs 7.92 60% 200 docs 5.22 R−Precision 500 docs 2.46 50% 1000 docs 1.29 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 21.94 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8710 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1111 Third Quartile 0.2807 Interquartile range 0.2807 Mean 0.2194 Standard Deviation 0.2863 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1330 Std With No Outliers 0.1689 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries sinaiEnEnExp2 Topic 026 33.33 Topic 039 0.00 0.8 Topic 027 10.53 Topic 040 21.43 Topic 028 26.32 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 83.33 Topic 043 0.00 Topic 031 25.42 Topic 044 23.68 0.4 Topic 032 87.10 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 8.33 Difference Topic 035 0.00 Topic 048 85.42 0 Topic 036 0.00 Topic 049 0.00 Topic 037 12.50 Topic 050 20.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 158 jaen sinaiEnEnExp4 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 322 Pooled false Geometric Mean Average Precision 0.0660 Expansión con tesauro Binary Preference (BPREF) 0.2102 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 47.34 sinaiEnEnExp4 10 40.25 90% 20 36.31 80% 30 32.43 40 27.67 70% 50 26.80 Average Precision 60% 60 26.06 70 19.56 50% 80 18.82 40% 90 14.10 30% 100 11.72 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 26.11 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9762 Minimum 0.0000 First Quartile 0.0405 Second Quartile 0.1002 Third Quartile 0.3065 Interquartile range 0.2660 Mean 0.2611 Standard Deviation 0.3112 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5435 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1419 Std With No Outliers 0.1439 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries sinaiEnEnExp4 Topic 026 28.64 Topic 039 4.74 0.8 Topic 027 4.07 Topic 040 28.43 Topic 028 8.27 Topic 041 2.22 0.6 Topic 029 3.99 Topic 042 9.55 Topic 030 97.62 Topic 043 1.34 Topic 031 25.33 Topic 044 21.97 0.4 Topic 032 95.56 Topic 045 18.33 Topic 033 0.00 Topic 046 70.67 0.2 Topic 034 54.35 Topic 047 6.47 Difference Topic 035 9.18 Topic 048 90.86 0 Topic 036 0.00 Topic 049 36.67 Topic 037 10.02 Topic 050 23.13 −0.2 Topic 038 1.27 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 159 jaen sinaiEnEnExp4 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 26.40 sinaiEnEnExp4 10 docs 23.20 90% 15 docs 20.27 80% 20 docs 19.40 30 docs 17.60 70% 100 docs 8.00 60% 200 docs 5.18 R−Precision 500 docs 2.46 50% 1000 docs 1.29 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 22.61 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8710 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1250 Third Quartile 0.2807 Interquartile range 0.2807 Mean 0.2261 Standard Deviation 0.2829 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1406 Std With No Outliers 0.1664 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries sinaiEnEnExp4 Topic 026 33.33 Topic 039 0.00 0.8 Topic 027 10.53 Topic 040 21.43 Topic 028 26.32 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 83.33 Topic 043 0.00 Topic 031 25.42 Topic 044 23.68 0.4 Topic 032 87.10 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 8.33 Difference Topic 035 16.67 Topic 048 85.42 0 Topic 036 0.00 Topic 049 0.00 Topic 037 12.50 Topic 050 20.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 160 jaen sinaiEnEnExp5 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 319 Pooled false Geometric Mean Average Precision 0.0533 Expansión con geonames y tesauro Binary Preference (BPREF) 0.1945 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 44.00 sinaiEnEnExp5 10 35.03 90% 20 30.37 80% 30 29.30 40 26.05 70% 50 25.39 Average Precision 60% 60 24.78 70 18.74 50% 80 18.17 40% 90 14.16 30% 100 11.70 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 24.07 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0373 Second Quartile 0.0918 Third Quartile 0.3049 Interquartile range 0.2676 Mean 0.2407 Standard Deviation 0.3186 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5435 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1170 Std With No Outliers 0.1373 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries sinaiEnEnExp5 Topic 026 0.00 Topic 039 4.74 0.8 Topic 027 4.24 Topic 040 28.43 Topic 028 8.27 Topic 041 2.22 0.6 Topic 029 7.53 Topic 042 9.55 Topic 030 100.00 Topic 043 1.34 Topic 031 13.09 Topic 044 16.59 0.4 Topic 032 94.60 Topic 045 8.23 Topic 033 0.40 Topic 046 70.67 0.2 Topic 034 54.35 Topic 047 6.47 Difference Topic 035 9.18 Topic 048 90.86 0 Topic 036 0.00 Topic 049 36.67 Topic 037 10.02 Topic 050 23.13 −0.2 Topic 038 1.27 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 161 jaen sinaiEnEnExp5 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 24.00 sinaiEnEnExp5 10 docs 20.00 90% 15 docs 17.87 80% 20 docs 16.80 30 docs 16.13 70% 100 docs 7.20 60% 200 docs 4.70 R−Precision 500 docs 2.38 50% 1000 docs 1.28 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 20.95 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1053 Third Quartile 0.2265 Interquartile range 0.2265 Mean 0.2095 Standard Deviation 0.3033 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0878 Std With No Outliers 0.1025 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries sinaiEnEnExp5 Topic 026 0.00 Topic 039 0.00 0.8 Topic 027 10.53 Topic 040 21.43 Topic 028 26.32 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 100.00 Topic 043 0.00 Topic 031 8.47 Topic 044 15.79 0.4 Topic 032 87.10 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 8.33 Difference Topic 035 16.67 Topic 048 85.42 0 Topic 036 0.00 Topic 049 0.00 Topic 037 12.50 Topic 050 20.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 162 ms-china msramanual GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction MANUAL Retrieved 5,258 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 187 Pooled true Geometric Mean Average Precision 0.0513 Geoclef 2006 English queries using geo knowledge Binary Preference (BPREF) 0.2279 base and manual query construction Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 55.00 msramanual 10 52.27 90% 20 42.80 80% 30 35.54 40 32.15 70% 50 30.81 Average Precision 60% 60 17.23 70 9.02 50% 80 5.69 40% 90 2.24 30% 100 2.24 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 23.95 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7500 Minimum 0.0000 First Quartile 0.0239 Second Quartile 0.1622 Third Quartile 0.4063 Interquartile range 0.3823 Mean 0.2395 Standard Deviation 0.2344 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7500 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2395 Std With No Outliers 0.2344 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries msramanual Topic 026 34.52 Topic 039 41.67 0.8 Topic 027 8.87 Topic 040 0.02 Topic 028 21.98 Topic 041 5.00 0.6 Topic 029 54.85 Topic 042 75.00 Topic 030 40.28 Topic 043 0.34 Topic 031 1.44 Topic 044 11.04 0.4 Topic 032 64.43 Topic 045 26.96 Topic 033 10.00 Topic 046 0.00 0.2 Topic 034 38.89 Topic 047 2.71 Difference Topic 035 16.22 Topic 048 59.89 0 Topic 036 0.00 Topic 049 50.00 Topic 037 27.31 Topic 050 1.41 −0.2 Topic 038 5.88 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 163 ms-china msramanual GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 34.40 msramanual 10 docs 23.20 90% 15 docs 20.00 80% 20 docs 18.00 30 docs 15.87 70% 100 docs 6.40 60% 200 docs 3.38 R−Precision 500 docs 1.48 50% 1000 docs 0.75 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 25.45 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6774 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1667 Third Quartile 0.5000 Interquartile range 0.5000 Mean 0.2545 Standard Deviation 0.2541 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6774 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2545 Std With No Outliers 0.2541 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries msramanual Topic 026 33.33 Topic 039 50.00 0.8 Topic 027 5.26 Topic 040 0.00 Topic 028 26.32 Topic 041 0.00 0.6 Topic 029 66.67 Topic 042 50.00 Topic 030 50.00 Topic 043 0.00 Topic 031 8.47 Topic 044 18.42 0.4 Topic 032 67.74 Topic 045 16.67 Topic 033 10.00 Topic 046 0.00 0.2 Topic 034 66.67 Topic 047 0.00 Difference Topic 035 16.67 Topic 048 62.50 0 Topic 036 0.00 Topic 049 50.00 Topic 037 37.50 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 164 ms-china msrawhitelist GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 4,906 Source Language English Relevant 378 Topic Fields title Relevant retrieved 172 Pooled true Geometric Mean Average Precision 0.0309 Geoclef 2006 English queries using geo knowledge Binary Preference (BPREF) 0.2078 base Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 53.81 msrawhitelist 10 46.86 90% 20 36.93 80% 30 30.41 40 27.72 70% 50 26.02 Average Precision 60% 60 12.06 70 5.08 50% 80 4.17 40% 90 1.09 30% 100 1.09 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 20.00 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6284 Minimum 0.0000 First Quartile 0.0125 Second Quartile 0.1087 Third Quartile 0.3322 Interquartile range 0.3197 Mean 0.2000 Standard Deviation 0.2081 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6284 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2000 Std With No Outliers 0.2081 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries msrawhitelist Topic 026 20.19 Topic 039 31.33 0.8 Topic 027 8.87 Topic 040 0.02 Topic 028 14.10 Topic 041 25.00 0.6 Topic 029 28.89 Topic 042 60.00 Topic 030 40.28 Topic 043 0.34 Topic 031 1.54 Topic 044 21.72 0.4 Topic 032 62.84 Topic 045 0.76 Topic 033 10.00 Topic 046 0.00 0.2 Topic 034 38.89 Topic 047 8.77 Difference Topic 035 0.00 Topic 048 57.10 0 Topic 036 0.00 Topic 049 50.00 Topic 037 10.87 Topic 050 1.41 −0.2 Topic 038 7.14 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 165 ms-china msrawhitelist GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 29.60 msrawhitelist 10 docs 21.20 90% 15 docs 18.13 80% 20 docs 16.80 30 docs 14.53 70% 100 docs 5.92 60% 200 docs 3.20 R−Precision 500 docs 1.36 50% 1000 docs 0.69 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 23.52 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6774 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.2105 Third Quartile 0.4583 Interquartile range 0.4583 Mean 0.2352 Standard Deviation 0.2369 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6774 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2352 Std With No Outliers 0.2369 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries msrawhitelist Topic 026 33.33 Topic 039 31.25 0.8 Topic 027 5.26 Topic 040 0.00 Topic 028 21.05 Topic 041 25.00 0.6 Topic 029 44.44 Topic 042 50.00 Topic 030 50.00 Topic 043 0.00 Topic 031 8.47 Topic 044 28.95 0.4 Topic 032 67.74 Topic 045 0.00 Topic 033 10.00 Topic 046 0.00 0.2 Topic 034 66.67 Topic 047 8.33 Difference Topic 035 0.00 Topic 048 62.50 0 Topic 036 0.00 Topic 049 50.00 Topic 037 25.00 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 166 ms-china msraexpansion GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 6,166 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 146 Pooled false Geometric Mean Average Precision 0.0112 msraexpansion Binary Preference (BPREF) 0.1730 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 42.10 msraexpansion 10 40.08 90% 20 31.38 80% 30 28.60 40 18.10 70% 50 16.43 Average Precision 60% 60 6.57 70 2.95 50% 80 2.32 40% 90 0.10 30% 100 0.10 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 15.21 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6284 Minimum 0.0000 First Quartile 0.0020 Second Quartile 0.0428 Third Quartile 0.3000 Interquartile range 0.2980 Mean 0.1521 Standard Deviation 0.1987 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6284 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1521 Std With No Outliers 0.1987 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries msraexpansion Topic 026 20.19 Topic 039 0.74 0.8 Topic 027 8.87 Topic 040 0.02 Topic 028 14.10 Topic 041 0.00 0.6 Topic 029 28.89 Topic 042 0.00 Topic 030 40.28 Topic 043 0.26 Topic 031 4.28 Topic 044 3.08 0.4 Topic 032 62.84 Topic 045 0.00 Topic 033 10.00 Topic 046 33.33 0.2 Topic 034 33.33 Topic 047 2.59 Difference Topic 035 0.00 Topic 048 57.10 0 Topic 036 0.00 Topic 049 50.00 Topic 037 6.67 Topic 050 1.29 −0.2 Topic 038 2.50 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 167 ms-china msraexpansion GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 23.20 msraexpansion 10 docs 16.40 90% 15 docs 13.87 80% 20 docs 13.00 30 docs 11.73 70% 100 docs 5.24 60% 200 docs 2.72 R−Precision 500 docs 1.14 50% 1000 docs 0.58 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 18.53 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6774 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0667 Third Quartile 0.3333 Interquartile range 0.3333 Mean 0.1853 Standard Deviation 0.2210 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6774 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1853 Std With No Outliers 0.2210 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries msraexpansion Topic 026 33.33 Topic 039 0.00 0.8 Topic 027 5.26 Topic 040 0.00 Topic 028 21.05 Topic 041 0.00 0.6 Topic 029 44.44 Topic 042 0.00 Topic 030 50.00 Topic 043 0.00 Topic 031 15.25 Topic 044 5.26 0.4 Topic 032 67.74 Topic 045 0.00 Topic 033 10.00 Topic 046 33.33 0.2 Topic 034 33.33 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 62.50 0 Topic 036 0.00 Topic 049 50.00 Topic 037 25.00 Topic 050 6.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 168 ms-china msralocal GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 9,129 Source Language English Relevant 378 Topic Fields title Relevant retrieved 183 Pooled false Geometric Mean Average Precision 0.0284 Geoclef 2006 English queries without geo knowledge Binary Preference (BPREF) 0.1966 base or query expansion Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 49.68 msralocal 10 44.32 90% 20 34.79 80% 30 28.96 40 26.40 70% 50 24.54 Average Precision 60% 60 10.65 70 3.48 50% 80 2.74 40% 90 0.41 30% 100 0.41 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 18.37 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6284 Minimum 0.0000 First Quartile 0.0151 Second Quartile 0.1000 Third Quartile 0.3139 Interquartile range 0.2988 Mean 0.1837 Standard Deviation 0.2043 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6284 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1837 Std With No Outliers 0.2043 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries msralocal Topic 026 20.19 Topic 039 10.62 0.8 Topic 027 8.87 Topic 040 0.02 Topic 028 14.10 Topic 041 25.00 0.6 Topic 029 28.89 Topic 042 54.00 Topic 030 40.28 Topic 043 0.23 Topic 031 1.54 Topic 044 13.90 0.4 Topic 032 62.84 Topic 045 0.00 Topic 033 10.00 Topic 046 0.00 0.2 Topic 034 38.89 Topic 047 8.77 Difference Topic 035 3.91 Topic 048 57.10 0 Topic 036 0.00 Topic 049 50.00 Topic 037 6.45 Topic 050 1.41 −0.2 Topic 038 2.13 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 169 ms-china msralocal GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 27.20 msralocal 10 docs 19.20 90% 15 docs 16.80 80% 20 docs 15.40 30 docs 13.07 70% 100 docs 5.64 60% 200 docs 3.12 R−Precision 500 docs 1.38 50% 1000 docs 0.73 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 22.45 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6774 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1875 Third Quartile 0.4583 Interquartile range 0.4583 Mean 0.2245 Standard Deviation 0.2363 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6774 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2245 Std With No Outliers 0.2363 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries msralocal Topic 026 33.33 Topic 039 18.75 0.8 Topic 027 5.26 Topic 040 0.00 Topic 028 21.05 Topic 041 25.00 0.6 Topic 029 44.44 Topic 042 50.00 Topic 030 50.00 Topic 043 0.00 Topic 031 8.47 Topic 044 21.05 0.4 Topic 032 67.74 Topic 045 0.00 Topic 033 10.00 Topic 046 0.00 0.2 Topic 034 66.67 Topic 047 8.33 Difference Topic 035 0.00 Topic 048 62.50 0 Topic 036 0.00 Topic 049 50.00 Topic 037 18.75 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 170 ms-china msratext GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 227 Pooled true Geometric Mean Average Precision 0.0176 Geoclef 2006 English queries using pure text Binary Preference (BPREF) 0.1754 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 43.99 msratext 10 39.63 90% 20 30.03 80% 30 27.86 40 22.86 70% 50 21.26 Average Precision 60% 60 11.78 70 6.73 50% 80 5.38 40% 90 3.41 30% 100 2.14 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 18.35 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8655 Minimum 0.0000 First Quartile 0.0031 Second Quartile 0.1378 Third Quartile 0.2361 Interquartile range 0.2330 Mean 0.1835 Standard Deviation 0.2372 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5017 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1132 Std With No Outliers 0.1386 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries msratext Topic 026 21.17 Topic 039 36.72 0.8 Topic 027 3.25 Topic 040 0.01 Topic 028 0.03 Topic 041 0.00 0.6 Topic 029 14.37 Topic 042 50.17 Topic 030 15.06 Topic 043 0.36 Topic 031 18.97 Topic 044 13.78 0.4 Topic 032 86.55 Topic 045 61.66 Topic 033 0.33 Topic 046 11.22 0.2 Topic 034 14.01 Topic 047 0.43 Difference Topic 035 1.77 Topic 048 61.49 0 Topic 036 0.00 Topic 049 16.20 Topic 037 0.25 Topic 050 30.94 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 171 ms-china msratext GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 24.00 msratext 10 docs 19.60 90% 15 docs 17.87 80% 20 docs 16.80 30 docs 13.73 70% 100 docs 6.04 60% 200 docs 3.64 R−Precision 500 docs 1.70 50% 1000 docs 0.91 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 21.23 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7419 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1111 Third Quartile 0.3797 Interquartile range 0.3797 Mean 0.2123 Standard Deviation 0.2387 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7419 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2123 Std With No Outliers 0.2387 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries msratext Topic 026 22.22 Topic 039 43.75 0.8 Topic 027 10.53 Topic 040 0.00 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 50.00 Topic 030 16.67 Topic 043 0.00 Topic 031 37.29 Topic 044 26.32 0.4 Topic 032 74.19 Topic 045 66.67 Topic 033 5.00 Topic 046 33.33 0.2 Topic 034 33.33 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 60.42 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 40.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 172 nicta MuTdnManQexpGeo GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction MANUAL Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 308 Pooled false Geometric Mean Average Precision 0.0580 title + desc + narr-exp + title-manexp, geo-query Binary Preference (BPREF) 0.2050 (only for title and desc), with manual title expansion Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 47.07 MuTdnManQexpGeo 10 41.49 90% 20 34.63 80% 30 32.91 40 31.48 70% 50 29.63 Average Precision 60% 60 19.88 70 13.74 50% 80 11.55 40% 90 8.74 30% 100 7.32 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 24.00 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8807 Minimum 0.0000 First Quartile 0.0175 Second Quartile 0.1407 Third Quartile 0.3799 Interquartile range 0.3625 Mean 0.2400 Standard Deviation 0.2692 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8807 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2400 Std With No Outliers 0.2692 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries MuTdnManQexpGeo Topic 026 14.66 Topic 039 9.31 0.8 Topic 027 4.33 Topic 040 32.12 Topic 028 0.12 Topic 041 0.14 0.6 Topic 029 18.38 Topic 042 58.33 Topic 030 59.84 Topic 043 0.82 Topic 031 35.89 Topic 044 14.07 0.4 Topic 032 88.07 Topic 045 8.86 Topic 033 2.00 Topic 046 75.00 0.2 Topic 034 44.29 Topic 047 3.81 Difference Topic 035 4.84 Topic 048 67.52 0 Topic 036 0.00 Topic 049 25.33 Topic 037 0.29 Topic 050 31.04 −0.2 Topic 038 0.97 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 173 nicta MuTdnManQexpGeo GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 25.60 MuTdnManQexpGeo 10 docs 21.20 90% 15 docs 18.93 80% 20 docs 18.00 30 docs 15.20 70% 100 docs 7.88 60% 200 docs 4.72 R−Precision 500 docs 2.22 50% 1000 docs 1.23 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 23.00 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7742 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1250 Third Quartile 0.4301 Interquartile range 0.4301 Mean 0.2300 Standard Deviation 0.2472 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7742 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2300 Std With No Outliers 0.2472 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries MuTdnManQexpGeo Topic 026 22.22 Topic 039 12.50 0.8 Topic 027 10.53 Topic 040 28.57 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 50.00 Topic 030 50.00 Topic 043 0.00 Topic 031 40.68 Topic 044 21.05 0.4 Topic 032 77.42 Topic 045 0.00 Topic 033 5.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 62.50 0 Topic 036 0.00 Topic 049 50.00 Topic 037 0.00 Topic 050 33.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 174 nicta MuTdnTxt GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 308 Pooled true Geometric Mean Average Precision 0.0760 Baseline Zetair: title + desc + narr-exp, text only Binary Preference (BPREF) 0.1993 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 54.38 MuTdnTxt 10 42.31 90% 20 32.76 80% 30 32.10 40 30.46 70% 50 29.11 Average Precision 60% 60 20.53 70 16.38 50% 80 13.20 40% 90 9.64 30% 100 8.27 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 24.44 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9019 Minimum 0.0000 First Quartile 0.0353 Second Quartile 0.1781 Third Quartile 0.3602 Interquartile range 0.3248 Mean 0.2444 Standard Deviation 0.2552 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7436 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2170 Std With No Outliers 0.2200 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries MuTdnTxt Topic 026 12.19 Topic 039 20.92 0.8 Topic 027 3.41 Topic 040 35.66 Topic 028 0.47 Topic 041 0.14 0.6 Topic 029 24.25 Topic 042 54.76 Topic 030 24.61 Topic 043 0.81 Topic 031 37.09 Topic 044 15.74 0.4 Topic 032 90.19 Topic 045 17.81 Topic 033 1.66 Topic 046 74.36 0.2 Topic 034 44.29 Topic 047 4.71 Difference Topic 035 4.31 Topic 048 69.29 0 Topic 036 0.00 Topic 049 33.33 Topic 037 7.17 Topic 050 30.36 −0.2 Topic 038 3.57 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 175 nicta MuTdnTxt GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 26.40 MuTdnTxt 10 docs 21.60 90% 15 docs 19.73 80% 20 docs 18.80 30 docs 16.53 70% 100 docs 8.44 60% 200 docs 4.94 R−Precision 500 docs 2.26 50% 1000 docs 1.23 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 21.84 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7419 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1111 Third Quartile 0.3333 Interquartile range 0.3333 Mean 0.2184 Standard Deviation 0.2354 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7419 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2184 Std With No Outliers 0.2354 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries MuTdnTxt Topic 026 11.11 Topic 039 18.75 0.8 Topic 027 5.26 Topic 040 28.57 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 50.00 Topic 030 33.33 Topic 043 0.00 Topic 031 32.20 Topic 044 21.05 0.4 Topic 032 74.19 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 8.33 Difference Topic 035 0.00 Topic 048 62.50 0 Topic 036 0.00 Topic 049 50.00 Topic 037 6.25 Topic 050 33.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 176 nicta MuTdQexpPrb GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 291 Pooled true Geometric Mean Average Precision 0.0626 title + desc, with automatic geographic query Binary Preference (BPREF) 0.1898 expansion, using probabilistic geo-index Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 42.04 MuTdQexpPrb 10 37.33 90% 20 31.67 80% 30 29.96 40 28.86 70% 50 26.88 Average Precision 60% 60 20.08 70 13.68 50% 80 11.53 40% 90 9.05 30% 100 7.77 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 22.18 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8763 Minimum 0.0000 First Quartile 0.0263 Second Quartile 0.0970 Third Quartile 0.3314 Interquartile range 0.3052 Mean 0.2218 Standard Deviation 0.2615 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7500 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1945 Std With No Outliers 0.2279 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries MuTdQexpPrb Topic 026 13.38 Topic 039 6.22 0.8 Topic 027 3.16 Topic 040 31.61 Topic 028 6.11 Topic 041 0.35 0.6 Topic 029 18.78 Topic 042 9.70 Topic 030 55.93 Topic 043 0.71 Topic 031 37.75 Topic 044 15.16 0.4 Topic 032 87.63 Topic 045 5.96 Topic 033 0.41 Topic 046 75.00 0.2 Topic 034 46.67 Topic 047 5.27 Difference Topic 035 4.47 Topic 048 70.71 0 Topic 036 0.00 Topic 049 28.70 Topic 037 1.05 Topic 050 28.68 −0.2 Topic 038 1.04 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 177 nicta MuTdQexpPrb GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 22.40 MuTdQexpPrb 10 docs 21.20 90% 15 docs 19.20 80% 20 docs 18.40 30 docs 16.00 70% 100 docs 8.16 60% 200 docs 5.00 R−Precision 500 docs 2.14 50% 1000 docs 1.16 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 22.40 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7742 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1579 Third Quartile 0.3924 Interquartile range 0.3924 Mean 0.2240 Standard Deviation 0.2438 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7742 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2240 Std With No Outliers 0.2438 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries MuTdQexpPrb Topic 026 22.22 Topic 039 12.50 0.8 Topic 027 5.26 Topic 040 28.57 Topic 028 15.79 Topic 041 0.00 0.6 Topic 029 22.22 Topic 042 0.00 Topic 030 50.00 Topic 043 0.00 Topic 031 38.98 Topic 044 26.32 0.4 Topic 032 77.42 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 64.58 0 Topic 036 0.00 Topic 049 50.00 Topic 037 6.25 Topic 050 40.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 178 nicta MuTdRedn GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 293 Pooled false Geometric Mean Average Precision 0.0648 title + desc, with document expansion no query Binary Preference (BPREF) 0.1870 expansion Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 43.70 MuTdRedn 10 39.58 90% 20 32.18 80% 30 30.93 40 29.93 70% 50 28.08 Average Precision 60% 60 22.13 70 15.73 50% 80 13.06 40% 90 10.46 30% 100 9.32 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 23.41 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8843 Minimum 0.0000 First Quartile 0.0192 Second Quartile 0.1516 Third Quartile 0.3406 Interquartile range 0.3214 Mean 0.2341 Standard Deviation 0.2650 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7576 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2071 Std With No Outliers 0.2327 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries MuTdRedn Topic 026 18.19 Topic 039 4.94 0.8 Topic 027 2.21 Topic 040 31.61 Topic 028 4.72 Topic 041 0.35 0.6 Topic 029 18.82 Topic 042 29.17 Topic 030 58.76 Topic 043 0.71 Topic 031 38.17 Topic 044 15.16 0.4 Topic 032 88.43 Topic 045 6.76 Topic 033 0.41 Topic 046 75.76 0.2 Topic 034 46.67 Topic 047 5.08 Difference Topic 035 4.47 Topic 048 71.52 0 Topic 036 0.00 Topic 049 32.69 Topic 037 1.05 Topic 050 28.68 −0.2 Topic 038 1.04 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 179 nicta MuTdRedn GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 24.00 MuTdRedn 10 docs 20.00 90% 15 docs 19.20 80% 20 docs 18.60 30 docs 15.73 70% 100 docs 8.28 60% 200 docs 4.96 R−Precision 500 docs 2.14 50% 1000 docs 1.17 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 21.92 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8065 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1111 Third Quartile 0.3924 Interquartile range 0.3924 Mean 0.2192 Standard Deviation 0.2507 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8065 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2192 Std With No Outliers 0.2507 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries MuTdRedn Topic 026 11.11 Topic 039 6.25 0.8 Topic 027 5.26 Topic 040 28.57 Topic 028 15.79 Topic 041 0.00 0.6 Topic 029 22.22 Topic 042 0.00 Topic 030 50.00 Topic 043 0.00 Topic 031 38.98 Topic 044 26.32 0.4 Topic 032 80.65 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 66.67 0 Topic 036 0.00 Topic 049 50.00 Topic 037 6.25 Topic 050 40.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 180 nicta MuTdTxt GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 301 Pooled false Geometric Mean Average Precision 0.0773 Baseline Zetair: title + desc, text only Binary Preference (BPREF) 0.1943 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 48.88 MuTdTxt 10 41.16 90% 20 32.71 80% 30 30.52 40 29.18 70% 50 27.95 Average Precision 60% 60 20.49 70 14.73 50% 80 11.66 40% 90 8.23 30% 100 7.11 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 23.12 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8980 Minimum 0.0000 First Quartile 0.0523 Second Quartile 0.1146 Third Quartile 0.3523 Interquartile range 0.3000 Mean 0.2312 Standard Deviation 0.2568 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7265 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2034 Std With No Outliers 0.2206 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 6 Number of Topics of the Experiment 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries MuTdTxt Topic 026 12.19 Topic 039 8.35 0.8 Topic 027 2.41 Topic 040 35.66 Topic 028 7.38 Topic 041 0.41 0.6 Topic 029 24.25 Topic 042 11.46 Topic 030 20.27 Topic 043 0.71 Topic 031 35.09 Topic 044 17.63 0.4 Topic 032 89.80 Topic 045 6.75 Topic 033 0.53 Topic 046 70.83 0.2 Topic 034 44.29 Topic 047 7.70 Difference Topic 035 4.25 Topic 048 72.65 0 Topic 036 0.00 Topic 049 58.33 Topic 037 10.13 Topic 050 31.43 −0.2 Topic 038 5.56 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 181 nicta MuTdTxt GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 25.60 MuTdTxt 10 docs 22.00 90% 15 docs 20.53 80% 20 docs 20.00 30 docs 17.07 70% 100 docs 8.64 60% 200 docs 5.00 R−Precision 500 docs 2.22 50% 1000 docs 1.20 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 21.55 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7742 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1667 Third Quartile 0.3121 Interquartile range 0.3121 Mean 0.2155 Standard Deviation 0.2358 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7742 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2155 Std With No Outliers 0.2358 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries MuTdTxt Topic 026 11.11 Topic 039 18.75 0.8 Topic 027 5.26 Topic 040 28.57 Topic 028 21.05 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 16.67 Topic 043 0.00 Topic 031 30.51 Topic 044 26.32 0.4 Topic 032 77.42 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 12.50 Difference Topic 035 0.00 Topic 048 70.83 0 Topic 036 0.00 Topic 049 50.00 Topic 037 18.75 Topic 050 40.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 182 rfia-upv rfiaUPV01 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 298 Pooled false Geometric Mean Average Precision 0.0689 Base system without GITE nor WN Binary Preference (BPREF) 0.2218 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 47.56 rfiaUPV01 10 39.05 90% 20 34.46 80% 30 33.01 40 29.95 70% 50 28.85 Average Precision 60% 60 23.28 70 17.51 50% 80 13.85 40% 90 10.74 30% 100 9.18 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 25.07 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8511 Minimum 0.0000 First Quartile 0.0266 Second Quartile 0.0995 Third Quartile 0.4107 Interquartile range 0.3841 Mean 0.2507 Standard Deviation 0.2946 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8511 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2507 Std With No Outliers 0.2946 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries rfiaUPV01 Topic 026 11.21 Topic 039 7.25 0.8 Topic 027 1.02 Topic 040 36.25 Topic 028 5.45 Topic 041 0.37 0.6 Topic 029 14.38 Topic 042 2.29 Topic 030 76.95 Topic 043 1.70 Topic 031 36.07 Topic 044 18.22 0.4 Topic 032 85.11 Topic 045 2.79 Topic 033 0.21 Topic 046 72.22 0.2 Topic 034 72.22 Topic 047 3.56 Difference Topic 035 9.95 Topic 048 75.66 0 Topic 036 0.00 Topic 049 55.56 Topic 037 6.59 Topic 050 22.68 −0.2 Topic 038 9.09 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 183 rfia-upv rfiaUPV01 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 25.60 rfiaUPV01 10 docs 21.20 90% 15 docs 19.47 80% 20 docs 19.00 30 docs 16.53 70% 100 docs 8.36 60% 200 docs 4.88 R−Precision 500 docs 2.22 50% 1000 docs 1.19 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 24.18 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7419 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1250 Third Quartile 0.4428 Interquartile range 0.4428 Mean 0.2418 Standard Deviation 0.2656 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7419 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2418 Std With No Outliers 0.2656 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries rfiaUPV01 Topic 026 11.11 Topic 039 18.75 0.8 Topic 027 5.26 Topic 040 28.57 Topic 028 10.53 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 66.67 Topic 043 0.00 Topic 031 42.37 Topic 044 26.32 0.4 Topic 032 74.19 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 66.67 Topic 047 4.17 Difference Topic 035 16.67 Topic 048 72.92 0 Topic 036 0.00 Topic 049 50.00 Topic 037 12.50 Topic 050 20.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 184 rfia-upv rfiaUPV02 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 303 Pooled false Geometric Mean Average Precision 0.0735 All Fields without GITE nor WN Binary Preference (BPREF) 0.2388 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 52.27 rfiaUPV02 10 44.75 90% 20 37.94 80% 30 37.54 40 34.77 70% 50 33.66 Average Precision 60% 60 25.23 70 16.83 50% 80 15.33 40% 90 10.02 30% 100 8.21 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 27.35 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8569 Minimum 0.0000 First Quartile 0.0183 Second Quartile 0.1960 Third Quartile 0.4531 Interquartile range 0.4348 Mean 0.2735 Standard Deviation 0.2852 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8569 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2735 Std With No Outliers 0.2852 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries rfiaUPV02 Topic 026 11.00 Topic 039 35.26 0.8 Topic 027 2.31 Topic 040 28.86 Topic 028 7.21 Topic 041 0.63 0.6 Topic 029 19.60 Topic 042 66.67 Topic 030 72.70 Topic 043 1.86 Topic 031 29.52 Topic 044 12.24 0.4 Topic 032 85.69 Topic 045 31.62 Topic 033 0.33 Topic 046 70.83 0.2 Topic 034 41.52 Topic 047 1.07 Difference Topic 035 2.76 Topic 048 75.94 0 Topic 036 0.00 Topic 049 56.67 Topic 037 0.39 Topic 050 27.26 −0.2 Topic 038 1.72 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 185 rfia-upv rfiaUPV02 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 24.00 rfiaUPV02 10 docs 21.20 90% 15 docs 20.00 80% 20 docs 18.40 30 docs 17.07 70% 100 docs 8.20 60% 200 docs 4.94 R−Precision 500 docs 2.28 50% 1000 docs 1.21 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 26.50 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7742 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1667 Third Quartile 0.5000 Interquartile range 0.5000 Mean 0.2650 Standard Deviation 0.2702 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7742 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2650 Std With No Outliers 0.2702 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries rfiaUPV02 Topic 026 11.11 Topic 039 43.75 0.8 Topic 027 5.26 Topic 040 28.57 Topic 028 15.79 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 50.00 Topic 030 66.67 Topic 043 0.00 Topic 031 32.20 Topic 044 18.42 0.4 Topic 032 77.42 Topic 045 16.67 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 66.67 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 68.75 0 Topic 036 0.00 Topic 049 50.00 Topic 037 0.00 Topic 050 33.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 186 rfia-upv rfiaUPV03 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 302 Pooled true Geometric Mean Average Precision 0.0643 Title-Desc with GITE Binary Preference (BPREF) 0.2045 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 43.17 rfiaUPV03 10 38.01 90% 20 34.07 80% 30 32.27 40 29.60 70% 50 27.56 Average Precision 60% 60 21.90 70 14.89 50% 80 11.47 40% 90 8.74 30% 100 6.73 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 23.35 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8056 Minimum 0.0000 First Quartile 0.0342 Second Quartile 0.1034 Third Quartile 0.4284 Interquartile range 0.3942 Mean 0.2335 Standard Deviation 0.2842 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8056 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2335 Std With No Outliers 0.2842 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries rfiaUPV03 Topic 026 3.91 Topic 039 11.19 0.8 Topic 027 0.30 Topic 040 7.75 Topic 028 8.95 Topic 041 0.37 0.6 Topic 029 4.45 Topic 042 17.87 Topic 030 76.32 Topic 043 1.71 Topic 031 38.70 Topic 044 11.94 0.4 Topic 032 69.75 Topic 045 2.78 Topic 033 0.35 Topic 046 73.81 0.2 Topic 034 80.56 Topic 047 3.64 Difference Topic 035 10.34 Topic 048 66.92 0 Topic 036 0.00 Topic 049 55.26 Topic 037 14.44 Topic 050 13.31 −0.2 Topic 038 9.09 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 187 rfia-upv rfiaUPV03 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 23.20 rfiaUPV03 10 docs 21.20 90% 15 docs 18.67 80% 20 docs 17.40 30 docs 14.40 70% 100 docs 7.24 60% 200 docs 4.52 R−Precision 500 docs 2.20 50% 1000 docs 1.21 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 21.93 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7419 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1333 Third Quartile 0.3792 Interquartile range 0.3792 Mean 0.2193 Standard Deviation 0.2668 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7419 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2193 Std With No Outliers 0.2668 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries rfiaUPV03 Topic 026 0.00 Topic 039 18.75 0.8 Topic 027 0.00 Topic 040 7.14 Topic 028 15.79 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 0.00 Topic 030 66.67 Topic 043 0.00 Topic 031 33.90 Topic 044 18.42 0.4 Topic 032 74.19 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 66.67 Topic 047 4.17 Difference Topic 035 16.67 Topic 048 64.58 0 Topic 036 0.00 Topic 049 50.00 Topic 037 31.25 Topic 050 13.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 188 rfia-upv rfiaUPV04 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 307 Pooled true Geometric Mean Average Precision 0.0681 All Fields with GITE Binary Preference (BPREF) 0.2393 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 49.64 rfiaUPV04 10 41.56 90% 20 39.02 80% 30 37.65 40 36.11 70% 50 34.81 Average Precision 60% 60 25.47 70 15.20 50% 80 12.54 40% 90 8.16 30% 100 7.01 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 26.60 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8382 Minimum 0.0000 First Quartile 0.0157 Second Quartile 0.1941 Third Quartile 0.4575 Interquartile range 0.4418 Mean 0.2660 Standard Deviation 0.2838 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8382 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2660 Std With No Outliers 0.2838 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries rfiaUPV04 Topic 026 8.39 Topic 039 38.59 0.8 Topic 027 0.83 Topic 040 32.18 Topic 028 19.41 Topic 041 0.63 0.6 Topic 029 5.69 Topic 042 64.29 Topic 030 73.54 Topic 043 1.95 Topic 031 30.59 Topic 044 9.28 0.4 Topic 032 83.82 Topic 045 21.91 Topic 033 0.22 Topic 046 71.67 0.2 Topic 034 42.11 Topic 047 1.10 Difference Topic 035 3.32 Topic 048 72.23 0 Topic 036 0.00 Topic 049 56.67 Topic 037 0.57 Topic 050 24.23 −0.2 Topic 038 1.72 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 189 rfia-upv rfiaUPV04 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 25.60 rfiaUPV04 10 docs 22.00 90% 15 docs 20.27 80% 20 docs 19.20 30 docs 16.27 70% 100 docs 7.84 60% 200 docs 4.74 R−Precision 500 docs 2.26 50% 1000 docs 1.23 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 26.67 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8065 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.2143 Third Quartile 0.5000 Interquartile range 0.5000 Mean 0.2667 Standard Deviation 0.2767 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8065 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2667 Std With No Outliers 0.2767 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries rfiaUPV04 Topic 026 11.11 Topic 039 43.75 0.8 Topic 027 5.26 Topic 040 21.43 Topic 028 26.32 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 50.00 Topic 030 66.67 Topic 043 0.00 Topic 031 32.20 Topic 044 10.53 0.4 Topic 032 80.65 Topic 045 33.33 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 66.67 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 68.75 0 Topic 036 0.00 Topic 049 50.00 Topic 037 0.00 Topic 050 33.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 190 sanmarcos SMGeoEN4 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 299 Pooled false Geometric Mean Average Precision 0.0843 Monolingual English no query expansion Binary Preference (BPREF) 0.2441 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 53.06 SMGeoEN4 10 46.30 90% 20 35.10 80% 30 34.12 40 32.48 70% 50 29.05 Average Precision 60% 60 24.63 70 17.24 50% 80 13.19 40% 90 11.10 30% 100 8.76 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 26.37 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0511 Second Quartile 0.1202 Third Quartile 0.3529 Interquartile range 0.3018 Mean 0.2637 Standard Deviation 0.2909 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7460 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2035 Std With No Outliers 0.2115 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 6 Number of Topics of the Experiment 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoEN4 Topic 026 41.15 Topic 039 11.20 0.8 Topic 027 2.72 Topic 040 25.50 Topic 028 8.02 Topic 041 0.27 0.6 Topic 029 19.83 Topic 042 6.92 Topic 030 100.00 Topic 043 1.67 Topic 031 32.14 Topic 044 28.95 0.4 Topic 032 91.32 Topic 045 11.66 Topic 033 0.22 Topic 046 69.23 0.2 Topic 034 44.77 Topic 047 5.89 Difference Topic 035 12.02 Topic 048 74.60 0 Topic 036 0.00 Topic 049 33.33 Topic 037 10.73 Topic 050 24.39 −0.2 Topic 038 2.78 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 191 sanmarcos SMGeoEN4 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 32.80 SMGeoEN4 10 docs 26.40 90% 15 docs 22.93 80% 20 docs 21.20 30 docs 17.87 70% 100 docs 8.28 60% 200 docs 5.02 R−Precision 500 docs 2.26 50% 1000 docs 1.20 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 28.57 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0625 Second Quartile 0.1667 Third Quartile 0.4583 Interquartile range 0.3958 Mean 0.2857 Standard Deviation 0.2905 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 1.0000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2857 Std With No Outliers 0.2905 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 6 Number of Topics of the Experiment 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoEN4 Topic 026 44.44 Topic 039 18.75 0.8 Topic 027 10.53 Topic 040 28.57 Topic 028 15.79 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 100.00 Topic 043 0.00 Topic 031 23.73 Topic 044 36.84 0.4 Topic 032 80.65 Topic 045 16.67 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 66.67 Topic 047 8.33 Difference Topic 035 16.67 Topic 048 72.92 0 Topic 036 0.00 Topic 049 50.00 Topic 037 12.50 Topic 050 33.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 192 sanmarcos SMGeoEN5 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 24,187 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 304 Pooled false Geometric Mean Average Precision 0.0755 Monolingual English no query expansion Binary Preference (BPREF) 0.2145 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 45.45 SMGeoEN5 10 40.90 90% 20 35.65 80% 30 31.80 40 29.51 70% 50 28.34 Average Precision 60% 60 24.70 70 14.72 50% 80 11.54 40% 90 9.08 30% 100 6.57 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 23.77 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8471 Minimum 0.0000 First Quartile 0.0344 Second Quartile 0.1325 Third Quartile 0.3426 Interquartile range 0.3082 Mean 0.2377 Standard Deviation 0.2689 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7165 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1849 Std With No Outliers 0.2061 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoEN5 Topic 026 3.45 Topic 039 38.98 0.8 Topic 027 7.16 Topic 040 18.90 Topic 028 22.18 Topic 041 0.27 0.6 Topic 029 13.25 Topic 042 32.69 Topic 030 84.26 Topic 043 3.44 Topic 031 68.79 Topic 044 8.62 0.4 Topic 032 84.71 Topic 045 32.06 Topic 033 5.23 Topic 046 15.42 0.2 Topic 034 40.58 Topic 047 4.32 Difference Topic 035 2.41 Topic 048 71.65 0 Topic 036 0.00 Topic 049 10.37 Topic 037 0.24 Topic 050 22.28 −0.2 Topic 038 2.94 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 193 sanmarcos SMGeoEN5 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 29.60 SMGeoEN5 10 docs 25.60 90% 15 docs 23.47 80% 20 docs 21.20 30 docs 19.47 70% 100 docs 8.52 60% 200 docs 5.06 R−Precision 500 docs 2.30 50% 1000 docs 1.22 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 25.81 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8387 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1316 Third Quartile 0.5000 Interquartile range 0.5000 Mean 0.2581 Standard Deviation 0.2744 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8387 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2581 Std With No Outliers 0.2744 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoEN5 Topic 026 0.00 Topic 039 37.50 0.8 Topic 027 5.26 Topic 040 21.43 Topic 028 31.58 Topic 041 0.00 0.6 Topic 029 22.22 Topic 042 50.00 Topic 030 66.67 Topic 043 12.50 Topic 031 66.10 Topic 044 13.16 0.4 Topic 032 83.87 Topic 045 50.00 Topic 033 10.00 Topic 046 0.00 0.2 Topic 034 66.67 Topic 047 8.33 Difference Topic 035 0.00 Topic 048 66.67 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 33.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 194 sanmarcos SMGeoEN1 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 299 Pooled true Geometric Mean Average Precision 0.0843 Monolingual English query expansion Binary Preference (BPREF) 0.2441 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 53.06 SMGeoEN1 10 46.30 90% 20 35.10 80% 30 34.12 40 32.48 70% 50 29.05 Average Precision 60% 60 24.63 70 17.24 50% 80 13.19 40% 90 11.10 30% 100 8.76 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 26.37 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0511 Second Quartile 0.1202 Third Quartile 0.3529 Interquartile range 0.3018 Mean 0.2637 Standard Deviation 0.2909 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7460 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2035 Std With No Outliers 0.2115 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 6 Number of Topics of the Experiment 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoEN1 Topic 026 41.15 Topic 039 11.20 0.8 Topic 027 2.72 Topic 040 25.50 Topic 028 8.02 Topic 041 0.27 0.6 Topic 029 19.83 Topic 042 6.92 Topic 030 100.00 Topic 043 1.67 Topic 031 32.14 Topic 044 28.95 0.4 Topic 032 91.32 Topic 045 11.66 Topic 033 0.22 Topic 046 69.23 0.2 Topic 034 44.77 Topic 047 5.89 Difference Topic 035 12.02 Topic 048 74.60 0 Topic 036 0.00 Topic 049 33.33 Topic 037 10.73 Topic 050 24.39 −0.2 Topic 038 2.78 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 195 sanmarcos SMGeoEN1 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 32.80 SMGeoEN1 10 docs 26.40 90% 15 docs 22.93 80% 20 docs 21.20 30 docs 17.87 70% 100 docs 8.28 60% 200 docs 5.02 R−Precision 500 docs 2.26 50% 1000 docs 1.20 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 28.57 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0625 Second Quartile 0.1667 Third Quartile 0.4583 Interquartile range 0.3958 Mean 0.2857 Standard Deviation 0.2905 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 1.0000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2857 Std With No Outliers 0.2905 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 6 Number of Topics of the Experiment 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoEN1 Topic 026 44.44 Topic 039 18.75 0.8 Topic 027 10.53 Topic 040 28.57 Topic 028 15.79 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 100.00 Topic 043 0.00 Topic 031 23.73 Topic 044 36.84 0.4 Topic 032 80.65 Topic 045 16.67 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 66.67 Topic 047 8.33 Difference Topic 035 16.67 Topic 048 72.92 0 Topic 036 0.00 Topic 049 50.00 Topic 037 12.50 Topic 050 33.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 196 sanmarcos SMGeoEN3 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 317 Pooled false Geometric Mean Average Precision 0.0818 Monolingual English query expansion Binary Preference (BPREF) 0.2519 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 52.35 SMGeoEN3 10 48.93 90% 20 38.86 80% 30 38.26 40 35.28 70% 50 33.08 Average Precision 60% 60 26.13 70 18.46 50% 80 16.52 40% 90 12.26 30% 100 9.70 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 28.57 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9350 Minimum 0.0000 First Quartile 0.0199 Second Quartile 0.2435 Third Quartile 0.3602 Interquartile range 0.3403 Mean 0.2857 Standard Deviation 0.2953 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7857 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2295 Std With No Outliers 0.2318 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoEN3 Topic 026 31.88 Topic 039 37.46 0.8 Topic 027 3.58 Topic 040 26.18 Topic 028 14.52 Topic 041 0.41 0.6 Topic 029 23.28 Topic 042 61.11 Topic 030 93.06 Topic 043 2.00 Topic 031 35.54 Topic 044 15.16 0.4 Topic 032 93.50 Topic 045 33.35 Topic 033 0.69 Topic 046 70.51 0.2 Topic 034 28.38 Topic 047 1.96 Difference Topic 035 3.37 Topic 048 78.57 0 Topic 036 0.00 Topic 049 34.09 Topic 037 0.35 Topic 050 24.35 −0.2 Topic 038 1.04 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 197 sanmarcos SMGeoEN3 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 28.00 SMGeoEN3 10 docs 26.80 90% 15 docs 22.67 80% 20 docs 20.40 30 docs 18.13 70% 100 docs 8.56 60% 200 docs 4.92 R−Precision 500 docs 2.38 50% 1000 docs 1.27 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 28.36 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9032 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.2542 Third Quartile 0.4583 Interquartile range 0.4583 Mean 0.2836 Standard Deviation 0.2788 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.9032 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2836 Std With No Outliers 0.2788 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoEN3 Topic 026 44.44 Topic 039 37.50 0.8 Topic 027 10.53 Topic 040 21.43 Topic 028 26.32 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 50.00 Topic 030 83.33 Topic 043 0.00 Topic 031 25.42 Topic 044 21.05 0.4 Topic 032 90.32 Topic 045 33.33 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 70.83 0 Topic 036 0.00 Topic 049 50.00 Topic 037 0.00 Topic 050 33.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 198 sanmarcos SMGeoEN5 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction MANUAL Retrieved 24,187 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 304 Pooled true Geometric Mean Average Precision 0.0755 Monolingual English added other sources of Binary Preference (BPREF) 0.2145 information Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 45.45 SMGeoEN5 10 40.90 90% 20 35.65 80% 30 31.80 40 29.51 70% 50 28.34 Average Precision 60% 60 24.70 70 14.72 50% 80 11.54 40% 90 9.08 30% 100 6.57 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 23.77 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8471 Minimum 0.0000 First Quartile 0.0344 Second Quartile 0.1325 Third Quartile 0.3426 Interquartile range 0.3082 Mean 0.2377 Standard Deviation 0.2689 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7165 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1849 Std With No Outliers 0.2061 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoEN5 Topic 026 3.45 Topic 039 38.98 0.8 Topic 027 7.16 Topic 040 18.90 Topic 028 22.18 Topic 041 0.27 0.6 Topic 029 13.25 Topic 042 32.69 Topic 030 84.26 Topic 043 3.44 Topic 031 68.79 Topic 044 8.62 0.4 Topic 032 84.71 Topic 045 32.06 Topic 033 5.23 Topic 046 15.42 0.2 Topic 034 40.58 Topic 047 4.32 Difference Topic 035 2.41 Topic 048 71.65 0 Topic 036 0.00 Topic 049 10.37 Topic 037 0.24 Topic 050 22.28 −0.2 Topic 038 2.94 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 199 sanmarcos SMGeoEN5 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 29.60 SMGeoEN5 10 docs 25.60 90% 15 docs 23.47 80% 20 docs 21.20 30 docs 19.47 70% 100 docs 8.52 60% 200 docs 5.06 R−Precision 500 docs 2.30 50% 1000 docs 1.22 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 25.81 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8387 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1316 Third Quartile 0.5000 Interquartile range 0.5000 Mean 0.2581 Standard Deviation 0.2744 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8387 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2581 Std With No Outliers 0.2744 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoEN5 Topic 026 0.00 Topic 039 37.50 0.8 Topic 027 5.26 Topic 040 21.43 Topic 028 31.58 Topic 041 0.00 0.6 Topic 029 22.22 Topic 042 50.00 Topic 030 66.67 Topic 043 12.50 Topic 031 66.10 Topic 044 13.16 0.4 Topic 032 83.87 Topic 045 50.00 Topic 033 10.00 Topic 046 0.00 0.2 Topic 034 66.67 Topic 047 8.33 Difference Topic 035 0.00 Topic 048 66.67 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 33.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 200 talp TALPGeoIRTDN2 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 8,805 Source Language English Relevant 289 Topic Fields title, description, narrative Relevant retrieved 181 Pooled false Geometric Mean Average Precision 0.0006 JIRS with lexical information and Lucene for Binary Preference (BPREF) 0.0773 Geographical Search Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 17.07 TALPGeoIRTDN2 10 12.96 90% 20 10.88 80% 30 9.84 40 9.61 70% 50 9.06 Average Precision 60% 60 4.29 70 3.45 50% 80 2.82 40% 90 0.38 30% 100 0.00 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 6.38 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.5000 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.0906 Interquartile range 0.0906 Mean 0.0638 Standard Deviation 0.1264 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1512 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0300 Std With No Outliers 0.0477 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries TALPGeoIRTDN2 Topic 026 0.00 Topic 039 11.07 0.8 Topic 027 0.16 Topic 040 0.00 Topic 028 1.27 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 0.00 Topic 030 40.56 Topic 043 0.00 Topic 031 6.66 Topic 044 9.48 0.4 Topic 032 0.00 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 0.00 Topic 047 10.56 Difference Topic 035 0.31 Topic 048 15.12 0 Topic 036 0.00 Topic 049 50.00 Topic 037 5.51 Topic 050 8.92 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 201 talp TALPGeoIRTDN2 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 8.00 TALPGeoIRTDN2 10 docs 7.20 90% 15 docs 6.67 80% 20 docs 5.60 30 docs 4.53 70% 100 docs 3.04 60% 200 docs 2.24 R−Precision 500 docs 1.32 50% 1000 docs 0.72 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 8.13 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.5000 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.1123 Interquartile range 0.1123 Mean 0.0813 Standard Deviation 0.1459 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.2500 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0449 Std With No Outliers 0.0767 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries TALPGeoIRTDN2 Topic 026 0.00 Topic 039 25.00 0.8 Topic 027 0.00 Topic 040 0.00 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 0.00 Topic 030 50.00 Topic 043 0.00 Topic 031 8.47 Topic 044 10.53 0.4 Topic 032 0.00 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 0.00 Topic 047 8.33 Difference Topic 035 0.00 Topic 048 18.75 0 Topic 036 0.00 Topic 049 50.00 Topic 037 18.75 Topic 050 13.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 202 talp TALPGeoIRTD1 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 24,742 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 230 Pooled true Geometric Mean Average Precision 0.0060 Uses the JIRS Passage Retrieval with lexical and Binary Preference (BPREF) 0.1189 geographical information Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 32.97 TALPGeoIRTD1 10 24.42 90% 20 23.54 80% 30 17.91 40 15.64 70% 50 15.12 Average Precision 60% 60 14.04 70 9.00 50% 80 7.28 40% 90 1.37 30% 100 1.12 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 13.42 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8424 Minimum 0.0000 First Quartile 0.0002 Second Quartile 0.0255 Third Quartile 0.1651 Interquartile range 0.1648 Mean 0.1342 Standard Deviation 0.2200 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3824 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0798 Std With No Outliers 0.1162 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries TALPGeoIRTD1 Topic 026 0.52 Topic 039 2.55 0.8 Topic 027 0.17 Topic 040 0.00 Topic 028 11.93 Topic 041 0.13 0.6 Topic 029 8.96 Topic 042 1.09 Topic 030 84.24 Topic 043 0.10 Topic 031 26.11 Topic 044 3.40 0.4 Topic 032 38.24 Topic 045 0.00 Topic 033 0.00 Topic 046 67.70 0.2 Topic 034 12.68 Topic 047 7.25 Difference Topic 035 0.03 Topic 048 33.64 0 Topic 036 0.00 Topic 049 22.22 Topic 037 0.02 Topic 050 14.60 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 203 talp TALPGeoIRTD1 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 16.80 TALPGeoIRTD1 10 docs 14.00 90% 15 docs 12.00 80% 20 docs 9.40 30 docs 7.60 70% 100 docs 5.40 60% 200 docs 3.26 R−Precision 500 docs 1.54 50% 1000 docs 0.92 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 13.70 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8333 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.2081 Interquartile range 0.2081 Mean 0.1370 Standard Deviation 0.2174 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0837 Std With No Outliers 0.1174 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries TALPGeoIRTD1 Topic 026 0.00 Topic 039 6.25 0.8 Topic 027 0.00 Topic 040 0.00 Topic 028 15.79 Topic 041 0.00 0.6 Topic 029 22.22 Topic 042 0.00 Topic 030 83.33 Topic 043 0.00 Topic 031 20.34 Topic 044 2.63 0.4 Topic 032 32.26 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 12.50 Difference Topic 035 0.00 Topic 048 27.08 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 20.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 204 talp TALPGeoIRTDN1 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 260 Pooled true Geometric Mean Average Precision 0.0093 JIRS for lexical and Geographical Search Binary Preference (BPREF) 0.1035 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 22.64 TALPGeoIRTDN1 10 21.19 90% 20 21.19 80% 30 18.41 40 13.91 70% 50 13.68 Average Precision 60% 60 11.35 70 10.05 50% 80 7.90 40% 90 2.66 30% 100 2.41 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 11.79 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7323 Minimum 0.0000 First Quartile 0.0013 Second Quartile 0.0437 Third Quartile 0.1846 Interquartile range 0.1833 Mean 0.1179 Standard Deviation 0.1798 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4500 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0923 Std With No Outliers 0.1290 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries TALPGeoIRTDN1 Topic 026 0.60 Topic 039 12.11 0.8 Topic 027 1.17 Topic 040 0.00 Topic 028 0.16 Topic 041 0.13 0.6 Topic 029 8.96 Topic 042 5.37 Topic 030 73.23 Topic 043 0.10 Topic 031 26.11 Topic 044 4.37 0.4 Topic 032 38.24 Topic 045 18.08 Topic 033 0.00 Topic 046 4.93 0.2 Topic 034 19.59 Topic 047 3.55 Difference Topic 035 1.20 Topic 048 25.12 0 Topic 036 0.00 Topic 049 45.00 Topic 037 0.01 Topic 050 6.74 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 205 talp TALPGeoIRTDN1 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 11.20 TALPGeoIRTDN1 10 docs 9.20 90% 15 docs 6.93 80% 20 docs 6.20 30 docs 6.67 70% 100 docs 5.56 60% 200 docs 3.64 R−Precision 500 docs 1.73 50% 1000 docs 1.04 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 13.16 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6667 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.2473 Interquartile range 0.2473 Mean 0.1316 Standard Deviation 0.1933 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1093 Std With No Outliers 0.1613 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries TALPGeoIRTDN1 Topic 026 0.00 Topic 039 12.50 0.8 Topic 027 0.00 Topic 040 0.00 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 22.22 Topic 042 0.00 Topic 030 66.67 Topic 043 0.00 Topic 031 20.34 Topic 044 7.89 0.4 Topic 032 32.26 Topic 045 33.33 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 33.33 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 43.75 0 Topic 036 0.00 Topic 049 50.00 Topic 037 0.00 Topic 050 6.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 206 talp TALPGeoIRTD2 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 4,851 Source Language English Relevant 276 Topic Fields title, description Relevant retrieved 123 Pooled false Geometric Mean Average Precision 0.0004 JIRS Passage Retrieval for lexical information and Binary Preference (BPREF) 0.0819 Lucene IR for geographical search Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 17.69 TALPGeoIRTD2 10 14.52 90% 20 14.27 80% 30 12.20 40 9.16 70% 50 9.16 Average Precision 60% 60 5.15 70 4.66 50% 80 4.21 40% 90 0.08 30% 100 0.08 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 7.66 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8333 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.0417 Interquartile range 0.0417 Mean 0.0766 Standard Deviation 0.1913 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.0666 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0060 Std With No Outliers 0.0162 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries TALPGeoIRTD2 Topic 026 0.00 Topic 039 3.34 0.8 Topic 027 0.00 Topic 040 0.00 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 0.32 Topic 030 83.33 Topic 043 0.26 Topic 031 6.66 Topic 044 10.71 0.4 Topic 032 0.00 Topic 045 0.27 Topic 033 0.00 Topic 046 1.25 0.2 Topic 034 0.00 Topic 047 19.14 Difference Topic 035 0.00 Topic 048 16.24 0 Topic 036 0.00 Topic 049 50.00 Topic 037 0.00 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 207 talp TALPGeoIRTD2 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 8.00 TALPGeoIRTD2 10 docs 6.40 90% 15 docs 6.40 80% 20 docs 5.60 30 docs 4.53 70% 100 docs 2.28 60% 200 docs 1.58 R−Precision 500 docs 0.98 50% 1000 docs 0.49 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 8.84 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8333 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.0681 Interquartile range 0.0681 Mean 0.0884 Standard Deviation 0.2008 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1053 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0120 Std With No Outliers 0.0309 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries TALPGeoIRTD2 Topic 026 0.00 Topic 039 6.25 0.8 Topic 027 0.00 Topic 040 0.00 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 0.00 Topic 030 83.33 Topic 043 0.00 Topic 031 8.47 Topic 044 10.53 0.4 Topic 032 0.00 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 0.00 Topic 047 29.17 Difference Topic 035 0.00 Topic 048 33.33 0 Topic 036 0.00 Topic 049 50.00 Topic 037 0.00 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 208 talp TALPGeoIRTDN3 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 243 Pooled false Geometric Mean Average Precision 0.0046 JIRS for lexical and geographical search with Binary Preference (BPREF) 0.1056 accumulated doc scoring Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 27.28 TALPGeoIRTDN3 10 18.32 90% 20 17.38 80% 30 14.20 40 11.17 70% 50 10.70 Average Precision 60% 60 8.53 70 7.85 50% 80 6.43 40% 90 1.46 30% 100 1.06 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 9.97 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7323 Minimum 0.0000 First Quartile 0.0001 Second Quartile 0.0117 Third Quartile 0.1395 Interquartile range 0.1394 Mean 0.0997 Standard Deviation 0.1677 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.2611 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0600 Std With No Outliers 0.0848 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries TALPGeoIRTDN3 Topic 026 0.60 Topic 039 12.11 0.8 Topic 027 1.17 Topic 040 0.00 Topic 028 0.16 Topic 041 0.00 0.6 Topic 029 8.96 Topic 042 5.89 Topic 030 73.23 Topic 043 0.01 Topic 031 26.11 Topic 044 8.40 0.4 Topic 032 38.24 Topic 045 24.86 Topic 033 0.00 Topic 046 0.18 0.2 Topic 034 19.59 Topic 047 0.05 Difference Topic 035 1.20 Topic 048 13.93 0 Topic 036 0.00 Topic 049 0.65 Topic 037 0.01 Topic 050 14.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 209 talp TALPGeoIRTDN3 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 11.20 TALPGeoIRTDN3 10 docs 9.20 90% 15 docs 7.47 80% 20 docs 7.60 30 docs 6.53 70% 100 docs 5.08 60% 200 docs 3.18 R−Precision 500 docs 1.57 50% 1000 docs 0.97 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 9.85 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6667 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.1758 Interquartile range 0.1758 Mean 0.0985 Standard Deviation 0.1619 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0748 Std With No Outliers 0.1129 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries TALPGeoIRTDN3 Topic 026 0.00 Topic 039 12.50 0.8 Topic 027 0.00 Topic 040 0.00 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 22.22 Topic 042 0.00 Topic 030 66.67 Topic 043 0.00 Topic 031 20.34 Topic 044 10.53 0.4 Topic 032 32.26 Topic 045 16.67 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 33.33 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 25.00 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 6.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 210 u.buffalo UBGTDrf1 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 301 Pooled false Geometric Mean Average Precision 0.0694 retr feedback run with parameters 10, 20 6 1 Binary Preference (BPREF) 0.2074 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 47.97 UBGTDrf1 10 38.15 90% 20 32.18 80% 30 30.65 40 29.13 70% 50 27.31 Average Precision 60% 60 23.85 70 15.37 50% 80 14.08 40% 90 6.95 30% 100 5.01 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 23.44 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9248 Minimum 0.0000 First Quartile 0.0230 Second Quartile 0.1410 Third Quartile 0.3015 Interquartile range 0.2785 Mean 0.2344 Standard Deviation 0.2785 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6828 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1513 Std With No Outliers 0.1669 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries UBGTDrf1 Topic 026 12.57 Topic 039 8.79 0.8 Topic 027 3.57 Topic 040 19.88 Topic 028 1.95 Topic 041 0.24 0.6 Topic 029 19.35 Topic 042 5.23 Topic 030 82.12 Topic 043 0.85 Topic 031 29.30 Topic 044 21.70 0.4 Topic 032 92.48 Topic 045 20.97 Topic 033 0.57 Topic 046 68.28 0.2 Topic 034 41.52 Topic 047 2.41 Difference Topic 035 9.97 Topic 048 78.56 0 Topic 036 0.00 Topic 049 32.69 Topic 037 14.10 Topic 050 17.55 −0.2 Topic 038 1.37 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 211 u.buffalo UBGTDrf1 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 26.40 UBGTDrf1 10 docs 21.60 90% 15 docs 19.73 80% 20 docs 18.60 30 docs 16.53 70% 100 docs 8.12 60% 200 docs 4.84 R−Precision 500 docs 2.18 50% 1000 docs 1.20 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 25.16 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8387 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1429 Third Quartile 0.3792 Interquartile range 0.3792 Mean 0.2516 Standard Deviation 0.2870 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8387 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2516 Std With No Outliers 0.2870 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries UBGTDrf1 Topic 026 22.22 Topic 039 18.75 0.8 Topic 027 5.26 Topic 040 14.29 Topic 028 10.53 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 83.33 Topic 043 0.00 Topic 031 33.90 Topic 044 31.58 0.4 Topic 032 83.87 Topic 045 16.67 Topic 033 5.00 Topic 046 66.67 0.2 Topic 034 66.67 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 77.08 0 Topic 036 0.00 Topic 049 50.00 Topic 037 18.75 Topic 050 13.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 212 u.buffalo UBGTDrf2 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 303 Pooled true Geometric Mean Average Precision 0.0697 Automatic retrieval feedback params 5 50 10 1 Binary Preference (BPREF) 0.1976 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 49.73 UBGTDrf2 10 39.89 90% 20 31.29 80% 30 29.50 40 28.68 70% 50 26.86 Average Precision 60% 60 23.90 70 15.01 50% 80 13.97 40% 90 7.03 30% 100 4.99 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 23.30 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9434 Minimum 0.0000 First Quartile 0.0289 Second Quartile 0.1484 Third Quartile 0.2741 Interquartile range 0.2451 Mean 0.2330 Standard Deviation 0.2773 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4139 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1251 Std With No Outliers 0.1186 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries UBGTDrf2 Topic 026 9.97 Topic 039 10.82 0.8 Topic 027 3.57 Topic 040 18.74 Topic 028 3.04 Topic 041 0.21 0.6 Topic 029 18.72 Topic 042 5.30 Topic 030 77.32 Topic 043 0.87 Topic 031 32.58 Topic 044 25.68 0.4 Topic 032 94.34 Topic 045 17.54 Topic 033 0.53 Topic 046 68.45 0.2 Topic 034 41.39 Topic 047 2.44 Difference Topic 035 8.35 Topic 048 79.78 0 Topic 036 0.00 Topic 049 24.36 Topic 037 14.84 Topic 050 22.21 −0.2 Topic 038 1.45 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 213 u.buffalo UBGTDrf2 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 29.60 UBGTDrf2 10 docs 22.00 90% 15 docs 20.00 80% 20 docs 18.60 30 docs 17.07 70% 100 docs 8.36 60% 200 docs 4.90 R−Precision 500 docs 2.21 50% 1000 docs 1.21 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 22.19 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8387 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1111 Third Quartile 0.3146 Interquartile range 0.3146 Mean 0.2219 Standard Deviation 0.2772 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7708 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1962 Std With No Outliers 0.2509 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries UBGTDrf2 Topic 026 22.22 Topic 039 18.75 0.8 Topic 027 5.26 Topic 040 14.29 Topic 028 10.53 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 66.67 Topic 043 0.00 Topic 031 38.98 Topic 044 28.95 0.4 Topic 032 83.87 Topic 045 0.00 Topic 033 5.00 Topic 046 66.67 0.2 Topic 034 66.67 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 77.08 0 Topic 036 0.00 Topic 049 0.00 Topic 037 18.75 Topic 050 20.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 214 u.buffalo UBManual2 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction MANUAL Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 311 Pooled false Geometric Mean Average Precision 0.0496 manaul run with auto feedback 10 50 10 1 Binary Preference (BPREF) 0.2054 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 49.24 UBManual2 10 43.62 90% 20 36.53 80% 30 32.38 40 27.26 70% 50 26.44 Average Precision 60% 60 22.35 70 16.67 50% 80 15.08 40% 90 9.15 30% 100 7.03 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 24.46 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9560 Minimum 0.0000 First Quartile 0.0388 Second Quartile 0.1509 Third Quartile 0.3812 Interquartile range 0.3424 Mean 0.2446 Standard Deviation 0.2665 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8256 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2150 Std With No Outliers 0.2262 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries UBManual2 Topic 026 6.18 Topic 039 46.74 0.8 Topic 027 12.57 Topic 040 10.82 Topic 028 30.17 Topic 041 0.00 0.6 Topic 029 36.77 Topic 042 70.00 Topic 030 15.95 Topic 043 0.48 Topic 031 26.09 Topic 044 10.94 0.4 Topic 032 95.60 Topic 045 82.56 Topic 033 1.05 Topic 046 37.59 0.2 Topic 034 39.70 Topic 047 0.03 Difference Topic 035 4.82 Topic 048 40.55 0 Topic 036 0.00 Topic 049 19.44 Topic 037 15.09 Topic 050 8.19 −0.2 Topic 038 0.19 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 215 u.buffalo UBManual2 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 25.60 UBManual2 10 docs 22.80 90% 15 docs 20.53 80% 20 docs 19.00 30 docs 16.40 70% 100 docs 7.68 60% 200 docs 4.84 R−Precision 500 docs 2.29 50% 1000 docs 1.24 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 24.59 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8710 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1875 Third Quartile 0.3701 Interquartile range 0.3701 Mean 0.2459 Standard Deviation 0.2568 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8710 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2459 Std With No Outliers 0.2568 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries UBManual2 Topic 026 11.11 Topic 039 37.50 0.8 Topic 027 21.05 Topic 040 21.43 Topic 028 36.84 Topic 041 0.00 0.6 Topic 029 33.33 Topic 042 50.00 Topic 030 16.67 Topic 043 0.00 Topic 031 28.81 Topic 044 15.79 0.4 Topic 032 87.10 Topic 045 83.33 Topic 033 0.00 Topic 046 33.33 0.2 Topic 034 66.67 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 39.58 0 Topic 036 0.00 Topic 049 0.00 Topic 037 18.75 Topic 050 13.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 216 u.buffalo UBGManual1 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction MANUAL Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 312 Pooled true Geometric Mean Average Precision 0.0503 Manual run 1 Binary Preference (BPREF) 0.1938 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 47.52 UBGManual1 10 41.99 90% 20 33.72 80% 30 30.33 40 24.98 70% 50 23.27 Average Precision 60% 60 21.15 70 15.69 50% 80 13.83 40% 90 8.94 30% 100 7.13 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 23.07 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9180 Minimum 0.0000 First Quartile 0.0306 Second Quartile 0.1708 Third Quartile 0.3749 Interquartile range 0.3443 Mean 0.2307 Standard Deviation 0.2369 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7041 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2021 Std With No Outliers 0.1928 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries UBGManual1 Topic 026 6.94 Topic 039 45.00 0.8 Topic 027 11.78 Topic 040 9.98 Topic 028 27.94 Topic 041 0.00 0.6 Topic 029 36.62 Topic 042 45.00 Topic 030 19.26 Topic 043 0.62 Topic 031 35.69 Topic 044 10.66 0.4 Topic 032 91.80 Topic 045 70.41 Topic 033 1.07 Topic 046 42.11 0.2 Topic 034 40.12 Topic 047 0.03 Difference Topic 035 3.73 Topic 048 31.67 0 Topic 036 0.00 Topic 049 20.00 Topic 037 17.08 Topic 050 8.94 −0.2 Topic 038 0.33 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 217 u.buffalo UBGManual1 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 26.40 UBGManual1 10 docs 22.80 90% 15 docs 21.60 80% 20 docs 19.20 30 docs 16.53 70% 100 docs 7.68 60% 200 docs 4.90 R−Precision 500 docs 2.30 50% 1000 docs 1.25 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 24.73 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8387 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1875 Third Quartile 0.3787 Interquartile range 0.3787 Mean 0.2473 Standard Deviation 0.2396 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8387 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2473 Std With No Outliers 0.2396 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries UBGManual1 Topic 026 11.11 Topic 039 37.50 0.8 Topic 027 15.79 Topic 040 21.43 Topic 028 42.11 Topic 041 0.00 0.6 Topic 029 33.33 Topic 042 50.00 Topic 030 33.33 Topic 043 0.00 Topic 031 38.98 Topic 044 15.79 0.4 Topic 032 83.87 Topic 045 66.67 Topic 033 5.00 Topic 046 33.33 0.2 Topic 034 66.67 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 31.25 0 Topic 036 0.00 Topic 049 0.00 Topic 037 18.75 Topic 050 13.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 218 u.groningen CLCGGeoEE1 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 265 Pooled true Geometric Mean Average Precision 0.0402 uploaded by N. Ferro Binary Preference (BPREF) 0.1589 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 33.37 CLCGGeoEE1 10 28.45 90% 20 25.08 80% 30 22.99 40 21.40 70% 50 20.37 Average Precision 60% 60 17.52 70 12.73 50% 80 9.08 40% 90 7.21 30% 100 4.88 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 17.30 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8145 Minimum 0.0000 First Quartile 0.0213 Second Quartile 0.0389 Third Quartile 0.1558 Interquartile range 0.1345 Mean 0.1730 Standard Deviation 0.2568 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1971 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0556 Std With No Outliers 0.0572 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries CLCGGeoEE1 Topic 026 2.34 Topic 039 7.02 0.8 Topic 027 3.35 Topic 040 13.72 Topic 028 1.48 Topic 041 0.05 0.6 Topic 029 10.41 Topic 042 3.89 Topic 030 19.71 Topic 043 0.31 Topic 031 35.79 Topic 044 13.93 0.4 Topic 032 81.45 Topic 045 3.54 Topic 033 0.59 Topic 046 77.78 0.2 Topic 034 55.56 Topic 047 1.40 Difference Topic 035 3.48 Topic 048 70.91 0 Topic 036 0.00 Topic 049 3.57 Topic 037 4.92 Topic 050 14.20 −0.2 Topic 038 3.23 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 219 u.groningen CLCGGeoEE1 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 20.80 CLCGGeoEE1 10 docs 18.00 90% 15 docs 18.13 80% 20 docs 17.00 30 docs 15.33 70% 100 docs 7.28 60% 200 docs 4.28 R−Precision 500 docs 1.99 50% 1000 docs 1.06 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 19.83 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7742 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1053 Third Quartile 0.2833 Interquartile range 0.2833 Mean 0.1983 Standard Deviation 0.2548 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7083 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1743 Std With No Outliers 0.2296 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries CLCGGeoEE1 Topic 026 0.00 Topic 039 18.75 0.8 Topic 027 10.53 Topic 040 21.43 Topic 028 5.26 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 33.33 Topic 043 0.00 Topic 031 42.37 Topic 044 21.05 0.4 Topic 032 77.42 Topic 045 0.00 Topic 033 5.00 Topic 046 66.67 0.2 Topic 034 66.67 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 70.83 0 Topic 036 0.00 Topic 049 0.00 Topic 037 18.75 Topic 050 26.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 220 u.groningen CLCGGeoEE2 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 278 Pooled true Geometric Mean Average Precision 0.0400 uploaded by N. Ferro Binary Preference (BPREF) 0.1821 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 45.21 CLCGGeoEE2 10 37.14 90% 20 30.61 80% 30 29.02 40 26.95 70% 50 25.08 Average Precision 60% 60 20.40 70 14.43 50% 80 10.66 40% 90 8.52 30% 100 6.31 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 21.63 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8095 Minimum 0.0000 First Quartile 0.0093 Second Quartile 0.1446 Third Quartile 0.3180 Interquartile range 0.3087 Mean 0.2163 Standard Deviation 0.2586 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7194 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1650 Std With No Outliers 0.1964 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries CLCGGeoEE2 Topic 026 11.00 Topic 039 25.92 0.8 Topic 027 6.04 Topic 040 22.52 Topic 028 0.24 Topic 041 0.00 0.6 Topic 029 21.67 Topic 042 36.11 Topic 030 14.46 Topic 043 0.60 Topic 031 30.37 Topic 044 18.53 0.4 Topic 032 80.25 Topic 045 56.57 Topic 033 0.64 Topic 046 80.95 0.2 Topic 034 38.89 Topic 047 1.03 Difference Topic 035 2.74 Topic 048 71.94 0 Topic 036 0.00 Topic 049 2.63 Topic 037 0.31 Topic 050 16.00 −0.2 Topic 038 1.32 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 221 u.groningen CLCGGeoEE2 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 23.20 CLCGGeoEE2 10 docs 20.00 90% 15 docs 18.40 80% 20 docs 18.00 30 docs 16.27 70% 100 docs 7.52 60% 200 docs 4.42 R−Precision 500 docs 2.10 50% 1000 docs 1.11 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 21.94 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7742 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1111 Third Quartile 0.3390 Interquartile range 0.3390 Mean 0.2194 Standard Deviation 0.2460 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7742 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2194 Std With No Outliers 0.2460 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries CLCGGeoEE2 Topic 026 11.11 Topic 039 31.25 0.8 Topic 027 10.53 Topic 040 28.57 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 50.00 Topic 030 16.67 Topic 043 0.00 Topic 031 35.59 Topic 044 23.68 0.4 Topic 032 77.42 Topic 045 50.00 Topic 033 5.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 70.83 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 26.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 222 u.groningen CLCGGeoEE5 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 257 Pooled false Geometric Mean Average Precision 0.0287 uploaded by N. Ferro Binary Preference (BPREF) 0.1672 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 30.63 CLCGGeoEE5 10 29.43 90% 20 26.34 80% 30 24.06 40 22.95 70% 50 22.52 Average Precision 60% 60 15.57 70 9.64 50% 80 8.02 40% 90 6.35 30% 100 3.50 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 17.57 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8510 Minimum 0.0000 First Quartile 0.0047 Second Quartile 0.0420 Third Quartile 0.3036 Interquartile range 0.2989 Mean 0.1757 Standard Deviation 0.2576 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1476 Std With No Outliers 0.2205 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries CLCGGeoEE5 Topic 026 0.49 Topic 039 5.10 0.8 Topic 027 2.23 Topic 040 4.20 Topic 028 0.09 Topic 041 0.10 0.6 Topic 029 6.07 Topic 042 4.19 Topic 030 25.67 Topic 043 0.09 Topic 031 49.10 Topic 044 12.20 0.4 Topic 032 85.10 Topic 045 3.03 Topic 033 0.40 Topic 046 73.33 0.2 Topic 034 44.44 Topic 047 1.06 Difference Topic 035 4.86 Topic 048 55.83 0 Topic 036 0.00 Topic 049 50.00 Topic 037 0.35 Topic 050 8.75 −0.2 Topic 038 2.63 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 223 u.groningen CLCGGeoEE5 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 19.20 CLCGGeoEE5 10 docs 17.20 90% 15 docs 15.20 80% 20 docs 14.40 30 docs 12.80 70% 100 docs 6.60 60% 200 docs 4.10 R−Precision 500 docs 1.91 50% 1000 docs 1.03 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 17.77 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7742 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0526 Third Quartile 0.3333 Interquartile range 0.3333 Mean 0.1777 Standard Deviation 0.2464 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7742 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1777 Std With No Outliers 0.2464 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries CLCGGeoEE5 Topic 026 0.00 Topic 039 6.25 0.8 Topic 027 5.26 Topic 040 14.29 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 33.33 Topic 043 0.00 Topic 031 55.93 Topic 044 21.05 0.4 Topic 032 77.42 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 56.25 0 Topic 036 0.00 Topic 049 50.00 Topic 037 0.00 Topic 050 13.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 224 u.groningen CLCGGeoEE10 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 257 Pooled false Geometric Mean Average Precision 0.0229 geographic query expansion - uploaded by N. Ferro Binary Preference (BPREF) 0.1481 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 37.92 CLCGGeoEE10 10 30.89 90% 20 24.40 80% 30 22.87 40 19.97 70% 50 19.34 Average Precision 60% 60 13.56 70 9.70 50% 80 8.75 40% 90 5.83 30% 100 3.62 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 16.90 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8481 Minimum 0.0000 First Quartile 0.0053 Second Quartile 0.0555 Third Quartile 0.2643 Interquartile range 0.2590 Mean 0.1690 Standard Deviation 0.2363 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5717 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1183 Std With No Outliers 0.1628 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries CLCGGeoEE10 Topic 026 2.11 Topic 039 26.91 0.8 Topic 027 5.55 Topic 040 0.91 Topic 028 0.06 Topic 041 0.00 0.6 Topic 029 7.90 Topic 042 14.25 Topic 030 30.02 Topic 043 0.56 Topic 031 48.20 Topic 044 9.79 0.4 Topic 032 84.81 Topic 045 26.27 Topic 033 1.35 Topic 046 65.56 0.2 Topic 034 4.11 Topic 047 0.05 Difference Topic 035 1.02 Topic 048 57.17 0 Topic 036 0.00 Topic 049 25.00 Topic 037 0.23 Topic 050 10.32 −0.2 Topic 038 0.41 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 225 u.groningen CLCGGeoEE10 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 20.80 CLCGGeoEE10 10 docs 18.80 90% 15 docs 16.27 80% 20 docs 14.80 30 docs 12.67 70% 100 docs 6.56 60% 200 docs 4.02 R−Precision 500 docs 1.90 50% 1000 docs 1.03 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 17.62 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7097 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0526 Third Quartile 0.2708 Interquartile range 0.2708 Mean 0.1762 Standard Deviation 0.2356 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1539 Std With No Outliers 0.2122 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries CLCGGeoEE10 Topic 026 11.11 Topic 039 25.00 0.8 Topic 027 5.26 Topic 040 0.00 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 33.33 Topic 043 0.00 Topic 031 52.54 Topic 044 21.05 0.4 Topic 032 70.97 Topic 045 16.67 Topic 033 5.00 Topic 046 66.67 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 58.33 0 Topic 036 0.00 Topic 049 50.00 Topic 037 0.00 Topic 050 13.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 226 u.groningen CLCGGeoEE11 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 277 Pooled false Geometric Mean Average Precision 0.0421 geographic query expansion - uploaded by N. Ferro Binary Preference (BPREF) 0.1810 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 49.21 CLCGGeoEE11 10 40.24 90% 20 31.10 80% 30 29.11 40 26.13 70% 50 24.61 Average Precision 60% 60 19.37 70 13.28 50% 80 11.12 40% 90 8.97 30% 100 6.63 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 21.94 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8333 Minimum 0.0000 First Quartile 0.0113 Second Quartile 0.1585 Third Quartile 0.3621 Interquartile range 0.3507 Mean 0.2194 Standard Deviation 0.2514 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2194 Std With No Outliers 0.2514 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries CLCGGeoEE11 Topic 026 12.68 Topic 039 26.02 0.8 Topic 027 6.08 Topic 040 21.75 Topic 028 0.26 Topic 041 0.00 0.6 Topic 029 21.67 Topic 042 36.11 Topic 030 27.11 Topic 043 0.60 Topic 031 36.49 Topic 044 18.08 0.4 Topic 032 80.49 Topic 045 56.75 Topic 033 1.67 Topic 046 83.33 0.2 Topic 034 38.89 Topic 047 0.64 Difference Topic 035 2.75 Topic 048 57.12 0 Topic 036 0.00 Topic 049 2.63 Topic 037 0.31 Topic 050 15.85 −0.2 Topic 038 1.30 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 227 u.groningen CLCGGeoEE11 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 23.20 CLCGGeoEE11 10 docs 21.20 90% 15 docs 18.13 80% 20 docs 17.80 30 docs 16.00 70% 100 docs 7.68 60% 200 docs 4.50 R−Precision 500 docs 2.07 50% 1000 docs 1.11 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 21.44 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7742 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1111 Third Quartile 0.3559 Interquartile range 0.3559 Mean 0.2144 Standard Deviation 0.2384 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7742 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2144 Std With No Outliers 0.2384 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries CLCGGeoEE11 Topic 026 11.11 Topic 039 31.25 0.8 Topic 027 10.53 Topic 040 21.43 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 50.00 Topic 030 16.67 Topic 043 0.00 Topic 031 42.37 Topic 044 23.68 0.4 Topic 032 77.42 Topic 045 50.00 Topic 033 10.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 60.42 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 20.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 228 u.twente utGeoTIB GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction MANUAL Retrieved 21,727 Source Language English Relevant 378 Topic Fields title Relevant retrieved 209 Pooled true Geometric Mean Average Precision 0.0247 Retrieval by content. Filtering by geographic Binary Preference (BPREF) 0.1512 location through boolean matching of query title locations and document locations. Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 34.06 utGeoTIB 10 29.52 90% 20 25.01 80% 30 23.94 40 19.17 70% 50 14.82 Average Precision 60% 60 10.91 70 9.70 50% 80 8.97 40% 90 7.75 30% 100 6.90 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 16.23 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0072 Second Quartile 0.0644 Third Quartile 0.1936 Interquartile range 0.1864 Mean 0.1623 Standard Deviation 0.2473 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4722 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0861 Std With No Outliers 0.1168 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries utGeoTIB Topic 026 0.81 Topic 039 6.93 0.8 Topic 027 1.05 Topic 040 0.00 Topic 028 8.95 Topic 041 0.13 0.6 Topic 029 9.58 Topic 042 1.09 Topic 030 20.08 Topic 043 0.17 Topic 031 27.59 Topic 044 14.93 0.4 Topic 032 57.26 Topic 045 0.85 Topic 033 0.31 Topic 046 100.00 0.2 Topic 034 4.76 Topic 047 6.44 Difference Topic 035 0.42 Topic 048 47.22 0 Topic 036 0.00 Topic 049 59.09 Topic 037 4.63 Topic 050 19.12 −0.2 Topic 038 14.29 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 229 u.twente utGeoTIB GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 18.40 utGeoTIB 10 docs 17.20 90% 15 docs 17.33 80% 20 docs 16.20 30 docs 13.73 70% 100 docs 6.00 60% 200 docs 3.48 R−Precision 500 docs 1.54 50% 1000 docs 0.84 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 17.38 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0625 Third Quartile 0.2215 Interquartile range 0.2215 Mean 0.1738 Standard Deviation 0.2558 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1160 Std With No Outliers 0.1591 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries utGeoTIB Topic 026 0.00 Topic 039 12.50 0.8 Topic 027 0.00 Topic 040 0.00 Topic 028 15.79 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 16.67 Topic 043 0.00 Topic 031 37.29 Topic 044 18.42 0.4 Topic 032 67.74 Topic 045 0.00 Topic 033 5.00 Topic 046 100.00 0.2 Topic 034 0.00 Topic 047 12.50 Difference Topic 035 0.00 Topic 048 47.92 0 Topic 036 0.00 Topic 049 50.00 Topic 037 6.25 Topic 050 33.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 230 u.twente utGeoTdIB GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction MANUAL Retrieved 9,845 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 76 Pooled true Geometric Mean Average Precision 0.0078 Retrieval by content. Filtering by geographic Binary Preference (BPREF) 0.0695 location through boolean matching of query title locations and document locations. Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 36.38 utGeoTdIB 10 20.45 90% 20 10.55 80% 30 6.87 40 6.78 70% 50 6.71 Average Precision 60% 60 5.11 70 0.62 50% 80 0.48 40% 90 0.23 30% 100 0.23 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 7.32 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6667 Minimum 0.0000 First Quartile 0.0075 Second Quartile 0.0260 Third Quartile 0.0758 Interquartile range 0.0683 Mean 0.0732 Standard Deviation 0.1360 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0425 Std With No Outliers 0.0512 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries utGeoTdIB Topic 026 1.24 Topic 039 0.89 0.8 Topic 027 0.93 Topic 040 2.38 Topic 028 11.05 Topic 041 1.09 0.6 Topic 029 10.46 Topic 042 1.28 Topic 030 3.05 Topic 043 0.33 Topic 031 18.49 Topic 044 6.51 0.4 Topic 032 3.23 Topic 045 6.62 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 16.67 Topic 048 6.25 0 Topic 036 0.00 Topic 049 16.67 Topic 037 2.60 Topic 050 6.51 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 231 u.twente utGeoTdIB GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 16.80 utGeoTdIB 10 docs 11.60 90% 15 docs 9.33 80% 20 docs 7.20 30 docs 5.87 70% 100 docs 2.40 60% 200 docs 1.32 R−Precision 500 docs 0.59 50% 1000 docs 0.30 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 7.62 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6667 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0323 Third Quartile 0.1067 Interquartile range 0.1067 Mean 0.0762 Standard Deviation 0.1388 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.2203 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0516 Std With No Outliers 0.0657 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries utGeoTdIB Topic 026 0.00 Topic 039 6.25 0.8 Topic 027 5.26 Topic 040 7.14 Topic 028 15.79 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 0.00 Topic 043 0.00 Topic 031 22.03 Topic 044 10.53 0.4 Topic 032 3.23 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 16.67 Topic 048 6.25 0 Topic 036 0.00 Topic 049 0.00 Topic 037 6.25 Topic 050 13.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 232 u.twente utGeoTIBm GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 271 Pooled false Geometric Mean Average Precision 0.0285 Filtering by geographic location and merging of Binary Preference (BPREF) 0.1528 filtered and unfiltered results. Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 34.07 utGeoTIBm 10 29.52 90% 20 25.01 80% 30 23.94 40 20.69 70% 50 16.81 Average Precision 60% 60 12.77 70 11.46 50% 80 10.60 40% 90 8.99 30% 100 7.38 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 17.18 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0072 Second Quartile 0.0644 Third Quartile 0.1936 Interquartile range 0.1864 Mean 0.1718 Standard Deviation 0.2562 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4674 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0769 Std With No Outliers 0.1107 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries utGeoTIBm Topic 026 0.81 Topic 039 6.93 0.8 Topic 027 1.05 Topic 040 0.02 Topic 028 8.95 Topic 041 0.13 0.6 Topic 029 9.58 Topic 042 1.09 Topic 030 20.08 Topic 043 0.17 Topic 031 46.74 Topic 044 14.93 0.4 Topic 032 57.26 Topic 045 0.85 Topic 033 0.31 Topic 046 100.00 0.2 Topic 034 4.93 Topic 047 6.44 Difference Topic 035 0.42 Topic 048 51.59 0 Topic 036 0.00 Topic 049 59.09 Topic 037 4.63 Topic 050 19.12 −0.2 Topic 038 14.29 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 233 u.twente utGeoTIBm GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 18.40 utGeoTIBm 10 docs 17.20 90% 15 docs 17.33 80% 20 docs 16.20 30 docs 13.73 70% 100 docs 6.52 60% 200 docs 4.10 R−Precision 500 docs 1.98 50% 1000 docs 1.08 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 17.38 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0625 Third Quartile 0.2215 Interquartile range 0.2215 Mean 0.1738 Standard Deviation 0.2558 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1160 Std With No Outliers 0.1591 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries utGeoTIBm Topic 026 0.00 Topic 039 12.50 0.8 Topic 027 0.00 Topic 040 0.00 Topic 028 15.79 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 16.67 Topic 043 0.00 Topic 031 37.29 Topic 044 18.42 0.4 Topic 032 67.74 Topic 045 0.00 Topic 033 5.00 Topic 046 100.00 0.2 Topic 034 0.00 Topic 047 12.50 Difference Topic 035 0.00 Topic 048 47.92 0 Topic 036 0.00 Topic 049 50.00 Topic 037 6.25 Topic 050 33.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 234 u.twente utGeoTdnIB GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction MANUAL Retrieved 10,175 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 85 Pooled true Geometric Mean Average Precision 0.0032 Retrieval by content. Filtering by geographic Binary Preference (BPREF) 0.1085 location through boolean matching of query title locations and document locations. Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 40.83 utGeoTdnIB 10 24.57 90% 20 20.48 80% 30 15.84 40 14.44 70% 50 13.83 Average Precision 60% 60 7.24 70 2.07 50% 80 1.87 40% 90 0.79 30% 100 0.47 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 11.34 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6667 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0323 Third Quartile 0.1979 Interquartile range 0.1979 Mean 0.1134 Standard Deviation 0.1773 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3389 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0726 Std With No Outliers 0.1089 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries utGeoTdnIB Topic 026 8.53 Topic 039 33.89 0.8 Topic 027 1.13 Topic 040 7.14 Topic 028 0.00 Topic 041 0.86 0.6 Topic 029 19.76 Topic 042 50.00 Topic 030 0.40 Topic 043 0.00 Topic 031 19.90 Topic 044 3.63 0.4 Topic 032 3.23 Topic 045 32.35 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 6.25 0 Topic 036 0.00 Topic 049 25.00 Topic 037 0.00 Topic 050 4.87 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 235 u.twente utGeoTdnIB GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 16.80 utGeoTdnIB 10 docs 12.80 90% 15 docs 10.13 80% 20 docs 9.00 30 docs 6.67 70% 100 docs 2.60 60% 200 docs 1.44 R−Precision 500 docs 0.63 50% 1000 docs 0.34 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 13.66 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6667 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0526 Third Quartile 0.2260 Interquartile range 0.2260 Mean 0.1366 Standard Deviation 0.2008 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1145 Std With No Outliers 0.1713 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries utGeoTdnIB Topic 026 22.22 Topic 039 31.25 0.8 Topic 027 5.26 Topic 040 7.14 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 50.00 Topic 030 0.00 Topic 043 0.00 Topic 031 23.73 Topic 044 7.89 0.4 Topic 032 3.23 Topic 045 50.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 6.25 0 Topic 036 0.00 Topic 049 50.00 Topic 037 0.00 Topic 050 6.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 236 u.twente utGeoTdnIBm GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction MANUAL Retrieved 25,000 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 285 Pooled false Geometric Mean Average Precision 0.0380 Filtering by geographic location and merging of Binary Preference (BPREF) 0.1484 filtered and unfiltered results. Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 43.07 utGeoTdnIBm 10 32.48 90% 20 26.90 80% 30 23.28 40 21.79 70% 50 21.18 Average Precision 60% 60 13.93 70 8.43 50% 80 7.31 40% 90 4.24 30% 100 3.11 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 16.77 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6816 Minimum 0.0000 First Quartile 0.0078 Second Quartile 0.0590 Third Quartile 0.3247 Interquartile range 0.3169 Mean 0.1677 Standard Deviation 0.2101 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6816 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1677 Std With No Outliers 0.2101 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries utGeoTdnIBm Topic 026 8.61 Topic 039 33.89 0.8 Topic 027 1.13 Topic 040 33.55 Topic 028 5.62 Topic 041 0.86 0.6 Topic 029 19.76 Topic 042 50.16 Topic 030 0.40 Topic 043 0.54 Topic 031 34.65 Topic 044 7.99 0.4 Topic 032 68.16 Topic 045 32.11 Topic 033 0.21 Topic 046 66.86 0.2 Topic 034 5.90 Topic 047 1.28 Difference Topic 035 0.34 Topic 048 15.20 0 Topic 036 0.00 Topic 049 25.00 Topic 037 0.40 Topic 050 4.87 −0.2 Topic 038 1.67 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 237 u.twente utGeoTdnIBm GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 19.20 utGeoTdnIBm 10 docs 16.80 90% 15 docs 14.40 80% 20 docs 13.60 30 docs 11.07 70% 100 docs 5.08 60% 200 docs 3.86 R−Precision 500 docs 2.02 50% 1000 docs 1.14 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 18.12 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7419 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0667 Third Quartile 0.2924 Interquartile range 0.2924 Mean 0.1812 Standard Deviation 0.2310 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1578 Std With No Outliers 0.2036 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries utGeoTdnIBm Topic 026 22.22 Topic 039 31.25 0.8 Topic 027 5.26 Topic 040 28.57 Topic 028 15.79 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 50.00 Topic 030 0.00 Topic 043 0.00 Topic 031 27.12 Topic 044 7.89 0.4 Topic 032 74.19 Topic 045 50.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 6.25 0 Topic 036 0.00 Topic 049 50.00 Topic 037 0.00 Topic 050 6.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 238 unsw unswTitleBaseline GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 12,919 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 260 Pooled true Geometric Mean Average Precision 0.0866 unswTitleBaseline Binary Preference (BPREF) 0.2374 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 59.03 unswTitleBaseline 10 50.44 90% 20 43.13 80% 30 37.14 40 32.73 70% 50 30.93 Average Precision 60% 60 23.12 70 15.43 50% 80 10.41 40% 90 6.19 30% 100 3.08 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 26.22 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7722 Minimum 0.0000 First Quartile 0.0753 Second Quartile 0.2134 Third Quartile 0.4690 Interquartile range 0.3937 Mean 0.2622 Standard Deviation 0.2395 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7722 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2622 Std With No Outliers 0.2395 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 4 Number of Topics of the Experiment 3.5 3 2.5 2 1.5 1 0.5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries unswTitleBaseline Topic 026 30.94 Topic 039 46.96 0.8 Topic 027 10.26 Topic 040 15.86 Topic 028 7.79 Topic 041 0.00 0.6 Topic 029 24.50 Topic 042 10.10 Topic 030 77.22 Topic 043 6.75 Topic 031 4.75 Topic 044 21.34 0.4 Topic 032 73.34 Topic 045 1.85 Topic 033 46.88 Topic 046 66.67 0.2 Topic 034 21.43 Topic 047 8.88 Difference Topic 035 32.79 Topic 048 58.52 0 Topic 036 0.00 Topic 049 50.00 Topic 037 21.38 Topic 050 11.06 −0.2 Topic 038 6.25 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 239 unsw unswTitleBaseline GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 26.40 unswTitleBaseline 10 docs 23.60 90% 15 docs 22.40 80% 20 docs 21.40 30 docs 18.93 70% 100 docs 8.68 60% 200 docs 4.66 R−Precision 500 docs 2.03 50% 1000 docs 1.04 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 28.21 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8333 Minimum 0.0000 First Quartile 0.0956 Second Quartile 0.2105 Third Quartile 0.4625 Interquartile range 0.3669 Mean 0.2821 Standard Deviation 0.2517 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2821 Std With No Outliers 0.2517 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 5 Number of Topics of the Experiment 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries unswTitleBaseline Topic 026 33.33 Topic 039 50.00 0.8 Topic 027 10.53 Topic 040 14.29 Topic 028 21.05 Topic 041 0.00 0.6 Topic 029 33.33 Topic 042 0.00 Topic 030 83.33 Topic 043 12.50 Topic 031 22.03 Topic 044 21.05 0.4 Topic 032 70.97 Topic 045 0.00 Topic 033 45.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 12.50 Difference Topic 035 16.67 Topic 048 70.83 0 Topic 036 0.00 Topic 049 50.00 Topic 037 31.25 Topic 050 6.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 240 unsw unswNarrBaseline GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 15,905 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 265 Pooled true Geometric Mean Average Precision 0.0854 unswNarrBaseline Binary Preference (BPREF) 0.2313 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 59.86 unswNarrBaseline 10 49.40 90% 20 41.66 80% 30 37.01 40 33.68 70% 50 32.42 Average Precision 60% 60 24.96 70 17.37 50% 80 12.94 40% 90 8.61 30% 100 5.55 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 27.58 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9384 Minimum 0.0000 First Quartile 0.0516 Second Quartile 0.1650 Third Quartile 0.4421 Interquartile range 0.3904 Mean 0.2758 Standard Deviation 0.2670 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.9384 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2758 Std With No Outliers 0.2670 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 6 Number of Topics of the Experiment 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries unswNarrBaseline Topic 026 30.94 Topic 039 45.42 0.8 Topic 027 10.26 Topic 040 15.86 Topic 028 3.35 Topic 041 0.00 0.6 Topic 029 4.55 Topic 042 36.67 Topic 030 77.22 Topic 043 16.50 Topic 031 5.37 Topic 044 17.23 0.4 Topic 032 93.84 Topic 045 3.96 Topic 033 38.88 Topic 046 66.67 0.2 Topic 034 2.30 Topic 047 11.41 Difference Topic 035 43.80 Topic 048 68.54 0 Topic 036 0.00 Topic 049 50.00 Topic 037 21.38 Topic 050 11.06 −0.2 Topic 038 14.29 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 241 unsw unswNarrBaseline GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 28.80 unswNarrBaseline 10 docs 23.60 90% 15 docs 21.87 80% 20 docs 21.00 30 docs 19.20 70% 100 docs 8.80 60% 200 docs 4.84 R−Precision 500 docs 2.07 50% 1000 docs 1.06 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 25.88 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8710 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1579 Third Quartile 0.4062 Interquartile range 0.4062 Mean 0.2588 Standard Deviation 0.2769 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8710 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2588 Std With No Outliers 0.2769 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries unswNarrBaseline Topic 026 33.33 Topic 039 37.50 0.8 Topic 027 10.53 Topic 040 14.29 Topic 028 5.26 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 0.00 Topic 030 83.33 Topic 043 12.50 Topic 031 22.03 Topic 044 15.79 0.4 Topic 032 87.10 Topic 045 0.00 Topic 033 50.00 Topic 046 66.67 0.2 Topic 034 0.00 Topic 047 16.67 Difference Topic 035 33.33 Topic 048 70.83 0 Topic 036 0.00 Topic 049 50.00 Topic 037 31.25 Topic 050 6.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 242 unsw unswNarrMap GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 15,905 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 262 Pooled false Geometric Mean Average Precision 0.0081 unswNarrMap Binary Preference (BPREF) 0.0410 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 8.89 unswNarrMap 10 8.45 90% 20 8.06 80% 30 7.21 40 7.10 70% 50 6.05 Average Precision 60% 60 3.90 70 3.21 50% 80 1.66 40% 90 1.23 30% 100 0.69 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 4.00 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.3371 Minimum 0.0000 First Quartile 0.0031 Second Quartile 0.0098 Third Quartile 0.0501 Interquartile range 0.0470 Mean 0.0400 Standard Deviation 0.0701 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1106 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0276 Std With No Outliers 0.0336 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries unswNarrMap Topic 026 0.56 Topic 039 3.50 0.8 Topic 027 10.26 Topic 040 0.30 Topic 028 0.31 Topic 041 0.00 0.6 Topic 029 0.53 Topic 042 0.33 Topic 030 6.55 Topic 043 0.54 Topic 031 3.31 Topic 044 4.78 0.4 Topic 032 5.71 Topic 045 1.42 Topic 033 33.71 Topic 046 3.90 0.2 Topic 034 0.13 Topic 047 0.98 Difference Topic 035 3.11 Topic 048 8.06 0 Topic 036 0.00 Topic 049 0.09 Topic 037 0.81 Topic 050 11.06 −0.2 Topic 038 0.12 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 243 unsw unswNarrMap GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 1.60 unswNarrMap 10 docs 3.60 90% 15 docs 4.00 80% 20 docs 4.20 30 docs 4.67 70% 100 docs 2.80 60% 200 docs 2.72 R−Precision 500 docs 1.54 50% 1000 docs 1.05 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 4.06 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.5000 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.0354 Interquartile range 0.0354 Mean 0.0406 Standard Deviation 0.1045 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.0667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0109 Std With No Outliers 0.0229 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries unswNarrMap Topic 026 0.00 Topic 039 6.25 0.8 Topic 027 10.53 Topic 040 0.00 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 0.00 Topic 030 0.00 Topic 043 0.00 Topic 031 16.95 Topic 044 2.63 0.4 Topic 032 6.45 Topic 045 0.00 Topic 033 50.00 Topic 046 0.00 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 2.08 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 6.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 244 unsw unswTitleF46 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 12,919 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 261 Pooled false Geometric Mean Average Precision 0.0600 unswTitleF46 Binary Preference (BPREF) 0.2307 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 59.19 unswTitleF46 10 45.72 90% 20 38.67 80% 30 34.63 40 27.38 70% 50 25.37 Average Precision 60% 60 17.59 70 7.14 50% 80 4.56 40% 90 1.89 30% 100 0.78 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 22.15 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6667 Minimum 0.0000 First Quartile 0.0490 Second Quartile 0.1365 Third Quartile 0.4004 Interquartile range 0.3514 Mean 0.2215 Standard Deviation 0.2138 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2215 Std With No Outliers 0.2138 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 6 Number of Topics of the Experiment 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries unswTitleF46 Topic 026 15.04 Topic 039 34.07 0.8 Topic 027 12.32 Topic 040 13.65 Topic 028 5.09 Topic 041 0.00 0.6 Topic 029 16.33 Topic 042 1.04 Topic 030 61.69 Topic 043 4.33 Topic 031 5.09 Topic 044 13.80 0.4 Topic 032 53.54 Topic 045 2.38 Topic 033 44.77 Topic 046 66.67 0.2 Topic 034 38.46 Topic 047 9.80 Difference Topic 035 28.06 Topic 048 51.55 0 Topic 036 0.00 Topic 049 50.00 Topic 037 13.17 Topic 050 12.73 −0.2 Topic 038 0.12 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 245 unsw unswTitleF46 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 29.60 unswTitleF46 10 docs 24.00 90% 15 docs 20.80 80% 20 docs 19.40 30 docs 16.53 70% 100 docs 7.00 60% 200 docs 4.04 R−Precision 500 docs 1.83 50% 1000 docs 1.04 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 26.87 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.6667 Minimum 0.0000 First Quartile 0.1104 Second Quartile 0.2203 Third Quartile 0.4062 Interquartile range 0.2958 Mean 0.2687 Standard Deviation 0.2167 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2687 Std With No Outliers 0.2167 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 5 Number of Topics of the Experiment 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries unswTitleF46 Topic 026 22.22 Topic 039 37.50 0.8 Topic 027 15.79 Topic 040 21.43 Topic 028 21.05 Topic 041 0.00 0.6 Topic 029 33.33 Topic 042 0.00 Topic 030 66.67 Topic 043 12.50 Topic 031 22.03 Topic 044 13.16 0.4 Topic 032 54.84 Topic 045 0.00 Topic 033 55.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 16.67 Difference Topic 035 33.33 Topic 048 58.33 0 Topic 036 0.00 Topic 049 50.00 Topic 037 31.25 Topic 050 6.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 246 unsw unswNarrF41 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 15,905 Source Language English Relevant 378 Topic Fields title, description, narrative Relevant retrieved 262 Pooled false Geometric Mean Average Precision 0.0083 unswNarrF41 Binary Preference (BPREF) 0.0411 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 8.91 unswNarrF41 10 8.48 90% 20 8.08 80% 30 7.23 40 7.12 70% 50 6.06 Average Precision 60% 60 3.91 70 3.21 50% 80 1.66 40% 90 1.23 30% 100 0.69 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 4.01 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.3371 Minimum 0.0000 First Quartile 0.0033 Second Quartile 0.0102 Third Quartile 0.0501 Interquartile range 0.0468 Mean 0.0401 Standard Deviation 0.0701 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1106 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0277 Std With No Outliers 0.0336 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries unswNarrF41 Topic 026 0.58 Topic 039 3.50 0.8 Topic 027 10.26 Topic 040 0.34 Topic 028 0.36 Topic 041 0.00 0.6 Topic 029 0.53 Topic 042 0.33 Topic 030 6.55 Topic 043 0.55 Topic 031 3.31 Topic 044 4.78 0.4 Topic 032 5.71 Topic 045 1.42 Topic 033 33.71 Topic 046 3.90 0.2 Topic 034 0.14 Topic 047 1.02 Difference Topic 035 3.19 Topic 048 8.06 0 Topic 036 0.00 Topic 049 0.09 Topic 037 0.81 Topic 050 11.06 −0.2 Topic 038 0.12 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 247 unsw unswNarrF41 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 1.60 unswNarrF41 10 docs 3.60 90% 15 docs 4.00 80% 20 docs 4.20 30 docs 4.67 70% 100 docs 2.80 60% 200 docs 2.72 R−Precision 500 docs 1.57 50% 1000 docs 1.05 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 4.06 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.5000 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.0354 Interquartile range 0.0354 Mean 0.0406 Standard Deviation 0.1045 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.0667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0109 Std With No Outliers 0.0229 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries unswNarrF41 Topic 026 0.00 Topic 039 6.25 0.8 Topic 027 10.53 Topic 040 0.00 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 0.00 Topic 030 0.00 Topic 043 0.00 Topic 031 16.95 Topic 044 2.63 0.4 Topic 032 6.45 Topic 045 0.00 Topic 033 50.00 Topic 046 0.00 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 2.08 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 6.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 248 xldb XLDBGeoENAut02 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 22,483 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 300 Pooled false Geometric Mean Average Precision 0.0268 Scope as topic term, no geoexpansion, QE 32 terms, Binary Preference (BPREF) 0.1397 20 top-kdocs, relaxed query construction Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 36.69 XLDBGeoENAut02 10 25.36 90% 20 21.54 80% 30 21.03 40 17.13 70% 50 16.18 Average Precision 60% 60 14.30 70 11.96 50% 80 10.72 40% 90 7.61 30% 100 5.59 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 15.79 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8876 Minimum 0.0000 First Quartile 0.0063 Second Quartile 0.0617 Third Quartile 0.2071 Interquartile range 0.2008 Mean 0.1579 Standard Deviation 0.2344 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4247 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0980 Std With No Outliers 0.1137 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries XLDBGeoENAut02 Topic 026 3.28 Topic 039 8.45 0.8 Topic 027 5.49 Topic 040 42.47 Topic 028 17.53 Topic 041 0.00 0.6 Topic 029 24.16 Topic 042 0.25 Topic 030 6.17 Topic 043 4.87 Topic 031 10.56 Topic 044 19.56 0.4 Topic 032 88.76 Topic 045 0.12 Topic 033 0.64 Topic 046 0.59 0.2 Topic 034 25.65 Topic 047 12.68 Difference Topic 035 2.07 Topic 048 80.50 0 Topic 036 0.00 Topic 049 26.67 Topic 037 12.41 Topic 050 0.10 −0.2 Topic 038 1.75 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 249 xldb XLDBGeoENAut02 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 20.80 XLDBGeoENAut02 10 docs 18.00 90% 15 docs 17.07 80% 20 docs 15.60 30 docs 14.40 70% 100 docs 7.28 60% 200 docs 4.40 R−Precision 500 docs 2.18 50% 1000 docs 1.20 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 15.28 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8387 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0526 Third Quartile 0.2566 Interquartile range 0.2566 Mean 0.1528 Standard Deviation 0.2334 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4286 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0970 Std With No Outliers 0.1362 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries XLDBGeoENAut02 Topic 026 0.00 Topic 039 6.25 0.8 Topic 027 5.26 Topic 040 42.86 Topic 028 31.58 Topic 041 0.00 0.6 Topic 029 33.33 Topic 042 0.00 Topic 030 0.00 Topic 043 12.50 Topic 031 13.56 Topic 044 23.68 0.4 Topic 032 83.87 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 33.33 Topic 047 8.33 Difference Topic 035 0.00 Topic 048 75.00 0 Topic 036 0.00 Topic 049 0.00 Topic 037 12.50 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 250 xldb XLDBGeoENAut05 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 10,652 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 260 Pooled false Geometric Mean Average Precision 0.0468 topic 16 QE terms expansion, top-20k docs, scope Binary Preference (BPREF) 0.1994 expansion to 10 scopes Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 54.28 XLDBGeoENAut05 10 37.95 90% 20 28.74 80% 30 26.64 40 22.26 70% 50 21.51 Average Precision 60% 60 19.68 70 17.01 50% 80 12.35 40% 90 11.28 30% 100 9.39 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 21.45 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.9317 Minimum 0.0000 First Quartile 0.0479 Second Quartile 0.1083 Third Quartile 0.2828 Interquartile range 0.2349 Mean 0.2145 Standard Deviation 0.2387 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5833 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1618 Std With No Outliers 0.1575 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 6 Number of Topics of the Experiment 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries XLDBGeoENAut05 Topic 026 3.67 Topic 039 26.18 0.8 Topic 027 7.89 Topic 040 42.41 Topic 028 6.58 Topic 041 0.00 0.6 Topic 029 33.12 Topic 042 26.67 Topic 030 1.42 Topic 043 5.84 Topic 031 37.25 Topic 044 25.95 0.4 Topic 032 93.17 Topic 045 10.83 Topic 033 26.65 Topic 046 9.26 0.2 Topic 034 0.00 Topic 047 5.17 Difference Topic 035 17.73 Topic 048 70.83 0 Topic 036 0.00 Topic 049 58.33 Topic 037 2.71 Topic 050 10.24 −0.2 Topic 038 14.29 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 251 xldb XLDBGeoENAut05 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 28.80 XLDBGeoENAut05 10 docs 24.00 90% 15 docs 22.40 80% 20 docs 21.20 30 docs 18.40 70% 100 docs 8.36 60% 200 docs 4.94 R−Precision 500 docs 2.08 50% 1000 docs 1.04 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 21.97 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.8710 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1250 Third Quartile 0.3600 Interquartile range 0.3600 Mean 0.2197 Standard Deviation 0.2367 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8710 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2197 Std With No Outliers 0.2367 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries XLDBGeoENAut05 Topic 026 0.00 Topic 039 31.25 0.8 Topic 027 10.53 Topic 040 35.71 Topic 028 10.53 Topic 041 0.00 0.6 Topic 029 44.44 Topic 042 0.00 Topic 030 0.00 Topic 043 12.50 Topic 031 38.98 Topic 044 36.84 0.4 Topic 032 87.10 Topic 045 33.33 Topic 033 25.00 Topic 046 0.00 0.2 Topic 034 0.00 Topic 047 12.50 Difference Topic 035 16.67 Topic 048 70.83 0 Topic 036 0.00 Topic 049 50.00 Topic 037 6.25 Topic 050 26.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 252 xldb XLDBGeoManualEN GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction MANUAL Retrieved 3,324 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 192 Pooled false Geometric Mean Average Precision 0.0654 Manual Query Binary Preference (BPREF) 0.3142 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 67.72 XLDBGeoManualEN 10 58.07 90% 20 41.48 80% 30 38.22 40 33.87 70% 50 30.42 Average Precision 60% 60 26.73 70 19.99 50% 80 15.56 40% 90 11.70 30% 100 11.58 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 30.34 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0604 Second Quartile 0.2343 Third Quartile 0.4979 Interquartile range 0.4375 Mean 0.3034 Standard Deviation 0.3075 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 1.0000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.3034 Std With No Outliers 0.3075 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 5 Number of Topics of the Experiment 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries XLDBGeoManualEN Topic 026 23.77 Topic 039 57.78 0.8 Topic 027 7.82 Topic 040 23.43 Topic 028 7.41 Topic 041 0.00 0.6 Topic 029 3.70 Topic 042 0.00 Topic 030 97.62 Topic 043 3.12 Topic 031 9.60 Topic 044 13.79 0.4 Topic 032 74.88 Topic 045 33.88 Topic 033 47.13 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 5.29 Difference Topic 035 24.83 Topic 048 70.23 0 Topic 036 0.00 Topic 049 25.00 Topic 037 6.29 Topic 050 23.00 −0.2 Topic 038 100.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 253 xldb XLDBGeoManualEN GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 38.40 XLDBGeoManualEN 10 docs 29.60 90% 15 docs 24.27 80% 20 docs 22.40 30 docs 19.73 70% 100 docs 7.20 60% 200 docs 3.70 R−Precision 500 docs 1.53 50% 1000 docs 0.77 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 33.60 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.1215 Second Quartile 0.3333 Third Quartile 0.5000 Interquartile range 0.3785 Mean 0.3360 Standard Deviation 0.2797 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 1.0000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.3360 Std With No Outliers 0.2797 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 5 Number of Topics of the Experiment 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries XLDBGeoManualEN Topic 026 33.33 Topic 039 43.75 0.8 Topic 027 15.79 Topic 040 35.71 Topic 028 10.53 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 83.33 Topic 043 12.50 Topic 031 10.17 Topic 044 21.05 0.4 Topic 032 77.42 Topic 045 33.33 Topic 033 50.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 12.50 Difference Topic 035 16.67 Topic 048 70.83 0 Topic 036 0.00 Topic 049 50.00 Topic 037 18.75 Topic 050 33.33 −0.2 Topic 038 100.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 254 xldb XLDBGeoENAut03_2 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 21,228 Source Language English Relevant 363 Topic Fields title, description Relevant retrieved 240 Pooled true Geometric Mean Average Precision 0.0235 XLDBGeoENAut03 run, improved Binary Preference (BPREF) 0.1912 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 50.31 XLDBGeoENAut03_2 10 44.28 90% 20 31.97 80% 30 29.25 40 23.36 70% 50 22.07 Average Precision 60% 60 16.40 70 12.12 50% 80 8.85 40% 90 3.17 30% 100 2.46 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 20.79 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7746 Minimum 0.0000 First Quartile 0.0114 Second Quartile 0.1462 Third Quartile 0.3866 Interquartile range 0.3752 Mean 0.2079 Standard Deviation 0.2365 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7746 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2079 Std With No Outliers 0.2365 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries XLDBGeoENAut03_2 Topic 026 1.29 Topic 039 23.93 0.8 Topic 027 0.72 Topic 040 42.47 Topic 028 14.62 Topic 041 0.00 0.6 Topic 029 37.96 Topic 042 61.11 Topic 030 45.91 Topic 043 14.83 Topic 031 19.48 Topic 044 3.12 0.4 Topic 032 77.46 Topic 045 0.05 Topic 033 2.65 Topic 046 40.74 0.2 Topic 034 21.98 Topic 047 7.23 Difference Topic 035 24.47 Topic 048 69.73 0 Topic 036 0.00 Topic 049 0.00 Topic 037 8.74 Topic 050 0.00 −0.2 Topic 038 1.28 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 255 xldb XLDBGeoENAut03_2 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 24.00 XLDBGeoENAut03_2 10 docs 22.80 90% 15 docs 19.47 80% 20 docs 17.00 30 docs 14.67 70% 100 docs 6.84 60% 200 docs 4.00 R−Precision 500 docs 1.86 50% 1000 docs 0.96 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 21.53 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7083 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1250 Third Quartile 0.3571 Interquartile range 0.3571 Mean 0.2153 Standard Deviation 0.2252 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7083 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2153 Std With No Outliers 0.2252 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries XLDBGeoENAut03_2 Topic 026 0.00 Topic 039 25.00 0.8 Topic 027 0.00 Topic 040 42.86 Topic 028 21.05 Topic 041 0.00 0.6 Topic 029 55.56 Topic 042 50.00 Topic 030 50.00 Topic 043 12.50 Topic 031 22.03 Topic 044 10.53 0.4 Topic 032 64.52 Topic 045 0.00 Topic 033 5.00 Topic 046 33.33 0.2 Topic 034 33.33 Topic 047 12.50 Difference Topic 035 16.67 Topic 048 70.83 0 Topic 036 0.00 Topic 049 0.00 Topic 037 12.50 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 256 xldb XLDBGeoENAut03 GC-MONO-EN-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 21,937 Source Language English Relevant 378 Topic Fields title, description Relevant retrieved 151 Pooled true Geometric Mean Average Precision 0.0096 Run with geosim, final correction Binary Preference (BPREF) 0.1812 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision 100% 0 47.52 XLDBGeoENAut03 10 37.27 90% 20 26.16 80% 30 24.45 40 22.20 70% 50 20.92 Average Precision 60% 60 16.77 70 11.11 50% 80 7.87 40% 90 2.91 30% 100 2.72 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 18.67 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7681 Minimum 0.0000 First Quartile 0.0004 Second Quartile 0.0490 Third Quartile 0.2300 Interquartile range 0.2296 Mean 0.1867 Standard Deviation 0.2728 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4247 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0608 Std With No Outliers 0.1002 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries XLDBGeoENAut03 Topic 026 6.72 Topic 039 16.51 0.8 Topic 027 0.00 Topic 040 42.47 Topic 028 12.31 Topic 041 0.00 0.6 Topic 029 5.56 Topic 042 64.29 Topic 030 67.86 Topic 043 15.63 Topic 031 2.88 Topic 044 3.03 0.4 Topic 032 76.81 Topic 045 0.05 Topic 033 1.12 Topic 046 66.67 0.2 Topic 034 0.00 Topic 047 6.90 Difference Topic 035 2.08 Topic 048 69.68 0 Topic 036 0.00 Topic 049 0.00 Topic 037 4.90 Topic 050 1.39 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 257 xldb XLDBGeoENAut03 GC-MONO-EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual English track − Retrieved documents vs Precision 100% 5 docs 22.40 XLDBGeoENAut03 10 docs 20.80 90% 15 docs 18.40 80% 20 docs 16.00 30 docs 13.20 70% 100 docs 5.24 60% 200 docs 2.76 R−Precision 500 docs 1.19 50% 1000 docs 0.60 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 19.47 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment Maximum 0.7083 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1111 Third Quartile 0.2946 Interquartile range 0.2946 Mean 0.1947 Standard Deviation 0.2360 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7083 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1947 Std With No Outliers 0.2360 GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries XLDBGeoENAut03 Topic 026 22.22 Topic 039 25.00 0.8 Topic 027 0.00 Topic 040 42.86 Topic 028 15.79 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 50.00 Topic 030 50.00 Topic 043 12.50 Topic 031 3.39 Topic 044 10.53 0.4 Topic 032 64.52 Topic 045 0.00 Topic 033 10.00 Topic 046 66.67 0.2 Topic 034 0.00 Topic 047 12.50 Difference Topic 035 0.00 Topic 048 70.83 0 Topic 036 0.00 Topic 049 0.00 Topic 037 18.75 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 258 alicante esTD GC-MONO-ES-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Spanish; Castilian Relevant 2,054 Topic Fields title, description Relevant retrieved 1,819 Pooled true Geometric Mean Average Precision 0.2182 Title and Description Binary Preference (BPREF) 0.3665 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision 100% 0 71.04 esTD 10 56.86 90% 20 48.51 80% 30 42.92 40 38.60 70% 50 35.48 Average Precision 60% 60 32.23 70 27.55 50% 80 22.42 40% 90 17.08 30% 100 7.26 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 35.08 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.9708 Minimum 0.0197 First Quartile 0.1368 Second Quartile 0.3246 Third Quartile 0.5148 Interquartile range 0.3779 Mean 0.3508 Standard Deviation 0.2744 Lower Outlier Threshold 0.0197 Upper Outlier Threshold 0.9708 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.3508 Std With No Outliers 0.2744 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 5 Number of Topics of the Experiment 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries esTD Topic 026 14.66 Topic 039 32.46 0.8 Topic 027 2.89 Topic 040 77.64 Topic 028 39.37 Topic 041 38.78 0.6 Topic 029 51.80 Topic 042 28.32 Topic 030 44.99 Topic 043 3.88 Topic 031 72.52 Topic 044 40.33 0.4 Topic 032 97.08 Topic 045 4.27 Topic 033 1.97 Topic 046 66.67 0.2 Topic 034 15.65 Topic 047 2.46 Difference Topic 035 16.45 Topic 048 74.87 0 Topic 036 51.37 Topic 049 50.89 Topic 037 21.59 Topic 050 15.37 −0.2 Topic 038 10.76 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 259 alicante esTD GC-MONO-ES-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision 100% 5 docs 56.00 esTD 10 docs 52.80 90% 15 docs 49.07 80% 20 docs 47.00 30 docs 42.53 70% 100 docs 33.48 60% 200 docs 24.80 R−Precision 500 docs 13.30 50% 1000 docs 7.28 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 35.83 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.9000 Minimum 0.0256 First Quartile 0.1588 Second Quartile 0.3208 Third Quartile 0.5836 Interquartile range 0.4248 Mean 0.3583 Standard Deviation 0.2414 Lower Outlier Threshold 0.0256 Upper Outlier Threshold 0.9000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.3583 Std With No Outliers 0.2414 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 4 Number of Topics of the Experiment 3.5 3 2.5 2 1.5 1 0.5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries esTD Topic 026 16.67 Topic 039 38.81 0.8 Topic 027 2.56 Topic 040 70.50 Topic 028 30.56 Topic 041 40.00 0.6 Topic 029 57.58 Topic 042 32.08 Topic 030 43.05 Topic 043 12.50 Topic 031 61.18 Topic 044 46.60 0.4 Topic 032 90.00 Topic 045 8.33 Topic 033 6.00 Topic 046 60.71 0.2 Topic 034 13.51 Topic 047 3.39 Difference Topic 035 21.05 Topic 048 66.04 0 Topic 036 60.91 Topic 049 51.15 Topic 037 20.69 Topic 050 22.00 −0.2 Topic 038 20.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 260 alicante esTDN GC-MONO-ES-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Spanish; Castilian Relevant 2,054 Topic Fields title, description Relevant retrieved 1,733 Pooled true Geometric Mean Average Precision 0.0962 Title, Description and Narrative Binary Preference (BPREF) 0.3400 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision 100% 0 67.84 esTDN 10 50.74 90% 20 43.80 80% 30 37.61 40 35.84 70% 50 33.16 Average Precision 60% 60 29.49 70 27.03 50% 80 23.01 40% 90 14.50 30% 100 6.75 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 32.37 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.9591 Minimum 0.0000 First Quartile 0.0755 Second Quartile 0.2143 Third Quartile 0.5113 Interquartile range 0.4358 Mean 0.3237 Standard Deviation 0.2986 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.9591 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.3237 Std With No Outliers 0.2986 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 5 Number of Topics of the Experiment 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries esTDN Topic 026 15.18 Topic 039 30.75 0.8 Topic 027 3.89 Topic 040 77.24 Topic 028 37.65 Topic 041 38.12 0.6 Topic 029 68.63 Topic 042 45.30 Topic 030 37.18 Topic 043 8.54 Topic 031 69.22 Topic 044 5.69 0.4 Topic 032 95.91 Topic 045 1.66 Topic 033 2.31 Topic 046 77.26 0.2 Topic 034 20.67 Topic 047 0.00 Difference Topic 035 8.17 Topic 048 81.18 0 Topic 036 21.43 Topic 049 38.69 Topic 037 0.00 Topic 050 12.85 −0.2 Topic 038 11.78 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 261 alicante esTDN GC-MONO-ES-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision 100% 5 docs 49.60 esTDN 10 docs 47.20 90% 15 docs 43.73 80% 20 docs 40.40 30 docs 37.47 70% 100 docs 29.84 60% 200 docs 23.10 R−Precision 500 docs 12.31 50% 1000 docs 6.93 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 33.77 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.9000 Minimum 0.0000 First Quartile 0.1032 Second Quartile 0.2818 Third Quartile 0.5194 Interquartile range 0.4162 Mean 0.3377 Standard Deviation 0.2667 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.9000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.3377 Std With No Outliers 0.2667 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 4 Number of Topics of the Experiment 3.5 3 2.5 2 1.5 1 0.5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries esTDN Topic 026 16.67 Topic 039 31.34 0.8 Topic 027 7.69 Topic 040 73.38 Topic 028 33.33 Topic 041 38.67 0.6 Topic 029 60.61 Topic 042 49.06 Topic 030 40.40 Topic 043 25.00 Topic 031 73.33 Topic 044 9.71 0.4 Topic 032 90.00 Topic 045 0.00 Topic 033 9.00 Topic 046 71.43 0.2 Topic 034 27.03 Topic 047 0.00 Difference Topic 035 10.53 Topic 048 70.57 0 Topic 036 28.18 Topic 049 44.24 Topic 037 0.00 Topic 050 14.00 −0.2 Topic 038 20.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 262 alicante esTDNGeoNames GC-MONO-ES-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction MANUAL Retrieved 25,000 Source Language Spanish; Castilian Relevant 2,054 Topic Fields title, description, narrative Relevant retrieved 1,069 Pooled true Geometric Mean Average Precision 0.0036 Title, Description and Narrative with GeoNames Binary Preference (BPREF) 0.1736 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision 100% 0 33.12 esTDNGeoNames 10 25.11 90% 20 20.55 80% 30 17.15 40 16.65 70% 50 16.53 Average Precision 60% 60 15.52 70 14.24 50% 80 10.01 40% 90 7.17 30% 100 5.40 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 15.25 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.9016 Minimum 0.0000 First Quartile 0.0001 Second Quartile 0.0047 Third Quartile 0.1699 Interquartile range 0.1699 Mean 0.1525 Standard Deviation 0.2635 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4128 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0508 Std With No Outliers 0.1021 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries esTDNGeoNames Topic 026 15.18 Topic 039 4.34 0.8 Topic 027 10.35 Topic 040 77.55 Topic 028 0.47 Topic 041 22.45 0.6 Topic 029 0.07 Topic 042 4.58 Topic 030 0.00 Topic 043 0.11 Topic 031 41.28 Topic 044 0.01 0.4 Topic 032 90.16 Topic 045 0.11 Topic 033 54.64 Topic 046 6.47 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 0.15 Topic 048 52.18 0 Topic 036 0.00 Topic 049 0.06 Topic 037 0.00 Topic 050 1.04 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 263 alicante esTDNGeoNames GC-MONO-ES-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision 100% 5 docs 20.80 esTDNGeoNames 10 docs 20.40 90% 15 docs 20.53 80% 20 docs 21.20 30 docs 19.60 70% 100 docs 15.64 60% 200 docs 12.54 R−Precision 500 docs 7.47 50% 1000 docs 4.28 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 16.23 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.8077 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0184 Third Quartile 0.1850 Interquartile range 0.1850 Mean 0.1623 Standard Deviation 0.2661 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.2400 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0378 Std With No Outliers 0.0668 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries esTDNGeoNames Topic 026 16.67 Topic 039 4.48 0.8 Topic 027 10.26 Topic 040 71.94 Topic 028 0.00 Topic 041 24.00 0.6 Topic 029 0.00 Topic 042 5.66 Topic 030 0.00 Topic 043 0.00 Topic 031 48.63 Topic 044 0.00 0.4 Topic 032 80.77 Topic 045 0.00 Topic 033 71.00 Topic 046 10.71 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 57.74 0 Topic 036 0.00 Topic 049 1.84 Topic 037 0.00 Topic 050 2.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 264 berkeley BKGeoS1 GC-MONO-ES-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Spanish; Castilian Relevant 2,054 Topic Fields title, description Relevant retrieved 1,646 Pooled true Geometric Mean Average Precision 0.0695 Baseline TD run using standard logistic regression Binary Preference (BPREF) 0.3406 algorithms and blind feedback Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision 100% 0 63.66 BKGeoS1 10 44.62 90% 20 42.22 80% 30 40.51 40 37.66 70% 50 34.19 Average Precision 60% 60 30.11 70 25.16 50% 80 21.68 40% 90 16.68 30% 100 5.32 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 31.82 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.9782 Minimum 0.0000 First Quartile 0.0377 Second Quartile 0.2212 Third Quartile 0.6252 Interquartile range 0.5875 Mean 0.3182 Standard Deviation 0.3239 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.9782 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.3182 Std With No Outliers 0.3239 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries BKGeoS1 Topic 026 0.05 Topic 039 1.48 0.8 Topic 027 0.00 Topic 040 60.86 Topic 028 11.37 Topic 041 22.12 0.6 Topic 029 61.67 Topic 042 31.72 Topic 030 68.69 Topic 043 1.33 Topic 031 65.07 Topic 044 41.34 0.4 Topic 032 97.82 Topic 045 4.56 Topic 033 0.53 Topic 046 81.92 0.2 Topic 034 25.28 Topic 047 12.03 Difference Topic 035 4.76 Topic 048 80.52 0 Topic 036 0.00 Topic 049 78.74 Topic 037 11.95 Topic 050 27.21 −0.2 Topic 038 4.53 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 265 berkeley BKGeoS1 GC-MONO-ES-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision 100% 5 docs 41.60 BKGeoS1 10 docs 39.20 90% 15 docs 39.47 80% 20 docs 38.40 30 docs 37.33 70% 100 docs 31.76 60% 200 docs 22.66 R−Precision 500 docs 12.13 50% 1000 docs 6.58 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 32.11 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.9000 Minimum 0.0000 First Quartile 0.0432 Second Quartile 0.3243 Third Quartile 0.6110 Interquartile range 0.5678 Mean 0.3211 Standard Deviation 0.2958 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.9000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.3211 Std With No Outliers 0.2958 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 6 Number of Topics of the Experiment 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries BKGeoS1 Topic 026 0.00 Topic 039 1.49 0.8 Topic 027 0.00 Topic 040 62.59 Topic 028 5.56 Topic 041 33.33 0.6 Topic 029 60.61 Topic 042 37.74 Topic 030 66.89 Topic 043 8.33 Topic 031 56.08 Topic 044 43.69 0.4 Topic 032 90.00 Topic 045 8.33 Topic 033 1.00 Topic 046 75.00 0.2 Topic 034 32.43 Topic 047 22.03 Difference Topic 035 5.26 Topic 048 72.08 0 Topic 036 0.00 Topic 049 70.51 Topic 037 13.79 Topic 050 36.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 266 berkeley BKGeoS2 GC-MONO-ES-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Spanish; Castilian Relevant 2,054 Topic Fields title, description, narrative Relevant retrieved 1,702 Pooled true Geometric Mean Average Precision 0.0622 Baseline TDN with standard logistic regression Binary Preference (BPREF) 0.3159 algorithms plus blind feedback Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision 100% 0 60.42 BKGeoS2 10 42.41 90% 20 38.92 80% 30 37.79 40 35.16 70% 50 32.88 Average Precision 60% 60 29.23 70 25.09 50% 80 20.87 40% 90 15.59 30% 100 5.60 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 30.03 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.9750 Minimum 0.0002 First Quartile 0.0174 Second Quartile 0.1834 Third Quartile 0.5828 Interquartile range 0.5654 Mean 0.3003 Standard Deviation 0.3199 Lower Outlier Threshold 0.0002 Upper Outlier Threshold 0.9750 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.3003 Std With No Outliers 0.3199 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries BKGeoS2 Topic 026 0.04 Topic 039 26.80 0.8 Topic 027 0.02 Topic 040 69.34 Topic 028 11.66 Topic 041 20.18 0.6 Topic 029 53.34 Topic 042 50.82 Topic 030 56.62 Topic 043 1.44 Topic 031 63.26 Topic 044 14.62 0.4 Topic 032 97.50 Topic 045 0.51 Topic 033 19.32 Topic 046 83.30 0.2 Topic 034 18.34 Topic 047 4.07 Difference Topic 035 3.24 Topic 048 74.68 0 Topic 036 0.02 Topic 049 73.34 Topic 037 0.02 Topic 050 6.47 −0.2 Topic 038 1.84 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 267 berkeley BKGeoS2 GC-MONO-ES-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision 100% 5 docs 43.20 BKGeoS2 10 docs 37.20 90% 15 docs 37.87 80% 20 docs 37.80 30 docs 35.20 70% 100 docs 30.68 60% 200 docs 22.34 R−Precision 500 docs 12.06 50% 1000 docs 6.81 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 29.94 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.9154 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.2090 Third Quartile 0.5906 Interquartile range 0.5906 Mean 0.2994 Standard Deviation 0.2997 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.9154 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2994 Std With No Outliers 0.2997 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries BKGeoS2 Topic 026 0.00 Topic 039 20.90 0.8 Topic 027 0.00 Topic 040 71.22 Topic 028 16.67 Topic 041 24.00 0.6 Topic 029 57.58 Topic 042 52.83 Topic 030 55.63 Topic 043 0.00 Topic 031 63.53 Topic 044 21.36 0.4 Topic 032 91.54 Topic 045 0.00 Topic 033 34.00 Topic 046 71.43 0.2 Topic 034 18.92 Topic 047 6.78 Difference Topic 035 5.26 Topic 048 64.15 0 Topic 036 0.00 Topic 049 68.66 Topic 037 0.00 Topic 050 4.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 268 daedalus GCesNA GC-MONO-ES-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction MANUAL Retrieved 4,744 Source Language Spanish; Castilian Relevant 1,965 Topic Fields title, description Relevant retrieved 523 Pooled true Geometric Mean Average Precision 0.0044 Mandatory run Binary Preference (BPREF) 0.1534 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision 100% 0 46.90 GCesNA 10 33.03 90% 20 21.89 80% 30 16.67 40 11.79 70% 50 10.93 Average Precision 60% 60 9.97 70 4.70 50% 80 4.07 40% 90 0.34 30% 100 0.00 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 12.73 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.8575 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0546 Third Quartile 0.1612 Interquartile range 0.1612 Mean 0.1273 Standard Deviation 0.2056 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.2161 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0613 Std With No Outliers 0.0725 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries GCesNA Topic 026 0.00 Topic 039 6.73 0.8 Topic 027 0.00 Topic 040 0.00 Topic 028 0.00 Topic 041 11.47 0.6 Topic 029 21.61 Topic 042 10.77 Topic 030 0.00 Topic 043 2.66 Topic 031 0.85 Topic 044 5.46 0.4 Topic 032 85.75 Topic 045 0.00 Topic 033 0.01 Topic 046 3.57 0.2 Topic 034 17.04 Topic 047 0.00 Difference Topic 035 15.82 Topic 048 57.06 0 Topic 036 40.72 Topic 049 10.45 Topic 037 8.37 Topic 050 0.00 −0.2 Topic 038 20.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 269 daedalus GCesNA GC-MONO-ES-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision 100% 5 docs 35.20 GCesNA 10 docs 30.40 90% 15 docs 28.27 80% 20 docs 26.40 30 docs 23.20 70% 100 docs 15.64 60% 200 docs 9.64 R−Precision 500 docs 4.17 50% 1000 docs 2.09 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 17.18 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.8462 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1165 Third Quartile 0.2482 Interquartile range 0.2482 Mean 0.1718 Standard Deviation 0.2234 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1227 Std With No Outliers 0.1479 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries GCesNA Topic 026 0.00 Topic 039 14.93 0.8 Topic 027 0.00 Topic 040 0.00 Topic 028 0.00 Topic 041 13.33 0.6 Topic 029 33.33 Topic 042 28.30 Topic 030 0.00 Topic 043 12.50 Topic 031 3.14 Topic 044 11.65 0.4 Topic 032 84.62 Topic 045 0.00 Topic 033 1.00 Topic 046 10.71 0.2 Topic 034 24.32 Topic 047 0.00 Difference Topic 035 26.32 Topic 048 62.64 0 Topic 036 60.00 Topic 049 12.44 Topic 037 10.34 Topic 050 0.00 −0.2 Topic 038 20.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 270 daedalus GCesAtLg GC-MONO-ES-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction MANUAL Retrieved 23,086 Source Language Spanish; Castilian Relevant 2,015 Topic Fields title, description, narrative Relevant retrieved 1,009 Pooled true Geometric Mean Average Precision 0.0282 All text Left geo run Binary Preference (BPREF) 0.1462 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision 100% 0 39.27 GCesAtLg 10 29.03 90% 20 24.95 80% 30 21.66 40 13.94 70% 50 11.26 Average Precision 60% 60 10.44 70 9.02 50% 80 7.67 40% 90 2.75 30% 100 0.22 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 14.13 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.8612 Minimum 0.0000 First Quartile 0.0122 Second Quartile 0.0689 Third Quartile 0.2049 Interquartile range 0.1927 Mean 0.1413 Standard Deviation 0.1926 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3504 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1113 Std With No Outliers 0.1234 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries GCesAtLg Topic 026 0.00 Topic 039 11.12 0.8 Topic 027 0.00 Topic 040 34.29 Topic 028 1.27 Topic 041 28.73 0.6 Topic 029 12.31 Topic 042 14.86 Topic 030 30.43 Topic 043 2.88 Topic 031 1.07 Topic 044 3.41 0.4 Topic 032 86.12 Topic 045 0.44 Topic 033 0.01 Topic 046 35.04 0.2 Topic 034 6.89 Topic 047 3.72 Difference Topic 035 9.81 Topic 048 34.86 0 Topic 036 0.59 Topic 049 3.77 Topic 037 17.74 Topic 050 8.07 −0.2 Topic 038 5.86 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 271 daedalus GCesAtLg GC-MONO-ES-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision 100% 5 docs 23.20 GCesAtLg 10 docs 23.20 90% 15 docs 22.67 80% 20 docs 22.00 30 docs 20.67 70% 100 docs 17.16 60% 200 docs 12.78 R−Precision 500 docs 6.98 50% 1000 docs 4.04 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 16.58 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.8538 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1053 Third Quartile 0.3249 Interquartile range 0.3249 Mean 0.1658 Standard Deviation 0.2010 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4038 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1371 Std With No Outliers 0.1440 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries GCesAtLg Topic 026 0.00 Topic 039 16.42 0.8 Topic 027 0.00 Topic 040 35.25 Topic 028 0.00 Topic 041 32.00 0.6 Topic 029 18.18 Topic 042 33.96 Topic 030 34.44 Topic 043 0.00 Topic 031 5.10 Topic 044 9.71 0.4 Topic 032 85.38 Topic 045 0.00 Topic 033 1.00 Topic 046 35.71 0.2 Topic 034 0.00 Topic 047 1.69 Difference Topic 035 10.53 Topic 048 40.38 0 Topic 036 5.45 Topic 049 11.06 Topic 037 24.14 Topic 050 14.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 272 daedalus GCesAO GC-MONO-ES-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction MANUAL Retrieved 25,000 Source Language Spanish; Castilian Relevant 2,054 Topic Fields title, description, narrative Relevant retrieved 980 Pooled true Geometric Mean Average Precision 0.0140 All text Or geo run Binary Preference (BPREF) 0.1234 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision 100% 0 35.48 GCesAO 10 25.34 90% 20 18.82 80% 30 17.13 40 13.15 70% 50 10.54 Average Precision 60% 60 9.74 70 8.63 50% 80 7.58 40% 90 2.67 30% 100 0.22 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 12.21 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.8612 Minimum 0.0000 First Quartile 0.0055 Second Quartile 0.0457 Third Quartile 0.1458 Interquartile range 0.1403 Mean 0.1221 Standard Deviation 0.1907 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3486 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0913 Std With No Outliers 0.1150 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries GCesAO Topic 026 0.00 Topic 039 11.12 0.8 Topic 027 1.08 Topic 040 34.29 Topic 028 0.00 Topic 041 26.36 0.6 Topic 029 12.31 Topic 042 4.57 Topic 030 30.43 Topic 043 2.88 Topic 031 1.07 Topic 044 3.41 0.4 Topic 032 86.12 Topic 045 0.44 Topic 033 0.00 Topic 046 21.40 0.2 Topic 034 6.89 Topic 047 0.04 Difference Topic 035 9.81 Topic 048 34.86 0 Topic 036 0.59 Topic 049 3.77 Topic 037 0.00 Topic 050 8.07 −0.2 Topic 038 5.86 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 273 daedalus GCesAO GC-MONO-ES-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision 100% 5 docs 19.20 GCesAO 10 docs 19.20 90% 15 docs 18.67 80% 20 docs 18.60 30 docs 17.87 70% 100 docs 15.48 60% 200 docs 11.78 R−Precision 500 docs 6.72 50% 1000 docs 3.92 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 13.82 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.8538 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0545 Third Quartile 0.2030 Interquartile range 0.2030 Mean 0.1382 Standard Deviation 0.1967 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4038 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1084 Std With No Outliers 0.1311 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries GCesAO Topic 026 0.00 Topic 039 16.42 0.8 Topic 027 2.56 Topic 040 35.25 Topic 028 0.00 Topic 041 26.67 0.6 Topic 029 18.18 Topic 042 1.89 Topic 030 34.44 Topic 043 0.00 Topic 031 5.10 Topic 044 9.71 0.4 Topic 032 85.38 Topic 045 0.00 Topic 033 0.00 Topic 046 28.57 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 10.53 Topic 048 40.38 0 Topic 036 5.45 Topic 049 11.06 Topic 037 0.00 Topic 050 14.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 274 daedalus GCesAA GC-MONO-ES-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction MANUAL Retrieved 5,728 Source Language Spanish; Castilian Relevant 2,015 Topic Fields title, description, narrative Relevant retrieved 487 Pooled true Geometric Mean Average Precision 0.0064 All text And geo run Binary Preference (BPREF) 0.1531 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision 100% 0 48.87 GCesAA 10 33.01 90% 20 25.26 80% 30 18.98 40 14.42 70% 50 8.21 Average Precision 60% 60 7.04 70 5.98 50% 80 5.09 40% 90 0.00 30% 100 0.00 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 13.48 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.8575 Minimum 0.0000 First Quartile 0.0001 Second Quartile 0.0421 Third Quartile 0.1723 Interquartile range 0.1722 Mean 0.1348 Standard Deviation 0.2070 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4031 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0881 Std With No Outliers 0.1232 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries GCesAA Topic 026 0.00 Topic 039 13.00 0.8 Topic 027 0.00 Topic 040 0.00 Topic 028 0.06 Topic 041 10.61 0.6 Topic 029 24.27 Topic 042 26.60 Topic 030 14.89 Topic 043 48.45 Topic 031 1.00 Topic 044 4.30 0.4 Topic 032 85.75 Topic 045 1.67 Topic 033 0.01 Topic 046 38.02 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 10.54 Topic 048 40.31 0 Topic 036 0.20 Topic 049 9.68 Topic 037 0.00 Topic 050 4.21 −0.2 Topic 038 3.33 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 275 daedalus GCesAA GC-MONO-ES-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision 100% 5 docs 37.60 GCesAA 10 docs 33.20 90% 15 docs 30.67 80% 20 docs 27.80 30 docs 25.60 70% 100 docs 15.68 60% 200 docs 9.04 R−Precision 500 docs 3.80 50% 1000 docs 1.95 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 17.01 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.8462 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1000 Third Quartile 0.2621 Interquartile range 0.2621 Mean 0.1701 Standard Deviation 0.2213 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5833 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1419 Std With No Outliers 0.1744 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries GCesAA Topic 026 0.00 Topic 039 17.91 0.8 Topic 027 0.00 Topic 040 0.00 Topic 028 0.00 Topic 041 14.67 0.6 Topic 029 33.33 Topic 042 43.40 Topic 030 23.84 Topic 043 58.33 Topic 031 3.92 Topic 044 11.65 0.4 Topic 032 84.62 Topic 045 8.33 Topic 033 1.00 Topic 046 42.86 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 15.79 Topic 048 42.26 0 Topic 036 1.82 Topic 049 11.52 Topic 037 0.00 Topic 050 10.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 276 daedalus GCesNtLg GC-MONO-ES-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction MANUAL Retrieved 21,736 Source Language Spanish; Castilian Relevant 1,965 Topic Fields title, description Relevant retrieved 1,207 Pooled true Geometric Mean Average Precision 0.0236 Normal text Left geo run Binary Preference (BPREF) 0.1640 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision 100% 0 41.02 GCesNtLg 10 29.36 90% 20 25.44 80% 30 21.66 40 18.85 70% 50 17.28 Average Precision 60% 60 15.97 70 11.64 50% 80 9.18 40% 90 3.52 30% 100 0.41 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 16.12 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.8612 Minimum 0.0000 First Quartile 0.0087 Second Quartile 0.1539 Third Quartile 0.2290 Interquartile range 0.2203 Mean 0.1612 Standard Deviation 0.2010 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3215 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1130 Std With No Outliers 0.1086 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries GCesNtLg Topic 026 0.00 Topic 039 5.03 0.8 Topic 027 0.00 Topic 040 28.45 Topic 028 27.48 Topic 041 32.15 0.6 Topic 029 25.40 Topic 042 1.04 Topic 030 15.39 Topic 043 0.74 Topic 031 0.91 Topic 044 15.87 0.4 Topic 032 86.12 Topic 045 0.21 Topic 033 0.01 Topic 046 4.61 0.2 Topic 034 17.04 Topic 047 4.72 Difference Topic 035 15.82 Topic 048 57.02 0 Topic 036 16.75 Topic 049 19.24 Topic 037 22.06 Topic 050 0.00 −0.2 Topic 038 6.94 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 277 daedalus GCesNtLg GC-MONO-ES-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision 100% 5 docs 24.80 GCesNtLg 10 docs 25.20 90% 15 docs 25.60 80% 20 docs 23.40 30 docs 21.47 70% 100 docs 18.24 60% 200 docs 14.46 R−Precision 500 docs 8.28 50% 1000 docs 4.83 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 18.59 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.8538 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1656 Third Quartile 0.2760 Interquartile range 0.2760 Mean 0.1859 Standard Deviation 0.2082 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6264 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1580 Std With No Outliers 0.1582 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries GCesNtLg Topic 026 0.00 Topic 039 10.45 0.8 Topic 027 0.00 Topic 040 17.99 Topic 028 33.33 Topic 041 32.00 0.6 Topic 029 27.27 Topic 042 0.00 Topic 030 16.56 Topic 043 0.00 Topic 031 3.92 Topic 044 28.16 0.4 Topic 032 85.38 Topic 045 0.00 Topic 033 1.00 Topic 046 10.71 0.2 Topic 034 24.32 Topic 047 8.47 Difference Topic 035 26.32 Topic 048 62.64 0 Topic 036 20.91 Topic 049 27.65 Topic 037 27.59 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 278 sanmarcos SMGeoES4 GC-MONO-ES-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Spanish; Castilian Relevant 2,054 Topic Fields title, description Relevant retrieved 746 Pooled true Geometric Mean Average Precision 0.0270 Monolingual Spanish query expansion Binary Preference (BPREF) 0.1608 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision 100% 0 57.59 SMGeoES4 10 37.87 90% 20 26.49 80% 30 18.15 40 14.18 70% 50 9.77 Average Precision 60% 60 7.66 70 2.17 50% 80 0.00 40% 90 0.00 30% 100 0.00 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 13.78 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.5530 Minimum 0.0000 First Quartile 0.0126 Second Quartile 0.0544 Third Quartile 0.2166 Interquartile range 0.2039 Mean 0.1378 Standard Deviation 0.1646 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3556 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1026 Std With No Outliers 0.1159 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoES4 Topic 026 1.71 Topic 039 30.24 0.8 Topic 027 1.90 Topic 040 55.30 Topic 028 26.13 Topic 041 1.03 0.6 Topic 029 35.56 Topic 042 33.36 Topic 030 11.09 Topic 043 0.00 Topic 031 1.34 Topic 044 12.63 0.4 Topic 032 20.17 Topic 045 0.60 Topic 033 0.10 Topic 046 10.51 0.2 Topic 034 18.91 Topic 047 5.44 Difference Topic 035 12.17 Topic 048 5.40 0 Topic 036 4.34 Topic 049 53.01 Topic 037 0.03 Topic 050 3.42 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 279 sanmarcos SMGeoES4 GC-MONO-ES-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision 100% 5 docs 44.00 SMGeoES4 10 docs 37.20 90% 15 docs 35.20 80% 20 docs 32.40 30 docs 27.87 70% 100 docs 16.28 60% 200 docs 10.68 R−Precision 500 docs 5.46 50% 1000 docs 2.98 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 18.63 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.6267 Minimum 0.0000 First Quartile 0.0535 Second Quartile 0.1273 Third Quartile 0.3102 Interquartile range 0.2568 Mean 0.1863 Standard Deviation 0.1845 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6267 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1863 Std With No Outliers 0.1845 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 6 Number of Topics of the Experiment 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoES4 Topic 026 11.11 Topic 039 35.82 0.8 Topic 027 7.69 Topic 040 61.87 Topic 028 30.56 Topic 041 8.00 0.6 Topic 029 42.42 Topic 042 39.62 Topic 030 21.19 Topic 043 0.00 Topic 031 6.27 Topic 044 19.42 0.4 Topic 032 20.77 Topic 045 0.00 Topic 033 1.00 Topic 046 14.29 0.2 Topic 034 32.43 Topic 047 3.39 Difference Topic 035 21.05 Topic 048 7.55 0 Topic 036 12.73 Topic 049 62.67 Topic 037 0.00 Topic 050 6.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 280 sanmarcos SMGeoES5 GC-MONO-ES-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Spanish; Castilian Relevant 2,054 Topic Fields title, description, narrative Relevant retrieved 743 Pooled true Geometric Mean Average Precision 0.0350 Monolingual Spanish no query expansion Binary Preference (BPREF) 0.1781 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision 100% 0 58.66 SMGeoES5 10 42.54 90% 20 29.48 80% 30 18.96 40 15.11 70% 50 10.69 Average Precision 60% 60 7.32 70 2.79 50% 80 0.00 40% 90 0.00 30% 100 0.00 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 14.71 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.5745 Minimum 0.0000 First Quartile 0.0099 Second Quartile 0.1102 Third Quartile 0.2251 Interquartile range 0.2152 Mean 0.1471 Standard Deviation 0.1590 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5408 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1293 Std With No Outliers 0.1345 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoES5 Topic 026 2.36 Topic 039 22.11 0.8 Topic 027 0.99 Topic 040 54.08 Topic 028 23.70 Topic 041 0.84 0.6 Topic 029 27.59 Topic 042 18.35 Topic 030 13.52 Topic 043 0.00 Topic 031 0.99 Topic 044 29.15 0.4 Topic 032 19.84 Topic 045 0.96 Topic 033 0.03 Topic 046 11.07 0.2 Topic 034 31.74 Topic 047 10.44 Difference Topic 035 13.95 Topic 048 5.88 0 Topic 036 11.02 Topic 049 57.45 Topic 037 7.46 Topic 050 4.15 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 281 sanmarcos SMGeoES5 GC-MONO-ES-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision 100% 5 docs 46.40 SMGeoES5 10 docs 42.00 90% 15 docs 38.93 80% 20 docs 34.60 30 docs 30.67 70% 100 docs 18.16 60% 200 docs 11.82 R−Precision 500 docs 5.49 50% 1000 docs 2.97 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 20.44 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.6682 Minimum 0.0000 First Quartile 0.0696 Second Quartile 0.1724 Third Quartile 0.2791 Interquartile range 0.2095 Mean 0.2044 Standard Deviation 0.1839 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4595 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1668 Std With No Outliers 0.1355 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 5 Number of Topics of the Experiment 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoES5 Topic 026 11.11 Topic 039 22.39 0.8 Topic 027 0.00 Topic 040 60.43 Topic 028 27.78 Topic 041 6.67 0.6 Topic 029 42.42 Topic 042 28.30 Topic 030 20.53 Topic 043 0.00 Topic 031 7.06 Topic 044 37.86 0.4 Topic 032 20.77 Topic 045 0.00 Topic 033 1.00 Topic 046 14.29 0.2 Topic 034 45.95 Topic 047 16.95 Difference Topic 035 21.05 Topic 048 7.55 0 Topic 036 22.73 Topic 049 66.82 Topic 037 17.24 Topic 050 12.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 282 sanmarcos SMGeoES1 GC-MONO-ES-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Spanish; Castilian Relevant 2,054 Topic Fields title, description Relevant retrieved 743 Pooled true Geometric Mean Average Precision 0.0350 Monolingual Spanish title desc automatic Binary Preference (BPREF) 0.1781 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision 100% 0 58.66 SMGeoES1 10 42.54 90% 20 29.49 80% 30 18.95 40 15.09 70% 50 10.69 Average Precision 60% 60 7.32 70 2.80 50% 80 0.00 40% 90 0.00 30% 100 0.00 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 14.71 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.5745 Minimum 0.0000 First Quartile 0.0099 Second Quartile 0.1102 Third Quartile 0.2252 Interquartile range 0.2153 Mean 0.1471 Standard Deviation 0.1590 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5408 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1293 Std With No Outliers 0.1345 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoES1 Topic 026 2.36 Topic 039 22.13 0.8 Topic 027 0.99 Topic 040 54.08 Topic 028 23.70 Topic 041 0.84 0.6 Topic 029 27.59 Topic 042 18.35 Topic 030 13.52 Topic 043 0.00 Topic 031 0.99 Topic 044 29.15 0.4 Topic 032 19.84 Topic 045 0.96 Topic 033 0.03 Topic 046 11.07 0.2 Topic 034 31.74 Topic 047 10.44 Difference Topic 035 13.92 Topic 048 5.88 0 Topic 036 11.02 Topic 049 57.45 Topic 037 7.46 Topic 050 4.15 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 283 sanmarcos SMGeoES1 GC-MONO-ES-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision 100% 5 docs 46.40 SMGeoES1 10 docs 42.00 90% 15 docs 38.93 80% 20 docs 34.60 30 docs 30.67 70% 100 docs 18.16 60% 200 docs 11.82 R−Precision 500 docs 5.49 50% 1000 docs 2.97 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 20.44 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.6682 Minimum 0.0000 First Quartile 0.0696 Second Quartile 0.1724 Third Quartile 0.2791 Interquartile range 0.2095 Mean 0.2044 Standard Deviation 0.1839 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4595 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1668 Std With No Outliers 0.1355 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 5 Number of Topics of the Experiment 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoES1 Topic 026 11.11 Topic 039 22.39 0.8 Topic 027 0.00 Topic 040 60.43 Topic 028 27.78 Topic 041 6.67 0.6 Topic 029 42.42 Topic 042 28.30 Topic 030 20.53 Topic 043 0.00 Topic 031 7.06 Topic 044 37.86 0.4 Topic 032 20.77 Topic 045 0.00 Topic 033 1.00 Topic 046 14.29 0.2 Topic 034 45.95 Topic 047 16.95 Difference Topic 035 21.05 Topic 048 7.55 0 Topic 036 22.73 Topic 049 66.82 Topic 037 17.24 Topic 050 12.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 284 sanmarcos SMGeoES2 GC-MONO-ES-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Spanish; Castilian Relevant 2,054 Topic Fields title, description, narrative Relevant retrieved 745 Pooled true Geometric Mean Average Precision 0.0366 Monolingual Spanish title + desc + narr Binary Preference (BPREF) 0.1806 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision 100% 0 60.75 SMGeoES2 10 44.52 90% 20 29.27 80% 30 20.27 40 15.21 70% 50 11.16 Average Precision 60% 60 7.54 70 2.82 50% 80 0.00 40% 90 0.00 30% 100 0.00 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 15.33 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.5854 Minimum 0.0000 First Quartile 0.0145 Second Quartile 0.1114 Third Quartile 0.2309 Interquartile range 0.2164 Mean 0.1533 Standard Deviation 0.1637 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5494 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1353 Std With No Outliers 0.1397 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoES2 Topic 026 1.92 Topic 039 22.49 0.8 Topic 027 1.13 Topic 040 54.94 Topic 028 24.89 Topic 041 0.86 0.6 Topic 029 29.49 Topic 042 19.64 Topic 030 14.26 Topic 043 0.00 Topic 031 1.56 Topic 044 29.43 0.4 Topic 032 19.94 Topic 045 1.13 Topic 033 0.03 Topic 046 13.10 0.2 Topic 034 35.33 Topic 047 11.14 Difference Topic 035 15.45 Topic 048 5.92 0 Topic 036 10.93 Topic 049 58.54 Topic 037 7.52 Topic 050 3.61 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 285 sanmarcos SMGeoES2 GC-MONO-ES-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision 100% 5 docs 47.20 SMGeoES2 10 docs 42.40 90% 15 docs 39.73 80% 20 docs 36.80 30 docs 30.67 70% 100 docs 19.08 60% 200 docs 11.82 R−Precision 500 docs 5.48 50% 1000 docs 2.98 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 20.29 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.6682 Minimum 0.0000 First Quartile 0.0633 Second Quartile 0.1724 Third Quartile 0.2676 Interquartile range 0.2042 Mean 0.2029 Standard Deviation 0.1872 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5135 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1649 Std With No Outliers 0.1389 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 5 Number of Topics of the Experiment 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoES2 Topic 026 11.11 Topic 039 22.39 0.8 Topic 027 0.00 Topic 040 61.15 Topic 028 27.78 Topic 041 5.33 0.6 Topic 029 36.36 Topic 042 26.42 Topic 030 19.87 Topic 043 0.00 Topic 031 6.67 Topic 044 37.86 0.4 Topic 032 20.77 Topic 045 0.00 Topic 033 1.00 Topic 046 14.29 0.2 Topic 034 51.35 Topic 047 15.25 Difference Topic 035 26.32 Topic 048 7.55 0 Topic 036 23.64 Topic 049 66.82 Topic 037 17.24 Topic 050 8.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 286 sanmarcos SMGeoES3 GC-MONO-ES-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction MANUAL Retrieved 25,000 Source Language Spanish; Castilian Relevant 2,054 Topic Fields title, description Relevant retrieved 743 Pooled true Geometric Mean Average Precision 0.0350 Monolingual Spanish adding information from Binary Preference (BPREF) 0.1781 gazzeteers and other sources Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision 100% 0 58.66 SMGeoES3 10 42.54 90% 20 29.49 80% 30 18.95 40 15.09 70% 50 10.69 Average Precision 60% 60 7.32 70 2.80 50% 80 0.00 40% 90 0.00 30% 100 0.00 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 14.71 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.5745 Minimum 0.0000 First Quartile 0.0099 Second Quartile 0.1102 Third Quartile 0.2252 Interquartile range 0.2153 Mean 0.1471 Standard Deviation 0.1590 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5408 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1293 Std With No Outliers 0.1345 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoES3 Topic 026 2.36 Topic 039 22.13 0.8 Topic 027 0.99 Topic 040 54.08 Topic 028 23.70 Topic 041 0.84 0.6 Topic 029 27.59 Topic 042 18.35 Topic 030 13.52 Topic 043 0.00 Topic 031 0.99 Topic 044 29.15 0.4 Topic 032 19.84 Topic 045 0.96 Topic 033 0.03 Topic 046 11.07 0.2 Topic 034 31.74 Topic 047 10.44 Difference Topic 035 13.92 Topic 048 5.88 0 Topic 036 11.02 Topic 049 57.45 Topic 037 7.46 Topic 050 4.15 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 287 sanmarcos SMGeoES3 GC-MONO-ES-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision 100% 5 docs 46.40 SMGeoES3 10 docs 42.00 90% 15 docs 38.93 80% 20 docs 34.60 30 docs 30.67 70% 100 docs 18.16 60% 200 docs 11.82 R−Precision 500 docs 5.49 50% 1000 docs 2.97 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 20.44 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.6682 Minimum 0.0000 First Quartile 0.0696 Second Quartile 0.1724 Third Quartile 0.2791 Interquartile range 0.2095 Mean 0.2044 Standard Deviation 0.1839 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4595 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1668 Std With No Outliers 0.1355 GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment 5 Number of Topics of the Experiment 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoES3 Topic 026 11.11 Topic 039 22.39 0.8 Topic 027 0.00 Topic 040 60.43 Topic 028 27.78 Topic 041 6.67 0.6 Topic 029 42.42 Topic 042 28.30 Topic 030 20.53 Topic 043 0.00 Topic 031 7.06 Topic 044 37.86 0.4 Topic 032 20.77 Topic 045 0.00 Topic 033 1.00 Topic 046 14.29 0.2 Topic 034 45.95 Topic 047 16.95 Difference Topic 035 21.05 Topic 048 7.55 0 Topic 036 22.73 Topic 049 66.82 Topic 037 17.24 Topic 050 12.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 288 berkeley BKGeoP2 GC-MONO-PT-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Portuguese Relevant 1,060 Topic Fields title, description, narrative Relevant retrieved 604 Pooled true Geometric Mean Average Precision 0.0124 Baseline TDN run using standard logistic regression Binary Preference (BPREF) 0.1469 algorithms plus blind feedback Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision 100% 0 36.34 BKGeoP2 10 30.25 90% 20 26.18 80% 30 21.39 40 19.96 70% 50 18.01 Average Precision 60% 60 14.41 70 11.42 50% 80 7.62 40% 90 4.50 30% 100 0.72 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 16.31 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.8680 Minimum 0.0000 First Quartile 0.0006 Second Quartile 0.0450 Third Quartile 0.2431 Interquartile range 0.2424 Mean 0.1631 Standard Deviation 0.2318 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5945 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1337 Std With No Outliers 0.1832 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries BKGeoP2 Topic 026 0.06 Topic 039 4.50 0.8 Topic 027 0.04 Topic 040 0.03 Topic 028 11.86 Topic 041 0.00 0.6 Topic 029 32.24 Topic 042 43.10 Topic 030 45.55 Topic 043 0.11 Topic 031 16.45 Topic 044 0.37 0.4 Topic 032 45.34 Topic 045 18.73 Topic 033 0.19 Topic 046 59.45 0.2 Topic 034 0.03 Topic 047 0.84 Difference Topic 035 0.51 Topic 048 86.80 0 Topic 036 0.00 Topic 049 9.93 Topic 037 0.07 Topic 050 21.66 −0.2 Topic 038 9.78 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 289 berkeley BKGeoP2 GC-MONO-PT-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision 100% 5 docs 23.20 BKGeoP2 10 docs 24.40 90% 15 docs 22.13 80% 20 docs 20.00 30 docs 19.33 70% 100 docs 13.08 60% 200 docs 8.98 R−Precision 500 docs 4.46 50% 1000 docs 2.42 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 16.46 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.7832 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0098 Third Quartile 0.3007 Interquartile range 0.3007 Mean 0.1646 Standard Deviation 0.2348 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5909 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1388 Std With No Outliers 0.2005 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries BKGeoP2 Topic 026 0.00 Topic 039 4.35 0.8 Topic 027 0.98 Topic 040 0.00 Topic 028 15.62 Topic 041 0.00 0.6 Topic 029 38.46 Topic 042 48.57 Topic 030 50.00 Topic 043 0.00 Topic 031 14.75 Topic 044 0.00 0.4 Topic 032 49.06 Topic 045 19.51 Topic 033 0.00 Topic 046 59.09 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 78.32 0 Topic 036 0.00 Topic 049 5.56 Topic 037 0.00 Topic 050 27.27 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 290 berkeley BKGeoP1 GC-MONO-PT-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Portuguese Relevant 1,060 Topic Fields title, description Relevant retrieved 644 Pooled true Geometric Mean Average Precision 0.0183 Baseline TD run using standard logistic regression Binary Preference (BPREF) 0.1478 algorithms plus blind feedback Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision 100% 0 38.63 BKGeoP1 10 29.01 90% 20 25.25 80% 30 22.30 40 19.52 70% 50 18.16 Average Precision 60% 60 13.30 70 10.47 50% 80 7.54 40% 90 4.95 30% 100 2.28 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 16.22 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.9241 Minimum 0.0000 First Quartile 0.0015 Second Quartile 0.0426 Third Quartile 0.2736 Interquartile range 0.2721 Mean 0.1622 Standard Deviation 0.2339 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6566 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1305 Std With No Outliers 0.1755 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries BKGeoP1 Topic 026 0.47 Topic 039 4.26 0.8 Topic 027 0.12 Topic 040 0.08 Topic 028 14.82 Topic 041 0.01 0.6 Topic 029 27.73 Topic 042 10.65 Topic 030 65.66 Topic 043 0.02 Topic 031 27.24 Topic 044 0.65 0.4 Topic 032 46.22 Topic 045 35.38 Topic 033 0.13 Topic 046 28.66 0.2 Topic 034 0.16 Topic 047 0.30 Difference Topic 035 0.94 Topic 048 92.41 0 Topic 036 0.00 Topic 049 17.81 Topic 037 3.90 Topic 050 19.57 −0.2 Topic 038 8.33 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 291 berkeley BKGeoP1 GC-MONO-PT-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision 100% 5 docs 25.60 BKGeoP1 10 docs 24.00 90% 15 docs 22.67 80% 20 docs 21.00 30 docs 20.00 70% 100 docs 12.60 60% 200 docs 9.00 R−Precision 500 docs 4.62 50% 1000 docs 2.58 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 16.43 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.8112 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0098 Third Quartile 0.2708 Interquartile range 0.2708 Mean 0.1643 Standard Deviation 0.2249 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5714 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1373 Std With No Outliers 0.1839 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries BKGeoP1 Topic 026 0.00 Topic 039 8.70 0.8 Topic 027 0.98 Topic 040 0.00 Topic 028 18.75 Topic 041 0.00 0.6 Topic 029 33.33 Topic 042 20.00 Topic 030 57.14 Topic 043 0.00 Topic 031 21.31 Topic 044 0.00 0.4 Topic 032 56.60 Topic 045 37.80 Topic 033 0.00 Topic 046 33.33 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 81.12 0 Topic 036 0.00 Topic 049 16.67 Topic 037 0.00 Topic 050 25.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 292 berkeley BKGeoP4 GC-MONO-PT-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Portuguese Relevant 1,060 Topic Fields title, description, narrative Relevant retrieved 607 Pooled true Geometric Mean Average Precision 0.0126 Portuguese Monolingual using title, description and Binary Preference (BPREF) 0.1555 narrative (Corrected Queries) Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision 100% 0 36.34 BKGeoP4 10 30.19 90% 20 27.04 80% 30 22.45 40 21.01 70% 50 19.92 Average Precision 60% 60 16.13 70 12.72 50% 80 7.85 40% 90 4.59 30% 100 0.72 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 17.36 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.8680 Minimum 0.0000 First Quartile 0.0006 Second Quartile 0.0450 Third Quartile 0.2431 Interquartile range 0.2424 Mean 0.1736 Standard Deviation 0.2495 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5945 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1210 Std With No Outliers 0.1762 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries BKGeoP4 Topic 026 0.06 Topic 039 4.50 0.8 Topic 027 0.04 Topic 040 0.03 Topic 028 11.86 Topic 041 0.00 0.6 Topic 029 32.24 Topic 042 45.81 Topic 030 45.55 Topic 043 0.11 Topic 031 16.45 Topic 044 0.37 0.4 Topic 032 68.91 Topic 045 18.73 Topic 033 0.19 Topic 046 59.45 0.2 Topic 034 0.03 Topic 047 0.84 Difference Topic 035 0.51 Topic 048 86.80 0 Topic 036 0.00 Topic 049 9.93 Topic 037 0.07 Topic 050 21.66 −0.2 Topic 038 9.79 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 293 berkeley BKGeoP4 GC-MONO-PT-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision 100% 5 docs 24.00 BKGeoP4 10 docs 25.60 90% 15 docs 22.67 80% 20 docs 21.40 30 docs 20.27 70% 100 docs 13.44 60% 200 docs 9.04 R−Precision 500 docs 4.50 50% 1000 docs 2.43 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 17.22 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.7832 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0098 Third Quartile 0.3007 Interquartile range 0.3007 Mean 0.1722 Standard Deviation 0.2484 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6792 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1467 Std With No Outliers 0.2179 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries BKGeoP4 Topic 026 0.00 Topic 039 4.35 0.8 Topic 027 0.98 Topic 040 0.00 Topic 028 15.62 Topic 041 0.00 0.6 Topic 029 38.46 Topic 042 48.57 Topic 030 50.00 Topic 043 0.00 Topic 031 14.75 Topic 044 0.00 0.4 Topic 032 67.92 Topic 045 19.51 Topic 033 0.00 Topic 046 59.09 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 78.32 0 Topic 036 0.00 Topic 049 5.56 Topic 037 0.00 Topic 050 27.27 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 294 berkeley BKGeoP3 GC-MONO-PT-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Portuguese Relevant 1,060 Topic Fields title, description Relevant retrieved 644 Pooled true Geometric Mean Average Precision 0.0186 Portuguese Monolingual using title and desc Binary Preference (BPREF) 0.1514 (corrected queries) Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision 100% 0 39.15 BKGeoP3 10 29.53 90% 20 25.77 80% 30 22.82 40 20.17 70% 50 18.78 Average Precision 60% 60 13.98 70 12.00 50% 80 7.54 40% 90 4.95 30% 100 2.28 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 16.92 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.9241 Minimum 0.0000 First Quartile 0.0015 Second Quartile 0.0426 Third Quartile 0.2736 Interquartile range 0.2721 Mean 0.1692 Standard Deviation 0.2457 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6566 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1378 Std With No Outliers 0.1928 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries BKGeoP3 Topic 026 0.47 Topic 039 4.26 0.8 Topic 027 0.12 Topic 040 0.08 Topic 028 14.82 Topic 041 0.01 0.6 Topic 029 27.73 Topic 042 10.65 Topic 030 65.66 Topic 043 0.02 Topic 031 27.24 Topic 044 0.65 0.4 Topic 032 63.83 Topic 045 35.38 Topic 033 0.13 Topic 046 28.66 0.2 Topic 034 0.16 Topic 047 0.30 Difference Topic 035 0.94 Topic 048 92.41 0 Topic 036 0.00 Topic 049 17.81 Topic 037 3.90 Topic 050 19.57 −0.2 Topic 038 8.34 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 295 berkeley BKGeoP3 GC-MONO-PT-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision 100% 5 docs 27.20 BKGeoP3 10 docs 24.80 90% 15 docs 23.47 80% 20 docs 21.60 30 docs 20.27 70% 100 docs 12.84 60% 200 docs 9.12 R−Precision 500 docs 4.65 50% 1000 docs 2.58 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 16.51 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.8112 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0098 Third Quartile 0.2708 Interquartile range 0.2708 Mean 0.1651 Standard Deviation 0.2263 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5849 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1381 Std With No Outliers 0.1858 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries BKGeoP3 Topic 026 0.00 Topic 039 8.70 0.8 Topic 027 0.98 Topic 040 0.00 Topic 028 18.75 Topic 041 0.00 0.6 Topic 029 33.33 Topic 042 20.00 Topic 030 57.14 Topic 043 0.00 Topic 031 21.31 Topic 044 0.00 0.4 Topic 032 58.49 Topic 045 37.80 Topic 033 0.00 Topic 046 33.33 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 81.12 0 Topic 036 0.00 Topic 049 16.67 Topic 037 0.00 Topic 050 25.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 296 sanmarcos SMGeoPT4 GC-MONO-PT-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Portuguese Relevant 1,060 Topic Fields title, description Relevant retrieved 535 Pooled true Geometric Mean Average Precision 0.0153 Automatic Portuguese title+desc no query expansion Binary Preference (BPREF) 0.1052 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision 100% 0 39.55 SMGeoPT4 10 24.96 90% 20 18.86 80% 30 14.25 40 10.88 70% 50 8.91 Average Precision 60% 60 6.55 70 5.01 50% 80 3.77 40% 90 1.55 30% 100 0.63 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 10.63 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.4830 Minimum 0.0000 First Quartile 0.0044 Second Quartile 0.0317 Third Quartile 0.1791 Interquartile range 0.1748 Mean 0.1063 Standard Deviation 0.1438 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3088 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0746 Std With No Outliers 0.0972 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoPT4 Topic 026 0.21 Topic 039 15.43 0.8 Topic 027 0.06 Topic 040 0.57 Topic 028 3.06 Topic 041 0.51 0.6 Topic 029 17.87 Topic 042 30.88 Topic 030 1.12 Topic 043 0.10 Topic 031 18.06 Topic 044 3.17 0.4 Topic 032 48.30 Topic 045 15.17 Topic 033 0.00 Topic 046 22.49 0.2 Topic 034 4.38 Topic 047 0.00 Difference Topic 035 1.61 Topic 048 45.81 0 Topic 036 1.67 Topic 049 26.45 Topic 037 0.00 Topic 050 3.91 −0.2 Topic 038 4.90 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 297 sanmarcos SMGeoPT4 GC-MONO-PT-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision 100% 5 docs 20.80 SMGeoPT4 10 docs 20.00 90% 15 docs 19.47 80% 20 docs 18.40 30 docs 16.27 70% 100 docs 10.20 60% 200 docs 6.72 R−Precision 500 docs 3.67 50% 1000 docs 2.14 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 13.57 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.5283 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0833 Third Quartile 0.2006 Interquartile range 0.2006 Mean 0.1357 Standard Deviation 0.1636 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3714 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1020 Std With No Outliers 0.1200 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoPT4 Topic 026 0.00 Topic 039 17.39 0.8 Topic 027 0.98 Topic 040 8.33 Topic 028 6.25 Topic 041 1.92 0.6 Topic 029 15.38 Topic 042 37.14 Topic 030 7.14 Topic 043 0.00 Topic 031 14.75 Topic 044 10.53 0.4 Topic 032 52.83 Topic 045 28.05 Topic 033 0.00 Topic 046 31.82 0.2 Topic 034 12.50 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 51.75 0 Topic 036 0.00 Topic 049 33.33 Topic 037 0.00 Topic 050 9.09 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 298 sanmarcos SMGeoPT2 GC-MONO-PT-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Portuguese Relevant 1,060 Topic Fields title, description Relevant retrieved 620 Pooled true Geometric Mean Average Precision 0.0524 Automatic Portuguese title+desc Binary Preference (BPREF) 0.1211 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision 100% 0 47.69 SMGeoPT2 10 29.63 90% 20 23.19 80% 30 18.01 40 13.97 70% 50 12.26 Average Precision 60% 60 9.27 70 6.21 50% 80 4.01 40% 90 2.35 30% 100 0.40 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 13.44 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.5240 Minimum 0.0006 First Quartile 0.0210 Second Quartile 0.0638 Third Quartile 0.1717 Interquartile range 0.1506 Mean 0.1344 Standard Deviation 0.1651 Lower Outlier Threshold 0.0006 Upper Outlier Threshold 0.3276 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0835 Std With No Outliers 0.0918 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoPT2 Topic 026 1.04 Topic 039 32.76 0.8 Topic 027 6.09 Topic 040 1.52 Topic 028 14.88 Topic 041 1.14 0.6 Topic 029 6.38 Topic 042 52.40 Topic 030 47.73 Topic 043 0.06 Topic 031 14.69 Topic 044 7.68 0.4 Topic 032 52.29 Topic 045 3.34 Topic 033 2.14 Topic 046 23.48 0.2 Topic 034 2.39 Topic 047 9.14 Difference Topic 035 1.99 Topic 048 2.86 0 Topic 036 2.19 Topic 049 15.07 Topic 037 0.21 Topic 050 8.54 −0.2 Topic 038 26.01 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 299 sanmarcos SMGeoPT2 GC-MONO-PT-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision 100% 5 docs 26.40 SMGeoPT2 10 docs 21.60 90% 15 docs 19.20 80% 20 docs 17.20 30 docs 15.47 70% 100 docs 9.64 60% 200 docs 6.52 R−Precision 500 docs 3.94 50% 1000 docs 2.48 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 15.02 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.6038 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0979 Third Quartile 0.1875 Interquartile range 0.1875 Mean 0.1502 Standard Deviation 0.1726 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4286 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1134 Std With No Outliers 0.1213 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoPT2 Topic 026 0.00 Topic 039 34.78 0.8 Topic 027 10.78 Topic 040 4.17 Topic 028 15.62 Topic 041 1.92 0.6 Topic 029 15.38 Topic 042 54.29 Topic 030 42.86 Topic 043 0.00 Topic 031 14.75 Topic 044 7.89 0.4 Topic 032 60.38 Topic 045 8.54 Topic 033 0.00 Topic 046 30.30 0.2 Topic 034 0.00 Topic 047 8.82 Difference Topic 035 0.00 Topic 048 9.79 0 Topic 036 0.00 Topic 049 16.67 Topic 037 0.00 Topic 050 13.64 −0.2 Topic 038 25.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 300 sanmarcos SMGeoPT1 GC-MONO-PT-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Portuguese Relevant 1,060 Topic Fields title, description, narrative Relevant retrieved 537 Pooled true Geometric Mean Average Precision 0.0154 Automatic Portuguese title+desc+narr Binary Preference (BPREF) 0.1094 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision 100% 0 39.88 SMGeoPT1 10 25.54 90% 20 18.96 80% 30 14.50 40 11.25 70% 50 8.76 Average Precision 60% 60 7.06 70 5.36 50% 80 3.99 40% 90 1.66 30% 100 0.50 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 10.98 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.5345 Minimum 0.0000 First Quartile 0.0047 Second Quartile 0.0316 Third Quartile 0.1796 Interquartile range 0.1748 Mean 0.1098 Standard Deviation 0.1511 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.2824 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0749 Std With No Outliers 0.0949 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoPT1 Topic 026 0.22 Topic 039 14.60 0.8 Topic 027 0.05 Topic 040 0.73 Topic 028 2.70 Topic 041 0.56 0.6 Topic 029 17.80 Topic 042 28.24 Topic 030 1.61 Topic 043 0.09 Topic 031 18.42 Topic 044 3.16 0.4 Topic 032 53.45 Topic 045 16.61 Topic 033 0.00 Topic 046 22.19 0.2 Topic 034 4.57 Topic 047 0.00 Difference Topic 035 1.53 Topic 048 48.65 0 Topic 036 2.66 Topic 049 27.21 Topic 037 0.01 Topic 050 3.96 −0.2 Topic 038 5.44 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 301 sanmarcos SMGeoPT1 GC-MONO-PT-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision 100% 5 docs 20.80 SMGeoPT1 10 docs 20.80 90% 15 docs 20.00 80% 20 docs 18.80 30 docs 16.80 70% 100 docs 10.32 60% 200 docs 6.90 R−Precision 500 docs 3.75 50% 1000 docs 2.15 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 13.91 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.5849 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0909 Third Quartile 0.2047 Interquartile range 0.2047 Mean 0.1391 Standard Deviation 0.1716 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3889 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1033 Std With No Outliers 0.1235 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoPT1 Topic 026 0.00 Topic 039 13.04 0.8 Topic 027 0.98 Topic 040 8.33 Topic 028 9.38 Topic 041 0.96 0.6 Topic 029 17.95 Topic 042 34.29 Topic 030 7.14 Topic 043 0.00 Topic 031 13.11 Topic 044 10.53 0.4 Topic 032 58.49 Topic 045 28.05 Topic 033 0.00 Topic 046 33.33 0.2 Topic 034 12.50 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 51.75 0 Topic 036 0.00 Topic 049 38.89 Topic 037 0.00 Topic 050 9.09 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 302 sanmarcos SMGeoPT3 GC-MONO-PT-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Portuguese Relevant 1,060 Topic Fields title, description, narrative Relevant retrieved 537 Pooled true Geometric Mean Average Precision 0.0154 Automatic Portuguese title+desc+narr no query Binary Preference (BPREF) 0.1094 expansion Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision 100% 0 39.88 SMGeoPT3 10 25.54 90% 20 18.96 80% 30 14.50 40 11.25 70% 50 8.76 Average Precision 60% 60 7.06 70 5.36 50% 80 3.99 40% 90 1.66 30% 100 0.50 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 10.98 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.5345 Minimum 0.0000 First Quartile 0.0047 Second Quartile 0.0316 Third Quartile 0.1796 Interquartile range 0.1748 Mean 0.1098 Standard Deviation 0.1511 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.2824 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0749 Std With No Outliers 0.0949 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoPT3 Topic 026 0.22 Topic 039 14.60 0.8 Topic 027 0.05 Topic 040 0.73 Topic 028 2.70 Topic 041 0.56 0.6 Topic 029 17.80 Topic 042 28.24 Topic 030 1.61 Topic 043 0.09 Topic 031 18.42 Topic 044 3.16 0.4 Topic 032 53.45 Topic 045 16.61 Topic 033 0.00 Topic 046 22.19 0.2 Topic 034 4.57 Topic 047 0.00 Difference Topic 035 1.53 Topic 048 48.65 0 Topic 036 2.66 Topic 049 27.21 Topic 037 0.01 Topic 050 3.96 −0.2 Topic 038 5.44 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 303 sanmarcos SMGeoPT3 GC-MONO-PT-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision 100% 5 docs 20.80 SMGeoPT3 10 docs 20.80 90% 15 docs 20.00 80% 20 docs 18.80 30 docs 16.80 70% 100 docs 10.32 60% 200 docs 6.90 R−Precision 500 docs 3.75 50% 1000 docs 2.15 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 13.91 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.5849 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0909 Third Quartile 0.2047 Interquartile range 0.2047 Mean 0.1391 Standard Deviation 0.1716 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3889 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1033 Std With No Outliers 0.1235 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoPT3 Topic 026 0.00 Topic 039 13.04 0.8 Topic 027 0.98 Topic 040 8.33 Topic 028 9.38 Topic 041 0.96 0.6 Topic 029 17.95 Topic 042 34.29 Topic 030 7.14 Topic 043 0.00 Topic 031 13.11 Topic 044 10.53 0.4 Topic 032 58.49 Topic 045 28.05 Topic 033 0.00 Topic 046 33.33 0.2 Topic 034 12.50 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 51.75 0 Topic 036 0.00 Topic 049 38.89 Topic 037 0.00 Topic 050 9.09 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 304 xldb XLDBGeoPTAut02 GC-MONO-PT-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction MANUAL Retrieved 23,350 Source Language Portuguese Relevant 1,060 Topic Fields title, description Relevant retrieved 828 Pooled true Geometric Mean Average Precision 0.1096 Scope as topic term, no geoexpansion, QE 32 terms, Binary Preference (BPREF) 0.2540 20 top-kdocs, relaxed query construction Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision 100% 0 55.25 XLDBGeoPTAut02 10 48.65 90% 20 43.78 80% 30 35.68 40 29.17 70% 50 25.56 Average Precision 60% 60 21.93 70 15.92 50% 80 11.48 40% 90 6.94 30% 100 1.24 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 25.70 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.7629 Minimum 0.0020 First Quartile 0.0303 Second Quartile 0.2519 Third Quartile 0.4039 Interquartile range 0.3736 Mean 0.2570 Standard Deviation 0.2333 Lower Outlier Threshold 0.0020 Upper Outlier Threshold 0.7629 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2570 Std With No Outliers 0.2333 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries XLDBGeoPTAut02 Topic 026 3.24 Topic 039 12.29 0.8 Topic 027 0.20 Topic 040 27.33 Topic 028 32.96 Topic 041 69.30 0.6 Topic 029 41.38 Topic 042 40.06 Topic 030 46.28 Topic 043 0.25 Topic 031 24.84 Topic 044 31.02 0.4 Topic 032 70.73 Topic 045 28.41 Topic 033 0.54 Topic 046 48.56 0.2 Topic 034 25.19 Topic 047 6.10 Difference Topic 035 1.23 Topic 048 76.29 0 Topic 036 2.41 Topic 049 27.60 Topic 037 12.16 Topic 050 12.26 −0.2 Topic 038 1.74 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 305 xldb XLDBGeoPTAut02 GC-MONO-PT-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision 100% 5 docs 41.60 XLDBGeoPTAut02 10 docs 39.20 90% 15 docs 36.00 80% 20 docs 35.00 30 docs 32.40 70% 100 docs 19.32 60% 200 docs 13.00 R−Precision 500 docs 6.26 50% 1000 docs 3.31 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 28.09 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.6783 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.3279 Third Quartile 0.4351 Interquartile range 0.4351 Mean 0.2809 Standard Deviation 0.2282 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6783 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2809 Std With No Outliers 0.2282 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries XLDBGeoPTAut02 Topic 026 0.00 Topic 039 17.39 0.8 Topic 027 0.00 Topic 040 33.33 Topic 028 37.50 Topic 041 67.31 0.6 Topic 029 48.72 Topic 042 48.57 Topic 030 42.86 Topic 043 0.00 Topic 031 32.79 Topic 044 42.11 0.4 Topic 032 66.04 Topic 045 36.59 Topic 033 0.00 Topic 046 45.45 0.2 Topic 034 37.50 Topic 047 0.00 Difference Topic 035 11.11 Topic 048 67.83 0 Topic 036 0.00 Topic 049 27.78 Topic 037 16.67 Topic 050 22.73 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 306 xldb XLDBGeoPTAut05 GC-MONO-PT-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 10,483 Source Language Portuguese Relevant 1,060 Topic Fields title, description Relevant retrieved 624 Pooled true Geometric Mean Average Precision 0.1208 topic 16 QE terms expansion, top-20k docs, scope Binary Preference (BPREF) 0.3055 expansion to 10 scopes Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision 100% 0 71.58 XLDBGeoPTAut05 10 57.65 90% 20 49.88 80% 30 45.47 40 38.91 70% 50 30.51 Average Precision 60% 60 23.50 70 16.29 50% 80 10.06 40% 90 2.05 30% 100 0.28 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 29.32 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.8587 Minimum 0.0000 First Quartile 0.0771 Second Quartile 0.2888 Third Quartile 0.4313 Interquartile range 0.3542 Mean 0.2932 Standard Deviation 0.2315 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8587 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2932 Std With No Outliers 0.2315 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 4 Number of Topics of the Experiment 3.5 3 2.5 2 1.5 1 0.5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries XLDBGeoPTAut05 Topic 026 5.06 Topic 039 6.60 0.8 Topic 027 12.35 Topic 040 28.88 Topic 028 42.52 Topic 041 36.49 0.6 Topic 029 44.98 Topic 042 38.40 Topic 030 61.23 Topic 043 8.09 Topic 031 30.86 Topic 044 36.60 0.4 Topic 032 85.87 Topic 045 21.39 Topic 033 0.60 Topic 046 59.87 0.2 Topic 034 14.18 Topic 047 2.94 Difference Topic 035 0.38 Topic 048 59.24 0 Topic 036 0.00 Topic 049 22.67 Topic 037 33.21 Topic 050 23.60 −0.2 Topic 038 57.03 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 307 xldb XLDBGeoPTAut05 GC-MONO-PT-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision 100% 5 docs 53.60 XLDBGeoPTAut05 10 docs 48.00 90% 15 docs 44.00 80% 20 docs 42.40 30 docs 36.93 70% 100 docs 21.80 60% 200 docs 11.80 R−Precision 500 docs 4.96 50% 1000 docs 2.50 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 34.57 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.8302 Minimum 0.0000 First Quartile 0.1326 Second Quartile 0.3415 Third Quartile 0.4712 Interquartile range 0.3385 Mean 0.3457 Standard Deviation 0.2446 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8302 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.3457 Std With No Outliers 0.2446 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 5 Number of Topics of the Experiment 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries XLDBGeoPTAut05 Topic 026 13.33 Topic 039 13.04 0.8 Topic 027 15.69 Topic 040 33.33 Topic 028 50.00 Topic 041 41.35 0.6 Topic 029 46.15 Topic 042 42.86 Topic 030 64.29 Topic 043 0.00 Topic 031 40.98 Topic 044 46.05 0.4 Topic 032 83.02 Topic 045 34.15 Topic 033 0.00 Topic 046 65.15 0.2 Topic 034 25.00 Topic 047 2.94 Difference Topic 035 0.00 Topic 048 60.14 0 Topic 036 0.00 Topic 049 33.33 Topic 037 44.44 Topic 050 34.09 −0.2 Topic 038 75.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 308 xldb XLDBGeoManualPT GC-MONO-PT-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction MANUAL Retrieved 5,232 Source Language Portuguese Relevant 1,060 Topic Fields title, description Relevant retrieved 607 Pooled true Geometric Mean Average Precision 0.2034 Manual query Binary Preference (BPREF) 0.3208 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision 100% 0 70.76 XLDBGeoManualPT 10 60.08 90% 20 51.15 80% 30 43.68 40 39.03 70% 50 34.68 Average Precision 60% 60 26.45 70 14.46 50% 80 7.91 40% 90 1.99 30% 100 0.20 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 30.12 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.7500 Minimum 0.0020 First Quartile 0.1306 Second Quartile 0.3169 Third Quartile 0.4245 Interquartile range 0.2940 Mean 0.3012 Standard Deviation 0.2099 Lower Outlier Threshold 0.0020 Upper Outlier Threshold 0.7500 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.3012 Std With No Outliers 0.2099 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 5 Number of Topics of the Experiment 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries XLDBGeoManualPT Topic 026 19.59 Topic 039 31.69 0.8 Topic 027 64.35 Topic 040 50.25 Topic 028 41.69 Topic 041 14.16 0.6 Topic 029 7.31 Topic 042 31.75 Topic 030 44.74 Topic 043 0.20 Topic 031 33.34 Topic 044 9.73 0.4 Topic 032 67.32 Topic 045 35.38 Topic 033 8.33 Topic 046 39.94 0.2 Topic 034 6.44 Topic 047 19.16 Difference Topic 035 15.05 Topic 048 58.04 0 Topic 036 9.52 Topic 049 33.44 Topic 037 14.25 Topic 050 22.26 −0.2 Topic 038 75.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 309 xldb XLDBGeoManualPT GC-MONO-PT-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision 100% 5 docs 48.80 XLDBGeoManualPT 10 docs 49.60 90% 15 docs 47.20 80% 20 docs 44.20 30 docs 39.87 70% 100 docs 21.84 60% 200 docs 11.88 R−Precision 500 docs 4.84 50% 1000 docs 2.43 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 35.89 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.7500 Minimum 0.0000 First Quartile 0.2036 Second Quartile 0.3409 Third Quartile 0.5076 Interquartile range 0.3040 Mean 0.3589 Standard Deviation 0.2182 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7500 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.3589 Std With No Outliers 0.2182 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 3 Number of Topics of the Experiment 2.5 2 1.5 1 0.5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries XLDBGeoManualPT Topic 026 26.67 Topic 039 34.78 0.8 Topic 027 73.53 Topic 040 50.00 Topic 028 50.00 Topic 041 18.27 0.6 Topic 029 17.95 Topic 042 45.71 Topic 030 57.14 Topic 043 4.17 Topic 031 40.98 Topic 044 21.05 0.4 Topic 032 67.92 Topic 045 47.56 Topic 033 25.00 Topic 046 53.03 0.2 Topic 034 12.50 Topic 047 23.53 Difference Topic 035 0.00 Topic 048 60.14 0 Topic 036 0.00 Topic 049 33.33 Topic 037 25.00 Topic 050 34.09 −0.2 Topic 038 75.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 310 xldb XLDBGeoPTAut03 GC-MONO-PT-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 22,617 Source Language Portuguese Relevant 1,060 Topic Fields title, description Relevant retrieved 519 Pooled true Geometric Mean Average Precision 0.0738 With geosim, final correction. Binary Preference (BPREF) 0.2081 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision 100% 0 71.53 XLDBGeoPTAut03 10 48.45 90% 20 36.48 80% 30 28.80 40 19.87 70% 50 16.26 Average Precision 60% 60 9.49 70 5.94 50% 80 3.42 40% 90 0.40 30% 100 0.00 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 19.29 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.7930 Minimum 0.0000 First Quartile 0.0645 Second Quartile 0.1309 Third Quartile 0.2902 Interquartile range 0.2257 Mean 0.1929 Standard Deviation 0.1888 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4798 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1679 Std With No Outliers 0.1446 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 5 Number of Topics of the Experiment 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries XLDBGeoPTAut03 Topic 026 0.40 Topic 039 8.93 0.8 Topic 027 2.11 Topic 040 23.16 Topic 028 13.09 Topic 041 42.24 0.6 Topic 029 14.92 Topic 042 18.83 Topic 030 44.12 Topic 043 6.74 Topic 031 6.99 Topic 044 5.57 0.4 Topic 032 29.95 Topic 045 24.92 Topic 033 8.05 Topic 046 47.98 0.2 Topic 034 22.02 Topic 047 12.86 Difference Topic 035 0.00 Topic 048 79.30 0 Topic 036 0.97 Topic 049 29.42 Topic 037 28.89 Topic 050 0.29 −0.2 Topic 038 10.42 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 311 xldb XLDBGeoPTAut03 GC-MONO-PT-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision 100% 5 docs 43.20 XLDBGeoPTAut03 10 docs 37.20 90% 15 docs 34.13 80% 20 docs 31.80 30 docs 28.67 70% 100 docs 16.16 60% 200 docs 9.02 R−Precision 500 docs 3.92 50% 1000 docs 2.08 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 23.91 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.7622 Minimum 0.0000 First Quartile 0.0909 Second Quartile 0.1875 Third Quartile 0.3802 Interquartile range 0.2893 Mean 0.2391 Standard Deviation 0.1981 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7622 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2391 Std With No Outliers 0.1981 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 5 Number of Topics of the Experiment 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries XLDBGeoPTAut03 Topic 026 0.00 Topic 039 13.04 0.8 Topic 027 6.86 Topic 040 29.17 Topic 028 18.75 Topic 041 54.81 0.6 Topic 029 28.21 Topic 042 25.71 Topic 030 42.86 Topic 043 16.67 Topic 031 9.84 Topic 044 17.11 0.4 Topic 032 37.74 Topic 045 35.37 Topic 033 0.00 Topic 046 53.03 0.2 Topic 034 12.50 Topic 047 14.71 Difference Topic 035 0.00 Topic 048 76.22 0 Topic 036 0.00 Topic 049 38.89 Topic 037 38.89 Topic 050 2.27 −0.2 Topic 038 25.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 312 xldb XLDBGeoPTAut03_2 GC-MONO-PT-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 15,622 Source Language Portuguese Relevant 647 Topic Fields title, description Relevant retrieved 326 Pooled true Geometric Mean Average Precision 0.0044 XLDBGeoPTAut03 run, improved Binary Preference (BPREF) 0.1533 Interploated Recall (%) Precision Averages (%) GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision 100% 0 45.44 XLDBGeoPTAut03_2 10 36.56 90% 20 29.06 80% 30 22.89 40 16.29 70% 50 13.01 Average Precision 60% 60 6.66 70 5.45 50% 80 5.03 40% 90 1.17 30% 100 0.65 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 15.13 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.7843 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0673 Third Quartile 0.2356 Interquartile range 0.2356 Mean 0.1513 Standard Deviation 0.2051 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5215 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1249 Std With No Outliers 0.1605 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries XLDBGeoPTAut03_2 Topic 026 0.00 Topic 039 8.96 0.8 Topic 027 0.64 Topic 040 23.16 Topic 028 0.00 Topic 041 42.23 0.6 Topic 029 52.15 Topic 042 18.81 Topic 030 44.58 Topic 043 6.73 Topic 031 0.00 Topic 044 5.81 0.4 Topic 032 78.43 Topic 045 24.76 Topic 033 9.88 Topic 046 0.00 0.2 Topic 034 21.76 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 0.00 0 Topic 036 0.97 Topic 049 0.00 Topic 037 28.92 Topic 050 0.00 −0.2 Topic 038 10.42 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 313 xldb XLDBGeoPTAut03_2 GC-MONO-PT-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision 100% 5 docs 30.40 XLDBGeoPTAut03_2 10 docs 28.40 90% 15 docs 24.27 80% 20 docs 21.60 30 docs 20.00 70% 100 docs 10.36 60% 200 docs 5.58 R−Precision 500 docs 2.50 50% 1000 docs 1.30 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 17.29 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.7547 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1250 Third Quartile 0.3041 Interquartile range 0.3041 Mean 0.1729 Standard Deviation 0.2122 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7547 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1729 Std With No Outliers 0.2122 GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries XLDBGeoPTAut03_2 Topic 026 0.00 Topic 039 13.04 0.8 Topic 027 4.90 Topic 040 29.17 Topic 028 0.00 Topic 041 54.81 0.6 Topic 029 46.15 Topic 042 25.71 Topic 030 42.86 Topic 043 12.50 Topic 031 0.00 Topic 044 17.11 0.4 Topic 032 75.47 Topic 045 34.15 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 12.50 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 0.00 0 Topic 036 0.00 Topic 049 0.00 Topic 037 38.89 Topic 050 0.00 −0.2 Topic 038 25.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 314 berkeley BKGeoED2 GC-BILI-X2DE-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction MANUAL Retrieved 25,000 Source Language English Relevant 602 Topic Fields title, description, narrative Relevant retrieved 450 Pooled true Geometric Mean Average Precision 0.0115 English-German TDN from expanded narrative, Binary Preference (BPREF) 0.1649 translation by L&H Power Translator, blind feedback Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision 100% 0 29.75 BKGeoED2 10 28.19 90% 20 26.59 80% 30 24.48 40 20.23 70% 50 17.35 Average Precision 60% 60 13.28 70 12.31 50% 80 10.86 40% 90 9.20 30% 100 2.07 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 16.82 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.9178 Minimum 0.0000 First Quartile 0.0038 Second Quartile 0.0235 Third Quartile 0.2800 Interquartile range 0.2762 Mean 0.1682 Standard Deviation 0.2563 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5092 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1082 Std With No Outliers 0.1557 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries BKGeoED2 Topic 026 0.00 Topic 039 17.14 0.8 Topic 027 0.08 Topic 040 36.77 Topic 028 34.93 Topic 041 2.35 0.6 Topic 029 35.10 Topic 042 24.04 Topic 030 7.28 Topic 043 25.69 Topic 031 4.22 Topic 044 0.48 0.4 Topic 032 79.95 Topic 045 1.60 Topic 033 0.01 Topic 046 1.36 0.2 Topic 034 50.92 Topic 047 0.00 Difference Topic 035 0.99 Topic 048 91.78 0 Topic 036 0.00 Topic 049 2.86 Topic 037 0.00 Topic 050 2.08 −0.2 Topic 038 0.96 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 315 berkeley BKGeoED2 GC-BILI-X2DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual German track − Retrieved documents vs Precision 100% 5 docs 17.60 BKGeoED2 10 docs 18.00 90% 15 docs 18.93 80% 20 docs 18.20 30 docs 17.07 70% 100 docs 10.64 60% 200 docs 6.44 R−Precision 500 docs 3.05 50% 1000 docs 1.80 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 18.56 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.8841 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.3807 Interquartile range 0.3807 Mean 0.1856 Standard Deviation 0.2781 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8841 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1856 Std With No Outliers 0.2781 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries BKGeoED2 Topic 026 0.00 Topic 039 22.22 0.8 Topic 027 1.54 Topic 040 43.90 Topic 028 43.75 Topic 041 0.00 0.6 Topic 029 33.33 Topic 042 36.17 Topic 030 6.67 Topic 043 50.00 Topic 031 0.00 Topic 044 0.00 0.4 Topic 032 85.19 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 52.94 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 88.41 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 316 berkeley BKGeoED1 GC-BILI-X2DE-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 602 Topic Fields title, description Relevant retrieved 391 Pooled true Geometric Mean Average Precision 0.0107 English-German TD run translation by L&H Binary Preference (BPREF) 0.1397 translator, blind feedback Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision 100% 0 28.82 BKGeoED1 10 22.23 90% 20 20.79 80% 30 20.15 40 18.47 70% 50 17.53 Average Precision 60% 60 15.67 70 13.45 50% 80 11.19 40% 90 8.68 30% 100 2.11 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 15.61 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.8867 Minimum 0.0000 First Quartile 0.0030 Second Quartile 0.0222 Third Quartile 0.1511 Interquartile range 0.1482 Mean 0.1561 Standard Deviation 0.2704 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1984 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0334 Std With No Outliers 0.0503 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries BKGeoED1 Topic 026 0.00 Topic 039 0.04 0.8 Topic 027 2.10 Topic 040 38.45 Topic 028 44.85 Topic 041 3.77 0.6 Topic 029 2.22 Topic 042 0.04 Topic 030 19.84 Topic 043 1.05 Topic 031 2.13 Topic 044 1.34 0.4 Topic 032 83.26 Topic 045 4.17 Topic 033 0.00 Topic 046 6.34 0.2 Topic 034 68.19 Topic 047 0.00 Difference Topic 035 2.05 Topic 048 88.67 0 Topic 036 0.00 Topic 049 5.21 Topic 037 0.38 Topic 050 2.66 −0.2 Topic 038 13.54 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 317 berkeley BKGeoED1 GC-BILI-X2DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual German track − Retrieved documents vs Precision 100% 5 docs 19.20 BKGeoED1 10 docs 17.20 90% 15 docs 17.33 80% 20 docs 17.80 30 docs 16.67 70% 100 docs 10.00 60% 200 docs 5.74 R−Precision 500 docs 2.66 50% 1000 docs 1.56 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 15.44 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.8696 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.2542 Interquartile range 0.2542 Mean 0.1544 Standard Deviation 0.2700 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4390 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0698 Std With No Outliers 0.1414 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries BKGeoED1 Topic 026 0.00 Topic 039 0.00 0.8 Topic 027 4.62 Topic 040 43.90 Topic 028 43.75 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 0.00 Topic 030 26.67 Topic 043 0.00 Topic 031 1.32 Topic 044 0.00 0.4 Topic 032 77.78 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 67.65 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 86.96 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 8.33 −0.2 Topic 038 25.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 318 hagen FUHedGNNNTDN GC-BILI-X2DE-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 602 Topic Fields title, description, narrative Relevant retrieved 333 Pooled true Geometric Mean Average Precision 0.0127 third run Binary Preference (BPREF) 0.0548 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision 100% 0 18.90 FUHedGNNNTDN 10 14.86 90% 20 10.13 80% 30 7.23 40 5.54 70% 50 4.30 Average Precision 60% 60 2.86 70 2.44 50% 80 1.29 40% 90 0.64 30% 100 0.06 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 5.48 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.5405 Minimum 0.0003 First Quartile 0.0052 Second Quartile 0.0096 Third Quartile 0.0412 Interquartile range 0.0359 Mean 0.0548 Standard Deviation 0.1217 Lower Outlier Threshold 0.0003 Upper Outlier Threshold 0.0886 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0213 Std With No Outliers 0.0238 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries FUHedGNNNTDN Topic 026 0.54 Topic 039 3.57 0.8 Topic 027 2.71 Topic 040 0.72 Topic 028 0.30 Topic 041 2.46 0.6 Topic 029 0.61 Topic 042 0.96 Topic 030 4.05 Topic 043 0.47 Topic 031 5.66 Topic 044 1.46 0.4 Topic 032 54.05 Topic 045 0.59 Topic 033 0.07 Topic 046 5.99 0.2 Topic 034 8.86 Topic 047 0.06 Difference Topic 035 4.31 Topic 048 33.99 0 Topic 036 3.84 Topic 049 0.03 Topic 037 0.61 Topic 050 0.73 −0.2 Topic 038 0.27 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 319 hagen FUHedGNNNTDN GC-BILI-X2DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual German track − Retrieved documents vs Precision 100% 5 docs 11.20 FUHedGNNNTDN 10 docs 11.60 90% 15 docs 10.93 80% 20 docs 10.20 30 docs 8.13 70% 100 docs 4.80 60% 200 docs 3.22 R−Precision 500 docs 2.01 50% 1000 docs 1.33 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 6.24 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.5370 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.1013 Interquartile range 0.1013 Mean 0.0624 Standard Deviation 0.1277 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1231 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0287 Std With No Outliers 0.0478 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries FUHedGNNNTDN Topic 026 0.00 Topic 039 11.11 0.8 Topic 027 12.31 Topic 040 2.44 Topic 028 0.00 Topic 041 10.53 0.6 Topic 029 0.00 Topic 042 0.00 Topic 030 10.00 Topic 043 0.00 Topic 031 7.89 Topic 044 0.00 0.4 Topic 032 53.70 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 11.76 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 36.23 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 320 hagen FUHedGYYYTDN GC-BILI-X2DE-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 602 Topic Fields title, description, narrative Relevant retrieved 375 Pooled true Geometric Mean Average Precision 0.0106 second run Binary Preference (BPREF) 0.1175 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision 100% 0 29.26 FUHedGYYYTDN 10 19.62 90% 20 17.13 80% 30 14.74 40 13.81 70% 50 12.22 Average Precision 60% 60 11.45 70 10.15 50% 80 8.03 40% 90 5.97 30% 100 0.24 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 12.34 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.8704 Minimum 0.0000 First Quartile 0.0023 Second Quartile 0.0123 Third Quartile 0.0651 Interquartile range 0.0628 Mean 0.1234 Standard Deviation 0.2421 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.0725 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0152 Std With No Outliers 0.0200 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries FUHedGYYYTDN Topic 026 0.05 Topic 039 26.52 0.8 Topic 027 1.10 Topic 040 39.12 Topic 028 1.90 Topic 041 6.26 0.6 Topic 029 1.23 Topic 042 0.25 Topic 030 1.83 Topic 043 0.16 Topic 031 68.71 Topic 044 0.98 0.4 Topic 032 56.78 Topic 045 0.83 Topic 033 2.72 Topic 046 0.13 0.2 Topic 034 7.25 Topic 047 0.00 Difference Topic 035 2.46 Topic 048 87.04 0 Topic 036 2.24 Topic 049 0.00 Topic 037 0.34 Topic 050 0.11 −0.2 Topic 038 0.54 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 321 hagen FUHedGYYYTDN GC-BILI-X2DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual German track − Retrieved documents vs Precision 100% 5 docs 19.20 FUHedGYYYTDN 10 docs 16.00 90% 15 docs 16.27 80% 20 docs 16.00 30 docs 14.40 70% 100 docs 10.08 60% 200 docs 5.86 R−Precision 500 docs 2.63 50% 1000 docs 1.50 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 12.45 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.7826 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.0794 Interquartile range 0.0794 Mean 0.1245 Standard Deviation 0.2320 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1176 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0196 Std With No Outliers 0.0340 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries FUHedGYYYTDN Topic 026 0.00 Topic 039 25.93 0.8 Topic 027 3.08 Topic 040 43.90 Topic 028 6.25 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 0.00 Topic 030 6.67 Topic 043 0.00 Topic 031 68.42 Topic 044 0.00 0.4 Topic 032 55.56 Topic 045 0.00 Topic 033 11.76 Topic 046 0.00 0.2 Topic 034 5.88 Topic 047 0.00 Difference Topic 035 5.56 Topic 048 78.26 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 322 hagen FUHedGNNNTD GC-BILI-X2DE-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 24,318 Source Language English Relevant 602 Topic Fields title, description Relevant retrieved 397 Pooled true Geometric Mean Average Precision 0.0231 fourth run Binary Preference (BPREF) 0.1219 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision 100% 0 35.08 FUHedGNNNTD 10 25.77 90% 20 22.00 80% 30 17.93 40 12.62 70% 50 11.01 Average Precision 60% 60 7.42 70 5.44 50% 80 4.33 40% 90 2.40 30% 100 0.05 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 12.11 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.7259 Minimum 0.0000 First Quartile 0.0053 Second Quartile 0.0645 Third Quartile 0.1710 Interquartile range 0.1657 Mean 0.1211 Standard Deviation 0.1773 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.2854 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0755 Std With No Outliers 0.0809 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries FUHedGNNNTD Topic 026 10.00 Topic 039 7.23 0.8 Topic 027 0.55 Topic 040 28.54 Topic 028 20.94 Topic 041 1.10 0.6 Topic 029 6.45 Topic 042 8.57 Topic 030 5.94 Topic 043 0.36 Topic 031 18.86 Topic 044 11.11 0.4 Topic 032 56.52 Topic 045 1.26 Topic 033 0.23 Topic 046 16.80 0.2 Topic 034 17.98 Topic 047 0.00 Difference Topic 035 5.75 Topic 048 72.59 0 Topic 036 0.47 Topic 049 0.00 Topic 037 7.95 Topic 050 3.55 −0.2 Topic 038 0.07 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 323 hagen FUHedGNNNTD GC-BILI-X2DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual German track − Retrieved documents vs Precision 100% 5 docs 20.80 FUHedGNNNTD 10 docs 21.20 90% 15 docs 18.67 80% 20 docs 17.20 30 docs 15.20 70% 100 docs 8.52 60% 200 docs 5.58 R−Precision 500 docs 2.90 50% 1000 docs 1.59 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 15.34 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.6087 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1111 Third Quartile 0.2537 Interquartile range 0.2537 Mean 0.1534 Standard Deviation 0.1738 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6087 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1534 Std With No Outliers 0.1738 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries FUHedGNNNTD Topic 026 0.00 Topic 039 11.11 0.8 Topic 027 6.15 Topic 040 36.59 Topic 028 21.88 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 23.40 Topic 030 13.33 Topic 043 0.00 Topic 031 30.26 Topic 044 33.33 0.4 Topic 032 51.85 Topic 045 0.00 Topic 033 0.00 Topic 046 25.00 0.2 Topic 034 26.47 Topic 047 0.00 Difference Topic 035 16.67 Topic 048 60.87 0 Topic 036 0.00 Topic 049 0.00 Topic 037 18.18 Topic 050 8.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 324 hagen FUHedGYYYTD GC-BILI-X2DE-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 602 Topic Fields title, description Relevant retrieved 383 Pooled true Geometric Mean Average Precision 0.0140 first run Binary Preference (BPREF) 0.1171 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision 100% 0 31.47 FUHedGYYYTD 10 21.14 90% 20 18.08 80% 30 15.41 40 14.08 70% 50 11.79 Average Precision 60% 60 10.77 70 9.75 50% 80 8.20 40% 90 6.80 30% 100 0.49 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 12.80 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.8412 Minimum 0.0000 First Quartile 0.0033 Second Quartile 0.0223 Third Quartile 0.0676 Interquartile range 0.0643 Mean 0.1280 Standard Deviation 0.2460 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1560 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0288 Std With No Outliers 0.0381 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries FUHedGYYYTD Topic 026 0.06 Topic 039 6.22 0.8 Topic 027 1.10 Topic 040 39.12 Topic 028 1.10 Topic 041 6.26 0.6 Topic 029 1.17 Topic 042 0.32 Topic 030 15.60 Topic 043 0.16 Topic 031 79.84 Topic 044 2.55 0.4 Topic 032 56.28 Topic 045 2.82 Topic 033 2.23 Topic 046 0.13 0.2 Topic 034 7.25 Topic 047 0.00 Difference Topic 035 1.35 Topic 048 84.12 0 Topic 036 6.60 Topic 049 0.00 Topic 037 0.34 Topic 050 4.70 −0.2 Topic 038 0.54 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 325 hagen FUHedGYYYTD GC-BILI-X2DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual German track − Retrieved documents vs Precision 100% 5 docs 20.00 FUHedGYYYTD 10 docs 17.60 90% 15 docs 16.53 80% 20 docs 16.60 30 docs 15.33 70% 100 docs 9.96 60% 200 docs 5.96 R−Precision 500 docs 2.75 50% 1000 docs 1.53 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 11.94 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.7681 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.0764 Interquartile range 0.0764 Mean 0.1194 Standard Deviation 0.2339 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.0833 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0150 Std With No Outliers 0.0269 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries FUHedGYYYTD Topic 026 0.00 Topic 039 7.41 0.8 Topic 027 3.08 Topic 040 43.90 Topic 028 3.12 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 2.13 Topic 030 23.33 Topic 043 0.00 Topic 031 76.32 Topic 044 0.00 0.4 Topic 032 48.15 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 5.88 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 76.81 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 8.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 326 hagen FUHedGYYYMTDN GC-BILI-X2DE-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 602 Topic Fields title, description, narrative Relevant retrieved 375 Pooled true Geometric Mean Average Precision 0.0118 fifth run Binary Preference (BPREF) 0.1104 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision 100% 0 28.51 FUHedGYYYMTDN 10 18.25 90% 20 16.52 80% 30 15.31 40 13.96 70% 50 11.77 Average Precision 60% 60 10.08 70 8.55 50% 80 6.70 40% 90 4.59 30% 100 0.11 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 11.48 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.8178 Minimum 0.0000 First Quartile 0.0040 Second Quartile 0.0155 Third Quartile 0.0438 Interquartile range 0.0398 Mean 0.1148 Standard Deviation 0.2280 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.0683 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0145 Std With No Outliers 0.0162 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries FUHedGYYYMTDN Topic 026 0.08 Topic 039 18.33 0.8 Topic 027 1.55 Topic 040 37.97 Topic 028 2.25 Topic 041 3.56 0.6 Topic 029 1.22 Topic 042 0.47 Topic 030 2.16 Topic 043 0.22 Topic 031 62.69 Topic 044 1.29 0.4 Topic 032 57.32 Topic 045 0.48 Topic 033 1.51 Topic 046 0.21 0.2 Topic 034 6.83 Topic 047 0.00 Difference Topic 035 1.93 Topic 048 81.78 0 Topic 036 2.73 Topic 049 0.00 Topic 037 1.77 Topic 050 0.16 −0.2 Topic 038 0.54 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 327 hagen FUHedGYYYMTDN GC-BILI-X2DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual German track − Retrieved documents vs Precision 100% 5 docs 15.20 FUHedGYYYMTDN 10 docs 16.00 90% 15 docs 15.20 80% 20 docs 15.20 30 docs 14.53 70% 100 docs 9.44 60% 200 docs 5.92 R−Precision 500 docs 2.70 50% 1000 docs 1.50 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 11.57 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.7681 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.0661 Interquartile range 0.0661 Mean 0.1157 Standard Deviation 0.2245 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.0769 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0154 Std With No Outliers 0.0263 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries FUHedGYYYMTDN Topic 026 0.00 Topic 039 18.52 0.8 Topic 027 7.69 Topic 040 46.34 Topic 028 6.25 Topic 041 0.00 0.6 Topic 029 0.00 Topic 042 2.13 Topic 030 3.33 Topic 043 0.00 Topic 031 63.16 Topic 044 0.00 0.4 Topic 032 53.70 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 5.88 Topic 047 0.00 Difference Topic 035 5.56 Topic 048 76.81 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 328 hildesheim HIGeoenderun21 GC-BILI-X2DE-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 602 Topic Fields title, description Relevant retrieved 349 Pooled true Geometric Mean Average Precision 0.0095 Experiment with BRF(5docs,25terms) base run Binary Preference (BPREF) 0.1218 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision 100% 0 30.08 HIGeoenderun21 10 24.56 90% 20 17.92 80% 30 16.49 40 15.59 70% 50 14.10 Average Precision 60% 60 9.86 70 6.35 50% 80 4.04 40% 90 2.67 30% 100 0.06 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 11.86 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.5317 Minimum 0.0000 First Quartile 0.0020 Second Quartile 0.0210 Third Quartile 0.2117 Interquartile range 0.2097 Mean 0.1186 Standard Deviation 0.1784 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5196 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1014 Std With No Outliers 0.1596 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries HIGeoenderun21 Topic 026 0.22 Topic 039 1.13 0.8 Topic 027 1.58 Topic 040 29.01 Topic 028 29.57 Topic 041 5.26 0.6 Topic 029 28.27 Topic 042 0.14 Topic 030 9.84 Topic 043 2.10 Topic 031 5.31 Topic 044 0.43 0.4 Topic 032 51.96 Topic 045 0.00 Topic 033 0.00 Topic 046 0.51 0.2 Topic 034 50.65 Topic 047 0.00 Difference Topic 035 4.13 Topic 048 53.17 0 Topic 036 0.08 Topic 049 18.80 Topic 037 0.59 Topic 050 3.83 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 329 hildesheim HIGeoenderun21 GC-BILI-X2DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual German track − Retrieved documents vs Precision 100% 5 docs 17.60 HIGeoenderun21 10 docs 14.40 90% 15 docs 14.13 80% 20 docs 14.40 30 docs 14.00 70% 100 docs 8.92 60% 200 docs 5.38 R−Precision 500 docs 2.56 50% 1000 docs 1.40 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 15.18 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.6087 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0714 Third Quartile 0.2083 Interquartile range 0.2083 Mean 0.1518 Standard Deviation 0.2043 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4375 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0942 Std With No Outliers 0.1364 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries HIGeoenderun21 Topic 026 0.00 Topic 039 3.70 0.8 Topic 027 12.31 Topic 040 41.46 Topic 028 43.75 Topic 041 10.53 0.6 Topic 029 33.33 Topic 042 0.00 Topic 030 16.67 Topic 043 7.14 Topic 031 10.53 Topic 044 0.00 0.4 Topic 032 55.56 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 55.88 Topic 047 0.00 Difference Topic 035 11.11 Topic 048 60.87 0 Topic 036 0.00 Topic 049 16.67 Topic 037 0.00 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 330 hildesheim HIGeoenderun22 GC-BILI-X2DE-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 23,198 Source Language English Relevant 568 Topic Fields title, description Relevant retrieved 307 Pooled true Geometric Mean Average Precision 0.0052 Experiment with BRF(5docs,25terms) with NE- Binary Preference (BPREF) 0.1043 recognition and weighting, also within the BRF Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision 100% 0 25.62 HIGeoenderun22 10 21.29 90% 20 17.48 80% 30 16.18 40 11.53 70% 50 9.37 Average Precision 60% 60 6.84 70 3.44 50% 80 2.35 40% 90 1.44 30% 100 0.05 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 9.69 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.5888 Minimum 0.0000 First Quartile 0.0006 Second Quartile 0.0114 Third Quartile 0.1055 Interquartile range 0.1049 Mean 0.0969 Standard Deviation 0.1651 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.2025 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0326 Std With No Outliers 0.0526 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries HIGeoenderun22 Topic 026 0.06 Topic 039 2.96 0.8 Topic 027 1.14 Topic 040 20.25 Topic 028 28.00 Topic 041 11.02 0.6 Topic 029 50.00 Topic 042 2.69 Topic 030 10.39 Topic 043 3.97 Topic 031 0.57 Topic 044 0.49 0.4 Topic 032 36.85 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 0.00 Topic 047 0.59 Difference Topic 035 0.16 Topic 048 58.88 0 Topic 036 0.06 Topic 049 8.50 Topic 037 0.08 Topic 050 5.48 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 331 hildesheim HIGeoenderun22 GC-BILI-X2DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual German track − Retrieved documents vs Precision 100% 5 docs 15.20 HIGeoenderun22 10 docs 14.80 90% 15 docs 13.60 80% 20 docs 12.00 30 docs 10.80 70% 100 docs 6.92 60% 200 docs 4.14 R−Precision 500 docs 2.07 50% 1000 docs 1.23 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 11.72 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.5942 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0370 Third Quartile 0.1776 Interquartile range 0.1776 Mean 0.1172 Standard Deviation 0.1669 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3750 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0822 Std With No Outliers 0.1179 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries HIGeoenderun22 Topic 026 0.00 Topic 039 3.70 0.8 Topic 027 10.77 Topic 040 29.27 Topic 028 37.50 Topic 041 21.05 0.6 Topic 029 33.33 Topic 042 4.26 Topic 030 10.00 Topic 043 14.29 Topic 031 0.00 Topic 044 0.00 0.4 Topic 032 44.44 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 59.42 0 Topic 036 0.00 Topic 049 16.67 Topic 037 0.00 Topic 050 8.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 332 hildesheim HIGeoenderun21n GC-BILI-X2DE-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 602 Topic Fields title, description, narrative Relevant retrieved 369 Pooled true Geometric Mean Average Precision 0.0125 Experiment with BRF(5docs,25terms) base Binary Preference (BPREF) 0.1346 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision 100% 0 32.78 HIGeoenderun21n 10 26.86 90% 20 20.68 80% 30 19.28 40 16.17 70% 50 14.60 Average Precision 60% 60 10.66 70 7.08 50% 80 3.74 40% 90 2.17 30% 100 0.05 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 13.15 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.5798 Minimum 0.0000 First Quartile 0.0024 Second Quartile 0.0198 Third Quartile 0.2114 Interquartile range 0.2090 Mean 0.1315 Standard Deviation 0.1941 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5040 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0946 Std With No Outliers 0.1523 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries HIGeoenderun21n Topic 026 0.22 Topic 039 1.49 0.8 Topic 027 1.69 Topic 040 25.86 Topic 028 32.67 Topic 041 9.42 0.6 Topic 029 50.40 Topic 042 0.71 Topic 030 8.43 Topic 043 1.81 Topic 031 5.08 Topic 044 0.25 0.4 Topic 032 47.45 Topic 045 0.00 Topic 033 0.00 Topic 046 1.98 0.2 Topic 034 53.30 Topic 047 0.03 Difference Topic 035 5.43 Topic 048 57.98 0 Topic 036 0.08 Topic 049 19.57 Topic 037 0.32 Topic 050 4.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 333 hildesheim HIGeoenderun21n GC-BILI-X2DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual German track − Retrieved documents vs Precision 100% 5 docs 15.20 HIGeoenderun21n 10 docs 15.60 90% 15 docs 15.20 80% 20 docs 15.00 30 docs 14.13 70% 100 docs 8.92 60% 200 docs 5.46 R−Precision 500 docs 2.62 50% 1000 docs 1.48 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 15.21 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.5882 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0714 Third Quartile 0.2083 Interquartile range 0.2083 Mean 0.1521 Standard Deviation 0.2011 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4146 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0945 Std With No Outliers 0.1313 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries HIGeoenderun21n Topic 026 0.00 Topic 039 7.41 0.8 Topic 027 13.85 Topic 040 41.46 Topic 028 40.62 Topic 041 5.26 0.6 Topic 029 33.33 Topic 042 6.38 Topic 030 10.00 Topic 043 7.14 Topic 031 9.21 Topic 044 0.00 0.4 Topic 032 55.56 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 58.82 Topic 047 0.00 Difference Topic 035 16.67 Topic 048 57.97 0 Topic 036 0.00 Topic 049 16.67 Topic 037 0.00 Topic 050 0.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 334 hildesheim HIGeoenderun22n GC-BILI-X2DE-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 602 Topic Fields title, description, narrative Relevant retrieved 303 Pooled true Geometric Mean Average Precision 0.0048 Experiment with BRF(5docs,25terms) with NE- Binary Preference (BPREF) 0.0977 recognition and weighting, also within the BRF Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision 100% 0 25.64 HIGeoenderun22n 10 20.12 90% 20 17.28 80% 30 16.36 40 12.18 70% 50 10.84 Average Precision 60% 60 7.61 70 5.36 50% 80 4.12 40% 90 2.64 30% 100 0.35 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 10.46 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.8120 Minimum 0.0000 First Quartile 0.0006 Second Quartile 0.0131 Third Quartile 0.0898 Interquartile range 0.0892 Mean 0.1046 Standard Deviation 0.1935 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.1877 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0323 Std With No Outliers 0.0521 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries HIGeoenderun22n Topic 026 0.00 Topic 039 1.31 0.8 Topic 027 0.16 Topic 040 18.77 Topic 028 34.50 Topic 041 6.32 0.6 Topic 029 46.97 Topic 042 3.28 Topic 030 4.61 Topic 043 0.63 Topic 031 0.51 Topic 044 0.17 0.4 Topic 032 31.07 Topic 045 0.00 Topic 033 0.00 Topic 046 3.21 0.2 Topic 034 6.67 Topic 047 0.00 Difference Topic 035 0.08 Topic 048 81.20 0 Topic 036 0.95 Topic 049 15.90 Topic 037 0.00 Topic 050 5.17 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 335 hildesheim HIGeoenderun22n GC-BILI-X2DE-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual German track − Retrieved documents vs Precision 100% 5 docs 16.00 HIGeoenderun22n 10 docs 14.00 90% 15 docs 12.53 80% 20 docs 12.20 30 docs 12.00 70% 100 docs 6.88 60% 200 docs 4.24 R−Precision 500 docs 2.12 50% 1000 docs 1.21 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 11.77 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment Maximum 0.7681 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0333 Third Quartile 0.1299 Interquartile range 0.1299 Mean 0.1177 Standard Deviation 0.1891 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.2927 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0480 Std With No Outliers 0.0737 GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries HIGeoenderun22n Topic 026 0.00 Topic 039 7.41 0.8 Topic 027 3.08 Topic 040 29.27 Topic 028 40.62 Topic 041 5.26 0.6 Topic 029 33.33 Topic 042 8.51 Topic 030 3.33 Topic 043 7.14 Topic 031 0.00 Topic 044 0.00 0.4 Topic 032 42.59 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 11.76 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 76.81 0 Topic 036 0.00 Topic 049 16.67 Topic 037 0.00 Topic 050 8.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 336 hildesheim HIGeodeenrun12 GC-BILI-X2EN-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language German Relevant 378 Topic Fields title, description Relevant retrieved 222 Pooled true Geometric Mean Average Precision 0.0103 Experiment with BRF(5docs,20terms) with Binary Preference (BPREF) 0.1489 GeoNEweighting within the BRF-algorithm Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision 100% 0 31.10 HIGeodeenrun12 10 28.46 90% 20 24.65 80% 30 21.50 40 19.76 70% 50 18.40 Average Precision 60% 60 16.78 70 9.17 50% 80 6.49 40% 90 5.12 30% 100 3.95 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 16.03 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 0.8837 Minimum 0.0000 First Quartile 0.0009 Second Quartile 0.0291 Third Quartile 0.2416 Interquartile range 0.2407 Mean 0.1603 Standard Deviation 0.2448 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5095 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1027 Std With No Outliers 0.1473 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries HIGeodeenrun12 Topic 026 1.18 Topic 039 0.63 0.8 Topic 027 0.04 Topic 040 88.37 Topic 028 0.10 Topic 041 0.00 0.6 Topic 029 13.37 Topic 042 0.09 Topic 030 27.62 Topic 043 0.03 Topic 031 23.00 Topic 044 3.18 0.4 Topic 032 50.95 Topic 045 0.35 Topic 033 0.00 Topic 046 39.81 0.2 Topic 034 28.47 Topic 047 2.91 Difference Topic 035 0.07 Topic 048 76.11 0 Topic 036 0.00 Topic 049 10.43 Topic 037 0.13 Topic 050 21.30 −0.2 Topic 038 12.50 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 337 hildesheim HIGeodeenrun12 GC-BILI-X2EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual English track − Retrieved documents vs Precision 100% 5 docs 20.80 HIGeodeenrun12 10 docs 18.40 90% 15 docs 16.80 80% 20 docs 15.40 30 docs 13.60 70% 100 docs 5.20 60% 200 docs 3.10 R−Precision 500 docs 1.64 50% 1000 docs 0.89 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 17.52 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 0.7857 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.2867 Interquartile range 0.2867 Mean 0.1752 Standard Deviation 0.2615 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7083 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1497 Std With No Outliers 0.2334 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries HIGeodeenrun12 Topic 026 0.00 Topic 039 0.00 0.8 Topic 027 0.00 Topic 040 78.57 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 22.22 Topic 042 0.00 Topic 030 33.33 Topic 043 0.00 Topic 031 27.12 Topic 044 10.53 0.4 Topic 032 64.52 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 4.17 Difference Topic 035 0.00 Topic 048 70.83 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 26.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 338 hildesheim HIGeodeenrun13n GC-BILI-X2EN-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 23,326 Source Language German Relevant 378 Topic Fields title, description, narrative Relevant retrieved 214 Pooled true Geometric Mean Average Precision 0.0054 Experiment with BRF(5docs,25terms) with NE- Binary Preference (BPREF) 0.1322 recognition and weighting, also within the BRF Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision 100% 0 29.63 HIGeodeenrun13n 10 23.84 90% 20 23.36 80% 30 22.09 40 21.26 70% 50 19.87 Average Precision 60% 60 17.58 70 9.80 50% 80 4.97 40% 90 3.66 30% 100 2.33 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 15.65 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 0.8353 Minimum 0.0000 First Quartile 0.0001 Second Quartile 0.0271 Third Quartile 0.1738 Interquartile range 0.1738 Mean 0.1565 Standard Deviation 0.2497 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.2324 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0422 Std With No Outliers 0.0658 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries HIGeodeenrun13n Topic 026 1.96 Topic 039 0.79 0.8 Topic 027 0.01 Topic 040 74.25 Topic 028 0.05 Topic 041 0.08 0.6 Topic 029 9.05 Topic 042 0.00 Topic 030 46.46 Topic 043 0.00 Topic 031 23.24 Topic 044 3.09 0.4 Topic 032 52.48 Topic 045 2.71 Topic 033 0.01 Topic 046 8.59 0.2 Topic 034 0.00 Topic 047 5.46 Difference Topic 035 0.00 Topic 048 83.53 0 Topic 036 0.00 Topic 049 13.91 Topic 037 0.05 Topic 050 15.43 −0.2 Topic 038 50.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 339 hildesheim HIGeodeenrun13n GC-BILI-X2EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual English track − Retrieved documents vs Precision 100% 5 docs 20.80 HIGeodeenrun13n 10 docs 17.60 90% 15 docs 16.53 80% 20 docs 14.40 30 docs 12.53 70% 100 docs 5.44 60% 200 docs 3.28 R−Precision 500 docs 1.58 50% 1000 docs 0.86 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 14.83 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 0.7500 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.2333 Interquartile range 0.2333 Mean 0.1483 Standard Deviation 0.2572 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3390 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0459 Std With No Outliers 0.1002 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries HIGeodeenrun13n Topic 026 0.00 Topic 039 0.00 0.8 Topic 027 0.00 Topic 040 71.43 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 22.22 Topic 042 0.00 Topic 030 66.67 Topic 043 0.00 Topic 031 33.90 Topic 044 5.26 0.4 Topic 032 61.29 Topic 045 0.00 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 0.00 Topic 047 8.33 Difference Topic 035 0.00 Topic 048 75.00 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 26.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 340 hildesheim HIGeodeenrun11n GC-BILI-X2EN-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language German Relevant 378 Topic Fields title, description, narrative Relevant retrieved 217 Pooled false Geometric Mean Average Precision 0.0096 no BRF base run, stem snowball Binary Preference (BPREF) 0.1824 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision 100% 0 32.96 HIGeodeenrun11n 10 30.42 90% 20 29.01 80% 30 26.47 40 22.46 70% 50 20.45 Average Precision 60% 60 17.94 70 10.35 50% 80 8.30 40% 90 7.74 30% 100 6.65 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 19.03 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0007 Second Quartile 0.0147 Third Quartile 0.2478 Interquartile range 0.2471 Mean 0.1903 Standard Deviation 0.3065 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5153 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0748 Std With No Outliers 0.1475 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries HIGeodeenrun11n Topic 026 0.92 Topic 039 0.75 0.8 Topic 027 0.01 Topic 040 81.81 Topic 028 0.05 Topic 041 0.00 0.6 Topic 029 8.67 Topic 042 0.37 Topic 030 64.80 Topic 043 0.03 Topic 031 16.15 Topic 044 1.60 0.4 Topic 032 51.53 Topic 045 0.64 Topic 033 0.00 Topic 046 46.26 0.2 Topic 034 8.82 Topic 047 2.02 Difference Topic 035 0.08 Topic 048 71.89 0 Topic 036 0.00 Topic 049 1.47 Topic 037 0.16 Topic 050 17.62 −0.2 Topic 038 100.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 341 hildesheim HIGeodeenrun11n GC-BILI-X2EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual English track − Retrieved documents vs Precision 100% 5 docs 18.40 HIGeodeenrun11n 10 docs 17.60 90% 15 docs 15.20 80% 20 docs 15.20 30 docs 12.13 70% 100 docs 4.72 60% 200 docs 2.90 R−Precision 500 docs 1.58 50% 1000 docs 0.87 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 19.01 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.2833 Interquartile range 0.2833 Mean 0.1901 Standard Deviation 0.2948 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1321 Std With No Outliers 0.2214 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries HIGeodeenrun11n Topic 026 0.00 Topic 039 0.00 0.8 Topic 027 0.00 Topic 040 71.43 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 22.22 Topic 042 0.00 Topic 030 66.67 Topic 043 0.00 Topic 031 20.34 Topic 044 7.89 0.4 Topic 032 58.06 Topic 045 0.00 Topic 033 0.00 Topic 046 33.33 0.2 Topic 034 0.00 Topic 047 4.17 Difference Topic 035 0.00 Topic 048 64.58 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 26.67 −0.2 Topic 038 100.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 342 hildesheim HIGeodeenrun11 GC-BILI-X2EN-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language German Relevant 378 Topic Fields title, description Relevant retrieved 216 Pooled false Geometric Mean Average Precision 0.0081 no BRF base run, stem snowball Binary Preference (BPREF) 0.1421 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision 100% 0 29.92 HIGeodeenrun11 10 27.68 90% 20 25.94 80% 30 23.21 40 17.72 70% 50 16.09 Average Precision 60% 60 14.39 70 6.24 50% 80 4.89 40% 90 4.32 30% 100 3.43 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 15.04 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 0.8091 Minimum 0.0000 First Quartile 0.0007 Second Quartile 0.0089 Third Quartile 0.2037 Interquartile range 0.2030 Mean 0.1504 Standard Deviation 0.2363 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5066 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0987 Std With No Outliers 0.1600 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries HIGeodeenrun11 Topic 026 0.89 Topic 039 0.67 0.8 Topic 027 0.04 Topic 040 80.91 Topic 028 0.10 Topic 041 0.00 0.6 Topic 029 9.63 Topic 042 0.08 Topic 030 43.57 Topic 043 0.03 Topic 031 16.46 Topic 044 0.63 0.4 Topic 032 50.66 Topic 045 0.30 Topic 033 0.00 Topic 046 41.68 0.2 Topic 034 28.89 Topic 047 2.06 Difference Topic 035 0.03 Topic 048 68.14 0 Topic 036 0.00 Topic 049 1.10 Topic 037 0.14 Topic 050 17.53 −0.2 Topic 038 12.50 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 343 hildesheim HIGeodeenrun11 GC-BILI-X2EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual English track − Retrieved documents vs Precision 100% 5 docs 19.20 HIGeodeenrun11 10 docs 17.60 90% 15 docs 15.73 80% 20 docs 14.20 30 docs 12.13 70% 100 docs 4.32 60% 200 docs 2.68 R−Precision 500 docs 1.49 50% 1000 docs 0.86 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 15.18 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 0.7143 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.2833 Interquartile range 0.2833 Mean 0.1518 Standard Deviation 0.2330 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6250 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1284 Std With No Outliers 0.2057 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries HIGeodeenrun11 Topic 026 0.00 Topic 039 0.00 0.8 Topic 027 0.00 Topic 040 71.43 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 22.22 Topic 042 0.00 Topic 030 50.00 Topic 043 0.00 Topic 031 22.03 Topic 044 0.00 0.4 Topic 032 58.06 Topic 045 0.00 Topic 033 0.00 Topic 046 33.33 0.2 Topic 034 33.33 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 62.50 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 26.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 344 hildesheim HIGeodeenrun13 GC-BILI-X2EN-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 21,396 Source Language German Relevant 378 Topic Fields title, description Relevant retrieved 178 Pooled false Geometric Mean Average Precision 0.0038 Experiment with BRF(5docs,25terms) with NE- Binary Preference (BPREF) 0.1337 recognition and weighting, also within the BRF Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision 100% 0 26.22 HIGeodeenrun13 10 22.77 90% 20 21.63 80% 30 20.42 40 18.83 70% 50 18.66 Average Precision 60% 60 16.47 70 9.67 50% 80 4.64 40% 90 2.43 30% 100 1.03 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 14.56 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 0.7828 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0070 Third Quartile 0.2040 Interquartile range 0.2040 Mean 0.1456 Standard Deviation 0.2567 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4530 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0624 Std With No Outliers 0.1216 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries HIGeodeenrun13 Topic 026 0.70 Topic 039 0.46 0.8 Topic 027 0.14 Topic 040 78.28 Topic 028 0.00 Topic 041 0.09 0.6 Topic 029 3.84 Topic 042 0.00 Topic 030 70.14 Topic 043 0.00 Topic 031 21.59 Topic 044 2.92 0.4 Topic 032 45.30 Topic 045 0.23 Topic 033 0.00 Topic 046 31.12 0.2 Topic 034 1.52 Topic 047 2.59 Difference Topic 035 0.00 Topic 048 78.12 0 Topic 036 0.00 Topic 049 0.52 Topic 037 0.00 Topic 050 6.38 −0.2 Topic 038 20.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 345 hildesheim HIGeodeenrun13 GC-BILI-X2EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual English track − Retrieved documents vs Precision 100% 5 docs 20.00 HIGeodeenrun13 10 docs 16.80 90% 15 docs 14.13 80% 20 docs 13.80 30 docs 11.87 70% 100 docs 4.88 60% 200 docs 3.08 R−Precision 500 docs 1.38 50% 1000 docs 0.71 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 14.65 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 0.7143 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0000 Third Quartile 0.2136 Interquartile range 0.2136 Mean 0.1465 Standard Deviation 0.2491 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.0473 Std With No Outliers 0.0963 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 20 Number of Topics of the Experiment 15 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries HIGeodeenrun13 Topic 026 0.00 Topic 039 0.00 0.8 Topic 027 0.00 Topic 040 71.43 Topic 028 0.00 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 66.67 Topic 043 0.00 Topic 031 25.42 Topic 044 5.26 0.4 Topic 032 58.06 Topic 045 0.00 Topic 033 0.00 Topic 046 33.33 0.2 Topic 034 0.00 Topic 047 4.17 Difference Topic 035 0.00 Topic 048 70.83 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 20.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 346 jaen sinaiEsEnExp1 GC-BILI-X2EN-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Spanish; Castilian Relevant 378 Topic Fields title, description, narrative Relevant retrieved 280 Pooled true Geometric Mean Average Precision 0.0185 Caso base ESEN Binary Preference (BPREF) 0.2420 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision 100% 0 47.54 sinaiEsEnExp1 10 44.48 90% 20 32.64 80% 30 29.39 40 29.19 70% 50 28.75 Average Precision 60% 60 25.59 70 21.77 50% 80 21.09 40% 90 18.68 30% 100 15.54 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 27.07 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 0.9713 Minimum 0.0000 First Quartile 0.0047 Second Quartile 0.1597 Third Quartile 0.3573 Interquartile range 0.3525 Mean 0.2707 Standard Deviation 0.3413 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6429 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1435 Std With No Outliers 0.1829 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries sinaiEsEnExp1 Topic 026 20.88 Topic 039 8.94 0.8 Topic 027 0.00 Topic 040 26.94 Topic 028 17.59 Topic 041 0.63 0.6 Topic 029 15.91 Topic 042 58.33 Topic 030 94.84 Topic 043 0.00 Topic 031 15.97 Topic 044 7.86 0.4 Topic 032 97.13 Topic 045 16.67 Topic 033 1.36 Topic 046 91.67 0.2 Topic 034 28.19 Topic 047 0.01 Difference Topic 035 0.93 Topic 048 91.79 0 Topic 036 0.00 Topic 049 64.29 Topic 037 0.00 Topic 050 16.93 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 347 jaen sinaiEsEnExp1 GC-BILI-X2EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual English track − Retrieved documents vs Precision 100% 5 docs 28.00 sinaiEsEnExp1 10 docs 19.60 90% 15 docs 16.80 80% 20 docs 15.40 30 docs 14.00 70% 100 docs 6.68 60% 200 docs 4.10 R−Precision 500 docs 2.06 50% 1000 docs 1.12 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 24.27 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 0.9355 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1333 Third Quartile 0.3750 Interquartile range 0.3750 Mean 0.2427 Standard Deviation 0.2976 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.9355 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2427 Std With No Outliers 0.2976 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries sinaiEsEnExp1 Topic 026 22.22 Topic 039 6.25 0.8 Topic 027 0.00 Topic 040 21.43 Topic 028 21.05 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 50.00 Topic 030 83.33 Topic 043 0.00 Topic 031 16.95 Topic 044 10.53 0.4 Topic 032 93.55 Topic 045 16.67 Topic 033 5.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 85.42 0 Topic 036 0.00 Topic 049 50.00 Topic 037 0.00 Topic 050 13.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 348 jaen sinaiDeEnExp2 GC-BILI-X2EN-CLEF2006 Overall statistics for 25 queries : Priority 5 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language German Relevant 378 Topic Fields title, description Relevant retrieved 324 Pooled false Geometric Mean Average Precision 0.0602 Caso Base DEEN Binary Preference (BPREF) 0.1675 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision 100% 0 41.27 sinaiDeEnExp2 10 34.02 90% 20 29.37 80% 30 25.85 40 23.64 70% 50 21.97 Average Precision 60% 60 19.99 70 17.72 50% 80 16.35 40% 90 12.07 30% 100 8.80 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 21.64 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 0.9086 Minimum 0.0000 First Quartile 0.0433 Second Quartile 0.0954 Third Quartile 0.3542 Interquartile range 0.3108 Mean 0.2164 Standard Deviation 0.2395 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7905 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1875 Std With No Outliers 0.1954 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries sinaiDeEnExp2 Topic 026 22.57 Topic 039 4.83 0.8 Topic 027 4.44 Topic 040 25.23 Topic 028 31.83 Topic 041 1.19 0.6 Topic 029 4.01 Topic 042 35.00 Topic 030 5.94 Topic 043 1.34 Topic 031 40.49 Topic 044 19.71 0.4 Topic 032 79.05 Topic 045 7.85 Topic 033 0.00 Topic 046 40.00 0.2 Topic 034 41.67 Topic 047 5.71 Difference Topic 035 2.28 Topic 048 90.86 0 Topic 036 0.00 Topic 049 36.67 Topic 037 9.54 Topic 050 23.00 −0.2 Topic 038 7.69 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 349 jaen sinaiDeEnExp2 GC-BILI-X2EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual English track − Retrieved documents vs Precision 100% 5 docs 24.80 sinaiDeEnExp2 10 docs 22.40 90% 15 docs 20.27 80% 20 docs 18.20 30 docs 16.67 70% 100 docs 8.96 60% 200 docs 5.42 R−Precision 500 docs 2.46 50% 1000 docs 1.30 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 19.55 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 0.8542 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1111 Third Quartile 0.3333 Interquartile range 0.3333 Mean 0.1955 Standard Deviation 0.2383 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.7419 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1681 Std With No Outliers 0.1990 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries sinaiDeEnExp2 Topic 026 22.22 Topic 039 0.00 0.8 Topic 027 10.53 Topic 040 21.43 Topic 028 36.84 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 50.00 Topic 030 0.00 Topic 043 0.00 Topic 031 40.68 Topic 044 28.95 0.4 Topic 032 74.19 Topic 045 0.00 Topic 033 0.00 Topic 046 33.33 0.2 Topic 034 33.33 Topic 047 8.33 Difference Topic 035 0.00 Topic 048 85.42 0 Topic 036 0.00 Topic 049 0.00 Topic 037 12.50 Topic 050 20.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 350 jaen sinaiEsEnExp3 GC-BILI-X2EN-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Spanish; Castilian Relevant 378 Topic Fields title, description Relevant retrieved 311 Pooled false Geometric Mean Average Precision 0.0353 Expansión con geonames Binary Preference (BPREF) 0.1751 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision 100% 0 41.03 sinaiEsEnExp3 10 29.06 90% 20 26.45 80% 30 25.83 40 25.05 70% 50 24.62 Average Precision 60% 60 23.98 70 19.48 50% 80 18.17 40% 90 14.71 30% 100 12.01 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 22.09 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0229 Second Quartile 0.0966 Third Quartile 0.2829 Interquartile range 0.2600 Mean 0.2209 Standard Deviation 0.3045 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4514 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1206 Std With No Outliers 0.1345 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries sinaiEsEnExp3 Topic 026 0.00 Topic 039 5.06 0.8 Topic 027 0.00 Topic 040 26.80 Topic 028 25.64 Topic 041 2.25 0.6 Topic 029 7.51 Topic 042 6.16 Topic 030 100.00 Topic 043 2.20 Topic 031 12.56 Topic 044 10.50 0.4 Topic 032 94.60 Topic 045 10.62 Topic 033 0.21 Topic 046 32.78 0.2 Topic 034 45.14 Topic 047 3.72 Difference Topic 035 2.31 Topic 048 92.19 0 Topic 036 0.00 Topic 049 36.67 Topic 037 9.66 Topic 050 23.30 −0.2 Topic 038 2.33 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 351 jaen sinaiEsEnExp3 GC-BILI-X2EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual English track − Retrieved documents vs Precision 100% 5 docs 22.40 sinaiEsEnExp3 10 docs 18.80 90% 15 docs 17.07 80% 20 docs 16.60 30 docs 14.67 70% 100 docs 7.32 60% 200 docs 4.78 R−Precision 500 docs 2.30 50% 1000 docs 1.24 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 20.42 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0789 Third Quartile 0.2789 Interquartile range 0.2789 Mean 0.2042 Standard Deviation 0.3055 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1091 Std With No Outliers 0.1643 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries sinaiEsEnExp3 Topic 026 0.00 Topic 039 0.00 0.8 Topic 027 0.00 Topic 040 14.29 Topic 028 31.58 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 100.00 Topic 043 12.50 Topic 031 6.78 Topic 044 7.89 0.4 Topic 032 87.10 Topic 045 16.67 Topic 033 0.00 Topic 046 33.33 0.2 Topic 034 66.67 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 83.33 0 Topic 036 0.00 Topic 049 0.00 Topic 037 12.50 Topic 050 26.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 352 jaen sinaiDeEnExp1 GC-BILI-X2EN-CLEF2006 Overall statistics for 25 queries : Priority 4 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language German Relevant 378 Topic Fields title, description, narrative Relevant retrieved 293 Pooled false Geometric Mean Average Precision 0.0369 Caso Base DEEN Binary Preference (BPREF) 0.1464 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision 100% 0 44.94 sinaiDeEnExp1 10 31.63 90% 20 24.65 80% 30 20.56 40 19.46 70% 50 18.90 Average Precision 60% 60 15.60 70 14.56 50% 80 14.04 40% 90 10.84 30% 100 8.34 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 18.68 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 0.9625 Minimum 0.0000 First Quartile 0.0119 Second Quartile 0.0884 Third Quartile 0.2174 Interquartile range 0.2054 Mean 0.1868 Standard Deviation 0.2643 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3203 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0990 Std With No Outliers 0.0971 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries sinaiDeEnExp1 Topic 026 19.05 Topic 039 15.98 0.8 Topic 027 6.72 Topic 040 32.03 Topic 028 11.88 Topic 041 1.14 0.6 Topic 029 11.53 Topic 042 25.00 Topic 030 3.49 Topic 043 3.49 Topic 031 24.41 Topic 044 7.41 0.4 Topic 032 96.25 Topic 045 20.85 Topic 033 0.25 Topic 046 5.91 0.2 Topic 034 8.84 Topic 047 0.16 Difference Topic 035 1.21 Topic 048 90.47 0 Topic 036 0.00 Topic 049 62.50 Topic 037 0.00 Topic 050 18.02 −0.2 Topic 038 0.51 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 353 jaen sinaiDeEnExp1 GC-BILI-X2EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual English track − Retrieved documents vs Precision 100% 5 docs 21.60 sinaiDeEnExp1 10 docs 17.60 90% 15 docs 16.27 80% 20 docs 15.60 30 docs 14.67 70% 100 docs 6.92 60% 200 docs 4.50 R−Precision 500 docs 2.14 50% 1000 docs 1.17 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 16.49 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 0.9032 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1053 Third Quartile 0.2208 Interquartile range 0.2208 Mean 0.1649 Standard Deviation 0.2517 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1029 Std With No Outliers 0.1369 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries sinaiDeEnExp1 Topic 026 22.22 Topic 039 12.50 0.8 Topic 027 10.53 Topic 040 28.57 Topic 028 15.79 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 0.00 Topic 043 0.00 Topic 031 22.03 Topic 044 10.53 0.4 Topic 032 90.32 Topic 045 33.33 Topic 033 0.00 Topic 046 0.00 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 85.42 0 Topic 036 0.00 Topic 049 50.00 Topic 037 0.00 Topic 050 20.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 354 jaen sinaiEsEnExp2 GC-BILI-X2EN-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Spanish; Castilian Relevant 378 Topic Fields title, description Relevant retrieved 312 Pooled true Geometric Mean Average Precision 0.0364 Caso Base ESEN Binary Preference (BPREF) 0.1811 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision 100% 0 38.36 sinaiEsEnExp2 10 32.16 90% 20 29.44 80% 30 26.16 40 25.05 70% 50 24.67 Average Precision 60% 60 23.93 70 19.69 50% 80 18.20 40% 90 14.24 30% 100 11.83 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 22.56 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 0.9762 Minimum 0.0000 First Quartile 0.0229 Second Quartile 0.0966 Third Quartile 0.2829 Interquartile range 0.2600 Mean 0.2256 Standard Deviation 0.3015 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4514 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1269 Std With No Outliers 0.1370 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries sinaiEsEnExp2 Topic 026 20.81 Topic 039 5.06 0.8 Topic 027 0.00 Topic 040 26.80 Topic 028 25.64 Topic 041 2.25 0.6 Topic 029 3.47 Topic 042 6.16 Topic 030 97.62 Topic 043 2.20 Topic 031 16.26 Topic 044 14.37 0.4 Topic 032 95.07 Topic 045 0.28 Topic 033 0.00 Topic 046 32.78 0.2 Topic 034 45.14 Topic 047 3.72 Difference Topic 035 2.31 Topic 048 92.19 0 Topic 036 0.00 Topic 049 36.67 Topic 037 9.66 Topic 050 23.30 −0.2 Topic 038 2.33 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 355 jaen sinaiEsEnExp2 GC-BILI-X2EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual English track − Retrieved documents vs Precision 100% 5 docs 21.60 sinaiEsEnExp2 10 docs 18.80 90% 15 docs 17.33 80% 20 docs 16.40 30 docs 14.80 70% 100 docs 7.80 60% 200 docs 5.02 R−Precision 500 docs 2.38 50% 1000 docs 1.25 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 20.63 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 0.8710 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1111 Third Quartile 0.2789 Interquartile range 0.2789 Mean 0.2063 Standard Deviation 0.2870 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1191 Std With No Outliers 0.1665 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries sinaiEsEnExp2 Topic 026 22.22 Topic 039 0.00 0.8 Topic 027 0.00 Topic 040 14.29 Topic 028 31.58 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 0.00 Topic 030 83.33 Topic 043 12.50 Topic 031 10.17 Topic 044 21.05 0.4 Topic 032 87.10 Topic 045 0.00 Topic 033 0.00 Topic 046 33.33 0.2 Topic 034 66.67 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 83.33 0 Topic 036 0.00 Topic 049 0.00 Topic 037 12.50 Topic 050 26.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 356 sanmarcos SMGeoESEN1 GC-BILI-X2EN-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction MANUAL Retrieved 24,252 Source Language Spanish; Castilian Relevant 378 Topic Fields title, description, narrative Relevant retrieved 318 Pooled true Geometric Mean Average Precision 0.0806 Spanish-English using all topic fields and other Binary Preference (BPREF) 0.2164 sources Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision 100% 0 50.00 SMGeoESEN1 10 43.95 90% 20 36.81 80% 30 32.72 40 31.41 70% 50 29.72 Average Precision 60% 60 25.24 70 13.74 50% 80 11.06 40% 90 8.59 30% 100 6.39 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 25.52 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 0.9601 Minimum 0.0000 First Quartile 0.0430 Second Quartile 0.1743 Third Quartile 0.3584 Interquartile range 0.3154 Mean 0.2552 Standard Deviation 0.2771 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8258 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2258 Std With No Outliers 0.2400 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoESEN1 Topic 026 4.43 Topic 039 36.86 0.8 Topic 027 7.30 Topic 040 32.50 Topic 028 20.03 Topic 041 0.30 0.6 Topic 029 5.72 Topic 042 30.26 Topic 030 82.58 Topic 043 1.22 Topic 031 49.12 Topic 044 9.27 0.4 Topic 032 96.01 Topic 045 35.50 Topic 033 5.52 Topic 046 67.30 0.2 Topic 034 33.99 Topic 047 4.75 Difference Topic 035 3.90 Topic 048 66.89 0 Topic 036 0.00 Topic 049 17.43 Topic 037 0.29 Topic 050 23.01 −0.2 Topic 038 3.85 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 357 sanmarcos SMGeoESEN1 GC-BILI-X2EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual English track − Retrieved documents vs Precision 100% 5 docs 28.80 SMGeoESEN1 10 docs 24.40 90% 15 docs 23.47 80% 20 docs 21.80 30 docs 18.40 70% 100 docs 8.16 60% 200 docs 5.00 R−Precision 500 docs 2.34 50% 1000 docs 1.27 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 24.80 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 0.9032 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1500 Third Quartile 0.3999 Interquartile range 0.3999 Mean 0.2480 Standard Deviation 0.2643 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.9032 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2480 Std With No Outliers 0.2643 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoESEN1 Topic 026 11.11 Topic 039 37.50 0.8 Topic 027 5.26 Topic 040 35.71 Topic 028 21.05 Topic 041 0.00 0.6 Topic 029 11.11 Topic 042 50.00 Topic 030 66.67 Topic 043 0.00 Topic 031 47.46 Topic 044 10.53 0.4 Topic 032 90.32 Topic 045 16.67 Topic 033 15.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 8.33 Difference Topic 035 0.00 Topic 048 66.67 0 Topic 036 0.00 Topic 049 0.00 Topic 037 0.00 Topic 050 26.67 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 358 sanmarcos SMGeoESEN2 GC-BILI-X2EN-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Spanish; Castilian Relevant 378 Topic Fields title, description Relevant retrieved 303 Pooled true Geometric Mean Average Precision 0.0512 Automatic Spanish English title+des Binary Preference (BPREF) 0.2039 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision 100% 0 43.77 SMGeoESEN2 10 35.49 90% 20 28.62 80% 30 27.29 40 26.08 70% 50 25.13 Average Precision 60% 60 22.38 70 16.74 50% 80 12.44 40% 90 11.27 30% 100 8.86 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 22.46 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0150 Second Quartile 0.1157 Third Quartile 0.2798 Interquartile range 0.2648 Mean 0.2246 Standard Deviation 0.2979 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4242 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1095 Std With No Outliers 0.1268 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoESEN2 Topic 026 6.35 Topic 039 16.01 0.8 Topic 027 1.18 Topic 040 26.20 Topic 028 11.57 Topic 041 0.23 0.6 Topic 029 21.08 Topic 042 0.82 Topic 030 100.00 Topic 043 1.67 Topic 031 13.41 Topic 044 22.48 0.4 Topic 032 91.32 Topic 045 1.67 Topic 033 0.46 Topic 046 69.30 0.2 Topic 034 42.42 Topic 047 3.44 Difference Topic 035 2.10 Topic 048 70.96 0 Topic 036 0.00 Topic 049 33.33 Topic 037 1.60 Topic 050 23.43 −0.2 Topic 038 0.56 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 359 sanmarcos SMGeoESEN2 GC-BILI-X2EN-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual English track − Retrieved documents vs Precision 100% 5 docs 24.80 SMGeoESEN2 10 docs 21.20 90% 15 docs 18.93 80% 20 docs 17.60 30 docs 15.20 70% 100 docs 7.32 60% 200 docs 4.40 R−Precision 500 docs 2.06 50% 1000 docs 1.21 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 23.29 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment Maximum 1.0000 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1111 Third Quartile 0.3355 Interquartile range 0.3355 Mean 0.2329 Standard Deviation 0.2905 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.8065 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2010 Std With No Outliers 0.2479 GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoESEN2 Topic 026 11.11 Topic 039 12.50 0.8 Topic 027 5.26 Topic 040 21.43 Topic 028 10.53 Topic 041 0.00 0.6 Topic 029 22.22 Topic 042 0.00 Topic 030 100.00 Topic 043 0.00 Topic 031 13.56 Topic 044 34.21 0.4 Topic 032 80.65 Topic 045 0.00 Topic 033 0.00 Topic 046 66.67 0.2 Topic 034 33.33 Topic 047 8.33 Difference Topic 035 0.00 Topic 048 72.92 0 Topic 036 0.00 Topic 049 50.00 Topic 037 6.25 Topic 050 33.33 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 360 berkeley BKGeoES1 GC-BILI-X2ES-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 2,054 Topic Fields title, description Relevant retrieved 1,504 Pooled true Geometric Mean Average Precision 0.0552 EN->ES using L&H query translation. title, desc Binary Preference (BPREF) 0.2676 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual Spanish track − Interpolated Recall vs Average Precision 100% 0 52.97 BKGeoES1 10 40.03 90% 20 33.75 80% 30 31.01 40 28.71 70% 50 26.83 Average Precision 60% 60 25.15 70 21.98 50% 80 19.08 40% 90 13.02 30% 100 4.85 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 25.71 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.9782 Minimum 0.0000 First Quartile 0.0184 Second Quartile 0.1361 Third Quartile 0.4076 Interquartile range 0.3892 Mean 0.2571 Standard Deviation 0.3008 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.9782 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2571 Std With No Outliers 0.3008 GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries BKGeoES1 Topic 026 0.01 Topic 039 0.98 0.8 Topic 027 0.84 Topic 040 56.39 Topic 028 11.51 Topic 041 22.43 0.6 Topic 029 25.94 Topic 042 35.55 Topic 030 0.00 Topic 043 0.13 Topic 031 65.65 Topic 044 18.03 0.4 Topic 032 97.82 Topic 045 5.67 Topic 033 2.12 Topic 046 61.44 0.2 Topic 034 13.61 Topic 047 28.60 Difference Topic 035 0.71 Topic 048 82.80 0 Topic 036 2.96 Topic 049 78.74 Topic 037 6.28 Topic 050 17.96 −0.2 Topic 038 6.63 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 361 berkeley BKGeoES1 GC-BILI-X2ES-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual Spanish track − Retrieved documents vs Precision 100% 5 docs 32.00 BKGeoES1 10 docs 34.00 90% 15 docs 33.07 80% 20 docs 31.80 30 docs 30.53 70% 100 docs 27.20 60% 200 docs 20.88 R−Precision 500 docs 11.08 50% 1000 docs 6.02 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 26.45 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.9000 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.1667 Third Quartile 0.4490 Interquartile range 0.4490 Mean 0.2645 Standard Deviation 0.2871 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.9000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2645 Std With No Outliers 0.2871 GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries BKGeoES1 Topic 026 0.00 Topic 039 2.99 0.8 Topic 027 0.00 Topic 040 61.87 Topic 028 5.56 Topic 041 34.67 0.6 Topic 029 30.30 Topic 042 39.62 Topic 030 0.00 Topic 043 0.00 Topic 031 67.45 Topic 044 28.16 0.4 Topic 032 90.00 Topic 045 16.67 Topic 033 3.00 Topic 046 60.71 0.2 Topic 034 13.51 Topic 047 33.90 Difference Topic 035 0.00 Topic 048 73.96 0 Topic 036 6.36 Topic 049 70.51 Topic 037 0.00 Topic 050 22.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 362 berkeley BKGeoES2 GC-BILI-X2ES-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 2,054 Topic Fields title, description, narrative Relevant retrieved 1,450 Pooled true Geometric Mean Average Precision 0.0421 EN->ES using L&H query translation. title, desc Binary Preference (BPREF) 0.2882 and narrative Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual Spanish track − Interpolated Recall vs Average Precision 100% 0 54.49 BKGeoES2 10 39.70 90% 20 38.13 80% 30 35.96 40 32.79 70% 50 29.58 Average Precision 60% 60 26.75 70 22.65 50% 80 19.31 40% 90 12.71 30% 100 3.53 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 27.45 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.9740 Minimum 0.0000 First Quartile 0.0128 Second Quartile 0.1018 Third Quartile 0.5579 Interquartile range 0.5451 Mean 0.2745 Standard Deviation 0.3135 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.9740 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.2745 Std With No Outliers 0.3135 GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries BKGeoES2 Topic 026 0.11 Topic 039 53.58 0.8 Topic 027 38.67 Topic 040 67.60 Topic 028 10.67 Topic 041 16.07 0.6 Topic 029 47.52 Topic 042 47.96 Topic 030 0.00 Topic 043 3.18 Topic 031 62.41 Topic 044 2.83 0.4 Topic 032 97.40 Topic 045 1.38 Topic 033 1.53 Topic 046 63.69 0.2 Topic 034 9.83 Topic 047 0.97 Difference Topic 035 2.03 Topic 048 73.23 0 Topic 036 0.02 Topic 049 74.35 Topic 037 0.00 Topic 050 10.18 −0.2 Topic 038 0.93 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 363 berkeley BKGeoES2 GC-BILI-X2ES-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual Spanish track − Retrieved documents vs Precision 100% 5 docs 33.60 BKGeoES2 10 docs 36.00 90% 15 docs 35.20 80% 20 docs 34.20 30 docs 32.93 70% 100 docs 28.48 60% 200 docs 20.48 R−Precision 500 docs 10.34 50% 1000 docs 5.80 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 27.04 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.9077 Minimum 0.0000 First Quartile 0.0127 Second Quartile 0.1351 Third Quartile 0.5381 Interquartile range 0.5254 Mean 0.2704 Standard Deviation 0.2934 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.9077 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.2704 Std With No Outliers 0.2934 GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries BKGeoES2 Topic 026 0.00 Topic 039 50.75 0.8 Topic 027 41.03 Topic 040 68.35 Topic 028 13.89 Topic 041 18.67 0.6 Topic 029 54.55 Topic 042 50.94 Topic 030 0.00 Topic 043 4.17 Topic 031 60.39 Topic 044 7.77 0.4 Topic 032 90.77 Topic 045 0.00 Topic 033 4.00 Topic 046 53.57 0.2 Topic 034 13.51 Topic 047 1.69 Difference Topic 035 5.26 Topic 048 63.02 0 Topic 036 0.00 Topic 049 69.59 Topic 037 0.00 Topic 050 4.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 364 sanmarcos SMGeoENES1 GC-BILI-X2ES-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 2,054 Topic Fields title, description Relevant retrieved 729 Pooled true Geometric Mean Average Precision 0.0291 Automatic English Spanish title + desc Binary Preference (BPREF) 0.1505 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual Spanish track − Interpolated Recall vs Average Precision 100% 0 57.89 SMGeoENES1 10 36.39 90% 20 24.05 80% 30 15.35 40 11.82 70% 50 9.72 Average Precision 60% 60 7.36 70 2.52 50% 80 0.00 40% 90 0.00 30% 100 0.00 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 12.82 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.5745 Minimum 0.0000 First Quartile 0.0179 Second Quartile 0.0991 Third Quartile 0.1747 Interquartile range 0.1568 Mean 0.1282 Standard Deviation 0.1538 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.2954 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0909 Std With No Outliers 0.0868 GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoENES1 Topic 026 2.12 Topic 039 16.72 0.8 Topic 027 2.04 Topic 040 54.13 Topic 028 24.55 Topic 041 0.86 0.6 Topic 029 0.98 Topic 042 20.06 Topic 030 13.11 Topic 043 0.00 Topic 031 1.02 Topic 044 29.54 0.4 Topic 032 19.71 Topic 045 2.21 Topic 033 0.01 Topic 046 11.07 0.2 Topic 034 15.87 Topic 047 6.94 Difference Topic 035 9.91 Topic 048 5.81 0 Topic 036 11.02 Topic 049 57.45 Topic 037 11.45 Topic 050 4.01 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 365 sanmarcos SMGeoENES1 GC-BILI-X2ES-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual Spanish track − Retrieved documents vs Precision 100% 5 docs 44.00 SMGeoENES1 10 docs 35.60 90% 15 docs 33.60 80% 20 docs 31.00 30 docs 28.53 70% 100 docs 17.28 60% 200 docs 11.28 R−Precision 500 docs 5.30 50% 1000 docs 2.92 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 16.89 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.6682 Minimum 0.0000 First Quartile 0.0291 Second Quartile 0.1429 Third Quartile 0.2302 Interquartile range 0.2010 Mean 0.1689 Standard Deviation 0.1763 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3786 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1279 Std With No Outliers 0.1091 GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoENES1 Topic 026 0.00 Topic 039 23.88 0.8 Topic 027 2.56 Topic 040 61.15 Topic 028 27.78 Topic 041 6.67 0.6 Topic 029 3.03 Topic 042 28.30 Topic 030 17.88 Topic 043 0.00 Topic 031 5.10 Topic 044 37.86 0.4 Topic 032 20.77 Topic 045 0.00 Topic 033 0.00 Topic 046 14.29 0.2 Topic 034 18.92 Topic 047 11.86 Difference Topic 035 15.79 Topic 048 7.55 0 Topic 036 22.73 Topic 049 66.82 Topic 037 17.24 Topic 050 12.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 366 sanmarcos SMGeoPTES2 GC-BILI-X2ES-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Portuguese Relevant 2,054 Topic Fields title, description Relevant retrieved 659 Pooled true Geometric Mean Average Precision 0.0221 Automatic Portuguese Spanish title+desc Binary Preference (BPREF) 0.1261 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual Spanish track − Interpolated Recall vs Average Precision 100% 0 52.67 SMGeoPTES2 10 26.69 90% 20 20.30 80% 30 14.71 40 11.08 70% 50 8.86 Average Precision 60% 60 6.92 70 1.65 50% 80 0.00 40% 90 0.00 30% 100 0.00 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 10.89 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.5335 Minimum 0.0000 First Quartile 0.0082 Second Quartile 0.0317 Third Quartile 0.2020 Interquartile range 0.1938 Mean 0.1089 Standard Deviation 0.1522 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4566 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0913 Std With No Outliers 0.1266 GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoPTES2 Topic 026 1.72 Topic 039 27.01 0.8 Topic 027 3.17 Topic 040 53.35 Topic 028 7.13 Topic 041 1.00 0.6 Topic 029 28.54 Topic 042 31.64 Topic 030 0.29 Topic 043 0.03 Topic 031 0.87 Topic 044 5.52 0.4 Topic 032 19.83 Topic 045 2.29 Topic 033 0.09 Topic 046 5.19 0.2 Topic 034 21.30 Topic 047 3.17 Difference Topic 035 2.99 Topic 048 5.65 0 Topic 036 0.68 Topic 049 45.66 Topic 037 0.05 Topic 050 5.21 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 367 sanmarcos SMGeoPTES2 GC-BILI-X2ES-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual Spanish track − Retrieved documents vs Precision 100% 5 docs 34.40 SMGeoPTES2 10 docs 30.40 90% 15 docs 28.53 80% 20 docs 25.20 30 docs 21.87 70% 100 docs 13.88 60% 200 docs 9.36 R−Precision 500 docs 4.75 50% 1000 docs 2.64 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 14.67 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.6043 Minimum 0.0000 First Quartile 0.0357 Second Quartile 0.0800 Third Quartile 0.2155 Interquartile range 0.1797 Mean 0.1467 Standard Deviation 0.1661 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3774 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1090 Std With No Outliers 0.1068 GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoPTES2 Topic 026 5.56 Topic 039 23.88 0.8 Topic 027 12.82 Topic 040 60.43 Topic 028 13.89 Topic 041 9.33 0.6 Topic 029 30.30 Topic 042 37.74 Topic 030 2.65 Topic 043 0.00 Topic 031 5.10 Topic 044 14.56 0.4 Topic 032 20.77 Topic 045 8.33 Topic 033 1.00 Topic 046 7.14 0.2 Topic 034 29.73 Topic 047 3.39 Difference Topic 035 5.26 Topic 048 7.55 0 Topic 036 3.64 Topic 049 55.76 Topic 037 0.00 Topic 050 8.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 368 sanmarcos SMGeoPTES3 GC-BILI-X2ES-CLEF2006 Overall statistics for 25 queries : Priority 3 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Portuguese Relevant 2,054 Topic Fields title, description, narrative Relevant retrieved 655 Pooled true Geometric Mean Average Precision 0.0230 Automatic Portuguese Spanish title-desc-narr Binary Preference (BPREF) 0.1310 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual Spanish track − Interpolated Recall vs Average Precision 100% 0 54.49 SMGeoPTES3 10 29.04 90% 20 22.69 80% 30 15.38 40 11.65 70% 50 9.36 Average Precision 60% 60 7.29 70 1.81 50% 80 0.00 40% 90 0.00 30% 100 0.00 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 11.50 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.5378 Minimum 0.0000 First Quartile 0.0092 Second Quartile 0.0401 Third Quartile 0.2080 Interquartile range 0.1987 Mean 0.1150 Standard Deviation 0.1604 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3255 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0794 Std With No Outliers 0.1070 GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoPTES3 Topic 026 1.78 Topic 039 27.15 0.8 Topic 027 2.67 Topic 040 53.78 Topic 028 8.07 Topic 041 0.99 0.6 Topic 029 32.55 Topic 042 31.54 Topic 030 0.28 Topic 043 0.03 Topic 031 1.43 Topic 044 5.84 0.4 Topic 032 20.01 Topic 045 2.15 Topic 033 0.06 Topic 046 5.76 0.2 Topic 034 23.16 Topic 047 3.37 Difference Topic 035 4.01 Topic 048 5.68 0 Topic 036 0.72 Topic 049 51.21 Topic 037 0.05 Topic 050 5.28 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 369 sanmarcos SMGeoPTES3 GC-BILI-X2ES-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual Spanish track − Retrieved documents vs Precision 100% 5 docs 33.60 SMGeoPTES3 10 docs 32.40 90% 15 docs 27.73 80% 20 docs 26.40 30 docs 22.53 70% 100 docs 14.76 60% 200 docs 9.62 R−Precision 500 docs 4.77 50% 1000 docs 2.62 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 15.27 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment Maximum 0.6043 Minimum 0.0000 First Quartile 0.0320 Second Quartile 0.0755 Third Quartile 0.2192 Interquartile range 0.1872 Mean 0.1527 Standard Deviation 0.1771 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4054 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1139 Std With No Outliers 0.1205 GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoPTES3 Topic 026 5.56 Topic 039 25.37 0.8 Topic 027 12.82 Topic 040 60.43 Topic 028 16.67 Topic 041 9.33 0.6 Topic 029 30.30 Topic 042 37.74 Topic 030 2.65 Topic 043 0.00 Topic 031 6.67 Topic 044 15.53 0.4 Topic 032 20.77 Topic 045 0.00 Topic 033 1.00 Topic 046 7.14 0.2 Topic 034 40.54 Topic 047 3.39 Difference Topic 035 5.26 Topic 048 7.55 0 Topic 036 3.64 Topic 049 59.45 Topic 037 0.00 Topic 050 10.00 −0.2 Topic 038 0.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 370 berkeley BKGeoEP1 GC-BILI-X2PT-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 1,060 Topic Fields title, description Relevant retrieved 591 Pooled true Geometric Mean Average Precision 0.0098 EN->PT using L&H query translation. title and Binary Preference (BPREF) 0.1291 description used Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual Portuguese track − Interpolated Recall vs Average Precision 100% 0 32.95 BKGeoEP1 10 26.74 90% 20 20.71 80% 30 17.10 40 13.29 70% 50 11.11 Average Precision 60% 60 9.49 70 7.84 50% 80 5.18 40% 90 3.64 30% 100 0.79 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 12.60 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.7932 Minimum 0.0000 First Quartile 0.0010 Second Quartile 0.0246 Third Quartile 0.1786 Interquartile range 0.1776 Mean 0.1260 Standard Deviation 0.1873 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.4374 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0982 Std With No Outliers 0.1283 GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries BKGeoEP1 Topic 026 1.22 Topic 039 5.45 0.8 Topic 027 0.03 Topic 040 0.01 Topic 028 12.27 Topic 041 0.01 0.6 Topic 029 27.03 Topic 042 10.65 Topic 030 0.81 Topic 043 2.46 Topic 031 43.74 Topic 044 0.00 0.4 Topic 032 14.19 Topic 045 27.39 Topic 033 0.00 Topic 046 33.97 0.2 Topic 034 0.14 Topic 047 0.13 Difference Topic 035 0.55 Topic 048 79.32 0 Topic 036 0.00 Topic 049 24.37 Topic 037 1.29 Topic 050 15.70 −0.2 Topic 038 14.35 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 371 berkeley BKGeoEP1 GC-BILI-X2PT-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual Portuguese track − Retrieved documents vs Precision 100% 5 docs 19.20 BKGeoEP1 10 docs 21.20 90% 15 docs 20.00 80% 20 docs 19.40 30 docs 17.47 70% 100 docs 12.04 60% 200 docs 8.22 R−Precision 500 docs 4.26 50% 1000 docs 2.36 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 14.77 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.7343 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0417 Third Quartile 0.2740 Interquartile range 0.2740 Mean 0.1477 Standard Deviation 0.1905 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.3934 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1233 Std With No Outliers 0.1493 GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries BKGeoEP1 Topic 026 0.00 Topic 039 8.70 0.8 Topic 027 0.00 Topic 040 0.00 Topic 028 21.88 Topic 041 0.00 0.6 Topic 029 33.33 Topic 042 20.00 Topic 030 0.00 Topic 043 4.17 Topic 031 39.34 Topic 044 0.00 0.4 Topic 032 26.42 Topic 045 36.59 Topic 033 0.00 Topic 046 37.88 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 73.43 0 Topic 036 0.00 Topic 049 27.78 Topic 037 0.00 Topic 050 27.27 −0.2 Topic 038 12.50 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 372 berkeley BKGeoEP2 GC-BILI-X2PT-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language English Relevant 1,060 Topic Fields title, description, narrative Relevant retrieved 630 Pooled true Geometric Mean Average Precision 0.0114 EN->PT using L&H query translation. title, desc and Binary Preference (BPREF) 0.1422 narrative Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual Portuguese track − Interpolated Recall vs Average Precision 100% 0 41.75 BKGeoEP2 10 27.68 90% 20 22.31 80% 30 19.80 40 16.82 70% 50 13.98 Average Precision 60% 60 11.01 70 8.17 50% 80 6.09 40% 90 3.68 30% 100 0.53 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 14.30 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.6974 Minimum 0.0000 First Quartile 0.0012 Second Quartile 0.0776 Third Quartile 0.2403 Interquartile range 0.2391 Mean 0.1430 Standard Deviation 0.1868 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5165 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.1199 Std With No Outliers 0.1500 GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries BKGeoEP2 Topic 026 1.27 Topic 039 8.98 0.8 Topic 027 8.97 Topic 040 0.01 Topic 028 24.05 Topic 041 0.00 0.6 Topic 029 33.34 Topic 042 43.84 Topic 030 0.69 Topic 043 6.26 Topic 031 21.61 Topic 044 0.00 0.4 Topic 032 7.76 Topic 045 24.53 Topic 033 0.00 Topic 046 51.65 0.2 Topic 034 0.15 Topic 047 2.96 Difference Topic 035 0.14 Topic 048 69.74 0 Topic 036 0.00 Topic 049 8.80 Topic 037 0.07 Topic 050 24.03 −0.2 Topic 038 18.63 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 373 berkeley BKGeoEP2 GC-BILI-X2PT-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual Portuguese track − Retrieved documents vs Precision 100% 5 docs 24.80 BKGeoEP2 10 docs 22.80 90% 15 docs 22.67 80% 20 docs 20.80 30 docs 19.73 70% 100 docs 13.36 60% 200 docs 9.14 R−Precision 500 docs 4.55 50% 1000 docs 2.52 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 16.34 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.6434 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0833 Third Quartile 0.3462 Interquartile range 0.3462 Mean 0.1634 Standard Deviation 0.1963 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.6434 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1634 Std With No Outliers 0.1963 GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries BKGeoEP2 Topic 026 0.00 Topic 039 13.04 0.8 Topic 027 20.59 Topic 040 0.00 Topic 028 34.38 Topic 041 0.00 0.6 Topic 029 38.46 Topic 042 42.86 Topic 030 0.00 Topic 043 8.33 Topic 031 21.31 Topic 044 0.00 0.4 Topic 032 20.75 Topic 045 35.37 Topic 033 0.00 Topic 046 54.55 0.2 Topic 034 0.00 Topic 047 0.00 Difference Topic 035 0.00 Topic 048 64.34 0 Topic 036 0.00 Topic 049 5.56 Topic 037 0.00 Topic 050 36.36 −0.2 Topic 038 12.50 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 374 sanmarcos SMGeoESPT1 GC-BILI-X2PT-CLEF2006 Overall statistics for 25 queries : Priority 1 Total number of documents over all queries Query Construction MANUAL Retrieved 25,000 Source Language Spanish; Castilian Relevant 1,060 Topic Fields title, description Relevant retrieved 608 Pooled true Geometric Mean Average Precision 0.0511 Spanish-Portuguese Binary Preference (BPREF) 0.1174 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual Portuguese track − Interpolated Recall vs Average Precision 100% 0 45.68 SMGeoESPT1 10 27.39 90% 20 23.53 80% 30 17.21 40 13.62 70% 50 10.70 Average Precision 60% 60 8.25 70 5.52 50% 80 4.28 40% 90 2.27 30% 100 0.27 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 12.81 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.4984 Minimum 0.0011 First Quartile 0.0203 Second Quartile 0.0697 Third Quartile 0.1727 Interquartile range 0.1524 Mean 0.1281 Standard Deviation 0.1540 Lower Outlier Threshold 0.0011 Upper Outlier Threshold 0.3866 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0960 Std With No Outliers 0.1114 GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment 15 Number of Topics of the Experiment 10 5 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoESPT1 Topic 026 1.17 Topic 039 30.28 0.8 Topic 027 6.97 Topic 040 1.41 Topic 028 8.54 Topic 041 1.15 0.6 Topic 029 6.86 Topic 042 49.61 Topic 030 38.66 Topic 043 0.11 Topic 031 14.38 Topic 044 7.75 0.4 Topic 032 49.84 Topic 045 2.95 Topic 033 1.16 Topic 046 25.94 0.2 Topic 034 2.24 Topic 047 8.84 Difference Topic 035 2.47 Topic 048 2.29 0 Topic 036 2.88 Topic 049 14.11 Topic 037 0.14 Topic 050 10.34 −0.2 Topic 038 30.10 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 375 sanmarcos SMGeoESPT1 GC-BILI-X2PT-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual Portuguese track − Retrieved documents vs Precision 100% 5 docs 23.20 SMGeoESPT1 10 docs 21.60 90% 15 docs 18.67 80% 20 docs 17.40 30 docs 16.00 70% 100 docs 9.88 60% 200 docs 6.52 R−Precision 500 docs 3.97 50% 1000 docs 2.43 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 14.88 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.5849 Minimum 0.0000 First Quartile 0.0000 Second Quartile 0.0979 Third Quartile 0.2527 Interquartile range 0.2527 Mean 0.1488 Standard Deviation 0.1652 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5849 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1488 Std With No Outliers 0.1652 GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoESPT1 Topic 026 0.00 Topic 039 26.09 0.8 Topic 027 12.75 Topic 040 4.17 Topic 028 15.62 Topic 041 2.88 0.6 Topic 029 7.69 Topic 042 48.57 Topic 030 42.86 Topic 043 0.00 Topic 031 13.11 Topic 044 10.53 0.4 Topic 032 58.49 Topic 045 8.54 Topic 033 0.00 Topic 046 33.33 0.2 Topic 034 0.00 Topic 047 8.82 Difference Topic 035 0.00 Topic 048 9.79 0 Topic 036 0.00 Topic 049 27.78 Topic 037 0.00 Topic 050 15.91 −0.2 Topic 038 25.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 376 sanmarcos SMGeoESPT2 GC-BILI-X2PT-CLEF2006 Overall statistics for 25 queries : Priority 2 Total number of documents over all queries Query Construction AUTOMATIC Retrieved 25,000 Source Language Spanish; Castilian Relevant 1,060 Topic Fields title, description Relevant retrieved 655 Pooled true Geometric Mean Average Precision 0.0497 Automatic Spanish Portuguese title + desc Binary Preference (BPREF) 0.1415 Interploated Recall (%) Precision Averages (%) GeoCLEF Bilingual Portuguese track − Interpolated Recall vs Average Precision 100% 0 45.27 SMGeoESPT2 10 27.06 90% 20 23.20 80% 30 19.45 40 16.32 70% 50 14.00 Average Precision 60% 60 11.33 70 8.11 50% 80 5.93 40% 90 3.39 30% 100 0.63 Average precision (non-interpolated) for all 20% relevant documents (averaged over queries) 14.16 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Interpolated Recall Mean Average Precision GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.7579 Minimum 0.0002 First Quartile 0.0187 Second Quartile 0.0797 Third Quartile 0.1811 Interquartile range 0.1624 Mean 0.1416 Standard Deviation 0.1830 Lower Outlier Threshold 0.0002 Upper Outlier Threshold 0.3997 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Mean With No Outliers 0.0983 Std With No Outliers 0.1041 GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment 10 Number of Topics of the Experiment 8 6 4 2 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Mean Average Precision Precision averages (%) for individual 1 GeoCLEF Bilingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050) queries SMGeoESPT2 Topic 026 7.79 Topic 039 8.25 0.8 Topic 027 0.16 Topic 040 2.84 Topic 028 13.15 Topic 041 1.12 0.6 Topic 029 23.65 Topic 042 25.88 Topic 030 1.64 Topic 043 0.02 Topic 031 16.14 Topic 044 7.97 0.4 Topic 032 75.79 Topic 045 23.82 Topic 033 0.17 Topic 046 9.22 0.2 Topic 034 3.21 Topic 047 5.42 Difference Topic 035 2.76 Topic 048 51.93 0 Topic 036 1.92 Topic 049 39.97 Topic 037 1.72 Topic 050 13.12 −0.2 Topic 038 16.26 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 377 sanmarcos SMGeoESPT2 GC-BILI-X2PT-CLEF2006 Docs Cutoff Levels Precision at DCL (%) GeoCLEF Bilingual Portuguese track − Retrieved documents vs Precision 100% 5 docs 21.60 SMGeoESPT2 10 docs 23.60 90% 15 docs 21.60 80% 20 docs 20.60 30 docs 19.60 70% 100 docs 12.76 60% 200 docs 8.46 R−Precision 500 docs 4.48 50% 1000 docs 2.62 40% R-Precision (precision after R document retrieved, where R = Relevant retrieved) 30% 17.42 20% 10% 0% 5 10 15 20 30 100 200 500 1000 Retrieved Documents (logarithmic scale) Exact R-Precision GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment Maximum 0.7547 Minimum 0.0000 First Quartile 0.0366 Second Quartile 0.1250 Third Quartile 0.2644 Interquartile range 0.2278 Mean 0.1742 Standard Deviation 0.1901 Lower Outlier Threshold 0.0000 Upper Outlier Threshold 0.5315 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Mean With No Outliers 0.1500 Std With No Outliers 0.1498 GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment 8 Number of Topics of the Experiment 7 6 5 4 3 2 1 0 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Exact R−Precision Precision averages (%) for individual 1 GeoCLEF Bilingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050) queries SMGeoESPT2 Topic 026 6.67 Topic 039 13.04 0.8 Topic 027 3.92 Topic 040 12.50 Topic 028 21.88 Topic 041 2.88 0.6 Topic 029 30.77 Topic 042 31.43 Topic 030 7.14 Topic 043 0.00 Topic 031 16.39 Topic 044 9.21 0.4 Topic 032 75.47 Topic 045 35.37 Topic 033 0.00 Topic 046 16.67 0.2 Topic 034 0.00 Topic 047 5.88 Difference Topic 035 0.00 Topic 048 53.15 0 Topic 036 0.00 Topic 049 44.44 Topic 037 5.56 Topic 050 18.18 −0.2 Topic 038 25.00 −0.4 −0.6 −0.8 −1 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 Topic Identifier 378