Appendix D
Results of the GeoCLEF Track
Prepared by:
Giorgio Maria Di Nunzio and Nicola Ferro
{dinunzio, ferro}@dei.unipd.it
Department of Information Engineering
University of Padua
Italy
1
2
Introduction
3
4
Results for CLEF 2006 GeoCLEF Tracks
The following pages contain the results and graphs for all the experiments that have been officially submitted to the CLEF 2006
campaign for the GeoCLEF track.
This document is divided in three main parts:
1. List of submitted experiments
2. Track overview results and graphs
3. Individual experiment results and graphs
1. List of Submitted Experiments
This section gives a listing of all experiments and their characteristics:
Participant: the name of the participant who submitted the experiment.
Country: country of the participant.
Identifier: unique identifier for each experiment.
Task: track/task to which the experiment belongs.
Topic language: language of the topics used to create the experiment (ISO identifiers for language).
Topic fields: identifies the parts of the topics used to create the experiment (T = title, D = Description, N = Narrative).
Query constr.: identifies how the query has been constructed from topic fields (manual/automatic).
Pool: specifies if experiment was used for relevance assessment pooling.
2. Track Overview Results and Graphs
For each track/task graphs and tables are shown in order to compare the experiments.
The graphs and tables contain the following information:
- Mandatory experiments title + description (TD) of at most top five participants
- Interpolated recall vs precision averages plot
- Average precision comparison to median plot
- All experiments
- Average precision box plot
- Average precision Tukey t-test plot
- Mandatory experiments title + description (TD) of at most top five participants
- Document cutoff levels (DCL) vs precision at DCL plot
- R-Precision comparison to median plot
- All experiments
- R-Precision box plot
- R-Precision Tukey t-test plot
- A table with descriptive statistics of performance figures for each topic
3. Individual Experiment Results and Graphs
This section provides the individual results for each official experiment. For each experiment the following tables and graphs are
shown:
- Overall statistics and information
- Interpolated recall vs precision averages plot
- Average precision statistics and box plot
- Average precision comparison to median plot
- Document cutoff levels vs precision at DCL plot
- R-Precision statistics and box plot
- R-Precision comparison to median plot
5
6
List of Submitted Experiments
7
8
Participant Country Experiment ID Task Topic Topic Query Pool
Lang. Fields Construction
berkeley United States BKGeoD2 GC-MONO-DE-CLEF2006 de TDN AUTOMATIC yes
berkeley United States BKGeoD1 GC-MONO-DE-CLEF2006 de TD AUTOMATIC yes
daedalus Spain GCdeNtLg GC-MONO-DE-CLEF2006 de TD AUTOMATIC yes
daedalus Spain GCdeAA GC-MONO-DE-CLEF2006 de TDN MANUAL yes
daedalus Spain GCdeAtLg GC-MONO-DE-CLEF2006 de TDN MANUAL yes
daedalus Spain GCdeNA GC-MONO-DE-CLEF2006 de TD MANUAL yes
daedalus Spain GCdeAO GC-MONO-DE-CLEF2006 de TDN MANUAL yes
hagen Germany FUHddGYYYTD GC-MONO-DE-CLEF2006 de TD AUTOMATIC yes
hagen Germany FUHddGNNNTD GC-MONO-DE-CLEF2006 de TD AUTOMATIC yes
hagen Germany FUHddGNNNTDN GC-MONO-DE-CLEF2006 de TDN AUTOMATIC yes
hagen Germany FUHddGYYYTDN GC-MONO-DE-CLEF2006 de TDN AUTOMATIC yes
hagen Germany FUHddGYYYMTDN GC-MONO-DE-CLEF2006 de TDN AUTOMATIC yes
hildesheim Germany HIGeodederun4n GC-MONO-DE-CLEF2006 de TDN AUTOMATIC yes
hildesheim Germany HIGeodederun4 GC-MONO-DE-CLEF2006 de TD AUTOMATIC yes
hildesheim Germany HIGeodederun6 GC-MONO-DE-CLEF2006 de TD AUTOMATIC yes
hildesheim Germany HIGeodederun6n GC-MONO-DE-CLEF2006 de TDN AUTOMATIC yes
alicante Spain enTD GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes
alicante Spain enTDN GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no
alicante Spain enTDNGeoNames GC-MONO-EN-CLEF2006 en TDN MANUAL yes
alicante Spain UAUJAUPVenenExp1 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no
berkeley United States BKGeoE4 GC-MONO-EN-CLEF2006 en TDN MANUAL yes
berkeley United States BKGeoE2 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no
berkeley United States BKGeoE3 GC-MONO-EN-CLEF2006 en TDN MANUAL no
berkeley United States BKGeoE1 GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes
daedalus Spain GCenAtLg GC-MONO-EN-CLEF2006 en TDN MANUAL no
daedalus Spain GCenNtLg GC-MONO-EN-CLEF2006 en TD MANUAL no
daedalus Spain GCenNA GC-MONO-EN-CLEF2006 en TD MANUAL yes
daedalus Spain GCenAA GC-MONO-EN-CLEF2006 en TDN MANUAL yes
daedalus Spain GCenAO GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no
hildesheim Germany HIGeoenenrun1n GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no
hildesheim Germany HIGeoenenrun2n GC-MONO-EN-CLEF2006 en TDN AUTOMATIC yes
hildesheim Germany HIGeoenenrun3 GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes
hildesheim Germany HIGeoenenrun1 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no
hildesheim Germany HIGeoenenrun2 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no
imp-coll United ICgeoMLtdn GC-MONO-EN-CLEF2006 en TDN MANUAL yes
Kingdom
imp-coll United ICgeoMLtd GC-MONO-EN-CLEF2006 en TD MANUAL yes
Kingdom
jaen Spain sinaiEnEnExp3 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no
jaen Spain sinaiEnEnExp1 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC yes
jaen Spain sinaiEnEnExp2 GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes
jaen Spain sinaiEnEnExp4 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no
jaen Spain sinaiEnEnExp5 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no
ms-china China msramanual GC-MONO-EN-CLEF2006 en TD MANUAL yes
ms-china China msrawhitelist GC-MONO-EN-CLEF2006 en T AUTOMATIC yes
ms-china China msraexpansion GC-MONO-EN-CLEF2006 en TD AUTOMATIC no
ms-china China msralocal GC-MONO-EN-CLEF2006 en T AUTOMATIC no
ms-china China msratext GC-MONO-EN-CLEF2006 en TDN AUTOMATIC yes
nicta Australia MuTdnManQexpGeo GC-MONO-EN-CLEF2006 en TDN MANUAL no
nicta Australia MuTdnTxt GC-MONO-EN-CLEF2006 en TDN AUTOMATIC yes
nicta Australia MuTdQexpPrb GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes
nicta Australia MuTdRedn GC-MONO-EN-CLEF2006 en TD AUTOMATIC no
nicta Australia MuTdTxt GC-MONO-EN-CLEF2006 en TD AUTOMATIC no
rfia-upv Spain rfiaUPV01 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no
rfia-upv Spain rfiaUPV02 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no
rfia-upv Spain rfiaUPV03 GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes
rfia-upv Spain rfiaUPV04 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC yes
sanmarcos United States SMGeoEN4 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no
9
Participant Country Experiment ID Task Topic Topic Query Pool
Lang. Fields Construction
sanmarcos United States SMGeoEN5 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no
sanmarcos United States SMGeoEN1 GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes
sanmarcos United States SMGeoEN3 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no
sanmarcos United States SMGeoEN5 GC-MONO-EN-CLEF2006 en TDN MANUAL yes
talp Spain TALPGeoIRTDN2 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no
talp Spain TALPGeoIRTD1 GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes
talp Spain TALPGeoIRTDN1 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC yes
talp Spain TALPGeoIRTD2 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no
talp Spain TALPGeoIRTDN3 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no
u.buffalo United States UBGTDrf1 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no
u.buffalo United States UBGTDrf2 GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes
u.buffalo United States UBManual2 GC-MONO-EN-CLEF2006 en TDN MANUAL no
u.buffalo United States UBGManual1 GC-MONO-EN-CLEF2006 en TDN MANUAL yes
u.groningen Netherlands CLCGGeoEE1 GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes
u.groningen Netherlands CLCGGeoEE2 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC yes
u.groningen Netherlands CLCGGeoEE5 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no
u.groningen Netherlands CLCGGeoEE10 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no
u.groningen Netherlands CLCGGeoEE11 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no
u.twente Netherlands utGeoTIB GC-MONO-EN-CLEF2006 en T MANUAL yes
u.twente Netherlands utGeoTdIB GC-MONO-EN-CLEF2006 en TD MANUAL yes
u.twente Netherlands utGeoTIBm GC-MONO-EN-CLEF2006 en TD AUTOMATIC no
u.twente Netherlands utGeoTdnIB GC-MONO-EN-CLEF2006 en TDN MANUAL yes
u.twente Netherlands utGeoTdnIBm GC-MONO-EN-CLEF2006 en TDN MANUAL no
unsw Australia unswTitleBaseline GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes
unsw Australia unswNarrBaseline GC-MONO-EN-CLEF2006 en TDN AUTOMATIC yes
unsw Australia unswNarrMap GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no
unsw Australia unswTitleF46 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no
unsw Australia unswNarrF41 GC-MONO-EN-CLEF2006 en TDN AUTOMATIC no
xldb Portugal XLDBGeoENAut02 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no
xldb Portugal XLDBGeoENAut05 GC-MONO-EN-CLEF2006 en TD AUTOMATIC no
xldb Portugal XLDBGeoManualEN GC-MONO-EN-CLEF2006 en TD MANUAL no
xldb Portugal XLDBGeoENAut03_2 GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes
xldb Portugal XLDBGeoENAut03 GC-MONO-EN-CLEF2006 en TD AUTOMATIC yes
alicante Spain esTD GC-MONO-ES-CLEF2006 es TD AUTOMATIC yes
alicante Spain esTDN GC-MONO-ES-CLEF2006 es TD AUTOMATIC yes
alicante Spain esTDNGeoNames GC-MONO-ES-CLEF2006 es TDN MANUAL yes
berkeley United States BKGeoS1 GC-MONO-ES-CLEF2006 es TD AUTOMATIC yes
berkeley United States BKGeoS2 GC-MONO-ES-CLEF2006 es TDN AUTOMATIC yes
daedalus Spain GCesNA GC-MONO-ES-CLEF2006 es TD MANUAL yes
daedalus Spain GCesAtLg GC-MONO-ES-CLEF2006 es TDN MANUAL yes
daedalus Spain GCesAO GC-MONO-ES-CLEF2006 es TDN MANUAL yes
daedalus Spain GCesAA GC-MONO-ES-CLEF2006 es TDN MANUAL yes
daedalus Spain GCesNtLg GC-MONO-ES-CLEF2006 es TD MANUAL yes
sanmarcos United States SMGeoES4 GC-MONO-ES-CLEF2006 es TD AUTOMATIC yes
sanmarcos United States SMGeoES5 GC-MONO-ES-CLEF2006 es TDN AUTOMATIC yes
sanmarcos United States SMGeoES1 GC-MONO-ES-CLEF2006 es TD AUTOMATIC yes
sanmarcos United States SMGeoES2 GC-MONO-ES-CLEF2006 es TDN AUTOMATIC yes
sanmarcos United States SMGeoES3 GC-MONO-ES-CLEF2006 es TD MANUAL yes
berkeley United States BKGeoP2 GC-MONO-PT-CLEF2006 pt TDN AUTOMATIC yes
berkeley United States BKGeoP1 GC-MONO-PT-CLEF2006 pt TD AUTOMATIC yes
berkeley United States BKGeoP4 GC-MONO-PT-CLEF2006 pt TDN AUTOMATIC yes
berkeley United States BKGeoP3 GC-MONO-PT-CLEF2006 pt TD AUTOMATIC yes
sanmarcos United States SMGeoPT4 GC-MONO-PT-CLEF2006 pt TD AUTOMATIC yes
sanmarcos United States SMGeoPT2 GC-MONO-PT-CLEF2006 pt TD AUTOMATIC yes
sanmarcos United States SMGeoPT1 GC-MONO-PT-CLEF2006 pt TDN AUTOMATIC yes
sanmarcos United States SMGeoPT3 GC-MONO-PT-CLEF2006 pt TDN AUTOMATIC yes
10
Participant Country Experiment ID Task Topic Topic Query Pool
Lang. Fields Construction
xldb Portugal XLDBGeoPTAut02 GC-MONO-PT-CLEF2006 pt TD MANUAL yes
xldb Portugal XLDBGeoPTAut05 GC-MONO-PT-CLEF2006 pt TD AUTOMATIC yes
xldb Portugal XLDBGeoManualPT GC-MONO-PT-CLEF2006 pt TD MANUAL yes
xldb Portugal XLDBGeoPTAut03 GC-MONO-PT-CLEF2006 pt TD AUTOMATIC yes
xldb Portugal XLDBGeoPTAut03_2 GC-MONO-PT-CLEF2006 pt TD AUTOMATIC yes
berkeley United States BKGeoED2 GC-BILI-X2DE-CLEF2006 en TDN MANUAL yes
berkeley United States BKGeoED1 GC-BILI-X2DE-CLEF2006 en TD AUTOMATIC yes
hagen Germany FUHedGNNNTDN GC-BILI-X2DE-CLEF2006 en TDN AUTOMATIC yes
hagen Germany FUHedGYYYTDN GC-BILI-X2DE-CLEF2006 en TDN AUTOMATIC yes
hagen Germany FUHedGNNNTD GC-BILI-X2DE-CLEF2006 en TD AUTOMATIC yes
hagen Germany FUHedGYYYTD GC-BILI-X2DE-CLEF2006 en TD AUTOMATIC yes
hagen Germany FUHedGYYYMTDN GC-BILI-X2DE-CLEF2006 en TDN AUTOMATIC yes
hildesheim Germany HIGeoenderun21 GC-BILI-X2DE-CLEF2006 en TD AUTOMATIC yes
hildesheim Germany HIGeoenderun22 GC-BILI-X2DE-CLEF2006 en TD AUTOMATIC yes
hildesheim Germany HIGeoenderun21n GC-BILI-X2DE-CLEF2006 en TDN AUTOMATIC yes
hildesheim Germany HIGeoenderun22n GC-BILI-X2DE-CLEF2006 en TDN AUTOMATIC yes
hildesheim Germany HIGeodeenrun12 GC-BILI-X2EN-CLEF2006 de TD AUTOMATIC yes
hildesheim Germany HIGeodeenrun13n GC-BILI-X2EN-CLEF2006 de TDN AUTOMATIC yes
hildesheim Germany HIGeodeenrun11n GC-BILI-X2EN-CLEF2006 de TDN AUTOMATIC no
hildesheim Germany HIGeodeenrun11 GC-BILI-X2EN-CLEF2006 de TD AUTOMATIC no
hildesheim Germany HIGeodeenrun13 GC-BILI-X2EN-CLEF2006 de TD AUTOMATIC no
jaen Spain sinaiEsEnExp1 GC-BILI-X2EN-CLEF2006 es TDN AUTOMATIC yes
jaen Spain sinaiDeEnExp2 GC-BILI-X2EN-CLEF2006 de TD AUTOMATIC no
jaen Spain sinaiEsEnExp3 GC-BILI-X2EN-CLEF2006 es TD AUTOMATIC no
jaen Spain sinaiDeEnExp1 GC-BILI-X2EN-CLEF2006 de TDN AUTOMATIC no
jaen Spain sinaiEsEnExp2 GC-BILI-X2EN-CLEF2006 es TD AUTOMATIC yes
sanmarcos United States SMGeoESEN1 GC-BILI-X2EN-CLEF2006 es TDN MANUAL yes
sanmarcos United States SMGeoESEN2 GC-BILI-X2EN-CLEF2006 es TD AUTOMATIC yes
berkeley United States BKGeoES1 GC-BILI-X2ES-CLEF2006 en TD AUTOMATIC yes
berkeley United States BKGeoES2 GC-BILI-X2ES-CLEF2006 en TDN AUTOMATIC yes
sanmarcos United States SMGeoENES1 GC-BILI-X2ES-CLEF2006 en TD AUTOMATIC yes
sanmarcos United States SMGeoPTES2 GC-BILI-X2ES-CLEF2006 pt TD AUTOMATIC yes
sanmarcos United States SMGeoPTES3 GC-BILI-X2ES-CLEF2006 pt TDN AUTOMATIC yes
berkeley United States BKGeoEP1 GC-BILI-X2PT-CLEF2006 en TD AUTOMATIC yes
berkeley United States BKGeoEP2 GC-BILI-X2PT-CLEF2006 en TDN AUTOMATIC yes
sanmarcos United States SMGeoESPT1 GC-BILI-X2PT-CLEF2006 es TD MANUAL yes
sanmarcos United States SMGeoESPT2 GC-BILI-X2PT-CLEF2006 es TD AUTOMATIC yes
11
12
Track Overview Results and Graphs
13
14
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-DE-CLEF2006
GeoCLEF Monolingual German track Top 5 Participants − Interpolated Recall vs Average Precision
100%
hagen [FUHddGYYYTD; MAP 22.29%; Pooled]
berkeley [BKGeoD1; MAP 21.51%; Pooled]
90% hildesheim [HIGeodederun4; MAP 15.58%; Pooled]
daedalus [GCdeNtLg; MAP 10.01%; Pooled]
80%
70%
Average Precision
60%
50%
40%
30%
20%
10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
GeoCLEF Monolingual German track Top 5 Participants − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050
)
1
hagen [FUHddGYYYTD; MAP 22.29%; Pooled]
berkeley [BKGeoD1; MAP 21.51%; Pooled]
hildesheim [HIGeodederun4; MAP 15.58%; Pooled]
daedalus [GCdeNtLg; MAP 10.01%; Pooled]
0.8
0.6
0.4
0.2
Difference
0
−0.2
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
15
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-DE-CLEF2006
GeoCLEF Monolingual German track − Box Plot of the Topics
FUHddGYYYTD [MAP 22.29%; Pooled]
BKGeoD1 [MAP 21.51%; Pooled]
FUHddGYYYTDN [MAP 21.41%; Pooled]
FUHddGYYYMTDN [MAP 19.99%; Pooled]
BKGeoD2 [MAP 18.22%; Pooled]
FUHddGNNNTD [MAP 16.94%; Pooled]
HIGeodederun4n [MAP 16.01%; Pooled]
Experiments
HIGeodederun4 [MAP 15.58%; Pooled]
FUHddGNNNTDN [MAP 12.23%; Pooled]
HIGeodederun6 [MAP 12.14%; Pooled]
HIGeodederun6n [MAP 11.34%; Pooled]
GCdeNtLg [MAP 10.01%; Pooled]
GCdeNA [MAP 9.28%; Pooled]
GCdeAtLg [MAP 7.36%; Pooled]
GCdeAA [MAP 7.15%; Pooled]
GCdeAO [MAP 5.48%; Pooled]
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Mean Average Precision
16
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-DE-CLEF2006
GeoCLEF Monolingual German track − Tukey T test with "top group" highlighted
BKGeoD1
FUHddGYYYTD
FUHddGYYYTDN
FUHddGYYYMTDN
BKGeoD2
FUHddGNNNTD
HIGeodederun4n
Experiments
HIGeodederun4
FUHddGNNNTDN
GCdeNtLg
HIGeodederun6
HIGeodederun6n
GCdeNA
GCdeAtLg
GCdeAA
GCdeAO
0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
arcsin(sqrt(Mean average precision))
17
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-DE-CLEF2006
GeoCLEF Monolingual German track Top 5 Participants − Retrieved documents vs Precision
100%
hagen [FUHddGYYYTD; R−Prec 21.53%; Pooled]
berkeley [BKGeoD1; R−Prec 19.99%; Pooled]
90% hildesheim [HIGeodederun4; R−Prec 18.15%; Pooled]
daedalus [GCdeNtLg; R−Prec 11.53%; Pooled]
80%
70%
60%
R−Precision
50%
40%
30%
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
GeoCLEF Monolingual German track Top 5 Participants − Comparison to Median R−Precision by Topic (Topics 026 to 050)
1
hagen [FUHddGYYYTD; R−Prec 21.53%; Pooled]
berkeley [BKGeoD1; R−Prec 19.99%; Pooled]
hildesheim [HIGeodederun4; R−Prec 18.15%; Pooled]
daedalus [GCdeNtLg; R−Prec 11.53%; Pooled]
0.8
0.6
0.4
0.2
Difference
0
−0.2
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
18
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-DE-CLEF2006
GeoCLEF Monolingual German track − Box Plot of the Topics
FUHddGYYYTD [R−Prec 21.53%; Pooled]
FUHddGYYYTDN [R−Prec 20.56%; Pooled]
FUHddGYYYMTDN [R−Prec 20.39%; Pooled]
BKGeoD1 [R−Prec 19.99%; Pooled]
BKGeoD2 [R−Prec 18.57%; Pooled]
HIGeodederun4 [R−Prec 18.15%; Pooled]
FUHddGNNNTD [R−Prec 18.00%; Pooled]
Experiments
HIGeodederun4n [R−Prec 17.68%; Pooled]
HIGeodederun6n [R−Prec 13.72%; Pooled]
HIGeodederun6 [R−Prec 13.45%; Pooled]
FUHddGNNNTDN [R−Prec 13.40%; Pooled]
GCdeNtLg [R−Prec 11.53%; Pooled]
GCdeNA [R−Prec 10.19%; Pooled]
GCdeAA [R−Prec 8.93%; Pooled]
GCdeAtLg [R−Prec 8.78%; Pooled]
GCdeAO [R−Prec 6.62%; Pooled]
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
R−Precision
19
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-DE-CLEF2006
GeoCLEF Monolingual German track − Tukey T test with "top group" highlighted
FUHddGYYYTD
FUHddGYYYMTDN
FUHddGYYYTDN
HIGeodederun4n
FUHddGNNNTD
HIGeodederun4
BKGeoD1
Experiments
BKGeoD2
FUHddGNNNTDN
HIGeodederun6n
HIGeodederun6
GCdeNtLg
GCdeNA
GCdeAtLg
GCdeAA
GCdeAO
0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
arcsin(sqrt(R Precision))
20
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-DE-CLEF2006
Average Precision R-Precision
Topic Minimum 1st Q. Median 3rd Q. Maximum Mean Std Minimum 1st Q. Median 3rd Q. Maximum Mean Std
026 0.0000 0.0004 0.0012 0.0040 0.0172 0.0034 0.0050 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
027 0.0000 0.0020 0.0135 0.0209 0.0359 0.0132 0.0106 0.0000 0.0231 0.0692 0.1154 0.1385 0.0683 0.0512
028 0.0000 0.0084 0.0745 0.2362 0.3916 0.1238 0.1293 0.0000 0.0156 0.1562 0.3438 0.4375 0.1855 0.1736
029 0.0011 0.0193 0.0716 0.3494 0.4667 0.1629 0.1739 0.0000 0.0000 0.0000 0.3333 0.3333 0.1458 0.1708
030 0.1256 0.4398 0.5873 0.6712 0.7862 0.5300 0.1906 0.1000 0.5000 0.6000 0.6333 0.7667 0.5354 0.1832
031 0.0023 0.0230 0.0507 0.1284 0.8249 0.1912 0.3124 0.0000 0.0329 0.0658 0.2171 0.8158 0.2097 0.2943
032 0.0023 0.2005 0.6070 0.6648 0.7861 0.4625 0.2885 0.0185 0.2407 0.5833 0.6667 0.8148 0.4711 0.2834
033 0.0018 0.0221 0.0573 0.0909 0.1176 0.0560 0.0413 0.0000 0.0000 0.0588 0.1176 0.1765 0.0772 0.0670
034 0.0000 0.0738 0.4100 0.4744 0.6704 0.3121 0.2253 0.0000 0.0882 0.4265 0.4706 0.6176 0.3143 0.2239
035 0.0000 0.0048 0.0305 0.0675 0.1231 0.0377 0.0349 0.0000 0.0000 0.0278 0.0556 0.1667 0.0417 0.0517
036 0.0000 0.0018 0.0079 0.0495 0.4167 0.0873 0.1627 0.0000 0.0000 0.0000 0.0000 0.3333 0.0625 0.1344
037 0.0000 0.0001 0.0065 0.0878 0.2303 0.0531 0.0851 0.0000 0.0000 0.0000 0.0909 0.2727 0.0511 0.0876
038 0.0000 0.0072 0.0221 0.0962 0.2381 0.0635 0.0765 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
039 0.0004 0.0133 0.0690 0.2032 0.3491 0.1119 0.1171 0.0000 0.0370 0.1296 0.2037 0.4074 0.1319 0.1194
040 0.0000 0.1421 0.2841 0.3259 0.4016 0.2285 0.1237 0.0000 0.2073 0.3659 0.4268 0.4878 0.3110 0.1507
041 0.0000 0.0091 0.0180 0.0491 0.2184 0.0441 0.0649 0.0000 0.0000 0.0526 0.1053 0.2105 0.0592 0.0716
042 0.0062 0.0167 0.0409 0.0706 0.1273 0.0468 0.0339 0.0000 0.0319 0.0532 0.1383 0.2128 0.0785 0.0635
043 0.0016 0.0090 0.0178 0.0329 0.0617 0.0239 0.0194 0.0000 0.0000 0.0357 0.0714 0.1429 0.0446 0.0513
044 0.0023 0.0079 0.0132 0.0233 0.3340 0.0463 0.0902 0.0000 0.0000 0.0000 0.0000 0.3333 0.0417 0.0962
045 0.0000 0.0003 0.0166 0.0635 0.6250 0.0652 0.1528 0.0000 0.0000 0.0000 0.0000 0.5000 0.0312 0.1250
046 0.0000 0.0000 0.1612 0.2803 0.5108 0.1782 0.1561 0.0000 0.0000 0.2500 0.2500 0.5000 0.1875 0.1443
047 0.0000 0.0060 0.0299 0.0622 0.3911 0.0515 0.0946 0.0000 0.0000 0.0000 0.0833 0.5000 0.0625 0.1344
048 0.0950 0.2137 0.5931 0.8834 0.9161 0.5646 0.3107 0.1594 0.2464 0.6232 0.8406 0.8841 0.5634 0.2775
049 0.0000 0.0095 0.0432 0.0807 0.1763 0.0529 0.0546 0.0000 0.0000 0.0000 0.0833 0.1667 0.0417 0.0745
050 0.0000 0.0053 0.0373 0.0599 0.0755 0.0352 0.0280 0.0000 0.0000 0.0833 0.0833 0.1667 0.0573 0.0587
ALL 0.0548 0.0965 0.1391 0.1910 0.2229 0.1418 0.0556 0.0662 0.1086 0.1570 0.1928 0.2153 0.1509 0.0486
21
22
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-EN-CLEF2006
GeoCLEF Monolingual English track Top 5 Participants − Interpolated Recall vs Average Precision
100%
xldb [XLDBGeoManualEN; MAP 30.34%; Not Pooled]
alicante [enTD; MAP 27.23%; Pooled]
90% sanmarcos [SMGeoEN4; MAP 26.37%; Not Pooled]
unsw [unswTitleBaseline; MAP 26.22%; Pooled]
jaen [sinaiEnEnExp4; MAP 26.11%; Not Pooled]
80%
70%
Average Precision
60%
50%
40%
30%
20%
10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
GeoCLEF Monolingual English track Top 5 Participants − Comparison to Median Mean Average Precision by Topic (Topics 026 to 05
0)
1
xldb [XLDBGeoManualEN; MAP 30.34%; Not Pooled]
alicante [enTD; MAP 27.23%; Pooled]
sanmarcos [SMGeoEN4; MAP 26.37%; Not Pooled]
unsw [unswTitleBaseline; MAP 26.22%; Pooled]
0.8 jaen [sinaiEnEnExp4; MAP 26.11%; Not Pooled]
0.6
0.4
0.2
Difference
0
−0.2
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
23
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-EN-CLEF2006
GeoCLEF Monolingual English track − Box Plot of the Topics
sinaiEnEnExp1 [MAP 32.24%; Pooled]
XLDBGeoManualEN [MAP 30.34%; Not Pooled]
enTDN [MAP 29.85%; Not Pooled]
BKGeoE4 [MAP 28.87%; Pooled]
SMGeoEN3 [MAP 28.57%; Not Pooled]
BKGeoE3 [MAP 28.27%; Not Pooled]
unswNarrBaseline [MAP 27.58%; Pooled]
rfiaUPV02 [MAP 27.35%; Not Pooled]
enTD [MAP 27.23%; Pooled]
rfiaUPV04 [MAP 26.60%; Pooled]
BKGeoE2 [MAP 26.56%; Not Pooled]
SMGeoEN4 [MAP 26.37%; Not Pooled]
SMGeoEN1 [MAP 26.37%; Pooled]
unswTitleBaseline [MAP 26.22%; Pooled]
sinaiEnEnExp4 [MAP 26.11%; Not Pooled]
rfiaUPV01 [MAP 25.07%; Not Pooled]
sinaiEnEnExp2 [MAP 25.04%; Pooled]
BKGeoE1 [MAP 24.99%; Pooled]
UBManual2 [MAP 24.46%; Not Pooled]
MuTdnTxt [MAP 24.44%; Pooled]
sinaiEnEnExp5 [MAP 24.07%; Not Pooled]
UAUJAUPVenenExp1 [MAP 24.03%; Not Pooled]
MuTdnManQexpGeo [MAP 24.00%; Not Pooled]
msramanual [MAP 23.95%; Pooled]
SMGeoEN5 [MAP 23.77%; Pooled]
SMGeoEN5 [MAP 23.77%; Not Pooled]
UBGTDrf1 [MAP 23.44%; Not Pooled]
MuTdRedn [MAP 23.41%; Not Pooled]
rfiaUPV03 [MAP 23.35%; Pooled]
UBGTDrf2 [MAP 23.30%; Pooled]
MuTdTxt [MAP 23.12%; Not Pooled]
UBGManual1 [MAP 23.07%; Pooled]
sinaiEnEnExp3 [MAP 22.95%; Not Pooled]
MuTdQexpPrb [MAP 22.18%; Pooled]
unswTitleF46 [MAP 22.15%; Not Pooled]
Experiments
CLCGGeoEE11 [MAP 21.94%; Not Pooled]
CLCGGeoEE2 [MAP 21.63%; Pooled]
XLDBGeoENAut05 [MAP 21.45%; Not Pooled]
XLDBGeoENAut03_2 [MAP 20.79%; Pooled]
msrawhitelist [MAP 20.00%; Pooled]
ICgeoMLtdn [MAP 19.53%; Pooled]
HIGeoenenrun3 [MAP 18.75%; Pooled]
XLDBGeoENAut03 [MAP 18.67%; Pooled]
msralocal [MAP 18.37%; Not Pooled]
msratext [MAP 18.35%; Pooled]
CLCGGeoEE5 [MAP 17.57%; Not Pooled]
HIGeoenenrun1n [MAP 17.47%; Not Pooled]
CLCGGeoEE1 [MAP 17.30%; Pooled]
utGeoTIBm [MAP 17.18%; Not Pooled]
CLCGGeoEE10 [MAP 16.90%; Not Pooled]
utGeoTdnIBm [MAP 16.77%; Not Pooled]
HIGeoenenrun1 [MAP 16.76%; Not Pooled]
ICgeoMLtd [MAP 16.49%; Pooled]
utGeoTIB [MAP 16.23%; Pooled]
XLDBGeoENAut02 [MAP 15.79%; Not Pooled]
msraexpansion [MAP 15.21%; Not Pooled]
GCenAA [MAP 13.60%; Pooled]
TALPGeoIRTD1 [MAP 13.42%; Pooled]
GCenAtLg [MAP 13.05%; Not Pooled]
HIGeoenenrun2n [MAP 12.13%; Pooled]
enTDNGeoNames [MAP 12.01%; Pooled]
TALPGeoIRTDN1 [MAP 11.79%; Pooled]
HIGeoenenrun2 [MAP 11.66%; Not Pooled]
utGeoTdnIB [MAP 11.34%; Pooled]
TALPGeoIRTDN3 [MAP 9.97%; Not Pooled]
GCenNtLg [MAP 9.37%; Not Pooled]
GCenNA [MAP 8.93%; Pooled]
GCenAO [MAP 8.91%; Not Pooled]
TALPGeoIRTD2 [MAP 7.66%; Not Pooled]
utGeoTdIB [MAP 7.32%; Pooled]
TALPGeoIRTDN2 [MAP 6.38%; Not Pooled]
unswNarrF41 [MAP 4.01%; Not Pooled]
unswNarrMap [MAP 4.00%; Not Pooled]
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Mean Average Precision
24
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-EN-CLEF2006
GeoCLEF Monolingual English track − Tukey T test with "top group" highlighted
XLDBGeoManualEN
sinaiEnEnExp1
enTDN
SMGeoEN3
unswNarrBaseline
BKGeoE4
BKGeoE3
SMGeoEN1
SMGeoEN4
unswTitleBaseline
sinaiEnEnExp4
enTD
rfiaUPV02
BKGeoE2
rfiaUPV04
sinaiEnEnExp2
BKGeoE1
MuTdnTxt
sinaiEnEnExp5
rfiaUPV01
UBManual2
SMGeoEN5
SMGeoEN5
MuTdTxt
UBGTDrf1
UBGTDrf2
msramanual
MuTdnManQexpGeo
sinaiEnEnExp3
UBGManual1
MuTdRedn
rfiaUPV03
Experiments
unswTitleF46
UAUJAUPVenenExp1
XLDBGeoENAut05
MuTdQexpPrb
CLCGGeoEE11
CLCGGeoEE2
XLDBGeoENAut03_2
msrawhitelist
ICgeoMLtdn
msralocal
msratext
CLCGGeoEE1
utGeoTIBm
utGeoTdnIBm
XLDBGeoENAut03
CLCGGeoEE5
utGeoTIB
CLCGGeoEE10
XLDBGeoENAut02
HIGeoenenrun3
ICgeoMLtd
HIGeoenenrun1n
msraexpansion
HIGeoenenrun1
GCenAtLg
TALPGeoIRTD1
TALPGeoIRTDN1
GCenAA
HIGeoenenrun2n
utGeoTdnIB
enTDNGeoNames
GCenNtLg
TALPGeoIRTDN3
HIGeoenenrun2
GCenAO
utGeoTdIB
GCenNA
TALPGeoIRTDN2
unswNarrF41
TALPGeoIRTD2
unswNarrMap
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
arcsin(sqrt(Mean average precision))
25
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-EN-CLEF2006
GeoCLEF Monolingual English track Top 5 Participants − Retrieved documents vs Precision
100%
xldb [XLDBGeoManualEN; R−Prec 33.60%; Not Pooled]
alicante [enTD; R−Prec 28.01%; Pooled]
90% sanmarcos [SMGeoEN4; R−Prec 28.57%; Not Pooled]
unsw [unswTitleBaseline; R−Prec 28.21%; Pooled]
jaen [sinaiEnEnExp4; R−Prec 22.61%; Not Pooled]
80%
70%
60%
R−Precision
50%
40%
30%
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
GeoCLEF Monolingual English track Top 5 Participants − Comparison to Median R−Precision by Topic (Topics 026 to 050)
1
xldb [XLDBGeoManualEN; R−Prec 33.60%; Not Pooled]
alicante [enTD; R−Prec 28.01%; Pooled]
sanmarcos [SMGeoEN4; R−Prec 28.57%; Not Pooled]
unsw [unswTitleBaseline; R−Prec 28.21%; Pooled]
0.8 jaen [sinaiEnEnExp4; R−Prec 22.61%; Not Pooled]
0.6
0.4
0.2
Difference
0
−0.2
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
26
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-EN-CLEF2006
GeoCLEF Monolingual English track − Box Plot of the Topics
XLDBGeoManualEN [R−Prec 33.60%; Not Pooled]
sinaiEnEnExp1 [R−Prec 29.34%; Pooled]
SMGeoEN4 [R−Prec 28.57%; Not Pooled]
SMGeoEN1 [R−Prec 28.57%; Pooled]
enTDN [R−Prec 28.51%; Not Pooled]
SMGeoEN3 [R−Prec 28.36%; Not Pooled]
unswTitleBaseline [R−Prec 28.21%; Pooled]
enTD [R−Prec 28.01%; Pooled]
BKGeoE4 [R−Prec 27.11%; Pooled]
unswTitleF46 [R−Prec 26.87%; Not Pooled]
rfiaUPV04 [R−Prec 26.67%; Pooled]
BKGeoE3 [R−Prec 26.58%; Not Pooled]
rfiaUPV02 [R−Prec 26.50%; Not Pooled]
unswNarrBaseline [R−Prec 25.88%; Pooled]
SMGeoEN5 [R−Prec 25.81%; Pooled]
SMGeoEN5 [R−Prec 25.81%; Not Pooled]
msramanual [R−Prec 25.45%; Pooled]
UBGTDrf1 [R−Prec 25.16%; Not Pooled]
BKGeoE2 [R−Prec 24.84%; Not Pooled]
UBGManual1 [R−Prec 24.73%; Pooled]
UBManual2 [R−Prec 24.59%; Not Pooled]
rfiaUPV01 [R−Prec 24.18%; Not Pooled]
ICgeoMLtdn [R−Prec 23.55%; Pooled]
msrawhitelist [R−Prec 23.52%; Pooled]
UAUJAUPVenenExp1 [R−Prec 23.19%; Not Pooled]
MuTdnManQexpGeo [R−Prec 23.00%; Not Pooled]
sinaiEnEnExp4 [R−Prec 22.61%; Not Pooled]
msralocal [R−Prec 22.45%; Not Pooled]
MuTdQexpPrb [R−Prec 22.40%; Pooled]
UBGTDrf2 [R−Prec 22.19%; Pooled]
XLDBGeoENAut05 [R−Prec 21.97%; Not Pooled]
BKGeoE1 [R−Prec 21.95%; Pooled]
sinaiEnEnExp2 [R−Prec 21.94%; Pooled]
CLCGGeoEE2 [R−Prec 21.94%; Pooled]
rfiaUPV03 [R−Prec 21.93%; Pooled]
Experiments
MuTdRedn [R−Prec 21.92%; Not Pooled]
MuTdnTxt [R−Prec 21.84%; Pooled]
MuTdTxt [R−Prec 21.55%; Not Pooled]
XLDBGeoENAut03_2 [R−Prec 21.53%; Pooled]
CLCGGeoEE11 [R−Prec 21.44%; Not Pooled]
msratext [R−Prec 21.23%; Pooled]
sinaiEnEnExp5 [R−Prec 20.95%; Not Pooled]
sinaiEnEnExp3 [R−Prec 20.28%; Not Pooled]
CLCGGeoEE1 [R−Prec 19.83%; Pooled]
ICgeoMLtd [R−Prec 19.69%; Pooled]
XLDBGeoENAut03 [R−Prec 19.47%; Pooled]
msraexpansion [R−Prec 18.53%; Not Pooled]
utGeoTdnIBm [R−Prec 18.12%; Not Pooled]
HIGeoenenrun3 [R−Prec 17.85%; Pooled]
CLCGGeoEE5 [R−Prec 17.77%; Not Pooled]
CLCGGeoEE10 [R−Prec 17.62%; Not Pooled]
utGeoTIBm [R−Prec 17.38%; Not Pooled]
utGeoTIB [R−Prec 17.38%; Pooled]
HIGeoenenrun1n [R−Prec 16.33%; Not Pooled]
HIGeoenenrun1 [R−Prec 15.95%; Not Pooled]
GCenAA [R−Prec 15.70%; Pooled]
XLDBGeoENAut02 [R−Prec 15.28%; Not Pooled]
TALPGeoIRTD1 [R−Prec 13.70%; Pooled]
utGeoTdnIB [R−Prec 13.66%; Pooled]
GCenAtLg [R−Prec 13.57%; Not Pooled]
TALPGeoIRTDN1 [R−Prec 13.16%; Pooled]
HIGeoenenrun2 [R−Prec 13.05%; Not Pooled]
HIGeoenenrun2n [R−Prec 13.04%; Pooled]
GCenNtLg [R−Prec 10.87%; Not Pooled]
enTDNGeoNames [R−Prec 10.30%; Pooled]
TALPGeoIRTDN3 [R−Prec 9.85%; Not Pooled]
GCenNA [R−Prec 9.70%; Pooled]
GCenAO [R−Prec 9.52%; Not Pooled]
TALPGeoIRTD2 [R−Prec 8.84%; Not Pooled]
TALPGeoIRTDN2 [R−Prec 8.13%; Not Pooled]
utGeoTdIB [R−Prec 7.62%; Pooled]
unswNarrF41 [R−Prec 4.06%; Not Pooled]
unswNarrMap [R−Prec 4.06%; Not Pooled]
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
R−Precision
27
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-EN-CLEF2006
GeoCLEF Monolingual English track − Tukey T test with "top group" highlighted
XLDBGeoManualEN
SMGeoEN1
SMGeoEN4
unswTitleBaseline
unswTitleF46
sinaiEnEnExp1
SMGeoEN3
enTDN
enTD
BKGeoE4
unswNarrBaseline
rfiaUPV02
BKGeoE3
UBGManual1
rfiaUPV04
UBGTDrf1
SMGeoEN5
SMGeoEN5
msramanual
UBManual2
rfiaUPV01
ICgeoMLtdn
BKGeoE2
msrawhitelist
msralocal
XLDBGeoENAut05
MuTdnManQexpGeo
sinaiEnEnExp4
MuTdnTxt
MuTdTxt
MuTdQexpPrb
XLDBGeoENAut03_2
Experiments
CLCGGeoEE2
UBGTDrf2
MuTdRedn
BKGeoE1
CLCGGeoEE11
UAUJAUPVenenExp1
sinaiEnEnExp2
sinaiEnEnExp5
rfiaUPV03
msratext
sinaiEnEnExp3
ICgeoMLtd
XLDBGeoENAut03
CLCGGeoEE1
msraexpansion
utGeoTdnIBm
utGeoTIB
utGeoTIBm
CLCGGeoEE10
CLCGGeoEE5
HIGeoenenrun3
XLDBGeoENAut02
HIGeoenenrun1n
GCenAA
HIGeoenenrun1
utGeoTdnIB
TALPGeoIRTD1
GCenAtLg
TALPGeoIRTDN1
HIGeoenenrun2n
HIGeoenenrun2
GCenNtLg
TALPGeoIRTDN3
GCenNA
utGeoTdIB
GCenAO
enTDNGeoNames
TALPGeoIRTDN2
TALPGeoIRTD2
unswNarrMap
unswNarrF41
−0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
arcsin(sqrt(R Precision))
28
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-EN-CLEF2006
Average Precision R-Precision
Topic Minimum 1st Q. Median 3rd Q. Maximum Mean Std Minimum 1st Q. Median 3rd Q. Maximum Mean Std
026 0.0000 0.0096 0.0997 0.1511 0.5009 0.1191 0.1294 0.0000 0.0000 0.1111 0.2500 0.5556 0.1553 0.1612
027 0.0000 0.0071 0.0272 0.0605 0.1257 0.0373 0.0367 0.0000 0.0000 0.0526 0.1053 0.2105 0.0620 0.0521
028 0.0000 0.0016 0.0509 0.0948 0.3017 0.0694 0.0789 0.0000 0.0000 0.1053 0.2105 0.4211 0.1211 0.1218
029 0.0000 0.0563 0.1046 0.1983 0.5485 0.1386 0.1084 0.0000 0.1111 0.1111 0.2222 0.6667 0.1598 0.1446
030 0.0000 0.2022 0.5984 0.8356 1.0000 0.5443 0.3362 0.0000 0.2917 0.6667 0.8333 1.0000 0.5137 0.3090
031 0.0105 0.0634 0.2611 0.3582 0.6879 0.2423 0.1662 0.0339 0.1525 0.2542 0.3771 0.6610 0.2730 0.1446
032 0.0000 0.5633 0.8145 0.9047 0.9631 0.6987 0.2725 0.0000 0.6452 0.7419 0.8387 0.9032 0.6757 0.2383
033 0.0000 0.0019 0.0035 0.0142 0.4713 0.0479 0.1163 0.0000 0.0000 0.0000 0.0500 0.5500 0.0644 0.1383
034 0.0000 0.0489 0.3693 0.4167 0.8056 0.2916 0.2039 0.0000 0.0000 0.3333 0.4167 0.6667 0.3242 0.2420
035 0.0000 0.0097 0.0276 0.0781 0.5397 0.0640 0.0994 0.0000 0.0000 0.0000 0.0000 0.5000 0.0479 0.0981
036 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
037 0.0000 0.0028 0.0183 0.1002 0.2731 0.0553 0.0668 0.0000 0.0000 0.0625 0.1875 0.3750 0.0933 0.1027
038 0.0000 0.0017 0.0128 0.0331 1.0000 0.0541 0.1384 0.0000 0.0000 0.0000 0.0000 1.0000 0.0137 0.1170
039 0.0000 0.0491 0.1120 0.3437 0.5778 0.1872 0.1558 0.0000 0.0625 0.1875 0.3750 0.5000 0.2149 0.1620
040 0.0000 0.0539 0.2175 0.3213 0.8560 0.2112 0.1861 0.0000 0.0536 0.2143 0.2857 0.7857 0.2084 0.1661
041 0.0000 0.0000 0.0024 0.0086 0.2500 0.0123 0.0412 0.0000 0.0000 0.0000 0.0000 0.2500 0.0068 0.0411
042 0.0000 0.0109 0.0970 0.5016 1.0000 0.2534 0.2866 0.0000 0.0000 0.0000 0.5000 1.0000 0.1918 0.2717
043 0.0000 0.0026 0.0082 0.0259 0.3115 0.0290 0.0583 0.0000 0.0000 0.0000 0.0000 0.3750 0.0342 0.0759
044 0.0000 0.0468 0.1071 0.1595 0.2895 0.1143 0.0731 0.0000 0.1053 0.1579 0.2105 0.3684 0.1583 0.0898
045 0.0000 0.0069 0.0823 0.2265 0.8256 0.1498 0.1894 0.0000 0.0000 0.0000 0.1667 0.8333 0.1301 0.2065
046 0.0000 0.1542 0.6686 0.7083 1.0000 0.5205 0.3059 0.0000 0.2500 0.6667 0.6667 1.0000 0.4840 0.2994
047 0.0000 0.0116 0.0364 0.0647 0.1914 0.0460 0.0418 0.0000 0.0000 0.0417 0.0833 0.2917 0.0559 0.0645
048 0.0625 0.5158 0.6973 0.7856 0.9086 0.6182 0.2347 0.0208 0.5573 0.6667 0.7292 0.8542 0.6084 0.2152
049 0.0000 0.1624 0.2667 0.5000 0.6429 0.2953 0.1969 0.0000 0.0000 0.5000 0.5000 0.5000 0.2534 0.2517
050 0.0000 0.0477 0.1323 0.2303 0.3143 0.1378 0.0974 0.0000 0.0667 0.1333 0.3333 0.4000 0.1726 0.1266
ALL 0.0400 0.1565 0.2163 0.2459 0.3224 0.1975 0.0682 0.0406 0.1589 0.2184 0.2492 0.3360 0.2009 0.0652
29
30
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-ES-CLEF2006
GeoCLEF Monolingual Spanish track Top 5 Participants − Interpolated Recall vs Average Precision
100%
alicante [esTD; MAP 35.08%; Pooled]
berkeley [BKGeoS1; MAP 31.82%; Pooled]
90% daedalus [GCesNtLg; MAP 16.12%; Pooled]
sanmarcos [SMGeoES1; MAP 14.71%; Pooled]
80%
70%
Average Precision
60%
50%
40%
30%
20%
10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
GeoCLEF Monolingual Spanish track Top 5 Participants − Comparison to Median Mean Average Precision by Topic (Topics 026 to 05
0)
1
alicante [esTD; MAP 35.08%; Pooled]
berkeley [BKGeoS1; MAP 31.82%; Pooled]
daedalus [GCesNtLg; MAP 16.12%; Pooled]
sanmarcos [SMGeoES1; MAP 14.71%; Pooled]
0.8
0.6
0.4
0.2
Difference
0
−0.2
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
31
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-ES-CLEF2006
GeoCLEF Monolingual Spanish track − Box Plot of the Topics
esTD [MAP 35.08%; Pooled]
esTDN [MAP 32.37%; Pooled]
BKGeoS1 [MAP 31.82%; Pooled]
BKGeoS2 [MAP 30.03%; Pooled]
GCesNtLg [MAP 16.12%; Pooled]
SMGeoES2 [MAP 15.33%; Pooled]
esTDNGeoNames [MAP 15.25%; Pooled]
Experiments
SMGeoES3 [MAP 14.71%; Pooled]
SMGeoES1 [MAP 14.71%; Pooled]
SMGeoES5 [MAP 14.71%; Pooled]
GCesAtLg [MAP 14.13%; Pooled]
SMGeoES4 [MAP 13.78%; Pooled]
GCesAA [MAP 13.48%; Pooled]
GCesNA [MAP 12.73%; Pooled]
GCesAO [MAP 12.21%; Pooled]
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Mean Average Precision
32
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-ES-CLEF2006
GeoCLEF Monolingual Spanish track − Tukey T test with "top group" highlighted
esTD
esTDN
BKGeoS1
BKGeoS2
GCesNtLg
SMGeoES2
SMGeoES5
Experiments
SMGeoES1
SMGeoES3
GCesAtLg
SMGeoES4
GCesAA
GCesAO
esTDNGeoNames
GCesNA
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
arcsin(sqrt(Mean average precision))
33
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-ES-CLEF2006
GeoCLEF Monolingual Spanish track Top 5 Participants − Retrieved documents vs Precision
100%
alicante [esTD; R−Prec 35.83%; Pooled]
berkeley [BKGeoS1; R−Prec 32.11%; Pooled]
90% daedalus [GCesNtLg; R−Prec 18.59%; Pooled]
sanmarcos [SMGeoES1; R−Prec 20.44%; Pooled]
80%
70%
60%
R−Precision
50%
40%
30%
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
GeoCLEF Monolingual Spanish track Top 5 Participants − Comparison to Median R−Precision by Topic (Topics 026 to 050)
1
alicante [esTD; R−Prec 35.83%; Pooled]
berkeley [BKGeoS1; R−Prec 32.11%; Pooled]
daedalus [GCesNtLg; R−Prec 18.59%; Pooled]
sanmarcos [SMGeoES1; R−Prec 20.44%; Pooled]
0.8
0.6
0.4
0.2
Difference
0
−0.2
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
34
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-ES-CLEF2006
GeoCLEF Monolingual Spanish track − Box Plot of the Topics
esTD [R−Prec 35.83%; Pooled]
esTDN [R−Prec 33.77%; Pooled]
BKGeoS1 [R−Prec 32.11%; Pooled]
BKGeoS2 [R−Prec 29.94%; Pooled]
SMGeoES5 [R−Prec 20.44%; Pooled]
SMGeoES3 [R−Prec 20.44%; Pooled]
SMGeoES1 [R−Prec 20.44%; Pooled]
Experiments
SMGeoES2 [R−Prec 20.29%; Pooled]
SMGeoES4 [R−Prec 18.63%; Pooled]
GCesNtLg [R−Prec 18.59%; Pooled]
GCesNA [R−Prec 17.18%; Pooled]
GCesAA [R−Prec 17.01%; Pooled]
GCesAtLg [R−Prec 16.58%; Pooled]
esTDNGeoNames [R−Prec 16.23%; Pooled]
GCesAO [R−Prec 13.82%; Pooled]
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
R−Precision
35
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-ES-CLEF2006
GeoCLEF Monolingual Spanish track − Tukey T test with "top group" highlighted
esTD
esTDN
BKGeoS1
BKGeoS2
SMGeoES1
SMGeoES3
SMGeoES5
Experiments
SMGeoES2
SMGeoES4
GCesNtLg
GCesNA
GCesAtLg
GCesAA
GCesAO
esTDNGeoNames
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
arcsin(sqrt(R Precision))
36
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-ES-CLEF2006
Average Precision R-Precision
Topic Minimum 1st Q. Median 3rd Q. Maximum Mean Std Minimum 1st Q. Median 3rd Q. Maximum Mean Std
026 0.0000 0.0000 0.0171 0.0236 0.1518 0.0372 0.0592 0.0000 0.0000 0.1111 0.1111 0.1667 0.0704 0.0711
027 0.0000 0.0000 0.0099 0.0171 0.1035 0.0162 0.0268 0.0000 0.0000 0.0000 0.0256 0.1026 0.0205 0.0352
028 0.0000 0.0067 0.2370 0.2582 0.3937 0.1676 0.1405 0.0000 0.0000 0.2778 0.2986 0.3333 0.1741 0.1449
029 0.0007 0.2228 0.2759 0.4774 0.6863 0.3195 0.1918 0.0000 0.2879 0.4242 0.5379 0.6061 0.3818 0.1744
030 0.0000 0.1352 0.1489 0.3549 0.6869 0.2430 0.2010 0.0000 0.2003 0.2119 0.3891 0.6689 0.2786 0.1839
031 0.0085 0.0099 0.0107 0.5776 0.7252 0.2147 0.3057 0.0314 0.0510 0.0706 0.5422 0.7333 0.2387 0.2731
032 0.1984 0.2000 0.8612 0.9447 0.9782 0.6720 0.3488 0.2077 0.2077 0.8462 0.8885 0.9154 0.6477 0.3232
033 0.0000 0.0001 0.0003 0.0161 0.5464 0.0527 0.1452 0.0000 0.0100 0.0100 0.0475 0.7100 0.0867 0.1927
034 0.0000 0.0908 0.1834 0.3013 0.3533 0.1848 0.1142 0.0000 0.0338 0.2432 0.4257 0.5135 0.2414 0.1849
035 0.0015 0.0858 0.1217 0.1507 0.1645 0.1093 0.0501 0.0000 0.1053 0.2105 0.2105 0.2632 0.1614 0.0855
036 0.0000 0.0029 0.1093 0.1531 0.5137 0.1200 0.1554 0.0000 0.0273 0.2091 0.2341 0.6091 0.1915 0.1959
037 0.0000 0.0001 0.0746 0.1105 0.2206 0.0744 0.0790 0.0000 0.0000 0.1379 0.1724 0.2759 0.1103 0.1011
038 0.0000 0.0000 0.0333 0.0667 0.2000 0.0473 0.0580 0.0000 0.0000 0.0000 0.0000 0.2000 0.0400 0.0828
039 0.0148 0.0783 0.2211 0.2572 0.3246 0.1746 0.1049 0.0149 0.1530 0.2090 0.2239 0.3881 0.1990 0.1030
040 0.0000 0.3429 0.5408 0.6722 0.7764 0.4881 0.2507 0.0000 0.3525 0.6043 0.6853 0.7338 0.4950 0.2550
041 0.0084 0.0090 0.2018 0.2814 0.3878 0.1703 0.1417 0.0533 0.0700 0.2400 0.3200 0.4000 0.2080 0.1267
042 0.0104 0.1179 0.1835 0.3087 0.5082 0.2177 0.1446 0.0000 0.2689 0.2830 0.3915 0.5283 0.2906 0.1588
043 0.0000 0.0000 0.0133 0.0288 0.4845 0.0486 0.1227 0.0000 0.0000 0.0000 0.1146 0.5833 0.0778 0.1582
044 0.0001 0.0459 0.1462 0.2915 0.4134 0.1760 0.1423 0.0000 0.1019 0.2136 0.3786 0.4660 0.2421 0.1513
045 0.0000 0.0044 0.0096 0.0153 0.0456 0.0123 0.0139 0.0000 0.0000 0.0000 0.0000 0.0833 0.0167 0.0345
046 0.0357 0.1065 0.1310 0.5951 0.8330 0.3167 0.3032 0.1071 0.1429 0.1429 0.5625 0.7500 0.3262 0.2514
047 0.0000 0.0001 0.0407 0.1044 0.1203 0.0500 0.0470 0.0000 0.0000 0.0339 0.1653 0.2203 0.0746 0.0797
048 0.0540 0.0589 0.4031 0.7027 0.8118 0.4110 0.2973 0.0755 0.0755 0.4226 0.6377 0.7208 0.4111 0.2648
049 0.0006 0.0987 0.5089 0.5745 0.7874 0.3817 0.2746 0.0184 0.1175 0.5115 0.6682 0.7051 0.4267 0.2677
050 0.0000 0.0347 0.0415 0.0807 0.2721 0.0685 0.0710 0.0000 0.0450 0.1200 0.1400 0.3600 0.1107 0.0919
ALL 0.1221 0.1386 0.1471 0.2655 0.3508 0.1910 0.0837 0.1382 0.1705 0.2029 0.2756 0.3583 0.2209 0.0710
37
38
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-PT-CLEF2006
GeoCLEF Monolingual Portuguese track Top 5 Participants − Interpolated Recall vs Average Precision
100%
xldb [XLDBGeoManualPT; MAP 30.12%; Pooled]
berkeley [BKGeoP3; MAP 16.92%; Pooled]
90%
sanmarcos [SMGeoPT2; MAP 13.44%; Pooled]
80%
70%
Average Precision
60%
50%
40%
30%
20%
10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
GeoCLEF Monolingual Portuguese track Top 5 Participants − Comparison to Median Mean Average Precision by Topic (Topics 026 to050)
1
xldb [XLDBGeoManualPT; MAP 30.12%; Pooled]
berkeley [BKGeoP3; MAP 16.92%; Pooled]
sanmarcos [SMGeoPT2; MAP 13.44%; Pooled]
0.8
0.6
0.4
0.2
Difference
0
−0.2
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
39
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-PT-CLEF2006
GeoCLEF Monolingual Portuguese track − Box Plot of the Topics
XLDBGeoManualPT [MAP 30.12%; Pooled]
XLDBGeoPTAut05 [MAP 29.32%; Pooled]
XLDBGeoPTAut02 [MAP 25.70%; Pooled]
XLDBGeoPTAut03 [MAP 19.29%; Pooled]
BKGeoP4 [MAP 17.36%; Pooled]
BKGeoP3 [MAP 16.92%; Pooled]
Experiments
BKGeoP2 [MAP 16.31%; Pooled]
BKGeoP1 [MAP 16.22%; Pooled]
XLDBGeoPTAut03_2 [MAP 15.13%; Pooled]
SMGeoPT2 [MAP 13.44%; Pooled]
SMGeoPT1 [MAP 10.98%; Pooled]
SMGeoPT3 [MAP 10.98%; Pooled]
SMGeoPT4 [MAP 10.63%; Pooled]
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Mean Average Precision
40
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-PT-CLEF2006
GeoCLEF Monolingual Portuguese track − Tukey T test with "top group" highlighted
XLDBGeoManualPT
XLDBGeoPTAut05
XLDBGeoPTAut02
XLDBGeoPTAut03
BKGeoP3
BKGeoP4
Experiments
BKGeoP1
SMGeoPT2
BKGeoP2
XLDBGeoPTAut03_2
SMGeoPT3
SMGeoPT1
SMGeoPT4
0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65
arcsin(sqrt(Mean average precision))
41
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-PT-CLEF2006
GeoCLEF Monolingual Portuguese track Top 5 Participants − Retrieved documents vs Precision
100%
xldb [XLDBGeoManualPT; R−Prec 35.89%; Pooled]
berkeley [BKGeoP3; R−Prec 16.51%; Pooled]
90%
sanmarcos [SMGeoPT2; R−Prec 15.02%; Pooled]
80%
70%
60%
R−Precision
50%
40%
30%
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
GeoCLEF Monolingual Portuguese track Top 5 Participants − Comparison to Median R−Precision by Topic (Topics 026 to 050)
1
xldb [XLDBGeoManualPT; R−Prec 35.89%; Pooled]
berkeley [BKGeoP3; R−Prec 16.51%; Pooled]
sanmarcos [SMGeoPT2; R−Prec 15.02%; Pooled]
0.8
0.6
0.4
0.2
Difference
0
−0.2
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
42
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-PT-CLEF2006
GeoCLEF Monolingual Portuguese track − Box Plot of the Topics
XLDBGeoManualPT [R−Prec 35.89%; Pooled]
XLDBGeoPTAut05 [R−Prec 34.57%; Pooled]
XLDBGeoPTAut02 [R−Prec 28.09%; Pooled]
XLDBGeoPTAut03 [R−Prec 23.91%; Pooled]
XLDBGeoPTAut03_2 [R−Prec 17.29%; Pooled]
BKGeoP4 [R−Prec 17.22%; Pooled]
Experiments
BKGeoP3 [R−Prec 16.51%; Pooled]
BKGeoP2 [R−Prec 16.46%; Pooled]
BKGeoP1 [R−Prec 16.43%; Pooled]
SMGeoPT2 [R−Prec 15.02%; Pooled]
SMGeoPT1 [R−Prec 13.91%; Pooled]
SMGeoPT3 [R−Prec 13.91%; Pooled]
SMGeoPT4 [R−Prec 13.57%; Pooled]
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
R−Precision
43
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-PT-CLEF2006
GeoCLEF Monolingual Portuguese track − Tukey T test with "top group" highlighted
XLDBGeoManualPT
XLDBGeoPTAut05
XLDBGeoPTAut02
XLDBGeoPTAut03
SMGeoPT2
XLDBGeoPTAut03_2
Experiments
BKGeoP4
BKGeoP3
BKGeoP1
SMGeoPT3
SMGeoPT1
SMGeoPT4
BKGeoP2
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
arcsin(sqrt(R Precision))
44
GC-MONO-CLEF2006 Track Overview Results and Graphs GC-MONO-PT-CLEF2006
Average Precision R-Precision
Topic Minimum 1st Q. Median 3rd Q. Maximum Mean Std Minimum 1st Q. Median 3rd Q. Maximum Mean Std
026 0.0000 0.0017 0.0040 0.0159 0.1959 0.0239 0.0538 0.0000 0.0000 0.0000 0.0000 0.2667 0.0308 0.0799
027 0.0004 0.0005 0.0012 0.0310 0.6435 0.0663 0.1771 0.0000 0.0098 0.0098 0.0784 0.7353 0.0913 0.1993
028 0.0000 0.0297 0.1309 0.1940 0.4252 0.1592 0.1436 0.0000 0.0938 0.1562 0.2344 0.5000 0.2043 0.1576
029 0.0638 0.1708 0.2773 0.3452 0.5215 0.2619 0.1421 0.1538 0.1795 0.3333 0.4038 0.4872 0.3057 0.1258
030 0.0112 0.3349 0.4555 0.5110 0.6566 0.3965 0.2318 0.0714 0.3393 0.4286 0.5714 0.6429 0.4066 0.2028
031 0.0000 0.1601 0.1842 0.2724 0.3334 0.1946 0.0936 0.0000 0.1311 0.1475 0.2418 0.4098 0.1942 0.1208
032 0.2995 0.4778 0.5345 0.6936 0.8587 0.5878 0.1539 0.3774 0.5566 0.5849 0.6792 0.8302 0.6096 0.1156
033 0.0000 0.0010 0.0019 0.0362 0.0988 0.0232 0.0373 0.0000 0.0000 0.0000 0.0000 0.2500 0.0192 0.0693
034 0.0003 0.0016 0.0457 0.1608 0.2519 0.0815 0.0931 0.0000 0.0000 0.1250 0.1250 0.3750 0.1058 0.1123
035 0.0000 0.0048 0.0094 0.0155 0.1505 0.0202 0.0397 0.0000 0.0000 0.0000 0.0000 0.1111 0.0085 0.0308
036 0.0000 0.0000 0.0097 0.0247 0.0952 0.0177 0.0257 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
037 0.0000 0.0005 0.0390 0.1791 0.3321 0.0966 0.1272 0.0000 0.0000 0.0000 0.2847 0.4444 0.1261 0.1788
038 0.0174 0.0544 0.0978 0.1432 0.7500 0.1790 0.2241 0.0000 0.0000 0.0000 0.2500 0.7500 0.1731 0.2774
039 0.0426 0.0450 0.0896 0.1481 0.3276 0.1257 0.0966 0.0435 0.0870 0.1304 0.1739 0.3478 0.1505 0.0966
040 0.0003 0.0008 0.0073 0.2421 0.5025 0.1204 0.1658 0.0000 0.0000 0.0833 0.3021 0.5000 0.1571 0.1694
041 0.0000 0.0001 0.0056 0.3792 0.6930 0.1594 0.2346 0.0000 0.0000 0.0192 0.4471 0.6731 0.1864 0.2594
042 0.1065 0.1883 0.3088 0.4082 0.5240 0.3060 0.1323 0.2000 0.2571 0.3714 0.4857 0.5429 0.3736 0.1175
043 0.0002 0.0008 0.0011 0.0187 0.0809 0.0174 0.0312 0.0000 0.0000 0.0000 0.0104 0.1667 0.0256 0.0552
044 0.0037 0.0065 0.0317 0.0819 0.3660 0.0830 0.1174 0.0000 0.0000 0.1053 0.1809 0.4605 0.1407 0.1512
045 0.0334 0.1661 0.2139 0.3015 0.3538 0.2268 0.0941 0.0854 0.2591 0.3415 0.3689 0.4756 0.3039 0.1011
046 0.0000 0.2242 0.2866 0.5128 0.5987 0.3561 0.1850 0.0000 0.3295 0.3333 0.5455 0.6515 0.4079 0.1747
047 0.0000 0.0000 0.0084 0.0686 0.1916 0.0404 0.0613 0.0000 0.0000 0.0000 0.0441 0.2353 0.0385 0.0744
048 0.0000 0.4794 0.5924 0.8680 0.9241 0.5979 0.3096 0.0000 0.5175 0.6014 0.7832 0.8112 0.5756 0.2604
049 0.0000 0.1378 0.2267 0.2731 0.3344 0.2035 0.0964 0.0000 0.1389 0.2778 0.3472 0.3889 0.2350 0.1401
050 0.0000 0.0395 0.1226 0.2166 0.2360 0.1240 0.0925 0.0000 0.0909 0.2273 0.2727 0.3409 0.1836 0.1168
ALL 0.1063 0.1283 0.1631 0.2089 0.3012 0.1788 0.0662 0.1357 0.1475 0.1651 0.2495 0.3589 0.2021 0.0784
45
46
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2DE-CLEF2006
GeoCLEF Bilingual German track Top 5 Participants − Interpolated Recall vs Average Precision
100%
berkeley [BKGeoED1; MAP 15.61%; Pooled]
hagen [FUHedGYYYTD; MAP 12.80%; Pooled]
90%
hildesheim [HIGeoenderun21; MAP 11.86%; Pooled]
80%
70%
Average Precision
60%
50%
40%
30%
20%
10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
GeoCLEF Bilingual German track Top 5 Participants − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
1
berkeley [BKGeoED1; MAP 15.61%; Pooled]
hagen [FUHedGYYYTD; MAP 12.80%; Pooled]
hildesheim [HIGeoenderun21; MAP 11.86%; Pooled]
0.8
0.6
0.4
0.2
Difference
0
−0.2
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
47
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2DE-CLEF2006
GeoCLEF Bilingual German track − Box Plot of the Topics
BKGeoED2 [MAP 16.82%; Pooled]
BKGeoED1 [MAP 15.61%; Pooled]
HIGeoenderun21n [MAP 13.15%; Pooled]
FUHedGYYYTD [MAP 12.80%; Pooled]
FUHedGYYYTDN [MAP 12.34%; Pooled]
Experiments
FUHedGNNNTD [MAP 12.11%; Pooled]
HIGeoenderun21 [MAP 11.86%; Pooled]
FUHedGYYYMTDN [MAP 11.48%; Pooled]
HIGeoenderun22n [MAP 10.46%; Pooled]
HIGeoenderun22 [MAP 9.69%; Pooled]
FUHedGNNNTDN [MAP 5.48%; Pooled]
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Mean Average Precision
48
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2DE-CLEF2006
GeoCLEF Bilingual German track − Tukey T test with "top group" highlighted
BKGeoED2
BKGeoED1
FUHedGNNNTD
HIGeoenderun21n
FUHedGYYYTD
Experiments
HIGeoenderun21
FUHedGYYYTDN
FUHedGYYYMTDN
HIGeoenderun22n
HIGeoenderun22
FUHedGNNNTDN
0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4
arcsin(sqrt(Mean average precision))
49
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2DE-CLEF2006
GeoCLEF Bilingual German track Top 5 Participants − Retrieved documents vs Precision
100%
berkeley [BKGeoED1; R−Prec 15.44%; Pooled]
hagen [FUHedGYYYTD; R−Prec 11.94%; Pooled]
90%
hildesheim [HIGeoenderun21; R−Prec 15.18%; Pooled]
80%
70%
60%
R−Precision
50%
40%
30%
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
GeoCLEF Bilingual German track Top 5 Participants − Comparison to Median R−Precision by Topic (Topics 026 to 050)
1
berkeley [BKGeoED1; R−Prec 15.44%; Pooled]
hagen [FUHedGYYYTD; R−Prec 11.94%; Pooled]
hildesheim [HIGeoenderun21; R−Prec 15.18%; Pooled]
0.8
0.6
0.4
0.2
Difference
0
−0.2
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
50
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2DE-CLEF2006
GeoCLEF Bilingual German track − Box Plot of the Topics
BKGeoED2 [R−Prec 18.56%; Pooled]
BKGeoED1 [R−Prec 15.44%; Pooled]
FUHedGNNNTD [R−Prec 15.34%; Pooled]
HIGeoenderun21n [R−Prec 15.21%; Pooled]
HIGeoenderun21 [R−Prec 15.18%; Pooled]
Experiments
FUHedGYYYTDN [R−Prec 12.45%; Pooled]
FUHedGYYYTD [R−Prec 11.94%; Pooled]
HIGeoenderun22n [R−Prec 11.77%; Pooled]
HIGeoenderun22 [R−Prec 11.72%; Pooled]
FUHedGYYYMTDN [R−Prec 11.57%; Pooled]
FUHedGNNNTDN [R−Prec 6.24%; Pooled]
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
R−Precision
51
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2DE-CLEF2006
GeoCLEF Bilingual German track − Tukey T test with "top group" highlighted
FUHedGNNNTD
BKGeoED2
HIGeoenderun21n
HIGeoenderun21
BKGeoED1
Experiments
HIGeoenderun22n
HIGeoenderun22
FUHedGYYYTDN
FUHedGYYYTD
FUHedGYYYMTDN
FUHedGNNNTDN
0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4
arcsin(sqrt(R Precision))
52
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2DE-CLEF2006
Average Precision R-Precision
Topic Minimum 1st Q. Median 3rd Q. Maximum Mean Std Minimum 1st Q. Median 3rd Q. Maximum Mean Std
026 0.0000 0.0001 0.0006 0.0022 0.1000 0.0102 0.0298 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
027 0.0008 0.0068 0.0114 0.0167 0.0271 0.0125 0.0080 0.0154 0.0308 0.0615 0.1192 0.1385 0.0713 0.0447
028 0.0030 0.0199 0.2800 0.3404 0.4485 0.2100 0.1657 0.0000 0.0625 0.3750 0.4297 0.4375 0.2614 0.1871
029 0.0061 0.0122 0.0645 0.4400 0.5040 0.2033 0.2187 0.0000 0.0000 0.0000 0.3333 0.3333 0.1515 0.1741
030 0.0183 0.0419 0.0728 0.1025 0.1984 0.0818 0.0558 0.0333 0.0667 0.1000 0.1583 0.2667 0.1182 0.0765
031 0.0051 0.0265 0.0531 0.5173 0.7984 0.2305 0.3106 0.0000 0.0033 0.0921 0.5493 0.7632 0.2428 0.3031
032 0.3107 0.4858 0.5628 0.5719 0.8326 0.5559 0.1549 0.4259 0.4907 0.5370 0.5556 0.8519 0.5673 0.1313
033 0.0000 0.0000 0.0001 0.0119 0.0272 0.0062 0.0103 0.0000 0.0000 0.0000 0.0000 0.1176 0.0107 0.0355
034 0.0000 0.0693 0.0886 0.5085 0.6819 0.2526 0.2495 0.0000 0.0588 0.1176 0.5515 0.6765 0.2754 0.2588
035 0.0008 0.0108 0.0205 0.0427 0.0575 0.0260 0.0201 0.0000 0.0000 0.0000 0.0972 0.1667 0.0505 0.0678
036 0.0000 0.0006 0.0047 0.0261 0.0660 0.0155 0.0213 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
037 0.0000 0.0014 0.0034 0.0061 0.0795 0.0113 0.0231 0.0000 0.0000 0.0000 0.0000 0.1818 0.0165 0.0548
038 0.0000 0.0000 0.0027 0.0054 0.1354 0.0150 0.0401 0.0000 0.0000 0.0000 0.0000 0.2500 0.0227 0.0754
039 0.0004 0.0136 0.0357 0.1466 0.2652 0.0781 0.0883 0.0000 0.0463 0.0741 0.1667 0.2593 0.1077 0.0819
040 0.0072 0.2165 0.2901 0.3833 0.3912 0.2860 0.1193 0.0244 0.3110 0.4146 0.4390 0.4634 0.3659 0.1277
041 0.0110 0.0273 0.0526 0.0630 0.1102 0.0525 0.0304 0.0000 0.0000 0.0000 0.0921 0.2105 0.0478 0.0684
042 0.0004 0.0027 0.0071 0.0314 0.2404 0.0377 0.0718 0.0000 0.0000 0.0213 0.0798 0.3617 0.0754 0.1171
043 0.0016 0.0026 0.0063 0.0203 0.2569 0.0333 0.0751 0.0000 0.0000 0.0000 0.0714 0.5000 0.0779 0.1480
044 0.0017 0.0044 0.0098 0.0143 0.1111 0.0187 0.0314 0.0000 0.0000 0.0000 0.0000 0.3333 0.0303 0.1005
045 0.0000 0.0000 0.0059 0.0151 0.0417 0.0107 0.0135 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
046 0.0000 0.0015 0.0136 0.0530 0.1680 0.0333 0.0503 0.0000 0.0000 0.0000 0.0000 0.2500 0.0227 0.0754
047 0.0000 0.0000 0.0000 0.0002 0.0059 0.0006 0.0018 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
048 0.3399 0.5821 0.8120 0.8631 0.9178 0.7193 0.1841 0.3623 0.5978 0.7681 0.7790 0.8841 0.6904 0.1545
049 0.0000 0.0000 0.0286 0.1405 0.1957 0.0644 0.0800 0.0000 0.0000 0.0000 0.1667 0.1667 0.0606 0.0841
050 0.0011 0.0107 0.0355 0.0470 0.0548 0.0301 0.0200 0.0000 0.0000 0.0000 0.0833 0.0833 0.0379 0.0435
ALL 0.0548 0.1072 0.1211 0.1306 0.1682 0.1198 0.0298 0.0624 0.1173 0.1245 0.1531 0.1856 0.1322 0.0322
53
54
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2EN-CLEF2006
GeoCLEF Bilingual English track Top 5 Participants − Interpolated Recall vs Average Precision
100%
jaen [sinaiEsEnExp2; MAP 22.56%; Pooled]
sanmarcos [SMGeoESEN2; MAP 22.46%; Pooled]
90%
hildesheim [HIGeodeenrun12; MAP 16.03%; Pooled]
80%
70%
Average Precision
60%
50%
40%
30%
20%
10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
GeoCLEF Bilingual English track Top 5 Participants − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
1
jaen [sinaiEsEnExp2; MAP 22.56%; Pooled]
sanmarcos [SMGeoESEN2; MAP 22.46%; Pooled]
hildesheim [HIGeodeenrun12; MAP 16.03%; Pooled]
0.8
0.6
0.4
0.2
Difference
0
−0.2
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
55
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2EN-CLEF2006
GeoCLEF Bilingual English track − Box Plot of the Topics
sinaiEsEnExp1 [MAP 27.07%; Pooled]
SMGeoESEN1 [MAP 25.52%; Pooled]
sinaiEsEnExp2 [MAP 22.56%; Pooled]
SMGeoESEN2 [MAP 22.46%; Pooled]
sinaiEsEnExp3 [MAP 22.09%; Not Pooled]
sinaiDeEnExp2 [MAP 21.64%; Not Pooled]
Experiments
HIGeodeenrun11n [MAP 19.03%; Not Pooled]
sinaiDeEnExp1 [MAP 18.68%; Not Pooled]
HIGeodeenrun12 [MAP 16.03%; Pooled]
HIGeodeenrun13n [MAP 15.65%; Pooled]
HIGeodeenrun11 [MAP 15.04%; Not Pooled]
HIGeodeenrun13 [MAP 14.56%; Not Pooled]
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Mean Average Precision
56
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2EN-CLEF2006
GeoCLEF Bilingual English track − Tukey T test with "top group" highlighted
SMGeoESEN1
sinaiEsEnExp1
sinaiEsEnExp2
sinaiEsEnExp3
sinaiDeEnExp2
Experiments
SMGeoESEN2
sinaiDeEnExp1
HIGeodeenrun11n
HIGeodeenrun12
HIGeodeenrun13n
HIGeodeenrun11
HIGeodeenrun13
0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65
arcsin(sqrt(Mean average precision))
57
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2EN-CLEF2006
GeoCLEF Bilingual English track Top 5 Participants − Retrieved documents vs Precision
100%
jaen [sinaiEsEnExp2; R−Prec 20.63%; Pooled]
sanmarcos [SMGeoESEN2; R−Prec 23.29%; Pooled]
90%
hildesheim [HIGeodeenrun12; R−Prec 17.52%; Pooled]
80%
70%
60%
R−Precision
50%
40%
30%
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
GeoCLEF Bilingual English track Top 5 Participants − Comparison to Median R−Precision by Topic (Topics 026 to 050)
1
jaen [sinaiEsEnExp2; R−Prec 20.63%; Pooled]
sanmarcos [SMGeoESEN2; R−Prec 23.29%; Pooled]
hildesheim [HIGeodeenrun12; R−Prec 17.52%; Pooled]
0.8
0.6
0.4
0.2
Difference
0
−0.2
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
58
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2EN-CLEF2006
GeoCLEF Bilingual English track − Box Plot of the Topics
SMGeoESEN1 [R−Prec 24.80%; Pooled]
sinaiEsEnExp1 [R−Prec 24.27%; Pooled]
SMGeoESEN2 [R−Prec 23.29%; Pooled]
sinaiEsEnExp2 [R−Prec 20.63%; Pooled]
sinaiEsEnExp3 [R−Prec 20.42%; Not Pooled]
sinaiDeEnExp2 [R−Prec 19.55%; Not Pooled]
Experiments
HIGeodeenrun11n [R−Prec 19.01%; Not Pooled]
HIGeodeenrun12 [R−Prec 17.52%; Pooled]
sinaiDeEnExp1 [R−Prec 16.49%; Not Pooled]
HIGeodeenrun11 [R−Prec 15.18%; Not Pooled]
HIGeodeenrun13n [R−Prec 14.83%; Pooled]
HIGeodeenrun13 [R−Prec 14.65%; Not Pooled]
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
R−Precision
59
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2EN-CLEF2006
GeoCLEF Bilingual English track − Tukey T test with "top group" highlighted
SMGeoESEN1
sinaiEsEnExp1
SMGeoESEN2
sinaiEsEnExp3
sinaiEsEnExp2
Experiments
sinaiDeEnExp2
HIGeodeenrun11n
sinaiDeEnExp1
HIGeodeenrun12
HIGeodeenrun11
HIGeodeenrun13
HIGeodeenrun13n
0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6
arcsin(sqrt(R Precision))
60
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2EN-CLEF2006
Average Precision R-Precision
Topic Minimum 1st Q. Median 3rd Q. Maximum Mean Std Minimum 1st Q. Median 3rd Q. Maximum Mean Std
026 0.0000 0.0090 0.0319 0.1993 0.2257 0.0831 0.0944 0.0000 0.0000 0.0556 0.2222 0.2222 0.0926 0.1042
027 0.0000 0.0000 0.0004 0.0281 0.0730 0.0166 0.0281 0.0000 0.0000 0.0000 0.0526 0.1053 0.0263 0.0420
028 0.0000 0.0008 0.1172 0.2284 0.3183 0.1204 0.1196 0.0000 0.0000 0.1316 0.2632 0.3684 0.1404 0.1425
029 0.0347 0.0486 0.0886 0.1245 0.2108 0.0948 0.0533 0.1111 0.1111 0.1111 0.2222 0.2222 0.1574 0.0572
030 0.0349 0.3559 0.6747 0.9623 1.0000 0.6142 0.3575 0.0000 0.4167 0.6667 0.8333 1.0000 0.5972 0.3368
031 0.1256 0.1606 0.1903 0.2383 0.4912 0.2272 0.1119 0.0678 0.1525 0.2203 0.3051 0.4746 0.2387 0.1208
032 0.4530 0.5124 0.8519 0.9554 0.9713 0.7503 0.2249 0.5806 0.5968 0.7742 0.8871 0.9355 0.7527 0.1445
033 0.0000 0.0000 0.0000 0.0035 0.0552 0.0065 0.0158 0.0000 0.0000 0.0000 0.0000 0.1500 0.0167 0.0444
034 0.0000 0.0883 0.2868 0.4205 0.4514 0.2609 0.1705 0.0000 0.0000 0.3333 0.3333 0.6667 0.2778 0.2392
035 0.0000 0.0005 0.0107 0.0229 0.0390 0.0127 0.0130 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
036 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
037 0.0000 0.0003 0.0015 0.0557 0.0966 0.0260 0.0425 0.0000 0.0000 0.0000 0.0938 0.1250 0.0365 0.0563
038 0.0000 0.0144 0.0577 0.1625 1.0000 0.1769 0.2945 0.0000 0.0000 0.0000 0.0000 1.0000 0.0833 0.2887
039 0.0046 0.0071 0.0494 0.1246 0.3686 0.0800 0.1070 0.0000 0.0000 0.0000 0.0938 0.3750 0.0573 0.1114
040 0.2523 0.2680 0.3227 0.7960 0.8837 0.5001 0.2738 0.1429 0.2143 0.3214 0.7143 0.7857 0.4345 0.2663
041 0.0000 0.0004 0.0027 0.0116 0.0225 0.0068 0.0084 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
042 0.0000 0.0009 0.0349 0.2763 0.5833 0.1352 0.1919 0.0000 0.0000 0.0000 0.2500 0.5000 0.1250 0.2261
043 0.0000 0.0001 0.0063 0.0194 0.0349 0.0102 0.0119 0.0000 0.0000 0.0000 0.0000 0.1250 0.0208 0.0487
044 0.0063 0.0300 0.0763 0.1244 0.2248 0.0858 0.0713 0.0000 0.0658 0.1053 0.1579 0.3421 0.1272 0.1015
045 0.0023 0.0032 0.0219 0.1364 0.3550 0.0814 0.1113 0.0000 0.0000 0.0000 0.1667 0.3333 0.0694 0.1114
046 0.0591 0.3195 0.3990 0.5678 0.9167 0.4226 0.2447 0.0000 0.3333 0.3333 0.6667 0.6667 0.3889 0.2392
047 0.0001 0.0204 0.0317 0.0423 0.0571 0.0305 0.0182 0.0000 0.0000 0.0417 0.0833 0.0833 0.0382 0.0375
048 0.6689 0.7143 0.8082 0.9132 0.9219 0.8110 0.1018 0.6250 0.6875 0.7396 0.8438 0.8542 0.7552 0.0871
049 0.0052 0.0595 0.2538 0.3667 0.6429 0.2625 0.2241 0.0000 0.0000 0.0000 0.2500 0.5000 0.1250 0.2261
050 0.0638 0.1723 0.1966 0.2315 0.2343 0.1910 0.0500 0.1333 0.2000 0.2667 0.2667 0.3333 0.2444 0.0519
ALL 0.1456 0.1584 0.2033 0.2251 0.2707 0.2003 0.0418 0.1465 0.1584 0.1928 0.2196 0.2480 0.1922 0.0361
61
62
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2ES-CLEF2006
GeoCLEF Bilingual Spanish track Top 5 Participants − Interpolated Recall vs Average Precision
100%
berkeley [BKGeoES1; MAP 25.71%; Pooled]
sanmarcos [SMGeoENES1; MAP 12.82%; Pooled]
90%
80%
70%
Average Precision
60%
50%
40%
30%
20%
10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
GeoCLEF Bilingual Spanish track Top 5 Participants − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
1
berkeley [BKGeoES1; MAP 25.71%; Pooled]
sanmarcos [SMGeoENES1; MAP 12.82%; Pooled]
0.8
0.6
0.4
0.2
Difference
0
−0.2
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
63
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2ES-CLEF2006
GeoCLEF Bilingual Spanish track − Box Plot of the Topics
BKGeoES2 [MAP 27.45%; Pooled]
BKGeoES1 [MAP 25.71%; Pooled]
Experiments
SMGeoENES1 [MAP 12.82%; Pooled]
SMGeoPTES3 [MAP 11.50%; Pooled]
SMGeoPTES2 [MAP 10.89%; Pooled]
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Mean Average Precision
64
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2ES-CLEF2006
GeoCLEF Bilingual Spanish track − Tukey T test with "top group" highlighted
BKGeoES2
BKGeoES1
Experiments
SMGeoENES1
SMGeoPTES3
SMGeoPTES2
0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55
arcsin(sqrt(Mean average precision))
65
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2ES-CLEF2006
GeoCLEF Bilingual Spanish track Top 5 Participants − Retrieved documents vs Precision
100%
berkeley [BKGeoES1; R−Prec 26.45%; Pooled]
sanmarcos [SMGeoENES1; R−Prec 16.89%; Pooled]
90%
80%
70%
60%
R−Precision
50%
40%
30%
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
GeoCLEF Bilingual Spanish track Top 5 Participants − Comparison to Median R−Precision by Topic (Topics 026 to 050)
1
berkeley [BKGeoES1; R−Prec 26.45%; Pooled]
sanmarcos [SMGeoENES1; R−Prec 16.89%; Pooled]
0.8
0.6
0.4
0.2
Difference
0
−0.2
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
66
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2ES-CLEF2006
GeoCLEF Bilingual Spanish track − Box Plot of the Topics
BKGeoES2 [R−Prec 27.04%; Pooled]
BKGeoES1 [R−Prec 26.45%; Pooled]
Experiments
SMGeoENES1 [R−Prec 16.89%; Pooled]
SMGeoPTES3 [R−Prec 15.27%; Pooled]
SMGeoPTES2 [R−Prec 14.67%; Pooled]
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
R−Precision
67
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2ES-CLEF2006
GeoCLEF Bilingual Spanish track − Tukey T test with "top group" highlighted
BKGeoES2
BKGeoES1
Experiments
SMGeoENES1
SMGeoPTES2
SMGeoPTES3
0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65
arcsin(sqrt(R Precision))
68
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2ES-CLEF2006
Average Precision R-Precision
Topic Minimum 1st Q. Median 3rd Q. Maximum Mean Std Minimum 1st Q. Median 3rd Q. Maximum Mean Std
026 0.0001 0.0009 0.0172 0.0186 0.0212 0.0115 0.0100 0.0000 0.0000 0.0000 0.0556 0.0556 0.0222 0.0304
027 0.0084 0.0174 0.0267 0.1205 0.3867 0.0948 0.1634 0.0000 0.0192 0.1282 0.1987 0.4103 0.1385 0.1628
028 0.0713 0.0784 0.1067 0.1477 0.2455 0.1239 0.0703 0.0556 0.1181 0.1389 0.1944 0.2778 0.1556 0.0800
029 0.0098 0.1970 0.2854 0.3630 0.4752 0.2711 0.1683 0.0303 0.2348 0.3030 0.3636 0.5455 0.2970 0.1823
030 0.0000 0.0000 0.0028 0.0350 0.1311 0.0274 0.0580 0.0000 0.0000 0.0265 0.0646 0.1788 0.0464 0.0752
031 0.0087 0.0098 0.0143 0.6322 0.6565 0.2628 0.3449 0.0510 0.0510 0.0667 0.6216 0.6745 0.2894 0.3204
032 0.1971 0.1980 0.2001 0.9751 0.9782 0.5096 0.4259 0.2077 0.2077 0.2077 0.9019 0.9077 0.4862 0.3813
033 0.0001 0.0005 0.0009 0.0168 0.0212 0.0076 0.0099 0.0000 0.0075 0.0100 0.0325 0.0400 0.0180 0.0164
034 0.0983 0.1267 0.1587 0.2176 0.2316 0.1676 0.0548 0.1351 0.1351 0.1892 0.3243 0.4054 0.2324 0.1172
035 0.0071 0.0170 0.0299 0.0549 0.0991 0.0393 0.0356 0.0000 0.0395 0.0526 0.0789 0.1579 0.0632 0.0577
036 0.0002 0.0051 0.0072 0.0497 0.1102 0.0308 0.0458 0.0000 0.0273 0.0364 0.1045 0.2273 0.0727 0.0893
037 0.0000 0.0004 0.0005 0.0757 0.1145 0.0357 0.0517 0.0000 0.0000 0.0000 0.0431 0.1724 0.0345 0.0771
038 0.0000 0.0000 0.0000 0.0236 0.0663 0.0151 0.0289 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
039 0.0098 0.1278 0.2701 0.3376 0.5358 0.2509 0.1918 0.0299 0.1866 0.2388 0.3172 0.5075 0.2537 0.1695
040 0.5335 0.5367 0.5413 0.5919 0.6760 0.5705 0.0601 0.6043 0.6043 0.6115 0.6349 0.6835 0.6245 0.0335
041 0.0086 0.0096 0.0100 0.1766 0.2243 0.0827 0.1027 0.0667 0.0867 0.0933 0.2267 0.3467 0.1573 0.1152
042 0.2006 0.2867 0.3164 0.3865 0.4796 0.3335 0.1001 0.2830 0.3538 0.3774 0.4245 0.5094 0.3887 0.0807
043 0.0000 0.0002 0.0003 0.0089 0.0318 0.0068 0.0140 0.0000 0.0000 0.0000 0.0104 0.0417 0.0083 0.0186
044 0.0283 0.0485 0.0584 0.2091 0.2954 0.1235 0.1126 0.0777 0.1286 0.1553 0.3058 0.3786 0.2078 0.1206
045 0.0138 0.0196 0.0221 0.0314 0.0567 0.0274 0.0168 0.0000 0.0000 0.0000 0.1042 0.1667 0.0500 0.0745
046 0.0519 0.0562 0.1107 0.6201 0.6369 0.2943 0.3035 0.0714 0.0714 0.1429 0.5536 0.6071 0.2857 0.2637
047 0.0097 0.0262 0.0337 0.1236 0.2860 0.0861 0.1138 0.0169 0.0297 0.0339 0.1737 0.3390 0.1085 0.1348
048 0.0565 0.0567 0.0581 0.7562 0.8280 0.3463 0.3975 0.0755 0.0755 0.0755 0.6575 0.7396 0.3192 0.3360
049 0.4566 0.4982 0.5745 0.7545 0.7874 0.6148 0.1445 0.5576 0.5853 0.6682 0.6982 0.7051 0.6442 0.0650
050 0.0401 0.0491 0.0528 0.1212 0.1796 0.0853 0.0578 0.0400 0.0700 0.1000 0.1450 0.2200 0.1120 0.0672
ALL 0.1089 0.1135 0.1282 0.2615 0.2745 0.1768 0.0818 0.1467 0.1512 0.1689 0.2660 0.2704 0.2006 0.0615
69
70
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2PT-CLEF2006
GeoCLEF Bilingual Portuguese track Top 5 Participants − Interpolated Recall vs Average Precision
100%
sanmarcos [SMGeoESPT2; MAP 14.16%; Pooled]
berkeley [BKGeoEP1; MAP 12.60%; Pooled]
90%
80%
70%
Average Precision
60%
50%
40%
30%
20%
10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
GeoCLEF Bilingual Portuguese track Top 5 Participants − Comparison to Median Mean Average Precision by Topic (Topics 026 to 5
00)
1
sanmarcos [SMGeoESPT2; MAP 14.16%; Pooled]
berkeley [BKGeoEP1; MAP 12.60%; Pooled]
0.8
0.6
0.4
0.2
Difference
0
−0.2
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
71
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2PT-CLEF2006
GeoCLEF Bilingual Portuguese track − Box Plot of the Topics
BKGeoEP2 [MAP 14.30%; Pooled]
SMGeoESPT2 [MAP 14.16%; Pooled]
Experiments
SMGeoESPT1 [MAP 12.81%; Pooled]
BKGeoEP1 [MAP 12.60%; Pooled]
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Mean Average Precision
72
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2PT-CLEF2006
GeoCLEF Bilingual Portuguese track − Tukey T test with "top group" highlighted
SMGeoESPT2
SMGeoESPT1
Experiments
BKGeoEP2
BKGeoEP1
0.2 0.22 0.24 0.26 0.28 0.3 0.32 0.34 0.36 0.38 0.4
arcsin(sqrt(Mean average precision))
73
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2PT-CLEF2006
GeoCLEF Bilingual Portuguese track Top 5 Participants − Retrieved documents vs Precision
100%
sanmarcos [SMGeoESPT2; R−Prec 17.42%; Pooled]
berkeley [BKGeoEP1; R−Prec 14.77%; Pooled]
90%
80%
70%
60%
R−Precision
50%
40%
30%
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
GeoCLEF Bilingual Portuguese track Top 5 Participants − Comparison to Median R−Precision by Topic (Topics 026 to 050)
1
sanmarcos [SMGeoESPT2; R−Prec 17.42%; Pooled]
berkeley [BKGeoEP1; R−Prec 14.77%; Pooled]
0.8
0.6
0.4
0.2
Difference
0
−0.2
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
74
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2PT-CLEF2006
GeoCLEF Bilingual Portuguese track − Box Plot of the Topics
SMGeoESPT2 [R−Prec 17.42%; Pooled]
BKGeoEP2 [R−Prec 16.34%; Pooled]
Experiments
SMGeoESPT1 [R−Prec 14.88%; Pooled]
BKGeoEP1 [R−Prec 14.77%; Pooled]
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
R−Precision
75
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2PT-CLEF2006
GeoCLEF Bilingual Portuguese track − Tukey T test with "top group" highlighted
SMGeoESPT2
SMGeoESPT1
Experiments
BKGeoEP2
BKGeoEP1
0.2 0.25 0.3 0.35 0.4 0.45
arcsin(sqrt(R Precision))
76
GC-BILI-CLEF2006 Track Overview Results and Graphs GC-BILI-X2PT-CLEF2006
Average Precision R-Precision
Topic Minimum 1st Q. Median 3rd Q. Maximum Mean Std Minimum 1st Q. Median 3rd Q. Maximum Mean Std
026 0.0117 0.0120 0.0125 0.0453 0.0779 0.0286 0.0328 0.0000 0.0000 0.0000 0.0333 0.0667 0.0167 0.0333
027 0.0003 0.0009 0.0357 0.0797 0.0897 0.0403 0.0462 0.0000 0.0196 0.0833 0.1667 0.2059 0.0931 0.0921
028 0.0854 0.1041 0.1271 0.1860 0.2405 0.1450 0.0667 0.1562 0.1875 0.2188 0.2812 0.3438 0.2344 0.0786
029 0.0686 0.1525 0.2534 0.3019 0.3334 0.2272 0.1131 0.0769 0.1923 0.3205 0.3590 0.3846 0.2756 0.1363
030 0.0069 0.0075 0.0122 0.2015 0.3866 0.1045 0.1881 0.0000 0.0000 0.0357 0.2500 0.4286 0.1250 0.2052
031 0.1438 0.1526 0.1887 0.3268 0.4374 0.2397 0.1354 0.1311 0.1475 0.1885 0.3033 0.3934 0.2254 0.1170
032 0.0776 0.1098 0.3201 0.6281 0.7579 0.3689 0.3186 0.2075 0.2358 0.4245 0.6698 0.7547 0.4528 0.2610
033 0.0000 0.0000 0.0009 0.0067 0.0116 0.0033 0.0056 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
034 0.0014 0.0015 0.0120 0.0272 0.0321 0.0143 0.0154 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
035 0.0014 0.0035 0.0151 0.0262 0.0276 0.0148 0.0133 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
036 0.0000 0.0000 0.0096 0.0240 0.0288 0.0120 0.0144 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
037 0.0007 0.0011 0.0072 0.0150 0.0172 0.0081 0.0082 0.0000 0.0000 0.0000 0.0278 0.0556 0.0139 0.0278
038 0.1435 0.1530 0.1745 0.2437 0.3010 0.1984 0.0707 0.1250 0.1250 0.1875 0.2500 0.2500 0.1875 0.0722
039 0.0545 0.0685 0.0861 0.1963 0.3028 0.1324 0.1146 0.0870 0.1087 0.1304 0.1957 0.2609 0.1522 0.0753
040 0.0001 0.0001 0.0071 0.0213 0.0284 0.0107 0.0135 0.0000 0.0000 0.0208 0.0833 0.1250 0.0417 0.0589
041 0.0000 0.0001 0.0057 0.0113 0.0115 0.0057 0.0065 0.0000 0.0000 0.0144 0.0288 0.0288 0.0144 0.0167
042 0.1065 0.1826 0.3486 0.4673 0.4961 0.3250 0.1773 0.2000 0.2571 0.3714 0.4571 0.4857 0.3571 0.1267
043 0.0002 0.0007 0.0129 0.0436 0.0626 0.0221 0.0292 0.0000 0.0000 0.0208 0.0625 0.0833 0.0313 0.0399
044 0.0000 0.0000 0.0387 0.0786 0.0797 0.0393 0.0454 0.0000 0.0000 0.0461 0.0987 0.1053 0.0493 0.0572
045 0.0295 0.1339 0.2418 0.2596 0.2739 0.1967 0.1125 0.0854 0.2195 0.3537 0.3598 0.3659 0.2896 0.1363
046 0.0922 0.1758 0.2996 0.4281 0.5165 0.3020 0.1763 0.1667 0.2500 0.3561 0.4621 0.5455 0.3561 0.1557
047 0.0013 0.0155 0.0419 0.0713 0.0884 0.0434 0.0370 0.0000 0.0000 0.0294 0.0735 0.0882 0.0368 0.0441
048 0.0229 0.2711 0.6083 0.7453 0.7932 0.5082 0.3428 0.0979 0.3147 0.5874 0.6888 0.7343 0.5017 0.2817
049 0.0880 0.1145 0.1924 0.3217 0.3997 0.2181 0.1372 0.0556 0.1667 0.2778 0.3611 0.4444 0.2639 0.1596
050 0.1034 0.1173 0.1441 0.1986 0.2403 0.1580 0.0591 0.1591 0.1705 0.2273 0.3182 0.3636 0.2443 0.0935
ALL 0.1260 0.1271 0.1348 0.1423 0.1430 0.1347 0.0088 0.1477 0.1482 0.1561 0.1688 0.1742 0.1585 0.0127
77
78
Individual Experiment Results and Graphs
79
80
berkeley BKGeoD2 GC-MONO-DE-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language German
Relevant 602 Topic Fields title, description, narrative
Relevant retrieved 490 Pooled true
Geometric Mean Average Precision 0.0434 German topics TDN with blind feedback
Binary Preference (BPREF) 0.1665
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision
100%
0 34.92
BKGeoD2
10 32.21 90%
20 28.25
80%
30 22.43
40 18.05 70%
50 17.57
Average Precision
60%
60 17.00
70 15.88 50%
80 13.29 40%
90 9.95
30%
100 2.16
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
18.22 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.9069
Minimum 0.0000
First Quartile 0.0160
Second Quartile 0.0682
Third Quartile 0.2145
Interquartile range 0.1986
Mean 0.1822
Standard Deviation 0.2623
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4016 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0965
Std With No Outliers 0.1178
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries BKGeoD2
Topic 026 0.36 Topic 039 16.21 0.8
Topic 027 2.34 Topic 040 40.16
Topic 028 12.02 Topic 041 18.63
0.6
Topic 029 34.86 Topic 042 12.73
Topic 030 78.49 Topic 043 3.34
Topic 031 6.82 Topic 044 0.23 0.4
Topic 032 73.90 Topic 045 4.61
Topic 033 0.37 Topic 046 11.88 0.2
Topic 034 29.92 Topic 047 8.61
Difference
Topic 035 1.76 Topic 048 90.69 0
Topic 036 0.80 Topic 049 3.74
Topic 037 0.00 Topic 050 1.11
−0.2
Topic 038 1.85
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
81
berkeley BKGeoD2 GC-MONO-DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual German track − Retrieved documents vs Precision
100%
5 docs 21.60
BKGeoD2
10 docs 20.00 90%
15 docs 20.53
80%
20 docs 19.40
30 docs 17.60 70%
100 docs 11.60
60%
200 docs 6.66
R−Precision
500 docs 3.33 50%
1000 docs 1.96 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
18.57
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.8841
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0556
Third Quartile 0.2500
Interquartile range 0.2500
Mean 0.1857
Standard Deviation 0.2650
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4390 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1014
Std With No Outliers 0.1337
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries BKGeoD2
Topic 026 0.00 Topic 039 11.11 0.8
Topic 027 10.77 Topic 040 43.90
Topic 028 25.00 Topic 041 21.05
0.6
Topic 029 33.33 Topic 042 21.28
Topic 030 76.67 Topic 043 0.00
Topic 031 2.63 Topic 044 0.00 0.4
Topic 032 75.93 Topic 045 0.00
Topic 033 0.00 Topic 046 25.00 0.2
Topic 034 23.53 Topic 047 0.00
Difference
Topic 035 5.56 Topic 048 88.41 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
82
berkeley BKGeoD1 GC-MONO-DE-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language German
Relevant 602 Topic Fields title, description
Relevant retrieved 435 Pooled true
Geometric Mean Average Precision 0.0466 German automatic TD with blind feedback
Binary Preference (BPREF) 0.1936
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision
100%
0 44.24
BKGeoD1
10 34.84 90%
20 33.46
80%
30 27.55
40 22.05 70%
50 21.30
Average Precision
60%
60 18.54
70 16.72 50%
80 14.63 40%
90 12.13
30%
100 4.05
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
21.51 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.8944
Minimum 0.0001
First Quartile 0.0128
Second Quartile 0.0610
Third Quartile 0.3398
Interquartile range 0.3270
Mean 0.2151
Standard Deviation 0.2827
Lower Outlier Threshold 0.0001
Upper Outlier Threshold 0.7862 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1868
Std With No Outliers 0.2500
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries BKGeoD1
Topic 026 0.08 Topic 039 0.04 0.8
Topic 027 1.37 Topic 040 33.63
Topic 028 39.16 Topic 041 21.84
0.6
Topic 029 35.02 Topic 042 1.68
Topic 030 78.62 Topic 043 0.93
Topic 031 2.13 Topic 044 0.50 0.4
Topic 032 78.61 Topic 045 2.69
Topic 033 8.34 Topic 046 27.94 0.2
Topic 034 67.04 Topic 047 6.10
Difference
Topic 035 12.31 Topic 048 89.44 0
Topic 036 1.03 Topic 049 5.17
Topic 037 0.01 Topic 050 3.33
−0.2
Topic 038 20.81
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
83
berkeley BKGeoD1 GC-MONO-DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual German track − Retrieved documents vs Precision
100%
5 docs 25.60
BKGeoD1
10 docs 24.40 90%
15 docs 22.13
80%
20 docs 20.80
30 docs 19.07 70%
100 docs 11.36
60%
200 docs 6.52
R−Precision
500 docs 3.09 50%
1000 docs 1.74 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
19.99
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.8696
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0462
Third Quartile 0.3476
Interquartile range 0.3476
Mean 0.1999
Standard Deviation 0.2812
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8148 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1720
Std With No Outliers 0.2494
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries BKGeoD1
Topic 026 0.00 Topic 039 0.00 0.8
Topic 027 4.62 Topic 040 39.02
Topic 028 43.75 Topic 041 21.05
0.6
Topic 029 33.33 Topic 042 0.00
Topic 030 70.00 Topic 043 0.00
Topic 031 1.32 Topic 044 0.00 0.4
Topic 032 81.48 Topic 045 0.00
Topic 033 17.65 Topic 046 25.00 0.2
Topic 034 61.76 Topic 047 0.00
Difference
Topic 035 5.56 Topic 048 86.96 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 8.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
84
daedalus GCdeNtLg GC-MONO-DE-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 13,055 Source Language German
Relevant 523 Topic Fields title, description
Relevant retrieved 256 Pooled true
Geometric Mean Average Precision 0.0152 Normal text Left geo run
Binary Preference (BPREF) 0.0876
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision
100%
0 34.44
GCdeNtLg
10 20.62 90%
20 18.05
80%
30 12.74
40 11.81 70%
50 9.92
Average Precision
60%
60 5.87
70 4.39 50%
80 3.59 40%
90 2.11
30%
100 2.07
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
10.01 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.5258
Minimum 0.0000
First Quartile 0.0150
Second Quartile 0.0411
Third Quartile 0.1501
Interquartile range 0.1351
Mean 0.1001
Standard Deviation 0.1364
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.2404 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0667
Std With No Outliers 0.0748
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries GCdeNtLg
Topic 026 0.00 Topic 039 1.73 0.8
Topic 027 0.00 Topic 040 13.83
Topic 028 24.04 Topic 041 2.14
0.6
Topic 029 4.17 Topic 042 4.11
Topic 030 44.21 Topic 043 0.55
Topic 031 3.80 Topic 044 4.96 0.4
Topic 032 1.85 Topic 045 0.00
Topic 033 2.96 Topic 046 18.29 0.2
Topic 034 52.58 Topic 047 3.16
Difference
Topic 035 6.74 Topic 048 18.82 0
Topic 036 0.79 Topic 049 7.86
Topic 037 19.80 Topic 050 0.00
−0.2
Topic 038 13.91
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
85
daedalus GCdeNtLg GC-MONO-DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual German track − Retrieved documents vs Precision
100%
5 docs 13.60
GCdeNtLg
10 docs 15.20 90%
15 docs 15.20
80%
20 docs 15.40
30 docs 13.20 70%
100 docs 6.60
60%
200 docs 4.10
R−Precision
500 docs 1.91 50%
1000 docs 1.02 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
11.53
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.5588
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0395
Third Quartile 0.1866
Interquartile range 0.1866
Mean 0.1153
Standard Deviation 0.1621
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3125 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0778
Std With No Outliers 0.1016
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries GCdeNtLg
Topic 026 0.00 Topic 039 3.70 0.8
Topic 027 0.00 Topic 040 26.83
Topic 028 31.25 Topic 041 10.53
0.6
Topic 029 0.00 Topic 042 8.51
Topic 030 53.33 Topic 043 0.00
Topic 031 3.95 Topic 044 16.67 0.4
Topic 032 1.85 Topic 045 0.00
Topic 033 5.88 Topic 046 25.00 0.2
Topic 034 55.88 Topic 047 0.00
Difference
Topic 035 11.11 Topic 048 24.64 0
Topic 036 0.00 Topic 049 0.00
Topic 037 9.09 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
86
daedalus GCdeAA GC-MONO-DE-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction MANUAL
Retrieved 793 Source Language German
Relevant 535 Topic Fields title, description, narrative
Relevant retrieved 80 Pooled true
Geometric Mean Average Precision 0.0026 All text And geo run
Binary Preference (BPREF) 0.0734
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision
100%
0 32.83
GCdeAA
10 24.61 90%
20 9.71
80%
30 9.01
40 6.13 70%
50 5.08
Average Precision
60%
60 3.79
70 0.19 50%
80 0.19 40%
90 0.19
30%
100 0.19
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
7.15 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.5862
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0323
Third Quartile 0.1003
Interquartile range 0.1003
Mean 0.0715
Standard Deviation 0.1222
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1850 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0501
Std With No Outliers 0.0600
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries GCdeAA
Topic 026 0.00 Topic 039 11.55 0.8
Topic 027 0.00 Topic 040 0.00
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 16.67 Topic 042 7.97
Topic 030 58.62 Topic 043 3.23
Topic 031 3.20 Topic 044 0.54 0.4
Topic 032 0.23 Topic 045 9.52
Topic 033 11.76 Topic 046 0.00 0.2
Topic 034 0.00 Topic 047 3.33
Difference
Topic 035 0.00 Topic 048 18.50 0
Topic 036 6.67 Topic 049 0.00
Topic 037 15.15 Topic 050 4.14
−0.2
Topic 038 7.67
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
87
daedalus GCdeAA GC-MONO-DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual German track − Retrieved documents vs Precision
100%
5 docs 16.80
GCdeAA
10 docs 16.80 90%
15 docs 14.40
80%
20 docs 12.20
30 docs 8.53 70%
100 docs 3.16
60%
200 docs 1.60
R−Precision
500 docs 0.64 50%
1000 docs 0.32 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
8.93
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.6000
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.1534
Interquartile range 0.1534
Mean 0.0893
Standard Deviation 0.1400
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0680
Std With No Outliers 0.0930
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries GCdeAA
Topic 026 0.00 Topic 039 18.52 0.8
Topic 027 0.00 Topic 040 0.00
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 33.33 Topic 042 14.89
Topic 030 60.00 Topic 043 14.29
Topic 031 6.58 Topic 044 0.00 0.4
Topic 032 1.85 Topic 045 0.00
Topic 033 11.76 Topic 046 0.00 0.2
Topic 034 0.00 Topic 047 16.67
Difference
Topic 035 0.00 Topic 048 18.84 0
Topic 036 0.00 Topic 049 0.00
Topic 037 18.18 Topic 050 8.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
88
daedalus GCdeAtLg GC-MONO-DE-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction MANUAL
Retrieved 20,707 Source Language German
Relevant 535 Topic Fields title, description, narrative
Relevant retrieved 208 Pooled true
Geometric Mean Average Precision 0.0122 All text Left geo run
Binary Preference (BPREF) 0.0657
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision
100%
0 30.53
GCdeAtLg
10 17.52 90%
20 10.02
80%
30 9.06
40 8.46 70%
50 7.83
Average Precision
60%
60 6.07
70 2.25 50%
80 1.43 40%
90 1.11
30%
100 0.52
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
7.36 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.5909
Minimum 0.0000
First Quartile 0.0030
Second Quartile 0.0306
Third Quartile 0.0838
Interquartile range 0.0809
Mean 0.0736
Standard Deviation 0.1227
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1658 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0443
Std With No Outliers 0.0473
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries GCdeAtLg
Topic 026 0.00 Topic 039 6.04 0.8
Topic 027 0.00 Topic 040 16.58
Topic 028 14.08 Topic 041 2.14
0.6
Topic 029 0.11 Topic 042 8.66
Topic 030 59.09 Topic 043 3.06
Topic 031 2.30 Topic 044 1.06 0.4
Topic 032 1.85 Topic 045 7.92
Topic 033 0.32 Topic 046 0.00 0.2
Topic 034 1.53 Topic 047 6.33
Difference
Topic 035 0.21 Topic 048 9.50 0
Topic 036 0.21 Topic 049 4.90
Topic 037 23.03 Topic 050 6.73
−0.2
Topic 038 8.29
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
89
daedalus GCdeAtLg GC-MONO-DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual German track − Retrieved documents vs Precision
100%
5 docs 16.80
GCdeAtLg
10 docs 12.40 90%
15 docs 11.73
80%
20 docs 11.60
30 docs 9.47 70%
100 docs 5.00
60%
200 docs 3.36
R−Precision
500 docs 1.56 50%
1000 docs 0.83 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
8.78
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.6000
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0185
Third Quartile 0.1516
Interquartile range 0.1516
Mean 0.0878
Standard Deviation 0.1332
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.2500 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0664
Std With No Outliers 0.0815
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries GCdeAtLg
Topic 026 0.00 Topic 039 14.81 0.8
Topic 027 0.00 Topic 040 19.51
Topic 028 25.00 Topic 041 10.53
0.6
Topic 029 0.00 Topic 042 14.89
Topic 030 60.00 Topic 043 7.14
Topic 031 6.58 Topic 044 0.00 0.4
Topic 032 1.85 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 0.00 Topic 047 16.67
Difference
Topic 035 0.00 Topic 048 15.94 0
Topic 036 0.00 Topic 049 0.00
Topic 037 18.18 Topic 050 8.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
90
daedalus GCdeNA GC-MONO-DE-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction MANUAL
Retrieved 4,128 Source Language German
Relevant 523 Topic Fields title, description
Relevant retrieved 152 Pooled true
Geometric Mean Average Precision 0.0032 run mandatory
Binary Preference (BPREF) 0.0851
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision
100%
0 34.22
GCdeNA
10 24.65 90%
20 17.18
80%
30 10.70
40 8.81 70%
50 8.53
Average Precision
60%
60 4.58
70 3.81 50%
80 2.07 40%
90 1.80
30%
100 1.76
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
9.28 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.5258
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0106
Third Quartile 0.1438
Interquartile range 0.1438
Mean 0.0928
Standard Deviation 0.1442
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.2392 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0590
Std With No Outliers 0.0870
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries GCdeNA
Topic 026 0.00 Topic 039 0.93 0.8
Topic 027 0.00 Topic 040 0.41
Topic 028 23.20 Topic 041 0.00
0.6
Topic 029 3.33 Topic 042 1.06
Topic 030 43.75 Topic 043 0.16
Topic 031 4.83 Topic 044 20.00 0.4
Topic 032 0.31 Topic 045 0.00
Topic 033 11.76 Topic 046 12.50 0.2
Topic 034 52.58 Topic 047 0.00
Difference
Topic 035 7.20 Topic 048 23.92 0
Topic 036 0.00 Topic 049 0.00
Topic 037 2.27 Topic 050 0.00
−0.2
Topic 038 23.81
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
91
daedalus GCdeNA GC-MONO-DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual German track − Retrieved documents vs Precision
100%
5 docs 16.80
GCdeNA
10 docs 16.80 90%
15 docs 14.93
80%
20 docs 13.60
30 docs 10.80 70%
100 docs 4.52
60%
200 docs 2.78
R−Precision
500 docs 1.21 50%
1000 docs 0.61 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
10.19
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.5588
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0213
Third Quartile 0.1299
Interquartile range 0.1299
Mean 0.1019
Standard Deviation 0.1657
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.2500 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0491
Std With No Outliers 0.0783
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries GCdeNA
Topic 026 0.00 Topic 039 3.70 0.8
Topic 027 0.00 Topic 040 2.44
Topic 028 37.50 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 2.13
Topic 030 53.33 Topic 043 0.00
Topic 031 5.26 Topic 044 16.67 0.4
Topic 032 1.85 Topic 045 0.00
Topic 033 11.76 Topic 046 25.00 0.2
Topic 034 55.88 Topic 047 0.00
Difference
Topic 035 5.56 Topic 048 24.64 0
Topic 036 0.00 Topic 049 0.00
Topic 037 9.09 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
92
daedalus GCdeAO GC-MONO-DE-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction MANUAL
Retrieved 24,328 Source Language German
Relevant 602 Topic Fields title, description, narrative
Relevant retrieved 259 Pooled true
Geometric Mean Average Precision 0.0125 All text Or geo run
Binary Preference (BPREF) 0.0442
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision
100%
0 23.17
GCdeAO
10 11.94 90%
20 8.26
80%
30 7.21
40 6.02 70%
50 5.35
Average Precision
60%
60 3.78
70 3.26 50%
80 2.67 40%
90 1.59
30%
100 0.40
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
5.48 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.3825
Minimum 0.0000
First Quartile 0.0055
Second Quartile 0.0238
Third Quartile 0.0703
Interquartile range 0.0647
Mean 0.0548
Standard Deviation 0.0809
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1658 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0411
Std With No Outliers 0.0443
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries GCdeAO
Topic 026 1.35 Topic 039 6.04 0.8
Topic 027 1.65 Topic 040 16.58
Topic 028 0.00 Topic 041 1.12
0.6
Topic 029 0.11 Topic 042 5.22
Topic 030 12.56 Topic 043 3.06
Topic 031 2.30 Topic 044 0.66 0.4
Topic 032 38.25 Topic 045 7.92
Topic 033 2.38 Topic 046 0.00 0.2
Topic 034 1.53 Topic 047 6.33
Difference
Topic 035 0.06 Topic 048 9.50 0
Topic 036 0.13 Topic 049 4.90
Topic 037 0.23 Topic 050 6.73
−0.2
Topic 038 8.29
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
93
daedalus GCdeAO GC-MONO-DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual German track − Retrieved documents vs Precision
100%
5 docs 9.60
GCdeAO
10 docs 7.60 90%
15 docs 6.40
80%
20 docs 6.80
30 docs 7.33 70%
100 docs 6.04
60%
200 docs 4.10
R−Precision
500 docs 1.92 50%
1000 docs 1.04 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
6.62
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.4630
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.1096
Interquartile range 0.1096
Mean 0.0662
Standard Deviation 0.1056
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1951 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0497
Std With No Outliers 0.0671
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries GCdeAO
Topic 026 0.00 Topic 039 14.81 0.8
Topic 027 13.85 Topic 040 19.51
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 6.38
Topic 030 10.00 Topic 043 7.14
Topic 031 6.58 Topic 044 0.00 0.4
Topic 032 46.30 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 0.00 Topic 047 16.67
Difference
Topic 035 0.00 Topic 048 15.94 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 8.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
94
hagen FUHddGYYYTD GC-MONO-DE-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 24,118 Source Language German
Relevant 602 Topic Fields title, description
Relevant retrieved 449 Pooled true
Geometric Mean Average Precision 0.0413 second run
Binary Preference (BPREF) 0.2011
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision
100%
0 41.56
FUHddGYYYTD
10 37.05 90%
20 34.25
80%
30 30.28
40 25.63 70%
50 24.85
Average Precision
60%
60 18.58
70 14.71 50%
80 12.24 40%
90 9.42
30%
100 1.79
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
22.29 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.9085
Minimum 0.0000
First Quartile 0.0095
Second Quartile 0.0307
Third Quartile 0.4244
Interquartile range 0.4149
Mean 0.2229
Standard Deviation 0.2992
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.9085 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2229
Std With No Outliers 0.2992
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries FUHddGYYYTD
Topic 026 0.12 Topic 039 7.76 0.8
Topic 027 1.14 Topic 040 32.59
Topic 028 0.78 Topic 041 6.26
0.6
Topic 029 1.93 Topic 042 0.62
Topic 030 69.84 Topic 043 0.97
Topic 031 82.49 Topic 044 2.55 0.4
Topic 032 63.16 Topic 045 62.50
Topic 033 4.02 Topic 046 33.63 0.2
Topic 034 44.77 Topic 047 0.54
Difference
Topic 035 3.07 Topic 048 90.85 0
Topic 036 41.67 Topic 049 0.00
Topic 037 2.41 Topic 050 2.82
−0.2
Topic 038 0.87
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
95
hagen FUHddGYYYTD GC-MONO-DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual German track − Retrieved documents vs Precision
100%
5 docs 24.00
FUHddGYYYTD
10 docs 21.60 90%
15 docs 21.33
80%
20 docs 21.00
30 docs 19.60 70%
100 docs 12.04
60%
200 docs 6.92
R−Precision
500 docs 3.22 50%
1000 docs 1.80 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
21.53
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.8841
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0741
Third Quartile 0.4467
Interquartile range 0.4467
Mean 0.2153
Standard Deviation 0.2865
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8841 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2153
Std With No Outliers 0.2865
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries FUHddGYYYTD
Topic 026 0.00 Topic 039 7.41 0.8
Topic 027 7.69 Topic 040 46.34
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 0.00
Topic 030 66.67 Topic 043 0.00
Topic 031 81.58 Topic 044 0.00 0.4
Topic 032 59.26 Topic 045 50.00
Topic 033 11.76 Topic 046 25.00 0.2
Topic 034 44.12 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 88.41 0
Topic 036 33.33 Topic 049 0.00
Topic 037 0.00 Topic 050 16.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
96
hagen FUHddGNNNTD GC-MONO-DE-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 23,756 Source Language German
Relevant 602 Topic Fields title, description
Relevant retrieved 439 Pooled true
Geometric Mean Average Precision 0.0510 fourth run
Binary Preference (BPREF) 0.1574
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision
100%
0 41.78
FUHddGNNNTD
10 35.24 90%
20 30.43
80%
30 24.74
40 18.09 70%
50 16.62
Average Precision
60%
60 13.78
70 8.27 50%
80 5.16 40%
90 3.98
30%
100 0.18
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
16.94 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.7469
Minimum 0.0000
First Quartile 0.0176
Second Quartile 0.0528
Third Quartile 0.2886
Interquartile range 0.2710
Mean 0.1694
Standard Deviation 0.2161
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5884 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1453
Std With No Outliers 0.1834
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries FUHddGNNNTD
Topic 026 1.72 Topic 039 2.60 0.8
Topic 027 3.59 Topic 040 31.10
Topic 028 2.89 Topic 041 1.10
0.6
Topic 029 9.86 Topic 042 1.39
Topic 030 58.84 Topic 043 1.15
Topic 031 18.85 Topic 044 33.40 0.4
Topic 032 58.23 Topic 045 0.63
Topic 033 9.85 Topic 046 28.11 0.2
Topic 034 44.77 Topic 047 3.53
Difference
Topic 035 5.28 Topic 048 74.69 0
Topic 036 3.23 Topic 049 1.77
Topic 037 19.30 Topic 050 7.55
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
97
hagen FUHddGNNNTD GC-MONO-DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual German track − Retrieved documents vs Precision
100%
5 docs 24.00
FUHddGNNNTD
10 docs 20.80 90%
15 docs 21.07
80%
20 docs 19.20
30 docs 18.40 70%
100 docs 9.96
60%
200 docs 6.20
R−Precision
500 docs 3.14 50%
1000 docs 1.76 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
18.00
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.6333
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0714
Third Quartile 0.2906
Interquartile range 0.2906
Mean 0.1800
Standard Deviation 0.2161
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1800
Std With No Outliers 0.2161
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries FUHddGNNNTD
Topic 026 0.00 Topic 039 3.70 0.8
Topic 027 13.85 Topic 040 48.78
Topic 028 6.25 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 4.26
Topic 030 63.33 Topic 043 7.14
Topic 031 27.63 Topic 044 33.33 0.4
Topic 032 57.41 Topic 045 0.00
Topic 033 11.76 Topic 046 25.00 0.2
Topic 034 44.12 Topic 047 0.00
Difference
Topic 035 5.56 Topic 048 62.32 0
Topic 036 0.00 Topic 049 0.00
Topic 037 27.27 Topic 050 8.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
98
hagen FUHddGNNNTDN GC-MONO-DE-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language German
Relevant 602 Topic Fields title, description, narrative
Relevant retrieved 426 Pooled true
Geometric Mean Average Precision 0.0385 third run
Binary Preference (BPREF) 0.1134
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision
100%
0 32.25
FUHddGNNNTDN
10 24.35 90%
20 18.96
80%
30 16.61
40 15.05 70%
50 13.18
Average Precision
60%
60 10.26
70 7.12 50%
80 5.24 40%
90 3.75
30%
100 0.57
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
12.23 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.6954
Minimum 0.0034
First Quartile 0.0123
Second Quartile 0.0298
Third Quartile 0.1342
Interquartile range 0.1218
Mean 0.1223
Standard Deviation 0.2016
Lower Outlier Threshold 0.0034
Upper Outlier Threshold 0.2754 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0558
Std With No Outliers 0.0780
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries FUHddGNNNTDN
Topic 026 0.36 Topic 039 26.16 0.8
Topic 027 2.47 Topic 040 27.54
Topic 028 0.91 Topic 041 3.52
0.6
Topic 029 4.45 Topic 042 2.59
Topic 030 45.60 Topic 043 6.17
Topic 031 5.58 Topic 044 1.38 0.4
Topic 032 67.99 Topic 045 0.34
Topic 033 2.03 Topic 046 13.95 0.2
Topic 034 13.24 Topic 047 0.66
Difference
Topic 035 2.98 Topic 048 69.54 0
Topic 036 1.65 Topic 049 3.41
Topic 037 1.28 Topic 050 0.95
−0.2
Topic 038 1.11
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
99
hagen FUHddGNNNTDN GC-MONO-DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual German track − Retrieved documents vs Precision
100%
5 docs 19.20
FUHddGNNNTDN
10 docs 19.60 90%
15 docs 17.60
80%
20 docs 15.60
30 docs 14.00 70%
100 docs 8.84
60%
200 docs 5.58
R−Precision
500 docs 2.86 50%
1000 docs 1.70 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
13.40
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.6852
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0312
Third Quartile 0.2100
Interquartile range 0.2100
Mean 0.1340
Standard Deviation 0.2037
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0875
Std With No Outliers 0.1304
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries FUHddGNNNTDN
Topic 026 0.00 Topic 039 22.22 0.8
Topic 027 4.62 Topic 040 34.15
Topic 028 3.12 Topic 041 10.53
0.6
Topic 029 0.00 Topic 042 4.26
Topic 030 46.67 Topic 043 14.29
Topic 031 15.79 Topic 044 0.00 0.4
Topic 032 68.52 Topic 045 0.00
Topic 033 0.00 Topic 046 25.00 0.2
Topic 034 20.59 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 65.22 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
100
hagen FUHddGYYYTDN GC-MONO-DE-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 24,363 Source Language German
Relevant 602 Topic Fields title, description, narrative
Relevant retrieved 462 Pooled true
Geometric Mean Average Precision 0.0337 first run
Binary Preference (BPREF) 0.1929
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision
100%
0 44.57
FUHddGYYYTDN
10 35.11 90%
20 32.13
80%
30 27.93
40 23.36 70%
50 22.49
Average Precision
60%
60 19.04
70 15.51 50%
80 12.08 40%
90 9.05
30%
100 1.16
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
21.41 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.9161
Minimum 0.0000
First Quartile 0.0086
Second Quartile 0.0478
Third Quartile 0.3660
Interquartile range 0.3574
Mean 0.2141
Standard Deviation 0.2865
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8249 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1849
Std With No Outliers 0.2517
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries FUHddGYYYTDN
Topic 026 0.13 Topic 039 34.91 0.8
Topic 027 1.14 Topic 040 32.59
Topic 028 1.74 Topic 041 6.26
0.6
Topic 029 1.93 Topic 042 4.07
Topic 030 67.56 Topic 043 5.87
Topic 031 82.49 Topic 044 0.91 0.4
Topic 032 63.16 Topic 045 4.78
Topic 033 11.66 Topic 046 33.63 0.2
Topic 034 44.77 Topic 047 0.00
Difference
Topic 035 3.07 Topic 048 91.61 0
Topic 036 41.67 Topic 049 0.58
Topic 037 0.02 Topic 050 0.09
−0.2
Topic 038 0.72
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
101
hagen FUHddGYYYTDN GC-MONO-DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual German track − Retrieved documents vs Precision
100%
5 docs 27.20
FUHddGYYYTDN
10 docs 23.60 90%
15 docs 22.40
80%
20 docs 22.20
30 docs 20.67 70%
100 docs 12.52
60%
200 docs 7.30
R−Precision
500 docs 3.41 50%
1000 docs 1.85 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
20.56
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.8841
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0625
Third Quartile 0.3603
Interquartile range 0.3603
Mean 0.2056
Standard Deviation 0.2785
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8841 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2056
Std With No Outliers 0.2785
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries FUHddGYYYTDN
Topic 026 0.00 Topic 039 29.63 0.8
Topic 027 7.69 Topic 040 46.34
Topic 028 6.25 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 4.26
Topic 030 63.33 Topic 043 7.14
Topic 031 81.58 Topic 044 0.00 0.4
Topic 032 59.26 Topic 045 0.00
Topic 033 17.65 Topic 046 25.00 0.2
Topic 034 44.12 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 88.41 0
Topic 036 33.33 Topic 049 0.00
Topic 037 0.00 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
102
hagen FUHddGYYYMTDN GC-MONO-DE-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 24,361 Source Language German
Relevant 602 Topic Fields title, description, narrative
Relevant retrieved 442 Pooled true
Geometric Mean Average Precision 0.0318 fifth run
Binary Preference (BPREF) 0.1905
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision
100%
0 39.98
FUHddGYYYMTDN
10 37.09 90%
20 32.92
80%
30 27.08
40 21.92 70%
50 20.54
Average Precision
60%
60 16.82
70 13.13 50%
80 9.93 40%
90 7.45
30%
100 0.73
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
19.99 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.8723
Minimum 0.0000
First Quartile 0.0103
Second Quartile 0.0356
Third Quartile 0.3296
Interquartile range 0.3193
Mean 0.1999
Standard Deviation 0.2771
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7952 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1719
Std With No Outliers 0.2442
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries FUHddGYYYMTDN
Topic 026 0.26 Topic 039 30.63 0.8
Topic 027 0.40 Topic 040 29.27
Topic 028 2.74 Topic 041 3.56
0.6
Topic 029 1.64 Topic 042 6.16
Topic 030 66.68 Topic 043 4.89
Topic 031 79.52 Topic 044 1.13 0.4
Topic 032 64.31 Topic 045 3.20
Topic 033 5.51 Topic 046 26.74 0.2
Topic 034 39.97 Topic 047 0.00
Difference
Topic 035 3.03 Topic 048 87.23 0
Topic 036 40.74 Topic 049 1.32
Topic 037 0.01 Topic 050 0.11
−0.2
Topic 038 0.72
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
103
hagen FUHddGYYYMTDN GC-MONO-DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual German track − Retrieved documents vs Precision
100%
5 docs 25.60
FUHddGYYYMTDN
10 docs 25.60 90%
15 docs 23.20
80%
20 docs 23.00
30 docs 20.93 70%
100 docs 11.68
60%
200 docs 7.34
R−Precision
500 docs 3.33 50%
1000 docs 1.77 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
20.39
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.8116
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0625
Third Quartile 0.4085
Interquartile range 0.4085
Mean 0.2039
Standard Deviation 0.2609
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8116 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2039
Std With No Outliers 0.2609
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries FUHddGYYYMTDN
Topic 026 0.00 Topic 039 40.74 0.8
Topic 027 4.62 Topic 040 41.46
Topic 028 6.25 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 10.64
Topic 030 63.33 Topic 043 7.14
Topic 031 72.37 Topic 044 0.00 0.4
Topic 032 59.26 Topic 045 0.00
Topic 033 17.65 Topic 046 25.00 0.2
Topic 034 41.18 Topic 047 0.00
Difference
Topic 035 5.56 Topic 048 81.16 0
Topic 036 33.33 Topic 049 0.00
Topic 037 0.00 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
104
hildesheim HIGeodederun4n GC-MONO-DE-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language German
Relevant 602 Topic Fields title, description, narrative
Relevant retrieved 419 Pooled true
Geometric Mean Average Precision 0.0562 Experiment with BRF(5docs,25terms) stem lucene
Binary Preference (BPREF) 0.1517
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision
100%
0 45.71
HIGeodederun4n
10 35.46 90%
20 27.03
80%
30 21.27
40 16.62 70%
50 14.56
Average Precision
60%
60 11.24
70 8.62 50%
80 5.50 40%
90 3.04
30%
100 0.46
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
16.01 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.7026
Minimum 0.0020
First Quartile 0.0170
Second Quartile 0.0611
Third Quartile 0.2795
Interquartile range 0.2625
Mean 0.1601
Standard Deviation 0.1994
Lower Outlier Threshold 0.0020
Upper Outlier Threshold 0.5947 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1375
Std With No Outliers 0.1678
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries HIGeodederun4n
Topic 026 0.43 Topic 039 8.87 0.8
Topic 027 1.90 Topic 040 33.91
Topic 028 29.49 Topic 041 0.73
0.6
Topic 029 41.39 Topic 042 8.44
Topic 030 24.39 Topic 043 2.42
Topic 031 5.75 Topic 044 2.10 0.4
Topic 032 70.26 Topic 045 0.20
Topic 033 6.11 Topic 046 27.44 0.2
Topic 034 42.04 Topic 047 1.11
Difference
Topic 035 6.85 Topic 048 59.47 0
Topic 036 0.71 Topic 049 17.63
Topic 037 0.54 Topic 050 5.49
−0.2
Topic 038 2.57
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
105
hildesheim HIGeodederun4n GC-MONO-DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual German track − Retrieved documents vs Precision
100%
5 docs 20.00
HIGeodederun4n
10 docs 19.20 90%
15 docs 18.67
80%
20 docs 18.00
30 docs 17.07 70%
100 docs 10.04
60%
200 docs 6.32
R−Precision
500 docs 2.97 50%
1000 docs 1.68 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
17.68
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.6852
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1231
Third Quartile 0.2708
Interquartile range 0.2708
Mean 0.1768
Standard Deviation 0.2002
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6087 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1557
Std With No Outliers 0.1735
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries HIGeodederun4n
Topic 026 0.00 Topic 039 14.81 0.8
Topic 027 12.31 Topic 040 39.02
Topic 028 43.75 Topic 041 5.26
0.6
Topic 029 33.33 Topic 042 14.89
Topic 030 23.33 Topic 043 7.14
Topic 031 10.53 Topic 044 0.00 0.4
Topic 032 68.52 Topic 045 0.00
Topic 033 5.88 Topic 046 25.00 0.2
Topic 034 44.12 Topic 047 0.00
Difference
Topic 035 16.67 Topic 048 60.87 0
Topic 036 0.00 Topic 049 16.67
Topic 037 0.00 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
106
hildesheim HIGeodederun4 GC-MONO-DE-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language German
Relevant 602 Topic Fields title, description
Relevant retrieved 390 Pooled true
Geometric Mean Average Precision 0.0455 Experiment with BRF(5docs,25terms) stem lucene
Binary Preference (BPREF) 0.1511
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision
100%
0 41.76
HIGeodederun4
10 34.02 90%
20 24.33
80%
30 23.05
40 20.46 70%
50 18.26
Average Precision
60%
60 11.03
70 6.94 50%
80 3.93 40%
90 2.04
30%
100 0.20
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
15.58 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.6498
Minimum 0.0005
First Quartile 0.0080
Second Quartile 0.0534
Third Quartile 0.2489
Interquartile range 0.2410
Mean 0.1558
Standard Deviation 0.2032
Lower Outlier Threshold 0.0005
Upper Outlier Threshold 0.5915 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1352
Std With No Outliers 0.1790
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries HIGeodederun4
Topic 026 0.44 Topic 039 0.81 0.8
Topic 027 2.28 Topic 040 30.80
Topic 028 24.04 Topic 041 0.54
0.6
Topic 029 16.90 Topic 042 4.83
Topic 030 27.44 Topic 043 1.12
Topic 031 5.31 Topic 044 2.08 0.4
Topic 032 64.98 Topic 045 0.05
Topic 033 5.95 Topic 046 51.08 0.2
Topic 034 50.11 Topic 047 0.74
Difference
Topic 035 6.76 Topic 048 59.15 0
Topic 036 0.26 Topic 049 16.67
Topic 037 0.75 Topic 050 5.34
−0.2
Topic 038 10.96
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
107
hildesheim HIGeodederun4 GC-MONO-DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual German track − Retrieved documents vs Precision
100%
5 docs 21.60
HIGeodederun4
10 docs 19.20 90%
15 docs 18.13
80%
20 docs 17.60
30 docs 16.80 70%
100 docs 9.48
60%
200 docs 5.70
R−Precision
500 docs 2.77 50%
1000 docs 1.56 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
18.15
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.6481
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1111
Third Quartile 0.3476
Interquartile range 0.3476
Mean 0.1815
Standard Deviation 0.2146
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6481 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1815
Std With No Outliers 0.2146
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries HIGeodederun4
Topic 026 0.00 Topic 039 0.00 0.8
Topic 027 12.31 Topic 040 39.02
Topic 028 40.62 Topic 041 5.26
0.6
Topic 029 33.33 Topic 042 12.77
Topic 030 26.67 Topic 043 0.00
Topic 031 13.16 Topic 044 0.00 0.4
Topic 032 64.81 Topic 045 0.00
Topic 033 5.88 Topic 046 50.00 0.2
Topic 034 50.00 Topic 047 0.00
Difference
Topic 035 11.11 Topic 048 63.77 0
Topic 036 0.00 Topic 049 16.67
Topic 037 0.00 Topic 050 8.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
108
hildesheim HIGeodederun6 GC-MONO-DE-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 21,416 Source Language German
Relevant 602 Topic Fields title, description
Relevant retrieved 301 Pooled true
Geometric Mean Average Precision 0.0076 Experiment with BRF(5docs,25terms) with NE-
Binary Preference (BPREF) 0.1210 recognition and weighting, also within the BRF
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision
100%
0 34.12
HIGeodederun6
10 25.23 90%
20 18.54
80%
30 17.40
40 14.32 70%
50 13.07
Average Precision
60%
60 9.86
70 5.72 50%
80 3.72 40%
90 1.84
30%
100 0.49
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
12.14 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.5888
Minimum 0.0000
First Quartile 0.0021
Second Quartile 0.0133
Third Quartile 0.1458
Interquartile range 0.1437
Mean 0.1214
Standard Deviation 0.1944
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1459 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0296
Std With No Outliers 0.0466
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries HIGeodederun6
Topic 026 0.10 Topic 039 0.25 0.8
Topic 027 1.33 Topic 040 14.58
Topic 028 0.62 Topic 041 1.32
0.6
Topic 029 46.67 Topic 042 3.73
Topic 030 58.88 Topic 043 0.52
Topic 031 0.25 Topic 044 1.33 0.4
Topic 032 45.82 Topic 045 0.00
Topic 033 6.40 Topic 046 0.00 0.2
Topic 034 14.59 Topic 047 39.11
Difference
Topic 035 0.28 Topic 048 53.78 0
Topic 036 0.00 Topic 049 8.28
Topic 037 0.01 Topic 050 5.71
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
109
hildesheim HIGeodederun6 GC-MONO-DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual German track − Retrieved documents vs Precision
100%
5 docs 19.20
HIGeodederun6
10 docs 16.00 90%
15 docs 13.87
80%
20 docs 12.80
30 docs 11.60 70%
100 docs 6.88
60%
200 docs 4.30
R−Precision
500 docs 2.14 50%
1000 docs 1.20 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
13.45
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.6333
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0213
Third Quartile 0.1872
Interquartile range 0.1872
Mean 0.1345
Standard Deviation 0.2032
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0559
Std With No Outliers 0.0922
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries HIGeodederun6
Topic 026 0.00 Topic 039 0.00 0.8
Topic 027 6.15 Topic 040 21.95
Topic 028 0.00 Topic 041 5.26
0.6
Topic 029 33.33 Topic 042 2.13
Topic 030 63.33 Topic 043 0.00
Topic 031 0.00 Topic 044 0.00 0.4
Topic 032 51.85 Topic 045 0.00
Topic 033 5.88 Topic 046 0.00 0.2
Topic 034 17.65 Topic 047 50.00
Difference
Topic 035 0.00 Topic 048 53.62 0
Topic 036 0.00 Topic 049 16.67
Topic 037 0.00 Topic 050 8.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
110
hildesheim HIGeodederun6n GC-MONO-DE-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 22,151 Source Language German
Relevant 568 Topic Fields title, description, narrative
Relevant retrieved 296 Pooled true
Geometric Mean Average Precision 0.0077 Experiment with BRF(5docs,25terms) with NE-
Binary Preference (BPREF) 0.1164 recognition and weighting, also within the BRF
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual German track − Interpolated Recall vs Average Precision
100%
0 29.91
HIGeodederun6n
10 23.90 90%
20 18.87
80%
30 16.83
40 12.56 70%
50 11.13
Average Precision
60%
60 9.42
70 5.94 50%
80 3.46 40%
90 1.51
30%
100 0.38
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
11.34 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.5669
Minimum 0.0000
First Quartile 0.0015
Second Quartile 0.0143
Third Quartile 0.1462
Interquartile range 0.1447
Mean 0.1134
Standard Deviation 0.1847
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.2443 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0402
Std With No Outliers 0.0717
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries HIGeodederun6n
Topic 026 0.12 Topic 039 24.43 0.8
Topic 027 1.43 Topic 040 12.04
Topic 028 22.37 Topic 041 1.46
0.6
Topic 029 41.64 Topic 042 1.66
Topic 030 53.48 Topic 043 0.86
Topic 031 0.23 Topic 044 1.31 0.4
Topic 032 47.14 Topic 045 0.00
Topic 033 0.18 Topic 046 0.00 0.2
Topic 034 0.00 Topic 047 2.83
Difference
Topic 035 0.67 Topic 048 56.69 0
Topic 036 0.16 Topic 049 8.34
Topic 037 0.12 Topic 050 6.27
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
111
hildesheim HIGeodederun6n GC-MONO-DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual German track − Retrieved documents vs Precision
100%
5 docs 16.00
HIGeodederun6n
10 docs 16.40 90%
15 docs 15.73
80%
20 docs 14.20
30 docs 12.80 70%
100 docs 7.60
60%
200 docs 4.48
R−Precision
500 docs 2.08 50%
1000 docs 1.18 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
13.72
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual German track − Box plot of the Topics of the Experiment
Maximum 0.6232
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.2648
Interquartile range 0.2648
Mean 0.1372
Standard Deviation 0.1981
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6232 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1372
Std With No Outliers 0.1981
GeoCLEF Monolingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries HIGeodederun6n
Topic 026 0.00 Topic 039 25.93 0.8
Topic 027 10.77 Topic 040 29.27
Topic 028 28.12 Topic 041 5.26
0.6
Topic 029 33.33 Topic 042 4.26
Topic 030 56.67 Topic 043 0.00
Topic 031 0.00 Topic 044 0.00 0.4
Topic 032 53.70 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 62.32 0
Topic 036 0.00 Topic 049 16.67
Topic 037 0.00 Topic 050 16.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
112
alicante enTD GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 325 Pooled true
Geometric Mean Average Precision 0.0700 Title and Description
Binary Preference (BPREF) 0.2415
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 47.13
enTD
10 39.05 90%
20 36.01
80%
30 34.33
40 32.37 70%
50 31.03
Average Precision
60%
60 26.86
70 19.84 50%
80 17.01 40%
90 13.36
30%
100 10.10
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
27.23 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9167
Minimum 0.0000
First Quartile 0.0324
Second Quartile 0.1134
Third Quartile 0.4357
Interquartile range 0.4034
Mean 0.2723
Standard Deviation 0.2969
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.9167 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2723
Std With No Outliers 0.2969
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries enTD
Topic 026 49.07 Topic 039 3.94 0.8
Topic 027 0.22 Topic 040 36.93
Topic 028 16.85 Topic 041 0.17
0.6
Topic 029 9.07 Topic 042 45.00
Topic 030 91.67 Topic 043 1.13
Topic 031 43.10 Topic 044 11.34 0.4
Topic 032 88.41 Topic 045 10.04
Topic 033 0.30 Topic 046 71.43 0.2
Topic 034 37.68 Topic 047 5.48
Difference
Topic 035 5.07 Topic 048 80.82 0
Topic 036 0.00 Topic 049 36.11
Topic 037 9.16 Topic 050 26.79
−0.2
Topic 038 1.03
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
113
alicante enTD GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 28.80
enTD
10 docs 24.00 90%
15 docs 21.60
80%
20 docs 19.60
30 docs 16.93 70%
100 docs 8.84
60%
200 docs 5.32
R−Precision
500 docs 2.38 50%
1000 docs 1.30 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
28.01
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8387
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.2105
Third Quartile 0.5000
Interquartile range 0.5000
Mean 0.2801
Standard Deviation 0.2895
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8387 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2801
Std With No Outliers 0.2895
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries enTD
Topic 026 55.56 Topic 039 0.00 0.8
Topic 027 0.00 Topic 040 35.71
Topic 028 26.32 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 50.00
Topic 030 83.33 Topic 043 0.00
Topic 031 45.76 Topic 044 21.05 0.4
Topic 032 83.87 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 8.33
Difference
Topic 035 0.00 Topic 048 77.08 0
Topic 036 0.00 Topic 049 50.00
Topic 037 18.75 Topic 050 33.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
114
alicante enTDN GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 310 Pooled false
Geometric Mean Average Precision 0.0592 Title, Description and Narrative
Binary Preference (BPREF) 0.2672
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 44.88
enTDN
10 43.52 90%
20 40.20
80%
30 39.30
40 34.54 70%
50 33.86
Average Precision
60%
60 29.44
70 22.40 50%
80 20.75 40%
90 15.50
30%
100 12.75
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
29.85 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0088
Second Quartile 0.1585
Third Quartile 0.5044
Interquartile range 0.4957
Mean 0.2985
Standard Deviation 0.3381
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 1.0000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2985
Std With No Outliers 0.3381
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries enTDN
Topic 026 50.09 Topic 039 36.53 0.8
Topic 027 0.66 Topic 040 32.11
Topic 028 3.93 Topic 041 0.18
0.6
Topic 029 12.84 Topic 042 100.00
Topic 030 95.83 Topic 043 0.95
Topic 031 35.57 Topic 044 8.38 0.4
Topic 032 90.05 Topic 045 29.89
Topic 033 0.50 Topic 046 69.05 0.2
Topic 034 51.52 Topic 047 1.69
Difference
Topic 035 2.40 Topic 048 82.66 0
Topic 036 0.00 Topic 049 25.00
Topic 037 0.06 Topic 050 15.85
−0.2
Topic 038 0.57
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
115
alicante enTDN GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 26.40
enTDN
10 docs 22.40 90%
15 docs 20.53
80%
20 docs 19.00
30 docs 16.67 70%
100 docs 8.28
60%
200 docs 4.90
R−Precision
500 docs 2.23 50%
1000 docs 1.24 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
28.51
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1316
Third Quartile 0.5139
Interquartile range 0.5139
Mean 0.2851
Standard Deviation 0.3290
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 1.0000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2851
Std With No Outliers 0.3290
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries enTDN
Topic 026 55.56 Topic 039 50.00 0.8
Topic 027 5.26 Topic 040 28.57
Topic 028 5.26 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 100.00
Topic 030 83.33 Topic 043 0.00
Topic 031 32.20 Topic 044 13.16 0.4
Topic 032 87.10 Topic 045 33.33
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 81.25 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 26.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
116
alicante enTDNGeoNames GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction MANUAL
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 195 Pooled true
Geometric Mean Average Precision 0.0041 Title, Description and Narrative with GeoNames
Binary Preference (BPREF) 0.0907
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 21.96
enTDNGeoNames
10 20.14 90%
20 16.45
80%
30 15.41
40 14.43 70%
50 13.74
Average Precision
60%
60 11.98
70 9.98 50%
80 8.39 40%
90 3.96
30%
100 1.87
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
12.01 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8169
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0144
Third Quartile 0.0822
Interquartile range 0.0821
Mean 0.1201
Standard Deviation 0.2220
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1323 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0219
Std With No Outliers 0.0339
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries enTDNGeoNames
Topic 026 48.34 Topic 039 6.28 0.8
Topic 027 0.04 Topic 040 26.93
Topic 028 0.59 Topic 041 0.67
0.6
Topic 029 4.91 Topic 042 6.55
Topic 030 0.00 Topic 043 0.05
Topic 031 2.03 Topic 044 0.00 0.4
Topic 032 64.59 Topic 045 34.89
Topic 033 2.59 Topic 046 4.08 0.2
Topic 034 1.44 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 81.69 0
Topic 036 0.00 Topic 049 0.00
Topic 037 1.30 Topic 050 13.23
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
117
alicante enTDNGeoNames GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 11.20
enTDNGeoNames
10 docs 11.60 90%
15 docs 12.00
80%
20 docs 11.60
30 docs 10.53 70%
100 docs 5.52
60%
200 docs 3.22
R−Precision
500 docs 1.39 50%
1000 docs 0.78 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
10.30
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7917
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.0638
Interquartile range 0.0638
Mean 0.1030
Standard Deviation 0.2240
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.0678 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0065
Std With No Outliers 0.0201
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries enTDNGeoNames
Topic 026 55.56 Topic 039 0.00 0.8
Topic 027 0.00 Topic 040 28.57
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 0.00
Topic 030 0.00 Topic 043 0.00
Topic 031 6.78 Topic 044 0.00 0.4
Topic 032 64.52 Topic 045 16.67
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 79.17 0
Topic 036 0.00 Topic 049 0.00
Topic 037 6.25 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
118
alicante UAUJAUPVenenExp1 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 312 Pooled false
Geometric Mean Average Precision 0.0552 Voting UA, UJA and UPV
Binary Preference (BPREF) 0.2172
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 43.89
UAUJAUPVenenExp1
10 36.13 90%
20 34.85
80%
30 33.53
40 33.09 70%
50 32.25
Average Precision
60%
60 24.06
70 14.30 50%
80 11.60 40%
90 7.30
30%
100 6.08
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
24.03 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8374
Minimum 0.0000
First Quartile 0.0174
Second Quartile 0.0550
Third Quartile 0.4547
Interquartile range 0.4373
Mean 0.2403
Standard Deviation 0.2870
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8374 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2403
Std With No Outliers 0.2870
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries UAUJAUPVenenExp1
Topic 026 1.93 Topic 039 21.79 0.8
Topic 027 1.00 Topic 040 32.18
Topic 028 0.64 Topic 041 0.86
0.6
Topic 029 5.50 Topic 042 64.29
Topic 030 73.47 Topic 043 1.87
Topic 031 30.41 Topic 044 3.81 0.4
Topic 032 83.74 Topic 045 11.05
Topic 033 0.22 Topic 046 66.84 0.2
Topic 034 42.11 Topic 047 1.84
Difference
Topic 035 3.23 Topic 048 71.84 0
Topic 036 0.00 Topic 049 55.56
Topic 037 1.83 Topic 050 23.27
−0.2
Topic 038 1.47
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
119
alicante UAUJAUPVenenExp1 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 21.60
UAUJAUPVenenExp1
10 docs 17.60 90%
15 docs 16.80
80%
20 docs 16.60
30 docs 13.73 70%
100 docs 7.40
60%
200 docs 4.42
R−Precision
500 docs 2.09 50%
1000 docs 1.25 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
23.19
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8065
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0526
Third Quartile 0.5000
Interquartile range 0.5000
Mean 0.2319
Standard Deviation 0.2847
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8065 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2319
Std With No Outliers 0.2847
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries UAUJAUPVenenExp1
Topic 026 0.00 Topic 039 25.00 0.8
Topic 027 5.26 Topic 040 21.43
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 50.00
Topic 030 66.67 Topic 043 0.00
Topic 031 32.20 Topic 044 2.63 0.4
Topic 032 80.65 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 66.67 Topic 047 4.17
Difference
Topic 035 0.00 Topic 048 68.75 0
Topic 036 0.00 Topic 049 50.00
Topic 037 6.25 Topic 050 33.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
120
berkeley BKGeoE4 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction MANUAL
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 320 Pooled true
Geometric Mean Average Precision 0.0612 Manual expansion of topics 27,43,50 with deletion
Binary Preference (BPREF) 0.2554 of country names from 50, blind feedback
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 49.77
BKGeoE4
10 43.96 90%
20 40.16
80%
30 38.47
40 29.84 70%
50 29.36
Average Precision
60%
60 26.20
70 21.49 50%
80 19.97 40%
90 15.51
30%
100 12.44
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
28.87 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9762
Minimum 0.0000
First Quartile 0.0204
Second Quartile 0.1581
Third Quartile 0.3803
Interquartile range 0.3600
Mean 0.2887
Standard Deviation 0.3232
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8898 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2295
Std With No Outliers 0.2610
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries BKGeoE4
Topic 026 11.00 Topic 039 35.56 0.8
Topic 027 9.30 Topic 040 33.60
Topic 028 0.16 Topic 041 1.16
0.6
Topic 029 5.86 Topic 042 75.00
Topic 030 97.62 Topic 043 31.15
Topic 031 41.33 Topic 044 9.91 0.4
Topic 032 96.31 Topic 045 15.81
Topic 033 0.06 Topic 046 70.24 0.2
Topic 034 36.93 Topic 047 2.96
Difference
Topic 035 2.33 Topic 048 88.98 0
Topic 036 0.00 Topic 049 25.00
Topic 037 0.07 Topic 050 30.91
−0.2
Topic 038 0.49
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
121
berkeley BKGeoE4 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 28.80
BKGeoE4
10 docs 25.60 90%
15 docs 21.07
80%
20 docs 19.80
30 docs 17.73 70%
100 docs 8.28
60%
200 docs 4.84
R−Precision
500 docs 2.28 50%
1000 docs 1.28 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
27.11
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9032
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1667
Third Quartile 0.3957
Interquartile range 0.3957
Mean 0.2711
Standard Deviation 0.2936
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.9032 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2711
Std With No Outliers 0.2936
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries BKGeoE4
Topic 026 22.22 Topic 039 37.50 0.8
Topic 027 15.79 Topic 040 35.71
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 50.00
Topic 030 83.33 Topic 043 37.50
Topic 031 45.76 Topic 044 13.16 0.4
Topic 032 90.32 Topic 045 16.67
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 85.42 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 33.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
122
berkeley BKGeoE2 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 313 Pooled false
Geometric Mean Average Precision 0.0503 English topics TDN with blind feedback, baseline
Binary Preference (BPREF) 0.2326 run
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 41.75
BKGeoE2
10 38.92 90%
20 36.18
80%
30 34.92
40 28.00 70%
50 27.65
Average Precision
60%
60 24.86
70 20.70 50%
80 19.36 40%
90 15.24
30%
100 12.16
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
26.56 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9762
Minimum 0.0000
First Quartile 0.0178
Second Quartile 0.0991
Third Quartile 0.3803
Interquartile range 0.3625
Mean 0.2656
Standard Deviation 0.3312
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8898 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2044
Std With No Outliers 0.2659
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries BKGeoE2
Topic 026 11.00 Topic 039 35.56 0.8
Topic 027 1.99 Topic 040 33.60
Topic 028 0.16 Topic 041 1.16
0.6
Topic 029 5.86 Topic 042 75.00
Topic 030 97.62 Topic 043 6.53
Topic 031 41.33 Topic 044 9.91 0.4
Topic 032 96.31 Topic 045 15.81
Topic 033 0.06 Topic 046 70.24 0.2
Topic 034 36.93 Topic 047 2.96
Difference
Topic 035 2.33 Topic 048 88.98 0
Topic 036 0.00 Topic 049 25.00
Topic 037 0.07 Topic 050 5.06
−0.2
Topic 038 0.49
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
123
berkeley BKGeoE2 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 26.40
BKGeoE2
10 docs 22.80 90%
15 docs 19.47
80%
20 docs 18.20
30 docs 16.27 70%
100 docs 7.84
60%
200 docs 4.70
R−Precision
500 docs 2.25 50%
1000 docs 1.25 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
24.84
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9032
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1250
Third Quartile 0.3957
Interquartile range 0.3957
Mean 0.2484
Standard Deviation 0.2971
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.9032 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2484
Std With No Outliers 0.2971
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries BKGeoE2
Topic 026 22.22 Topic 039 37.50 0.8
Topic 027 10.53 Topic 040 35.71
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 50.00
Topic 030 83.33 Topic 043 12.50
Topic 031 45.76 Topic 044 13.16 0.4
Topic 032 90.32 Topic 045 16.67
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 85.42 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 6.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
124
berkeley BKGeoE3 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction MANUAL
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 320 Pooled false
Geometric Mean Average Precision 0.0596 Manual expansion of topics 27, 43,50 blind feedback
Binary Preference (BPREF) 0.2493
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 48.77
BKGeoE3
10 43.31 90%
20 39.52
80%
30 37.69
40 29.00 70%
50 28.59
Average Precision
60%
60 25.42
70 20.94 50%
80 19.52 40%
90 15.34
30%
100 12.28
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
28.27 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9762
Minimum 0.0000
First Quartile 0.0204
Second Quartile 0.1581
Third Quartile 0.3803
Interquartile range 0.3600
Mean 0.2827
Standard Deviation 0.3242
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8898 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2229
Std With No Outliers 0.2608
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries BKGeoE3
Topic 026 11.00 Topic 039 35.56 0.8
Topic 027 9.30 Topic 040 33.60
Topic 028 0.16 Topic 041 1.16
0.6
Topic 029 5.86 Topic 042 75.00
Topic 030 97.62 Topic 043 31.15
Topic 031 41.33 Topic 044 9.91 0.4
Topic 032 96.31 Topic 045 15.81
Topic 033 0.06 Topic 046 70.24 0.2
Topic 034 36.93 Topic 047 2.96
Difference
Topic 035 2.33 Topic 048 88.98 0
Topic 036 0.00 Topic 049 25.00
Topic 037 0.07 Topic 050 15.86
−0.2
Topic 038 0.49
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
125
berkeley BKGeoE3 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 28.80
BKGeoE3
10 docs 24.80 90%
15 docs 20.53
80%
20 docs 19.20
30 docs 17.20 70%
100 docs 8.20
60%
200 docs 4.82
R−Precision
500 docs 2.27 50%
1000 docs 1.28 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
26.58
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9032
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1667
Third Quartile 0.3957
Interquartile range 0.3957
Mean 0.2658
Standard Deviation 0.2936
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.9032 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2658
Std With No Outliers 0.2936
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries BKGeoE3
Topic 026 22.22 Topic 039 37.50 0.8
Topic 027 15.79 Topic 040 35.71
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 50.00
Topic 030 83.33 Topic 043 37.50
Topic 031 45.76 Topic 044 13.16 0.4
Topic 032 90.32 Topic 045 16.67
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 85.42 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 20.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
126
berkeley BKGeoE1 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 332 Pooled true
Geometric Mean Average Precision 0.0743 Automatic with Blind Feedback TD
Binary Preference (BPREF) 0.2044
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 46.31
BKGeoE1
10 37.74 90%
20 31.60
80%
30 28.99
40 27.55 70%
50 27.23
Average Precision
60%
60 25.04
70 18.52 50%
80 16.78 40%
90 13.44
30%
100 10.72
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
24.99 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9565
Minimum 0.0000
First Quartile 0.0388
Second Quartile 0.1402
Third Quartile 0.3776
Interquartile range 0.3387
Mean 0.2499
Standard Deviation 0.3045
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6952 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1586
Std With No Outliers 0.1819
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries BKGeoE1
Topic 026 6.02 Topic 039 4.51 0.8
Topic 027 3.96 Topic 040 38.53
Topic 028 3.65 Topic 041 0.56
0.6
Topic 029 14.02 Topic 042 14.35
Topic 030 91.07 Topic 043 2.48
Topic 031 48.24 Topic 044 24.20 0.4
Topic 032 95.65 Topic 045 14.14
Topic 033 0.23 Topic 046 69.52 0.2
Topic 034 25.34 Topic 047 9.80
Difference
Topic 035 4.41 Topic 048 89.08 0
Topic 036 0.00 Topic 049 37.50
Topic 037 17.91 Topic 050 8.49
−0.2
Topic 038 1.11
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
127
berkeley BKGeoE1 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 24.80
BKGeoE1
10 docs 21.20 90%
15 docs 20.53
80%
20 docs 19.40
30 docs 17.07 70%
100 docs 9.20
60%
200 docs 5.54
R−Precision
500 docs 2.52 50%
1000 docs 1.33 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
21.95
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8710
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1053
Third Quartile 0.3559
Interquartile range 0.3559
Mean 0.2195
Standard Deviation 0.2780
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8710 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2195
Std With No Outliers 0.2780
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries BKGeoE1
Topic 026 11.11 Topic 039 0.00 0.8
Topic 027 10.53 Topic 040 28.57
Topic 028 5.26 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 66.67 Topic 043 0.00
Topic 031 42.37 Topic 044 21.05 0.4
Topic 032 87.10 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 8.33
Difference
Topic 035 0.00 Topic 048 81.25 0
Topic 036 0.00 Topic 049 50.00
Topic 037 18.75 Topic 050 6.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
128
daedalus GCenAtLg GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction MANUAL
Retrieved 21,339 Source Language English
Relevant 359 Topic Fields title, description, narrative
Relevant retrieved 196 Pooled false
Geometric Mean Average Precision 0.0131 All text Left geo run
Binary Preference (BPREF) 0.1142
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 25.07
GCenAtLg
10 24.25 90%
20 21.27
80%
30 20.73
40 17.72 70%
50 15.12
Average Precision
60%
60 10.07
70 8.72 50%
80 6.70 40%
90 3.19
30%
100 2.90
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
13.05 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6795
Minimum 0.0000
First Quartile 0.0053
Second Quartile 0.0233
Third Quartile 0.1854
Interquartile range 0.1801
Mean 0.1305
Standard Deviation 0.1919
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.2444 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0681
Std With No Outliers 0.0860
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries GCenAtLg
Topic 026 11.98 Topic 039 24.44 0.8
Topic 027 0.00 Topic 040 23.19
Topic 028 1.46 Topic 041 1.04
0.6
Topic 029 0.23 Topic 042 16.86
Topic 030 56.94 Topic 043 6.49
Topic 031 1.40 Topic 044 0.62 0.4
Topic 032 67.95 Topic 045 0.00
Topic 033 0.25 Topic 046 13.10 0.2
Topic 034 2.33 Topic 047 5.03
Difference
Topic 035 0.81 Topic 048 51.55 0
Topic 036 0.00 Topic 049 21.11
Topic 037 1.81 Topic 050 17.68
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
129
daedalus GCenAtLg GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 16.80
GCenAtLg
10 docs 15.20 90%
15 docs 15.20
80%
20 docs 14.20
30 docs 12.27 70%
100 docs 5.16
60%
200 docs 3.10
R−Precision
500 docs 1.42 50%
1000 docs 0.78 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
13.57
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7097
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.2333
Interquartile range 0.2333
Mean 0.1357
Standard Deviation 0.2126
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5417 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1118
Std With No Outliers 0.1796
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries GCenAtLg
Topic 026 33.33 Topic 039 43.75 0.8
Topic 027 0.00 Topic 040 35.71
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 0.00
Topic 030 50.00 Topic 043 12.50
Topic 031 8.47 Topic 044 0.00 0.4
Topic 032 70.97 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 0.00 Topic 047 4.17
Difference
Topic 035 0.00 Topic 048 54.17 0
Topic 036 0.00 Topic 049 0.00
Topic 037 6.25 Topic 050 20.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
130
daedalus GCenNtLg GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction MANUAL
Retrieved 19,789 Source Language English
Relevant 344 Topic Fields title, description
Relevant retrieved 171 Pooled false
Geometric Mean Average Precision 0.0095 Normal text Left geo run
Binary Preference (BPREF) 0.0885
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 23.71
GCenNtLg
10 22.17 90%
20 21.00
80%
30 19.08
40 8.05 70%
50 7.24
Average Precision
60%
60 6.05
70 3.39 50%
80 2.55 40%
90 1.66
30%
100 1.50
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
9.37 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6399
Minimum 0.0000
First Quartile 0.0083
Second Quartile 0.0323
Third Quartile 0.1189
Interquartile range 0.1106
Mean 0.0937
Standard Deviation 0.1469
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1957 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0483
Std With No Outliers 0.0574
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries GCenNtLg
Topic 026 11.98 Topic 039 11.86 0.8
Topic 027 0.00 Topic 040 19.57
Topic 028 7.53 Topic 041 1.04
0.6
Topic 029 4.52 Topic 042 0.00
Topic 030 9.38 Topic 043 4.58
Topic 031 1.95 Topic 044 1.36 0.4
Topic 032 29.09 Topic 045 0.00
Topic 033 0.20 Topic 046 2.11 0.2
Topic 034 35.09 Topic 047 3.23
Difference
Topic 035 7.81 Topic 048 63.99 0
Topic 036 0.00 Topic 049 16.25
Topic 037 1.24 Topic 050 0.00
−0.2
Topic 038 1.59
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
131
daedalus GCenNtLg GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 11.20
GCenNtLg
10 docs 15.20 90%
15 docs 13.87
80%
20 docs 12.20
30 docs 10.27 70%
100 docs 4.56
60%
200 docs 2.76
R−Precision
500 docs 1.24 50%
1000 docs 0.68 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
10.87
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6250
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.1875
Interquartile range 0.1875
Mean 0.1087
Standard Deviation 0.1678
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3871 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0872
Std With No Outliers 0.1316
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries GCenNtLg
Topic 026 33.33 Topic 039 25.00 0.8
Topic 027 0.00 Topic 040 28.57
Topic 028 15.79 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 0.00
Topic 030 0.00 Topic 043 0.00
Topic 031 8.47 Topic 044 5.26 0.4
Topic 032 38.71 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 33.33 Topic 047 4.17
Difference
Topic 035 16.67 Topic 048 62.50 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
132
daedalus GCenNA GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction MANUAL
Retrieved 4,772 Source Language English
Relevant 344 Topic Fields title, description
Relevant retrieved 95 Pooled true
Geometric Mean Average Precision 0.0020 Mandatory run
Binary Preference (BPREF) 0.0830
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 28.20
GCenNA
10 25.28 90%
20 16.49
80%
30 14.80
40 9.93 70%
50 9.93
Average Precision
60%
60 5.70
70 0.80 50%
80 0.80 40%
90 0.80
30%
100 0.80
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
8.93 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6128
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0147
Third Quartile 0.1148
Interquartile range 0.1148
Mean 0.0893
Standard Deviation 0.1520
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.2000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0410
Std With No Outliers 0.0628
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries GCenNA
Topic 026 11.11 Topic 039 19.25 0.8
Topic 027 0.00 Topic 040 7.01
Topic 028 0.00 Topic 041 3.12
0.6
Topic 029 12.59 Topic 042 0.00
Topic 030 0.00 Topic 043 2.92
Topic 031 1.47 Topic 044 3.36 0.4
Topic 032 36.86 Topic 045 0.00
Topic 033 0.20 Topic 046 0.00 0.2
Topic 034 35.09 Topic 047 0.00
Difference
Topic 035 7.81 Topic 048 61.28 0
Topic 036 0.00 Topic 049 0.00
Topic 037 1.25 Topic 050 0.00
−0.2
Topic 038 20.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
133
daedalus GCenNA GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 12.80
GCenNA
10 docs 12.00 90%
15 docs 12.27
80%
20 docs 11.60
30 docs 10.13 70%
100 docs 3.60
60%
200 docs 1.82
R−Precision
500 docs 0.74 50%
1000 docs 0.38 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
9.70
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6667
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.1111
Interquartile range 0.1111
Mean 0.0970
Standard Deviation 0.1741
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.2500 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0413
Std With No Outliers 0.0700
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries GCenNA
Topic 026 11.11 Topic 039 25.00 0.8
Topic 027 0.00 Topic 040 0.00
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 0.00 Topic 043 0.00
Topic 031 10.17 Topic 044 10.53 0.4
Topic 032 51.61 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 33.33 Topic 047 0.00
Difference
Topic 035 16.67 Topic 048 66.67 0
Topic 036 0.00 Topic 049 0.00
Topic 037 6.25 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
134
daedalus GCenAA GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction MANUAL
Retrieved 2,209 Source Language English
Relevant 359 Topic Fields title, description, narrative
Relevant retrieved 117 Pooled true
Geometric Mean Average Precision 0.0036 All text And geo run
Binary Preference (BPREF) 0.1343
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 28.80
GCenAA
10 27.19 90%
20 18.59
80%
30 17.41
40 17.14 70%
50 16.35
Average Precision
60%
60 14.05
70 7.31 50%
80 7.16 40%
90 5.01
30%
100 4.74
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
13.60 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8607
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0221
Third Quartile 0.1250
Interquartile range 0.1250
Mean 0.1360
Standard Deviation 0.2474
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3033 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0388
Std With No Outliers 0.0739
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries GCenAA
Topic 026 11.11 Topic 039 30.33 0.8
Topic 027 0.00 Topic 040 6.32
Topic 028 0.00 Topic 041 3.12
0.6
Topic 029 2.41 Topic 042 16.67
Topic 030 72.66 Topic 043 3.79
Topic 031 1.05 Topic 044 0.80 0.4
Topic 032 86.07 Topic 045 0.00
Topic 033 0.31 Topic 046 38.89 0.2
Topic 034 0.00 Topic 047 0.30
Difference
Topic 035 0.00 Topic 048 60.91 0
Topic 036 0.00 Topic 049 0.00
Topic 037 2.21 Topic 050 3.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
135
daedalus GCenAA GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 17.60
GCenAA
10 docs 14.80 90%
15 docs 14.67
80%
20 docs 13.00
30 docs 11.73 70%
100 docs 4.44
60%
200 docs 2.22
R−Precision
500 docs 0.90 50%
1000 docs 0.47 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
15.70
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8387
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.1271
Interquartile range 0.1271
Mean 0.1570
Standard Deviation 0.2610
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0345
Std With No Outliers 0.0525
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries GCenAA
Topic 026 11.11 Topic 039 43.75 0.8
Topic 027 0.00 Topic 040 0.00
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 0.00
Topic 030 66.67 Topic 043 12.50
Topic 031 10.17 Topic 044 5.26 0.4
Topic 032 83.87 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 0.00 Topic 047 4.17
Difference
Topic 035 0.00 Topic 048 62.50 0
Topic 036 0.00 Topic 049 0.00
Topic 037 12.50 Topic 050 13.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
136
daedalus GCenAO GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 24,149 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 180 Pooled false
Geometric Mean Average Precision 0.0063 All text Or geo run
Binary Preference (BPREF) 0.0871
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 18.95
GCenAO
10 17.28 90%
20 14.55
80%
30 14.30
40 11.22 70%
50 9.56
Average Precision
60%
60 6.96
70 5.81 50%
80 4.19 40%
90 1.22
30%
100 1.00
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
8.91 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6795
Minimum 0.0000
First Quartile 0.0018
Second Quartile 0.0140
Third Quartile 0.0846
Interquartile range 0.0828
Mean 0.0891
Standard Deviation 0.1679
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1310 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0281
Std With No Outliers 0.0404
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries GCenAO
Topic 026 11.98 Topic 039 7.29 0.8
Topic 027 0.03 Topic 040 23.19
Topic 028 0.00 Topic 041 0.33
0.6
Topic 029 0.23 Topic 042 0.99
Topic 030 5.46 Topic 043 6.49
Topic 031 1.40 Topic 044 0.62 0.4
Topic 032 67.95 Topic 045 0.00
Topic 033 0.25 Topic 046 13.10 0.2
Topic 034 2.33 Topic 047 6.33
Difference
Topic 035 0.03 Topic 048 51.55 0
Topic 036 0.00 Topic 049 21.11
Topic 037 1.81 Topic 050 0.31
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
137
daedalus GCenAO GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 12.80
GCenAO
10 docs 12.00 90%
15 docs 12.00
80%
20 docs 11.20
30 docs 9.33 70%
100 docs 4.44
60%
200 docs 2.62
R−Precision
500 docs 1.28 50%
1000 docs 0.72 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
9.52
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7097
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.0948
Interquartile range 0.0948
Mean 0.0952
Standard Deviation 0.1885
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1250 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0209
Std With No Outliers 0.0418
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries GCenAO
Topic 026 33.33 Topic 039 12.50 0.8
Topic 027 0.00 Topic 040 35.71
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 0.00
Topic 030 0.00 Topic 043 12.50
Topic 031 8.47 Topic 044 0.00 0.4
Topic 032 70.97 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 0.00 Topic 047 4.17
Difference
Topic 035 0.00 Topic 048 54.17 0
Topic 036 0.00 Topic 049 0.00
Topic 037 6.25 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
138
hildesheim HIGeoenenrun1n GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 211 Pooled false
Geometric Mean Average Precision 0.0109 no BRF base run, stem snowball
Binary Preference (BPREF) 0.1532
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 34.92
HIGeoenenrun1n
10 28.72 90%
20 26.71
80%
30 23.54
40 21.02 70%
50 19.70
Average Precision
60%
60 16.56
70 8.67 50%
80 6.50 40%
90 5.51
30%
100 5.05
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
17.47 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7729
Minimum 0.0000
First Quartile 0.0015
Second Quartile 0.0178
Third Quartile 0.2501
Interquartile range 0.2486
Mean 0.1747
Standard Deviation 0.2683
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5755 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0987
Std With No Outliers 0.1777
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries HIGeoenenrun1n
Topic 026 0.38 Topic 039 0.90 0.8
Topic 027 0.00 Topic 040 77.29
Topic 028 0.12 Topic 041 0.37
0.6
Topic 029 6.72 Topic 042 0.24
Topic 030 57.55 Topic 043 0.03
Topic 031 17.77 Topic 044 5.03 0.4
Topic 032 46.75 Topic 045 0.65
Topic 033 0.00 Topic 046 68.94 0.2
Topic 034 17.12 Topic 047 1.87
Difference
Topic 035 0.05 Topic 048 73.51 0
Topic 036 0.00 Topic 049 1.78
Topic 037 0.16 Topic 050 9.58
−0.2
Topic 038 50.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
139
hildesheim HIGeoenenrun1n GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 21.60
HIGeoenenrun1n
10 docs 18.00 90%
15 docs 15.47
80%
20 docs 15.00
30 docs 11.87 70%
100 docs 4.68
60%
200 docs 2.80
R−Precision
500 docs 1.50 50%
1000 docs 0.84 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
16.33
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7083
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.2359
Interquartile range 0.2359
Mean 0.1633
Standard Deviation 0.2515
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5714 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0928
Std With No Outliers 0.1697
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries HIGeoenenrun1n
Topic 026 0.00 Topic 039 0.00 0.8
Topic 027 0.00 Topic 040 57.14
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 66.67 Topic 043 0.00
Topic 031 20.34 Topic 044 13.16 0.4
Topic 032 51.61 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 4.17
Difference
Topic 035 0.00 Topic 048 70.83 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 13.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
140
hildesheim HIGeoenenrun2n GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 23,116 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 179 Pooled true
Geometric Mean Average Precision 0.0037 Experiment with BRF(5docs,25terms) with NE-
Binary Preference (BPREF) 0.1174 recognition and weighting, also within the BRF
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 24.44
HIGeoenenrun2n
10 21.79 90%
20 17.90
80%
30 16.62
40 15.59 70%
50 14.86
Average Precision
60%
60 12.33
70 5.82 50%
80 3.35 40%
90 1.68
30%
100 0.84
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
12.13 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7971
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0238
Third Quartile 0.0577
Interquartile range 0.0577
Mean 0.1213
Standard Deviation 0.2278
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.0595 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0192
Std With No Outliers 0.0223
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries HIGeoenenrun2n
Topic 026 0.98 Topic 039 1.14 0.8
Topic 027 0.01 Topic 040 5.95
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 2.82 Topic 042 2.38
Topic 030 70.63 Topic 043 0.00
Topic 031 27.10 Topic 044 5.71 0.4
Topic 032 49.66 Topic 045 1.26
Topic 033 0.00 Topic 046 37.68 0.2
Topic 034 0.00 Topic 047 5.19
Difference
Topic 035 0.00 Topic 048 79.71 0
Topic 036 0.00 Topic 049 4.06
Topic 037 0.05 Topic 050 4.48
−0.2
Topic 038 4.35
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
141
hildesheim HIGeoenenrun2n GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 18.40
HIGeoenenrun2n
10 docs 15.20 90%
15 docs 12.53
80%
20 docs 11.60
30 docs 11.60 70%
100 docs 5.32
60%
200 docs 3.10
R−Precision
500 docs 1.39 50%
1000 docs 0.72 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
13.04
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7292
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.1466
Interquartile range 0.1466
Mean 0.1304
Standard Deviation 0.2222
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0584
Std With No Outliers 0.1029
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries HIGeoenenrun2n
Topic 026 0.00 Topic 039 0.00 0.8
Topic 027 0.00 Topic 040 14.29
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 66.67 Topic 043 0.00
Topic 031 32.20 Topic 044 15.79 0.4
Topic 032 58.06 Topic 045 0.00
Topic 033 0.00 Topic 046 33.33 0.2
Topic 034 0.00 Topic 047 8.33
Difference
Topic 035 0.00 Topic 048 72.92 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 13.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
142
hildesheim HIGeoenenrun3 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 214 Pooled true
Geometric Mean Average Precision 0.0070 Experiment with BRF(5docs,20terms) with
Binary Preference (BPREF) 0.1641 GeoNEweighting within the BRF-algorithm
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 31.16
HIGeoenenrun3
10 27.90 90%
20 27.18
80%
30 25.60
40 22.05 70%
50 20.71
Average Precision
60%
60 19.23
70 11.95 50%
80 9.30 40%
90 6.56
30%
100 4.04
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
18.75 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8680
Minimum 0.0000
First Quartile 0.0004
Second Quartile 0.0231
Third Quartile 0.3109
Interquartile range 0.3105
Mean 0.1875
Standard Deviation 0.2904
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6457 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1288
Std With No Outliers 0.2169
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries HIGeoenenrun3
Topic 026 0.70 Topic 039 0.61 0.8
Topic 027 0.00 Topic 040 85.60
Topic 028 0.01 Topic 041 0.05
0.6
Topic 029 9.28 Topic 042 0.00
Topic 030 64.57 Topic 043 0.00
Topic 031 30.35 Topic 044 4.13 0.4
Topic 032 60.36 Topic 045 0.30
Topic 033 0.00 Topic 046 62.22 0.2
Topic 034 4.07 Topic 047 6.94
Difference
Topic 035 0.13 Topic 048 86.80 0
Topic 036 0.00 Topic 049 16.67
Topic 037 0.23 Topic 050 2.31
−0.2
Topic 038 33.33
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
143
hildesheim HIGeoenenrun3 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 24.00
HIGeoenenrun3
10 docs 19.60 90%
15 docs 18.13
80%
20 docs 16.40
30 docs 12.93 70%
100 docs 5.68
60%
200 docs 3.30
R−Precision
500 docs 1.55 50%
1000 docs 0.86 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
17.85
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7917
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.2599
Interquartile range 0.2599
Mean 0.1785
Standard Deviation 0.2872
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6452 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0739
Std With No Outliers 0.1625
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries HIGeoenenrun3
Topic 026 0.00 Topic 039 0.00 0.8
Topic 027 0.00 Topic 040 78.57
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 22.22 Topic 042 0.00
Topic 030 66.67 Topic 043 0.00
Topic 031 37.29 Topic 044 7.89 0.4
Topic 032 64.52 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 0.00 Topic 047 16.67
Difference
Topic 035 0.00 Topic 048 79.17 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 6.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
144
hildesheim HIGeoenenrun1 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 210 Pooled false
Geometric Mean Average Precision 0.0080 no BRF base run, stem snowball
Binary Preference (BPREF) 0.1544
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 31.94
HIGeoenenrun1
10 27.76 90%
20 25.08
80%
30 23.05
40 20.67 70%
50 19.02
Average Precision
60%
60 17.63
70 7.65 50%
80 5.76 40%
90 5.35
30%
100 4.50
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
16.76 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7728
Minimum 0.0000
First Quartile 0.0013
Second Quartile 0.0118
Third Quartile 0.2704
Interquartile range 0.2692
Mean 0.1676
Standard Deviation 0.2634
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5799 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0909
Std With No Outliers 0.1669
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries HIGeoenenrun1
Topic 026 0.62 Topic 039 0.83 0.8
Topic 027 0.00 Topic 040 77.28
Topic 028 0.16 Topic 041 0.38
0.6
Topic 029 5.66 Topic 042 0.00
Topic 030 57.99 Topic 043 0.02
Topic 031 18.88 Topic 044 3.69 0.4
Topic 032 46.48 Topic 045 0.30
Topic 033 0.00 Topic 046 69.17 0.2
Topic 034 33.16 Topic 047 1.18
Difference
Topic 035 0.03 Topic 048 72.40 0
Topic 036 0.00 Topic 049 1.20
Topic 037 0.32 Topic 050 4.16
−0.2
Topic 038 25.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
145
hildesheim HIGeoenenrun1 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 20.80
HIGeoenenrun1
10 docs 17.60 90%
15 docs 16.00
80%
20 docs 14.60
30 docs 11.87 70%
100 docs 4.56
60%
200 docs 2.70
R−Precision
500 docs 1.38 50%
1000 docs 0.84 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
15.95
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6667
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.2359
Interquartile range 0.2359
Mean 0.1595
Standard Deviation 0.2522
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4839 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0641
Std With No Outliers 0.1285
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries HIGeoenenrun1
Topic 026 0.00 Topic 039 0.00 0.8
Topic 027 0.00 Topic 040 64.29
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 66.67 Topic 043 0.00
Topic 031 20.34 Topic 044 10.53 0.4
Topic 032 48.39 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 4.17
Difference
Topic 035 0.00 Topic 048 66.67 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 6.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
146
hildesheim HIGeoenenrun2 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 21,673 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 164 Pooled false
Geometric Mean Average Precision 0.0018 Experiment with BRF(5docs,25terms) with NE-
Binary Preference (BPREF) 0.1140 recognition and weighting, also within the BRF
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 22.44
HIGeoenenrun2
10 19.60 90%
20 18.23
80%
30 16.75
40 15.53 70%
50 15.04
Average Precision
60%
60 12.73
70 5.54 50%
80 2.57 40%
90 1.73
30%
100 0.72
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
11.66 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8007
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0024
Third Quartile 0.0413
Interquartile range 0.0413
Mean 0.1166
Standard Deviation 0.2331
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.0500 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0103
Std With No Outliers 0.0165
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries HIGeoenenrun2
Topic 026 0.89 Topic 039 0.35 0.8
Topic 027 0.03 Topic 040 0.00
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 3.19 Topic 042 0.00
Topic 030 69.12 Topic 043 0.00
Topic 031 28.44 Topic 044 3.57 0.4
Topic 032 50.00 Topic 045 0.23
Topic 033 0.00 Topic 046 43.24 0.2
Topic 034 0.24 Topic 047 2.95
Difference
Topic 035 0.00 Topic 048 80.07 0
Topic 036 0.00 Topic 049 0.22
Topic 037 0.01 Topic 050 3.84
−0.2
Topic 038 5.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
147
hildesheim HIGeoenenrun2 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 17.60
HIGeoenenrun2
10 docs 13.60 90%
15 docs 11.47
80%
20 docs 11.00
30 docs 10.67 70%
100 docs 4.64
60%
200 docs 2.78
R−Precision
500 docs 1.26 50%
1000 docs 0.66 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
13.05
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7292
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.0870
Interquartile range 0.0870
Mean 0.1305
Standard Deviation 0.2467
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1111 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0149
Std With No Outliers 0.0327
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries HIGeoenenrun2
Topic 026 0.00 Topic 039 0.00 0.8
Topic 027 0.00 Topic 040 0.00
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 66.67 Topic 043 0.00
Topic 031 32.20 Topic 044 7.89 0.4
Topic 032 58.06 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 0.00 Topic 047 4.17
Difference
Topic 035 0.00 Topic 048 72.92 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 6.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
148
imp-coll ICgeoMLtdn GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction MANUAL
Retrieved 7,030 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 182 Pooled true
Geometric Mean Average Precision 0.0166 String and Geographic terms were taken from the
Binary Preference (BPREF) 0.1984 Title, Discription and Narrative and manually
parsed into queries with no extra world knowledge
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 42.69
ICgeoMLtdn
10 34.66 90%
20 31.13
80%
30 29.65
40 24.96 70%
50 24.57
Average Precision
60%
60 18.10
70 14.22 50%
80 8.66 40%
90 5.37
30%
100 4.50
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
19.53 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9167
Minimum 0.0000
First Quartile 0.0063
Second Quartile 0.0578
Third Quartile 0.3506
Interquartile range 0.3443
Mean 0.1953
Standard Deviation 0.2353
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5397 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1653
Std With No Outliers 0.1850
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries ICgeoMLtdn
Topic 026 0.84 Topic 039 40.21 0.8
Topic 027 2.53 Topic 040 5.78
Topic 028 25.06 Topic 041 0.00
0.6
Topic 029 22.77 Topic 042 0.00
Topic 030 29.64 Topic 043 0.00
Topic 031 2.43 Topic 044 2.63 0.4
Topic 032 42.52 Topic 045 0.69
Topic 033 0.42 Topic 046 91.67 0.2
Topic 034 38.46 Topic 047 18.25
Difference
Topic 035 53.97 Topic 048 33.92 0
Topic 036 0.00 Topic 049 50.00
Topic 037 23.09 Topic 050 3.38
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
149
imp-coll ICgeoMLtdn GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 21.60
ICgeoMLtdn
10 docs 18.00 90%
15 docs 17.60
80%
20 docs 16.20
30 docs 12.93 70%
100 docs 6.16
60%
200 docs 3.38
R−Precision
500 docs 1.43 50%
1000 docs 0.73 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
23.55
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6667
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1525
Third Quartile 0.4080
Interquartile range 0.4080
Mean 0.2355
Standard Deviation 0.2213
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2355
Std With No Outliers 0.2213
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries ICgeoMLtdn
Topic 026 0.00 Topic 039 50.00 0.8
Topic 027 10.53 Topic 040 14.29
Topic 028 36.84 Topic 041 0.00
0.6
Topic 029 44.44 Topic 042 0.00
Topic 030 33.33 Topic 043 0.00
Topic 031 15.25 Topic 044 5.26 0.4
Topic 032 64.52 Topic 045 0.00
Topic 033 5.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 25.00
Difference
Topic 035 50.00 Topic 048 39.58 0
Topic 036 0.00 Topic 049 50.00
Topic 037 31.25 Topic 050 13.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
150
imp-coll ICgeoMLtd GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction MANUAL
Retrieved 8,863 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 165 Pooled true
Geometric Mean Average Precision 0.0123 String and Geographic terms were taken from the
Binary Preference (BPREF) 0.1658 Title and Discription and manually parsed into
queries with no extra world knowledge
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 39.84
ICgeoMLtd
10 28.03 90%
20 24.39
80%
30 23.08
40 22.54 70%
50 22.23
Average Precision
60%
60 15.89
70 10.81 50%
80 6.91 40%
90 4.56
30%
100 3.53
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
16.49 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9167
Minimum 0.0000
First Quartile 0.0063
Second Quartile 0.0263
Third Quartile 0.2621
Interquartile range 0.2559
Mean 0.1649
Standard Deviation 0.2456
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1087
Std With No Outliers 0.1531
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries ICgeoMLtd
Topic 026 0.84 Topic 039 0.00 0.8
Topic 027 2.53 Topic 040 5.78
Topic 028 25.07 Topic 041 0.00
0.6
Topic 029 22.77 Topic 042 0.86
Topic 030 29.64 Topic 043 0.00
Topic 031 2.43 Topic 044 2.63 0.4
Topic 032 42.52 Topic 045 0.69
Topic 033 0.42 Topic 046 91.67 0.2
Topic 034 70.67 Topic 047 16.04
Difference
Topic 035 2.61 Topic 048 33.92 0
Topic 036 0.00 Topic 049 50.00
Topic 037 7.81 Topic 050 3.38
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
151
imp-coll ICgeoMLtd GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 16.00
ICgeoMLtd
10 docs 13.20 90%
15 docs 13.33
80%
20 docs 11.80
30 docs 9.73 70%
100 docs 5.24
60%
200 docs 2.98
R−Precision
500 docs 1.28 50%
1000 docs 0.66 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
19.69
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6667
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1250
Third Quartile 0.3753
Interquartile range 0.3753
Mean 0.1969
Standard Deviation 0.2333
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1969
Std With No Outliers 0.2333
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries ICgeoMLtd
Topic 026 0.00 Topic 039 0.00 0.8
Topic 027 10.53 Topic 040 14.29
Topic 028 36.84 Topic 041 0.00
0.6
Topic 029 44.44 Topic 042 0.00
Topic 030 33.33 Topic 043 0.00
Topic 031 15.25 Topic 044 2.63 0.4
Topic 032 64.52 Topic 045 0.00
Topic 033 5.00 Topic 046 66.67 0.2
Topic 034 66.67 Topic 047 16.67
Difference
Topic 035 0.00 Topic 048 39.58 0
Topic 036 0.00 Topic 049 50.00
Topic 037 12.50 Topic 050 13.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
152
jaen sinaiEnEnExp3 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 317 Pooled false
Geometric Mean Average Precision 0.0465 Expansión con geonames
Binary Preference (BPREF) 0.1879
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 40.37
sinaiEnEnExp3
10 32.13 90%
20 27.43
80%
30 26.36
40 25.47 70%
50 24.85
Average Precision
60%
60 24.22
70 18.88 50%
80 18.34 40%
90 14.28
30%
100 11.75
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
22.95 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0234
Second Quartile 0.0823
Third Quartile 0.3049
Interquartile range 0.2815
Mean 0.2295
Standard Deviation 0.3175
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7067 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1311
Std With No Outliers 0.1743
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries sinaiEnEnExp3
Topic 026 0.00 Topic 039 4.83 0.8
Topic 027 3.39 Topic 040 28.43
Topic 028 8.27 Topic 041 2.22
0.6
Topic 029 7.53 Topic 042 3.14
Topic 030 100.00 Topic 043 1.34
Topic 031 13.09 Topic 044 16.59 0.4
Topic 032 94.60 Topic 045 8.23
Topic 033 0.24 Topic 046 70.67 0.2
Topic 034 40.48 Topic 047 6.47
Difference
Topic 035 2.38 Topic 048 90.86 0
Topic 036 0.00 Topic 049 36.67
Topic 037 10.02 Topic 050 23.13
−0.2
Topic 038 1.27
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
153
jaen sinaiEnEnExp3 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 24.00
sinaiEnEnExp3
10 docs 19.60 90%
15 docs 17.33
80%
20 docs 16.20
30 docs 15.73 70%
100 docs 7.08
60%
200 docs 4.70
R−Precision
500 docs 2.38 50%
1000 docs 1.27 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
20.28
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0847
Third Quartile 0.2265
Interquartile range 0.2265
Mean 0.2028
Standard Deviation 0.3061
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0799
Std With No Outliers 0.1025
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries sinaiEnEnExp3
Topic 026 0.00 Topic 039 0.00 0.8
Topic 027 10.53 Topic 040 21.43
Topic 028 26.32 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 100.00 Topic 043 0.00
Topic 031 8.47 Topic 044 15.79 0.4
Topic 032 87.10 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 8.33
Difference
Topic 035 0.00 Topic 048 85.42 0
Topic 036 0.00 Topic 049 0.00
Topic 037 12.50 Topic 050 20.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
154
jaen sinaiEnEnExp1 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 291 Pooled true
Geometric Mean Average Precision 0.0482 Caso base
Binary Preference (BPREF) 0.2907
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 55.37
sinaiEnEnExp1
10 50.26 90%
20 39.83
80%
30 39.10
40 36.40 70%
50 35.78
Average Precision
60%
60 28.73
70 24.12 50%
80 23.30 40%
90 19.80
30%
100 17.44
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
32.24 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0111
Second Quartile 0.1533
Third Quartile 0.6417
Interquartile range 0.6306
Mean 0.3224
Standard Deviation 0.3685
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 1.0000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.3224
Std With No Outliers 0.3685
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries sinaiEnEnExp1
Topic 026 15.33 Topic 039 37.12 0.8
Topic 027 4.32 Topic 040 31.81
Topic 028 0.18 Topic 041 2.05
0.6
Topic 029 11.49 Topic 042 100.00
Topic 030 94.84 Topic 043 2.39
Topic 031 25.61 Topic 044 8.85 0.4
Topic 032 96.05 Topic 045 64.13
Topic 033 0.15 Topic 046 86.67 0.2
Topic 034 50.62 Topic 047 0.05
Difference
Topic 035 1.34 Topic 048 90.47 0
Topic 036 0.00 Topic 049 64.29
Topic 037 0.01 Topic 050 17.70
−0.2
Topic 038 0.40
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
155
jaen sinaiEnEnExp1 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 29.60
sinaiEnEnExp1
10 docs 22.40 90%
15 docs 20.27
80%
20 docs 18.80
30 docs 16.00 70%
100 docs 6.92
60%
200 docs 4.40
R−Precision
500 docs 2.14 50%
1000 docs 1.16 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
29.34
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1316
Third Quartile 0.5000
Interquartile range 0.5000
Mean 0.2934
Standard Deviation 0.3329
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 1.0000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2934
Std With No Outliers 0.3329
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries sinaiEnEnExp1
Topic 026 11.11 Topic 039 43.75 0.8
Topic 027 10.53 Topic 040 42.86
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 100.00
Topic 030 83.33 Topic 043 0.00
Topic 031 22.03 Topic 044 13.16 0.4
Topic 032 90.32 Topic 045 50.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 85.42 0
Topic 036 0.00 Topic 049 50.00
Topic 037 0.00 Topic 050 20.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
156
jaen sinaiEnEnExp2 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 323 Pooled true
Geometric Mean Average Precision 0.0594 Caso base
Binary Preference (BPREF) 0.2039
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 44.41
sinaiEnEnExp2
10 37.60 90%
20 33.38
80%
30 29.48
40 27.16 70%
50 26.25
Average Precision
60%
60 25.50
70 19.70 50%
80 18.99 40%
90 14.22
30%
100 11.77
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
25.04 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9762
Minimum 0.0000
First Quartile 0.0295
Second Quartile 0.1002
Third Quartile 0.3065
Interquartile range 0.2770
Mean 0.2504
Standard Deviation 0.3105
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7067 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1554
Std With No Outliers 0.1770
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries sinaiEnEnExp2
Topic 026 28.64 Topic 039 4.83 0.8
Topic 027 4.42 Topic 040 28.43
Topic 028 8.27 Topic 041 2.22
0.6
Topic 029 3.99 Topic 042 3.14
Topic 030 97.62 Topic 043 1.34
Topic 031 25.33 Topic 044 21.97 0.4
Topic 032 95.56 Topic 045 18.33
Topic 033 0.00 Topic 046 70.67 0.2
Topic 034 40.48 Topic 047 6.47
Difference
Topic 035 2.38 Topic 048 90.86 0
Topic 036 0.00 Topic 049 36.67
Topic 037 10.02 Topic 050 23.13
−0.2
Topic 038 1.27
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
157
jaen sinaiEnEnExp2 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 26.40
sinaiEnEnExp2
10 docs 22.80 90%
15 docs 19.73
80%
20 docs 18.80
30 docs 17.20 70%
100 docs 7.92
60%
200 docs 5.22
R−Precision
500 docs 2.46 50%
1000 docs 1.29 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
21.94
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8710
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1111
Third Quartile 0.2807
Interquartile range 0.2807
Mean 0.2194
Standard Deviation 0.2863
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1330
Std With No Outliers 0.1689
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries sinaiEnEnExp2
Topic 026 33.33 Topic 039 0.00 0.8
Topic 027 10.53 Topic 040 21.43
Topic 028 26.32 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 83.33 Topic 043 0.00
Topic 031 25.42 Topic 044 23.68 0.4
Topic 032 87.10 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 8.33
Difference
Topic 035 0.00 Topic 048 85.42 0
Topic 036 0.00 Topic 049 0.00
Topic 037 12.50 Topic 050 20.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
158
jaen sinaiEnEnExp4 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 322 Pooled false
Geometric Mean Average Precision 0.0660 Expansión con tesauro
Binary Preference (BPREF) 0.2102
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 47.34
sinaiEnEnExp4
10 40.25 90%
20 36.31
80%
30 32.43
40 27.67 70%
50 26.80
Average Precision
60%
60 26.06
70 19.56 50%
80 18.82 40%
90 14.10
30%
100 11.72
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
26.11 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9762
Minimum 0.0000
First Quartile 0.0405
Second Quartile 0.1002
Third Quartile 0.3065
Interquartile range 0.2660
Mean 0.2611
Standard Deviation 0.3112
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5435 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1419
Std With No Outliers 0.1439
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries sinaiEnEnExp4
Topic 026 28.64 Topic 039 4.74 0.8
Topic 027 4.07 Topic 040 28.43
Topic 028 8.27 Topic 041 2.22
0.6
Topic 029 3.99 Topic 042 9.55
Topic 030 97.62 Topic 043 1.34
Topic 031 25.33 Topic 044 21.97 0.4
Topic 032 95.56 Topic 045 18.33
Topic 033 0.00 Topic 046 70.67 0.2
Topic 034 54.35 Topic 047 6.47
Difference
Topic 035 9.18 Topic 048 90.86 0
Topic 036 0.00 Topic 049 36.67
Topic 037 10.02 Topic 050 23.13
−0.2
Topic 038 1.27
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
159
jaen sinaiEnEnExp4 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 26.40
sinaiEnEnExp4
10 docs 23.20 90%
15 docs 20.27
80%
20 docs 19.40
30 docs 17.60 70%
100 docs 8.00
60%
200 docs 5.18
R−Precision
500 docs 2.46 50%
1000 docs 1.29 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
22.61
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8710
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1250
Third Quartile 0.2807
Interquartile range 0.2807
Mean 0.2261
Standard Deviation 0.2829
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1406
Std With No Outliers 0.1664
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries sinaiEnEnExp4
Topic 026 33.33 Topic 039 0.00 0.8
Topic 027 10.53 Topic 040 21.43
Topic 028 26.32 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 83.33 Topic 043 0.00
Topic 031 25.42 Topic 044 23.68 0.4
Topic 032 87.10 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 8.33
Difference
Topic 035 16.67 Topic 048 85.42 0
Topic 036 0.00 Topic 049 0.00
Topic 037 12.50 Topic 050 20.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
160
jaen sinaiEnEnExp5 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 319 Pooled false
Geometric Mean Average Precision 0.0533 Expansión con geonames y tesauro
Binary Preference (BPREF) 0.1945
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 44.00
sinaiEnEnExp5
10 35.03 90%
20 30.37
80%
30 29.30
40 26.05 70%
50 25.39
Average Precision
60%
60 24.78
70 18.74 50%
80 18.17 40%
90 14.16
30%
100 11.70
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
24.07 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0373
Second Quartile 0.0918
Third Quartile 0.3049
Interquartile range 0.2676
Mean 0.2407
Standard Deviation 0.3186
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5435 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1170
Std With No Outliers 0.1373
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries sinaiEnEnExp5
Topic 026 0.00 Topic 039 4.74 0.8
Topic 027 4.24 Topic 040 28.43
Topic 028 8.27 Topic 041 2.22
0.6
Topic 029 7.53 Topic 042 9.55
Topic 030 100.00 Topic 043 1.34
Topic 031 13.09 Topic 044 16.59 0.4
Topic 032 94.60 Topic 045 8.23
Topic 033 0.40 Topic 046 70.67 0.2
Topic 034 54.35 Topic 047 6.47
Difference
Topic 035 9.18 Topic 048 90.86 0
Topic 036 0.00 Topic 049 36.67
Topic 037 10.02 Topic 050 23.13
−0.2
Topic 038 1.27
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
161
jaen sinaiEnEnExp5 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 24.00
sinaiEnEnExp5
10 docs 20.00 90%
15 docs 17.87
80%
20 docs 16.80
30 docs 16.13 70%
100 docs 7.20
60%
200 docs 4.70
R−Precision
500 docs 2.38 50%
1000 docs 1.28 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
20.95
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1053
Third Quartile 0.2265
Interquartile range 0.2265
Mean 0.2095
Standard Deviation 0.3033
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0878
Std With No Outliers 0.1025
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries sinaiEnEnExp5
Topic 026 0.00 Topic 039 0.00 0.8
Topic 027 10.53 Topic 040 21.43
Topic 028 26.32 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 100.00 Topic 043 0.00
Topic 031 8.47 Topic 044 15.79 0.4
Topic 032 87.10 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 8.33
Difference
Topic 035 16.67 Topic 048 85.42 0
Topic 036 0.00 Topic 049 0.00
Topic 037 12.50 Topic 050 20.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
162
ms-china msramanual GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction MANUAL
Retrieved 5,258 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 187 Pooled true
Geometric Mean Average Precision 0.0513 Geoclef 2006 English queries using geo knowledge
Binary Preference (BPREF) 0.2279 base and manual query construction
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 55.00
msramanual
10 52.27 90%
20 42.80
80%
30 35.54
40 32.15 70%
50 30.81
Average Precision
60%
60 17.23
70 9.02 50%
80 5.69 40%
90 2.24
30%
100 2.24
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
23.95 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7500
Minimum 0.0000
First Quartile 0.0239
Second Quartile 0.1622
Third Quartile 0.4063
Interquartile range 0.3823
Mean 0.2395
Standard Deviation 0.2344
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7500 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2395
Std With No Outliers 0.2344
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries msramanual
Topic 026 34.52 Topic 039 41.67 0.8
Topic 027 8.87 Topic 040 0.02
Topic 028 21.98 Topic 041 5.00
0.6
Topic 029 54.85 Topic 042 75.00
Topic 030 40.28 Topic 043 0.34
Topic 031 1.44 Topic 044 11.04 0.4
Topic 032 64.43 Topic 045 26.96
Topic 033 10.00 Topic 046 0.00 0.2
Topic 034 38.89 Topic 047 2.71
Difference
Topic 035 16.22 Topic 048 59.89 0
Topic 036 0.00 Topic 049 50.00
Topic 037 27.31 Topic 050 1.41
−0.2
Topic 038 5.88
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
163
ms-china msramanual GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 34.40
msramanual
10 docs 23.20 90%
15 docs 20.00
80%
20 docs 18.00
30 docs 15.87 70%
100 docs 6.40
60%
200 docs 3.38
R−Precision
500 docs 1.48 50%
1000 docs 0.75 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
25.45
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6774
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1667
Third Quartile 0.5000
Interquartile range 0.5000
Mean 0.2545
Standard Deviation 0.2541
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6774 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2545
Std With No Outliers 0.2541
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries msramanual
Topic 026 33.33 Topic 039 50.00 0.8
Topic 027 5.26 Topic 040 0.00
Topic 028 26.32 Topic 041 0.00
0.6
Topic 029 66.67 Topic 042 50.00
Topic 030 50.00 Topic 043 0.00
Topic 031 8.47 Topic 044 18.42 0.4
Topic 032 67.74 Topic 045 16.67
Topic 033 10.00 Topic 046 0.00 0.2
Topic 034 66.67 Topic 047 0.00
Difference
Topic 035 16.67 Topic 048 62.50 0
Topic 036 0.00 Topic 049 50.00
Topic 037 37.50 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
164
ms-china msrawhitelist GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 4,906 Source Language English
Relevant 378 Topic Fields title
Relevant retrieved 172 Pooled true
Geometric Mean Average Precision 0.0309 Geoclef 2006 English queries using geo knowledge
Binary Preference (BPREF) 0.2078 base
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 53.81
msrawhitelist
10 46.86 90%
20 36.93
80%
30 30.41
40 27.72 70%
50 26.02
Average Precision
60%
60 12.06
70 5.08 50%
80 4.17 40%
90 1.09
30%
100 1.09
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
20.00 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6284
Minimum 0.0000
First Quartile 0.0125
Second Quartile 0.1087
Third Quartile 0.3322
Interquartile range 0.3197
Mean 0.2000
Standard Deviation 0.2081
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6284 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2000
Std With No Outliers 0.2081
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries msrawhitelist
Topic 026 20.19 Topic 039 31.33 0.8
Topic 027 8.87 Topic 040 0.02
Topic 028 14.10 Topic 041 25.00
0.6
Topic 029 28.89 Topic 042 60.00
Topic 030 40.28 Topic 043 0.34
Topic 031 1.54 Topic 044 21.72 0.4
Topic 032 62.84 Topic 045 0.76
Topic 033 10.00 Topic 046 0.00 0.2
Topic 034 38.89 Topic 047 8.77
Difference
Topic 035 0.00 Topic 048 57.10 0
Topic 036 0.00 Topic 049 50.00
Topic 037 10.87 Topic 050 1.41
−0.2
Topic 038 7.14
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
165
ms-china msrawhitelist GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 29.60
msrawhitelist
10 docs 21.20 90%
15 docs 18.13
80%
20 docs 16.80
30 docs 14.53 70%
100 docs 5.92
60%
200 docs 3.20
R−Precision
500 docs 1.36 50%
1000 docs 0.69 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
23.52
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6774
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.2105
Third Quartile 0.4583
Interquartile range 0.4583
Mean 0.2352
Standard Deviation 0.2369
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6774 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2352
Std With No Outliers 0.2369
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries msrawhitelist
Topic 026 33.33 Topic 039 31.25 0.8
Topic 027 5.26 Topic 040 0.00
Topic 028 21.05 Topic 041 25.00
0.6
Topic 029 44.44 Topic 042 50.00
Topic 030 50.00 Topic 043 0.00
Topic 031 8.47 Topic 044 28.95 0.4
Topic 032 67.74 Topic 045 0.00
Topic 033 10.00 Topic 046 0.00 0.2
Topic 034 66.67 Topic 047 8.33
Difference
Topic 035 0.00 Topic 048 62.50 0
Topic 036 0.00 Topic 049 50.00
Topic 037 25.00 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
166
ms-china msraexpansion GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 6,166 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 146 Pooled false
Geometric Mean Average Precision 0.0112 msraexpansion
Binary Preference (BPREF) 0.1730
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 42.10
msraexpansion
10 40.08 90%
20 31.38
80%
30 28.60
40 18.10 70%
50 16.43
Average Precision
60%
60 6.57
70 2.95 50%
80 2.32 40%
90 0.10
30%
100 0.10
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
15.21 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6284
Minimum 0.0000
First Quartile 0.0020
Second Quartile 0.0428
Third Quartile 0.3000
Interquartile range 0.2980
Mean 0.1521
Standard Deviation 0.1987
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6284 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1521
Std With No Outliers 0.1987
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries msraexpansion
Topic 026 20.19 Topic 039 0.74 0.8
Topic 027 8.87 Topic 040 0.02
Topic 028 14.10 Topic 041 0.00
0.6
Topic 029 28.89 Topic 042 0.00
Topic 030 40.28 Topic 043 0.26
Topic 031 4.28 Topic 044 3.08 0.4
Topic 032 62.84 Topic 045 0.00
Topic 033 10.00 Topic 046 33.33 0.2
Topic 034 33.33 Topic 047 2.59
Difference
Topic 035 0.00 Topic 048 57.10 0
Topic 036 0.00 Topic 049 50.00
Topic 037 6.67 Topic 050 1.29
−0.2
Topic 038 2.50
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
167
ms-china msraexpansion GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 23.20
msraexpansion
10 docs 16.40 90%
15 docs 13.87
80%
20 docs 13.00
30 docs 11.73 70%
100 docs 5.24
60%
200 docs 2.72
R−Precision
500 docs 1.14 50%
1000 docs 0.58 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
18.53
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6774
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0667
Third Quartile 0.3333
Interquartile range 0.3333
Mean 0.1853
Standard Deviation 0.2210
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6774 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1853
Std With No Outliers 0.2210
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries msraexpansion
Topic 026 33.33 Topic 039 0.00 0.8
Topic 027 5.26 Topic 040 0.00
Topic 028 21.05 Topic 041 0.00
0.6
Topic 029 44.44 Topic 042 0.00
Topic 030 50.00 Topic 043 0.00
Topic 031 15.25 Topic 044 5.26 0.4
Topic 032 67.74 Topic 045 0.00
Topic 033 10.00 Topic 046 33.33 0.2
Topic 034 33.33 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 62.50 0
Topic 036 0.00 Topic 049 50.00
Topic 037 25.00 Topic 050 6.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
168
ms-china msralocal GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 9,129 Source Language English
Relevant 378 Topic Fields title
Relevant retrieved 183 Pooled false
Geometric Mean Average Precision 0.0284 Geoclef 2006 English queries without geo knowledge
Binary Preference (BPREF) 0.1966 base or query expansion
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 49.68
msralocal
10 44.32 90%
20 34.79
80%
30 28.96
40 26.40 70%
50 24.54
Average Precision
60%
60 10.65
70 3.48 50%
80 2.74 40%
90 0.41
30%
100 0.41
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
18.37 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6284
Minimum 0.0000
First Quartile 0.0151
Second Quartile 0.1000
Third Quartile 0.3139
Interquartile range 0.2988
Mean 0.1837
Standard Deviation 0.2043
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6284 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1837
Std With No Outliers 0.2043
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries msralocal
Topic 026 20.19 Topic 039 10.62 0.8
Topic 027 8.87 Topic 040 0.02
Topic 028 14.10 Topic 041 25.00
0.6
Topic 029 28.89 Topic 042 54.00
Topic 030 40.28 Topic 043 0.23
Topic 031 1.54 Topic 044 13.90 0.4
Topic 032 62.84 Topic 045 0.00
Topic 033 10.00 Topic 046 0.00 0.2
Topic 034 38.89 Topic 047 8.77
Difference
Topic 035 3.91 Topic 048 57.10 0
Topic 036 0.00 Topic 049 50.00
Topic 037 6.45 Topic 050 1.41
−0.2
Topic 038 2.13
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
169
ms-china msralocal GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 27.20
msralocal
10 docs 19.20 90%
15 docs 16.80
80%
20 docs 15.40
30 docs 13.07 70%
100 docs 5.64
60%
200 docs 3.12
R−Precision
500 docs 1.38 50%
1000 docs 0.73 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
22.45
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6774
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1875
Third Quartile 0.4583
Interquartile range 0.4583
Mean 0.2245
Standard Deviation 0.2363
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6774 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2245
Std With No Outliers 0.2363
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries msralocal
Topic 026 33.33 Topic 039 18.75 0.8
Topic 027 5.26 Topic 040 0.00
Topic 028 21.05 Topic 041 25.00
0.6
Topic 029 44.44 Topic 042 50.00
Topic 030 50.00 Topic 043 0.00
Topic 031 8.47 Topic 044 21.05 0.4
Topic 032 67.74 Topic 045 0.00
Topic 033 10.00 Topic 046 0.00 0.2
Topic 034 66.67 Topic 047 8.33
Difference
Topic 035 0.00 Topic 048 62.50 0
Topic 036 0.00 Topic 049 50.00
Topic 037 18.75 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
170
ms-china msratext GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 227 Pooled true
Geometric Mean Average Precision 0.0176 Geoclef 2006 English queries using pure text
Binary Preference (BPREF) 0.1754
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 43.99
msratext
10 39.63 90%
20 30.03
80%
30 27.86
40 22.86 70%
50 21.26
Average Precision
60%
60 11.78
70 6.73 50%
80 5.38 40%
90 3.41
30%
100 2.14
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
18.35 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8655
Minimum 0.0000
First Quartile 0.0031
Second Quartile 0.1378
Third Quartile 0.2361
Interquartile range 0.2330
Mean 0.1835
Standard Deviation 0.2372
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5017 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1132
Std With No Outliers 0.1386
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries msratext
Topic 026 21.17 Topic 039 36.72 0.8
Topic 027 3.25 Topic 040 0.01
Topic 028 0.03 Topic 041 0.00
0.6
Topic 029 14.37 Topic 042 50.17
Topic 030 15.06 Topic 043 0.36
Topic 031 18.97 Topic 044 13.78 0.4
Topic 032 86.55 Topic 045 61.66
Topic 033 0.33 Topic 046 11.22 0.2
Topic 034 14.01 Topic 047 0.43
Difference
Topic 035 1.77 Topic 048 61.49 0
Topic 036 0.00 Topic 049 16.20
Topic 037 0.25 Topic 050 30.94
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
171
ms-china msratext GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 24.00
msratext
10 docs 19.60 90%
15 docs 17.87
80%
20 docs 16.80
30 docs 13.73 70%
100 docs 6.04
60%
200 docs 3.64
R−Precision
500 docs 1.70 50%
1000 docs 0.91 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
21.23
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7419
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1111
Third Quartile 0.3797
Interquartile range 0.3797
Mean 0.2123
Standard Deviation 0.2387
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7419 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2123
Std With No Outliers 0.2387
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries msratext
Topic 026 22.22 Topic 039 43.75 0.8
Topic 027 10.53 Topic 040 0.00
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 50.00
Topic 030 16.67 Topic 043 0.00
Topic 031 37.29 Topic 044 26.32 0.4
Topic 032 74.19 Topic 045 66.67
Topic 033 5.00 Topic 046 33.33 0.2
Topic 034 33.33 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 60.42 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 40.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
172
nicta MuTdnManQexpGeo GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction MANUAL
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 308 Pooled false
Geometric Mean Average Precision 0.0580 title + desc + narr-exp + title-manexp, geo-query
Binary Preference (BPREF) 0.2050 (only for title and desc), with manual title
expansion
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 47.07
MuTdnManQexpGeo
10 41.49 90%
20 34.63
80%
30 32.91
40 31.48 70%
50 29.63
Average Precision
60%
60 19.88
70 13.74 50%
80 11.55 40%
90 8.74
30%
100 7.32
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
24.00 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8807
Minimum 0.0000
First Quartile 0.0175
Second Quartile 0.1407
Third Quartile 0.3799
Interquartile range 0.3625
Mean 0.2400
Standard Deviation 0.2692
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8807 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2400
Std With No Outliers 0.2692
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries MuTdnManQexpGeo
Topic 026 14.66 Topic 039 9.31 0.8
Topic 027 4.33 Topic 040 32.12
Topic 028 0.12 Topic 041 0.14
0.6
Topic 029 18.38 Topic 042 58.33
Topic 030 59.84 Topic 043 0.82
Topic 031 35.89 Topic 044 14.07 0.4
Topic 032 88.07 Topic 045 8.86
Topic 033 2.00 Topic 046 75.00 0.2
Topic 034 44.29 Topic 047 3.81
Difference
Topic 035 4.84 Topic 048 67.52 0
Topic 036 0.00 Topic 049 25.33
Topic 037 0.29 Topic 050 31.04
−0.2
Topic 038 0.97
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
173
nicta MuTdnManQexpGeo GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 25.60
MuTdnManQexpGeo
10 docs 21.20 90%
15 docs 18.93
80%
20 docs 18.00
30 docs 15.20 70%
100 docs 7.88
60%
200 docs 4.72
R−Precision
500 docs 2.22 50%
1000 docs 1.23 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
23.00
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7742
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1250
Third Quartile 0.4301
Interquartile range 0.4301
Mean 0.2300
Standard Deviation 0.2472
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7742 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2300
Std With No Outliers 0.2472
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries MuTdnManQexpGeo
Topic 026 22.22 Topic 039 12.50 0.8
Topic 027 10.53 Topic 040 28.57
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 50.00
Topic 030 50.00 Topic 043 0.00
Topic 031 40.68 Topic 044 21.05 0.4
Topic 032 77.42 Topic 045 0.00
Topic 033 5.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 62.50 0
Topic 036 0.00 Topic 049 50.00
Topic 037 0.00 Topic 050 33.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
174
nicta MuTdnTxt GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 308 Pooled true
Geometric Mean Average Precision 0.0760 Baseline Zetair: title + desc + narr-exp, text only
Binary Preference (BPREF) 0.1993
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 54.38
MuTdnTxt
10 42.31 90%
20 32.76
80%
30 32.10
40 30.46 70%
50 29.11
Average Precision
60%
60 20.53
70 16.38 50%
80 13.20 40%
90 9.64
30%
100 8.27
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
24.44 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9019
Minimum 0.0000
First Quartile 0.0353
Second Quartile 0.1781
Third Quartile 0.3602
Interquartile range 0.3248
Mean 0.2444
Standard Deviation 0.2552
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7436 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2170
Std With No Outliers 0.2200
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries MuTdnTxt
Topic 026 12.19 Topic 039 20.92 0.8
Topic 027 3.41 Topic 040 35.66
Topic 028 0.47 Topic 041 0.14
0.6
Topic 029 24.25 Topic 042 54.76
Topic 030 24.61 Topic 043 0.81
Topic 031 37.09 Topic 044 15.74 0.4
Topic 032 90.19 Topic 045 17.81
Topic 033 1.66 Topic 046 74.36 0.2
Topic 034 44.29 Topic 047 4.71
Difference
Topic 035 4.31 Topic 048 69.29 0
Topic 036 0.00 Topic 049 33.33
Topic 037 7.17 Topic 050 30.36
−0.2
Topic 038 3.57
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
175
nicta MuTdnTxt GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 26.40
MuTdnTxt
10 docs 21.60 90%
15 docs 19.73
80%
20 docs 18.80
30 docs 16.53 70%
100 docs 8.44
60%
200 docs 4.94
R−Precision
500 docs 2.26 50%
1000 docs 1.23 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
21.84
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7419
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1111
Third Quartile 0.3333
Interquartile range 0.3333
Mean 0.2184
Standard Deviation 0.2354
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7419 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2184
Std With No Outliers 0.2354
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries MuTdnTxt
Topic 026 11.11 Topic 039 18.75 0.8
Topic 027 5.26 Topic 040 28.57
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 50.00
Topic 030 33.33 Topic 043 0.00
Topic 031 32.20 Topic 044 21.05 0.4
Topic 032 74.19 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 8.33
Difference
Topic 035 0.00 Topic 048 62.50 0
Topic 036 0.00 Topic 049 50.00
Topic 037 6.25 Topic 050 33.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
176
nicta MuTdQexpPrb GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 291 Pooled true
Geometric Mean Average Precision 0.0626 title + desc, with automatic geographic query
Binary Preference (BPREF) 0.1898 expansion, using probabilistic geo-index
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 42.04
MuTdQexpPrb
10 37.33 90%
20 31.67
80%
30 29.96
40 28.86 70%
50 26.88
Average Precision
60%
60 20.08
70 13.68 50%
80 11.53 40%
90 9.05
30%
100 7.77
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
22.18 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8763
Minimum 0.0000
First Quartile 0.0263
Second Quartile 0.0970
Third Quartile 0.3314
Interquartile range 0.3052
Mean 0.2218
Standard Deviation 0.2615
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7500 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1945
Std With No Outliers 0.2279
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries MuTdQexpPrb
Topic 026 13.38 Topic 039 6.22 0.8
Topic 027 3.16 Topic 040 31.61
Topic 028 6.11 Topic 041 0.35
0.6
Topic 029 18.78 Topic 042 9.70
Topic 030 55.93 Topic 043 0.71
Topic 031 37.75 Topic 044 15.16 0.4
Topic 032 87.63 Topic 045 5.96
Topic 033 0.41 Topic 046 75.00 0.2
Topic 034 46.67 Topic 047 5.27
Difference
Topic 035 4.47 Topic 048 70.71 0
Topic 036 0.00 Topic 049 28.70
Topic 037 1.05 Topic 050 28.68
−0.2
Topic 038 1.04
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
177
nicta MuTdQexpPrb GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 22.40
MuTdQexpPrb
10 docs 21.20 90%
15 docs 19.20
80%
20 docs 18.40
30 docs 16.00 70%
100 docs 8.16
60%
200 docs 5.00
R−Precision
500 docs 2.14 50%
1000 docs 1.16 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
22.40
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7742
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1579
Third Quartile 0.3924
Interquartile range 0.3924
Mean 0.2240
Standard Deviation 0.2438
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7742 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2240
Std With No Outliers 0.2438
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries MuTdQexpPrb
Topic 026 22.22 Topic 039 12.50 0.8
Topic 027 5.26 Topic 040 28.57
Topic 028 15.79 Topic 041 0.00
0.6
Topic 029 22.22 Topic 042 0.00
Topic 030 50.00 Topic 043 0.00
Topic 031 38.98 Topic 044 26.32 0.4
Topic 032 77.42 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 64.58 0
Topic 036 0.00 Topic 049 50.00
Topic 037 6.25 Topic 050 40.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
178
nicta MuTdRedn GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 293 Pooled false
Geometric Mean Average Precision 0.0648 title + desc, with document expansion no query
Binary Preference (BPREF) 0.1870 expansion
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 43.70
MuTdRedn
10 39.58 90%
20 32.18
80%
30 30.93
40 29.93 70%
50 28.08
Average Precision
60%
60 22.13
70 15.73 50%
80 13.06 40%
90 10.46
30%
100 9.32
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
23.41 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8843
Minimum 0.0000
First Quartile 0.0192
Second Quartile 0.1516
Third Quartile 0.3406
Interquartile range 0.3214
Mean 0.2341
Standard Deviation 0.2650
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7576 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2071
Std With No Outliers 0.2327
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries MuTdRedn
Topic 026 18.19 Topic 039 4.94 0.8
Topic 027 2.21 Topic 040 31.61
Topic 028 4.72 Topic 041 0.35
0.6
Topic 029 18.82 Topic 042 29.17
Topic 030 58.76 Topic 043 0.71
Topic 031 38.17 Topic 044 15.16 0.4
Topic 032 88.43 Topic 045 6.76
Topic 033 0.41 Topic 046 75.76 0.2
Topic 034 46.67 Topic 047 5.08
Difference
Topic 035 4.47 Topic 048 71.52 0
Topic 036 0.00 Topic 049 32.69
Topic 037 1.05 Topic 050 28.68
−0.2
Topic 038 1.04
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
179
nicta MuTdRedn GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 24.00
MuTdRedn
10 docs 20.00 90%
15 docs 19.20
80%
20 docs 18.60
30 docs 15.73 70%
100 docs 8.28
60%
200 docs 4.96
R−Precision
500 docs 2.14 50%
1000 docs 1.17 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
21.92
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8065
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1111
Third Quartile 0.3924
Interquartile range 0.3924
Mean 0.2192
Standard Deviation 0.2507
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8065 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2192
Std With No Outliers 0.2507
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries MuTdRedn
Topic 026 11.11 Topic 039 6.25 0.8
Topic 027 5.26 Topic 040 28.57
Topic 028 15.79 Topic 041 0.00
0.6
Topic 029 22.22 Topic 042 0.00
Topic 030 50.00 Topic 043 0.00
Topic 031 38.98 Topic 044 26.32 0.4
Topic 032 80.65 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 66.67 0
Topic 036 0.00 Topic 049 50.00
Topic 037 6.25 Topic 050 40.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
180
nicta MuTdTxt GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 301 Pooled false
Geometric Mean Average Precision 0.0773 Baseline Zetair: title + desc, text only
Binary Preference (BPREF) 0.1943
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 48.88
MuTdTxt
10 41.16 90%
20 32.71
80%
30 30.52
40 29.18 70%
50 27.95
Average Precision
60%
60 20.49
70 14.73 50%
80 11.66 40%
90 8.23
30%
100 7.11
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
23.12 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8980
Minimum 0.0000
First Quartile 0.0523
Second Quartile 0.1146
Third Quartile 0.3523
Interquartile range 0.3000
Mean 0.2312
Standard Deviation 0.2568
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7265 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2034
Std With No Outliers 0.2206
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
6
Number of Topics of the Experiment
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries MuTdTxt
Topic 026 12.19 Topic 039 8.35 0.8
Topic 027 2.41 Topic 040 35.66
Topic 028 7.38 Topic 041 0.41
0.6
Topic 029 24.25 Topic 042 11.46
Topic 030 20.27 Topic 043 0.71
Topic 031 35.09 Topic 044 17.63 0.4
Topic 032 89.80 Topic 045 6.75
Topic 033 0.53 Topic 046 70.83 0.2
Topic 034 44.29 Topic 047 7.70
Difference
Topic 035 4.25 Topic 048 72.65 0
Topic 036 0.00 Topic 049 58.33
Topic 037 10.13 Topic 050 31.43
−0.2
Topic 038 5.56
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
181
nicta MuTdTxt GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 25.60
MuTdTxt
10 docs 22.00 90%
15 docs 20.53
80%
20 docs 20.00
30 docs 17.07 70%
100 docs 8.64
60%
200 docs 5.00
R−Precision
500 docs 2.22 50%
1000 docs 1.20 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
21.55
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7742
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1667
Third Quartile 0.3121
Interquartile range 0.3121
Mean 0.2155
Standard Deviation 0.2358
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7742 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2155
Std With No Outliers 0.2358
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries MuTdTxt
Topic 026 11.11 Topic 039 18.75 0.8
Topic 027 5.26 Topic 040 28.57
Topic 028 21.05 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 16.67 Topic 043 0.00
Topic 031 30.51 Topic 044 26.32 0.4
Topic 032 77.42 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 12.50
Difference
Topic 035 0.00 Topic 048 70.83 0
Topic 036 0.00 Topic 049 50.00
Topic 037 18.75 Topic 050 40.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
182
rfia-upv rfiaUPV01 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 298 Pooled false
Geometric Mean Average Precision 0.0689 Base system without GITE nor WN
Binary Preference (BPREF) 0.2218
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 47.56
rfiaUPV01
10 39.05 90%
20 34.46
80%
30 33.01
40 29.95 70%
50 28.85
Average Precision
60%
60 23.28
70 17.51 50%
80 13.85 40%
90 10.74
30%
100 9.18
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
25.07 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8511
Minimum 0.0000
First Quartile 0.0266
Second Quartile 0.0995
Third Quartile 0.4107
Interquartile range 0.3841
Mean 0.2507
Standard Deviation 0.2946
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8511 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2507
Std With No Outliers 0.2946
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries rfiaUPV01
Topic 026 11.21 Topic 039 7.25 0.8
Topic 027 1.02 Topic 040 36.25
Topic 028 5.45 Topic 041 0.37
0.6
Topic 029 14.38 Topic 042 2.29
Topic 030 76.95 Topic 043 1.70
Topic 031 36.07 Topic 044 18.22 0.4
Topic 032 85.11 Topic 045 2.79
Topic 033 0.21 Topic 046 72.22 0.2
Topic 034 72.22 Topic 047 3.56
Difference
Topic 035 9.95 Topic 048 75.66 0
Topic 036 0.00 Topic 049 55.56
Topic 037 6.59 Topic 050 22.68
−0.2
Topic 038 9.09
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
183
rfia-upv rfiaUPV01 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 25.60
rfiaUPV01
10 docs 21.20 90%
15 docs 19.47
80%
20 docs 19.00
30 docs 16.53 70%
100 docs 8.36
60%
200 docs 4.88
R−Precision
500 docs 2.22 50%
1000 docs 1.19 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
24.18
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7419
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1250
Third Quartile 0.4428
Interquartile range 0.4428
Mean 0.2418
Standard Deviation 0.2656
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7419 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2418
Std With No Outliers 0.2656
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries rfiaUPV01
Topic 026 11.11 Topic 039 18.75 0.8
Topic 027 5.26 Topic 040 28.57
Topic 028 10.53 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 66.67 Topic 043 0.00
Topic 031 42.37 Topic 044 26.32 0.4
Topic 032 74.19 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 66.67 Topic 047 4.17
Difference
Topic 035 16.67 Topic 048 72.92 0
Topic 036 0.00 Topic 049 50.00
Topic 037 12.50 Topic 050 20.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
184
rfia-upv rfiaUPV02 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 303 Pooled false
Geometric Mean Average Precision 0.0735 All Fields without GITE nor WN
Binary Preference (BPREF) 0.2388
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 52.27
rfiaUPV02
10 44.75 90%
20 37.94
80%
30 37.54
40 34.77 70%
50 33.66
Average Precision
60%
60 25.23
70 16.83 50%
80 15.33 40%
90 10.02
30%
100 8.21
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
27.35 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8569
Minimum 0.0000
First Quartile 0.0183
Second Quartile 0.1960
Third Quartile 0.4531
Interquartile range 0.4348
Mean 0.2735
Standard Deviation 0.2852
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8569 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2735
Std With No Outliers 0.2852
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries rfiaUPV02
Topic 026 11.00 Topic 039 35.26 0.8
Topic 027 2.31 Topic 040 28.86
Topic 028 7.21 Topic 041 0.63
0.6
Topic 029 19.60 Topic 042 66.67
Topic 030 72.70 Topic 043 1.86
Topic 031 29.52 Topic 044 12.24 0.4
Topic 032 85.69 Topic 045 31.62
Topic 033 0.33 Topic 046 70.83 0.2
Topic 034 41.52 Topic 047 1.07
Difference
Topic 035 2.76 Topic 048 75.94 0
Topic 036 0.00 Topic 049 56.67
Topic 037 0.39 Topic 050 27.26
−0.2
Topic 038 1.72
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
185
rfia-upv rfiaUPV02 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 24.00
rfiaUPV02
10 docs 21.20 90%
15 docs 20.00
80%
20 docs 18.40
30 docs 17.07 70%
100 docs 8.20
60%
200 docs 4.94
R−Precision
500 docs 2.28 50%
1000 docs 1.21 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
26.50
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7742
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1667
Third Quartile 0.5000
Interquartile range 0.5000
Mean 0.2650
Standard Deviation 0.2702
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7742 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2650
Std With No Outliers 0.2702
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries rfiaUPV02
Topic 026 11.11 Topic 039 43.75 0.8
Topic 027 5.26 Topic 040 28.57
Topic 028 15.79 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 50.00
Topic 030 66.67 Topic 043 0.00
Topic 031 32.20 Topic 044 18.42 0.4
Topic 032 77.42 Topic 045 16.67
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 66.67 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 68.75 0
Topic 036 0.00 Topic 049 50.00
Topic 037 0.00 Topic 050 33.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
186
rfia-upv rfiaUPV03 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 302 Pooled true
Geometric Mean Average Precision 0.0643 Title-Desc with GITE
Binary Preference (BPREF) 0.2045
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 43.17
rfiaUPV03
10 38.01 90%
20 34.07
80%
30 32.27
40 29.60 70%
50 27.56
Average Precision
60%
60 21.90
70 14.89 50%
80 11.47 40%
90 8.74
30%
100 6.73
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
23.35 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8056
Minimum 0.0000
First Quartile 0.0342
Second Quartile 0.1034
Third Quartile 0.4284
Interquartile range 0.3942
Mean 0.2335
Standard Deviation 0.2842
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8056 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2335
Std With No Outliers 0.2842
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries rfiaUPV03
Topic 026 3.91 Topic 039 11.19 0.8
Topic 027 0.30 Topic 040 7.75
Topic 028 8.95 Topic 041 0.37
0.6
Topic 029 4.45 Topic 042 17.87
Topic 030 76.32 Topic 043 1.71
Topic 031 38.70 Topic 044 11.94 0.4
Topic 032 69.75 Topic 045 2.78
Topic 033 0.35 Topic 046 73.81 0.2
Topic 034 80.56 Topic 047 3.64
Difference
Topic 035 10.34 Topic 048 66.92 0
Topic 036 0.00 Topic 049 55.26
Topic 037 14.44 Topic 050 13.31
−0.2
Topic 038 9.09
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
187
rfia-upv rfiaUPV03 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 23.20
rfiaUPV03
10 docs 21.20 90%
15 docs 18.67
80%
20 docs 17.40
30 docs 14.40 70%
100 docs 7.24
60%
200 docs 4.52
R−Precision
500 docs 2.20 50%
1000 docs 1.21 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
21.93
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7419
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1333
Third Quartile 0.3792
Interquartile range 0.3792
Mean 0.2193
Standard Deviation 0.2668
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7419 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2193
Std With No Outliers 0.2668
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries rfiaUPV03
Topic 026 0.00 Topic 039 18.75 0.8
Topic 027 0.00 Topic 040 7.14
Topic 028 15.79 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 0.00
Topic 030 66.67 Topic 043 0.00
Topic 031 33.90 Topic 044 18.42 0.4
Topic 032 74.19 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 66.67 Topic 047 4.17
Difference
Topic 035 16.67 Topic 048 64.58 0
Topic 036 0.00 Topic 049 50.00
Topic 037 31.25 Topic 050 13.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
188
rfia-upv rfiaUPV04 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 307 Pooled true
Geometric Mean Average Precision 0.0681 All Fields with GITE
Binary Preference (BPREF) 0.2393
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 49.64
rfiaUPV04
10 41.56 90%
20 39.02
80%
30 37.65
40 36.11 70%
50 34.81
Average Precision
60%
60 25.47
70 15.20 50%
80 12.54 40%
90 8.16
30%
100 7.01
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
26.60 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8382
Minimum 0.0000
First Quartile 0.0157
Second Quartile 0.1941
Third Quartile 0.4575
Interquartile range 0.4418
Mean 0.2660
Standard Deviation 0.2838
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8382 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2660
Std With No Outliers 0.2838
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries rfiaUPV04
Topic 026 8.39 Topic 039 38.59 0.8
Topic 027 0.83 Topic 040 32.18
Topic 028 19.41 Topic 041 0.63
0.6
Topic 029 5.69 Topic 042 64.29
Topic 030 73.54 Topic 043 1.95
Topic 031 30.59 Topic 044 9.28 0.4
Topic 032 83.82 Topic 045 21.91
Topic 033 0.22 Topic 046 71.67 0.2
Topic 034 42.11 Topic 047 1.10
Difference
Topic 035 3.32 Topic 048 72.23 0
Topic 036 0.00 Topic 049 56.67
Topic 037 0.57 Topic 050 24.23
−0.2
Topic 038 1.72
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
189
rfia-upv rfiaUPV04 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 25.60
rfiaUPV04
10 docs 22.00 90%
15 docs 20.27
80%
20 docs 19.20
30 docs 16.27 70%
100 docs 7.84
60%
200 docs 4.74
R−Precision
500 docs 2.26 50%
1000 docs 1.23 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
26.67
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8065
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.2143
Third Quartile 0.5000
Interquartile range 0.5000
Mean 0.2667
Standard Deviation 0.2767
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8065 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2667
Std With No Outliers 0.2767
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries rfiaUPV04
Topic 026 11.11 Topic 039 43.75 0.8
Topic 027 5.26 Topic 040 21.43
Topic 028 26.32 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 50.00
Topic 030 66.67 Topic 043 0.00
Topic 031 32.20 Topic 044 10.53 0.4
Topic 032 80.65 Topic 045 33.33
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 66.67 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 68.75 0
Topic 036 0.00 Topic 049 50.00
Topic 037 0.00 Topic 050 33.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
190
sanmarcos SMGeoEN4 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 299 Pooled false
Geometric Mean Average Precision 0.0843 Monolingual English no query expansion
Binary Preference (BPREF) 0.2441
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 53.06
SMGeoEN4
10 46.30 90%
20 35.10
80%
30 34.12
40 32.48 70%
50 29.05
Average Precision
60%
60 24.63
70 17.24 50%
80 13.19 40%
90 11.10
30%
100 8.76
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
26.37 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0511
Second Quartile 0.1202
Third Quartile 0.3529
Interquartile range 0.3018
Mean 0.2637
Standard Deviation 0.2909
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7460 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2035
Std With No Outliers 0.2115
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
6
Number of Topics of the Experiment
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoEN4
Topic 026 41.15 Topic 039 11.20 0.8
Topic 027 2.72 Topic 040 25.50
Topic 028 8.02 Topic 041 0.27
0.6
Topic 029 19.83 Topic 042 6.92
Topic 030 100.00 Topic 043 1.67
Topic 031 32.14 Topic 044 28.95 0.4
Topic 032 91.32 Topic 045 11.66
Topic 033 0.22 Topic 046 69.23 0.2
Topic 034 44.77 Topic 047 5.89
Difference
Topic 035 12.02 Topic 048 74.60 0
Topic 036 0.00 Topic 049 33.33
Topic 037 10.73 Topic 050 24.39
−0.2
Topic 038 2.78
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
191
sanmarcos SMGeoEN4 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 32.80
SMGeoEN4
10 docs 26.40 90%
15 docs 22.93
80%
20 docs 21.20
30 docs 17.87 70%
100 docs 8.28
60%
200 docs 5.02
R−Precision
500 docs 2.26 50%
1000 docs 1.20 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
28.57
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0625
Second Quartile 0.1667
Third Quartile 0.4583
Interquartile range 0.3958
Mean 0.2857
Standard Deviation 0.2905
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 1.0000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2857
Std With No Outliers 0.2905
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
6
Number of Topics of the Experiment
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoEN4
Topic 026 44.44 Topic 039 18.75 0.8
Topic 027 10.53 Topic 040 28.57
Topic 028 15.79 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 100.00 Topic 043 0.00
Topic 031 23.73 Topic 044 36.84 0.4
Topic 032 80.65 Topic 045 16.67
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 66.67 Topic 047 8.33
Difference
Topic 035 16.67 Topic 048 72.92 0
Topic 036 0.00 Topic 049 50.00
Topic 037 12.50 Topic 050 33.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
192
sanmarcos SMGeoEN5 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 24,187 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 304 Pooled false
Geometric Mean Average Precision 0.0755 Monolingual English no query expansion
Binary Preference (BPREF) 0.2145
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 45.45
SMGeoEN5
10 40.90 90%
20 35.65
80%
30 31.80
40 29.51 70%
50 28.34
Average Precision
60%
60 24.70
70 14.72 50%
80 11.54 40%
90 9.08
30%
100 6.57
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
23.77 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8471
Minimum 0.0000
First Quartile 0.0344
Second Quartile 0.1325
Third Quartile 0.3426
Interquartile range 0.3082
Mean 0.2377
Standard Deviation 0.2689
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7165 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1849
Std With No Outliers 0.2061
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoEN5
Topic 026 3.45 Topic 039 38.98 0.8
Topic 027 7.16 Topic 040 18.90
Topic 028 22.18 Topic 041 0.27
0.6
Topic 029 13.25 Topic 042 32.69
Topic 030 84.26 Topic 043 3.44
Topic 031 68.79 Topic 044 8.62 0.4
Topic 032 84.71 Topic 045 32.06
Topic 033 5.23 Topic 046 15.42 0.2
Topic 034 40.58 Topic 047 4.32
Difference
Topic 035 2.41 Topic 048 71.65 0
Topic 036 0.00 Topic 049 10.37
Topic 037 0.24 Topic 050 22.28
−0.2
Topic 038 2.94
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
193
sanmarcos SMGeoEN5 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 29.60
SMGeoEN5
10 docs 25.60 90%
15 docs 23.47
80%
20 docs 21.20
30 docs 19.47 70%
100 docs 8.52
60%
200 docs 5.06
R−Precision
500 docs 2.30 50%
1000 docs 1.22 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
25.81
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8387
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1316
Third Quartile 0.5000
Interquartile range 0.5000
Mean 0.2581
Standard Deviation 0.2744
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8387 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2581
Std With No Outliers 0.2744
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoEN5
Topic 026 0.00 Topic 039 37.50 0.8
Topic 027 5.26 Topic 040 21.43
Topic 028 31.58 Topic 041 0.00
0.6
Topic 029 22.22 Topic 042 50.00
Topic 030 66.67 Topic 043 12.50
Topic 031 66.10 Topic 044 13.16 0.4
Topic 032 83.87 Topic 045 50.00
Topic 033 10.00 Topic 046 0.00 0.2
Topic 034 66.67 Topic 047 8.33
Difference
Topic 035 0.00 Topic 048 66.67 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 33.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
194
sanmarcos SMGeoEN1 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 299 Pooled true
Geometric Mean Average Precision 0.0843 Monolingual English query expansion
Binary Preference (BPREF) 0.2441
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 53.06
SMGeoEN1
10 46.30 90%
20 35.10
80%
30 34.12
40 32.48 70%
50 29.05
Average Precision
60%
60 24.63
70 17.24 50%
80 13.19 40%
90 11.10
30%
100 8.76
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
26.37 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0511
Second Quartile 0.1202
Third Quartile 0.3529
Interquartile range 0.3018
Mean 0.2637
Standard Deviation 0.2909
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7460 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2035
Std With No Outliers 0.2115
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
6
Number of Topics of the Experiment
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoEN1
Topic 026 41.15 Topic 039 11.20 0.8
Topic 027 2.72 Topic 040 25.50
Topic 028 8.02 Topic 041 0.27
0.6
Topic 029 19.83 Topic 042 6.92
Topic 030 100.00 Topic 043 1.67
Topic 031 32.14 Topic 044 28.95 0.4
Topic 032 91.32 Topic 045 11.66
Topic 033 0.22 Topic 046 69.23 0.2
Topic 034 44.77 Topic 047 5.89
Difference
Topic 035 12.02 Topic 048 74.60 0
Topic 036 0.00 Topic 049 33.33
Topic 037 10.73 Topic 050 24.39
−0.2
Topic 038 2.78
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
195
sanmarcos SMGeoEN1 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 32.80
SMGeoEN1
10 docs 26.40 90%
15 docs 22.93
80%
20 docs 21.20
30 docs 17.87 70%
100 docs 8.28
60%
200 docs 5.02
R−Precision
500 docs 2.26 50%
1000 docs 1.20 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
28.57
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0625
Second Quartile 0.1667
Third Quartile 0.4583
Interquartile range 0.3958
Mean 0.2857
Standard Deviation 0.2905
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 1.0000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2857
Std With No Outliers 0.2905
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
6
Number of Topics of the Experiment
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoEN1
Topic 026 44.44 Topic 039 18.75 0.8
Topic 027 10.53 Topic 040 28.57
Topic 028 15.79 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 100.00 Topic 043 0.00
Topic 031 23.73 Topic 044 36.84 0.4
Topic 032 80.65 Topic 045 16.67
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 66.67 Topic 047 8.33
Difference
Topic 035 16.67 Topic 048 72.92 0
Topic 036 0.00 Topic 049 50.00
Topic 037 12.50 Topic 050 33.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
196
sanmarcos SMGeoEN3 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 317 Pooled false
Geometric Mean Average Precision 0.0818 Monolingual English query expansion
Binary Preference (BPREF) 0.2519
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 52.35
SMGeoEN3
10 48.93 90%
20 38.86
80%
30 38.26
40 35.28 70%
50 33.08
Average Precision
60%
60 26.13
70 18.46 50%
80 16.52 40%
90 12.26
30%
100 9.70
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
28.57 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9350
Minimum 0.0000
First Quartile 0.0199
Second Quartile 0.2435
Third Quartile 0.3602
Interquartile range 0.3403
Mean 0.2857
Standard Deviation 0.2953
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7857 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2295
Std With No Outliers 0.2318
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoEN3
Topic 026 31.88 Topic 039 37.46 0.8
Topic 027 3.58 Topic 040 26.18
Topic 028 14.52 Topic 041 0.41
0.6
Topic 029 23.28 Topic 042 61.11
Topic 030 93.06 Topic 043 2.00
Topic 031 35.54 Topic 044 15.16 0.4
Topic 032 93.50 Topic 045 33.35
Topic 033 0.69 Topic 046 70.51 0.2
Topic 034 28.38 Topic 047 1.96
Difference
Topic 035 3.37 Topic 048 78.57 0
Topic 036 0.00 Topic 049 34.09
Topic 037 0.35 Topic 050 24.35
−0.2
Topic 038 1.04
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
197
sanmarcos SMGeoEN3 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 28.00
SMGeoEN3
10 docs 26.80 90%
15 docs 22.67
80%
20 docs 20.40
30 docs 18.13 70%
100 docs 8.56
60%
200 docs 4.92
R−Precision
500 docs 2.38 50%
1000 docs 1.27 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
28.36
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9032
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.2542
Third Quartile 0.4583
Interquartile range 0.4583
Mean 0.2836
Standard Deviation 0.2788
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.9032 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2836
Std With No Outliers 0.2788
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoEN3
Topic 026 44.44 Topic 039 37.50 0.8
Topic 027 10.53 Topic 040 21.43
Topic 028 26.32 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 50.00
Topic 030 83.33 Topic 043 0.00
Topic 031 25.42 Topic 044 21.05 0.4
Topic 032 90.32 Topic 045 33.33
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 70.83 0
Topic 036 0.00 Topic 049 50.00
Topic 037 0.00 Topic 050 33.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
198
sanmarcos SMGeoEN5 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction MANUAL
Retrieved 24,187 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 304 Pooled true
Geometric Mean Average Precision 0.0755 Monolingual English added other sources of
Binary Preference (BPREF) 0.2145 information
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 45.45
SMGeoEN5
10 40.90 90%
20 35.65
80%
30 31.80
40 29.51 70%
50 28.34
Average Precision
60%
60 24.70
70 14.72 50%
80 11.54 40%
90 9.08
30%
100 6.57
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
23.77 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8471
Minimum 0.0000
First Quartile 0.0344
Second Quartile 0.1325
Third Quartile 0.3426
Interquartile range 0.3082
Mean 0.2377
Standard Deviation 0.2689
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7165 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1849
Std With No Outliers 0.2061
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoEN5
Topic 026 3.45 Topic 039 38.98 0.8
Topic 027 7.16 Topic 040 18.90
Topic 028 22.18 Topic 041 0.27
0.6
Topic 029 13.25 Topic 042 32.69
Topic 030 84.26 Topic 043 3.44
Topic 031 68.79 Topic 044 8.62 0.4
Topic 032 84.71 Topic 045 32.06
Topic 033 5.23 Topic 046 15.42 0.2
Topic 034 40.58 Topic 047 4.32
Difference
Topic 035 2.41 Topic 048 71.65 0
Topic 036 0.00 Topic 049 10.37
Topic 037 0.24 Topic 050 22.28
−0.2
Topic 038 2.94
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
199
sanmarcos SMGeoEN5 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 29.60
SMGeoEN5
10 docs 25.60 90%
15 docs 23.47
80%
20 docs 21.20
30 docs 19.47 70%
100 docs 8.52
60%
200 docs 5.06
R−Precision
500 docs 2.30 50%
1000 docs 1.22 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
25.81
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8387
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1316
Third Quartile 0.5000
Interquartile range 0.5000
Mean 0.2581
Standard Deviation 0.2744
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8387 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2581
Std With No Outliers 0.2744
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoEN5
Topic 026 0.00 Topic 039 37.50 0.8
Topic 027 5.26 Topic 040 21.43
Topic 028 31.58 Topic 041 0.00
0.6
Topic 029 22.22 Topic 042 50.00
Topic 030 66.67 Topic 043 12.50
Topic 031 66.10 Topic 044 13.16 0.4
Topic 032 83.87 Topic 045 50.00
Topic 033 10.00 Topic 046 0.00 0.2
Topic 034 66.67 Topic 047 8.33
Difference
Topic 035 0.00 Topic 048 66.67 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 33.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
200
talp TALPGeoIRTDN2 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 8,805 Source Language English
Relevant 289 Topic Fields title, description, narrative
Relevant retrieved 181 Pooled false
Geometric Mean Average Precision 0.0006 JIRS with lexical information and Lucene for
Binary Preference (BPREF) 0.0773 Geographical Search
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 17.07
TALPGeoIRTDN2
10 12.96 90%
20 10.88
80%
30 9.84
40 9.61 70%
50 9.06
Average Precision
60%
60 4.29
70 3.45 50%
80 2.82 40%
90 0.38
30%
100 0.00
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
6.38 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.5000
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.0906
Interquartile range 0.0906
Mean 0.0638
Standard Deviation 0.1264
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1512 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0300
Std With No Outliers 0.0477
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries TALPGeoIRTDN2
Topic 026 0.00 Topic 039 11.07 0.8
Topic 027 0.16 Topic 040 0.00
Topic 028 1.27 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 0.00
Topic 030 40.56 Topic 043 0.00
Topic 031 6.66 Topic 044 9.48 0.4
Topic 032 0.00 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 0.00 Topic 047 10.56
Difference
Topic 035 0.31 Topic 048 15.12 0
Topic 036 0.00 Topic 049 50.00
Topic 037 5.51 Topic 050 8.92
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
201
talp TALPGeoIRTDN2 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 8.00
TALPGeoIRTDN2
10 docs 7.20 90%
15 docs 6.67
80%
20 docs 5.60
30 docs 4.53 70%
100 docs 3.04
60%
200 docs 2.24
R−Precision
500 docs 1.32 50%
1000 docs 0.72 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
8.13
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.5000
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.1123
Interquartile range 0.1123
Mean 0.0813
Standard Deviation 0.1459
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.2500 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0449
Std With No Outliers 0.0767
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries TALPGeoIRTDN2
Topic 026 0.00 Topic 039 25.00 0.8
Topic 027 0.00 Topic 040 0.00
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 0.00
Topic 030 50.00 Topic 043 0.00
Topic 031 8.47 Topic 044 10.53 0.4
Topic 032 0.00 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 0.00 Topic 047 8.33
Difference
Topic 035 0.00 Topic 048 18.75 0
Topic 036 0.00 Topic 049 50.00
Topic 037 18.75 Topic 050 13.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
202
talp TALPGeoIRTD1 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 24,742 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 230 Pooled true
Geometric Mean Average Precision 0.0060 Uses the JIRS Passage Retrieval with lexical and
Binary Preference (BPREF) 0.1189 geographical information
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 32.97
TALPGeoIRTD1
10 24.42 90%
20 23.54
80%
30 17.91
40 15.64 70%
50 15.12
Average Precision
60%
60 14.04
70 9.00 50%
80 7.28 40%
90 1.37
30%
100 1.12
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
13.42 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8424
Minimum 0.0000
First Quartile 0.0002
Second Quartile 0.0255
Third Quartile 0.1651
Interquartile range 0.1648
Mean 0.1342
Standard Deviation 0.2200
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3824 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0798
Std With No Outliers 0.1162
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries TALPGeoIRTD1
Topic 026 0.52 Topic 039 2.55 0.8
Topic 027 0.17 Topic 040 0.00
Topic 028 11.93 Topic 041 0.13
0.6
Topic 029 8.96 Topic 042 1.09
Topic 030 84.24 Topic 043 0.10
Topic 031 26.11 Topic 044 3.40 0.4
Topic 032 38.24 Topic 045 0.00
Topic 033 0.00 Topic 046 67.70 0.2
Topic 034 12.68 Topic 047 7.25
Difference
Topic 035 0.03 Topic 048 33.64 0
Topic 036 0.00 Topic 049 22.22
Topic 037 0.02 Topic 050 14.60
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
203
talp TALPGeoIRTD1 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 16.80
TALPGeoIRTD1
10 docs 14.00 90%
15 docs 12.00
80%
20 docs 9.40
30 docs 7.60 70%
100 docs 5.40
60%
200 docs 3.26
R−Precision
500 docs 1.54 50%
1000 docs 0.92 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
13.70
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8333
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.2081
Interquartile range 0.2081
Mean 0.1370
Standard Deviation 0.2174
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0837
Std With No Outliers 0.1174
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries TALPGeoIRTD1
Topic 026 0.00 Topic 039 6.25 0.8
Topic 027 0.00 Topic 040 0.00
Topic 028 15.79 Topic 041 0.00
0.6
Topic 029 22.22 Topic 042 0.00
Topic 030 83.33 Topic 043 0.00
Topic 031 20.34 Topic 044 2.63 0.4
Topic 032 32.26 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 12.50
Difference
Topic 035 0.00 Topic 048 27.08 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 20.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
204
talp TALPGeoIRTDN1 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 260 Pooled true
Geometric Mean Average Precision 0.0093 JIRS for lexical and Geographical Search
Binary Preference (BPREF) 0.1035
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 22.64
TALPGeoIRTDN1
10 21.19 90%
20 21.19
80%
30 18.41
40 13.91 70%
50 13.68
Average Precision
60%
60 11.35
70 10.05 50%
80 7.90 40%
90 2.66
30%
100 2.41
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
11.79 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7323
Minimum 0.0000
First Quartile 0.0013
Second Quartile 0.0437
Third Quartile 0.1846
Interquartile range 0.1833
Mean 0.1179
Standard Deviation 0.1798
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4500 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0923
Std With No Outliers 0.1290
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries TALPGeoIRTDN1
Topic 026 0.60 Topic 039 12.11 0.8
Topic 027 1.17 Topic 040 0.00
Topic 028 0.16 Topic 041 0.13
0.6
Topic 029 8.96 Topic 042 5.37
Topic 030 73.23 Topic 043 0.10
Topic 031 26.11 Topic 044 4.37 0.4
Topic 032 38.24 Topic 045 18.08
Topic 033 0.00 Topic 046 4.93 0.2
Topic 034 19.59 Topic 047 3.55
Difference
Topic 035 1.20 Topic 048 25.12 0
Topic 036 0.00 Topic 049 45.00
Topic 037 0.01 Topic 050 6.74
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
205
talp TALPGeoIRTDN1 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 11.20
TALPGeoIRTDN1
10 docs 9.20 90%
15 docs 6.93
80%
20 docs 6.20
30 docs 6.67 70%
100 docs 5.56
60%
200 docs 3.64
R−Precision
500 docs 1.73 50%
1000 docs 1.04 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
13.16
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6667
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.2473
Interquartile range 0.2473
Mean 0.1316
Standard Deviation 0.1933
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1093
Std With No Outliers 0.1613
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries TALPGeoIRTDN1
Topic 026 0.00 Topic 039 12.50 0.8
Topic 027 0.00 Topic 040 0.00
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 22.22 Topic 042 0.00
Topic 030 66.67 Topic 043 0.00
Topic 031 20.34 Topic 044 7.89 0.4
Topic 032 32.26 Topic 045 33.33
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 33.33 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 43.75 0
Topic 036 0.00 Topic 049 50.00
Topic 037 0.00 Topic 050 6.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
206
talp TALPGeoIRTD2 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 4,851 Source Language English
Relevant 276 Topic Fields title, description
Relevant retrieved 123 Pooled false
Geometric Mean Average Precision 0.0004 JIRS Passage Retrieval for lexical information and
Binary Preference (BPREF) 0.0819 Lucene IR for geographical search
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 17.69
TALPGeoIRTD2
10 14.52 90%
20 14.27
80%
30 12.20
40 9.16 70%
50 9.16
Average Precision
60%
60 5.15
70 4.66 50%
80 4.21 40%
90 0.08
30%
100 0.08
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
7.66 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8333
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.0417
Interquartile range 0.0417
Mean 0.0766
Standard Deviation 0.1913
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.0666 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0060
Std With No Outliers 0.0162
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries TALPGeoIRTD2
Topic 026 0.00 Topic 039 3.34 0.8
Topic 027 0.00 Topic 040 0.00
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 0.32
Topic 030 83.33 Topic 043 0.26
Topic 031 6.66 Topic 044 10.71 0.4
Topic 032 0.00 Topic 045 0.27
Topic 033 0.00 Topic 046 1.25 0.2
Topic 034 0.00 Topic 047 19.14
Difference
Topic 035 0.00 Topic 048 16.24 0
Topic 036 0.00 Topic 049 50.00
Topic 037 0.00 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
207
talp TALPGeoIRTD2 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 8.00
TALPGeoIRTD2
10 docs 6.40 90%
15 docs 6.40
80%
20 docs 5.60
30 docs 4.53 70%
100 docs 2.28
60%
200 docs 1.58
R−Precision
500 docs 0.98 50%
1000 docs 0.49 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
8.84
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8333
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.0681
Interquartile range 0.0681
Mean 0.0884
Standard Deviation 0.2008
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1053 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0120
Std With No Outliers 0.0309
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries TALPGeoIRTD2
Topic 026 0.00 Topic 039 6.25 0.8
Topic 027 0.00 Topic 040 0.00
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 0.00
Topic 030 83.33 Topic 043 0.00
Topic 031 8.47 Topic 044 10.53 0.4
Topic 032 0.00 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 0.00 Topic 047 29.17
Difference
Topic 035 0.00 Topic 048 33.33 0
Topic 036 0.00 Topic 049 50.00
Topic 037 0.00 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
208
talp TALPGeoIRTDN3 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 243 Pooled false
Geometric Mean Average Precision 0.0046 JIRS for lexical and geographical search with
Binary Preference (BPREF) 0.1056 accumulated doc scoring
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 27.28
TALPGeoIRTDN3
10 18.32 90%
20 17.38
80%
30 14.20
40 11.17 70%
50 10.70
Average Precision
60%
60 8.53
70 7.85 50%
80 6.43 40%
90 1.46
30%
100 1.06
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
9.97 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7323
Minimum 0.0000
First Quartile 0.0001
Second Quartile 0.0117
Third Quartile 0.1395
Interquartile range 0.1394
Mean 0.0997
Standard Deviation 0.1677
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.2611 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0600
Std With No Outliers 0.0848
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries TALPGeoIRTDN3
Topic 026 0.60 Topic 039 12.11 0.8
Topic 027 1.17 Topic 040 0.00
Topic 028 0.16 Topic 041 0.00
0.6
Topic 029 8.96 Topic 042 5.89
Topic 030 73.23 Topic 043 0.01
Topic 031 26.11 Topic 044 8.40 0.4
Topic 032 38.24 Topic 045 24.86
Topic 033 0.00 Topic 046 0.18 0.2
Topic 034 19.59 Topic 047 0.05
Difference
Topic 035 1.20 Topic 048 13.93 0
Topic 036 0.00 Topic 049 0.65
Topic 037 0.01 Topic 050 14.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
209
talp TALPGeoIRTDN3 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 11.20
TALPGeoIRTDN3
10 docs 9.20 90%
15 docs 7.47
80%
20 docs 7.60
30 docs 6.53 70%
100 docs 5.08
60%
200 docs 3.18
R−Precision
500 docs 1.57 50%
1000 docs 0.97 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
9.85
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6667
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.1758
Interquartile range 0.1758
Mean 0.0985
Standard Deviation 0.1619
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0748
Std With No Outliers 0.1129
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries TALPGeoIRTDN3
Topic 026 0.00 Topic 039 12.50 0.8
Topic 027 0.00 Topic 040 0.00
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 22.22 Topic 042 0.00
Topic 030 66.67 Topic 043 0.00
Topic 031 20.34 Topic 044 10.53 0.4
Topic 032 32.26 Topic 045 16.67
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 33.33 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 25.00 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 6.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
210
u.buffalo UBGTDrf1 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 301 Pooled false
Geometric Mean Average Precision 0.0694 retr feedback run with parameters 10, 20 6 1
Binary Preference (BPREF) 0.2074
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 47.97
UBGTDrf1
10 38.15 90%
20 32.18
80%
30 30.65
40 29.13 70%
50 27.31
Average Precision
60%
60 23.85
70 15.37 50%
80 14.08 40%
90 6.95
30%
100 5.01
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
23.44 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9248
Minimum 0.0000
First Quartile 0.0230
Second Quartile 0.1410
Third Quartile 0.3015
Interquartile range 0.2785
Mean 0.2344
Standard Deviation 0.2785
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6828 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1513
Std With No Outliers 0.1669
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries UBGTDrf1
Topic 026 12.57 Topic 039 8.79 0.8
Topic 027 3.57 Topic 040 19.88
Topic 028 1.95 Topic 041 0.24
0.6
Topic 029 19.35 Topic 042 5.23
Topic 030 82.12 Topic 043 0.85
Topic 031 29.30 Topic 044 21.70 0.4
Topic 032 92.48 Topic 045 20.97
Topic 033 0.57 Topic 046 68.28 0.2
Topic 034 41.52 Topic 047 2.41
Difference
Topic 035 9.97 Topic 048 78.56 0
Topic 036 0.00 Topic 049 32.69
Topic 037 14.10 Topic 050 17.55
−0.2
Topic 038 1.37
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
211
u.buffalo UBGTDrf1 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 26.40
UBGTDrf1
10 docs 21.60 90%
15 docs 19.73
80%
20 docs 18.60
30 docs 16.53 70%
100 docs 8.12
60%
200 docs 4.84
R−Precision
500 docs 2.18 50%
1000 docs 1.20 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
25.16
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8387
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1429
Third Quartile 0.3792
Interquartile range 0.3792
Mean 0.2516
Standard Deviation 0.2870
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8387 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2516
Std With No Outliers 0.2870
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries UBGTDrf1
Topic 026 22.22 Topic 039 18.75 0.8
Topic 027 5.26 Topic 040 14.29
Topic 028 10.53 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 83.33 Topic 043 0.00
Topic 031 33.90 Topic 044 31.58 0.4
Topic 032 83.87 Topic 045 16.67
Topic 033 5.00 Topic 046 66.67 0.2
Topic 034 66.67 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 77.08 0
Topic 036 0.00 Topic 049 50.00
Topic 037 18.75 Topic 050 13.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
212
u.buffalo UBGTDrf2 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 303 Pooled true
Geometric Mean Average Precision 0.0697 Automatic retrieval feedback params 5 50 10 1
Binary Preference (BPREF) 0.1976
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 49.73
UBGTDrf2
10 39.89 90%
20 31.29
80%
30 29.50
40 28.68 70%
50 26.86
Average Precision
60%
60 23.90
70 15.01 50%
80 13.97 40%
90 7.03
30%
100 4.99
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
23.30 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9434
Minimum 0.0000
First Quartile 0.0289
Second Quartile 0.1484
Third Quartile 0.2741
Interquartile range 0.2451
Mean 0.2330
Standard Deviation 0.2773
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4139 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1251
Std With No Outliers 0.1186
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries UBGTDrf2
Topic 026 9.97 Topic 039 10.82 0.8
Topic 027 3.57 Topic 040 18.74
Topic 028 3.04 Topic 041 0.21
0.6
Topic 029 18.72 Topic 042 5.30
Topic 030 77.32 Topic 043 0.87
Topic 031 32.58 Topic 044 25.68 0.4
Topic 032 94.34 Topic 045 17.54
Topic 033 0.53 Topic 046 68.45 0.2
Topic 034 41.39 Topic 047 2.44
Difference
Topic 035 8.35 Topic 048 79.78 0
Topic 036 0.00 Topic 049 24.36
Topic 037 14.84 Topic 050 22.21
−0.2
Topic 038 1.45
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
213
u.buffalo UBGTDrf2 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 29.60
UBGTDrf2
10 docs 22.00 90%
15 docs 20.00
80%
20 docs 18.60
30 docs 17.07 70%
100 docs 8.36
60%
200 docs 4.90
R−Precision
500 docs 2.21 50%
1000 docs 1.21 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
22.19
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8387
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1111
Third Quartile 0.3146
Interquartile range 0.3146
Mean 0.2219
Standard Deviation 0.2772
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7708 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1962
Std With No Outliers 0.2509
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries UBGTDrf2
Topic 026 22.22 Topic 039 18.75 0.8
Topic 027 5.26 Topic 040 14.29
Topic 028 10.53 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 66.67 Topic 043 0.00
Topic 031 38.98 Topic 044 28.95 0.4
Topic 032 83.87 Topic 045 0.00
Topic 033 5.00 Topic 046 66.67 0.2
Topic 034 66.67 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 77.08 0
Topic 036 0.00 Topic 049 0.00
Topic 037 18.75 Topic 050 20.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
214
u.buffalo UBManual2 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction MANUAL
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 311 Pooled false
Geometric Mean Average Precision 0.0496 manaul run with auto feedback 10 50 10 1
Binary Preference (BPREF) 0.2054
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 49.24
UBManual2
10 43.62 90%
20 36.53
80%
30 32.38
40 27.26 70%
50 26.44
Average Precision
60%
60 22.35
70 16.67 50%
80 15.08 40%
90 9.15
30%
100 7.03
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
24.46 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9560
Minimum 0.0000
First Quartile 0.0388
Second Quartile 0.1509
Third Quartile 0.3812
Interquartile range 0.3424
Mean 0.2446
Standard Deviation 0.2665
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8256 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2150
Std With No Outliers 0.2262
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries UBManual2
Topic 026 6.18 Topic 039 46.74 0.8
Topic 027 12.57 Topic 040 10.82
Topic 028 30.17 Topic 041 0.00
0.6
Topic 029 36.77 Topic 042 70.00
Topic 030 15.95 Topic 043 0.48
Topic 031 26.09 Topic 044 10.94 0.4
Topic 032 95.60 Topic 045 82.56
Topic 033 1.05 Topic 046 37.59 0.2
Topic 034 39.70 Topic 047 0.03
Difference
Topic 035 4.82 Topic 048 40.55 0
Topic 036 0.00 Topic 049 19.44
Topic 037 15.09 Topic 050 8.19
−0.2
Topic 038 0.19
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
215
u.buffalo UBManual2 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 25.60
UBManual2
10 docs 22.80 90%
15 docs 20.53
80%
20 docs 19.00
30 docs 16.40 70%
100 docs 7.68
60%
200 docs 4.84
R−Precision
500 docs 2.29 50%
1000 docs 1.24 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
24.59
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8710
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1875
Third Quartile 0.3701
Interquartile range 0.3701
Mean 0.2459
Standard Deviation 0.2568
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8710 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2459
Std With No Outliers 0.2568
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries UBManual2
Topic 026 11.11 Topic 039 37.50 0.8
Topic 027 21.05 Topic 040 21.43
Topic 028 36.84 Topic 041 0.00
0.6
Topic 029 33.33 Topic 042 50.00
Topic 030 16.67 Topic 043 0.00
Topic 031 28.81 Topic 044 15.79 0.4
Topic 032 87.10 Topic 045 83.33
Topic 033 0.00 Topic 046 33.33 0.2
Topic 034 66.67 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 39.58 0
Topic 036 0.00 Topic 049 0.00
Topic 037 18.75 Topic 050 13.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
216
u.buffalo UBGManual1 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction MANUAL
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 312 Pooled true
Geometric Mean Average Precision 0.0503 Manual run 1
Binary Preference (BPREF) 0.1938
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 47.52
UBGManual1
10 41.99 90%
20 33.72
80%
30 30.33
40 24.98 70%
50 23.27
Average Precision
60%
60 21.15
70 15.69 50%
80 13.83 40%
90 8.94
30%
100 7.13
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
23.07 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9180
Minimum 0.0000
First Quartile 0.0306
Second Quartile 0.1708
Third Quartile 0.3749
Interquartile range 0.3443
Mean 0.2307
Standard Deviation 0.2369
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7041 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2021
Std With No Outliers 0.1928
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries UBGManual1
Topic 026 6.94 Topic 039 45.00 0.8
Topic 027 11.78 Topic 040 9.98
Topic 028 27.94 Topic 041 0.00
0.6
Topic 029 36.62 Topic 042 45.00
Topic 030 19.26 Topic 043 0.62
Topic 031 35.69 Topic 044 10.66 0.4
Topic 032 91.80 Topic 045 70.41
Topic 033 1.07 Topic 046 42.11 0.2
Topic 034 40.12 Topic 047 0.03
Difference
Topic 035 3.73 Topic 048 31.67 0
Topic 036 0.00 Topic 049 20.00
Topic 037 17.08 Topic 050 8.94
−0.2
Topic 038 0.33
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
217
u.buffalo UBGManual1 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 26.40
UBGManual1
10 docs 22.80 90%
15 docs 21.60
80%
20 docs 19.20
30 docs 16.53 70%
100 docs 7.68
60%
200 docs 4.90
R−Precision
500 docs 2.30 50%
1000 docs 1.25 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
24.73
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8387
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1875
Third Quartile 0.3787
Interquartile range 0.3787
Mean 0.2473
Standard Deviation 0.2396
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8387 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2473
Std With No Outliers 0.2396
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries UBGManual1
Topic 026 11.11 Topic 039 37.50 0.8
Topic 027 15.79 Topic 040 21.43
Topic 028 42.11 Topic 041 0.00
0.6
Topic 029 33.33 Topic 042 50.00
Topic 030 33.33 Topic 043 0.00
Topic 031 38.98 Topic 044 15.79 0.4
Topic 032 83.87 Topic 045 66.67
Topic 033 5.00 Topic 046 33.33 0.2
Topic 034 66.67 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 31.25 0
Topic 036 0.00 Topic 049 0.00
Topic 037 18.75 Topic 050 13.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
218
u.groningen CLCGGeoEE1 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 265 Pooled true
Geometric Mean Average Precision 0.0402 uploaded by N. Ferro
Binary Preference (BPREF) 0.1589
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 33.37
CLCGGeoEE1
10 28.45 90%
20 25.08
80%
30 22.99
40 21.40 70%
50 20.37
Average Precision
60%
60 17.52
70 12.73 50%
80 9.08 40%
90 7.21
30%
100 4.88
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
17.30 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8145
Minimum 0.0000
First Quartile 0.0213
Second Quartile 0.0389
Third Quartile 0.1558
Interquartile range 0.1345
Mean 0.1730
Standard Deviation 0.2568
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1971 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0556
Std With No Outliers 0.0572
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries CLCGGeoEE1
Topic 026 2.34 Topic 039 7.02 0.8
Topic 027 3.35 Topic 040 13.72
Topic 028 1.48 Topic 041 0.05
0.6
Topic 029 10.41 Topic 042 3.89
Topic 030 19.71 Topic 043 0.31
Topic 031 35.79 Topic 044 13.93 0.4
Topic 032 81.45 Topic 045 3.54
Topic 033 0.59 Topic 046 77.78 0.2
Topic 034 55.56 Topic 047 1.40
Difference
Topic 035 3.48 Topic 048 70.91 0
Topic 036 0.00 Topic 049 3.57
Topic 037 4.92 Topic 050 14.20
−0.2
Topic 038 3.23
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
219
u.groningen CLCGGeoEE1 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 20.80
CLCGGeoEE1
10 docs 18.00 90%
15 docs 18.13
80%
20 docs 17.00
30 docs 15.33 70%
100 docs 7.28
60%
200 docs 4.28
R−Precision
500 docs 1.99 50%
1000 docs 1.06 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
19.83
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7742
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1053
Third Quartile 0.2833
Interquartile range 0.2833
Mean 0.1983
Standard Deviation 0.2548
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7083 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1743
Std With No Outliers 0.2296
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries CLCGGeoEE1
Topic 026 0.00 Topic 039 18.75 0.8
Topic 027 10.53 Topic 040 21.43
Topic 028 5.26 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 33.33 Topic 043 0.00
Topic 031 42.37 Topic 044 21.05 0.4
Topic 032 77.42 Topic 045 0.00
Topic 033 5.00 Topic 046 66.67 0.2
Topic 034 66.67 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 70.83 0
Topic 036 0.00 Topic 049 0.00
Topic 037 18.75 Topic 050 26.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
220
u.groningen CLCGGeoEE2 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 278 Pooled true
Geometric Mean Average Precision 0.0400 uploaded by N. Ferro
Binary Preference (BPREF) 0.1821
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 45.21
CLCGGeoEE2
10 37.14 90%
20 30.61
80%
30 29.02
40 26.95 70%
50 25.08
Average Precision
60%
60 20.40
70 14.43 50%
80 10.66 40%
90 8.52
30%
100 6.31
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
21.63 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8095
Minimum 0.0000
First Quartile 0.0093
Second Quartile 0.1446
Third Quartile 0.3180
Interquartile range 0.3087
Mean 0.2163
Standard Deviation 0.2586
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7194 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1650
Std With No Outliers 0.1964
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries CLCGGeoEE2
Topic 026 11.00 Topic 039 25.92 0.8
Topic 027 6.04 Topic 040 22.52
Topic 028 0.24 Topic 041 0.00
0.6
Topic 029 21.67 Topic 042 36.11
Topic 030 14.46 Topic 043 0.60
Topic 031 30.37 Topic 044 18.53 0.4
Topic 032 80.25 Topic 045 56.57
Topic 033 0.64 Topic 046 80.95 0.2
Topic 034 38.89 Topic 047 1.03
Difference
Topic 035 2.74 Topic 048 71.94 0
Topic 036 0.00 Topic 049 2.63
Topic 037 0.31 Topic 050 16.00
−0.2
Topic 038 1.32
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
221
u.groningen CLCGGeoEE2 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 23.20
CLCGGeoEE2
10 docs 20.00 90%
15 docs 18.40
80%
20 docs 18.00
30 docs 16.27 70%
100 docs 7.52
60%
200 docs 4.42
R−Precision
500 docs 2.10 50%
1000 docs 1.11 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
21.94
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7742
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1111
Third Quartile 0.3390
Interquartile range 0.3390
Mean 0.2194
Standard Deviation 0.2460
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7742 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2194
Std With No Outliers 0.2460
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries CLCGGeoEE2
Topic 026 11.11 Topic 039 31.25 0.8
Topic 027 10.53 Topic 040 28.57
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 50.00
Topic 030 16.67 Topic 043 0.00
Topic 031 35.59 Topic 044 23.68 0.4
Topic 032 77.42 Topic 045 50.00
Topic 033 5.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 70.83 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 26.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
222
u.groningen CLCGGeoEE5 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 257 Pooled false
Geometric Mean Average Precision 0.0287 uploaded by N. Ferro
Binary Preference (BPREF) 0.1672
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 30.63
CLCGGeoEE5
10 29.43 90%
20 26.34
80%
30 24.06
40 22.95 70%
50 22.52
Average Precision
60%
60 15.57
70 9.64 50%
80 8.02 40%
90 6.35
30%
100 3.50
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
17.57 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8510
Minimum 0.0000
First Quartile 0.0047
Second Quartile 0.0420
Third Quartile 0.3036
Interquartile range 0.2989
Mean 0.1757
Standard Deviation 0.2576
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1476
Std With No Outliers 0.2205
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries CLCGGeoEE5
Topic 026 0.49 Topic 039 5.10 0.8
Topic 027 2.23 Topic 040 4.20
Topic 028 0.09 Topic 041 0.10
0.6
Topic 029 6.07 Topic 042 4.19
Topic 030 25.67 Topic 043 0.09
Topic 031 49.10 Topic 044 12.20 0.4
Topic 032 85.10 Topic 045 3.03
Topic 033 0.40 Topic 046 73.33 0.2
Topic 034 44.44 Topic 047 1.06
Difference
Topic 035 4.86 Topic 048 55.83 0
Topic 036 0.00 Topic 049 50.00
Topic 037 0.35 Topic 050 8.75
−0.2
Topic 038 2.63
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
223
u.groningen CLCGGeoEE5 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 19.20
CLCGGeoEE5
10 docs 17.20 90%
15 docs 15.20
80%
20 docs 14.40
30 docs 12.80 70%
100 docs 6.60
60%
200 docs 4.10
R−Precision
500 docs 1.91 50%
1000 docs 1.03 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
17.77
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7742
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0526
Third Quartile 0.3333
Interquartile range 0.3333
Mean 0.1777
Standard Deviation 0.2464
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7742 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1777
Std With No Outliers 0.2464
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries CLCGGeoEE5
Topic 026 0.00 Topic 039 6.25 0.8
Topic 027 5.26 Topic 040 14.29
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 33.33 Topic 043 0.00
Topic 031 55.93 Topic 044 21.05 0.4
Topic 032 77.42 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 56.25 0
Topic 036 0.00 Topic 049 50.00
Topic 037 0.00 Topic 050 13.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
224
u.groningen CLCGGeoEE10 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 257 Pooled false
Geometric Mean Average Precision 0.0229 geographic query expansion - uploaded by N. Ferro
Binary Preference (BPREF) 0.1481
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 37.92
CLCGGeoEE10
10 30.89 90%
20 24.40
80%
30 22.87
40 19.97 70%
50 19.34
Average Precision
60%
60 13.56
70 9.70 50%
80 8.75 40%
90 5.83
30%
100 3.62
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
16.90 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8481
Minimum 0.0000
First Quartile 0.0053
Second Quartile 0.0555
Third Quartile 0.2643
Interquartile range 0.2590
Mean 0.1690
Standard Deviation 0.2363
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5717 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1183
Std With No Outliers 0.1628
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries CLCGGeoEE10
Topic 026 2.11 Topic 039 26.91 0.8
Topic 027 5.55 Topic 040 0.91
Topic 028 0.06 Topic 041 0.00
0.6
Topic 029 7.90 Topic 042 14.25
Topic 030 30.02 Topic 043 0.56
Topic 031 48.20 Topic 044 9.79 0.4
Topic 032 84.81 Topic 045 26.27
Topic 033 1.35 Topic 046 65.56 0.2
Topic 034 4.11 Topic 047 0.05
Difference
Topic 035 1.02 Topic 048 57.17 0
Topic 036 0.00 Topic 049 25.00
Topic 037 0.23 Topic 050 10.32
−0.2
Topic 038 0.41
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
225
u.groningen CLCGGeoEE10 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 20.80
CLCGGeoEE10
10 docs 18.80 90%
15 docs 16.27
80%
20 docs 14.80
30 docs 12.67 70%
100 docs 6.56
60%
200 docs 4.02
R−Precision
500 docs 1.90 50%
1000 docs 1.03 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
17.62
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7097
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0526
Third Quartile 0.2708
Interquartile range 0.2708
Mean 0.1762
Standard Deviation 0.2356
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1539
Std With No Outliers 0.2122
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries CLCGGeoEE10
Topic 026 11.11 Topic 039 25.00 0.8
Topic 027 5.26 Topic 040 0.00
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 33.33 Topic 043 0.00
Topic 031 52.54 Topic 044 21.05 0.4
Topic 032 70.97 Topic 045 16.67
Topic 033 5.00 Topic 046 66.67 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 58.33 0
Topic 036 0.00 Topic 049 50.00
Topic 037 0.00 Topic 050 13.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
226
u.groningen CLCGGeoEE11 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 277 Pooled false
Geometric Mean Average Precision 0.0421 geographic query expansion - uploaded by N. Ferro
Binary Preference (BPREF) 0.1810
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 49.21
CLCGGeoEE11
10 40.24 90%
20 31.10
80%
30 29.11
40 26.13 70%
50 24.61
Average Precision
60%
60 19.37
70 13.28 50%
80 11.12 40%
90 8.97
30%
100 6.63
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
21.94 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8333
Minimum 0.0000
First Quartile 0.0113
Second Quartile 0.1585
Third Quartile 0.3621
Interquartile range 0.3507
Mean 0.2194
Standard Deviation 0.2514
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2194
Std With No Outliers 0.2514
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries CLCGGeoEE11
Topic 026 12.68 Topic 039 26.02 0.8
Topic 027 6.08 Topic 040 21.75
Topic 028 0.26 Topic 041 0.00
0.6
Topic 029 21.67 Topic 042 36.11
Topic 030 27.11 Topic 043 0.60
Topic 031 36.49 Topic 044 18.08 0.4
Topic 032 80.49 Topic 045 56.75
Topic 033 1.67 Topic 046 83.33 0.2
Topic 034 38.89 Topic 047 0.64
Difference
Topic 035 2.75 Topic 048 57.12 0
Topic 036 0.00 Topic 049 2.63
Topic 037 0.31 Topic 050 15.85
−0.2
Topic 038 1.30
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
227
u.groningen CLCGGeoEE11 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 23.20
CLCGGeoEE11
10 docs 21.20 90%
15 docs 18.13
80%
20 docs 17.80
30 docs 16.00 70%
100 docs 7.68
60%
200 docs 4.50
R−Precision
500 docs 2.07 50%
1000 docs 1.11 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
21.44
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7742
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1111
Third Quartile 0.3559
Interquartile range 0.3559
Mean 0.2144
Standard Deviation 0.2384
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7742 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2144
Std With No Outliers 0.2384
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries CLCGGeoEE11
Topic 026 11.11 Topic 039 31.25 0.8
Topic 027 10.53 Topic 040 21.43
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 50.00
Topic 030 16.67 Topic 043 0.00
Topic 031 42.37 Topic 044 23.68 0.4
Topic 032 77.42 Topic 045 50.00
Topic 033 10.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 60.42 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 20.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
228
u.twente utGeoTIB GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction MANUAL
Retrieved 21,727 Source Language English
Relevant 378 Topic Fields title
Relevant retrieved 209 Pooled true
Geometric Mean Average Precision 0.0247 Retrieval by content. Filtering by geographic
Binary Preference (BPREF) 0.1512 location through boolean matching of query title
locations and document locations.
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 34.06
utGeoTIB
10 29.52 90%
20 25.01
80%
30 23.94
40 19.17 70%
50 14.82
Average Precision
60%
60 10.91
70 9.70 50%
80 8.97 40%
90 7.75
30%
100 6.90
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
16.23 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0072
Second Quartile 0.0644
Third Quartile 0.1936
Interquartile range 0.1864
Mean 0.1623
Standard Deviation 0.2473
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4722 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0861
Std With No Outliers 0.1168
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries utGeoTIB
Topic 026 0.81 Topic 039 6.93 0.8
Topic 027 1.05 Topic 040 0.00
Topic 028 8.95 Topic 041 0.13
0.6
Topic 029 9.58 Topic 042 1.09
Topic 030 20.08 Topic 043 0.17
Topic 031 27.59 Topic 044 14.93 0.4
Topic 032 57.26 Topic 045 0.85
Topic 033 0.31 Topic 046 100.00 0.2
Topic 034 4.76 Topic 047 6.44
Difference
Topic 035 0.42 Topic 048 47.22 0
Topic 036 0.00 Topic 049 59.09
Topic 037 4.63 Topic 050 19.12
−0.2
Topic 038 14.29
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
229
u.twente utGeoTIB GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 18.40
utGeoTIB
10 docs 17.20 90%
15 docs 17.33
80%
20 docs 16.20
30 docs 13.73 70%
100 docs 6.00
60%
200 docs 3.48
R−Precision
500 docs 1.54 50%
1000 docs 0.84 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
17.38
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0625
Third Quartile 0.2215
Interquartile range 0.2215
Mean 0.1738
Standard Deviation 0.2558
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1160
Std With No Outliers 0.1591
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries utGeoTIB
Topic 026 0.00 Topic 039 12.50 0.8
Topic 027 0.00 Topic 040 0.00
Topic 028 15.79 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 16.67 Topic 043 0.00
Topic 031 37.29 Topic 044 18.42 0.4
Topic 032 67.74 Topic 045 0.00
Topic 033 5.00 Topic 046 100.00 0.2
Topic 034 0.00 Topic 047 12.50
Difference
Topic 035 0.00 Topic 048 47.92 0
Topic 036 0.00 Topic 049 50.00
Topic 037 6.25 Topic 050 33.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
230
u.twente utGeoTdIB GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction MANUAL
Retrieved 9,845 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 76 Pooled true
Geometric Mean Average Precision 0.0078 Retrieval by content. Filtering by geographic
Binary Preference (BPREF) 0.0695 location through boolean matching of query title
locations and document locations.
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 36.38
utGeoTdIB
10 20.45 90%
20 10.55
80%
30 6.87
40 6.78 70%
50 6.71
Average Precision
60%
60 5.11
70 0.62 50%
80 0.48 40%
90 0.23
30%
100 0.23
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
7.32 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6667
Minimum 0.0000
First Quartile 0.0075
Second Quartile 0.0260
Third Quartile 0.0758
Interquartile range 0.0683
Mean 0.0732
Standard Deviation 0.1360
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0425
Std With No Outliers 0.0512
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries utGeoTdIB
Topic 026 1.24 Topic 039 0.89 0.8
Topic 027 0.93 Topic 040 2.38
Topic 028 11.05 Topic 041 1.09
0.6
Topic 029 10.46 Topic 042 1.28
Topic 030 3.05 Topic 043 0.33
Topic 031 18.49 Topic 044 6.51 0.4
Topic 032 3.23 Topic 045 6.62
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 16.67 Topic 048 6.25 0
Topic 036 0.00 Topic 049 16.67
Topic 037 2.60 Topic 050 6.51
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
231
u.twente utGeoTdIB GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 16.80
utGeoTdIB
10 docs 11.60 90%
15 docs 9.33
80%
20 docs 7.20
30 docs 5.87 70%
100 docs 2.40
60%
200 docs 1.32
R−Precision
500 docs 0.59 50%
1000 docs 0.30 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
7.62
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6667
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0323
Third Quartile 0.1067
Interquartile range 0.1067
Mean 0.0762
Standard Deviation 0.1388
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.2203 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0516
Std With No Outliers 0.0657
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries utGeoTdIB
Topic 026 0.00 Topic 039 6.25 0.8
Topic 027 5.26 Topic 040 7.14
Topic 028 15.79 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 0.00 Topic 043 0.00
Topic 031 22.03 Topic 044 10.53 0.4
Topic 032 3.23 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 16.67 Topic 048 6.25 0
Topic 036 0.00 Topic 049 0.00
Topic 037 6.25 Topic 050 13.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
232
u.twente utGeoTIBm GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 271 Pooled false
Geometric Mean Average Precision 0.0285 Filtering by geographic location and merging of
Binary Preference (BPREF) 0.1528 filtered and unfiltered results.
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 34.07
utGeoTIBm
10 29.52 90%
20 25.01
80%
30 23.94
40 20.69 70%
50 16.81
Average Precision
60%
60 12.77
70 11.46 50%
80 10.60 40%
90 8.99
30%
100 7.38
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
17.18 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0072
Second Quartile 0.0644
Third Quartile 0.1936
Interquartile range 0.1864
Mean 0.1718
Standard Deviation 0.2562
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4674 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0769
Std With No Outliers 0.1107
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries utGeoTIBm
Topic 026 0.81 Topic 039 6.93 0.8
Topic 027 1.05 Topic 040 0.02
Topic 028 8.95 Topic 041 0.13
0.6
Topic 029 9.58 Topic 042 1.09
Topic 030 20.08 Topic 043 0.17
Topic 031 46.74 Topic 044 14.93 0.4
Topic 032 57.26 Topic 045 0.85
Topic 033 0.31 Topic 046 100.00 0.2
Topic 034 4.93 Topic 047 6.44
Difference
Topic 035 0.42 Topic 048 51.59 0
Topic 036 0.00 Topic 049 59.09
Topic 037 4.63 Topic 050 19.12
−0.2
Topic 038 14.29
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
233
u.twente utGeoTIBm GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 18.40
utGeoTIBm
10 docs 17.20 90%
15 docs 17.33
80%
20 docs 16.20
30 docs 13.73 70%
100 docs 6.52
60%
200 docs 4.10
R−Precision
500 docs 1.98 50%
1000 docs 1.08 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
17.38
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0625
Third Quartile 0.2215
Interquartile range 0.2215
Mean 0.1738
Standard Deviation 0.2558
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1160
Std With No Outliers 0.1591
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries utGeoTIBm
Topic 026 0.00 Topic 039 12.50 0.8
Topic 027 0.00 Topic 040 0.00
Topic 028 15.79 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 16.67 Topic 043 0.00
Topic 031 37.29 Topic 044 18.42 0.4
Topic 032 67.74 Topic 045 0.00
Topic 033 5.00 Topic 046 100.00 0.2
Topic 034 0.00 Topic 047 12.50
Difference
Topic 035 0.00 Topic 048 47.92 0
Topic 036 0.00 Topic 049 50.00
Topic 037 6.25 Topic 050 33.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
234
u.twente utGeoTdnIB GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction MANUAL
Retrieved 10,175 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 85 Pooled true
Geometric Mean Average Precision 0.0032 Retrieval by content. Filtering by geographic
Binary Preference (BPREF) 0.1085 location through boolean matching of query title
locations and document locations.
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 40.83
utGeoTdnIB
10 24.57 90%
20 20.48
80%
30 15.84
40 14.44 70%
50 13.83
Average Precision
60%
60 7.24
70 2.07 50%
80 1.87 40%
90 0.79
30%
100 0.47
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
11.34 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6667
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0323
Third Quartile 0.1979
Interquartile range 0.1979
Mean 0.1134
Standard Deviation 0.1773
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3389 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0726
Std With No Outliers 0.1089
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries utGeoTdnIB
Topic 026 8.53 Topic 039 33.89 0.8
Topic 027 1.13 Topic 040 7.14
Topic 028 0.00 Topic 041 0.86
0.6
Topic 029 19.76 Topic 042 50.00
Topic 030 0.40 Topic 043 0.00
Topic 031 19.90 Topic 044 3.63 0.4
Topic 032 3.23 Topic 045 32.35
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 6.25 0
Topic 036 0.00 Topic 049 25.00
Topic 037 0.00 Topic 050 4.87
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
235
u.twente utGeoTdnIB GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 16.80
utGeoTdnIB
10 docs 12.80 90%
15 docs 10.13
80%
20 docs 9.00
30 docs 6.67 70%
100 docs 2.60
60%
200 docs 1.44
R−Precision
500 docs 0.63 50%
1000 docs 0.34 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
13.66
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6667
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0526
Third Quartile 0.2260
Interquartile range 0.2260
Mean 0.1366
Standard Deviation 0.2008
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1145
Std With No Outliers 0.1713
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries utGeoTdnIB
Topic 026 22.22 Topic 039 31.25 0.8
Topic 027 5.26 Topic 040 7.14
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 50.00
Topic 030 0.00 Topic 043 0.00
Topic 031 23.73 Topic 044 7.89 0.4
Topic 032 3.23 Topic 045 50.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 6.25 0
Topic 036 0.00 Topic 049 50.00
Topic 037 0.00 Topic 050 6.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
236
u.twente utGeoTdnIBm GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction MANUAL
Retrieved 25,000 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 285 Pooled false
Geometric Mean Average Precision 0.0380 Filtering by geographic location and merging of
Binary Preference (BPREF) 0.1484 filtered and unfiltered results.
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 43.07
utGeoTdnIBm
10 32.48 90%
20 26.90
80%
30 23.28
40 21.79 70%
50 21.18
Average Precision
60%
60 13.93
70 8.43 50%
80 7.31 40%
90 4.24
30%
100 3.11
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
16.77 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6816
Minimum 0.0000
First Quartile 0.0078
Second Quartile 0.0590
Third Quartile 0.3247
Interquartile range 0.3169
Mean 0.1677
Standard Deviation 0.2101
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6816 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1677
Std With No Outliers 0.2101
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries utGeoTdnIBm
Topic 026 8.61 Topic 039 33.89 0.8
Topic 027 1.13 Topic 040 33.55
Topic 028 5.62 Topic 041 0.86
0.6
Topic 029 19.76 Topic 042 50.16
Topic 030 0.40 Topic 043 0.54
Topic 031 34.65 Topic 044 7.99 0.4
Topic 032 68.16 Topic 045 32.11
Topic 033 0.21 Topic 046 66.86 0.2
Topic 034 5.90 Topic 047 1.28
Difference
Topic 035 0.34 Topic 048 15.20 0
Topic 036 0.00 Topic 049 25.00
Topic 037 0.40 Topic 050 4.87
−0.2
Topic 038 1.67
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
237
u.twente utGeoTdnIBm GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 19.20
utGeoTdnIBm
10 docs 16.80 90%
15 docs 14.40
80%
20 docs 13.60
30 docs 11.07 70%
100 docs 5.08
60%
200 docs 3.86
R−Precision
500 docs 2.02 50%
1000 docs 1.14 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
18.12
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7419
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0667
Third Quartile 0.2924
Interquartile range 0.2924
Mean 0.1812
Standard Deviation 0.2310
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1578
Std With No Outliers 0.2036
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries utGeoTdnIBm
Topic 026 22.22 Topic 039 31.25 0.8
Topic 027 5.26 Topic 040 28.57
Topic 028 15.79 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 50.00
Topic 030 0.00 Topic 043 0.00
Topic 031 27.12 Topic 044 7.89 0.4
Topic 032 74.19 Topic 045 50.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 6.25 0
Topic 036 0.00 Topic 049 50.00
Topic 037 0.00 Topic 050 6.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
238
unsw unswTitleBaseline GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 12,919 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 260 Pooled true
Geometric Mean Average Precision 0.0866 unswTitleBaseline
Binary Preference (BPREF) 0.2374
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 59.03
unswTitleBaseline
10 50.44 90%
20 43.13
80%
30 37.14
40 32.73 70%
50 30.93
Average Precision
60%
60 23.12
70 15.43 50%
80 10.41 40%
90 6.19
30%
100 3.08
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
26.22 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7722
Minimum 0.0000
First Quartile 0.0753
Second Quartile 0.2134
Third Quartile 0.4690
Interquartile range 0.3937
Mean 0.2622
Standard Deviation 0.2395
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7722 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2622
Std With No Outliers 0.2395
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
4
Number of Topics of the Experiment
3.5
3
2.5
2
1.5
1
0.5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries unswTitleBaseline
Topic 026 30.94 Topic 039 46.96 0.8
Topic 027 10.26 Topic 040 15.86
Topic 028 7.79 Topic 041 0.00
0.6
Topic 029 24.50 Topic 042 10.10
Topic 030 77.22 Topic 043 6.75
Topic 031 4.75 Topic 044 21.34 0.4
Topic 032 73.34 Topic 045 1.85
Topic 033 46.88 Topic 046 66.67 0.2
Topic 034 21.43 Topic 047 8.88
Difference
Topic 035 32.79 Topic 048 58.52 0
Topic 036 0.00 Topic 049 50.00
Topic 037 21.38 Topic 050 11.06
−0.2
Topic 038 6.25
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
239
unsw unswTitleBaseline GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 26.40
unswTitleBaseline
10 docs 23.60 90%
15 docs 22.40
80%
20 docs 21.40
30 docs 18.93 70%
100 docs 8.68
60%
200 docs 4.66
R−Precision
500 docs 2.03 50%
1000 docs 1.04 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
28.21
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8333
Minimum 0.0000
First Quartile 0.0956
Second Quartile 0.2105
Third Quartile 0.4625
Interquartile range 0.3669
Mean 0.2821
Standard Deviation 0.2517
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2821
Std With No Outliers 0.2517
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
5
Number of Topics of the Experiment
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries unswTitleBaseline
Topic 026 33.33 Topic 039 50.00 0.8
Topic 027 10.53 Topic 040 14.29
Topic 028 21.05 Topic 041 0.00
0.6
Topic 029 33.33 Topic 042 0.00
Topic 030 83.33 Topic 043 12.50
Topic 031 22.03 Topic 044 21.05 0.4
Topic 032 70.97 Topic 045 0.00
Topic 033 45.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 12.50
Difference
Topic 035 16.67 Topic 048 70.83 0
Topic 036 0.00 Topic 049 50.00
Topic 037 31.25 Topic 050 6.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
240
unsw unswNarrBaseline GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 15,905 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 265 Pooled true
Geometric Mean Average Precision 0.0854 unswNarrBaseline
Binary Preference (BPREF) 0.2313
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 59.86
unswNarrBaseline
10 49.40 90%
20 41.66
80%
30 37.01
40 33.68 70%
50 32.42
Average Precision
60%
60 24.96
70 17.37 50%
80 12.94 40%
90 8.61
30%
100 5.55
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
27.58 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9384
Minimum 0.0000
First Quartile 0.0516
Second Quartile 0.1650
Third Quartile 0.4421
Interquartile range 0.3904
Mean 0.2758
Standard Deviation 0.2670
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.9384 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2758
Std With No Outliers 0.2670
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
6
Number of Topics of the Experiment
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries unswNarrBaseline
Topic 026 30.94 Topic 039 45.42 0.8
Topic 027 10.26 Topic 040 15.86
Topic 028 3.35 Topic 041 0.00
0.6
Topic 029 4.55 Topic 042 36.67
Topic 030 77.22 Topic 043 16.50
Topic 031 5.37 Topic 044 17.23 0.4
Topic 032 93.84 Topic 045 3.96
Topic 033 38.88 Topic 046 66.67 0.2
Topic 034 2.30 Topic 047 11.41
Difference
Topic 035 43.80 Topic 048 68.54 0
Topic 036 0.00 Topic 049 50.00
Topic 037 21.38 Topic 050 11.06
−0.2
Topic 038 14.29
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
241
unsw unswNarrBaseline GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 28.80
unswNarrBaseline
10 docs 23.60 90%
15 docs 21.87
80%
20 docs 21.00
30 docs 19.20 70%
100 docs 8.80
60%
200 docs 4.84
R−Precision
500 docs 2.07 50%
1000 docs 1.06 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
25.88
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8710
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1579
Third Quartile 0.4062
Interquartile range 0.4062
Mean 0.2588
Standard Deviation 0.2769
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8710 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2588
Std With No Outliers 0.2769
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries unswNarrBaseline
Topic 026 33.33 Topic 039 37.50 0.8
Topic 027 10.53 Topic 040 14.29
Topic 028 5.26 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 0.00
Topic 030 83.33 Topic 043 12.50
Topic 031 22.03 Topic 044 15.79 0.4
Topic 032 87.10 Topic 045 0.00
Topic 033 50.00 Topic 046 66.67 0.2
Topic 034 0.00 Topic 047 16.67
Difference
Topic 035 33.33 Topic 048 70.83 0
Topic 036 0.00 Topic 049 50.00
Topic 037 31.25 Topic 050 6.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
242
unsw unswNarrMap GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 15,905 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 262 Pooled false
Geometric Mean Average Precision 0.0081 unswNarrMap
Binary Preference (BPREF) 0.0410
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 8.89
unswNarrMap
10 8.45 90%
20 8.06
80%
30 7.21
40 7.10 70%
50 6.05
Average Precision
60%
60 3.90
70 3.21 50%
80 1.66 40%
90 1.23
30%
100 0.69
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
4.00 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.3371
Minimum 0.0000
First Quartile 0.0031
Second Quartile 0.0098
Third Quartile 0.0501
Interquartile range 0.0470
Mean 0.0400
Standard Deviation 0.0701
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1106 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0276
Std With No Outliers 0.0336
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries unswNarrMap
Topic 026 0.56 Topic 039 3.50 0.8
Topic 027 10.26 Topic 040 0.30
Topic 028 0.31 Topic 041 0.00
0.6
Topic 029 0.53 Topic 042 0.33
Topic 030 6.55 Topic 043 0.54
Topic 031 3.31 Topic 044 4.78 0.4
Topic 032 5.71 Topic 045 1.42
Topic 033 33.71 Topic 046 3.90 0.2
Topic 034 0.13 Topic 047 0.98
Difference
Topic 035 3.11 Topic 048 8.06 0
Topic 036 0.00 Topic 049 0.09
Topic 037 0.81 Topic 050 11.06
−0.2
Topic 038 0.12
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
243
unsw unswNarrMap GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 1.60
unswNarrMap
10 docs 3.60 90%
15 docs 4.00
80%
20 docs 4.20
30 docs 4.67 70%
100 docs 2.80
60%
200 docs 2.72
R−Precision
500 docs 1.54 50%
1000 docs 1.05 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
4.06
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.5000
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.0354
Interquartile range 0.0354
Mean 0.0406
Standard Deviation 0.1045
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.0667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0109
Std With No Outliers 0.0229
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries unswNarrMap
Topic 026 0.00 Topic 039 6.25 0.8
Topic 027 10.53 Topic 040 0.00
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 0.00
Topic 030 0.00 Topic 043 0.00
Topic 031 16.95 Topic 044 2.63 0.4
Topic 032 6.45 Topic 045 0.00
Topic 033 50.00 Topic 046 0.00 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 2.08 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 6.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
244
unsw unswTitleF46 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 12,919 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 261 Pooled false
Geometric Mean Average Precision 0.0600 unswTitleF46
Binary Preference (BPREF) 0.2307
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 59.19
unswTitleF46
10 45.72 90%
20 38.67
80%
30 34.63
40 27.38 70%
50 25.37
Average Precision
60%
60 17.59
70 7.14 50%
80 4.56 40%
90 1.89
30%
100 0.78
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
22.15 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6667
Minimum 0.0000
First Quartile 0.0490
Second Quartile 0.1365
Third Quartile 0.4004
Interquartile range 0.3514
Mean 0.2215
Standard Deviation 0.2138
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2215
Std With No Outliers 0.2138
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
6
Number of Topics of the Experiment
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries unswTitleF46
Topic 026 15.04 Topic 039 34.07 0.8
Topic 027 12.32 Topic 040 13.65
Topic 028 5.09 Topic 041 0.00
0.6
Topic 029 16.33 Topic 042 1.04
Topic 030 61.69 Topic 043 4.33
Topic 031 5.09 Topic 044 13.80 0.4
Topic 032 53.54 Topic 045 2.38
Topic 033 44.77 Topic 046 66.67 0.2
Topic 034 38.46 Topic 047 9.80
Difference
Topic 035 28.06 Topic 048 51.55 0
Topic 036 0.00 Topic 049 50.00
Topic 037 13.17 Topic 050 12.73
−0.2
Topic 038 0.12
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
245
unsw unswTitleF46 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 29.60
unswTitleF46
10 docs 24.00 90%
15 docs 20.80
80%
20 docs 19.40
30 docs 16.53 70%
100 docs 7.00
60%
200 docs 4.04
R−Precision
500 docs 1.83 50%
1000 docs 1.04 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
26.87
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.6667
Minimum 0.0000
First Quartile 0.1104
Second Quartile 0.2203
Third Quartile 0.4062
Interquartile range 0.2958
Mean 0.2687
Standard Deviation 0.2167
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2687
Std With No Outliers 0.2167
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
5
Number of Topics of the Experiment
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries unswTitleF46
Topic 026 22.22 Topic 039 37.50 0.8
Topic 027 15.79 Topic 040 21.43
Topic 028 21.05 Topic 041 0.00
0.6
Topic 029 33.33 Topic 042 0.00
Topic 030 66.67 Topic 043 12.50
Topic 031 22.03 Topic 044 13.16 0.4
Topic 032 54.84 Topic 045 0.00
Topic 033 55.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 16.67
Difference
Topic 035 33.33 Topic 048 58.33 0
Topic 036 0.00 Topic 049 50.00
Topic 037 31.25 Topic 050 6.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
246
unsw unswNarrF41 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 15,905 Source Language English
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 262 Pooled false
Geometric Mean Average Precision 0.0083 unswNarrF41
Binary Preference (BPREF) 0.0411
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 8.91
unswNarrF41
10 8.48 90%
20 8.08
80%
30 7.23
40 7.12 70%
50 6.06
Average Precision
60%
60 3.91
70 3.21 50%
80 1.66 40%
90 1.23
30%
100 0.69
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
4.01 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.3371
Minimum 0.0000
First Quartile 0.0033
Second Quartile 0.0102
Third Quartile 0.0501
Interquartile range 0.0468
Mean 0.0401
Standard Deviation 0.0701
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1106 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0277
Std With No Outliers 0.0336
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries unswNarrF41
Topic 026 0.58 Topic 039 3.50 0.8
Topic 027 10.26 Topic 040 0.34
Topic 028 0.36 Topic 041 0.00
0.6
Topic 029 0.53 Topic 042 0.33
Topic 030 6.55 Topic 043 0.55
Topic 031 3.31 Topic 044 4.78 0.4
Topic 032 5.71 Topic 045 1.42
Topic 033 33.71 Topic 046 3.90 0.2
Topic 034 0.14 Topic 047 1.02
Difference
Topic 035 3.19 Topic 048 8.06 0
Topic 036 0.00 Topic 049 0.09
Topic 037 0.81 Topic 050 11.06
−0.2
Topic 038 0.12
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
247
unsw unswNarrF41 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 1.60
unswNarrF41
10 docs 3.60 90%
15 docs 4.00
80%
20 docs 4.20
30 docs 4.67 70%
100 docs 2.80
60%
200 docs 2.72
R−Precision
500 docs 1.57 50%
1000 docs 1.05 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
4.06
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.5000
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.0354
Interquartile range 0.0354
Mean 0.0406
Standard Deviation 0.1045
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.0667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0109
Std With No Outliers 0.0229
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries unswNarrF41
Topic 026 0.00 Topic 039 6.25 0.8
Topic 027 10.53 Topic 040 0.00
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 0.00
Topic 030 0.00 Topic 043 0.00
Topic 031 16.95 Topic 044 2.63 0.4
Topic 032 6.45 Topic 045 0.00
Topic 033 50.00 Topic 046 0.00 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 2.08 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 6.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
248
xldb XLDBGeoENAut02 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 22,483 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 300 Pooled false
Geometric Mean Average Precision 0.0268 Scope as topic term, no geoexpansion, QE 32 terms,
Binary Preference (BPREF) 0.1397 20 top-kdocs, relaxed query construction
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 36.69
XLDBGeoENAut02
10 25.36 90%
20 21.54
80%
30 21.03
40 17.13 70%
50 16.18
Average Precision
60%
60 14.30
70 11.96 50%
80 10.72 40%
90 7.61
30%
100 5.59
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
15.79 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8876
Minimum 0.0000
First Quartile 0.0063
Second Quartile 0.0617
Third Quartile 0.2071
Interquartile range 0.2008
Mean 0.1579
Standard Deviation 0.2344
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4247 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0980
Std With No Outliers 0.1137
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries XLDBGeoENAut02
Topic 026 3.28 Topic 039 8.45 0.8
Topic 027 5.49 Topic 040 42.47
Topic 028 17.53 Topic 041 0.00
0.6
Topic 029 24.16 Topic 042 0.25
Topic 030 6.17 Topic 043 4.87
Topic 031 10.56 Topic 044 19.56 0.4
Topic 032 88.76 Topic 045 0.12
Topic 033 0.64 Topic 046 0.59 0.2
Topic 034 25.65 Topic 047 12.68
Difference
Topic 035 2.07 Topic 048 80.50 0
Topic 036 0.00 Topic 049 26.67
Topic 037 12.41 Topic 050 0.10
−0.2
Topic 038 1.75
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
249
xldb XLDBGeoENAut02 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 20.80
XLDBGeoENAut02
10 docs 18.00 90%
15 docs 17.07
80%
20 docs 15.60
30 docs 14.40 70%
100 docs 7.28
60%
200 docs 4.40
R−Precision
500 docs 2.18 50%
1000 docs 1.20 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
15.28
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8387
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0526
Third Quartile 0.2566
Interquartile range 0.2566
Mean 0.1528
Standard Deviation 0.2334
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4286 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0970
Std With No Outliers 0.1362
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries XLDBGeoENAut02
Topic 026 0.00 Topic 039 6.25 0.8
Topic 027 5.26 Topic 040 42.86
Topic 028 31.58 Topic 041 0.00
0.6
Topic 029 33.33 Topic 042 0.00
Topic 030 0.00 Topic 043 12.50
Topic 031 13.56 Topic 044 23.68 0.4
Topic 032 83.87 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 33.33 Topic 047 8.33
Difference
Topic 035 0.00 Topic 048 75.00 0
Topic 036 0.00 Topic 049 0.00
Topic 037 12.50 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
250
xldb XLDBGeoENAut05 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 10,652 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 260 Pooled false
Geometric Mean Average Precision 0.0468 topic 16 QE terms expansion, top-20k docs, scope
Binary Preference (BPREF) 0.1994 expansion to 10 scopes
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 54.28
XLDBGeoENAut05
10 37.95 90%
20 28.74
80%
30 26.64
40 22.26 70%
50 21.51
Average Precision
60%
60 19.68
70 17.01 50%
80 12.35 40%
90 11.28
30%
100 9.39
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
21.45 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.9317
Minimum 0.0000
First Quartile 0.0479
Second Quartile 0.1083
Third Quartile 0.2828
Interquartile range 0.2349
Mean 0.2145
Standard Deviation 0.2387
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5833 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1618
Std With No Outliers 0.1575
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
6
Number of Topics of the Experiment
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries XLDBGeoENAut05
Topic 026 3.67 Topic 039 26.18 0.8
Topic 027 7.89 Topic 040 42.41
Topic 028 6.58 Topic 041 0.00
0.6
Topic 029 33.12 Topic 042 26.67
Topic 030 1.42 Topic 043 5.84
Topic 031 37.25 Topic 044 25.95 0.4
Topic 032 93.17 Topic 045 10.83
Topic 033 26.65 Topic 046 9.26 0.2
Topic 034 0.00 Topic 047 5.17
Difference
Topic 035 17.73 Topic 048 70.83 0
Topic 036 0.00 Topic 049 58.33
Topic 037 2.71 Topic 050 10.24
−0.2
Topic 038 14.29
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
251
xldb XLDBGeoENAut05 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 28.80
XLDBGeoENAut05
10 docs 24.00 90%
15 docs 22.40
80%
20 docs 21.20
30 docs 18.40 70%
100 docs 8.36
60%
200 docs 4.94
R−Precision
500 docs 2.08 50%
1000 docs 1.04 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
21.97
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.8710
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1250
Third Quartile 0.3600
Interquartile range 0.3600
Mean 0.2197
Standard Deviation 0.2367
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8710 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2197
Std With No Outliers 0.2367
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries XLDBGeoENAut05
Topic 026 0.00 Topic 039 31.25 0.8
Topic 027 10.53 Topic 040 35.71
Topic 028 10.53 Topic 041 0.00
0.6
Topic 029 44.44 Topic 042 0.00
Topic 030 0.00 Topic 043 12.50
Topic 031 38.98 Topic 044 36.84 0.4
Topic 032 87.10 Topic 045 33.33
Topic 033 25.00 Topic 046 0.00 0.2
Topic 034 0.00 Topic 047 12.50
Difference
Topic 035 16.67 Topic 048 70.83 0
Topic 036 0.00 Topic 049 50.00
Topic 037 6.25 Topic 050 26.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
252
xldb XLDBGeoManualEN GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction MANUAL
Retrieved 3,324 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 192 Pooled false
Geometric Mean Average Precision 0.0654 Manual Query
Binary Preference (BPREF) 0.3142
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 67.72
XLDBGeoManualEN
10 58.07 90%
20 41.48
80%
30 38.22
40 33.87 70%
50 30.42
Average Precision
60%
60 26.73
70 19.99 50%
80 15.56 40%
90 11.70
30%
100 11.58
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
30.34 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0604
Second Quartile 0.2343
Third Quartile 0.4979
Interquartile range 0.4375
Mean 0.3034
Standard Deviation 0.3075
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 1.0000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.3034
Std With No Outliers 0.3075
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
5
Number of Topics of the Experiment
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries XLDBGeoManualEN
Topic 026 23.77 Topic 039 57.78 0.8
Topic 027 7.82 Topic 040 23.43
Topic 028 7.41 Topic 041 0.00
0.6
Topic 029 3.70 Topic 042 0.00
Topic 030 97.62 Topic 043 3.12
Topic 031 9.60 Topic 044 13.79 0.4
Topic 032 74.88 Topic 045 33.88
Topic 033 47.13 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 5.29
Difference
Topic 035 24.83 Topic 048 70.23 0
Topic 036 0.00 Topic 049 25.00
Topic 037 6.29 Topic 050 23.00
−0.2
Topic 038 100.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
253
xldb XLDBGeoManualEN GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 38.40
XLDBGeoManualEN
10 docs 29.60 90%
15 docs 24.27
80%
20 docs 22.40
30 docs 19.73 70%
100 docs 7.20
60%
200 docs 3.70
R−Precision
500 docs 1.53 50%
1000 docs 0.77 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
33.60
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.1215
Second Quartile 0.3333
Third Quartile 0.5000
Interquartile range 0.3785
Mean 0.3360
Standard Deviation 0.2797
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 1.0000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.3360
Std With No Outliers 0.2797
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
5
Number of Topics of the Experiment
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries XLDBGeoManualEN
Topic 026 33.33 Topic 039 43.75 0.8
Topic 027 15.79 Topic 040 35.71
Topic 028 10.53 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 83.33 Topic 043 12.50
Topic 031 10.17 Topic 044 21.05 0.4
Topic 032 77.42 Topic 045 33.33
Topic 033 50.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 12.50
Difference
Topic 035 16.67 Topic 048 70.83 0
Topic 036 0.00 Topic 049 50.00
Topic 037 18.75 Topic 050 33.33
−0.2
Topic 038 100.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
254
xldb XLDBGeoENAut03_2 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 21,228 Source Language English
Relevant 363 Topic Fields title, description
Relevant retrieved 240 Pooled true
Geometric Mean Average Precision 0.0235 XLDBGeoENAut03 run, improved
Binary Preference (BPREF) 0.1912
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 50.31
XLDBGeoENAut03_2
10 44.28 90%
20 31.97
80%
30 29.25
40 23.36 70%
50 22.07
Average Precision
60%
60 16.40
70 12.12 50%
80 8.85 40%
90 3.17
30%
100 2.46
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
20.79 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7746
Minimum 0.0000
First Quartile 0.0114
Second Quartile 0.1462
Third Quartile 0.3866
Interquartile range 0.3752
Mean 0.2079
Standard Deviation 0.2365
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7746 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2079
Std With No Outliers 0.2365
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries XLDBGeoENAut03_2
Topic 026 1.29 Topic 039 23.93 0.8
Topic 027 0.72 Topic 040 42.47
Topic 028 14.62 Topic 041 0.00
0.6
Topic 029 37.96 Topic 042 61.11
Topic 030 45.91 Topic 043 14.83
Topic 031 19.48 Topic 044 3.12 0.4
Topic 032 77.46 Topic 045 0.05
Topic 033 2.65 Topic 046 40.74 0.2
Topic 034 21.98 Topic 047 7.23
Difference
Topic 035 24.47 Topic 048 69.73 0
Topic 036 0.00 Topic 049 0.00
Topic 037 8.74 Topic 050 0.00
−0.2
Topic 038 1.28
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
255
xldb XLDBGeoENAut03_2 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 24.00
XLDBGeoENAut03_2
10 docs 22.80 90%
15 docs 19.47
80%
20 docs 17.00
30 docs 14.67 70%
100 docs 6.84
60%
200 docs 4.00
R−Precision
500 docs 1.86 50%
1000 docs 0.96 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
21.53
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7083
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1250
Third Quartile 0.3571
Interquartile range 0.3571
Mean 0.2153
Standard Deviation 0.2252
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7083 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2153
Std With No Outliers 0.2252
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries XLDBGeoENAut03_2
Topic 026 0.00 Topic 039 25.00 0.8
Topic 027 0.00 Topic 040 42.86
Topic 028 21.05 Topic 041 0.00
0.6
Topic 029 55.56 Topic 042 50.00
Topic 030 50.00 Topic 043 12.50
Topic 031 22.03 Topic 044 10.53 0.4
Topic 032 64.52 Topic 045 0.00
Topic 033 5.00 Topic 046 33.33 0.2
Topic 034 33.33 Topic 047 12.50
Difference
Topic 035 16.67 Topic 048 70.83 0
Topic 036 0.00 Topic 049 0.00
Topic 037 12.50 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
256
xldb XLDBGeoENAut03 GC-MONO-EN-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 21,937 Source Language English
Relevant 378 Topic Fields title, description
Relevant retrieved 151 Pooled true
Geometric Mean Average Precision 0.0096 Run with geosim, final correction
Binary Preference (BPREF) 0.1812
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual English track − Interpolated Recall vs Average Precision
100%
0 47.52
XLDBGeoENAut03
10 37.27 90%
20 26.16
80%
30 24.45
40 22.20 70%
50 20.92
Average Precision
60%
60 16.77
70 11.11 50%
80 7.87 40%
90 2.91
30%
100 2.72
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
18.67 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7681
Minimum 0.0000
First Quartile 0.0004
Second Quartile 0.0490
Third Quartile 0.2300
Interquartile range 0.2296
Mean 0.1867
Standard Deviation 0.2728
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4247 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0608
Std With No Outliers 0.1002
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries XLDBGeoENAut03
Topic 026 6.72 Topic 039 16.51 0.8
Topic 027 0.00 Topic 040 42.47
Topic 028 12.31 Topic 041 0.00
0.6
Topic 029 5.56 Topic 042 64.29
Topic 030 67.86 Topic 043 15.63
Topic 031 2.88 Topic 044 3.03 0.4
Topic 032 76.81 Topic 045 0.05
Topic 033 1.12 Topic 046 66.67 0.2
Topic 034 0.00 Topic 047 6.90
Difference
Topic 035 2.08 Topic 048 69.68 0
Topic 036 0.00 Topic 049 0.00
Topic 037 4.90 Topic 050 1.39
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
257
xldb XLDBGeoENAut03 GC-MONO-EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual English track − Retrieved documents vs Precision
100%
5 docs 22.40
XLDBGeoENAut03
10 docs 20.80 90%
15 docs 18.40
80%
20 docs 16.00
30 docs 13.20 70%
100 docs 5.24
60%
200 docs 2.76
R−Precision
500 docs 1.19 50%
1000 docs 0.60 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
19.47
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual English track − Box plot of the Topics of the Experiment
Maximum 0.7083
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1111
Third Quartile 0.2946
Interquartile range 0.2946
Mean 0.1947
Standard Deviation 0.2360
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7083 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1947
Std With No Outliers 0.2360
GeoCLEF Monolingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries XLDBGeoENAut03
Topic 026 22.22 Topic 039 25.00 0.8
Topic 027 0.00 Topic 040 42.86
Topic 028 15.79 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 50.00
Topic 030 50.00 Topic 043 12.50
Topic 031 3.39 Topic 044 10.53 0.4
Topic 032 64.52 Topic 045 0.00
Topic 033 10.00 Topic 046 66.67 0.2
Topic 034 0.00 Topic 047 12.50
Difference
Topic 035 0.00 Topic 048 70.83 0
Topic 036 0.00 Topic 049 0.00
Topic 037 18.75 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
258
alicante esTD GC-MONO-ES-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Spanish; Castilian
Relevant 2,054 Topic Fields title, description
Relevant retrieved 1,819 Pooled true
Geometric Mean Average Precision 0.2182 Title and Description
Binary Preference (BPREF) 0.3665
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision
100%
0 71.04
esTD
10 56.86 90%
20 48.51
80%
30 42.92
40 38.60 70%
50 35.48
Average Precision
60%
60 32.23
70 27.55 50%
80 22.42 40%
90 17.08
30%
100 7.26
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
35.08 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.9708
Minimum 0.0197
First Quartile 0.1368
Second Quartile 0.3246
Third Quartile 0.5148
Interquartile range 0.3779
Mean 0.3508
Standard Deviation 0.2744
Lower Outlier Threshold 0.0197
Upper Outlier Threshold 0.9708 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.3508
Std With No Outliers 0.2744
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
5
Number of Topics of the Experiment
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries esTD
Topic 026 14.66 Topic 039 32.46 0.8
Topic 027 2.89 Topic 040 77.64
Topic 028 39.37 Topic 041 38.78
0.6
Topic 029 51.80 Topic 042 28.32
Topic 030 44.99 Topic 043 3.88
Topic 031 72.52 Topic 044 40.33 0.4
Topic 032 97.08 Topic 045 4.27
Topic 033 1.97 Topic 046 66.67 0.2
Topic 034 15.65 Topic 047 2.46
Difference
Topic 035 16.45 Topic 048 74.87 0
Topic 036 51.37 Topic 049 50.89
Topic 037 21.59 Topic 050 15.37
−0.2
Topic 038 10.76
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
259
alicante esTD GC-MONO-ES-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision
100%
5 docs 56.00
esTD
10 docs 52.80 90%
15 docs 49.07
80%
20 docs 47.00
30 docs 42.53 70%
100 docs 33.48
60%
200 docs 24.80
R−Precision
500 docs 13.30 50%
1000 docs 7.28 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
35.83
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.9000
Minimum 0.0256
First Quartile 0.1588
Second Quartile 0.3208
Third Quartile 0.5836
Interquartile range 0.4248
Mean 0.3583
Standard Deviation 0.2414
Lower Outlier Threshold 0.0256
Upper Outlier Threshold 0.9000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.3583
Std With No Outliers 0.2414
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
4
Number of Topics of the Experiment
3.5
3
2.5
2
1.5
1
0.5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries esTD
Topic 026 16.67 Topic 039 38.81 0.8
Topic 027 2.56 Topic 040 70.50
Topic 028 30.56 Topic 041 40.00
0.6
Topic 029 57.58 Topic 042 32.08
Topic 030 43.05 Topic 043 12.50
Topic 031 61.18 Topic 044 46.60 0.4
Topic 032 90.00 Topic 045 8.33
Topic 033 6.00 Topic 046 60.71 0.2
Topic 034 13.51 Topic 047 3.39
Difference
Topic 035 21.05 Topic 048 66.04 0
Topic 036 60.91 Topic 049 51.15
Topic 037 20.69 Topic 050 22.00
−0.2
Topic 038 20.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
260
alicante esTDN GC-MONO-ES-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Spanish; Castilian
Relevant 2,054 Topic Fields title, description
Relevant retrieved 1,733 Pooled true
Geometric Mean Average Precision 0.0962 Title, Description and Narrative
Binary Preference (BPREF) 0.3400
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision
100%
0 67.84
esTDN
10 50.74 90%
20 43.80
80%
30 37.61
40 35.84 70%
50 33.16
Average Precision
60%
60 29.49
70 27.03 50%
80 23.01 40%
90 14.50
30%
100 6.75
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
32.37 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.9591
Minimum 0.0000
First Quartile 0.0755
Second Quartile 0.2143
Third Quartile 0.5113
Interquartile range 0.4358
Mean 0.3237
Standard Deviation 0.2986
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.9591 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.3237
Std With No Outliers 0.2986
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
5
Number of Topics of the Experiment
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries esTDN
Topic 026 15.18 Topic 039 30.75 0.8
Topic 027 3.89 Topic 040 77.24
Topic 028 37.65 Topic 041 38.12
0.6
Topic 029 68.63 Topic 042 45.30
Topic 030 37.18 Topic 043 8.54
Topic 031 69.22 Topic 044 5.69 0.4
Topic 032 95.91 Topic 045 1.66
Topic 033 2.31 Topic 046 77.26 0.2
Topic 034 20.67 Topic 047 0.00
Difference
Topic 035 8.17 Topic 048 81.18 0
Topic 036 21.43 Topic 049 38.69
Topic 037 0.00 Topic 050 12.85
−0.2
Topic 038 11.78
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
261
alicante esTDN GC-MONO-ES-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision
100%
5 docs 49.60
esTDN
10 docs 47.20 90%
15 docs 43.73
80%
20 docs 40.40
30 docs 37.47 70%
100 docs 29.84
60%
200 docs 23.10
R−Precision
500 docs 12.31 50%
1000 docs 6.93 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
33.77
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.9000
Minimum 0.0000
First Quartile 0.1032
Second Quartile 0.2818
Third Quartile 0.5194
Interquartile range 0.4162
Mean 0.3377
Standard Deviation 0.2667
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.9000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.3377
Std With No Outliers 0.2667
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
4
Number of Topics of the Experiment
3.5
3
2.5
2
1.5
1
0.5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries esTDN
Topic 026 16.67 Topic 039 31.34 0.8
Topic 027 7.69 Topic 040 73.38
Topic 028 33.33 Topic 041 38.67
0.6
Topic 029 60.61 Topic 042 49.06
Topic 030 40.40 Topic 043 25.00
Topic 031 73.33 Topic 044 9.71 0.4
Topic 032 90.00 Topic 045 0.00
Topic 033 9.00 Topic 046 71.43 0.2
Topic 034 27.03 Topic 047 0.00
Difference
Topic 035 10.53 Topic 048 70.57 0
Topic 036 28.18 Topic 049 44.24
Topic 037 0.00 Topic 050 14.00
−0.2
Topic 038 20.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
262
alicante esTDNGeoNames GC-MONO-ES-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction MANUAL
Retrieved 25,000 Source Language Spanish; Castilian
Relevant 2,054 Topic Fields title, description, narrative
Relevant retrieved 1,069 Pooled true
Geometric Mean Average Precision 0.0036 Title, Description and Narrative with GeoNames
Binary Preference (BPREF) 0.1736
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision
100%
0 33.12
esTDNGeoNames
10 25.11 90%
20 20.55
80%
30 17.15
40 16.65 70%
50 16.53
Average Precision
60%
60 15.52
70 14.24 50%
80 10.01 40%
90 7.17
30%
100 5.40
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
15.25 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.9016
Minimum 0.0000
First Quartile 0.0001
Second Quartile 0.0047
Third Quartile 0.1699
Interquartile range 0.1699
Mean 0.1525
Standard Deviation 0.2635
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4128 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0508
Std With No Outliers 0.1021
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries esTDNGeoNames
Topic 026 15.18 Topic 039 4.34 0.8
Topic 027 10.35 Topic 040 77.55
Topic 028 0.47 Topic 041 22.45
0.6
Topic 029 0.07 Topic 042 4.58
Topic 030 0.00 Topic 043 0.11
Topic 031 41.28 Topic 044 0.01 0.4
Topic 032 90.16 Topic 045 0.11
Topic 033 54.64 Topic 046 6.47 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 0.15 Topic 048 52.18 0
Topic 036 0.00 Topic 049 0.06
Topic 037 0.00 Topic 050 1.04
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
263
alicante esTDNGeoNames GC-MONO-ES-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision
100%
5 docs 20.80
esTDNGeoNames
10 docs 20.40 90%
15 docs 20.53
80%
20 docs 21.20
30 docs 19.60 70%
100 docs 15.64
60%
200 docs 12.54
R−Precision
500 docs 7.47 50%
1000 docs 4.28 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
16.23
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.8077
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0184
Third Quartile 0.1850
Interquartile range 0.1850
Mean 0.1623
Standard Deviation 0.2661
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.2400 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0378
Std With No Outliers 0.0668
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries esTDNGeoNames
Topic 026 16.67 Topic 039 4.48 0.8
Topic 027 10.26 Topic 040 71.94
Topic 028 0.00 Topic 041 24.00
0.6
Topic 029 0.00 Topic 042 5.66
Topic 030 0.00 Topic 043 0.00
Topic 031 48.63 Topic 044 0.00 0.4
Topic 032 80.77 Topic 045 0.00
Topic 033 71.00 Topic 046 10.71 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 57.74 0
Topic 036 0.00 Topic 049 1.84
Topic 037 0.00 Topic 050 2.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
264
berkeley BKGeoS1 GC-MONO-ES-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Spanish; Castilian
Relevant 2,054 Topic Fields title, description
Relevant retrieved 1,646 Pooled true
Geometric Mean Average Precision 0.0695 Baseline TD run using standard logistic regression
Binary Preference (BPREF) 0.3406 algorithms and blind feedback
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision
100%
0 63.66
BKGeoS1
10 44.62 90%
20 42.22
80%
30 40.51
40 37.66 70%
50 34.19
Average Precision
60%
60 30.11
70 25.16 50%
80 21.68 40%
90 16.68
30%
100 5.32
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
31.82 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.9782
Minimum 0.0000
First Quartile 0.0377
Second Quartile 0.2212
Third Quartile 0.6252
Interquartile range 0.5875
Mean 0.3182
Standard Deviation 0.3239
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.9782 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.3182
Std With No Outliers 0.3239
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries BKGeoS1
Topic 026 0.05 Topic 039 1.48 0.8
Topic 027 0.00 Topic 040 60.86
Topic 028 11.37 Topic 041 22.12
0.6
Topic 029 61.67 Topic 042 31.72
Topic 030 68.69 Topic 043 1.33
Topic 031 65.07 Topic 044 41.34 0.4
Topic 032 97.82 Topic 045 4.56
Topic 033 0.53 Topic 046 81.92 0.2
Topic 034 25.28 Topic 047 12.03
Difference
Topic 035 4.76 Topic 048 80.52 0
Topic 036 0.00 Topic 049 78.74
Topic 037 11.95 Topic 050 27.21
−0.2
Topic 038 4.53
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
265
berkeley BKGeoS1 GC-MONO-ES-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision
100%
5 docs 41.60
BKGeoS1
10 docs 39.20 90%
15 docs 39.47
80%
20 docs 38.40
30 docs 37.33 70%
100 docs 31.76
60%
200 docs 22.66
R−Precision
500 docs 12.13 50%
1000 docs 6.58 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
32.11
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.9000
Minimum 0.0000
First Quartile 0.0432
Second Quartile 0.3243
Third Quartile 0.6110
Interquartile range 0.5678
Mean 0.3211
Standard Deviation 0.2958
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.9000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.3211
Std With No Outliers 0.2958
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
6
Number of Topics of the Experiment
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries BKGeoS1
Topic 026 0.00 Topic 039 1.49 0.8
Topic 027 0.00 Topic 040 62.59
Topic 028 5.56 Topic 041 33.33
0.6
Topic 029 60.61 Topic 042 37.74
Topic 030 66.89 Topic 043 8.33
Topic 031 56.08 Topic 044 43.69 0.4
Topic 032 90.00 Topic 045 8.33
Topic 033 1.00 Topic 046 75.00 0.2
Topic 034 32.43 Topic 047 22.03
Difference
Topic 035 5.26 Topic 048 72.08 0
Topic 036 0.00 Topic 049 70.51
Topic 037 13.79 Topic 050 36.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
266
berkeley BKGeoS2 GC-MONO-ES-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Spanish; Castilian
Relevant 2,054 Topic Fields title, description, narrative
Relevant retrieved 1,702 Pooled true
Geometric Mean Average Precision 0.0622 Baseline TDN with standard logistic regression
Binary Preference (BPREF) 0.3159 algorithms plus blind feedback
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision
100%
0 60.42
BKGeoS2
10 42.41 90%
20 38.92
80%
30 37.79
40 35.16 70%
50 32.88
Average Precision
60%
60 29.23
70 25.09 50%
80 20.87 40%
90 15.59
30%
100 5.60
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
30.03 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.9750
Minimum 0.0002
First Quartile 0.0174
Second Quartile 0.1834
Third Quartile 0.5828
Interquartile range 0.5654
Mean 0.3003
Standard Deviation 0.3199
Lower Outlier Threshold 0.0002
Upper Outlier Threshold 0.9750 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.3003
Std With No Outliers 0.3199
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries BKGeoS2
Topic 026 0.04 Topic 039 26.80 0.8
Topic 027 0.02 Topic 040 69.34
Topic 028 11.66 Topic 041 20.18
0.6
Topic 029 53.34 Topic 042 50.82
Topic 030 56.62 Topic 043 1.44
Topic 031 63.26 Topic 044 14.62 0.4
Topic 032 97.50 Topic 045 0.51
Topic 033 19.32 Topic 046 83.30 0.2
Topic 034 18.34 Topic 047 4.07
Difference
Topic 035 3.24 Topic 048 74.68 0
Topic 036 0.02 Topic 049 73.34
Topic 037 0.02 Topic 050 6.47
−0.2
Topic 038 1.84
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
267
berkeley BKGeoS2 GC-MONO-ES-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision
100%
5 docs 43.20
BKGeoS2
10 docs 37.20 90%
15 docs 37.87
80%
20 docs 37.80
30 docs 35.20 70%
100 docs 30.68
60%
200 docs 22.34
R−Precision
500 docs 12.06 50%
1000 docs 6.81 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
29.94
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.9154
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.2090
Third Quartile 0.5906
Interquartile range 0.5906
Mean 0.2994
Standard Deviation 0.2997
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.9154 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2994
Std With No Outliers 0.2997
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries BKGeoS2
Topic 026 0.00 Topic 039 20.90 0.8
Topic 027 0.00 Topic 040 71.22
Topic 028 16.67 Topic 041 24.00
0.6
Topic 029 57.58 Topic 042 52.83
Topic 030 55.63 Topic 043 0.00
Topic 031 63.53 Topic 044 21.36 0.4
Topic 032 91.54 Topic 045 0.00
Topic 033 34.00 Topic 046 71.43 0.2
Topic 034 18.92 Topic 047 6.78
Difference
Topic 035 5.26 Topic 048 64.15 0
Topic 036 0.00 Topic 049 68.66
Topic 037 0.00 Topic 050 4.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
268
daedalus GCesNA GC-MONO-ES-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction MANUAL
Retrieved 4,744 Source Language Spanish; Castilian
Relevant 1,965 Topic Fields title, description
Relevant retrieved 523 Pooled true
Geometric Mean Average Precision 0.0044 Mandatory run
Binary Preference (BPREF) 0.1534
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision
100%
0 46.90
GCesNA
10 33.03 90%
20 21.89
80%
30 16.67
40 11.79 70%
50 10.93
Average Precision
60%
60 9.97
70 4.70 50%
80 4.07 40%
90 0.34
30%
100 0.00
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
12.73 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.8575
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0546
Third Quartile 0.1612
Interquartile range 0.1612
Mean 0.1273
Standard Deviation 0.2056
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.2161 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0613
Std With No Outliers 0.0725
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries GCesNA
Topic 026 0.00 Topic 039 6.73 0.8
Topic 027 0.00 Topic 040 0.00
Topic 028 0.00 Topic 041 11.47
0.6
Topic 029 21.61 Topic 042 10.77
Topic 030 0.00 Topic 043 2.66
Topic 031 0.85 Topic 044 5.46 0.4
Topic 032 85.75 Topic 045 0.00
Topic 033 0.01 Topic 046 3.57 0.2
Topic 034 17.04 Topic 047 0.00
Difference
Topic 035 15.82 Topic 048 57.06 0
Topic 036 40.72 Topic 049 10.45
Topic 037 8.37 Topic 050 0.00
−0.2
Topic 038 20.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
269
daedalus GCesNA GC-MONO-ES-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision
100%
5 docs 35.20
GCesNA
10 docs 30.40 90%
15 docs 28.27
80%
20 docs 26.40
30 docs 23.20 70%
100 docs 15.64
60%
200 docs 9.64
R−Precision
500 docs 4.17 50%
1000 docs 2.09 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
17.18
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.8462
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1165
Third Quartile 0.2482
Interquartile range 0.2482
Mean 0.1718
Standard Deviation 0.2234
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1227
Std With No Outliers 0.1479
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries GCesNA
Topic 026 0.00 Topic 039 14.93 0.8
Topic 027 0.00 Topic 040 0.00
Topic 028 0.00 Topic 041 13.33
0.6
Topic 029 33.33 Topic 042 28.30
Topic 030 0.00 Topic 043 12.50
Topic 031 3.14 Topic 044 11.65 0.4
Topic 032 84.62 Topic 045 0.00
Topic 033 1.00 Topic 046 10.71 0.2
Topic 034 24.32 Topic 047 0.00
Difference
Topic 035 26.32 Topic 048 62.64 0
Topic 036 60.00 Topic 049 12.44
Topic 037 10.34 Topic 050 0.00
−0.2
Topic 038 20.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
270
daedalus GCesAtLg GC-MONO-ES-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction MANUAL
Retrieved 23,086 Source Language Spanish; Castilian
Relevant 2,015 Topic Fields title, description, narrative
Relevant retrieved 1,009 Pooled true
Geometric Mean Average Precision 0.0282 All text Left geo run
Binary Preference (BPREF) 0.1462
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision
100%
0 39.27
GCesAtLg
10 29.03 90%
20 24.95
80%
30 21.66
40 13.94 70%
50 11.26
Average Precision
60%
60 10.44
70 9.02 50%
80 7.67 40%
90 2.75
30%
100 0.22
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
14.13 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.8612
Minimum 0.0000
First Quartile 0.0122
Second Quartile 0.0689
Third Quartile 0.2049
Interquartile range 0.1927
Mean 0.1413
Standard Deviation 0.1926
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3504 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1113
Std With No Outliers 0.1234
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries GCesAtLg
Topic 026 0.00 Topic 039 11.12 0.8
Topic 027 0.00 Topic 040 34.29
Topic 028 1.27 Topic 041 28.73
0.6
Topic 029 12.31 Topic 042 14.86
Topic 030 30.43 Topic 043 2.88
Topic 031 1.07 Topic 044 3.41 0.4
Topic 032 86.12 Topic 045 0.44
Topic 033 0.01 Topic 046 35.04 0.2
Topic 034 6.89 Topic 047 3.72
Difference
Topic 035 9.81 Topic 048 34.86 0
Topic 036 0.59 Topic 049 3.77
Topic 037 17.74 Topic 050 8.07
−0.2
Topic 038 5.86
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
271
daedalus GCesAtLg GC-MONO-ES-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision
100%
5 docs 23.20
GCesAtLg
10 docs 23.20 90%
15 docs 22.67
80%
20 docs 22.00
30 docs 20.67 70%
100 docs 17.16
60%
200 docs 12.78
R−Precision
500 docs 6.98 50%
1000 docs 4.04 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
16.58
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.8538
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1053
Third Quartile 0.3249
Interquartile range 0.3249
Mean 0.1658
Standard Deviation 0.2010
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4038 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1371
Std With No Outliers 0.1440
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries GCesAtLg
Topic 026 0.00 Topic 039 16.42 0.8
Topic 027 0.00 Topic 040 35.25
Topic 028 0.00 Topic 041 32.00
0.6
Topic 029 18.18 Topic 042 33.96
Topic 030 34.44 Topic 043 0.00
Topic 031 5.10 Topic 044 9.71 0.4
Topic 032 85.38 Topic 045 0.00
Topic 033 1.00 Topic 046 35.71 0.2
Topic 034 0.00 Topic 047 1.69
Difference
Topic 035 10.53 Topic 048 40.38 0
Topic 036 5.45 Topic 049 11.06
Topic 037 24.14 Topic 050 14.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
272
daedalus GCesAO GC-MONO-ES-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction MANUAL
Retrieved 25,000 Source Language Spanish; Castilian
Relevant 2,054 Topic Fields title, description, narrative
Relevant retrieved 980 Pooled true
Geometric Mean Average Precision 0.0140 All text Or geo run
Binary Preference (BPREF) 0.1234
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision
100%
0 35.48
GCesAO
10 25.34 90%
20 18.82
80%
30 17.13
40 13.15 70%
50 10.54
Average Precision
60%
60 9.74
70 8.63 50%
80 7.58 40%
90 2.67
30%
100 0.22
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
12.21 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.8612
Minimum 0.0000
First Quartile 0.0055
Second Quartile 0.0457
Third Quartile 0.1458
Interquartile range 0.1403
Mean 0.1221
Standard Deviation 0.1907
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3486 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0913
Std With No Outliers 0.1150
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries GCesAO
Topic 026 0.00 Topic 039 11.12 0.8
Topic 027 1.08 Topic 040 34.29
Topic 028 0.00 Topic 041 26.36
0.6
Topic 029 12.31 Topic 042 4.57
Topic 030 30.43 Topic 043 2.88
Topic 031 1.07 Topic 044 3.41 0.4
Topic 032 86.12 Topic 045 0.44
Topic 033 0.00 Topic 046 21.40 0.2
Topic 034 6.89 Topic 047 0.04
Difference
Topic 035 9.81 Topic 048 34.86 0
Topic 036 0.59 Topic 049 3.77
Topic 037 0.00 Topic 050 8.07
−0.2
Topic 038 5.86
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
273
daedalus GCesAO GC-MONO-ES-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision
100%
5 docs 19.20
GCesAO
10 docs 19.20 90%
15 docs 18.67
80%
20 docs 18.60
30 docs 17.87 70%
100 docs 15.48
60%
200 docs 11.78
R−Precision
500 docs 6.72 50%
1000 docs 3.92 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
13.82
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.8538
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0545
Third Quartile 0.2030
Interquartile range 0.2030
Mean 0.1382
Standard Deviation 0.1967
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4038 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1084
Std With No Outliers 0.1311
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries GCesAO
Topic 026 0.00 Topic 039 16.42 0.8
Topic 027 2.56 Topic 040 35.25
Topic 028 0.00 Topic 041 26.67
0.6
Topic 029 18.18 Topic 042 1.89
Topic 030 34.44 Topic 043 0.00
Topic 031 5.10 Topic 044 9.71 0.4
Topic 032 85.38 Topic 045 0.00
Topic 033 0.00 Topic 046 28.57 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 10.53 Topic 048 40.38 0
Topic 036 5.45 Topic 049 11.06
Topic 037 0.00 Topic 050 14.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
274
daedalus GCesAA GC-MONO-ES-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction MANUAL
Retrieved 5,728 Source Language Spanish; Castilian
Relevant 2,015 Topic Fields title, description, narrative
Relevant retrieved 487 Pooled true
Geometric Mean Average Precision 0.0064 All text And geo run
Binary Preference (BPREF) 0.1531
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision
100%
0 48.87
GCesAA
10 33.01 90%
20 25.26
80%
30 18.98
40 14.42 70%
50 8.21
Average Precision
60%
60 7.04
70 5.98 50%
80 5.09 40%
90 0.00
30%
100 0.00
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
13.48 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.8575
Minimum 0.0000
First Quartile 0.0001
Second Quartile 0.0421
Third Quartile 0.1723
Interquartile range 0.1722
Mean 0.1348
Standard Deviation 0.2070
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4031 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0881
Std With No Outliers 0.1232
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries GCesAA
Topic 026 0.00 Topic 039 13.00 0.8
Topic 027 0.00 Topic 040 0.00
Topic 028 0.06 Topic 041 10.61
0.6
Topic 029 24.27 Topic 042 26.60
Topic 030 14.89 Topic 043 48.45
Topic 031 1.00 Topic 044 4.30 0.4
Topic 032 85.75 Topic 045 1.67
Topic 033 0.01 Topic 046 38.02 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 10.54 Topic 048 40.31 0
Topic 036 0.20 Topic 049 9.68
Topic 037 0.00 Topic 050 4.21
−0.2
Topic 038 3.33
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
275
daedalus GCesAA GC-MONO-ES-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision
100%
5 docs 37.60
GCesAA
10 docs 33.20 90%
15 docs 30.67
80%
20 docs 27.80
30 docs 25.60 70%
100 docs 15.68
60%
200 docs 9.04
R−Precision
500 docs 3.80 50%
1000 docs 1.95 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
17.01
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.8462
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1000
Third Quartile 0.2621
Interquartile range 0.2621
Mean 0.1701
Standard Deviation 0.2213
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5833 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1419
Std With No Outliers 0.1744
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries GCesAA
Topic 026 0.00 Topic 039 17.91 0.8
Topic 027 0.00 Topic 040 0.00
Topic 028 0.00 Topic 041 14.67
0.6
Topic 029 33.33 Topic 042 43.40
Topic 030 23.84 Topic 043 58.33
Topic 031 3.92 Topic 044 11.65 0.4
Topic 032 84.62 Topic 045 8.33
Topic 033 1.00 Topic 046 42.86 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 15.79 Topic 048 42.26 0
Topic 036 1.82 Topic 049 11.52
Topic 037 0.00 Topic 050 10.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
276
daedalus GCesNtLg GC-MONO-ES-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction MANUAL
Retrieved 21,736 Source Language Spanish; Castilian
Relevant 1,965 Topic Fields title, description
Relevant retrieved 1,207 Pooled true
Geometric Mean Average Precision 0.0236 Normal text Left geo run
Binary Preference (BPREF) 0.1640
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision
100%
0 41.02
GCesNtLg
10 29.36 90%
20 25.44
80%
30 21.66
40 18.85 70%
50 17.28
Average Precision
60%
60 15.97
70 11.64 50%
80 9.18 40%
90 3.52
30%
100 0.41
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
16.12 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.8612
Minimum 0.0000
First Quartile 0.0087
Second Quartile 0.1539
Third Quartile 0.2290
Interquartile range 0.2203
Mean 0.1612
Standard Deviation 0.2010
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3215 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1130
Std With No Outliers 0.1086
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries GCesNtLg
Topic 026 0.00 Topic 039 5.03 0.8
Topic 027 0.00 Topic 040 28.45
Topic 028 27.48 Topic 041 32.15
0.6
Topic 029 25.40 Topic 042 1.04
Topic 030 15.39 Topic 043 0.74
Topic 031 0.91 Topic 044 15.87 0.4
Topic 032 86.12 Topic 045 0.21
Topic 033 0.01 Topic 046 4.61 0.2
Topic 034 17.04 Topic 047 4.72
Difference
Topic 035 15.82 Topic 048 57.02 0
Topic 036 16.75 Topic 049 19.24
Topic 037 22.06 Topic 050 0.00
−0.2
Topic 038 6.94
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
277
daedalus GCesNtLg GC-MONO-ES-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision
100%
5 docs 24.80
GCesNtLg
10 docs 25.20 90%
15 docs 25.60
80%
20 docs 23.40
30 docs 21.47 70%
100 docs 18.24
60%
200 docs 14.46
R−Precision
500 docs 8.28 50%
1000 docs 4.83 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
18.59
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.8538
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1656
Third Quartile 0.2760
Interquartile range 0.2760
Mean 0.1859
Standard Deviation 0.2082
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6264 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1580
Std With No Outliers 0.1582
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries GCesNtLg
Topic 026 0.00 Topic 039 10.45 0.8
Topic 027 0.00 Topic 040 17.99
Topic 028 33.33 Topic 041 32.00
0.6
Topic 029 27.27 Topic 042 0.00
Topic 030 16.56 Topic 043 0.00
Topic 031 3.92 Topic 044 28.16 0.4
Topic 032 85.38 Topic 045 0.00
Topic 033 1.00 Topic 046 10.71 0.2
Topic 034 24.32 Topic 047 8.47
Difference
Topic 035 26.32 Topic 048 62.64 0
Topic 036 20.91 Topic 049 27.65
Topic 037 27.59 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
278
sanmarcos SMGeoES4 GC-MONO-ES-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Spanish; Castilian
Relevant 2,054 Topic Fields title, description
Relevant retrieved 746 Pooled true
Geometric Mean Average Precision 0.0270 Monolingual Spanish query expansion
Binary Preference (BPREF) 0.1608
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision
100%
0 57.59
SMGeoES4
10 37.87 90%
20 26.49
80%
30 18.15
40 14.18 70%
50 9.77
Average Precision
60%
60 7.66
70 2.17 50%
80 0.00 40%
90 0.00
30%
100 0.00
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
13.78 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.5530
Minimum 0.0000
First Quartile 0.0126
Second Quartile 0.0544
Third Quartile 0.2166
Interquartile range 0.2039
Mean 0.1378
Standard Deviation 0.1646
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3556 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1026
Std With No Outliers 0.1159
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoES4
Topic 026 1.71 Topic 039 30.24 0.8
Topic 027 1.90 Topic 040 55.30
Topic 028 26.13 Topic 041 1.03
0.6
Topic 029 35.56 Topic 042 33.36
Topic 030 11.09 Topic 043 0.00
Topic 031 1.34 Topic 044 12.63 0.4
Topic 032 20.17 Topic 045 0.60
Topic 033 0.10 Topic 046 10.51 0.2
Topic 034 18.91 Topic 047 5.44
Difference
Topic 035 12.17 Topic 048 5.40 0
Topic 036 4.34 Topic 049 53.01
Topic 037 0.03 Topic 050 3.42
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
279
sanmarcos SMGeoES4 GC-MONO-ES-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision
100%
5 docs 44.00
SMGeoES4
10 docs 37.20 90%
15 docs 35.20
80%
20 docs 32.40
30 docs 27.87 70%
100 docs 16.28
60%
200 docs 10.68
R−Precision
500 docs 5.46 50%
1000 docs 2.98 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
18.63
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.6267
Minimum 0.0000
First Quartile 0.0535
Second Quartile 0.1273
Third Quartile 0.3102
Interquartile range 0.2568
Mean 0.1863
Standard Deviation 0.1845
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6267 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1863
Std With No Outliers 0.1845
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
6
Number of Topics of the Experiment
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoES4
Topic 026 11.11 Topic 039 35.82 0.8
Topic 027 7.69 Topic 040 61.87
Topic 028 30.56 Topic 041 8.00
0.6
Topic 029 42.42 Topic 042 39.62
Topic 030 21.19 Topic 043 0.00
Topic 031 6.27 Topic 044 19.42 0.4
Topic 032 20.77 Topic 045 0.00
Topic 033 1.00 Topic 046 14.29 0.2
Topic 034 32.43 Topic 047 3.39
Difference
Topic 035 21.05 Topic 048 7.55 0
Topic 036 12.73 Topic 049 62.67
Topic 037 0.00 Topic 050 6.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
280
sanmarcos SMGeoES5 GC-MONO-ES-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Spanish; Castilian
Relevant 2,054 Topic Fields title, description, narrative
Relevant retrieved 743 Pooled true
Geometric Mean Average Precision 0.0350 Monolingual Spanish no query expansion
Binary Preference (BPREF) 0.1781
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision
100%
0 58.66
SMGeoES5
10 42.54 90%
20 29.48
80%
30 18.96
40 15.11 70%
50 10.69
Average Precision
60%
60 7.32
70 2.79 50%
80 0.00 40%
90 0.00
30%
100 0.00
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
14.71 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.5745
Minimum 0.0000
First Quartile 0.0099
Second Quartile 0.1102
Third Quartile 0.2251
Interquartile range 0.2152
Mean 0.1471
Standard Deviation 0.1590
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5408 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1293
Std With No Outliers 0.1345
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoES5
Topic 026 2.36 Topic 039 22.11 0.8
Topic 027 0.99 Topic 040 54.08
Topic 028 23.70 Topic 041 0.84
0.6
Topic 029 27.59 Topic 042 18.35
Topic 030 13.52 Topic 043 0.00
Topic 031 0.99 Topic 044 29.15 0.4
Topic 032 19.84 Topic 045 0.96
Topic 033 0.03 Topic 046 11.07 0.2
Topic 034 31.74 Topic 047 10.44
Difference
Topic 035 13.95 Topic 048 5.88 0
Topic 036 11.02 Topic 049 57.45
Topic 037 7.46 Topic 050 4.15
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
281
sanmarcos SMGeoES5 GC-MONO-ES-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision
100%
5 docs 46.40
SMGeoES5
10 docs 42.00 90%
15 docs 38.93
80%
20 docs 34.60
30 docs 30.67 70%
100 docs 18.16
60%
200 docs 11.82
R−Precision
500 docs 5.49 50%
1000 docs 2.97 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
20.44
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.6682
Minimum 0.0000
First Quartile 0.0696
Second Quartile 0.1724
Third Quartile 0.2791
Interquartile range 0.2095
Mean 0.2044
Standard Deviation 0.1839
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4595 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1668
Std With No Outliers 0.1355
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
5
Number of Topics of the Experiment
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoES5
Topic 026 11.11 Topic 039 22.39 0.8
Topic 027 0.00 Topic 040 60.43
Topic 028 27.78 Topic 041 6.67
0.6
Topic 029 42.42 Topic 042 28.30
Topic 030 20.53 Topic 043 0.00
Topic 031 7.06 Topic 044 37.86 0.4
Topic 032 20.77 Topic 045 0.00
Topic 033 1.00 Topic 046 14.29 0.2
Topic 034 45.95 Topic 047 16.95
Difference
Topic 035 21.05 Topic 048 7.55 0
Topic 036 22.73 Topic 049 66.82
Topic 037 17.24 Topic 050 12.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
282
sanmarcos SMGeoES1 GC-MONO-ES-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Spanish; Castilian
Relevant 2,054 Topic Fields title, description
Relevant retrieved 743 Pooled true
Geometric Mean Average Precision 0.0350 Monolingual Spanish title desc automatic
Binary Preference (BPREF) 0.1781
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision
100%
0 58.66
SMGeoES1
10 42.54 90%
20 29.49
80%
30 18.95
40 15.09 70%
50 10.69
Average Precision
60%
60 7.32
70 2.80 50%
80 0.00 40%
90 0.00
30%
100 0.00
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
14.71 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.5745
Minimum 0.0000
First Quartile 0.0099
Second Quartile 0.1102
Third Quartile 0.2252
Interquartile range 0.2153
Mean 0.1471
Standard Deviation 0.1590
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5408 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1293
Std With No Outliers 0.1345
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoES1
Topic 026 2.36 Topic 039 22.13 0.8
Topic 027 0.99 Topic 040 54.08
Topic 028 23.70 Topic 041 0.84
0.6
Topic 029 27.59 Topic 042 18.35
Topic 030 13.52 Topic 043 0.00
Topic 031 0.99 Topic 044 29.15 0.4
Topic 032 19.84 Topic 045 0.96
Topic 033 0.03 Topic 046 11.07 0.2
Topic 034 31.74 Topic 047 10.44
Difference
Topic 035 13.92 Topic 048 5.88 0
Topic 036 11.02 Topic 049 57.45
Topic 037 7.46 Topic 050 4.15
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
283
sanmarcos SMGeoES1 GC-MONO-ES-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision
100%
5 docs 46.40
SMGeoES1
10 docs 42.00 90%
15 docs 38.93
80%
20 docs 34.60
30 docs 30.67 70%
100 docs 18.16
60%
200 docs 11.82
R−Precision
500 docs 5.49 50%
1000 docs 2.97 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
20.44
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.6682
Minimum 0.0000
First Quartile 0.0696
Second Quartile 0.1724
Third Quartile 0.2791
Interquartile range 0.2095
Mean 0.2044
Standard Deviation 0.1839
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4595 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1668
Std With No Outliers 0.1355
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
5
Number of Topics of the Experiment
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoES1
Topic 026 11.11 Topic 039 22.39 0.8
Topic 027 0.00 Topic 040 60.43
Topic 028 27.78 Topic 041 6.67
0.6
Topic 029 42.42 Topic 042 28.30
Topic 030 20.53 Topic 043 0.00
Topic 031 7.06 Topic 044 37.86 0.4
Topic 032 20.77 Topic 045 0.00
Topic 033 1.00 Topic 046 14.29 0.2
Topic 034 45.95 Topic 047 16.95
Difference
Topic 035 21.05 Topic 048 7.55 0
Topic 036 22.73 Topic 049 66.82
Topic 037 17.24 Topic 050 12.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
284
sanmarcos SMGeoES2 GC-MONO-ES-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Spanish; Castilian
Relevant 2,054 Topic Fields title, description, narrative
Relevant retrieved 745 Pooled true
Geometric Mean Average Precision 0.0366 Monolingual Spanish title + desc + narr
Binary Preference (BPREF) 0.1806
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision
100%
0 60.75
SMGeoES2
10 44.52 90%
20 29.27
80%
30 20.27
40 15.21 70%
50 11.16
Average Precision
60%
60 7.54
70 2.82 50%
80 0.00 40%
90 0.00
30%
100 0.00
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
15.33 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.5854
Minimum 0.0000
First Quartile 0.0145
Second Quartile 0.1114
Third Quartile 0.2309
Interquartile range 0.2164
Mean 0.1533
Standard Deviation 0.1637
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5494 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1353
Std With No Outliers 0.1397
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoES2
Topic 026 1.92 Topic 039 22.49 0.8
Topic 027 1.13 Topic 040 54.94
Topic 028 24.89 Topic 041 0.86
0.6
Topic 029 29.49 Topic 042 19.64
Topic 030 14.26 Topic 043 0.00
Topic 031 1.56 Topic 044 29.43 0.4
Topic 032 19.94 Topic 045 1.13
Topic 033 0.03 Topic 046 13.10 0.2
Topic 034 35.33 Topic 047 11.14
Difference
Topic 035 15.45 Topic 048 5.92 0
Topic 036 10.93 Topic 049 58.54
Topic 037 7.52 Topic 050 3.61
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
285
sanmarcos SMGeoES2 GC-MONO-ES-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision
100%
5 docs 47.20
SMGeoES2
10 docs 42.40 90%
15 docs 39.73
80%
20 docs 36.80
30 docs 30.67 70%
100 docs 19.08
60%
200 docs 11.82
R−Precision
500 docs 5.48 50%
1000 docs 2.98 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
20.29
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.6682
Minimum 0.0000
First Quartile 0.0633
Second Quartile 0.1724
Third Quartile 0.2676
Interquartile range 0.2042
Mean 0.2029
Standard Deviation 0.1872
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5135 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1649
Std With No Outliers 0.1389
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
5
Number of Topics of the Experiment
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoES2
Topic 026 11.11 Topic 039 22.39 0.8
Topic 027 0.00 Topic 040 61.15
Topic 028 27.78 Topic 041 5.33
0.6
Topic 029 36.36 Topic 042 26.42
Topic 030 19.87 Topic 043 0.00
Topic 031 6.67 Topic 044 37.86 0.4
Topic 032 20.77 Topic 045 0.00
Topic 033 1.00 Topic 046 14.29 0.2
Topic 034 51.35 Topic 047 15.25
Difference
Topic 035 26.32 Topic 048 7.55 0
Topic 036 23.64 Topic 049 66.82
Topic 037 17.24 Topic 050 8.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
286
sanmarcos SMGeoES3 GC-MONO-ES-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction MANUAL
Retrieved 25,000 Source Language Spanish; Castilian
Relevant 2,054 Topic Fields title, description
Relevant retrieved 743 Pooled true
Geometric Mean Average Precision 0.0350 Monolingual Spanish adding information from
Binary Preference (BPREF) 0.1781 gazzeteers and other sources
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Spanish track − Interpolated Recall vs Average Precision
100%
0 58.66
SMGeoES3
10 42.54 90%
20 29.49
80%
30 18.95
40 15.09 70%
50 10.69
Average Precision
60%
60 7.32
70 2.80 50%
80 0.00 40%
90 0.00
30%
100 0.00
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
14.71 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.5745
Minimum 0.0000
First Quartile 0.0099
Second Quartile 0.1102
Third Quartile 0.2252
Interquartile range 0.2153
Mean 0.1471
Standard Deviation 0.1590
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5408 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1293
Std With No Outliers 0.1345
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoES3
Topic 026 2.36 Topic 039 22.13 0.8
Topic 027 0.99 Topic 040 54.08
Topic 028 23.70 Topic 041 0.84
0.6
Topic 029 27.59 Topic 042 18.35
Topic 030 13.52 Topic 043 0.00
Topic 031 0.99 Topic 044 29.15 0.4
Topic 032 19.84 Topic 045 0.96
Topic 033 0.03 Topic 046 11.07 0.2
Topic 034 31.74 Topic 047 10.44
Difference
Topic 035 13.92 Topic 048 5.88 0
Topic 036 11.02 Topic 049 57.45
Topic 037 7.46 Topic 050 4.15
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
287
sanmarcos SMGeoES3 GC-MONO-ES-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Spanish track − Retrieved documents vs Precision
100%
5 docs 46.40
SMGeoES3
10 docs 42.00 90%
15 docs 38.93
80%
20 docs 34.60
30 docs 30.67 70%
100 docs 18.16
60%
200 docs 11.82
R−Precision
500 docs 5.49 50%
1000 docs 2.97 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
20.44
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.6682
Minimum 0.0000
First Quartile 0.0696
Second Quartile 0.1724
Third Quartile 0.2791
Interquartile range 0.2095
Mean 0.2044
Standard Deviation 0.1839
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4595 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1668
Std With No Outliers 0.1355
GeoCLEF Monolingual Spanish track − Distribution of the Topics of the Experiment
5
Number of Topics of the Experiment
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoES3
Topic 026 11.11 Topic 039 22.39 0.8
Topic 027 0.00 Topic 040 60.43
Topic 028 27.78 Topic 041 6.67
0.6
Topic 029 42.42 Topic 042 28.30
Topic 030 20.53 Topic 043 0.00
Topic 031 7.06 Topic 044 37.86 0.4
Topic 032 20.77 Topic 045 0.00
Topic 033 1.00 Topic 046 14.29 0.2
Topic 034 45.95 Topic 047 16.95
Difference
Topic 035 21.05 Topic 048 7.55 0
Topic 036 22.73 Topic 049 66.82
Topic 037 17.24 Topic 050 12.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
288
berkeley BKGeoP2 GC-MONO-PT-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Portuguese
Relevant 1,060 Topic Fields title, description, narrative
Relevant retrieved 604 Pooled true
Geometric Mean Average Precision 0.0124 Baseline TDN run using standard logistic regression
Binary Preference (BPREF) 0.1469 algorithms plus blind feedback
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision
100%
0 36.34
BKGeoP2
10 30.25 90%
20 26.18
80%
30 21.39
40 19.96 70%
50 18.01
Average Precision
60%
60 14.41
70 11.42 50%
80 7.62 40%
90 4.50
30%
100 0.72
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
16.31 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.8680
Minimum 0.0000
First Quartile 0.0006
Second Quartile 0.0450
Third Quartile 0.2431
Interquartile range 0.2424
Mean 0.1631
Standard Deviation 0.2318
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5945 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1337
Std With No Outliers 0.1832
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries BKGeoP2
Topic 026 0.06 Topic 039 4.50 0.8
Topic 027 0.04 Topic 040 0.03
Topic 028 11.86 Topic 041 0.00
0.6
Topic 029 32.24 Topic 042 43.10
Topic 030 45.55 Topic 043 0.11
Topic 031 16.45 Topic 044 0.37 0.4
Topic 032 45.34 Topic 045 18.73
Topic 033 0.19 Topic 046 59.45 0.2
Topic 034 0.03 Topic 047 0.84
Difference
Topic 035 0.51 Topic 048 86.80 0
Topic 036 0.00 Topic 049 9.93
Topic 037 0.07 Topic 050 21.66
−0.2
Topic 038 9.78
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
289
berkeley BKGeoP2 GC-MONO-PT-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision
100%
5 docs 23.20
BKGeoP2
10 docs 24.40 90%
15 docs 22.13
80%
20 docs 20.00
30 docs 19.33 70%
100 docs 13.08
60%
200 docs 8.98
R−Precision
500 docs 4.46 50%
1000 docs 2.42 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
16.46
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.7832
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0098
Third Quartile 0.3007
Interquartile range 0.3007
Mean 0.1646
Standard Deviation 0.2348
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5909 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1388
Std With No Outliers 0.2005
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries BKGeoP2
Topic 026 0.00 Topic 039 4.35 0.8
Topic 027 0.98 Topic 040 0.00
Topic 028 15.62 Topic 041 0.00
0.6
Topic 029 38.46 Topic 042 48.57
Topic 030 50.00 Topic 043 0.00
Topic 031 14.75 Topic 044 0.00 0.4
Topic 032 49.06 Topic 045 19.51
Topic 033 0.00 Topic 046 59.09 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 78.32 0
Topic 036 0.00 Topic 049 5.56
Topic 037 0.00 Topic 050 27.27
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
290
berkeley BKGeoP1 GC-MONO-PT-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Portuguese
Relevant 1,060 Topic Fields title, description
Relevant retrieved 644 Pooled true
Geometric Mean Average Precision 0.0183 Baseline TD run using standard logistic regression
Binary Preference (BPREF) 0.1478 algorithms plus blind feedback
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision
100%
0 38.63
BKGeoP1
10 29.01 90%
20 25.25
80%
30 22.30
40 19.52 70%
50 18.16
Average Precision
60%
60 13.30
70 10.47 50%
80 7.54 40%
90 4.95
30%
100 2.28
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
16.22 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.9241
Minimum 0.0000
First Quartile 0.0015
Second Quartile 0.0426
Third Quartile 0.2736
Interquartile range 0.2721
Mean 0.1622
Standard Deviation 0.2339
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6566 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1305
Std With No Outliers 0.1755
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries BKGeoP1
Topic 026 0.47 Topic 039 4.26 0.8
Topic 027 0.12 Topic 040 0.08
Topic 028 14.82 Topic 041 0.01
0.6
Topic 029 27.73 Topic 042 10.65
Topic 030 65.66 Topic 043 0.02
Topic 031 27.24 Topic 044 0.65 0.4
Topic 032 46.22 Topic 045 35.38
Topic 033 0.13 Topic 046 28.66 0.2
Topic 034 0.16 Topic 047 0.30
Difference
Topic 035 0.94 Topic 048 92.41 0
Topic 036 0.00 Topic 049 17.81
Topic 037 3.90 Topic 050 19.57
−0.2
Topic 038 8.33
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
291
berkeley BKGeoP1 GC-MONO-PT-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision
100%
5 docs 25.60
BKGeoP1
10 docs 24.00 90%
15 docs 22.67
80%
20 docs 21.00
30 docs 20.00 70%
100 docs 12.60
60%
200 docs 9.00
R−Precision
500 docs 4.62 50%
1000 docs 2.58 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
16.43
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.8112
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0098
Third Quartile 0.2708
Interquartile range 0.2708
Mean 0.1643
Standard Deviation 0.2249
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5714 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1373
Std With No Outliers 0.1839
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries BKGeoP1
Topic 026 0.00 Topic 039 8.70 0.8
Topic 027 0.98 Topic 040 0.00
Topic 028 18.75 Topic 041 0.00
0.6
Topic 029 33.33 Topic 042 20.00
Topic 030 57.14 Topic 043 0.00
Topic 031 21.31 Topic 044 0.00 0.4
Topic 032 56.60 Topic 045 37.80
Topic 033 0.00 Topic 046 33.33 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 81.12 0
Topic 036 0.00 Topic 049 16.67
Topic 037 0.00 Topic 050 25.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
292
berkeley BKGeoP4 GC-MONO-PT-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Portuguese
Relevant 1,060 Topic Fields title, description, narrative
Relevant retrieved 607 Pooled true
Geometric Mean Average Precision 0.0126 Portuguese Monolingual using title, description and
Binary Preference (BPREF) 0.1555 narrative (Corrected Queries)
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision
100%
0 36.34
BKGeoP4
10 30.19 90%
20 27.04
80%
30 22.45
40 21.01 70%
50 19.92
Average Precision
60%
60 16.13
70 12.72 50%
80 7.85 40%
90 4.59
30%
100 0.72
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
17.36 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.8680
Minimum 0.0000
First Quartile 0.0006
Second Quartile 0.0450
Third Quartile 0.2431
Interquartile range 0.2424
Mean 0.1736
Standard Deviation 0.2495
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5945 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1210
Std With No Outliers 0.1762
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries BKGeoP4
Topic 026 0.06 Topic 039 4.50 0.8
Topic 027 0.04 Topic 040 0.03
Topic 028 11.86 Topic 041 0.00
0.6
Topic 029 32.24 Topic 042 45.81
Topic 030 45.55 Topic 043 0.11
Topic 031 16.45 Topic 044 0.37 0.4
Topic 032 68.91 Topic 045 18.73
Topic 033 0.19 Topic 046 59.45 0.2
Topic 034 0.03 Topic 047 0.84
Difference
Topic 035 0.51 Topic 048 86.80 0
Topic 036 0.00 Topic 049 9.93
Topic 037 0.07 Topic 050 21.66
−0.2
Topic 038 9.79
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
293
berkeley BKGeoP4 GC-MONO-PT-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision
100%
5 docs 24.00
BKGeoP4
10 docs 25.60 90%
15 docs 22.67
80%
20 docs 21.40
30 docs 20.27 70%
100 docs 13.44
60%
200 docs 9.04
R−Precision
500 docs 4.50 50%
1000 docs 2.43 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
17.22
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.7832
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0098
Third Quartile 0.3007
Interquartile range 0.3007
Mean 0.1722
Standard Deviation 0.2484
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6792 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1467
Std With No Outliers 0.2179
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries BKGeoP4
Topic 026 0.00 Topic 039 4.35 0.8
Topic 027 0.98 Topic 040 0.00
Topic 028 15.62 Topic 041 0.00
0.6
Topic 029 38.46 Topic 042 48.57
Topic 030 50.00 Topic 043 0.00
Topic 031 14.75 Topic 044 0.00 0.4
Topic 032 67.92 Topic 045 19.51
Topic 033 0.00 Topic 046 59.09 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 78.32 0
Topic 036 0.00 Topic 049 5.56
Topic 037 0.00 Topic 050 27.27
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
294
berkeley BKGeoP3 GC-MONO-PT-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Portuguese
Relevant 1,060 Topic Fields title, description
Relevant retrieved 644 Pooled true
Geometric Mean Average Precision 0.0186 Portuguese Monolingual using title and desc
Binary Preference (BPREF) 0.1514 (corrected queries)
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision
100%
0 39.15
BKGeoP3
10 29.53 90%
20 25.77
80%
30 22.82
40 20.17 70%
50 18.78
Average Precision
60%
60 13.98
70 12.00 50%
80 7.54 40%
90 4.95
30%
100 2.28
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
16.92 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.9241
Minimum 0.0000
First Quartile 0.0015
Second Quartile 0.0426
Third Quartile 0.2736
Interquartile range 0.2721
Mean 0.1692
Standard Deviation 0.2457
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6566 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1378
Std With No Outliers 0.1928
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries BKGeoP3
Topic 026 0.47 Topic 039 4.26 0.8
Topic 027 0.12 Topic 040 0.08
Topic 028 14.82 Topic 041 0.01
0.6
Topic 029 27.73 Topic 042 10.65
Topic 030 65.66 Topic 043 0.02
Topic 031 27.24 Topic 044 0.65 0.4
Topic 032 63.83 Topic 045 35.38
Topic 033 0.13 Topic 046 28.66 0.2
Topic 034 0.16 Topic 047 0.30
Difference
Topic 035 0.94 Topic 048 92.41 0
Topic 036 0.00 Topic 049 17.81
Topic 037 3.90 Topic 050 19.57
−0.2
Topic 038 8.34
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
295
berkeley BKGeoP3 GC-MONO-PT-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision
100%
5 docs 27.20
BKGeoP3
10 docs 24.80 90%
15 docs 23.47
80%
20 docs 21.60
30 docs 20.27 70%
100 docs 12.84
60%
200 docs 9.12
R−Precision
500 docs 4.65 50%
1000 docs 2.58 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
16.51
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.8112
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0098
Third Quartile 0.2708
Interquartile range 0.2708
Mean 0.1651
Standard Deviation 0.2263
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5849 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1381
Std With No Outliers 0.1858
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries BKGeoP3
Topic 026 0.00 Topic 039 8.70 0.8
Topic 027 0.98 Topic 040 0.00
Topic 028 18.75 Topic 041 0.00
0.6
Topic 029 33.33 Topic 042 20.00
Topic 030 57.14 Topic 043 0.00
Topic 031 21.31 Topic 044 0.00 0.4
Topic 032 58.49 Topic 045 37.80
Topic 033 0.00 Topic 046 33.33 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 81.12 0
Topic 036 0.00 Topic 049 16.67
Topic 037 0.00 Topic 050 25.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
296
sanmarcos SMGeoPT4 GC-MONO-PT-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Portuguese
Relevant 1,060 Topic Fields title, description
Relevant retrieved 535 Pooled true
Geometric Mean Average Precision 0.0153 Automatic Portuguese title+desc no query expansion
Binary Preference (BPREF) 0.1052
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision
100%
0 39.55
SMGeoPT4
10 24.96 90%
20 18.86
80%
30 14.25
40 10.88 70%
50 8.91
Average Precision
60%
60 6.55
70 5.01 50%
80 3.77 40%
90 1.55
30%
100 0.63
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
10.63 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.4830
Minimum 0.0000
First Quartile 0.0044
Second Quartile 0.0317
Third Quartile 0.1791
Interquartile range 0.1748
Mean 0.1063
Standard Deviation 0.1438
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3088 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0746
Std With No Outliers 0.0972
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoPT4
Topic 026 0.21 Topic 039 15.43 0.8
Topic 027 0.06 Topic 040 0.57
Topic 028 3.06 Topic 041 0.51
0.6
Topic 029 17.87 Topic 042 30.88
Topic 030 1.12 Topic 043 0.10
Topic 031 18.06 Topic 044 3.17 0.4
Topic 032 48.30 Topic 045 15.17
Topic 033 0.00 Topic 046 22.49 0.2
Topic 034 4.38 Topic 047 0.00
Difference
Topic 035 1.61 Topic 048 45.81 0
Topic 036 1.67 Topic 049 26.45
Topic 037 0.00 Topic 050 3.91
−0.2
Topic 038 4.90
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
297
sanmarcos SMGeoPT4 GC-MONO-PT-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision
100%
5 docs 20.80
SMGeoPT4
10 docs 20.00 90%
15 docs 19.47
80%
20 docs 18.40
30 docs 16.27 70%
100 docs 10.20
60%
200 docs 6.72
R−Precision
500 docs 3.67 50%
1000 docs 2.14 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
13.57
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.5283
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0833
Third Quartile 0.2006
Interquartile range 0.2006
Mean 0.1357
Standard Deviation 0.1636
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3714 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1020
Std With No Outliers 0.1200
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoPT4
Topic 026 0.00 Topic 039 17.39 0.8
Topic 027 0.98 Topic 040 8.33
Topic 028 6.25 Topic 041 1.92
0.6
Topic 029 15.38 Topic 042 37.14
Topic 030 7.14 Topic 043 0.00
Topic 031 14.75 Topic 044 10.53 0.4
Topic 032 52.83 Topic 045 28.05
Topic 033 0.00 Topic 046 31.82 0.2
Topic 034 12.50 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 51.75 0
Topic 036 0.00 Topic 049 33.33
Topic 037 0.00 Topic 050 9.09
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
298
sanmarcos SMGeoPT2 GC-MONO-PT-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Portuguese
Relevant 1,060 Topic Fields title, description
Relevant retrieved 620 Pooled true
Geometric Mean Average Precision 0.0524 Automatic Portuguese title+desc
Binary Preference (BPREF) 0.1211
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision
100%
0 47.69
SMGeoPT2
10 29.63 90%
20 23.19
80%
30 18.01
40 13.97 70%
50 12.26
Average Precision
60%
60 9.27
70 6.21 50%
80 4.01 40%
90 2.35
30%
100 0.40
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
13.44 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.5240
Minimum 0.0006
First Quartile 0.0210
Second Quartile 0.0638
Third Quartile 0.1717
Interquartile range 0.1506
Mean 0.1344
Standard Deviation 0.1651
Lower Outlier Threshold 0.0006
Upper Outlier Threshold 0.3276 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0835
Std With No Outliers 0.0918
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoPT2
Topic 026 1.04 Topic 039 32.76 0.8
Topic 027 6.09 Topic 040 1.52
Topic 028 14.88 Topic 041 1.14
0.6
Topic 029 6.38 Topic 042 52.40
Topic 030 47.73 Topic 043 0.06
Topic 031 14.69 Topic 044 7.68 0.4
Topic 032 52.29 Topic 045 3.34
Topic 033 2.14 Topic 046 23.48 0.2
Topic 034 2.39 Topic 047 9.14
Difference
Topic 035 1.99 Topic 048 2.86 0
Topic 036 2.19 Topic 049 15.07
Topic 037 0.21 Topic 050 8.54
−0.2
Topic 038 26.01
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
299
sanmarcos SMGeoPT2 GC-MONO-PT-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision
100%
5 docs 26.40
SMGeoPT2
10 docs 21.60 90%
15 docs 19.20
80%
20 docs 17.20
30 docs 15.47 70%
100 docs 9.64
60%
200 docs 6.52
R−Precision
500 docs 3.94 50%
1000 docs 2.48 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
15.02
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.6038
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0979
Third Quartile 0.1875
Interquartile range 0.1875
Mean 0.1502
Standard Deviation 0.1726
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4286 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1134
Std With No Outliers 0.1213
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoPT2
Topic 026 0.00 Topic 039 34.78 0.8
Topic 027 10.78 Topic 040 4.17
Topic 028 15.62 Topic 041 1.92
0.6
Topic 029 15.38 Topic 042 54.29
Topic 030 42.86 Topic 043 0.00
Topic 031 14.75 Topic 044 7.89 0.4
Topic 032 60.38 Topic 045 8.54
Topic 033 0.00 Topic 046 30.30 0.2
Topic 034 0.00 Topic 047 8.82
Difference
Topic 035 0.00 Topic 048 9.79 0
Topic 036 0.00 Topic 049 16.67
Topic 037 0.00 Topic 050 13.64
−0.2
Topic 038 25.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
300
sanmarcos SMGeoPT1 GC-MONO-PT-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Portuguese
Relevant 1,060 Topic Fields title, description, narrative
Relevant retrieved 537 Pooled true
Geometric Mean Average Precision 0.0154 Automatic Portuguese title+desc+narr
Binary Preference (BPREF) 0.1094
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision
100%
0 39.88
SMGeoPT1
10 25.54 90%
20 18.96
80%
30 14.50
40 11.25 70%
50 8.76
Average Precision
60%
60 7.06
70 5.36 50%
80 3.99 40%
90 1.66
30%
100 0.50
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
10.98 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.5345
Minimum 0.0000
First Quartile 0.0047
Second Quartile 0.0316
Third Quartile 0.1796
Interquartile range 0.1748
Mean 0.1098
Standard Deviation 0.1511
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.2824 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0749
Std With No Outliers 0.0949
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoPT1
Topic 026 0.22 Topic 039 14.60 0.8
Topic 027 0.05 Topic 040 0.73
Topic 028 2.70 Topic 041 0.56
0.6
Topic 029 17.80 Topic 042 28.24
Topic 030 1.61 Topic 043 0.09
Topic 031 18.42 Topic 044 3.16 0.4
Topic 032 53.45 Topic 045 16.61
Topic 033 0.00 Topic 046 22.19 0.2
Topic 034 4.57 Topic 047 0.00
Difference
Topic 035 1.53 Topic 048 48.65 0
Topic 036 2.66 Topic 049 27.21
Topic 037 0.01 Topic 050 3.96
−0.2
Topic 038 5.44
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
301
sanmarcos SMGeoPT1 GC-MONO-PT-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision
100%
5 docs 20.80
SMGeoPT1
10 docs 20.80 90%
15 docs 20.00
80%
20 docs 18.80
30 docs 16.80 70%
100 docs 10.32
60%
200 docs 6.90
R−Precision
500 docs 3.75 50%
1000 docs 2.15 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
13.91
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.5849
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0909
Third Quartile 0.2047
Interquartile range 0.2047
Mean 0.1391
Standard Deviation 0.1716
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3889 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1033
Std With No Outliers 0.1235
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoPT1
Topic 026 0.00 Topic 039 13.04 0.8
Topic 027 0.98 Topic 040 8.33
Topic 028 9.38 Topic 041 0.96
0.6
Topic 029 17.95 Topic 042 34.29
Topic 030 7.14 Topic 043 0.00
Topic 031 13.11 Topic 044 10.53 0.4
Topic 032 58.49 Topic 045 28.05
Topic 033 0.00 Topic 046 33.33 0.2
Topic 034 12.50 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 51.75 0
Topic 036 0.00 Topic 049 38.89
Topic 037 0.00 Topic 050 9.09
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
302
sanmarcos SMGeoPT3 GC-MONO-PT-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Portuguese
Relevant 1,060 Topic Fields title, description, narrative
Relevant retrieved 537 Pooled true
Geometric Mean Average Precision 0.0154 Automatic Portuguese title+desc+narr no query
Binary Preference (BPREF) 0.1094 expansion
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision
100%
0 39.88
SMGeoPT3
10 25.54 90%
20 18.96
80%
30 14.50
40 11.25 70%
50 8.76
Average Precision
60%
60 7.06
70 5.36 50%
80 3.99 40%
90 1.66
30%
100 0.50
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
10.98 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.5345
Minimum 0.0000
First Quartile 0.0047
Second Quartile 0.0316
Third Quartile 0.1796
Interquartile range 0.1748
Mean 0.1098
Standard Deviation 0.1511
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.2824 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0749
Std With No Outliers 0.0949
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoPT3
Topic 026 0.22 Topic 039 14.60 0.8
Topic 027 0.05 Topic 040 0.73
Topic 028 2.70 Topic 041 0.56
0.6
Topic 029 17.80 Topic 042 28.24
Topic 030 1.61 Topic 043 0.09
Topic 031 18.42 Topic 044 3.16 0.4
Topic 032 53.45 Topic 045 16.61
Topic 033 0.00 Topic 046 22.19 0.2
Topic 034 4.57 Topic 047 0.00
Difference
Topic 035 1.53 Topic 048 48.65 0
Topic 036 2.66 Topic 049 27.21
Topic 037 0.01 Topic 050 3.96
−0.2
Topic 038 5.44
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
303
sanmarcos SMGeoPT3 GC-MONO-PT-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision
100%
5 docs 20.80
SMGeoPT3
10 docs 20.80 90%
15 docs 20.00
80%
20 docs 18.80
30 docs 16.80 70%
100 docs 10.32
60%
200 docs 6.90
R−Precision
500 docs 3.75 50%
1000 docs 2.15 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
13.91
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.5849
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0909
Third Quartile 0.2047
Interquartile range 0.2047
Mean 0.1391
Standard Deviation 0.1716
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3889 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1033
Std With No Outliers 0.1235
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoPT3
Topic 026 0.00 Topic 039 13.04 0.8
Topic 027 0.98 Topic 040 8.33
Topic 028 9.38 Topic 041 0.96
0.6
Topic 029 17.95 Topic 042 34.29
Topic 030 7.14 Topic 043 0.00
Topic 031 13.11 Topic 044 10.53 0.4
Topic 032 58.49 Topic 045 28.05
Topic 033 0.00 Topic 046 33.33 0.2
Topic 034 12.50 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 51.75 0
Topic 036 0.00 Topic 049 38.89
Topic 037 0.00 Topic 050 9.09
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
304
xldb XLDBGeoPTAut02 GC-MONO-PT-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction MANUAL
Retrieved 23,350 Source Language Portuguese
Relevant 1,060 Topic Fields title, description
Relevant retrieved 828 Pooled true
Geometric Mean Average Precision 0.1096 Scope as topic term, no geoexpansion, QE 32 terms,
Binary Preference (BPREF) 0.2540 20 top-kdocs, relaxed query construction
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision
100%
0 55.25
XLDBGeoPTAut02
10 48.65 90%
20 43.78
80%
30 35.68
40 29.17 70%
50 25.56
Average Precision
60%
60 21.93
70 15.92 50%
80 11.48 40%
90 6.94
30%
100 1.24
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
25.70 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.7629
Minimum 0.0020
First Quartile 0.0303
Second Quartile 0.2519
Third Quartile 0.4039
Interquartile range 0.3736
Mean 0.2570
Standard Deviation 0.2333
Lower Outlier Threshold 0.0020
Upper Outlier Threshold 0.7629 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2570
Std With No Outliers 0.2333
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries XLDBGeoPTAut02
Topic 026 3.24 Topic 039 12.29 0.8
Topic 027 0.20 Topic 040 27.33
Topic 028 32.96 Topic 041 69.30
0.6
Topic 029 41.38 Topic 042 40.06
Topic 030 46.28 Topic 043 0.25
Topic 031 24.84 Topic 044 31.02 0.4
Topic 032 70.73 Topic 045 28.41
Topic 033 0.54 Topic 046 48.56 0.2
Topic 034 25.19 Topic 047 6.10
Difference
Topic 035 1.23 Topic 048 76.29 0
Topic 036 2.41 Topic 049 27.60
Topic 037 12.16 Topic 050 12.26
−0.2
Topic 038 1.74
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
305
xldb XLDBGeoPTAut02 GC-MONO-PT-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision
100%
5 docs 41.60
XLDBGeoPTAut02
10 docs 39.20 90%
15 docs 36.00
80%
20 docs 35.00
30 docs 32.40 70%
100 docs 19.32
60%
200 docs 13.00
R−Precision
500 docs 6.26 50%
1000 docs 3.31 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
28.09
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.6783
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.3279
Third Quartile 0.4351
Interquartile range 0.4351
Mean 0.2809
Standard Deviation 0.2282
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6783 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2809
Std With No Outliers 0.2282
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries XLDBGeoPTAut02
Topic 026 0.00 Topic 039 17.39 0.8
Topic 027 0.00 Topic 040 33.33
Topic 028 37.50 Topic 041 67.31
0.6
Topic 029 48.72 Topic 042 48.57
Topic 030 42.86 Topic 043 0.00
Topic 031 32.79 Topic 044 42.11 0.4
Topic 032 66.04 Topic 045 36.59
Topic 033 0.00 Topic 046 45.45 0.2
Topic 034 37.50 Topic 047 0.00
Difference
Topic 035 11.11 Topic 048 67.83 0
Topic 036 0.00 Topic 049 27.78
Topic 037 16.67 Topic 050 22.73
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
306
xldb XLDBGeoPTAut05 GC-MONO-PT-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 10,483 Source Language Portuguese
Relevant 1,060 Topic Fields title, description
Relevant retrieved 624 Pooled true
Geometric Mean Average Precision 0.1208 topic 16 QE terms expansion, top-20k docs, scope
Binary Preference (BPREF) 0.3055 expansion to 10 scopes
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision
100%
0 71.58
XLDBGeoPTAut05
10 57.65 90%
20 49.88
80%
30 45.47
40 38.91 70%
50 30.51
Average Precision
60%
60 23.50
70 16.29 50%
80 10.06 40%
90 2.05
30%
100 0.28
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
29.32 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.8587
Minimum 0.0000
First Quartile 0.0771
Second Quartile 0.2888
Third Quartile 0.4313
Interquartile range 0.3542
Mean 0.2932
Standard Deviation 0.2315
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8587 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2932
Std With No Outliers 0.2315
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
4
Number of Topics of the Experiment
3.5
3
2.5
2
1.5
1
0.5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries XLDBGeoPTAut05
Topic 026 5.06 Topic 039 6.60 0.8
Topic 027 12.35 Topic 040 28.88
Topic 028 42.52 Topic 041 36.49
0.6
Topic 029 44.98 Topic 042 38.40
Topic 030 61.23 Topic 043 8.09
Topic 031 30.86 Topic 044 36.60 0.4
Topic 032 85.87 Topic 045 21.39
Topic 033 0.60 Topic 046 59.87 0.2
Topic 034 14.18 Topic 047 2.94
Difference
Topic 035 0.38 Topic 048 59.24 0
Topic 036 0.00 Topic 049 22.67
Topic 037 33.21 Topic 050 23.60
−0.2
Topic 038 57.03
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
307
xldb XLDBGeoPTAut05 GC-MONO-PT-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision
100%
5 docs 53.60
XLDBGeoPTAut05
10 docs 48.00 90%
15 docs 44.00
80%
20 docs 42.40
30 docs 36.93 70%
100 docs 21.80
60%
200 docs 11.80
R−Precision
500 docs 4.96 50%
1000 docs 2.50 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
34.57
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.8302
Minimum 0.0000
First Quartile 0.1326
Second Quartile 0.3415
Third Quartile 0.4712
Interquartile range 0.3385
Mean 0.3457
Standard Deviation 0.2446
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8302 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.3457
Std With No Outliers 0.2446
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
5
Number of Topics of the Experiment
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries XLDBGeoPTAut05
Topic 026 13.33 Topic 039 13.04 0.8
Topic 027 15.69 Topic 040 33.33
Topic 028 50.00 Topic 041 41.35
0.6
Topic 029 46.15 Topic 042 42.86
Topic 030 64.29 Topic 043 0.00
Topic 031 40.98 Topic 044 46.05 0.4
Topic 032 83.02 Topic 045 34.15
Topic 033 0.00 Topic 046 65.15 0.2
Topic 034 25.00 Topic 047 2.94
Difference
Topic 035 0.00 Topic 048 60.14 0
Topic 036 0.00 Topic 049 33.33
Topic 037 44.44 Topic 050 34.09
−0.2
Topic 038 75.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
308
xldb XLDBGeoManualPT GC-MONO-PT-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction MANUAL
Retrieved 5,232 Source Language Portuguese
Relevant 1,060 Topic Fields title, description
Relevant retrieved 607 Pooled true
Geometric Mean Average Precision 0.2034 Manual query
Binary Preference (BPREF) 0.3208
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision
100%
0 70.76
XLDBGeoManualPT
10 60.08 90%
20 51.15
80%
30 43.68
40 39.03 70%
50 34.68
Average Precision
60%
60 26.45
70 14.46 50%
80 7.91 40%
90 1.99
30%
100 0.20
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
30.12 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.7500
Minimum 0.0020
First Quartile 0.1306
Second Quartile 0.3169
Third Quartile 0.4245
Interquartile range 0.2940
Mean 0.3012
Standard Deviation 0.2099
Lower Outlier Threshold 0.0020
Upper Outlier Threshold 0.7500 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.3012
Std With No Outliers 0.2099
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
5
Number of Topics of the Experiment
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries XLDBGeoManualPT
Topic 026 19.59 Topic 039 31.69 0.8
Topic 027 64.35 Topic 040 50.25
Topic 028 41.69 Topic 041 14.16
0.6
Topic 029 7.31 Topic 042 31.75
Topic 030 44.74 Topic 043 0.20
Topic 031 33.34 Topic 044 9.73 0.4
Topic 032 67.32 Topic 045 35.38
Topic 033 8.33 Topic 046 39.94 0.2
Topic 034 6.44 Topic 047 19.16
Difference
Topic 035 15.05 Topic 048 58.04 0
Topic 036 9.52 Topic 049 33.44
Topic 037 14.25 Topic 050 22.26
−0.2
Topic 038 75.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
309
xldb XLDBGeoManualPT GC-MONO-PT-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision
100%
5 docs 48.80
XLDBGeoManualPT
10 docs 49.60 90%
15 docs 47.20
80%
20 docs 44.20
30 docs 39.87 70%
100 docs 21.84
60%
200 docs 11.88
R−Precision
500 docs 4.84 50%
1000 docs 2.43 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
35.89
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.7500
Minimum 0.0000
First Quartile 0.2036
Second Quartile 0.3409
Third Quartile 0.5076
Interquartile range 0.3040
Mean 0.3589
Standard Deviation 0.2182
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7500 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.3589
Std With No Outliers 0.2182
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
3
Number of Topics of the Experiment
2.5
2
1.5
1
0.5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries XLDBGeoManualPT
Topic 026 26.67 Topic 039 34.78 0.8
Topic 027 73.53 Topic 040 50.00
Topic 028 50.00 Topic 041 18.27
0.6
Topic 029 17.95 Topic 042 45.71
Topic 030 57.14 Topic 043 4.17
Topic 031 40.98 Topic 044 21.05 0.4
Topic 032 67.92 Topic 045 47.56
Topic 033 25.00 Topic 046 53.03 0.2
Topic 034 12.50 Topic 047 23.53
Difference
Topic 035 0.00 Topic 048 60.14 0
Topic 036 0.00 Topic 049 33.33
Topic 037 25.00 Topic 050 34.09
−0.2
Topic 038 75.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
310
xldb XLDBGeoPTAut03 GC-MONO-PT-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 22,617 Source Language Portuguese
Relevant 1,060 Topic Fields title, description
Relevant retrieved 519 Pooled true
Geometric Mean Average Precision 0.0738 With geosim, final correction.
Binary Preference (BPREF) 0.2081
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision
100%
0 71.53
XLDBGeoPTAut03
10 48.45 90%
20 36.48
80%
30 28.80
40 19.87 70%
50 16.26
Average Precision
60%
60 9.49
70 5.94 50%
80 3.42 40%
90 0.40
30%
100 0.00
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
19.29 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.7930
Minimum 0.0000
First Quartile 0.0645
Second Quartile 0.1309
Third Quartile 0.2902
Interquartile range 0.2257
Mean 0.1929
Standard Deviation 0.1888
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4798 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1679
Std With No Outliers 0.1446
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
5
Number of Topics of the Experiment
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries XLDBGeoPTAut03
Topic 026 0.40 Topic 039 8.93 0.8
Topic 027 2.11 Topic 040 23.16
Topic 028 13.09 Topic 041 42.24
0.6
Topic 029 14.92 Topic 042 18.83
Topic 030 44.12 Topic 043 6.74
Topic 031 6.99 Topic 044 5.57 0.4
Topic 032 29.95 Topic 045 24.92
Topic 033 8.05 Topic 046 47.98 0.2
Topic 034 22.02 Topic 047 12.86
Difference
Topic 035 0.00 Topic 048 79.30 0
Topic 036 0.97 Topic 049 29.42
Topic 037 28.89 Topic 050 0.29
−0.2
Topic 038 10.42
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
311
xldb XLDBGeoPTAut03 GC-MONO-PT-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision
100%
5 docs 43.20
XLDBGeoPTAut03
10 docs 37.20 90%
15 docs 34.13
80%
20 docs 31.80
30 docs 28.67 70%
100 docs 16.16
60%
200 docs 9.02
R−Precision
500 docs 3.92 50%
1000 docs 2.08 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
23.91
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.7622
Minimum 0.0000
First Quartile 0.0909
Second Quartile 0.1875
Third Quartile 0.3802
Interquartile range 0.2893
Mean 0.2391
Standard Deviation 0.1981
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7622 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2391
Std With No Outliers 0.1981
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
5
Number of Topics of the Experiment
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries XLDBGeoPTAut03
Topic 026 0.00 Topic 039 13.04 0.8
Topic 027 6.86 Topic 040 29.17
Topic 028 18.75 Topic 041 54.81
0.6
Topic 029 28.21 Topic 042 25.71
Topic 030 42.86 Topic 043 16.67
Topic 031 9.84 Topic 044 17.11 0.4
Topic 032 37.74 Topic 045 35.37
Topic 033 0.00 Topic 046 53.03 0.2
Topic 034 12.50 Topic 047 14.71
Difference
Topic 035 0.00 Topic 048 76.22 0
Topic 036 0.00 Topic 049 38.89
Topic 037 38.89 Topic 050 2.27
−0.2
Topic 038 25.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
312
xldb XLDBGeoPTAut03_2 GC-MONO-PT-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 15,622 Source Language Portuguese
Relevant 647 Topic Fields title, description
Relevant retrieved 326 Pooled true
Geometric Mean Average Precision 0.0044 XLDBGeoPTAut03 run, improved
Binary Preference (BPREF) 0.1533
Interploated Recall (%) Precision Averages (%)
GeoCLEF Monolingual Portuguese track − Interpolated Recall vs Average Precision
100%
0 45.44
XLDBGeoPTAut03_2
10 36.56 90%
20 29.06
80%
30 22.89
40 16.29 70%
50 13.01
Average Precision
60%
60 6.66
70 5.45 50%
80 5.03 40%
90 1.17
30%
100 0.65
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
15.13 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.7843
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0673
Third Quartile 0.2356
Interquartile range 0.2356
Mean 0.1513
Standard Deviation 0.2051
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5215 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1249
Std With No Outliers 0.1605
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries XLDBGeoPTAut03_2
Topic 026 0.00 Topic 039 8.96 0.8
Topic 027 0.64 Topic 040 23.16
Topic 028 0.00 Topic 041 42.23
0.6
Topic 029 52.15 Topic 042 18.81
Topic 030 44.58 Topic 043 6.73
Topic 031 0.00 Topic 044 5.81 0.4
Topic 032 78.43 Topic 045 24.76
Topic 033 9.88 Topic 046 0.00 0.2
Topic 034 21.76 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 0.00 0
Topic 036 0.97 Topic 049 0.00
Topic 037 28.92 Topic 050 0.00
−0.2
Topic 038 10.42
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
313
xldb XLDBGeoPTAut03_2 GC-MONO-PT-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Monolingual Portuguese track − Retrieved documents vs Precision
100%
5 docs 30.40
XLDBGeoPTAut03_2
10 docs 28.40 90%
15 docs 24.27
80%
20 docs 21.60
30 docs 20.00 70%
100 docs 10.36
60%
200 docs 5.58
R−Precision
500 docs 2.50 50%
1000 docs 1.30 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
17.29
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Monolingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.7547
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1250
Third Quartile 0.3041
Interquartile range 0.3041
Mean 0.1729
Standard Deviation 0.2122
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7547 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1729
Std With No Outliers 0.2122
GeoCLEF Monolingual Portuguese track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Monolingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries XLDBGeoPTAut03_2
Topic 026 0.00 Topic 039 13.04 0.8
Topic 027 4.90 Topic 040 29.17
Topic 028 0.00 Topic 041 54.81
0.6
Topic 029 46.15 Topic 042 25.71
Topic 030 42.86 Topic 043 12.50
Topic 031 0.00 Topic 044 17.11 0.4
Topic 032 75.47 Topic 045 34.15
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 12.50 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 0.00 0
Topic 036 0.00 Topic 049 0.00
Topic 037 38.89 Topic 050 0.00
−0.2
Topic 038 25.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
314
berkeley BKGeoED2 GC-BILI-X2DE-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction MANUAL
Retrieved 25,000 Source Language English
Relevant 602 Topic Fields title, description, narrative
Relevant retrieved 450 Pooled true
Geometric Mean Average Precision 0.0115 English-German TDN from expanded narrative,
Binary Preference (BPREF) 0.1649 translation by L&H Power Translator, blind feedback
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision
100%
0 29.75
BKGeoED2
10 28.19 90%
20 26.59
80%
30 24.48
40 20.23 70%
50 17.35
Average Precision
60%
60 13.28
70 12.31 50%
80 10.86 40%
90 9.20
30%
100 2.07
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
16.82 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.9178
Minimum 0.0000
First Quartile 0.0038
Second Quartile 0.0235
Third Quartile 0.2800
Interquartile range 0.2762
Mean 0.1682
Standard Deviation 0.2563
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5092 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1082
Std With No Outliers 0.1557
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries BKGeoED2
Topic 026 0.00 Topic 039 17.14 0.8
Topic 027 0.08 Topic 040 36.77
Topic 028 34.93 Topic 041 2.35
0.6
Topic 029 35.10 Topic 042 24.04
Topic 030 7.28 Topic 043 25.69
Topic 031 4.22 Topic 044 0.48 0.4
Topic 032 79.95 Topic 045 1.60
Topic 033 0.01 Topic 046 1.36 0.2
Topic 034 50.92 Topic 047 0.00
Difference
Topic 035 0.99 Topic 048 91.78 0
Topic 036 0.00 Topic 049 2.86
Topic 037 0.00 Topic 050 2.08
−0.2
Topic 038 0.96
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
315
berkeley BKGeoED2 GC-BILI-X2DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual German track − Retrieved documents vs Precision
100%
5 docs 17.60
BKGeoED2
10 docs 18.00 90%
15 docs 18.93
80%
20 docs 18.20
30 docs 17.07 70%
100 docs 10.64
60%
200 docs 6.44
R−Precision
500 docs 3.05 50%
1000 docs 1.80 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
18.56
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.8841
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.3807
Interquartile range 0.3807
Mean 0.1856
Standard Deviation 0.2781
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8841 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1856
Std With No Outliers 0.2781
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries BKGeoED2
Topic 026 0.00 Topic 039 22.22 0.8
Topic 027 1.54 Topic 040 43.90
Topic 028 43.75 Topic 041 0.00
0.6
Topic 029 33.33 Topic 042 36.17
Topic 030 6.67 Topic 043 50.00
Topic 031 0.00 Topic 044 0.00 0.4
Topic 032 85.19 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 52.94 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 88.41 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
316
berkeley BKGeoED1 GC-BILI-X2DE-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 602 Topic Fields title, description
Relevant retrieved 391 Pooled true
Geometric Mean Average Precision 0.0107 English-German TD run translation by L&H
Binary Preference (BPREF) 0.1397 translator, blind feedback
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision
100%
0 28.82
BKGeoED1
10 22.23 90%
20 20.79
80%
30 20.15
40 18.47 70%
50 17.53
Average Precision
60%
60 15.67
70 13.45 50%
80 11.19 40%
90 8.68
30%
100 2.11
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
15.61 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.8867
Minimum 0.0000
First Quartile 0.0030
Second Quartile 0.0222
Third Quartile 0.1511
Interquartile range 0.1482
Mean 0.1561
Standard Deviation 0.2704
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1984 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0334
Std With No Outliers 0.0503
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries BKGeoED1
Topic 026 0.00 Topic 039 0.04 0.8
Topic 027 2.10 Topic 040 38.45
Topic 028 44.85 Topic 041 3.77
0.6
Topic 029 2.22 Topic 042 0.04
Topic 030 19.84 Topic 043 1.05
Topic 031 2.13 Topic 044 1.34 0.4
Topic 032 83.26 Topic 045 4.17
Topic 033 0.00 Topic 046 6.34 0.2
Topic 034 68.19 Topic 047 0.00
Difference
Topic 035 2.05 Topic 048 88.67 0
Topic 036 0.00 Topic 049 5.21
Topic 037 0.38 Topic 050 2.66
−0.2
Topic 038 13.54
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
317
berkeley BKGeoED1 GC-BILI-X2DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual German track − Retrieved documents vs Precision
100%
5 docs 19.20
BKGeoED1
10 docs 17.20 90%
15 docs 17.33
80%
20 docs 17.80
30 docs 16.67 70%
100 docs 10.00
60%
200 docs 5.74
R−Precision
500 docs 2.66 50%
1000 docs 1.56 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
15.44
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.8696
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.2542
Interquartile range 0.2542
Mean 0.1544
Standard Deviation 0.2700
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4390 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0698
Std With No Outliers 0.1414
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries BKGeoED1
Topic 026 0.00 Topic 039 0.00 0.8
Topic 027 4.62 Topic 040 43.90
Topic 028 43.75 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 0.00
Topic 030 26.67 Topic 043 0.00
Topic 031 1.32 Topic 044 0.00 0.4
Topic 032 77.78 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 67.65 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 86.96 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 8.33
−0.2
Topic 038 25.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
318
hagen FUHedGNNNTDN GC-BILI-X2DE-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 602 Topic Fields title, description, narrative
Relevant retrieved 333 Pooled true
Geometric Mean Average Precision 0.0127 third run
Binary Preference (BPREF) 0.0548
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision
100%
0 18.90
FUHedGNNNTDN
10 14.86 90%
20 10.13
80%
30 7.23
40 5.54 70%
50 4.30
Average Precision
60%
60 2.86
70 2.44 50%
80 1.29 40%
90 0.64
30%
100 0.06
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
5.48 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.5405
Minimum 0.0003
First Quartile 0.0052
Second Quartile 0.0096
Third Quartile 0.0412
Interquartile range 0.0359
Mean 0.0548
Standard Deviation 0.1217
Lower Outlier Threshold 0.0003
Upper Outlier Threshold 0.0886 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0213
Std With No Outliers 0.0238
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries FUHedGNNNTDN
Topic 026 0.54 Topic 039 3.57 0.8
Topic 027 2.71 Topic 040 0.72
Topic 028 0.30 Topic 041 2.46
0.6
Topic 029 0.61 Topic 042 0.96
Topic 030 4.05 Topic 043 0.47
Topic 031 5.66 Topic 044 1.46 0.4
Topic 032 54.05 Topic 045 0.59
Topic 033 0.07 Topic 046 5.99 0.2
Topic 034 8.86 Topic 047 0.06
Difference
Topic 035 4.31 Topic 048 33.99 0
Topic 036 3.84 Topic 049 0.03
Topic 037 0.61 Topic 050 0.73
−0.2
Topic 038 0.27
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
319
hagen FUHedGNNNTDN GC-BILI-X2DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual German track − Retrieved documents vs Precision
100%
5 docs 11.20
FUHedGNNNTDN
10 docs 11.60 90%
15 docs 10.93
80%
20 docs 10.20
30 docs 8.13 70%
100 docs 4.80
60%
200 docs 3.22
R−Precision
500 docs 2.01 50%
1000 docs 1.33 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
6.24
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.5370
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.1013
Interquartile range 0.1013
Mean 0.0624
Standard Deviation 0.1277
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1231 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0287
Std With No Outliers 0.0478
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries FUHedGNNNTDN
Topic 026 0.00 Topic 039 11.11 0.8
Topic 027 12.31 Topic 040 2.44
Topic 028 0.00 Topic 041 10.53
0.6
Topic 029 0.00 Topic 042 0.00
Topic 030 10.00 Topic 043 0.00
Topic 031 7.89 Topic 044 0.00 0.4
Topic 032 53.70 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 11.76 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 36.23 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
320
hagen FUHedGYYYTDN GC-BILI-X2DE-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 602 Topic Fields title, description, narrative
Relevant retrieved 375 Pooled true
Geometric Mean Average Precision 0.0106 second run
Binary Preference (BPREF) 0.1175
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision
100%
0 29.26
FUHedGYYYTDN
10 19.62 90%
20 17.13
80%
30 14.74
40 13.81 70%
50 12.22
Average Precision
60%
60 11.45
70 10.15 50%
80 8.03 40%
90 5.97
30%
100 0.24
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
12.34 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.8704
Minimum 0.0000
First Quartile 0.0023
Second Quartile 0.0123
Third Quartile 0.0651
Interquartile range 0.0628
Mean 0.1234
Standard Deviation 0.2421
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.0725 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0152
Std With No Outliers 0.0200
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries FUHedGYYYTDN
Topic 026 0.05 Topic 039 26.52 0.8
Topic 027 1.10 Topic 040 39.12
Topic 028 1.90 Topic 041 6.26
0.6
Topic 029 1.23 Topic 042 0.25
Topic 030 1.83 Topic 043 0.16
Topic 031 68.71 Topic 044 0.98 0.4
Topic 032 56.78 Topic 045 0.83
Topic 033 2.72 Topic 046 0.13 0.2
Topic 034 7.25 Topic 047 0.00
Difference
Topic 035 2.46 Topic 048 87.04 0
Topic 036 2.24 Topic 049 0.00
Topic 037 0.34 Topic 050 0.11
−0.2
Topic 038 0.54
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
321
hagen FUHedGYYYTDN GC-BILI-X2DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual German track − Retrieved documents vs Precision
100%
5 docs 19.20
FUHedGYYYTDN
10 docs 16.00 90%
15 docs 16.27
80%
20 docs 16.00
30 docs 14.40 70%
100 docs 10.08
60%
200 docs 5.86
R−Precision
500 docs 2.63 50%
1000 docs 1.50 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
12.45
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.7826
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.0794
Interquartile range 0.0794
Mean 0.1245
Standard Deviation 0.2320
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1176 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0196
Std With No Outliers 0.0340
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries FUHedGYYYTDN
Topic 026 0.00 Topic 039 25.93 0.8
Topic 027 3.08 Topic 040 43.90
Topic 028 6.25 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 0.00
Topic 030 6.67 Topic 043 0.00
Topic 031 68.42 Topic 044 0.00 0.4
Topic 032 55.56 Topic 045 0.00
Topic 033 11.76 Topic 046 0.00 0.2
Topic 034 5.88 Topic 047 0.00
Difference
Topic 035 5.56 Topic 048 78.26 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
322
hagen FUHedGNNNTD GC-BILI-X2DE-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 24,318 Source Language English
Relevant 602 Topic Fields title, description
Relevant retrieved 397 Pooled true
Geometric Mean Average Precision 0.0231 fourth run
Binary Preference (BPREF) 0.1219
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision
100%
0 35.08
FUHedGNNNTD
10 25.77 90%
20 22.00
80%
30 17.93
40 12.62 70%
50 11.01
Average Precision
60%
60 7.42
70 5.44 50%
80 4.33 40%
90 2.40
30%
100 0.05
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
12.11 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.7259
Minimum 0.0000
First Quartile 0.0053
Second Quartile 0.0645
Third Quartile 0.1710
Interquartile range 0.1657
Mean 0.1211
Standard Deviation 0.1773
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.2854 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0755
Std With No Outliers 0.0809
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries FUHedGNNNTD
Topic 026 10.00 Topic 039 7.23 0.8
Topic 027 0.55 Topic 040 28.54
Topic 028 20.94 Topic 041 1.10
0.6
Topic 029 6.45 Topic 042 8.57
Topic 030 5.94 Topic 043 0.36
Topic 031 18.86 Topic 044 11.11 0.4
Topic 032 56.52 Topic 045 1.26
Topic 033 0.23 Topic 046 16.80 0.2
Topic 034 17.98 Topic 047 0.00
Difference
Topic 035 5.75 Topic 048 72.59 0
Topic 036 0.47 Topic 049 0.00
Topic 037 7.95 Topic 050 3.55
−0.2
Topic 038 0.07
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
323
hagen FUHedGNNNTD GC-BILI-X2DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual German track − Retrieved documents vs Precision
100%
5 docs 20.80
FUHedGNNNTD
10 docs 21.20 90%
15 docs 18.67
80%
20 docs 17.20
30 docs 15.20 70%
100 docs 8.52
60%
200 docs 5.58
R−Precision
500 docs 2.90 50%
1000 docs 1.59 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
15.34
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.6087
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1111
Third Quartile 0.2537
Interquartile range 0.2537
Mean 0.1534
Standard Deviation 0.1738
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6087 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1534
Std With No Outliers 0.1738
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries FUHedGNNNTD
Topic 026 0.00 Topic 039 11.11 0.8
Topic 027 6.15 Topic 040 36.59
Topic 028 21.88 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 23.40
Topic 030 13.33 Topic 043 0.00
Topic 031 30.26 Topic 044 33.33 0.4
Topic 032 51.85 Topic 045 0.00
Topic 033 0.00 Topic 046 25.00 0.2
Topic 034 26.47 Topic 047 0.00
Difference
Topic 035 16.67 Topic 048 60.87 0
Topic 036 0.00 Topic 049 0.00
Topic 037 18.18 Topic 050 8.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
324
hagen FUHedGYYYTD GC-BILI-X2DE-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 602 Topic Fields title, description
Relevant retrieved 383 Pooled true
Geometric Mean Average Precision 0.0140 first run
Binary Preference (BPREF) 0.1171
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision
100%
0 31.47
FUHedGYYYTD
10 21.14 90%
20 18.08
80%
30 15.41
40 14.08 70%
50 11.79
Average Precision
60%
60 10.77
70 9.75 50%
80 8.20 40%
90 6.80
30%
100 0.49
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
12.80 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.8412
Minimum 0.0000
First Quartile 0.0033
Second Quartile 0.0223
Third Quartile 0.0676
Interquartile range 0.0643
Mean 0.1280
Standard Deviation 0.2460
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1560 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0288
Std With No Outliers 0.0381
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries FUHedGYYYTD
Topic 026 0.06 Topic 039 6.22 0.8
Topic 027 1.10 Topic 040 39.12
Topic 028 1.10 Topic 041 6.26
0.6
Topic 029 1.17 Topic 042 0.32
Topic 030 15.60 Topic 043 0.16
Topic 031 79.84 Topic 044 2.55 0.4
Topic 032 56.28 Topic 045 2.82
Topic 033 2.23 Topic 046 0.13 0.2
Topic 034 7.25 Topic 047 0.00
Difference
Topic 035 1.35 Topic 048 84.12 0
Topic 036 6.60 Topic 049 0.00
Topic 037 0.34 Topic 050 4.70
−0.2
Topic 038 0.54
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
325
hagen FUHedGYYYTD GC-BILI-X2DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual German track − Retrieved documents vs Precision
100%
5 docs 20.00
FUHedGYYYTD
10 docs 17.60 90%
15 docs 16.53
80%
20 docs 16.60
30 docs 15.33 70%
100 docs 9.96
60%
200 docs 5.96
R−Precision
500 docs 2.75 50%
1000 docs 1.53 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
11.94
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.7681
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.0764
Interquartile range 0.0764
Mean 0.1194
Standard Deviation 0.2339
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.0833 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0150
Std With No Outliers 0.0269
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries FUHedGYYYTD
Topic 026 0.00 Topic 039 7.41 0.8
Topic 027 3.08 Topic 040 43.90
Topic 028 3.12 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 2.13
Topic 030 23.33 Topic 043 0.00
Topic 031 76.32 Topic 044 0.00 0.4
Topic 032 48.15 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 5.88 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 76.81 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 8.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
326
hagen FUHedGYYYMTDN GC-BILI-X2DE-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 602 Topic Fields title, description, narrative
Relevant retrieved 375 Pooled true
Geometric Mean Average Precision 0.0118 fifth run
Binary Preference (BPREF) 0.1104
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision
100%
0 28.51
FUHedGYYYMTDN
10 18.25 90%
20 16.52
80%
30 15.31
40 13.96 70%
50 11.77
Average Precision
60%
60 10.08
70 8.55 50%
80 6.70 40%
90 4.59
30%
100 0.11
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
11.48 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.8178
Minimum 0.0000
First Quartile 0.0040
Second Quartile 0.0155
Third Quartile 0.0438
Interquartile range 0.0398
Mean 0.1148
Standard Deviation 0.2280
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.0683 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0145
Std With No Outliers 0.0162
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries FUHedGYYYMTDN
Topic 026 0.08 Topic 039 18.33 0.8
Topic 027 1.55 Topic 040 37.97
Topic 028 2.25 Topic 041 3.56
0.6
Topic 029 1.22 Topic 042 0.47
Topic 030 2.16 Topic 043 0.22
Topic 031 62.69 Topic 044 1.29 0.4
Topic 032 57.32 Topic 045 0.48
Topic 033 1.51 Topic 046 0.21 0.2
Topic 034 6.83 Topic 047 0.00
Difference
Topic 035 1.93 Topic 048 81.78 0
Topic 036 2.73 Topic 049 0.00
Topic 037 1.77 Topic 050 0.16
−0.2
Topic 038 0.54
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
327
hagen FUHedGYYYMTDN GC-BILI-X2DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual German track − Retrieved documents vs Precision
100%
5 docs 15.20
FUHedGYYYMTDN
10 docs 16.00 90%
15 docs 15.20
80%
20 docs 15.20
30 docs 14.53 70%
100 docs 9.44
60%
200 docs 5.92
R−Precision
500 docs 2.70 50%
1000 docs 1.50 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
11.57
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.7681
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.0661
Interquartile range 0.0661
Mean 0.1157
Standard Deviation 0.2245
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.0769 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0154
Std With No Outliers 0.0263
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries FUHedGYYYMTDN
Topic 026 0.00 Topic 039 18.52 0.8
Topic 027 7.69 Topic 040 46.34
Topic 028 6.25 Topic 041 0.00
0.6
Topic 029 0.00 Topic 042 2.13
Topic 030 3.33 Topic 043 0.00
Topic 031 63.16 Topic 044 0.00 0.4
Topic 032 53.70 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 5.88 Topic 047 0.00
Difference
Topic 035 5.56 Topic 048 76.81 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
328
hildesheim HIGeoenderun21 GC-BILI-X2DE-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 602 Topic Fields title, description
Relevant retrieved 349 Pooled true
Geometric Mean Average Precision 0.0095 Experiment with BRF(5docs,25terms) base run
Binary Preference (BPREF) 0.1218
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision
100%
0 30.08
HIGeoenderun21
10 24.56 90%
20 17.92
80%
30 16.49
40 15.59 70%
50 14.10
Average Precision
60%
60 9.86
70 6.35 50%
80 4.04 40%
90 2.67
30%
100 0.06
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
11.86 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.5317
Minimum 0.0000
First Quartile 0.0020
Second Quartile 0.0210
Third Quartile 0.2117
Interquartile range 0.2097
Mean 0.1186
Standard Deviation 0.1784
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5196 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1014
Std With No Outliers 0.1596
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries HIGeoenderun21
Topic 026 0.22 Topic 039 1.13 0.8
Topic 027 1.58 Topic 040 29.01
Topic 028 29.57 Topic 041 5.26
0.6
Topic 029 28.27 Topic 042 0.14
Topic 030 9.84 Topic 043 2.10
Topic 031 5.31 Topic 044 0.43 0.4
Topic 032 51.96 Topic 045 0.00
Topic 033 0.00 Topic 046 0.51 0.2
Topic 034 50.65 Topic 047 0.00
Difference
Topic 035 4.13 Topic 048 53.17 0
Topic 036 0.08 Topic 049 18.80
Topic 037 0.59 Topic 050 3.83
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
329
hildesheim HIGeoenderun21 GC-BILI-X2DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual German track − Retrieved documents vs Precision
100%
5 docs 17.60
HIGeoenderun21
10 docs 14.40 90%
15 docs 14.13
80%
20 docs 14.40
30 docs 14.00 70%
100 docs 8.92
60%
200 docs 5.38
R−Precision
500 docs 2.56 50%
1000 docs 1.40 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
15.18
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.6087
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0714
Third Quartile 0.2083
Interquartile range 0.2083
Mean 0.1518
Standard Deviation 0.2043
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4375 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0942
Std With No Outliers 0.1364
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries HIGeoenderun21
Topic 026 0.00 Topic 039 3.70 0.8
Topic 027 12.31 Topic 040 41.46
Topic 028 43.75 Topic 041 10.53
0.6
Topic 029 33.33 Topic 042 0.00
Topic 030 16.67 Topic 043 7.14
Topic 031 10.53 Topic 044 0.00 0.4
Topic 032 55.56 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 55.88 Topic 047 0.00
Difference
Topic 035 11.11 Topic 048 60.87 0
Topic 036 0.00 Topic 049 16.67
Topic 037 0.00 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
330
hildesheim HIGeoenderun22 GC-BILI-X2DE-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 23,198 Source Language English
Relevant 568 Topic Fields title, description
Relevant retrieved 307 Pooled true
Geometric Mean Average Precision 0.0052 Experiment with BRF(5docs,25terms) with NE-
Binary Preference (BPREF) 0.1043 recognition and weighting, also within the BRF
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision
100%
0 25.62
HIGeoenderun22
10 21.29 90%
20 17.48
80%
30 16.18
40 11.53 70%
50 9.37
Average Precision
60%
60 6.84
70 3.44 50%
80 2.35 40%
90 1.44
30%
100 0.05
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
9.69 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.5888
Minimum 0.0000
First Quartile 0.0006
Second Quartile 0.0114
Third Quartile 0.1055
Interquartile range 0.1049
Mean 0.0969
Standard Deviation 0.1651
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.2025 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0326
Std With No Outliers 0.0526
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries HIGeoenderun22
Topic 026 0.06 Topic 039 2.96 0.8
Topic 027 1.14 Topic 040 20.25
Topic 028 28.00 Topic 041 11.02
0.6
Topic 029 50.00 Topic 042 2.69
Topic 030 10.39 Topic 043 3.97
Topic 031 0.57 Topic 044 0.49 0.4
Topic 032 36.85 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 0.00 Topic 047 0.59
Difference
Topic 035 0.16 Topic 048 58.88 0
Topic 036 0.06 Topic 049 8.50
Topic 037 0.08 Topic 050 5.48
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
331
hildesheim HIGeoenderun22 GC-BILI-X2DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual German track − Retrieved documents vs Precision
100%
5 docs 15.20
HIGeoenderun22
10 docs 14.80 90%
15 docs 13.60
80%
20 docs 12.00
30 docs 10.80 70%
100 docs 6.92
60%
200 docs 4.14
R−Precision
500 docs 2.07 50%
1000 docs 1.23 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
11.72
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.5942
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0370
Third Quartile 0.1776
Interquartile range 0.1776
Mean 0.1172
Standard Deviation 0.1669
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3750 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0822
Std With No Outliers 0.1179
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries HIGeoenderun22
Topic 026 0.00 Topic 039 3.70 0.8
Topic 027 10.77 Topic 040 29.27
Topic 028 37.50 Topic 041 21.05
0.6
Topic 029 33.33 Topic 042 4.26
Topic 030 10.00 Topic 043 14.29
Topic 031 0.00 Topic 044 0.00 0.4
Topic 032 44.44 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 59.42 0
Topic 036 0.00 Topic 049 16.67
Topic 037 0.00 Topic 050 8.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
332
hildesheim HIGeoenderun21n GC-BILI-X2DE-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 602 Topic Fields title, description, narrative
Relevant retrieved 369 Pooled true
Geometric Mean Average Precision 0.0125 Experiment with BRF(5docs,25terms) base
Binary Preference (BPREF) 0.1346
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision
100%
0 32.78
HIGeoenderun21n
10 26.86 90%
20 20.68
80%
30 19.28
40 16.17 70%
50 14.60
Average Precision
60%
60 10.66
70 7.08 50%
80 3.74 40%
90 2.17
30%
100 0.05
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
13.15 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.5798
Minimum 0.0000
First Quartile 0.0024
Second Quartile 0.0198
Third Quartile 0.2114
Interquartile range 0.2090
Mean 0.1315
Standard Deviation 0.1941
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5040 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0946
Std With No Outliers 0.1523
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries HIGeoenderun21n
Topic 026 0.22 Topic 039 1.49 0.8
Topic 027 1.69 Topic 040 25.86
Topic 028 32.67 Topic 041 9.42
0.6
Topic 029 50.40 Topic 042 0.71
Topic 030 8.43 Topic 043 1.81
Topic 031 5.08 Topic 044 0.25 0.4
Topic 032 47.45 Topic 045 0.00
Topic 033 0.00 Topic 046 1.98 0.2
Topic 034 53.30 Topic 047 0.03
Difference
Topic 035 5.43 Topic 048 57.98 0
Topic 036 0.08 Topic 049 19.57
Topic 037 0.32 Topic 050 4.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
333
hildesheim HIGeoenderun21n GC-BILI-X2DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual German track − Retrieved documents vs Precision
100%
5 docs 15.20
HIGeoenderun21n
10 docs 15.60 90%
15 docs 15.20
80%
20 docs 15.00
30 docs 14.13 70%
100 docs 8.92
60%
200 docs 5.46
R−Precision
500 docs 2.62 50%
1000 docs 1.48 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
15.21
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.5882
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0714
Third Quartile 0.2083
Interquartile range 0.2083
Mean 0.1521
Standard Deviation 0.2011
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4146 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0945
Std With No Outliers 0.1313
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries HIGeoenderun21n
Topic 026 0.00 Topic 039 7.41 0.8
Topic 027 13.85 Topic 040 41.46
Topic 028 40.62 Topic 041 5.26
0.6
Topic 029 33.33 Topic 042 6.38
Topic 030 10.00 Topic 043 7.14
Topic 031 9.21 Topic 044 0.00 0.4
Topic 032 55.56 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 58.82 Topic 047 0.00
Difference
Topic 035 16.67 Topic 048 57.97 0
Topic 036 0.00 Topic 049 16.67
Topic 037 0.00 Topic 050 0.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
334
hildesheim HIGeoenderun22n GC-BILI-X2DE-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 602 Topic Fields title, description, narrative
Relevant retrieved 303 Pooled true
Geometric Mean Average Precision 0.0048 Experiment with BRF(5docs,25terms) with NE-
Binary Preference (BPREF) 0.0977 recognition and weighting, also within the BRF
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual German track − Interpolated Recall vs Average Precision
100%
0 25.64
HIGeoenderun22n
10 20.12 90%
20 17.28
80%
30 16.36
40 12.18 70%
50 10.84
Average Precision
60%
60 7.61
70 5.36 50%
80 4.12 40%
90 2.64
30%
100 0.35
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
10.46 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.8120
Minimum 0.0000
First Quartile 0.0006
Second Quartile 0.0131
Third Quartile 0.0898
Interquartile range 0.0892
Mean 0.1046
Standard Deviation 0.1935
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.1877 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0323
Std With No Outliers 0.0521
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries HIGeoenderun22n
Topic 026 0.00 Topic 039 1.31 0.8
Topic 027 0.16 Topic 040 18.77
Topic 028 34.50 Topic 041 6.32
0.6
Topic 029 46.97 Topic 042 3.28
Topic 030 4.61 Topic 043 0.63
Topic 031 0.51 Topic 044 0.17 0.4
Topic 032 31.07 Topic 045 0.00
Topic 033 0.00 Topic 046 3.21 0.2
Topic 034 6.67 Topic 047 0.00
Difference
Topic 035 0.08 Topic 048 81.20 0
Topic 036 0.95 Topic 049 15.90
Topic 037 0.00 Topic 050 5.17
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
335
hildesheim HIGeoenderun22n GC-BILI-X2DE-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual German track − Retrieved documents vs Precision
100%
5 docs 16.00
HIGeoenderun22n
10 docs 14.00 90%
15 docs 12.53
80%
20 docs 12.20
30 docs 12.00 70%
100 docs 6.88
60%
200 docs 4.24
R−Precision
500 docs 2.12 50%
1000 docs 1.21 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
11.77
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual German track − Box plot of the Topics of the Experiment
Maximum 0.7681
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0333
Third Quartile 0.1299
Interquartile range 0.1299
Mean 0.1177
Standard Deviation 0.1891
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.2927 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0480
Std With No Outliers 0.0737
GeoCLEF Bilingual German track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual German track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries HIGeoenderun22n
Topic 026 0.00 Topic 039 7.41 0.8
Topic 027 3.08 Topic 040 29.27
Topic 028 40.62 Topic 041 5.26
0.6
Topic 029 33.33 Topic 042 8.51
Topic 030 3.33 Topic 043 7.14
Topic 031 0.00 Topic 044 0.00 0.4
Topic 032 42.59 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 11.76 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 76.81 0
Topic 036 0.00 Topic 049 16.67
Topic 037 0.00 Topic 050 8.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
336
hildesheim HIGeodeenrun12 GC-BILI-X2EN-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language German
Relevant 378 Topic Fields title, description
Relevant retrieved 222 Pooled true
Geometric Mean Average Precision 0.0103 Experiment with BRF(5docs,20terms) with
Binary Preference (BPREF) 0.1489 GeoNEweighting within the BRF-algorithm
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision
100%
0 31.10
HIGeodeenrun12
10 28.46 90%
20 24.65
80%
30 21.50
40 19.76 70%
50 18.40
Average Precision
60%
60 16.78
70 9.17 50%
80 6.49 40%
90 5.12
30%
100 3.95
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
16.03 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 0.8837
Minimum 0.0000
First Quartile 0.0009
Second Quartile 0.0291
Third Quartile 0.2416
Interquartile range 0.2407
Mean 0.1603
Standard Deviation 0.2448
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5095 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1027
Std With No Outliers 0.1473
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries HIGeodeenrun12
Topic 026 1.18 Topic 039 0.63 0.8
Topic 027 0.04 Topic 040 88.37
Topic 028 0.10 Topic 041 0.00
0.6
Topic 029 13.37 Topic 042 0.09
Topic 030 27.62 Topic 043 0.03
Topic 031 23.00 Topic 044 3.18 0.4
Topic 032 50.95 Topic 045 0.35
Topic 033 0.00 Topic 046 39.81 0.2
Topic 034 28.47 Topic 047 2.91
Difference
Topic 035 0.07 Topic 048 76.11 0
Topic 036 0.00 Topic 049 10.43
Topic 037 0.13 Topic 050 21.30
−0.2
Topic 038 12.50
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
337
hildesheim HIGeodeenrun12 GC-BILI-X2EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual English track − Retrieved documents vs Precision
100%
5 docs 20.80
HIGeodeenrun12
10 docs 18.40 90%
15 docs 16.80
80%
20 docs 15.40
30 docs 13.60 70%
100 docs 5.20
60%
200 docs 3.10
R−Precision
500 docs 1.64 50%
1000 docs 0.89 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
17.52
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 0.7857
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.2867
Interquartile range 0.2867
Mean 0.1752
Standard Deviation 0.2615
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7083 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1497
Std With No Outliers 0.2334
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries HIGeodeenrun12
Topic 026 0.00 Topic 039 0.00 0.8
Topic 027 0.00 Topic 040 78.57
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 22.22 Topic 042 0.00
Topic 030 33.33 Topic 043 0.00
Topic 031 27.12 Topic 044 10.53 0.4
Topic 032 64.52 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 4.17
Difference
Topic 035 0.00 Topic 048 70.83 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 26.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
338
hildesheim HIGeodeenrun13n GC-BILI-X2EN-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 23,326 Source Language German
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 214 Pooled true
Geometric Mean Average Precision 0.0054 Experiment with BRF(5docs,25terms) with NE-
Binary Preference (BPREF) 0.1322 recognition and weighting, also within the BRF
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision
100%
0 29.63
HIGeodeenrun13n
10 23.84 90%
20 23.36
80%
30 22.09
40 21.26 70%
50 19.87
Average Precision
60%
60 17.58
70 9.80 50%
80 4.97 40%
90 3.66
30%
100 2.33
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
15.65 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 0.8353
Minimum 0.0000
First Quartile 0.0001
Second Quartile 0.0271
Third Quartile 0.1738
Interquartile range 0.1738
Mean 0.1565
Standard Deviation 0.2497
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.2324 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0422
Std With No Outliers 0.0658
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries HIGeodeenrun13n
Topic 026 1.96 Topic 039 0.79 0.8
Topic 027 0.01 Topic 040 74.25
Topic 028 0.05 Topic 041 0.08
0.6
Topic 029 9.05 Topic 042 0.00
Topic 030 46.46 Topic 043 0.00
Topic 031 23.24 Topic 044 3.09 0.4
Topic 032 52.48 Topic 045 2.71
Topic 033 0.01 Topic 046 8.59 0.2
Topic 034 0.00 Topic 047 5.46
Difference
Topic 035 0.00 Topic 048 83.53 0
Topic 036 0.00 Topic 049 13.91
Topic 037 0.05 Topic 050 15.43
−0.2
Topic 038 50.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
339
hildesheim HIGeodeenrun13n GC-BILI-X2EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual English track − Retrieved documents vs Precision
100%
5 docs 20.80
HIGeodeenrun13n
10 docs 17.60 90%
15 docs 16.53
80%
20 docs 14.40
30 docs 12.53 70%
100 docs 5.44
60%
200 docs 3.28
R−Precision
500 docs 1.58 50%
1000 docs 0.86 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
14.83
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 0.7500
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.2333
Interquartile range 0.2333
Mean 0.1483
Standard Deviation 0.2572
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3390 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0459
Std With No Outliers 0.1002
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries HIGeodeenrun13n
Topic 026 0.00 Topic 039 0.00 0.8
Topic 027 0.00 Topic 040 71.43
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 22.22 Topic 042 0.00
Topic 030 66.67 Topic 043 0.00
Topic 031 33.90 Topic 044 5.26 0.4
Topic 032 61.29 Topic 045 0.00
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 0.00 Topic 047 8.33
Difference
Topic 035 0.00 Topic 048 75.00 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 26.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
340
hildesheim HIGeodeenrun11n GC-BILI-X2EN-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language German
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 217 Pooled false
Geometric Mean Average Precision 0.0096 no BRF base run, stem snowball
Binary Preference (BPREF) 0.1824
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision
100%
0 32.96
HIGeodeenrun11n
10 30.42 90%
20 29.01
80%
30 26.47
40 22.46 70%
50 20.45
Average Precision
60%
60 17.94
70 10.35 50%
80 8.30 40%
90 7.74
30%
100 6.65
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
19.03 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0007
Second Quartile 0.0147
Third Quartile 0.2478
Interquartile range 0.2471
Mean 0.1903
Standard Deviation 0.3065
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5153 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0748
Std With No Outliers 0.1475
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries HIGeodeenrun11n
Topic 026 0.92 Topic 039 0.75 0.8
Topic 027 0.01 Topic 040 81.81
Topic 028 0.05 Topic 041 0.00
0.6
Topic 029 8.67 Topic 042 0.37
Topic 030 64.80 Topic 043 0.03
Topic 031 16.15 Topic 044 1.60 0.4
Topic 032 51.53 Topic 045 0.64
Topic 033 0.00 Topic 046 46.26 0.2
Topic 034 8.82 Topic 047 2.02
Difference
Topic 035 0.08 Topic 048 71.89 0
Topic 036 0.00 Topic 049 1.47
Topic 037 0.16 Topic 050 17.62
−0.2
Topic 038 100.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
341
hildesheim HIGeodeenrun11n GC-BILI-X2EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual English track − Retrieved documents vs Precision
100%
5 docs 18.40
HIGeodeenrun11n
10 docs 17.60 90%
15 docs 15.20
80%
20 docs 15.20
30 docs 12.13 70%
100 docs 4.72
60%
200 docs 2.90
R−Precision
500 docs 1.58 50%
1000 docs 0.87 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
19.01
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.2833
Interquartile range 0.2833
Mean 0.1901
Standard Deviation 0.2948
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1321
Std With No Outliers 0.2214
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries HIGeodeenrun11n
Topic 026 0.00 Topic 039 0.00 0.8
Topic 027 0.00 Topic 040 71.43
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 22.22 Topic 042 0.00
Topic 030 66.67 Topic 043 0.00
Topic 031 20.34 Topic 044 7.89 0.4
Topic 032 58.06 Topic 045 0.00
Topic 033 0.00 Topic 046 33.33 0.2
Topic 034 0.00 Topic 047 4.17
Difference
Topic 035 0.00 Topic 048 64.58 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 26.67
−0.2
Topic 038 100.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
342
hildesheim HIGeodeenrun11 GC-BILI-X2EN-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language German
Relevant 378 Topic Fields title, description
Relevant retrieved 216 Pooled false
Geometric Mean Average Precision 0.0081 no BRF base run, stem snowball
Binary Preference (BPREF) 0.1421
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision
100%
0 29.92
HIGeodeenrun11
10 27.68 90%
20 25.94
80%
30 23.21
40 17.72 70%
50 16.09
Average Precision
60%
60 14.39
70 6.24 50%
80 4.89 40%
90 4.32
30%
100 3.43
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
15.04 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 0.8091
Minimum 0.0000
First Quartile 0.0007
Second Quartile 0.0089
Third Quartile 0.2037
Interquartile range 0.2030
Mean 0.1504
Standard Deviation 0.2363
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5066 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0987
Std With No Outliers 0.1600
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries HIGeodeenrun11
Topic 026 0.89 Topic 039 0.67 0.8
Topic 027 0.04 Topic 040 80.91
Topic 028 0.10 Topic 041 0.00
0.6
Topic 029 9.63 Topic 042 0.08
Topic 030 43.57 Topic 043 0.03
Topic 031 16.46 Topic 044 0.63 0.4
Topic 032 50.66 Topic 045 0.30
Topic 033 0.00 Topic 046 41.68 0.2
Topic 034 28.89 Topic 047 2.06
Difference
Topic 035 0.03 Topic 048 68.14 0
Topic 036 0.00 Topic 049 1.10
Topic 037 0.14 Topic 050 17.53
−0.2
Topic 038 12.50
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
343
hildesheim HIGeodeenrun11 GC-BILI-X2EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual English track − Retrieved documents vs Precision
100%
5 docs 19.20
HIGeodeenrun11
10 docs 17.60 90%
15 docs 15.73
80%
20 docs 14.20
30 docs 12.13 70%
100 docs 4.32
60%
200 docs 2.68
R−Precision
500 docs 1.49 50%
1000 docs 0.86 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
15.18
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 0.7143
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.2833
Interquartile range 0.2833
Mean 0.1518
Standard Deviation 0.2330
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6250 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1284
Std With No Outliers 0.2057
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries HIGeodeenrun11
Topic 026 0.00 Topic 039 0.00 0.8
Topic 027 0.00 Topic 040 71.43
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 22.22 Topic 042 0.00
Topic 030 50.00 Topic 043 0.00
Topic 031 22.03 Topic 044 0.00 0.4
Topic 032 58.06 Topic 045 0.00
Topic 033 0.00 Topic 046 33.33 0.2
Topic 034 33.33 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 62.50 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 26.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
344
hildesheim HIGeodeenrun13 GC-BILI-X2EN-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 21,396 Source Language German
Relevant 378 Topic Fields title, description
Relevant retrieved 178 Pooled false
Geometric Mean Average Precision 0.0038 Experiment with BRF(5docs,25terms) with NE-
Binary Preference (BPREF) 0.1337 recognition and weighting, also within the BRF
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision
100%
0 26.22
HIGeodeenrun13
10 22.77 90%
20 21.63
80%
30 20.42
40 18.83 70%
50 18.66
Average Precision
60%
60 16.47
70 9.67 50%
80 4.64 40%
90 2.43
30%
100 1.03
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
14.56 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 0.7828
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0070
Third Quartile 0.2040
Interquartile range 0.2040
Mean 0.1456
Standard Deviation 0.2567
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4530 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0624
Std With No Outliers 0.1216
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries HIGeodeenrun13
Topic 026 0.70 Topic 039 0.46 0.8
Topic 027 0.14 Topic 040 78.28
Topic 028 0.00 Topic 041 0.09
0.6
Topic 029 3.84 Topic 042 0.00
Topic 030 70.14 Topic 043 0.00
Topic 031 21.59 Topic 044 2.92 0.4
Topic 032 45.30 Topic 045 0.23
Topic 033 0.00 Topic 046 31.12 0.2
Topic 034 1.52 Topic 047 2.59
Difference
Topic 035 0.00 Topic 048 78.12 0
Topic 036 0.00 Topic 049 0.52
Topic 037 0.00 Topic 050 6.38
−0.2
Topic 038 20.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
345
hildesheim HIGeodeenrun13 GC-BILI-X2EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual English track − Retrieved documents vs Precision
100%
5 docs 20.00
HIGeodeenrun13
10 docs 16.80 90%
15 docs 14.13
80%
20 docs 13.80
30 docs 11.87 70%
100 docs 4.88
60%
200 docs 3.08
R−Precision
500 docs 1.38 50%
1000 docs 0.71 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
14.65
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 0.7143
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0000
Third Quartile 0.2136
Interquartile range 0.2136
Mean 0.1465
Standard Deviation 0.2491
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3333 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.0473
Std With No Outliers 0.0963
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
20
Number of Topics of the Experiment
15
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries HIGeodeenrun13
Topic 026 0.00 Topic 039 0.00 0.8
Topic 027 0.00 Topic 040 71.43
Topic 028 0.00 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 66.67 Topic 043 0.00
Topic 031 25.42 Topic 044 5.26 0.4
Topic 032 58.06 Topic 045 0.00
Topic 033 0.00 Topic 046 33.33 0.2
Topic 034 0.00 Topic 047 4.17
Difference
Topic 035 0.00 Topic 048 70.83 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 20.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
346
jaen sinaiEsEnExp1 GC-BILI-X2EN-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Spanish; Castilian
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 280 Pooled true
Geometric Mean Average Precision 0.0185 Caso base ESEN
Binary Preference (BPREF) 0.2420
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision
100%
0 47.54
sinaiEsEnExp1
10 44.48 90%
20 32.64
80%
30 29.39
40 29.19 70%
50 28.75
Average Precision
60%
60 25.59
70 21.77 50%
80 21.09 40%
90 18.68
30%
100 15.54
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
27.07 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 0.9713
Minimum 0.0000
First Quartile 0.0047
Second Quartile 0.1597
Third Quartile 0.3573
Interquartile range 0.3525
Mean 0.2707
Standard Deviation 0.3413
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6429 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1435
Std With No Outliers 0.1829
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries sinaiEsEnExp1
Topic 026 20.88 Topic 039 8.94 0.8
Topic 027 0.00 Topic 040 26.94
Topic 028 17.59 Topic 041 0.63
0.6
Topic 029 15.91 Topic 042 58.33
Topic 030 94.84 Topic 043 0.00
Topic 031 15.97 Topic 044 7.86 0.4
Topic 032 97.13 Topic 045 16.67
Topic 033 1.36 Topic 046 91.67 0.2
Topic 034 28.19 Topic 047 0.01
Difference
Topic 035 0.93 Topic 048 91.79 0
Topic 036 0.00 Topic 049 64.29
Topic 037 0.00 Topic 050 16.93
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
347
jaen sinaiEsEnExp1 GC-BILI-X2EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual English track − Retrieved documents vs Precision
100%
5 docs 28.00
sinaiEsEnExp1
10 docs 19.60 90%
15 docs 16.80
80%
20 docs 15.40
30 docs 14.00 70%
100 docs 6.68
60%
200 docs 4.10
R−Precision
500 docs 2.06 50%
1000 docs 1.12 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
24.27
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 0.9355
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1333
Third Quartile 0.3750
Interquartile range 0.3750
Mean 0.2427
Standard Deviation 0.2976
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.9355 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2427
Std With No Outliers 0.2976
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries sinaiEsEnExp1
Topic 026 22.22 Topic 039 6.25 0.8
Topic 027 0.00 Topic 040 21.43
Topic 028 21.05 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 50.00
Topic 030 83.33 Topic 043 0.00
Topic 031 16.95 Topic 044 10.53 0.4
Topic 032 93.55 Topic 045 16.67
Topic 033 5.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 85.42 0
Topic 036 0.00 Topic 049 50.00
Topic 037 0.00 Topic 050 13.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
348
jaen sinaiDeEnExp2 GC-BILI-X2EN-CLEF2006
Overall statistics for 25 queries : Priority 5
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language German
Relevant 378 Topic Fields title, description
Relevant retrieved 324 Pooled false
Geometric Mean Average Precision 0.0602 Caso Base DEEN
Binary Preference (BPREF) 0.1675
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision
100%
0 41.27
sinaiDeEnExp2
10 34.02 90%
20 29.37
80%
30 25.85
40 23.64 70%
50 21.97
Average Precision
60%
60 19.99
70 17.72 50%
80 16.35 40%
90 12.07
30%
100 8.80
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
21.64 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 0.9086
Minimum 0.0000
First Quartile 0.0433
Second Quartile 0.0954
Third Quartile 0.3542
Interquartile range 0.3108
Mean 0.2164
Standard Deviation 0.2395
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7905 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1875
Std With No Outliers 0.1954
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries sinaiDeEnExp2
Topic 026 22.57 Topic 039 4.83 0.8
Topic 027 4.44 Topic 040 25.23
Topic 028 31.83 Topic 041 1.19
0.6
Topic 029 4.01 Topic 042 35.00
Topic 030 5.94 Topic 043 1.34
Topic 031 40.49 Topic 044 19.71 0.4
Topic 032 79.05 Topic 045 7.85
Topic 033 0.00 Topic 046 40.00 0.2
Topic 034 41.67 Topic 047 5.71
Difference
Topic 035 2.28 Topic 048 90.86 0
Topic 036 0.00 Topic 049 36.67
Topic 037 9.54 Topic 050 23.00
−0.2
Topic 038 7.69
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
349
jaen sinaiDeEnExp2 GC-BILI-X2EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual English track − Retrieved documents vs Precision
100%
5 docs 24.80
sinaiDeEnExp2
10 docs 22.40 90%
15 docs 20.27
80%
20 docs 18.20
30 docs 16.67 70%
100 docs 8.96
60%
200 docs 5.42
R−Precision
500 docs 2.46 50%
1000 docs 1.30 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
19.55
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 0.8542
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1111
Third Quartile 0.3333
Interquartile range 0.3333
Mean 0.1955
Standard Deviation 0.2383
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.7419 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1681
Std With No Outliers 0.1990
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries sinaiDeEnExp2
Topic 026 22.22 Topic 039 0.00 0.8
Topic 027 10.53 Topic 040 21.43
Topic 028 36.84 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 50.00
Topic 030 0.00 Topic 043 0.00
Topic 031 40.68 Topic 044 28.95 0.4
Topic 032 74.19 Topic 045 0.00
Topic 033 0.00 Topic 046 33.33 0.2
Topic 034 33.33 Topic 047 8.33
Difference
Topic 035 0.00 Topic 048 85.42 0
Topic 036 0.00 Topic 049 0.00
Topic 037 12.50 Topic 050 20.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
350
jaen sinaiEsEnExp3 GC-BILI-X2EN-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Spanish; Castilian
Relevant 378 Topic Fields title, description
Relevant retrieved 311 Pooled false
Geometric Mean Average Precision 0.0353 Expansión con geonames
Binary Preference (BPREF) 0.1751
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision
100%
0 41.03
sinaiEsEnExp3
10 29.06 90%
20 26.45
80%
30 25.83
40 25.05 70%
50 24.62
Average Precision
60%
60 23.98
70 19.48 50%
80 18.17 40%
90 14.71
30%
100 12.01
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
22.09 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0229
Second Quartile 0.0966
Third Quartile 0.2829
Interquartile range 0.2600
Mean 0.2209
Standard Deviation 0.3045
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4514 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1206
Std With No Outliers 0.1345
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries sinaiEsEnExp3
Topic 026 0.00 Topic 039 5.06 0.8
Topic 027 0.00 Topic 040 26.80
Topic 028 25.64 Topic 041 2.25
0.6
Topic 029 7.51 Topic 042 6.16
Topic 030 100.00 Topic 043 2.20
Topic 031 12.56 Topic 044 10.50 0.4
Topic 032 94.60 Topic 045 10.62
Topic 033 0.21 Topic 046 32.78 0.2
Topic 034 45.14 Topic 047 3.72
Difference
Topic 035 2.31 Topic 048 92.19 0
Topic 036 0.00 Topic 049 36.67
Topic 037 9.66 Topic 050 23.30
−0.2
Topic 038 2.33
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
351
jaen sinaiEsEnExp3 GC-BILI-X2EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual English track − Retrieved documents vs Precision
100%
5 docs 22.40
sinaiEsEnExp3
10 docs 18.80 90%
15 docs 17.07
80%
20 docs 16.60
30 docs 14.67 70%
100 docs 7.32
60%
200 docs 4.78
R−Precision
500 docs 2.30 50%
1000 docs 1.24 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
20.42
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0789
Third Quartile 0.2789
Interquartile range 0.2789
Mean 0.2042
Standard Deviation 0.3055
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1091
Std With No Outliers 0.1643
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries sinaiEsEnExp3
Topic 026 0.00 Topic 039 0.00 0.8
Topic 027 0.00 Topic 040 14.29
Topic 028 31.58 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 100.00 Topic 043 12.50
Topic 031 6.78 Topic 044 7.89 0.4
Topic 032 87.10 Topic 045 16.67
Topic 033 0.00 Topic 046 33.33 0.2
Topic 034 66.67 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 83.33 0
Topic 036 0.00 Topic 049 0.00
Topic 037 12.50 Topic 050 26.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
352
jaen sinaiDeEnExp1 GC-BILI-X2EN-CLEF2006
Overall statistics for 25 queries : Priority 4
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language German
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 293 Pooled false
Geometric Mean Average Precision 0.0369 Caso Base DEEN
Binary Preference (BPREF) 0.1464
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision
100%
0 44.94
sinaiDeEnExp1
10 31.63 90%
20 24.65
80%
30 20.56
40 19.46 70%
50 18.90
Average Precision
60%
60 15.60
70 14.56 50%
80 14.04 40%
90 10.84
30%
100 8.34
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
18.68 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 0.9625
Minimum 0.0000
First Quartile 0.0119
Second Quartile 0.0884
Third Quartile 0.2174
Interquartile range 0.2054
Mean 0.1868
Standard Deviation 0.2643
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3203 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0990
Std With No Outliers 0.0971
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries sinaiDeEnExp1
Topic 026 19.05 Topic 039 15.98 0.8
Topic 027 6.72 Topic 040 32.03
Topic 028 11.88 Topic 041 1.14
0.6
Topic 029 11.53 Topic 042 25.00
Topic 030 3.49 Topic 043 3.49
Topic 031 24.41 Topic 044 7.41 0.4
Topic 032 96.25 Topic 045 20.85
Topic 033 0.25 Topic 046 5.91 0.2
Topic 034 8.84 Topic 047 0.16
Difference
Topic 035 1.21 Topic 048 90.47 0
Topic 036 0.00 Topic 049 62.50
Topic 037 0.00 Topic 050 18.02
−0.2
Topic 038 0.51
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
353
jaen sinaiDeEnExp1 GC-BILI-X2EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual English track − Retrieved documents vs Precision
100%
5 docs 21.60
sinaiDeEnExp1
10 docs 17.60 90%
15 docs 16.27
80%
20 docs 15.60
30 docs 14.67 70%
100 docs 6.92
60%
200 docs 4.50
R−Precision
500 docs 2.14 50%
1000 docs 1.17 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
16.49
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 0.9032
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1053
Third Quartile 0.2208
Interquartile range 0.2208
Mean 0.1649
Standard Deviation 0.2517
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1029
Std With No Outliers 0.1369
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries sinaiDeEnExp1
Topic 026 22.22 Topic 039 12.50 0.8
Topic 027 10.53 Topic 040 28.57
Topic 028 15.79 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 0.00 Topic 043 0.00
Topic 031 22.03 Topic 044 10.53 0.4
Topic 032 90.32 Topic 045 33.33
Topic 033 0.00 Topic 046 0.00 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 85.42 0
Topic 036 0.00 Topic 049 50.00
Topic 037 0.00 Topic 050 20.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
354
jaen sinaiEsEnExp2 GC-BILI-X2EN-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Spanish; Castilian
Relevant 378 Topic Fields title, description
Relevant retrieved 312 Pooled true
Geometric Mean Average Precision 0.0364 Caso Base ESEN
Binary Preference (BPREF) 0.1811
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision
100%
0 38.36
sinaiEsEnExp2
10 32.16 90%
20 29.44
80%
30 26.16
40 25.05 70%
50 24.67
Average Precision
60%
60 23.93
70 19.69 50%
80 18.20 40%
90 14.24
30%
100 11.83
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
22.56 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 0.9762
Minimum 0.0000
First Quartile 0.0229
Second Quartile 0.0966
Third Quartile 0.2829
Interquartile range 0.2600
Mean 0.2256
Standard Deviation 0.3015
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4514 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1269
Std With No Outliers 0.1370
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries sinaiEsEnExp2
Topic 026 20.81 Topic 039 5.06 0.8
Topic 027 0.00 Topic 040 26.80
Topic 028 25.64 Topic 041 2.25
0.6
Topic 029 3.47 Topic 042 6.16
Topic 030 97.62 Topic 043 2.20
Topic 031 16.26 Topic 044 14.37 0.4
Topic 032 95.07 Topic 045 0.28
Topic 033 0.00 Topic 046 32.78 0.2
Topic 034 45.14 Topic 047 3.72
Difference
Topic 035 2.31 Topic 048 92.19 0
Topic 036 0.00 Topic 049 36.67
Topic 037 9.66 Topic 050 23.30
−0.2
Topic 038 2.33
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
355
jaen sinaiEsEnExp2 GC-BILI-X2EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual English track − Retrieved documents vs Precision
100%
5 docs 21.60
sinaiEsEnExp2
10 docs 18.80 90%
15 docs 17.33
80%
20 docs 16.40
30 docs 14.80 70%
100 docs 7.80
60%
200 docs 5.02
R−Precision
500 docs 2.38 50%
1000 docs 1.25 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
20.63
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 0.8710
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1111
Third Quartile 0.2789
Interquartile range 0.2789
Mean 0.2063
Standard Deviation 0.2870
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6667 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1191
Std With No Outliers 0.1665
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries sinaiEsEnExp2
Topic 026 22.22 Topic 039 0.00 0.8
Topic 027 0.00 Topic 040 14.29
Topic 028 31.58 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 0.00
Topic 030 83.33 Topic 043 12.50
Topic 031 10.17 Topic 044 21.05 0.4
Topic 032 87.10 Topic 045 0.00
Topic 033 0.00 Topic 046 33.33 0.2
Topic 034 66.67 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 83.33 0
Topic 036 0.00 Topic 049 0.00
Topic 037 12.50 Topic 050 26.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
356
sanmarcos SMGeoESEN1 GC-BILI-X2EN-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction MANUAL
Retrieved 24,252 Source Language Spanish; Castilian
Relevant 378 Topic Fields title, description, narrative
Relevant retrieved 318 Pooled true
Geometric Mean Average Precision 0.0806 Spanish-English using all topic fields and other
Binary Preference (BPREF) 0.2164 sources
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision
100%
0 50.00
SMGeoESEN1
10 43.95 90%
20 36.81
80%
30 32.72
40 31.41 70%
50 29.72
Average Precision
60%
60 25.24
70 13.74 50%
80 11.06 40%
90 8.59
30%
100 6.39
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
25.52 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 0.9601
Minimum 0.0000
First Quartile 0.0430
Second Quartile 0.1743
Third Quartile 0.3584
Interquartile range 0.3154
Mean 0.2552
Standard Deviation 0.2771
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8258 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2258
Std With No Outliers 0.2400
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoESEN1
Topic 026 4.43 Topic 039 36.86 0.8
Topic 027 7.30 Topic 040 32.50
Topic 028 20.03 Topic 041 0.30
0.6
Topic 029 5.72 Topic 042 30.26
Topic 030 82.58 Topic 043 1.22
Topic 031 49.12 Topic 044 9.27 0.4
Topic 032 96.01 Topic 045 35.50
Topic 033 5.52 Topic 046 67.30 0.2
Topic 034 33.99 Topic 047 4.75
Difference
Topic 035 3.90 Topic 048 66.89 0
Topic 036 0.00 Topic 049 17.43
Topic 037 0.29 Topic 050 23.01
−0.2
Topic 038 3.85
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
357
sanmarcos SMGeoESEN1 GC-BILI-X2EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual English track − Retrieved documents vs Precision
100%
5 docs 28.80
SMGeoESEN1
10 docs 24.40 90%
15 docs 23.47
80%
20 docs 21.80
30 docs 18.40 70%
100 docs 8.16
60%
200 docs 5.00
R−Precision
500 docs 2.34 50%
1000 docs 1.27 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
24.80
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 0.9032
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1500
Third Quartile 0.3999
Interquartile range 0.3999
Mean 0.2480
Standard Deviation 0.2643
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.9032 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2480
Std With No Outliers 0.2643
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoESEN1
Topic 026 11.11 Topic 039 37.50 0.8
Topic 027 5.26 Topic 040 35.71
Topic 028 21.05 Topic 041 0.00
0.6
Topic 029 11.11 Topic 042 50.00
Topic 030 66.67 Topic 043 0.00
Topic 031 47.46 Topic 044 10.53 0.4
Topic 032 90.32 Topic 045 16.67
Topic 033 15.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 8.33
Difference
Topic 035 0.00 Topic 048 66.67 0
Topic 036 0.00 Topic 049 0.00
Topic 037 0.00 Topic 050 26.67
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
358
sanmarcos SMGeoESEN2 GC-BILI-X2EN-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Spanish; Castilian
Relevant 378 Topic Fields title, description
Relevant retrieved 303 Pooled true
Geometric Mean Average Precision 0.0512 Automatic Spanish English title+des
Binary Preference (BPREF) 0.2039
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual English track − Interpolated Recall vs Average Precision
100%
0 43.77
SMGeoESEN2
10 35.49 90%
20 28.62
80%
30 27.29
40 26.08 70%
50 25.13
Average Precision
60%
60 22.38
70 16.74 50%
80 12.44 40%
90 11.27
30%
100 8.86
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
22.46 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0150
Second Quartile 0.1157
Third Quartile 0.2798
Interquartile range 0.2648
Mean 0.2246
Standard Deviation 0.2979
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4242 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1095
Std With No Outliers 0.1268
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoESEN2
Topic 026 6.35 Topic 039 16.01 0.8
Topic 027 1.18 Topic 040 26.20
Topic 028 11.57 Topic 041 0.23
0.6
Topic 029 21.08 Topic 042 0.82
Topic 030 100.00 Topic 043 1.67
Topic 031 13.41 Topic 044 22.48 0.4
Topic 032 91.32 Topic 045 1.67
Topic 033 0.46 Topic 046 69.30 0.2
Topic 034 42.42 Topic 047 3.44
Difference
Topic 035 2.10 Topic 048 70.96 0
Topic 036 0.00 Topic 049 33.33
Topic 037 1.60 Topic 050 23.43
−0.2
Topic 038 0.56
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
359
sanmarcos SMGeoESEN2 GC-BILI-X2EN-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual English track − Retrieved documents vs Precision
100%
5 docs 24.80
SMGeoESEN2
10 docs 21.20 90%
15 docs 18.93
80%
20 docs 17.60
30 docs 15.20 70%
100 docs 7.32
60%
200 docs 4.40
R−Precision
500 docs 2.06 50%
1000 docs 1.21 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
23.29
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual English track − Box plot of the Topics of the Experiment
Maximum 1.0000
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1111
Third Quartile 0.3355
Interquartile range 0.3355
Mean 0.2329
Standard Deviation 0.2905
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.8065 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2010
Std With No Outliers 0.2479
GeoCLEF Bilingual English track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual English track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoESEN2
Topic 026 11.11 Topic 039 12.50 0.8
Topic 027 5.26 Topic 040 21.43
Topic 028 10.53 Topic 041 0.00
0.6
Topic 029 22.22 Topic 042 0.00
Topic 030 100.00 Topic 043 0.00
Topic 031 13.56 Topic 044 34.21 0.4
Topic 032 80.65 Topic 045 0.00
Topic 033 0.00 Topic 046 66.67 0.2
Topic 034 33.33 Topic 047 8.33
Difference
Topic 035 0.00 Topic 048 72.92 0
Topic 036 0.00 Topic 049 50.00
Topic 037 6.25 Topic 050 33.33
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
360
berkeley BKGeoES1 GC-BILI-X2ES-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 2,054 Topic Fields title, description
Relevant retrieved 1,504 Pooled true
Geometric Mean Average Precision 0.0552 EN->ES using L&H query translation. title, desc
Binary Preference (BPREF) 0.2676
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual Spanish track − Interpolated Recall vs Average Precision
100%
0 52.97
BKGeoES1
10 40.03 90%
20 33.75
80%
30 31.01
40 28.71 70%
50 26.83
Average Precision
60%
60 25.15
70 21.98 50%
80 19.08 40%
90 13.02
30%
100 4.85
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
25.71 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.9782
Minimum 0.0000
First Quartile 0.0184
Second Quartile 0.1361
Third Quartile 0.4076
Interquartile range 0.3892
Mean 0.2571
Standard Deviation 0.3008
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.9782 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2571
Std With No Outliers 0.3008
GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries BKGeoES1
Topic 026 0.01 Topic 039 0.98 0.8
Topic 027 0.84 Topic 040 56.39
Topic 028 11.51 Topic 041 22.43
0.6
Topic 029 25.94 Topic 042 35.55
Topic 030 0.00 Topic 043 0.13
Topic 031 65.65 Topic 044 18.03 0.4
Topic 032 97.82 Topic 045 5.67
Topic 033 2.12 Topic 046 61.44 0.2
Topic 034 13.61 Topic 047 28.60
Difference
Topic 035 0.71 Topic 048 82.80 0
Topic 036 2.96 Topic 049 78.74
Topic 037 6.28 Topic 050 17.96
−0.2
Topic 038 6.63
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
361
berkeley BKGeoES1 GC-BILI-X2ES-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual Spanish track − Retrieved documents vs Precision
100%
5 docs 32.00
BKGeoES1
10 docs 34.00 90%
15 docs 33.07
80%
20 docs 31.80
30 docs 30.53 70%
100 docs 27.20
60%
200 docs 20.88
R−Precision
500 docs 11.08 50%
1000 docs 6.02 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
26.45
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.9000
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.1667
Third Quartile 0.4490
Interquartile range 0.4490
Mean 0.2645
Standard Deviation 0.2871
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.9000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2645
Std With No Outliers 0.2871
GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries BKGeoES1
Topic 026 0.00 Topic 039 2.99 0.8
Topic 027 0.00 Topic 040 61.87
Topic 028 5.56 Topic 041 34.67
0.6
Topic 029 30.30 Topic 042 39.62
Topic 030 0.00 Topic 043 0.00
Topic 031 67.45 Topic 044 28.16 0.4
Topic 032 90.00 Topic 045 16.67
Topic 033 3.00 Topic 046 60.71 0.2
Topic 034 13.51 Topic 047 33.90
Difference
Topic 035 0.00 Topic 048 73.96 0
Topic 036 6.36 Topic 049 70.51
Topic 037 0.00 Topic 050 22.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
362
berkeley BKGeoES2 GC-BILI-X2ES-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 2,054 Topic Fields title, description, narrative
Relevant retrieved 1,450 Pooled true
Geometric Mean Average Precision 0.0421 EN->ES using L&H query translation. title, desc
Binary Preference (BPREF) 0.2882 and narrative
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual Spanish track − Interpolated Recall vs Average Precision
100%
0 54.49
BKGeoES2
10 39.70 90%
20 38.13
80%
30 35.96
40 32.79 70%
50 29.58
Average Precision
60%
60 26.75
70 22.65 50%
80 19.31 40%
90 12.71
30%
100 3.53
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
27.45 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.9740
Minimum 0.0000
First Quartile 0.0128
Second Quartile 0.1018
Third Quartile 0.5579
Interquartile range 0.5451
Mean 0.2745
Standard Deviation 0.3135
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.9740 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.2745
Std With No Outliers 0.3135
GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries BKGeoES2
Topic 026 0.11 Topic 039 53.58 0.8
Topic 027 38.67 Topic 040 67.60
Topic 028 10.67 Topic 041 16.07
0.6
Topic 029 47.52 Topic 042 47.96
Topic 030 0.00 Topic 043 3.18
Topic 031 62.41 Topic 044 2.83 0.4
Topic 032 97.40 Topic 045 1.38
Topic 033 1.53 Topic 046 63.69 0.2
Topic 034 9.83 Topic 047 0.97
Difference
Topic 035 2.03 Topic 048 73.23 0
Topic 036 0.02 Topic 049 74.35
Topic 037 0.00 Topic 050 10.18
−0.2
Topic 038 0.93
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
363
berkeley BKGeoES2 GC-BILI-X2ES-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual Spanish track − Retrieved documents vs Precision
100%
5 docs 33.60
BKGeoES2
10 docs 36.00 90%
15 docs 35.20
80%
20 docs 34.20
30 docs 32.93 70%
100 docs 28.48
60%
200 docs 20.48
R−Precision
500 docs 10.34 50%
1000 docs 5.80 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
27.04
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.9077
Minimum 0.0000
First Quartile 0.0127
Second Quartile 0.1351
Third Quartile 0.5381
Interquartile range 0.5254
Mean 0.2704
Standard Deviation 0.2934
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.9077 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.2704
Std With No Outliers 0.2934
GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries BKGeoES2
Topic 026 0.00 Topic 039 50.75 0.8
Topic 027 41.03 Topic 040 68.35
Topic 028 13.89 Topic 041 18.67
0.6
Topic 029 54.55 Topic 042 50.94
Topic 030 0.00 Topic 043 4.17
Topic 031 60.39 Topic 044 7.77 0.4
Topic 032 90.77 Topic 045 0.00
Topic 033 4.00 Topic 046 53.57 0.2
Topic 034 13.51 Topic 047 1.69
Difference
Topic 035 5.26 Topic 048 63.02 0
Topic 036 0.00 Topic 049 69.59
Topic 037 0.00 Topic 050 4.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
364
sanmarcos SMGeoENES1 GC-BILI-X2ES-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 2,054 Topic Fields title, description
Relevant retrieved 729 Pooled true
Geometric Mean Average Precision 0.0291 Automatic English Spanish title + desc
Binary Preference (BPREF) 0.1505
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual Spanish track − Interpolated Recall vs Average Precision
100%
0 57.89
SMGeoENES1
10 36.39 90%
20 24.05
80%
30 15.35
40 11.82 70%
50 9.72
Average Precision
60%
60 7.36
70 2.52 50%
80 0.00 40%
90 0.00
30%
100 0.00
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
12.82 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.5745
Minimum 0.0000
First Quartile 0.0179
Second Quartile 0.0991
Third Quartile 0.1747
Interquartile range 0.1568
Mean 0.1282
Standard Deviation 0.1538
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.2954 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0909
Std With No Outliers 0.0868
GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoENES1
Topic 026 2.12 Topic 039 16.72 0.8
Topic 027 2.04 Topic 040 54.13
Topic 028 24.55 Topic 041 0.86
0.6
Topic 029 0.98 Topic 042 20.06
Topic 030 13.11 Topic 043 0.00
Topic 031 1.02 Topic 044 29.54 0.4
Topic 032 19.71 Topic 045 2.21
Topic 033 0.01 Topic 046 11.07 0.2
Topic 034 15.87 Topic 047 6.94
Difference
Topic 035 9.91 Topic 048 5.81 0
Topic 036 11.02 Topic 049 57.45
Topic 037 11.45 Topic 050 4.01
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
365
sanmarcos SMGeoENES1 GC-BILI-X2ES-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual Spanish track − Retrieved documents vs Precision
100%
5 docs 44.00
SMGeoENES1
10 docs 35.60 90%
15 docs 33.60
80%
20 docs 31.00
30 docs 28.53 70%
100 docs 17.28
60%
200 docs 11.28
R−Precision
500 docs 5.30 50%
1000 docs 2.92 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
16.89
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.6682
Minimum 0.0000
First Quartile 0.0291
Second Quartile 0.1429
Third Quartile 0.2302
Interquartile range 0.2010
Mean 0.1689
Standard Deviation 0.1763
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3786 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1279
Std With No Outliers 0.1091
GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoENES1
Topic 026 0.00 Topic 039 23.88 0.8
Topic 027 2.56 Topic 040 61.15
Topic 028 27.78 Topic 041 6.67
0.6
Topic 029 3.03 Topic 042 28.30
Topic 030 17.88 Topic 043 0.00
Topic 031 5.10 Topic 044 37.86 0.4
Topic 032 20.77 Topic 045 0.00
Topic 033 0.00 Topic 046 14.29 0.2
Topic 034 18.92 Topic 047 11.86
Difference
Topic 035 15.79 Topic 048 7.55 0
Topic 036 22.73 Topic 049 66.82
Topic 037 17.24 Topic 050 12.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
366
sanmarcos SMGeoPTES2 GC-BILI-X2ES-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Portuguese
Relevant 2,054 Topic Fields title, description
Relevant retrieved 659 Pooled true
Geometric Mean Average Precision 0.0221 Automatic Portuguese Spanish title+desc
Binary Preference (BPREF) 0.1261
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual Spanish track − Interpolated Recall vs Average Precision
100%
0 52.67
SMGeoPTES2
10 26.69 90%
20 20.30
80%
30 14.71
40 11.08 70%
50 8.86
Average Precision
60%
60 6.92
70 1.65 50%
80 0.00 40%
90 0.00
30%
100 0.00
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
10.89 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.5335
Minimum 0.0000
First Quartile 0.0082
Second Quartile 0.0317
Third Quartile 0.2020
Interquartile range 0.1938
Mean 0.1089
Standard Deviation 0.1522
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4566 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0913
Std With No Outliers 0.1266
GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoPTES2
Topic 026 1.72 Topic 039 27.01 0.8
Topic 027 3.17 Topic 040 53.35
Topic 028 7.13 Topic 041 1.00
0.6
Topic 029 28.54 Topic 042 31.64
Topic 030 0.29 Topic 043 0.03
Topic 031 0.87 Topic 044 5.52 0.4
Topic 032 19.83 Topic 045 2.29
Topic 033 0.09 Topic 046 5.19 0.2
Topic 034 21.30 Topic 047 3.17
Difference
Topic 035 2.99 Topic 048 5.65 0
Topic 036 0.68 Topic 049 45.66
Topic 037 0.05 Topic 050 5.21
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
367
sanmarcos SMGeoPTES2 GC-BILI-X2ES-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual Spanish track − Retrieved documents vs Precision
100%
5 docs 34.40
SMGeoPTES2
10 docs 30.40 90%
15 docs 28.53
80%
20 docs 25.20
30 docs 21.87 70%
100 docs 13.88
60%
200 docs 9.36
R−Precision
500 docs 4.75 50%
1000 docs 2.64 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
14.67
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.6043
Minimum 0.0000
First Quartile 0.0357
Second Quartile 0.0800
Third Quartile 0.2155
Interquartile range 0.1797
Mean 0.1467
Standard Deviation 0.1661
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3774 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1090
Std With No Outliers 0.1068
GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoPTES2
Topic 026 5.56 Topic 039 23.88 0.8
Topic 027 12.82 Topic 040 60.43
Topic 028 13.89 Topic 041 9.33
0.6
Topic 029 30.30 Topic 042 37.74
Topic 030 2.65 Topic 043 0.00
Topic 031 5.10 Topic 044 14.56 0.4
Topic 032 20.77 Topic 045 8.33
Topic 033 1.00 Topic 046 7.14 0.2
Topic 034 29.73 Topic 047 3.39
Difference
Topic 035 5.26 Topic 048 7.55 0
Topic 036 3.64 Topic 049 55.76
Topic 037 0.00 Topic 050 8.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
368
sanmarcos SMGeoPTES3 GC-BILI-X2ES-CLEF2006
Overall statistics for 25 queries : Priority 3
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Portuguese
Relevant 2,054 Topic Fields title, description, narrative
Relevant retrieved 655 Pooled true
Geometric Mean Average Precision 0.0230 Automatic Portuguese Spanish title-desc-narr
Binary Preference (BPREF) 0.1310
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual Spanish track − Interpolated Recall vs Average Precision
100%
0 54.49
SMGeoPTES3
10 29.04 90%
20 22.69
80%
30 15.38
40 11.65 70%
50 9.36
Average Precision
60%
60 7.29
70 1.81 50%
80 0.00 40%
90 0.00
30%
100 0.00
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
11.50 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.5378
Minimum 0.0000
First Quartile 0.0092
Second Quartile 0.0401
Third Quartile 0.2080
Interquartile range 0.1987
Mean 0.1150
Standard Deviation 0.1604
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3255 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0794
Std With No Outliers 0.1070
GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual Spanish track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoPTES3
Topic 026 1.78 Topic 039 27.15 0.8
Topic 027 2.67 Topic 040 53.78
Topic 028 8.07 Topic 041 0.99
0.6
Topic 029 32.55 Topic 042 31.54
Topic 030 0.28 Topic 043 0.03
Topic 031 1.43 Topic 044 5.84 0.4
Topic 032 20.01 Topic 045 2.15
Topic 033 0.06 Topic 046 5.76 0.2
Topic 034 23.16 Topic 047 3.37
Difference
Topic 035 4.01 Topic 048 5.68 0
Topic 036 0.72 Topic 049 51.21
Topic 037 0.05 Topic 050 5.28
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
369
sanmarcos SMGeoPTES3 GC-BILI-X2ES-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual Spanish track − Retrieved documents vs Precision
100%
5 docs 33.60
SMGeoPTES3
10 docs 32.40 90%
15 docs 27.73
80%
20 docs 26.40
30 docs 22.53 70%
100 docs 14.76
60%
200 docs 9.62
R−Precision
500 docs 4.77 50%
1000 docs 2.62 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
15.27
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual Spanish track − Box plot of the Topics of the Experiment
Maximum 0.6043
Minimum 0.0000
First Quartile 0.0320
Second Quartile 0.0755
Third Quartile 0.2192
Interquartile range 0.1872
Mean 0.1527
Standard Deviation 0.1771
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4054 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1139
Std With No Outliers 0.1205
GeoCLEF Bilingual Spanish track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual Spanish track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoPTES3
Topic 026 5.56 Topic 039 25.37 0.8
Topic 027 12.82 Topic 040 60.43
Topic 028 16.67 Topic 041 9.33
0.6
Topic 029 30.30 Topic 042 37.74
Topic 030 2.65 Topic 043 0.00
Topic 031 6.67 Topic 044 15.53 0.4
Topic 032 20.77 Topic 045 0.00
Topic 033 1.00 Topic 046 7.14 0.2
Topic 034 40.54 Topic 047 3.39
Difference
Topic 035 5.26 Topic 048 7.55 0
Topic 036 3.64 Topic 049 59.45
Topic 037 0.00 Topic 050 10.00
−0.2
Topic 038 0.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
370
berkeley BKGeoEP1 GC-BILI-X2PT-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 1,060 Topic Fields title, description
Relevant retrieved 591 Pooled true
Geometric Mean Average Precision 0.0098 EN->PT using L&H query translation. title and
Binary Preference (BPREF) 0.1291 description used
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual Portuguese track − Interpolated Recall vs Average Precision
100%
0 32.95
BKGeoEP1
10 26.74 90%
20 20.71
80%
30 17.10
40 13.29 70%
50 11.11
Average Precision
60%
60 9.49
70 7.84 50%
80 5.18 40%
90 3.64
30%
100 0.79
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
12.60 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.7932
Minimum 0.0000
First Quartile 0.0010
Second Quartile 0.0246
Third Quartile 0.1786
Interquartile range 0.1776
Mean 0.1260
Standard Deviation 0.1873
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.4374 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0982
Std With No Outliers 0.1283
GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries BKGeoEP1
Topic 026 1.22 Topic 039 5.45 0.8
Topic 027 0.03 Topic 040 0.01
Topic 028 12.27 Topic 041 0.01
0.6
Topic 029 27.03 Topic 042 10.65
Topic 030 0.81 Topic 043 2.46
Topic 031 43.74 Topic 044 0.00 0.4
Topic 032 14.19 Topic 045 27.39
Topic 033 0.00 Topic 046 33.97 0.2
Topic 034 0.14 Topic 047 0.13
Difference
Topic 035 0.55 Topic 048 79.32 0
Topic 036 0.00 Topic 049 24.37
Topic 037 1.29 Topic 050 15.70
−0.2
Topic 038 14.35
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
371
berkeley BKGeoEP1 GC-BILI-X2PT-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual Portuguese track − Retrieved documents vs Precision
100%
5 docs 19.20
BKGeoEP1
10 docs 21.20 90%
15 docs 20.00
80%
20 docs 19.40
30 docs 17.47 70%
100 docs 12.04
60%
200 docs 8.22
R−Precision
500 docs 4.26 50%
1000 docs 2.36 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
14.77
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.7343
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0417
Third Quartile 0.2740
Interquartile range 0.2740
Mean 0.1477
Standard Deviation 0.1905
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.3934 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1233
Std With No Outliers 0.1493
GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries BKGeoEP1
Topic 026 0.00 Topic 039 8.70 0.8
Topic 027 0.00 Topic 040 0.00
Topic 028 21.88 Topic 041 0.00
0.6
Topic 029 33.33 Topic 042 20.00
Topic 030 0.00 Topic 043 4.17
Topic 031 39.34 Topic 044 0.00 0.4
Topic 032 26.42 Topic 045 36.59
Topic 033 0.00 Topic 046 37.88 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 73.43 0
Topic 036 0.00 Topic 049 27.78
Topic 037 0.00 Topic 050 27.27
−0.2
Topic 038 12.50
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
372
berkeley BKGeoEP2 GC-BILI-X2PT-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language English
Relevant 1,060 Topic Fields title, description, narrative
Relevant retrieved 630 Pooled true
Geometric Mean Average Precision 0.0114 EN->PT using L&H query translation. title, desc and
Binary Preference (BPREF) 0.1422 narrative
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual Portuguese track − Interpolated Recall vs Average Precision
100%
0 41.75
BKGeoEP2
10 27.68 90%
20 22.31
80%
30 19.80
40 16.82 70%
50 13.98
Average Precision
60%
60 11.01
70 8.17 50%
80 6.09 40%
90 3.68
30%
100 0.53
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
14.30 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.6974
Minimum 0.0000
First Quartile 0.0012
Second Quartile 0.0776
Third Quartile 0.2403
Interquartile range 0.2391
Mean 0.1430
Standard Deviation 0.1868
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5165 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.1199
Std With No Outliers 0.1500
GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries BKGeoEP2
Topic 026 1.27 Topic 039 8.98 0.8
Topic 027 8.97 Topic 040 0.01
Topic 028 24.05 Topic 041 0.00
0.6
Topic 029 33.34 Topic 042 43.84
Topic 030 0.69 Topic 043 6.26
Topic 031 21.61 Topic 044 0.00 0.4
Topic 032 7.76 Topic 045 24.53
Topic 033 0.00 Topic 046 51.65 0.2
Topic 034 0.15 Topic 047 2.96
Difference
Topic 035 0.14 Topic 048 69.74 0
Topic 036 0.00 Topic 049 8.80
Topic 037 0.07 Topic 050 24.03
−0.2
Topic 038 18.63
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
373
berkeley BKGeoEP2 GC-BILI-X2PT-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual Portuguese track − Retrieved documents vs Precision
100%
5 docs 24.80
BKGeoEP2
10 docs 22.80 90%
15 docs 22.67
80%
20 docs 20.80
30 docs 19.73 70%
100 docs 13.36
60%
200 docs 9.14
R−Precision
500 docs 4.55 50%
1000 docs 2.52 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
16.34
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.6434
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0833
Third Quartile 0.3462
Interquartile range 0.3462
Mean 0.1634
Standard Deviation 0.1963
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.6434 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1634
Std With No Outliers 0.1963
GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries BKGeoEP2
Topic 026 0.00 Topic 039 13.04 0.8
Topic 027 20.59 Topic 040 0.00
Topic 028 34.38 Topic 041 0.00
0.6
Topic 029 38.46 Topic 042 42.86
Topic 030 0.00 Topic 043 8.33
Topic 031 21.31 Topic 044 0.00 0.4
Topic 032 20.75 Topic 045 35.37
Topic 033 0.00 Topic 046 54.55 0.2
Topic 034 0.00 Topic 047 0.00
Difference
Topic 035 0.00 Topic 048 64.34 0
Topic 036 0.00 Topic 049 5.56
Topic 037 0.00 Topic 050 36.36
−0.2
Topic 038 12.50
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
374
sanmarcos SMGeoESPT1 GC-BILI-X2PT-CLEF2006
Overall statistics for 25 queries : Priority 1
Total number of documents over all queries Query Construction MANUAL
Retrieved 25,000 Source Language Spanish; Castilian
Relevant 1,060 Topic Fields title, description
Relevant retrieved 608 Pooled true
Geometric Mean Average Precision 0.0511 Spanish-Portuguese
Binary Preference (BPREF) 0.1174
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual Portuguese track − Interpolated Recall vs Average Precision
100%
0 45.68
SMGeoESPT1
10 27.39 90%
20 23.53
80%
30 17.21
40 13.62 70%
50 10.70
Average Precision
60%
60 8.25
70 5.52 50%
80 4.28 40%
90 2.27
30%
100 0.27
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
12.81 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.4984
Minimum 0.0011
First Quartile 0.0203
Second Quartile 0.0697
Third Quartile 0.1727
Interquartile range 0.1524
Mean 0.1281
Standard Deviation 0.1540
Lower Outlier Threshold 0.0011
Upper Outlier Threshold 0.3866 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0960
Std With No Outliers 0.1114
GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment
15
Number of Topics of the Experiment
10
5
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoESPT1
Topic 026 1.17 Topic 039 30.28 0.8
Topic 027 6.97 Topic 040 1.41
Topic 028 8.54 Topic 041 1.15
0.6
Topic 029 6.86 Topic 042 49.61
Topic 030 38.66 Topic 043 0.11
Topic 031 14.38 Topic 044 7.75 0.4
Topic 032 49.84 Topic 045 2.95
Topic 033 1.16 Topic 046 25.94 0.2
Topic 034 2.24 Topic 047 8.84
Difference
Topic 035 2.47 Topic 048 2.29 0
Topic 036 2.88 Topic 049 14.11
Topic 037 0.14 Topic 050 10.34
−0.2
Topic 038 30.10
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
375
sanmarcos SMGeoESPT1 GC-BILI-X2PT-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual Portuguese track − Retrieved documents vs Precision
100%
5 docs 23.20
SMGeoESPT1
10 docs 21.60 90%
15 docs 18.67
80%
20 docs 17.40
30 docs 16.00 70%
100 docs 9.88
60%
200 docs 6.52
R−Precision
500 docs 3.97 50%
1000 docs 2.43 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
14.88
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.5849
Minimum 0.0000
First Quartile 0.0000
Second Quartile 0.0979
Third Quartile 0.2527
Interquartile range 0.2527
Mean 0.1488
Standard Deviation 0.1652
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5849 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1488
Std With No Outliers 0.1652
GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoESPT1
Topic 026 0.00 Topic 039 26.09 0.8
Topic 027 12.75 Topic 040 4.17
Topic 028 15.62 Topic 041 2.88
0.6
Topic 029 7.69 Topic 042 48.57
Topic 030 42.86 Topic 043 0.00
Topic 031 13.11 Topic 044 10.53 0.4
Topic 032 58.49 Topic 045 8.54
Topic 033 0.00 Topic 046 33.33 0.2
Topic 034 0.00 Topic 047 8.82
Difference
Topic 035 0.00 Topic 048 9.79 0
Topic 036 0.00 Topic 049 27.78
Topic 037 0.00 Topic 050 15.91
−0.2
Topic 038 25.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
376
sanmarcos SMGeoESPT2 GC-BILI-X2PT-CLEF2006
Overall statistics for 25 queries : Priority 2
Total number of documents over all queries Query Construction AUTOMATIC
Retrieved 25,000 Source Language Spanish; Castilian
Relevant 1,060 Topic Fields title, description
Relevant retrieved 655 Pooled true
Geometric Mean Average Precision 0.0497 Automatic Spanish Portuguese title + desc
Binary Preference (BPREF) 0.1415
Interploated Recall (%) Precision Averages (%)
GeoCLEF Bilingual Portuguese track − Interpolated Recall vs Average Precision
100%
0 45.27
SMGeoESPT2
10 27.06 90%
20 23.20
80%
30 19.45
40 16.32 70%
50 14.00
Average Precision
60%
60 11.33
70 8.11 50%
80 5.93 40%
90 3.39
30%
100 0.63
Average precision (non-interpolated) for all 20%
relevant documents (averaged over queries)
14.16 10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Interpolated Recall
Mean Average Precision GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.7579
Minimum 0.0002
First Quartile 0.0187
Second Quartile 0.0797
Third Quartile 0.1811
Interquartile range 0.1624
Mean 0.1416
Standard Deviation 0.1830
Lower Outlier Threshold 0.0002
Upper Outlier Threshold 0.3997 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Mean With No Outliers 0.0983
Std With No Outliers 0.1041
GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment
10
Number of Topics of the Experiment
8
6
4
2
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual Portuguese track − Comparison to Median Mean Average Precision by Topic (Topics 026 to 050)
queries SMGeoESPT2
Topic 026 7.79 Topic 039 8.25 0.8
Topic 027 0.16 Topic 040 2.84
Topic 028 13.15 Topic 041 1.12
0.6
Topic 029 23.65 Topic 042 25.88
Topic 030 1.64 Topic 043 0.02
Topic 031 16.14 Topic 044 7.97 0.4
Topic 032 75.79 Topic 045 23.82
Topic 033 0.17 Topic 046 9.22 0.2
Topic 034 3.21 Topic 047 5.42
Difference
Topic 035 2.76 Topic 048 51.93 0
Topic 036 1.92 Topic 049 39.97
Topic 037 1.72 Topic 050 13.12
−0.2
Topic 038 16.26
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
377
sanmarcos SMGeoESPT2 GC-BILI-X2PT-CLEF2006
Docs Cutoff Levels Precision at DCL (%)
GeoCLEF Bilingual Portuguese track − Retrieved documents vs Precision
100%
5 docs 21.60
SMGeoESPT2
10 docs 23.60 90%
15 docs 21.60
80%
20 docs 20.60
30 docs 19.60 70%
100 docs 12.76
60%
200 docs 8.46
R−Precision
500 docs 4.48 50%
1000 docs 2.62 40%
R-Precision (precision after R document retrieved,
where R = Relevant retrieved) 30%
17.42
20%
10%
0%
5 10 15 20 30 100 200 500 1000
Retrieved Documents (logarithmic scale)
Exact R-Precision GeoCLEF Bilingual Portuguese track − Box plot of the Topics of the Experiment
Maximum 0.7547
Minimum 0.0000
First Quartile 0.0366
Second Quartile 0.1250
Third Quartile 0.2644
Interquartile range 0.2278
Mean 0.1742
Standard Deviation 0.1901
Lower Outlier Threshold 0.0000
Upper Outlier Threshold 0.5315 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Mean With No Outliers 0.1500
Std With No Outliers 0.1498
GeoCLEF Bilingual Portuguese track − Distribution of the Topics of the Experiment
8
Number of Topics of the Experiment
7
6
5
4
3
2
1
0
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Exact R−Precision
Precision averages (%) for individual 1
GeoCLEF Bilingual Portuguese track − Comparison to Median Mean Exact R−Precision by Topic (Topics 026 to 050)
queries SMGeoESPT2
Topic 026 6.67 Topic 039 13.04 0.8
Topic 027 3.92 Topic 040 12.50
Topic 028 21.88 Topic 041 2.88
0.6
Topic 029 30.77 Topic 042 31.43
Topic 030 7.14 Topic 043 0.00
Topic 031 16.39 Topic 044 9.21 0.4
Topic 032 75.47 Topic 045 35.37
Topic 033 0.00 Topic 046 16.67 0.2
Topic 034 0.00 Topic 047 5.88
Difference
Topic 035 0.00 Topic 048 53.15 0
Topic 036 0.00 Topic 049 44.44
Topic 037 5.56 Topic 050 18.18
−0.2
Topic 038 25.00
−0.4
−0.6
−0.8
−1
026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Topic Identifier
378