List of Submitted Runs
A.7 u.budapest u.budapest u.budapest u.budapest u.budapest u.budapest clips clips clips clips clips clips clips cmu cmu cmu cmu cmu cmu cmu cmu cmu cmu cmu cmu miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle miracle dcu dcu dcu dcu dcu dcu dcu dcu cocri cocri cocri cocri cocri hummingbird hummingbird hummingbird hummingbird hummingbird hummingbird hummingbird hummingbird hummingbird hummingbird hummingbird hummingbird irit irit irit irit irit jhu/apl jhu/apl jhu/apl jhu/apl jhu/apl jhu/apl jhu/apl jhu/apl jhu/apl jhu/apl jhu/apl jhu/apl jhu/apl jhu/apl jhu/apl jhu/apl jhu/apl jhu/apl jhu/apl jhu/apl automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic manual manual manual manual manual automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic no no yes no yes no no no no no no no no no no no yes no no yes yes yes yes no no yes yes yes yes no no yes yes yes yes yes yes yes yes yes no yes no yes yes yes no yes yes no yes no no no yes yes jhu/apl jhu/apl jhu/apl jhu/apl jhu/apl msu-nivc msu-nivc msu-nivc msu-nivc msu-nivc sics sics sics tlr tlr tlr tlr tlr tlr tlr tlr tlr alicante alicante alicante alicante alicante alicante alicante alicante alicante alicante alicante alicante alicante alicante alicante alicante alicante alicante alicante ilps ilps ilps ilps berkeley berkeley berkeley berkeley berkeley berkeley berkeley berkeley berkeley berkeley berkeley AH-Monolingual-HU AH-Monolingual-PT AH-Monolingual-PT AH-Monolingual-PT AH-Monolingual-PT AH-Bilingual-X2EN DS-Monolingual-EN DS-Bilingual-X2EN DS-Bilingual-X2RU DS-Monolingual-RU AH-Monolingual-FR AH-Monolingual-FR AH-Monolingual-FR AH-Bilingual-X2PT AH-Bilingual-X2PT AH-Monolingual-FR AH-Monolingual-FR AH-Monolingual-HU AH-Monolingual-HU AH-Monolingual-PT AH-Monolingual-PT AH-Monolingual-PT AH-Monolingual-BG AH-Monolingual-BG AH-Monolingual-BG AH-Bilingual-X2FR AH-Bilingual-X2FR AH-Bilingual-X2FR AH-Bilingual-X2PT AH-Bilingual-X2PT AH-Bilingual-X2PT AH-Bilingual-X2PT AH-Monolingual-FR AH-Monolingual-FR AH-Monolingual-FR AH-Monolingual-HU AH-Monolingual-HU AH-Monolingual-HU AH-Monolingual-PT AH-Monolingual-PT AH-Monolingual-PT AH-Monolingual-HU AH-Monolingual-HU AH-Monolingual-HU AH-Monolingual-HU DS-Bilingual-X2EN DS-Bilingual-X2RU DS-Bilingual-X2DE DS-Bilingual-X2RU DS-Bilingual-X2DE DS-Bilingual-X2EN DS-Monolingual-DE DS-Monolingual-EN DS-Monolingual-RU DS-Multi
DS-Multi automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic no no yes no no yes yes yes yes yes yes no no no yes no yes yes yes no no yes yes yes no no no no no yes no no no yes no yes yes no no yes no yes yes no no yes yes yes yes yes yes yes yes yes no no berkeley berkeley-2 berkeley-2 berkeley-2 berkeley-2 berkeley-2 berkeley-2 berkeley-2 berkeley-2 berkeley-2 berkeley-2 berkeley-2 berkeley-2 berkeley-2 berkeley-2 berkeley-2 berkeley-2 berkeley-2 berkeley-2 buffalo buffalo buffalo buffalo buffalo buffalo buffalo buffalo isi-unige isi-unige isi-unige isi-unige isi-unige isi-unige u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow
USA USA USA USA USA USA USA USA USA USA USA USA USA USA USA USA USA USA USA USA USA USA USA USA USA USA USA Switzerland Switzerland Switzerland Switzerland Switzerland Switzerland UK UK UK UK UK UK UK UK UK UK UK UK UK UK UK UK UK UK UK UK UK UK UK automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic manual manual manual manual automatic manual automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic no yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes no no no no yes no no no no no no no no yes yes no yes yes yes yes yes yes yes yes no no yes yes yes yes yes yes yes no no yes u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow u.glasgow hagen hagen hagen hagen hagen hagen hagen hagen hagen hagen hildesheim hildesheim hildesheim hildesheim hildesheim hildesheim hildesheim hildesheim hildesheim hildesheim hildesheim hildesheim hildesheim hildesheim depok depok depok depok depok depok jaen jaen jaen jaen jaen jaen jaen jaen xldb
UK UK UK UK UK UK UK UK UK UK UK UK UK UK UK UK UK Germany Germany Germany Germany Germany Germany Germany Germany Germany Germany Germany Germany Germany Germany Germany Germany Germany Germany Germany Germany Germany Germany Germany Germany Indonesia Indonesia Indonesia Indonesia Indonesia Indonesia Spain Spain Spain Spain Spain Spain Spain Spain Portugal automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic manual no yes yes yes no yes no yes no yes no no yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes no no yes no no no yes yes yes yes yes yes yes no yes no no yes no no no no no no no no yes xldb xldb xldb xldb xldb xldb xldb xldb unine unine unine unine unine unine unine unine unine unine unine unine unine unine unine unine unine unine unine unine unine unine unine unine unine unine unine unine unine unine unine unine unine dsv-stockholm dsv-stockholm dsv-stockholm dsv-stockholm u.surugadai u.surugadai u.surugadai u.surugadai u.surugadai automatic automatic automatic manual automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic automatic no yes no yes no no no no yes yes no yes yes no yes no no yes no no yes no no yes no no yes yes yes yes yes yes yes yes yes yes yes no yes no no yes yes no no no no yes yes no Overview Graphs
A.15 CLEF 2005 − Top 4 participants of Ad−Hoc Multilingual Multi−8 2−years−on − Interpolated Recall vs Average Precision 100% 40% 30% 20% 10% 0%
5 0% 0% 10% 20% 30%
30 100 200 Retrieved Documents (logarithmic scale) 500
1000 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180
AH-2-years-on CLEF 2005 − Top 4 participants of Ad−Hoc Multilingual Multi−8 2−years−on − Interpolated Recall vs Average Precision 100% 40% 30% 20% 10% 0%
5 0% 0% 10% 20% 30%
30 100 200 Retrieved Documents (logarithmic scale) 500
1000 ) n o is 0.5 i c e r p e g rae 0 v a ( e c n e re−0.5 iff D
−1 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200
Average precision (non-interpolated) for all relevant documents (averaged over queries) R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 41.73 57.43 66.92 2.59 64.51 61.50 88.87 3.13 25.02 15.38 54.33 16.60 28.66
Topic 139 Topic 140 Topic 141 Topic 142 Topic 143 Topic 144 Topic 145 Topic 146 Topic 147 Topic 148 Topic 149 Topic 150 DS-Monolingual-EN 10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 500 1000 CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 126 to 138) 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
CLEF 2005 DS−Monolingual−EN − Retrieved documents vs Precision
CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 139 to 150) Average precision (non-interpolated) for all relevant documents (averaged over queries) R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 48.01 39.61 66.44 4.89 57.02 21.95 86.82 0.98 32.19 29.85 49.24 6.63 6.05
Topic 139 Topic 140 Topic 141 Topic 142 Topic 143 Topic 144 Topic 145 Topic 146 Topic 147 Topic 148 Topic 149 Topic 150 DS-Monolingual-EN 10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 500 1000 CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 126 to 138) 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
CLEF 2005 DS−Monolingual−EN − Retrieved documents vs Precision
CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 139 to 150) Average precision (non-interpolated) for all relevant documents (averaged over queries) R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 46.90 38.96 73.46 8.14 57.04 59.05 84.02 0.57 28.42 48.13 58.58 8.63 26.79
Topic 139 Topic 140 Topic 141 Topic 142 Topic 143 Topic 144 Topic 145 Topic 146 Topic 147 Topic 148 Topic 149 Topic 150 DS-Monolingual-EN 10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 500 1000 CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 126 to 138) 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
CLEF 2005 DS−Monolingual−EN − Retrieved documents vs Precision
CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 139 to 150) 100 Average precision (non-interpolated) for all relevant documents (averaged over queries) 32.51 R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 0.43 25.64 68.90 7.77 47.10 41.04 0.00 1.99 9.99 16.32 58.48 22.79 5.16
Topic 139 Topic 140 Topic 141 Topic 142 Topic 143 Topic 144 Topic 145 Topic 146 Topic 147 Topic 148 Topic 149 Topic 150
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
CLEF 2005 DS−Monolingual−EN − Retrieved documents vs Precision 10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 500 1000 CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 126 to 138) irit DS-Monolingual-EN −1 126 127 128 129 130 131 132
CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 139 to 150) irit DS-Monolingual-EN 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 36.09 34.66 72.64 5.62 47.32 38.59 0.00 7.98 44.46 11.81 12.06 21.91 4.44
Topic 139 Topic 140 Topic 141 Topic 142 Topic 143 Topic 144 Topic 145 Topic 146 Topic 147 Topic 148 Topic 149 Topic 150
10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 500 1000
CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 126 to 138)
CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 139 to 150) irit DS-Monolingual-EN 100 0.34 Average precision (non-interpolated) for all relevant documents (averaged over queries) 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
CLEF 2005 DS−Monolingual−EN − Retrieved documents vs Precision ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D
R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 500 1000 CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 126 to 138)
132
Topic Number 127 128 129 130 131 133 134 135 136 137 138 CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 139 to 150) 144 145
Topic Number 140 141 142 143 146 147 148 149 150 irit DS-Monolingual-EN 100 0.52 Average precision (non-interpolated) for all relevant documents (averaged over queries) 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
CLEF 2005 DS−Monolingual−EN − Retrieved documents vs Precision ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D
R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 500 1000 CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 126 to 138)
132
Topic Number 127 128 129 130 131 133 134 135 136 137 138 CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 139 to 150) 144 145
Topic Number 140 141 142 143 146 147 148 149 150 irit DS-Monolingual-EN 100 0.29 Average precision (non-interpolated) for all relevant documents (averaged over queries) 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
CLEF 2005 DS−Monolingual−EN − Retrieved documents vs Precision ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 500 1000 CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 126 to 138)
132
Topic Number 127 128 129 130 131 133 134 135 136 137 138 CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 139 to 150) 144 145
Topic Number 140 141 142 143 146 147 148 149 150 Average precision (non-interpolated) for all relevant documents (averaged over queries) R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 4.08 41.37 0.16 0.17 37.38 2.20 76.68 25.19 4.28 9.86 20.55 0.22 12.56
Topic 139 Topic 140 Topic 141 Topic 142 Topic 143 Topic 144 Topic 145 Topic 146 Topic 147 Topic 148 Topic 149 Topic 150 11.55 59.86 84.12 65.22 32.07 33.83 3.96 75.00 42.95 9.78 69.91 2.72 ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D
0%0% 100% 90% 80% 70% cirGIRTen2en 10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 500 1000 CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 126 to 138) 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Precision at DCL (%) CLEF 2005 DS−Monolingual−EN − Retrieved documents vs Precision 132
CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 139 to 150) glaentd Average precision (non-interpolated) for all relevant documents (averaged over queries) 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
CLEF 2005 DS−Monolingual−EN − Retrieved documents vs Precision ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 500 1000 CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 126 to 138)
132
Topic Number 127 128 129 130 131 133 134 135 136 137 CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 139 to 150) 144 145
Topic Number 140 141 142 143 146 147 148 149 150 glaentdnqe Average precision (non-interpolated) for all relevant documents (averaged over queries) 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
CLEF 2005 DS−Monolingual−EN − Retrieved documents vs Precision 0 10 20 30 40 50 60 70 80 90 100 R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 23.13 52.29 81.44 0.66 62.27 60.14 91.44 11.76 45.61 6.67 49.54 23.96 26.93
Topic 139 Topic 140 Topic 141 Topic 142 Topic 143 Topic 144 Topic 145 Topic 146 Topic 147 Topic 148 Topic 149 Topic 150 18.13 55.89 68.76 28.33 36.89 31.40 50.67 90.08 55.61 11.93 4.48 54.77 ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D
0%0% 100% 90% 80% 70% 10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 500 1000
CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 126 to 138)
CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 139 to 150) unine UniNEgen1 Average precision (non-interpolated) for all relevant documents (averaged over queries) 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Precision at DCL (%) CLEF 2005 DS−Monolingual−EN − Retrieved documents vs Precision 0 10 20 30 40 50 60 70 80 90 100 R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 47.24 50.01 77.92 0.56 62.22 72.37 93.40 1.39 49.03 16.88 55.32 74.42 30.03
Topic 139 Topic 140 Topic 141 Topic 142 Topic 143 Topic 144 Topic 145 Topic 146 Topic 147 Topic 148 Topic 149 Topic 150 34.79 48.86 82.42 60.35 39.89 56.25 62.34 92.89 67.07 14.69 4.86 71.08 ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D
0%0% 100% 90% 80% 70% 10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 1000
CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 126 to 138)
CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 139 to 150) unine UniNEgen2 Average precision (non-interpolated) for all relevant documents (averaged over queries) 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Precision at DCL (%) CLEF 2005 DS−Monolingual−EN − Retrieved documents vs Precision 0 10 20 30 40 50 60 70 80 90 100 R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 47.29 49.81 76.96 0.55 66.75 76.00 93.12 0.36 48.95 21.28 48.89 73.92 32.01
Topic 139 Topic 140 Topic 141 Topic 142 Topic 143 Topic 144 Topic 145 Topic 146 Topic 147 Topic 148 Topic 149 Topic 150 34.15 49.42 77.96 62.05 39.86 52.24 63.20 92.71 64.74 13.96 5.13 69.45 ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D
0%0% 100% 90% 80% 70% 10% 0%5
10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 1000
CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 126 to 138)
CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 139 to 150) −1 139 144 145
Topic Number 140 141 142 143 146 147 148 149 150 unine UniNEgen3 Average precision (non-interpolated) for all relevant documents (averaged over queries) 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Precision at DCL (%) CLEF 2005 DS−Monolingual−EN − Retrieved documents vs Precision 0 10 20 30 40 50 60 70 80 90 100 R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 44.85 41.64 63.64 0.32 46.54 64.81 89.44 4.08 32.31 11.25 40.29 70.83 31.26
Topic 139 Topic 140 Topic 141 Topic 142 Topic 143 Topic 144 Topic 145 Topic 146 Topic 147 Topic 148 Topic 149 Topic 150 19.14 44.95 79.91 53.97 38.09 48.60 48.21 89.94 60.35 14.74 6.05 59.62 ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D
0%0% 100% 90% 80% 70% 30% 20% 10% 100% 90% 80% 70% 60% 40% 30% 20% 10% 0%5
10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 1000
CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 126 to 138)
CLEF 2005 DS−Monolingual−EN − Comparison to median by topic (topics 139 to 150) −1 139 144 145
Topic Number 140 141 142 143 146 147 148 149 150 berkeley Average precision (non-interpolated) for all relevant documents (averaged over queries) 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Precision at DCL (%) CLEF 2005 DS−Monolingual−RU − Retrieved documents vs Precision
CLEF 2005 DS−Monolingual−RU − Comparison to median by topic (topics 139 to 150) −1 139 144 145
Topic Number 140 141 142 143 146 147 148 149 150 0 10 20 30 40 50 60 70 80 90 100 R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 37.19 79.28 24.42 4.65 12.27 3.12 12.50 17.91 8.25 38.78 34.87 20.22 52.69 ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D
0%0% 100% 90% 80% 70% 30% 20% 10% 100% 90% 80% 70% 60% 40% 30% 20% 10% 0%5 10 15 20 30 100 200
Retrieved Documents (logarithmic scale) CLEF 2005 DS−Monolingual−RU − Comparison to median by topic (topics 126 to 138)
Average precision (non-interpolated) for all relevant documents (averaged over queries) R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 54.37 Topic 127 90.36 Topic 128 27.83 Topic 129 5.96 Topic 130 8.01 Topic 131 100.00 Topic 132 12.50 Topic 133 17.16 Topic 134 10.24 Topic 135 14.19 Topic 136 58.50 Topic 137 28.70 Topic 138 47.27
Topic 139 39.66 Topic 140 2.92 Topic 141 5.39 Topic 142 37.31 Topic 143 100.00 Topic 144 0.56 Topic 145 53.35 Topic 146 0.40 Topic 147 0.00 Topic 148 24.05 Topic 149 20.66 Topic 150 0.00
1 ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D
0%0% 100% 90% 80% 70% 30% 20% 10% 100% 90% 80% 70% 60% 40% 30% 20% 10% 0%5 Overall statistics for 25 queries : Total number of documents over all queries Retrieved 25,000 Relevant 831 Relevant retrieved 726 10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 1000 CLEF 2005 DS−Monolingual−RU − Comparison to median by topic (topics 126 to 138) 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Precision at DCL (%) CLEF 2005 DS−Monolingual−RU − Retrieved documents vs Precision
CLEF 2005 DS−Monolingual−RU − Comparison to median by topic (topics 139 to 150) −1 139 144 145
Topic Number 140 141 142 143 146 147 148 149 150 Average precision (non-interpolated) for all relevant documents (averaged over queries) R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 20.83 87.89 19.73 0.95 2.60 50.89 3.12 11.52 9.59 53.40 43.24 18.55 33.37
Topic 139 Topic 140 Topic 141 Topic 142 Topic 143 Topic 144 Topic 145 Topic 146 Topic 147 Topic 148 Topic 149 Topic 150 42.23 3.42 28.47 24.39 45.00 0.91 20.58 0.91 12.50 35.87 17.34 0.00 ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D
1 ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D
0%0% 100% 90% 80% 70% 30% 20% 10% 100% 90% 80% 70% 60% 40% 30% 20% 10% 0%5 DS-Monolingual-RU 10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 1000 CLEF 2005 DS−Monolingual−RU − Comparison to median by topic (topics 126 to 138) 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Precision at DCL (%) CLEF 2005 DS−Monolingual−RU − Retrieved documents vs Precision −1 126 127 128 129 130 131 132
CLEF 2005 DS−Monolingual−RU − Comparison to median by topic (topics 139 to 150) −1 139 144 145
Topic Number 140 141 142 143 146 147 148 149 150 Average precision (non-interpolated) for all relevant documents (averaged over queries) R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 0.75 10.38 0.59 0.00 3.70 0.00 0.00 0.00 0.67 0.01 4.84 0.00 2.41
Topic 139 Topic 140 Topic 141 Topic 142 Topic 143 Topic 144 Topic 145 Topic 146 Topic 147 Topic 148 Topic 149 Topic 150 7.42 0.72 0.00 6.80 0.00 1.94 12.02 0.00 0.00 0.00 0.00 0.00 ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D
1 ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D
0%0% 100% 90% 80% 70% 30% 20% 10% 100% 90% 80% 70% 60% 40% 30% 20% 10% 0%5 10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 500 1000 CLEF 2005 DS−Monolingual−RU − Comparison to median by topic (topics 126 to 138) 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Precision at DCL (%) CLEF 2005 DS−Monolingual−RU − Retrieved documents vs Precision −1 126 127 128 129 130 131 132
CLEF 2005 DS−Monolingual−RU − Comparison to median by topic (topics 139 to 150) −1 139 144 145
Topic Number 140 141 142 143 146 147 148 149 150 DS-Monolingual-RU Average precision (non-interpolated) for all relevant documents (averaged over queries) 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Precision at DCL (%) CLEF 2005 DS−Monolingual−RU − Retrieved documents vs Precision 0 10 20 30 40 50 60 70 80 90 100 R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 2.63 64.23 20.44 0.31 0.01 0.10 0.89 0.16 0.18 12.92 0.50 0.43 2.60
Topic 139 Topic 140 Topic 141 Topic 142 Topic 143 Topic 144 Topic 145 Topic 146 Topic 147 Topic 148 Topic 149 Topic 150 2.54 5.03 3.74 10.76 8.33 0.43 0.87 0.00 0.00 0.45 0.77 0.00 ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D
1 ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D
0%0% 100% 90% 80% 70% 30% 20% 10% 100% 90% 80% 70% 60% 40% 30% 20% 10% 0%5 glarutdqe glarutdqe glarutdqe glarutdqe 10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 500 1000
CLEF 2005 DS−Monolingual−RU − Comparison to median by topic (topics 126 to 138)
CLEF 2005 DS−Monolingual−RU − Comparison to median by topic (topics 139 to 150) −1 139 144 145
Topic Number 140 141 142 143 146 147 148 149 150 UniNEgru1 CLEF 2005 DS−Monolingual−RU − Interpolated Recall vs Average Precision Average precision (non-interpolated) for all relevant documents (averaged over queries) 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Precision at DCL (%) CLEF 2005 DS−Monolingual−RU − Retrieved documents vs Precision 0 10 20 30 40 50 60 70 80 90 100 R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 19.24 84.33 67.74 3.07 0.20 0.00 2.27 1.01 38.25 15.08 65.76 14.02 52.03
Topic 139 Topic 140 Topic 141 Topic 142 Topic 143 Topic 144 Topic 145 Topic 146 Topic 147 Topic 148 Topic 149 Topic 150 61.60 4.22 2.66 76.26 7.20 1.65 63.02 4.35 0.00 37.87 1.01 0.00 ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D
1 ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D
0%0% 100% 90% 80% 70% 30% 20% 10% 100% 90% 80% 70% 60% n o ii s rce 50% P − R 40% 30% 20% 10% 0%5
10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 1000
CLEF 2005 DS−Monolingual−RU − Comparison to median by topic (topics 126 to 138) −1 126 127 128 129 130 131 132
133 134 135 136 137 138
CLEF 2005 DS−Monolingual−RU − Comparison to median by topic (topics 139 to 150) −1 139 144 145
Topic Number 140 141 142 143 146 147 148 149 150 UniNEgru2 CLEF 2005 DS−Monolingual−RU − Interpolated Recall vs Average Precision Average precision (non-interpolated) for all relevant documents (averaged over queries) 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Precision at DCL (%) CLEF 2005 DS−Monolingual−RU − Retrieved documents vs Precision 0 10 20 30 40 50 60 70 80 90 100 R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 41.73 81.57 52.86 4.96 0.26 0.00 6.25 11.11 15.80 27.68 69.05 17.38 53.13
Topic 139 58.44 Topic 140 4.66 Topic 141 5.33 Topic 142 43.43 Topic 143 100.00 Topic 144 4.02 Topic 145 65.53 Topic 146 3.23 Topic 147 0.10 Topic 148 25.79 Topic 149 1.14 Topic 150 0.00 ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D
1 ) n isco 0.5 i e r p e g rvea 0 a ( e c n ree−0.5 iff D
0%0% 100% 90% 80% 70% 30% 20% 10% 100% 90% 80% 70% 60% n o ii s rce 50% P − R 40% 30% 20% 10% 0%5
10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 1000
CLEF 2005 DS−Monolingual−RU − Comparison to median by topic (topics 126 to 138) −1 126 127 128 129 130 131 132
133 134 135 136 137 138
CLEF 2005 DS−Monolingual−RU − Comparison to median by topic (topics 139 to 150) −1 139 144 145
Topic Number 140 141 142 143 146 147 148 149 150 Average precision (non-interpolated) for all relevant documents (averaged over queries) 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Precision at DCL (%) CLEF 2005 DS−Monolingual−RU − Retrieved documents vs Precision 0 10 20 30 40 50 60 70 80 90 100 5 docs 10 docs 15 docs 20 docs 30 docs 100 docs 200 docs 500 docs 1000 docs R-Precision (precision after R document retrieved, where R = Relevant retrieved) Precision averages (%) for individual queries Topic 126 Topic 127 Topic 128 Topic 129 Topic 130 Topic 131 Topic 132 Topic 133 Topic 134 Topic 135 Topic 136 Topic 137 Topic 138 1 90% 80% 70% 1000
CLEF 2005 DS−Monolingual−RU − Comparison to median by topic (topics 126 to 138) 140