Introduction 3
Results for CLEF 2006 Ad-Hoc Tracks List of Submitted Experiments 7 daedalus daedalus daedalus hummingbird hummingbird hummingbird rsi-jhu rsi-jhu unine unine unine alicante alicante alicante alicante daedalus daedalus daedalus erss-toulouse erss-toulouse erss-toulouse hummingbird hummingbird hummingbird indianstat indianstat indianstat indianstat rim rim rim rim rsi-jhu rsi-jhu rsi-jhu unine unine unine alicante alicante alicante alicante daedalus daedalus daedalus hummingbird hummingbird hummingbird mokk mokk rsi-jhu rsi-jhu unine unine unine alicante
AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC MANUAL MANUAL MANUAL AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes no alicante alicante alicante colesir colesir daedalus daedalus daedalus hummingbird hummingbird hummingbird nilc-usp nilc-usp nilc-usp pucrs rsi-jhu rsi-jhu rsi-jhu saocarlos saocarlos saocarlos u.buffalo u.buffalo u.buffalo u.buffalo ufrgs-usp ufrgs-usp ufrgs-usp ufrgs-usp unine unine unine xldb xldb xldb xldb daedalus daedalus celi celi celi celi celi celi celi celi celi celi celi celi depok depok depok depok depok depok
AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC MANUAL MANUAL MANUAL MANUAL AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC MANUAL AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC no no yes yes no no yes no yes no no yes no no yes no yes no no yes no yes no no no yes no no no yes no no yes no no no yes yes no no yes no no no no no no no no no yes no no no no no depok depok dsv dsv dsv dsv ltrc ltrc ltrc ltrc ltrc ltrc ltrc rsi-jhu rsi-jhu daedalus daedalus daedalus daedalus queenmary rsi-jhu rsi-jhu rsi-jhu rsi-jhu unine unine unine daedalus daedalus daedalus daedalus daedalus daedalus queenmary
AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC MANUAL AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC AUTOMATIC Track Overview Results and Graphs 13 −0.4 −0.6 −0.8 −1 1 0.8 0.6 0.4 0.2 −0.4 −0.6 −0.8 −1 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325
Ad−Hoc Monolingual Bulgarian track Top 5 Participants − Comparison to Median Mean Average Precision by Topic (Topics 351 to 735) 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375
Ad−Hoc Monolingual Bulgarian track − Box Plot of the Topics UniNEbg2 [MAP 33.14%; Pooled]
UniNEbg3 [MAP 32.98%; Pooled] 02aplmobgtd4 [MAP 31.98%; Pooled] 02aplmobgtdn4 [MAP 31.48%; Pooled] s t n e m i r e p x E
UniNEbg1 [MAP 31.29%; Pooled] bgFSbg3S [MAP 30.80%; Pooled] humBG06tde [MAP 30.47%; Pooled] bgx011bgFW4FS4 [MAP 29.57%; Pooled] humBG06t [MAP 26.12%; Pooled] 0% 10% 20% 30% 70% 80%
Ad−Hoc Monolingual Bulgarian track − Tukey T test with "top group" highlighted s t n e m i r e p x E 02aplmobgtdn4 Track Overview
Results and Graphs −0.4 −0.6 −0.8 −1 1 0.8 0.6 0.2 −0.4 −0.6 −0.8 −1 Ad−Hoc Monolingual Bulgarian track − Box Plot of the Topics UniNEbg2 [R−Prec 33.14%; Pooled] UniNEbg3 [R−Prec 33.00%; Pooled]
UniNEbg1 [R−Prec 31.76%; Pooled] s t n e m i r e p x E 02aplmobgtdn4 [R−Prec 31.02%; Pooled] 02aplmobgtd4 [R−Prec 30.95%; Pooled] bgFSbg3S [R−Prec 30.50%; Pooled] bgFSbg2S [R−Prec 30.27%; Pooled] 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
R−Precision
Ad−Hoc Monolingual Bulgarian track − Tukey T test with "top group" highlighted s t n e m i r e p x E 02aplmobgtdn4 02aplmobgtd4 0.0837 Automatic Retrieval Feedback using TD, 10 docs and 0.2840 300 terms 0%0% 10% 20% 30%
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment
Ad−HocBilingualPortuguesetrack−ComparisontoMedianMeanAveragePrecisionbyTopic(Topics326to350) Mean Average Precision Maximum Minimum First Quartile Second Quartile Third Quartile Interquartile range Mean Standard Deviation Lower Outlier Threshold Upper Outlier Threshold Mean With No Outliers Std With No Outliers Topic 301 Topic 302 Topic 303 Topic 304 Topic 305 Topic 306 Topic 307 Topic 308 Topic 309 Topic 310 Topic 311 Topic 312 Topic 313 Topic 314 Topic 315 Topic 316 Topic 317 Topic 318 Topic 319 Topic 320 Topic 321 Topic 322 Topic 323 Topic 324 Topic 325 41.38 1.36 11.47 2.79 0.51 52.11 0.00 83.89 37.25 3.99 38.39 13.87 0.72 13.23 5.87 0.00 2.87 0.09 0.13 1.11 63.31 9.11 1.33 3.95 1.01
Topic 326 Topic 327 Topic 328 Topic 329 Topic 330 Topic 331 Topic 332 Topic 333 Topic 334 Topic 335 Topic 336 Topic 337 Topic 338 Topic 339 Topic 340 Topic 341 Topic 342 Topic 343 Topic 344 Topic 345 Topic 346 Topic 347 Topic 348 Topic 349 Topic 350 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 Precision averages (%) for individual queries Ad−HocBilingualPortuguesetrack−ComparisontoMedianMeanExactR−PrecisionbyTopic(Topics301to325) Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment 15 t n e m ir e xp E10 e h ft so cp i o fT 5 o r e b m u N
Ad−HocBilingualPortuguesetrack−ComparisontoMedianMeanExactR−PrecisionbyTopic(Topics326to350) 0.0842 Automatic Retrieval Feedback using TDN, 5 docs and 0.2834 30 terms 0%0% 10% 20% 30%
Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment
Ad−HocBilingualPortuguesetrack−ComparisontoMedianMeanAveragePrecisionbyTopic(Topics326to350) Mean Average Precision Maximum Minimum First Quartile Second Quartile Third Quartile Interquartile range Mean Standard Deviation Lower Outlier Threshold Upper Outlier Threshold Mean With No Outliers Std With No Outliers Topic 301 Topic 302 Topic 303 Topic 304 Topic 305 Topic 306 Topic 307 Topic 308 Topic 309 Topic 310 Topic 311 Topic 312 Topic 313 Topic 314 Topic 315 Topic 316 Topic 317 Topic 318 Topic 319 Topic 320 Topic 321 Topic 322 Topic 323 Topic 324 Topic 325 39.60 1.94 12.52 2.54 0.52 50.21 0.00 84.79 37.16 3.91 41.40 14.01 0.46 12.68 5.23 0.00 3.06 0.06 0.14 1.86 63.61 9.92 1.80 5.95 0.99
Topic 326 Topic 327 Topic 328 Topic 329 Topic 330 Topic 331 Topic 332 Topic 333 Topic 334 Topic 335 Topic 336 Topic 337 Topic 338 Topic 339 Topic 340 Topic 341 Topic 342 Topic 343 Topic 344 Topic 345 Topic 346 Topic 347 Topic 348 Topic 349 Topic 350 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 15 t n e m ir e xp E10 e h ft so cp i o fT 5 o r e b m u N
Precision averages (%) for individual queries Ad−HocBilingualPortuguesetrack−ComparisontoMedianMeanExactR−PrecisionbyTopic(Topics301to325) Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment 0%5 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment 10 t n e irm 8 e xp E the 6 f so c iop 4 T f o r e b 2 m u N
Ad−HocBilingualPortuguesetrack−ComparisontoMedianMeanExactR−PrecisionbyTopic(Topics326to350) 0.0962 Automatic Retrieval Feedback using TDN, 10 docs and 0.2949 300 terms 0%0% 10% 20% 30%
Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision Maximum Minimum First Quartile Second Quartile Third Quartile Interquartile range Mean Standard Deviation Lower Outlier Threshold Upper Outlier Threshold Mean With No Outliers Std With No Outliers Topic 301 Topic 302 Topic 303 Topic 304 Topic 305 Topic 306 Topic 307 Topic 308 Topic 309 Topic 310 Topic 311 Topic 312 Topic 313 Topic 314 Topic 315 Topic 316 Topic 317 Topic 318 Topic 319 Topic 320 Topic 321 Topic 322 Topic 323 Topic 324 Topic 325 41.18 0.38 6.67 12.75 0.78 48.61 0.28 80.49 29.63 0.94 47.05 15.67 0.14 8.76 5.08 0.00 4.38 1.03 0.15 0.97 64.55 15.47 1.42 4.33 1.04
Topic 326 Topic 327 Topic 328 Topic 329 Topic 330 Topic 331 Topic 332 Topic 333 Topic 334 Topic 335 Topic 336 Topic 337 Topic 338 Topic 339 Topic 340 Topic 341 Topic 342 Topic 343 Topic 344 Topic 345 Topic 346 Topic 347 Topic 348 Topic 349 Topic 350 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 20 t n e m irpe15 x E e h ftso10 cp i o T f o re 5 b m u N
Precision averages (%) for individual queries Ad−HocBilingualPortuguesetrack−ComparisontoMedianMeanExactR−PrecisionbyTopic(Topics301to325) Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment 0%5 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment 15 t n e m ir e xp E10 e h tf so cp i o fT 5 o r e b m u N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Ad−HocBilingualPortuguesetrack−ComparisontoMedianMeanExactR−PrecisionbyTopic(Topics326to350) 0.0994 Automatic Retrieval Feedback using TDN, 5 docs and 0.2967 30 terms 0%0% 10% 20% 30%
Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision Maximum Minimum First Quartile Second Quartile Third Quartile Interquartile range Mean Standard Deviation Lower Outlier Threshold Upper Outlier Threshold Mean With No Outliers Std With No Outliers Topic 301 Topic 302 Topic 303 Topic 304 Topic 305 Topic 306 Topic 307 Topic 308 Topic 309 Topic 310 Topic 311 Topic 312 Topic 313 Topic 314 Topic 315 Topic 316 Topic 317 Topic 318 Topic 319 Topic 320 Topic 321 Topic 322 Topic 323 Topic 324 Topic 325 38.48 0.36 6.62 11.21 0.81 51.41 0.34 80.80 30.99 1.04 46.66 14.91 0.15 8.41 5.72 0.00 5.24 1.54 0.16 1.08 64.47 15.88 1.41 5.70 1.14
Topic 326 Topic 327 Topic 328 Topic 329 Topic 330 Topic 331 Topic 332 Topic 333 Topic 334 Topic 335 Topic 336 Topic 337 Topic 338 Topic 339 Topic 340 Topic 341 Topic 342 Topic 343 Topic 344 Topic 345 Topic 346 Topic 347 Topic 348 Topic 349 Topic 350 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 Precision averages (%) for individual queries Ad−HocBilingualPortuguesetrack−ComparisontoMedianMeanExactR−PrecisionbyTopic(Topics301to325) Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment 60% 40% 30% 20% 10% 0%5 Exact R-Precision Maximum Minimum First Quartile Second Quartile Third Quartile Interquartile range Mean Standard Deviation Lower Outlier Threshold Upper Outlier Threshold Mean With No Outliers Std With No Outliers Topic 301 Topic 302 Topic 303 Topic 304 Topic 305 Topic 306 Topic 307 Topic 308 Topic 309 Topic 310 Topic 311 Topic 312 Topic 313 Topic 314 Topic 315 Topic 316 Topic 317 Topic 318 Topic 319 Topic 320 Topic 321 Topic 322 Topic 323 Topic 324 Topic 325 35.71 2.50 10.00 15.52 3.17 54.39 0.00 76.67 34.88 3.82 46.41 19.23 2.33 15.38 9.76 0.00 12.70 0.00 2.70 3.45 65.00 20.00 7.14 11.82 4.41
Topic 326 Topic 327 Topic 328 Topic 329 Topic 330 Topic 331 Topic 332 Topic 333 Topic 334 Topic 335 Topic 336 Topic 337 Topic 338 Topic 339 Topic 340 Topic 341 Topic 342 Topic 343 Topic 344 Topic 345 Topic 346 Topic 347 Topic 348 Topic 349 Topic 350 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment 15 t n e m ir e xp E10 e h tf so cp i o fT 5 o r e b m u N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
unine Priority
Query Construction 50,000 Source Language 2,677 Topic Fields 2,289 Pooled 2 AUTOMATIC English title, description false 0.2130 data fusion of DFR, Okapi & LM (QT: promt & free) 0.3917 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 Precision averages (%) for individual queries 0%0% 10% 20% 30%
Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment 6 t n ire5 m e xp E4 e h ftcspo3 i o f 2 T o r e b m1 u N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Docs Cutoff Levels Precision at DCL (%) 5 docs 55.60 10 docs 51.60 15 docs 48.80 20 docs 46.10 30 docs 42.60 100 docs 26.62 200 docs 16.89 500 docs 8.12 1000 docs 4.58 R-Precision (precision after R document retrieved, where R = Relevant retrieved) 38.77 Exact R-Precision Maximum Minimum First Quartile Second Quartile Third Quartile Interquartile range Mean Standard Deviation Lower Outlier Threshold Upper Outlier Threshold Mean With No Outliers Std With No Outliers Topic 301 Topic 302 Topic 303 Topic 304 Topic 305 Topic 306 Topic 307 Topic 308 Topic 309 Topic 310 Topic 311 Topic 312 Topic 313 Topic 314 Topic 315 Topic 316 Topic 317 Topic 318 Topic 319 Topic 320 Topic 321 Topic 322 Topic 323 Topic 324 Topic 325 60.71 51.25 26.00 29.31 0.00 52.63 0.00 73.33 25.58 67.18 51.38 0.00 39.53 11.54 15.85 40.23 55.56 9.09 64.86 0.00 65.00 14.29 12.50 56.36 8.82
Topic 326 Topic 327 Topic 328 Topic 329 Topic 330 Topic 331 Topic 332 Topic 333 Topic 334 Topic 335 Topic 336 Topic 337 Topic 338 Topic 339 Topic 340 Topic 341 Topic 342 Topic 343 Topic 344 Topic 345 Topic 346 Topic 347 Topic 348 Topic 349 Topic 350 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 100% 90% 80% 70% 60% 40% 30% 20% 10% 0%5 unine Precision averages (%) for individual queries Ad−HocBilingualPortuguesetrack−ComparisontoMedianMeanExactR−PrecisionbyTopic(Topics301to325)
Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment 8 tn7 e m ire6 xp E ftscope45 h i o3 T f o r2 e b m u1 N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
unine 0.2077 data fusion of DFR & LM (QT: promt, free & online) 0.4000 0%0% 10% 20% 30% 70% 80%
Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment 6 t n ire5 m e xp E4 e h ftscop3 i o f 2 T o r e b m1 u N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Precision averages (%) for individual queries 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8
Ad−HocBilingualPortuguesetrack−ComparisontoMedianMeanAveragePrecisionbyTopic(Topics326to350) Docs Cutoff Levels Precision at DCL (%) 5 docs 56.00 10 docs 52.80 15 docs 49.60 20 docs 47.60 30 docs 43.20 100 docs 26.56 200 docs 16.95 500 docs 8.30 1000 docs 4.63 R-Precision (precision after R document retrieved, where R = Relevant retrieved) 40.94 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 100% 90% 80% 70% 60% 40% 30% 20% 10% 0%5 unine Precision averages (%) for individual queries Ad−HocBilingualPortuguesetrack−ComparisontoMedianMeanExactR−PrecisionbyTopic(Topics301to325)
Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment 6 t n ire5 m e xp E4 e h ftscop3 i o f 2 T o r e b m1 u N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
unine 0.2288 data fusion of DFR, LM & Okapi (QT: promt & online) 0.4027 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 Precision averages (%) for individual queries 0%0% 10% 20% 30% 70% 80%
Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment 8 tn7 e m ire6 xp E tfsceop54 h i o3 T f o r2 e b m u1 N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Docs Cutoff Levels Precision at DCL (%) 5 docs 56.00 10 docs 51.60 15 docs 49.20 20 docs 46.50 30 docs 42.27 100 docs 26.02 200 docs 16.69 500 docs 8.22 1000 docs 4.63 R-Precision (precision after R document retrieved, where R = Relevant retrieved) 39.30 Exact R-Precision Maximum Minimum First Quartile Second Quartile Third Quartile Interquartile range Mean Standard Deviation Lower Outlier Threshold Upper Outlier Threshold Mean With No Outliers Std With No Outliers Topic 301 Topic 302 Topic 303 Topic 304 Topic 305 Topic 306 Topic 307 Topic 308 Topic 309 Topic 310 Topic 311 Topic 312 Topic 313 Topic 314 Topic 315 Topic 316 Topic 317 Topic 318 Topic 319 Topic 320 Topic 321 Topic 322 Topic 323 Topic 324 Topic 325 64.29 53.75 22.00 8.62 0.00 52.63 24.24 71.67 32.56 69.47 45.86 3.85 49.30 11.54 47.56 41.35 39.68 36.36 64.86 0.00 65.00 17.14 7.14 54.55 10.29
Topic 326 Topic 327 Topic 328 Topic 329 Topic 330 Topic 331 Topic 332 Topic 333 Topic 334 Topic 335 Topic 336 Topic 337 Topic 338 Topic 339 Topic 340 Topic 341 Topic 342 Topic 343 Topic 344 Topic 345 Topic 346 Topic 347 Topic 348 Topic 349 Topic 350 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 100% 90% 80% 70% 60% 40% 30% 20% 10% 0%5 unine Precision averages (%) for individual queries Ad−HocBilingualPortuguesetrack−ComparisontoMedianMeanExactR−PrecisionbyTopic(Topics301to325)
Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment 6 t n ire5 m e xp E4 e h tfscop3 i o f 2 T o r e b m1 u N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
xldb Priority
Query Construction 43,409 Source Language 2,677 Topic Fields 1,303 Pooled 1 AUTOMATIC English title true 0.0364 Automatic query, 32 QE terms, top20k docs, relaxed 0.2362 QC 0%0% 10% 20% 30% 70% 80%
Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Mean Average Precision Maximum Minimum First Quartile Second Quartile Third Quartile Interquartile range Mean Standard Deviation Lower Outlier Threshold Upper Outlier Threshold Mean With No Outliers Std With No Outliers Topic 301 Topic 302 Topic 303 Topic 304 Topic 305 Topic 306 Topic 307 Topic 308 Topic 309 Topic 310 Topic 311 Topic 312 Topic 313 Topic 314 Topic 315 Topic 316 Topic 317 Topic 318 Topic 319 Topic 320 Topic 321 Topic 322 Topic 323 Topic 324 Topic 325 27.34 0.26 4.25 0.00 0.00 14.81 0.01 66.82 1.17 0.65 49.92 7.18 0.93 0.02 47.17 0.00 15.12 0.02 0.15 0.03 19.39 20.63 1.23 3.15 0.98
Topic 326 Topic 327 Topic 328 Topic 329 Topic 330 Topic 331 Topic 332 Topic 333 Topic 334 Topic 335 Topic 336 Topic 337 Topic 338 Topic 339 Topic 340 Topic 341 Topic 342 Topic 343 Topic 344 Topic 345 Topic 346 Topic 347 Topic 348 Topic 349 Topic 350 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 25 t n e irm20 e xp E the15 f so c iop10 T f o r e b 5 m u N
xldb Precision averages (%) for individual queries Ad−HocBilingualPortuguesetrack−ComparisontoMedianMeanExactR−PrecisionbyTopic(Topics301to325) Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment 60% 40% 30% 20% 10% 0%5 Exact R-Precision Maximum Minimum First Quartile Second Quartile Third Quartile Interquartile range Mean Standard Deviation Lower Outlier Threshold Upper Outlier Threshold Mean With No Outliers Std With No Outliers Topic 301 Topic 302 Topic 303 Topic 304 Topic 305 Topic 306 Topic 307 Topic 308 Topic 309 Topic 310 Topic 311 Topic 312 Topic 313 Topic 314 Topic 315 Topic 316 Topic 317 Topic 318 Topic 319 Topic 320 Topic 321 Topic 322 Topic 323 Topic 324 Topic 325 32.14 2.50 12.00 0.00 0.00 24.56 0.00 58.33 4.65 2.29 50.83 11.54 0.93 0.00 51.22 0.00 26.98 0.00 2.70 0.00 32.50 28.57 7.14 5.45 5.88
Topic 326 Topic 327 Topic 328 Topic 329 Topic 330 Topic 331 Topic 332 Topic 333 Topic 334 Topic 335 Topic 336 Topic 337 Topic 338 Topic 339 Topic 340 Topic 341 Topic 342 Topic 343 Topic 344 Topic 345 Topic 346 Topic 347 Topic 348 Topic 349 Topic 350 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment 15 t n e m ir e xp E10 e h tf so cp i o fT 5 o r e b m u N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
xldb Overall statistics for 50 queries : Total number of documents over all queries Retrieved Relevant Relevant retrieved Priority
Query Construction 41,952 Source Language 2,677 Topic Fields 1,236 Pooled 4 AUTOMATIC English title false 0.0245 Automatic query, 16 QE terms, top10k docs, relaxed 0.2357 QC 0%0% 10% 20% 30% 70% 80% Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment Precision averages (%) for individual queries 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 20 t n e m irpe15 x E e h ftso10 cp i o T f o re 5 b m u N 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 100% 90% 80% 70% 60% 40% 30% 20% 10% 0%5
Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 20 t n e m irpe15 x E e h ftso10 cp i o T f o re 5 b m u N Precision averages (%) for individual queries
xldb Priority
Query Construction 42,401 Source Language 2,677 Topic Fields 1,254 Pooled 3 AUTOMATIC English title false 0.0300 Automatic query, 32 QE terms, top10k docs, relaxed 0.2439 QC 100% 90% 80% 70% 30% 20% 10% 0%0% 10% 20% 30% 40% 50% 60%
Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment Precision averages (%) for individual queries 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 25 t n e irm20 e xp E the15 f so c iop10 T f o r e b 5 m u N 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
10 15 20 30 100 200
Retrieved Documents (logarithmic scale) 100% 90% 80% 70% 60% 40% 30% 20% 10% 0%5
Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 20 t n e m irpe15 x E e h ftso10 cp i o T f o re 5 b m u N Precision averages (%) for individual queries
xldb Priority
Query Construction 42,790 Source Language 2,677 Topic Fields 1,275 Pooled 2 AUTOMATIC English title false 0.0377 Automatic query, 16 QE terms, top20k docs, relaxed 0.2434 QC 100% 90% 80% 70% 30% 20% 10% 0%0% 10% 20% 30% 40% 50% 60%
Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment Precision averages (%) for individual queries 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 1 0.8 0.6 0.4 e 0.2 c reen 0 ifD−0.2 −0.4 −0.6 −0.8 20 t n e m irpe15 x E e h ftso10 cp i o T f o re 5 b m u N 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
90% 80% 70% 60% 40%
Ad−Hoc Bilingual Portuguese track − Box plot of the Topics of the Experiment
Ad−Hoc Bilingual Portuguese track − Distribution of the Topics of the Experiment Exact R-Precision Maximum Minimum First Quartile Second Quartile Third Quartile Interquartile range Mean Standard Deviation Lower Outlier Threshold Upper Outlier Threshold Mean With No Outliers Std With No Outliers