Appendix – Results for 2002 Campaign The following pages contain the results for all valid experiments for the multilingual, bilingual, monolingual, Amaryllis and GIRT tasks that were officially submitted to the organizers for the CLEF 2002 campaign. Results for the interactive track are presented elsewhere. The first four pages give a listing of all runs and their characteristics: Institution: the name of the participant’s organization responsible for the run Country: origin of participant’s organization Run Tag: unique identifier for every experiment. Can be used to match individual result pages with this list. Task: which track/task the experiment is part of. Topic language: language of the topics used to create the experiment (ISO identifiers for language). Topic fields: identifies the parts of the topics used for query creation (combination of T=title, D=description and N=narrative). Run Type: type of experiment (automatic/manual). Pool: specifies if experiment was used for relevance assessment pooling. The following eleven pages compare the top entries for every track/task. Shown are the recall/precision curves for at most five groups. Chosen are the best entries by the respective groups, with the topic fields fixed to title+description (mandatory experiment for every participant) and only automatic experiments used. For the multilingual and bilingual experiments, the top entries regardless of topic language are presented. The remainder of the pages is made up by the individual results for every official experiment. Each experiment is presented on one page, with the following details given: 1. A table providing the following information: • Average precision figures for every individual query. This allows comparison of system performance for single queries, which is important since variation of performance across queries is often very high and can be significant. • Overall statistics, giving: - the total number of documents retrieved by the system - the total number of overall relevant documents in the collection, and - the total number of relevant documents actually found by the system - interpolated precision averages at specific recall levels - non-interpolated average precision over all queries - precision numbers after inspecting a specific number of documents - R-precision: precision after the last relevant document was retrieved. 2. Two graphs consisting of: • a recall/precision graph, providing a plot of the precision values for various recall levels. This is the standard statistic and is the one most commonly reported in the literature. • a comparison to median performance. For each query, the difference in average precision, when compared to the median performance for the given task, is plotted. This graph gives valuable insight into which type of queries is handled well by different systems. The results page for a specific experiment can be most quickly located by using the table of contents. This table is sorted by group, task, topic language and topic fields. The individual pages are sorted by task and run tag. More information on the interpretation of the standard measures used for scoring experiments (average precision, recall levels, precision/recall graphs, etc.) can be found e.g. in: Martin Braschler, Carol Peters: CLEF Methodology and Metrics, CLEF 2001, pages 394-404 (Carol Peters, Martin Braschler, Julio Gonzalo, Michael Kluck (Eds.), Evaluation of Cross-Language Information Retrieval Systems, Second Workshop of the Cross-Language Evaluation Forum, CLEF 2001, Darmstadt, Germany, September 3-4, 2001. Revised Papers. Lecture Notes in Computer Science, Vol. 2406, Springer, 2002, ISBN 3- 540-44042-9) ,QVWLWXWLRQ &RXQWU\ 5XQ7DJ 7DVN 7RS/DQJ 7RS)OG 5XQ7\SH -XGJHG" City University United Kingdom plge02t Mono DE T Auto City University United Kingdom plge02td Mono DE TD Auto Y City University United Kingdom plsp02t Mono ES T Auto City University United Kingdom plsp02td Mono ES TD Auto Y City University United Kingdom plfn02t Mono FI T Auto Y City University United Kingdom plfn02td Mono FI TD Auto Y City University United Kingdom plfr02td Mono FR TD Auto Y City University United Kingdom plit02td Mono IT TD Auto Y City University United Kingdom pldu02t Mono NL T Auto City University United Kingdom pldu02td Mono NL TD Auto Y Clairvoyance Corp. USA cles2enw Bi-EN ES TD Auto Y Clairvoyance Corp. USA cles2ent1 Bi-EN ES TD Auto Y Clairvoyance Corp. USA cles2ent2 Bi-EN ES TD Auto Y Clairvoyance Corp. USA cles2enc1 Bi-EN ES TD Auto Clairvoyance Corp. USA clch2enw Bi-EN ZH TD Auto Y Clairvoyance Corp. USA clch2ent1 Bi-EN ZH TD Auto Y Clairvoyance Corp. USA clch2ent2 Bi-EN ZH TD Auto Y Clairvoyance Corp. USA clch2enc1 Bi-EN ZH TD Auto COLE Group Univ. La Coruna Spain coleTDlem02 Mono ES TD Auto COLE Group Univ. La Coruna Spain coleTDNlem02 Mono ES TDN Auto Y COLE Group Univ. La Coruna Spain coleTDNpds02 Mono ES TDN Auto Y COLE Group Univ. La Coruna Spain coleTDNsyn02 Mono ES TDN Auto CWI/CNLP Netherlands/USA AAbiENNLt Bi-NL EN T Auto CWI/CNLP Netherlands/USA AAbiENNLtd Bi-NL EN TD Auto Y CWI/CNLP Netherlands/USA AAmoNLt Mono NL T Auto CWI/CNLP Netherlands/USA AAmoNLtd Mono NL TD Auto Y Eurospider IT AG Switzerland EIT2GDM1 Mono DE TD Auto Y Eurospider IT AG Switzerland EIT2GDL1 Mono DE TD Auto Y Eurospider IT AG Switzerland EIT2GDB1 Mono DE TD Auto Eurospider IT AG Switzerland EIT2GNM1 Mono DE TDN Auto Eurospider IT AG Switzerland EIT2MDF3 Multi DE TD Auto Y Eurospider IT AG Switzerland EIT2MDC3 Multi DE TD Auto Eurospider IT AG Switzerland EIT2MNF3 Multi DE TDN Auto Y Eurospider IT AG Switzerland EIT2MNU1 Multi DE TDN Auto Eurospider IT AG Switzerland EAN2MDF4 Multi EN TD Auto Fondazione Ugo Bordini Italy fub02bl Mono IT TD Auto Y Fondazione Ugo Bordini Italy fub02l Mono IT TD Auto Y Fondazione Ugo Bordini Italy fub02b Mono IT TD Auto Fondazione Ugo Bordini Italy fub02 Mono IT TD Auto Hummingbird Canada humDE02 Mono DE TD Auto Y Hummingbird Canada humES02 Mono ES TD Auto Y Hummingbird Canada humFI02 Mono FI TD Auto Y Hummingbird Canada humFI02n Mono FI TDN Auto Y Hummingbird Canada humFR02 Mono FR TD Auto Y Hummingbird Canada humIT02 Mono IT TD Auto Y Hummingbird Canada humNL02 Mono NL TD Auto Y Hummingbird Canada humNL02n Mono NL TDN Auto Hummingbird Canada humSV02 Mono SV TD Auto Y Hummingbird Canada humSV02n Mono SV TDN Auto Y IMBIT Inst., Univ. Hildesheim Germany IMBIT Mono DE TDN Manual Y IMS Univ. Padova Italy PDDS2PLL3 Mono IT TD Auto Y IMS Univ. Padova Italy PDDP Mono IT TD Auto Y IMS Univ. Padova Italy PDDS2PL Mono IT TD Auto IMS Univ. Padova Italy PDDN Mono IT TD Auto IRIT, Toulouse France iritBFr2En Bi-EN FR TD Auto Y IRIT, Toulouse France RunMonoSp Mono ES TD Auto Y IRIT, Toulouse France RunMonFrench Mono FR TD Auto Y IRIT, Toulouse France RunMonoIt Mono IT TD Auto Y IRIT, Toulouse France iritMEn2All Multi EN TD Auto Y ISU, Univ. Hildesheim Germany UHi02r1 GIRT DE TD Auto Y ISU, Univ. Hildesheim Germany UHi02r2 GIRT DE TD Auto Y ITC-IRST Italy IRSTen2it1 Bi-IT EN TD Auto Y ITC-IRST Italy IRSTen2it2 Bi-IT EN TD Auto ITC-IRST Italy IRSTen2it3 Bi-IT EN TD Auto ITC-IRST Italy IRSTit1 Mono IT TD Auto Y JHU/APL USA aplbiende Bi-DE EN TD Auto Y JHU/APL USA aplbiptena Bi-EN PT TD Auto Y JHU/APL USA aplbienes Bi-ES EN TD Auto JHU/APL USA aplbiptesa Bi-ES PT TD Auto JHU/APL USA aplbiptesb Bi-ES PT TD Auto JHU/APL USA aplbienfi Bi-FI EN TD Auto Y JHU/APL USA aplbienfr Bi-FR EN TD Auto JHU/APL USA aplbienit Bi-IT EN TD Auto JHU/APL USA aplbiennl Bi-NL EN TD Auto JHU/APL USA aplbiensv Bi-SV EN TD Auto Y JHU/APL USA aplmode Mono DE TD Auto Y JHU/APL USA aplmoes Mono ES TD Auto Y JHU/APL USA aplmofi Mono FI TD Auto Y JHU/APL USA aplmofr Mono FR TD Auto Y JHU/APL USA aplmoit Mono IT TD Auto Y JHU/APL USA aplmonl Mono NL TD Auto Y JHU/APL USA aplmosv Mono SV TD Auto Y JHU/APL USA aplmuena Multi EN TD Auto Y JHU/APL USA aplmuenb Multi EN TD Auto Y Lexware Labs Ltd. Sweden lexware2002 Mono SV TDN Auto Y MediaLab BV Netherlands medialab4 Mono NL TD Auto Y MediaLab BV Netherlands medialab3 Mono NL TD Auto MediaLab BV Netherlands medialab2 Mono NL TD Auto MediaLab BV Netherlands medialab5 Mono NL TD Auto Y Middlesex Univ. United Kingdom MDXtpc Bi-EN PT T Auto Middlesex Univ. United Kingdom MDXtd Bi-EN PT TD Auto Y Middlesex Univ. United Kingdom MDXman Bi-EN PT TDN Manual Y National Taiwan Univ. Taiwan NTUmulti01 Multi EN TD Auto National Taiwan Univ. Taiwan NTUmulti02 Multi EN TD Auto Y National Taiwan Univ. Taiwan NTUmulti03 Multi EN TD Auto National Taiwan Univ. Taiwan NTUmulti04 Multi EN TD Auto Y National Taiwan Univ. Taiwan NTUmulti05 Multi EN TD Auto Océ Netherlands oce02es2enLO Bi-EN ES TD Auto Y Océ Netherlands oce02es2enBF Bi-EN ES TD Auto Océ Netherlands oce02nl2enER Bi-EN NL TD Auto Y Océ Netherlands oce02en2esLO Bi-ES EN TD Auto Océ Netherlands oce02en2esBF Bi-ES EN TD Auto Océ Netherlands oce02en2nlER Bi-NL EN TD Auto Y Océ Netherlands oce02monDEto Mono DE T Auto Océ Netherlands oce02monDE Mono DE TD Auto Y Océ Netherlands oce02monESto Mono ES T Auto Océ Netherlands oce02monES Mono ES TD Auto Y Océ Netherlands oce02monFRto Mono FR T Auto Océ Netherlands oce02monFR Mono FR TD Auto Y Océ Netherlands oce02monITto Mono IT T Auto Océ Netherlands oce02monIT Mono IT TD Auto Y Océ Netherlands oce02monNLto Mono NL T Auto Océ Netherlands oce02monNL Mono NL TD Auto Y Océ Netherlands oce02mulRRloTO Multi EN T Auto Océ Netherlands oce02mulRRlo Multi EN TD Auto Y Océ Netherlands oce02mulMSlo Multi EN TD Auto Y Océ Netherlands oce02mulRRbf Multi EN TD Auto Océ Netherlands oce02mulMSbf Multi EN TD Auto RALI U Montreal Canada run1 Multi EN TDN Auto Y RALI U Montreal Canada run2 Multi EN TDN Auto Y RALI U Montreal Canada run3 Multi EN TDN Auto SICS/Conexor Sweden/Finland sicsFRFR0 Mono FR TD Auto SICS/Conexor Sweden/Finland sicsFRFRX0 Mono FR TD Auto Y SICS/Conexor Sweden/Finland sicsITIT Mono IT TD Auto SICS/Conexor Sweden/Finland sicsITITX Mono IT TD Auto Y SICS/Conexor Sweden/Finland siteseeker Mono SV T Auto Y SICS/Conexor Sweden/Finland sicsSVSV Mono SV TD Auto Y SICS/Conexor Sweden/Finland sicsSVSVX Mono SV TD Auto Y Tagmatica France runA Mono FR DN Manual Y TLR Research USA tlren2es Bi-ES EN TD Auto Y TLR Research USA tlren2fr Bi-FR EN TD Auto TLR Research USA tlrde Mono DE TD Auto Y TLR Research USA tlres Mono ES TD Auto Y TLR Research USA tlrfr Mono FR TD Auto Y TLR Research USA tlrit Mono IT TD Auto Y TLR Research USA tlrnl Mono NL TD Auto Y TLR Research USA tlrsv Mono SV TD Auto Y TLR Research USA tlren2multi Multi EN TD Auto Y Univ. Dortmund Germany GIRTsppc GIRT DE TD Auto Y Univ. Dortmund Germany GIRTstem GIRT DE TD Auto Y Univ. Dortmund Germany MLstem Mono DE TD Auto Y Univ. Exeter United Kingdom exespgemgcntbi Bi-ES DE TD Auto Univ. Exeter United Kingdom exespengorgbi Bi-ES EN TD Auto Univ. Exeter United Kingdom exespengpiwgt Bi-ES EN TD Auto Y Univ. Exeter United Kingdom exespfrmgcntbi Bi-ES FR TD Auto Univ. Exeter United Kingdom exespitmgcntbi Bi-ES IT TD Auto Univ. Exeter United Kingdom exeitgemgcntbi Bi-IT DE TD Auto Univ. Exeter United Kingdom exeitengorgbi Bi-IT EN TD Auto Univ. Exeter United Kingdom exeitengpiwgt Bi-IT EN TD Auto Univ. Exeter United Kingdom exeitspmgbi Bi-IT ES TD Auto Y Univ. Exeter United Kingdom exeitfrmgbi Bi-IT FR TD Auto Univ. Exeter United Kingdom exespmgcntmn Mono ES TD Auto Univ. Exeter United Kingdom exespmgmn Mono ES TD Auto Y Univ. Exeter United Kingdom exesporgmn Mono ES TD Auto Univ. Exeter United Kingdom exespwtpimn Mono ES TD Auto Univ. Exeter United Kingdom exeitmgcntmn Mono IT TD Auto Univ. Exeter United Kingdom exeitmgmn Mono IT TD Auto Univ. Exeter United Kingdom exeitorgmn Mono IT TD Auto Y Univ. Exeter United Kingdom exeitwtpimn Mono IT TD Auto Univ. Jaen/SINAI Spain UJABIDE Bi-DE EN TD Auto Y Univ. Jaen/SINAI Spain UJABISP Bi-ES EN TD Auto Y Univ. Jaen/SINAI Spain UJABIFR Bi-FR EN TD Auto Y Univ. Jaen/SINAI Spain UJABIIT Bi-IT EN TD Auto Y Univ. Jaen/SINAI Spain UJAMLTDRSV2RR Multi EN TD Auto Univ. Jaen/SINAI Spain UJAMLTDNORM Multi EN TD Auto Univ. Jaen/SINAI Spain UJAMLTDRR Multi EN TD Auto Univ. Jaen/SINAI Spain UJAMLTDRSV2 Multi EN TD Auto Y Univ. Jaen/SINAI Spain UJAMLTD2RSV2 Multi EN TD Auto Y Univ. of Ca. at Berkeley 1 USA BKAMEF1 Amaryllis EN TD Auto Y Univ. of Ca. at Berkeley 1 USA BKAMEF2 Amaryllis EN TD Auto Y Univ. of Ca. at Berkeley 1 USA BKAMEF3 Amaryllis EN TD Auto Y Univ. of Ca. at Berkeley 1 USA BKAMFF2 Amaryllis FR TD Auto Y Univ. of Ca. at Berkeley 1 USA BKAMFF1 Amaryllis FR TDN Auto Y Univ. of Ca. at Berkeley 1 USA BKBIEG2 Bi-DE EN TD Auto Univ. of Ca. at Berkeley 1 USA BKBIEG1 Bi-DE EN TDN Auto Univ. of Ca. at Berkeley 1 USA BKBIFG2 Bi-DE FR TD Auto Univ. of Ca. at Berkeley 1 USA BKBIFG1 Bi-DE FR TDN Auto Univ. of Ca. at Berkeley 1 USA BKBIRG2 Bi-DE RU TD Auto Y Univ. of Ca. at Berkeley 1 USA BKBIRG1 Bi-DE RU TDN Auto Y Univ. of Ca. at Berkeley 1 USA BKBIGF1 Bi-FR DE TD Auto Univ. of Ca. at Berkeley 1 USA BKBIEF1 Bi-FR EN TD Auto Univ. of Ca. at Berkeley 1 USA BKBIEF2 Bi-FR EN TDN Auto Univ. of Ca. at Berkeley 1 USA BKBIRF1 Bi-FR RU TD Auto Univ. of Ca. at Berkeley 1 USA BKGRGG2 GIRT DE TD Auto Y Univ. of Ca. at Berkeley 1 USA BKGRGG1 GIRT DE TDN Auto Y Univ. of Ca. at Berkeley 1 USA BKGREG1 GIRT EN TD Auto Y Univ. of Ca. at Berkeley 1 USA BKGRRG2 GIRT RU TD Auto Y Univ. of Ca. at Berkeley 1 USA BKGRRG1 GIRT RU TDN Auto Y Univ. of Ca. at Berkeley 1 USA BKMLGG2 Mono DE TD Auto Univ. of Ca. at Berkeley 1 USA BKMLGG1 Mono DE TDN Auto Y Univ. of Ca. at Berkeley 1 USA BKMLFF2 Mono FR TD Auto Univ. of Ca. at Berkeley 1 USA BKMLFF1 Mono FR TDN Auto Y Univ. of Ca. at Berkeley 2 USA bky2biende Bi-DE EN TD Auto Y Univ. of Ca. at Berkeley 2 USA bky2bifrde Bi-DE FR TD Auto Univ. of Ca. at Berkeley 2 USA bky2bienes Bi-ES EN TD Auto Y Univ. of Ca. at Berkeley 2 USA bky2bidefr Bi-FR DE TD Auto Univ. of Ca. at Berkeley 2 USA bky2bienfr Bi-FR EN TD Auto Y Univ. of Ca. at Berkeley 2 USA bky2bienfr2 Bi-FR EN TD Auto Univ. of Ca. at Berkeley 2 USA bky2bienit Bi-IT EN TD Auto Univ. of Ca. at Berkeley 2 USA bky2biennl Bi-NL EN TD Auto Y Univ. of Ca. at Berkeley 2 USA bky2mode Mono DE TD Auto Y Univ. of Ca. at Berkeley 2 USA bky2moes Mono ES TD Auto Y Univ. of Ca. at Berkeley 2 USA bky2mofr Mono FR TD Auto Y Univ. of Ca. at Berkeley 2 USA bky2moit Mono IT TD Auto Y Univ. of Ca. at Berkeley 2 USA bky2monl Mono NL TD Auto Y Univ. of Ca. at Berkeley 2 USA bky2muen1 Multi EN TD Auto Y Univ. of Ca. at Berkeley 2 USA bky2muen2 Multi EN TD Auto Y Univ. of Twente Netherlands tnoen1 Bi-NL EN TD Auto Univ. of Twente Netherlands tnofifi1 Mono FI TD Auto Y Univ. of Twente Netherlands tnoutn1 Mono NL TDN Manual Y Univ. Tampere, Dept. IS Finland finbi2 Bi-FI EN TD Auto Y Univ. Tampere, Dept. IS Finland bifren Bi-FR EN TD Auto Y Univ. Tampere, Dept. IS Finland dualge Bi-NL EN TD Auto Y Univ. Tampere, Dept. IS Finland finmo1 Mono FI TD Auto Y Univ. Tampere, Dept. IS Finland finmo2 Mono FI TD Auto Y Univ. Tampere, Dept. IS Finland tremu1 Multi EN TD Auto Y Univ. Tampere, Dept. IS Finland tremu2 Multi EN TD Auto Y Université de Neuchâtel Switzerland UniNEama1 Amaryllis FR TD Auto Y Université de Neuchâtel Switzerland UniNEama2 Amaryllis FR TD Auto Y Université de Neuchâtel Switzerland UniNEama3 Amaryllis FR TD Auto Y Université de Neuchâtel Switzerland UniNEama4 Amaryllis FR TD Auto Y Université de Neuchâtel Switzerland UniNEamaN1 Amaryllis FR TDN Auto Y Université de Neuchâtel Switzerland UniNEdeBi Bi-DE EN TD Auto Y Université de Neuchâtel Switzerland UniNEdeBi2 Bi-DE EN TD Auto Université de Neuchâtel Switzerland UniNEesBi Bi-ES EN TD Auto Université de Neuchâtel Switzerland UniNEesBi3 Bi-ES EN TD Auto Université de Neuchâtel Switzerland UniNEesBi2 Bi-ES EN TD Auto Université de Neuchâtel Switzerland UniNEfrBi Bi-FR EN TD Auto Université de Neuchâtel Switzerland UniNEfrBi3 Bi-FR EN TD Auto Université de Neuchâtel Switzerland UniNEfrBi2 Bi-FR EN TD Auto Université de Neuchâtel Switzerland UniNEitBi Bi-IT EN TD Auto Y Université de Neuchâtel Switzerland UniNEitBi2 Bi-IT EN TD Auto Université de Neuchâtel Switzerland UniNEde Mono DE TD Auto Université de Neuchâtel Switzerland UniNEdetdn Mono DE TDN Manual Y Université de Neuchâtel Switzerland UniNEes Mono ES TD Auto Université de Neuchâtel Switzerland UniNEestdn Mono ES TDN Auto Y Université de Neuchâtel Switzerland UniNEfi1 Mono FI TD Auto Y Université de Neuchâtel Switzerland UniNEfi2 Mono FI TD Auto Y Université de Neuchâtel Switzerland UniNEfr Mono FR TD Auto Université de Neuchâtel Switzerland UniNEfrtdn Mono FR TDN Manual Y Université de Neuchâtel Switzerland UniNEit Mono IT TD Auto Y Université de Neuchâtel Switzerland UniNEnl Mono NL TD Auto Y Université de Neuchâtel Switzerland UniNEm1 Multi EN TD Auto Y Université de Neuchâtel Switzerland UniNEm2 Multi EN TD Auto Y Université de Neuchâtel Switzerland UniNEm3 Multi EN TD Auto Université de Neuchâtel Switzerland UniNEm4 Multi EN TD Auto Université de Neuchâtel Switzerland UniNEm5 Multi EN TD Auto University of Alicante Spain irn1 Mono ES TD Auto Y University of Alicante Spain irn3 Mono ES TD Auto University of Alicante Spain irn2 Mono ES TDN Auto University of Alicante Spain irn4 Mono ES TDN Auto Y University of Amsterdam Netherlands UAmsC02EnAmTTiKW Amaryllis EN TD Auto Y University of Amsterdam Netherlands UAmsC02EnAmTTiRR Amaryllis EN TD Auto Y University of Amsterdam Netherlands UAmsC02FrAmKW Amaryllis FR N Auto Y University of Amsterdam Netherlands UAmsC02FrAmTT Amaryllis FR TD Auto Y University of Amsterdam Netherlands UAmsC02FrAmTTiKW Amaryllis FR TDN Auto Y University of Amsterdam Netherlands UAmsC02EnGeNGram Bi-DE EN TD Auto University of Amsterdam Netherlands UAmsC02EnDuMorph Bi-NL EN TD Auto University of Amsterdam Netherlands UAmsC02EnDuNGiMO Bi-NL EN TD Auto University of Amsterdam Netherlands UAmsC02EnDuNGram Bi-NL EN TD Auto University of Amsterdam Netherlands UAmsC02GeGiTT GIRT DE TD Auto Y University of Amsterdam Netherlands UAmsC02GeGiTTiKW GIRT DE TD Auto Y University of Amsterdam Netherlands UAmsC02GeGiTTiRR GIRT DE TD Auto Y University of Amsterdam Netherlands UAmsC02EnGiTTiKW GIRT EN TD Auto Y University of Amsterdam Netherlands UAmsC02EnGiTTiRR GIRT EN TD Auto Y University of Amsterdam Netherlands UAmsC02GeGeLC2F Mono DE TD Auto University of Amsterdam Netherlands UAmsC02GeGeNGiMO Mono DE TD Auto Y University of Amsterdam Netherlands UAmsC02GeGeNGram Mono DE TD Auto University of Amsterdam Netherlands UAmsC02SpSpNGiSt Mono ES TD Auto Y University of Amsterdam Netherlands UAmsC02FiFiNGram Mono FI TD Auto Y University of Amsterdam Netherlands UAmsC02FrFrNGiMO Mono FR TD Auto Y University of Amsterdam Netherlands UAmsC02ItItNGiMO Mono IT TD Auto Y University of Amsterdam Netherlands UAmsC02DuDuNGiMO Mono NL TD Auto Y University of Amsterdam Netherlands UAmsC02DuDuNGram Mono NL TD Auto University of Amsterdam Netherlands UAmsC02SwSwNGram Mono SV TD Auto Y University of Salamanca Spain usalNST Mono ES T Auto University of Salamanca Spain usalNAT Mono ES T Auto University of Salamanca Spain usalNNTDN Mono ES TDN Auto Y University of Salamanca Spain usalFNTDN Mono ES TDN Auto Y XRCE Xerox France xrcegirt1 GIRT EN T Auto Y XRCE Xerox France xrcegirt3 GIRT EN T Auto Y XRCE Xerox France xrcegirt4 GIRT EN T Auto Y CLEF 2002 − Multilingual Task (Average Precision) Automatic, Title + Description, Five Best Groups 1.0 0.3783 U Neuchâtel 0.3762 UC Berkeley 2 0.8 0.3409 Eurospider 0.2774 U Jaen/SINAI 0.2331 Océ 0.6 Precision 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Recall CLEF 2002 − Bilingual to German Task (Average Precision) Automatic, Title + Description, Five Best Groups 1.0 0.4759 UC Berkeley 2 0.4448 UC Berkeley 1 0.8 0.4129 U Neuchâtel 0.3137 JHU/APL 0.2747 U Jaen/SINAI 0.6 Precision 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Recall CLEF 2002 − Bilingual to English Task (Average Precision) Automatic, Title + Description, Five Best Groups 1.0 0.4158 JHU/APL 0.3502 Clairvoyance 0.8 0.3235 Océ 0.2449 IRIT 0.2088 Middlesex U 0.6 Precision 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Recall CLEF 2002 − Bilingual to Spanish Task (Average Precision) Automatic, Title + Description, Five Best Groups 1.0 0.4786 U Neuchâtel 0.4567 UC Berkeley 2 0.8 0.4202 U Exeter 0.3699 Océ 0.3602 JHU/APL 0.6 Precision 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Recall CLEF 2002 − Bilingual to Finnish Task (Average Precision) Automatic, Title + Description, Five Best Groups 1.0 0.2016 U Tampere 0.2003 JHU/APL 0.8 0.6 Precision 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Recall CLEF 2002 − Bilingual to French Task (Average Precision) Automatic, Title + Description, Five Best Groups 1.0 0.4935 U Neuchâtel 0.4773 UC Berkeley 2 0.8 0.4312 UC Berkeley 1 0.3505 JHU/APL 0.3467 U Jaen/SINAI 0.6 Precision 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Recall CLEF 2002 − Bilingual to Italian Task (Average Precision) Automatic, Title + Description, Five Best Groups 1.0 0.4090 UC Berkeley 2 0.3994 U Exeter 0.8  0.3756 U Neuchâtel 0.3552 ITC−IRST 0.2794 JHU/APL 0.6 Precision 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Recall CLEF 2002 − Bilingual to Dutch Task (Average Precision) Automatic, Title + Description, Five Best Groups 0.8 0.3516 JHU/APL 0.3369 U Twente/TNO 0.3199 UC Berkeley 2 0.6 0.2933 U Amsterdam 0.1912 Océ Precision 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Recall CLEF 2002 − Bilingual to Swedish Task (Average Precision) Automatic, Title + Description, Five Best Groups 1.0 0.3003 JHU/APL 0.8 0.6 Precision 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Recall CLEF 2002 − German Monolingual Task (Average Precision) Automatic, Title + Description, Five Best Groups 1.0 0.5234 UC Berkeley 2 0.4802 U Amsterdam 0.8 0.4672 U Neuchâtel 0.4663 JHU/APL 0.4577 Eurospider 0.6 Precision 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Recall CLEF 2002 − Spanish Monolingual Task (Average Precision) Automatic, Title + Description, Five Best Groups 1.0 0.5441 U Neuchâtel 0.5338 UC Berkeley 2 0.8 0.5192 JHU/APL 0.4993 TLR Research 0.4980 U Alicante 0.6 Precision 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Recall CLEF 2002 − Finnish Monolingual Task (Average Precision) Automatic, Title + Description, Five Best Groups 1.0 0.4090 U Neuchâtel 0.4056 U Twente/TNO 0.8 0.3956 Hummingbird 0.3280 JHU/APL 0.3034 U Amsterdam 0.6 Precision 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Recall CLEF 2002 − French Monolingual Task (Average Precision) Automatic, Title + Description, Five Best Groups 1.0 0.5191 UC Berkeley 2 0.4841 U Neuchâtel 0.8 0.4558 UC Berkeley 1 0.4535 U Amsterdam 0.4509 JHU/APL 0.6 Precision 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Recall CLEF 2002 − Italian Monolingual Task (Average Precision) Automatic, Title + Description, Five Best Groups 1.0  0.5088 F. U. Bordoni 0.4920 ITC−IRST 0.8 0.4750 UC Berkeley 2 0.4618 U Neuchâtel 0.4599 JHU/APL 0.6 Precision 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Recall CLEF 2002 − Dutch Monolingual Task (Average Precision) Automatic, Title + Description, Five Best Groups 1.0 0.5028 JHU/APL 0.4878 U Neuchâtel 0.8 0.4847 UC Berkeley 2 0.4598 U Amsterdam 0.4447 Hummingbird 0.6 Precision 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Recall CLEF 2002 − Swedish Monolingual Task (Average Precision) Automatic, Title + Description, Five Best Groups 1.0 0.4317 JHU/APL 0.4187 U Amsterdam 0.8 0.3441 Hummingbird 0.2439 TLR Research 0.1347 SICS/Conexor 0.6 Precision 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Recall  CLEF 2002 − Amaryllis Bilingual Task (Average Precision) Automatic, Title + Description, Five Best Groups 1.0 0.4272 UC Berkeley 1 0.2660 U Amsterdam 0.8 0.6 Precision 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Recall CLEF 2002 − Amaryllis Monolingual Task (Average Precision) Automatic, Title + Description, Five Best Groups 1.0 0.4802 U Neuchâtel 0.4396 UC Berkeley 1 0.8 0.2681 U Amsterdam 0.6 Precision 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Recall CLEF 2002 − GIRT Bilingual Task (Average Precision) Automatic, Title + Description, Five Best Groups 1.0 0.2330 UC Berkeley 1 0.0704 U Amsterdam 0.8 0.6 Precision 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Recall CLEF 2002 − GIRT Monolingual Task (Average Precision) Automatic, Title + Description, Five Best Groups 1.0 0.2587 UC Berkeley 1 0.1906 U Amsterdam 0.8 0.1890 U Dortmund 0.1097 U Hildesheim 0.6 Precision 0.4 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Recall JHU/APL, Multilingual English, Auto, TD Run aplmuena Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0432 Total number of documents Query 92: 0.4077 over all queries Query 93: 0.0001 Retrieved: 50000 0.8 Query 94: 0.4458 Relevant: 8068 Query 95: 0.1565 Rel_ret: 4729 Query 96: 0.0007 Query 97: 0.1591 Interpolated Recall - Query 98: 0.0322 Precision Averages: 0.6 Query 99: 0.0758 at 0.00 0.6876 Query 100: 0.1888 at 0.10 0.4105 Query 101: 0.1633 at 0.20 0.3359 Precision Query 102: 0.5017 at 0.30 0.2832 0.4 Query 103: 0.1062 at 0.40 0.2274 Query 104: 0.3265 at 0.50 0.1755 Query 105: 0.5721 at 0.60 0.1524 Query 106: 0.0605 at 0.70 0.1219 0.2 Query 107: 0.1172 at 0.80 0.0903 Query 108: 0.2891 at 0.90 0.0579 Query 109: 0.1746 at 1.00 0.0186 Query 110: 0.1680 0.0 Query 111: 0.1382 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.1430 for all rel. documents Query 113: 0.0214 0.2070 Recall Query 114: 0.3196 Precision: Query 115: 0.1753 At 5 docs: 0.4680 Run aplmuena Query 116: 0.1621 At 10 docs: 0.4180 Query 117: 0.0334 At 15 docs: 0.3973 Comparison to median by topic Query 118: 0.0043 At 20 docs: 0.3790 +1.0 Query 119: 0.6915 At 30 docs: 0.3513 Query 120: 0.1655 At 100 docs: 0.2792 Query 121: 0.2549 At 200 docs: 0.2189 Query 122: 0.1637 At 500 docs: 0.1425 Query 123: 0.6085 At 1000 docs: 0.0946 +0.5 Query 124: 0.1104 Query 125: 0.0921 R-Precision (prec. after R Query 126: 0.1512 (=num_rel) docs. retrieved): Query 127: 0.0481 Exact: 0.2650 Query 128: 0.0401 0.0 Query 129: 0.2482 Query 130: 0.6098 Query 131: 0.5279 Query 132: 0.5204 Query 133: 0.0131 Difference (average precision) Query 134: 0.1615 −0.5 Query 135: 0.1195 Query 136: 0.1839 Query 137: 0.0800 Query 138: 0.2859 Query 139: 0.0398 −1.0 Query 140: 0.2495 91 100 110 120 130 140 Topic number JHU/APL, Multilingual English, Auto, TD Run aplmuenb Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0832 Total number of documents Query 92: 0.3773 over all queries Query 93: 0.0001 Retrieved: 50000 0.8 Query 94: 0.5247 Relevant: 8068 Query 95: 0.1735 Rel_ret: 4660 Query 96: 0.0018 Query 97: 0.1872 Interpolated Recall - Query 98: 0.0623 Precision Averages: 0.6 Query 99: 0.0733 at 0.00 0.7118 Query 100: 0.2563 at 0.10 0.4276 Query 101: 0.3103 at 0.20 0.3526 Precision Query 102: 0.3980 at 0.30 0.2969 0.4 Query 103: 0.1100 at 0.40 0.2424 Query 104: 0.2544 at 0.50 0.1793 Query 105: 0.5276 at 0.60 0.1421 Query 106: 0.0806 at 0.70 0.1111 0.2 Query 107: 0.1124 at 0.80 0.0691 Query 108: 0.2181 at 0.90 0.0375 Query 109: 0.1477 at 1.00 0.0099 Query 110: 0.1937 0.0 Query 111: 0.1157 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.0753 for all rel. documents Query 113: 0.0193 0.2082 Recall Query 114: 0.3674 Precision: Query 115: 0.1810 At 5 docs: 0.4480 Run aplmuenb Query 116: 0.1705 At 10 docs: 0.4420 Query 117: 0.0533 At 15 docs: 0.4267 Comparison to median by topic Query 118: 0.0032 At 20 docs: 0.4050 +1.0 Query 119: 0.6800 At 30 docs: 0.3847 Query 120: 0.0827 At 100 docs: 0.2956 Query 121: 0.1442 At 200 docs: 0.2285 Query 122: 0.1522 At 500 docs: 0.1434 Query 123: 0.6079 At 1000 docs: 0.0932 +0.5 Query 124: 0.1322 Query 125: 0.1585 R-Precision (prec. after R Query 126: 0.1748 (=num_rel) docs. retrieved): Query 127: 0.0726 Exact: 0.2765 Query 128: 0.0301 0.0 Query 129: 0.2650 Query 130: 0.5426 Query 131: 0.5086 Query 132: 0.4081 Query 133: 0.0045 Difference (average precision) Query 134: 0.2922 −0.5 Query 135: 0.1567 Query 136: 0.2169 Query 137: 0.0717 Query 138: 0.3366 Query 139: 0.0608 −1.0 Query 140: 0.2316 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 2, Multilingual English, Auto, TD Run bky2muen1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1832 Total number of documents Query 92: 0.3782 over all queries Query 93: 0.6555 Retrieved: 50000 0.8 Query 94: 0.4686 Relevant: 8068 Query 95: 0.2080 Rel_ret: 5880 Query 96: 0.2726 Query 97: 0.4967 Interpolated Recall - Query 98: 0.5991 Precision Averages: 0.6 Query 99: 0.2957 at 0.00 0.8646 Query 100: 0.4329 at 0.10 0.6252 Query 101: 0.2298 at 0.20 0.5606 Precision Query 102: 0.5244 at 0.30 0.4992 0.4 Query 103: 0.5829 at 0.40 0.4397 Query 104: 0.1913 at 0.50 0.3898 Query 105: 0.5235 at 0.60 0.3364 Query 106: 0.1210 at 0.70 0.2759 0.2 Query 107: 0.1798 at 0.80 0.2115 Query 108: 0.3819 at 0.90 0.1229 Query 109: 0.1130 at 1.00 0.0136 Query 110: 0.6656 0.0 Query 111: 0.0893 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5905 for all rel. documents Query 113: 0.0067 0.3762 Recall Query 114: 0.2074 Precision: Query 115: 0.3806 At 5 docs: 0.5960 Run bky2muen1 Query 116: 0.5823 At 10 docs: 0.5880 Query 117: 0.2375 At 15 docs: 0.5867 Comparison to median by topic Query 118: 0.0340 At 20 docs: 0.5800 +1.0 Query 119: 0.7770 At 30 docs: 0.5587 Query 120: 0.1850 At 100 docs: 0.4092 Query 121: 0.4959 At 200 docs: 0.3196 Query 122: 0.2526 At 500 docs: 0.1961 Query 123: 0.7943 At 1000 docs: 0.1176 +0.5 Query 124: 0.0272 Query 125: 0.1486 R-Precision (prec. after R Query 126: 0.2036 (=num_rel) docs. retrieved): Query 127: 0.6207 Exact: 0.4153 Query 128: 0.0385 0.0 Query 129: 0.3792 Query 130: 0.4976 Query 131: 0.4905 Query 132: 0.4855 Query 133: 0.1314 Difference (average precision) Query 134: 0.8288 −0.5 Query 135: 0.6999 Query 136: 0.8458 Query 137: 0.2403 Query 138: 0.3528 Query 139: 0.2913 −1.0 Query 140: 0.3931 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 2, Multilingual English, Auto, TD Run bky2muen2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1036 Total number of documents Query 92: 0.3794 over all queries Query 93: 0.5229 Retrieved: 50000 0.8 Query 94: 0.3986 Relevant: 8068 Query 95: 0.2249 Rel_ret: 5765 Query 96: 0.2914 Query 97: 0.4919 Interpolated Recall - Query 98: 0.4665 Precision Averages: 0.6 Query 99: 0.3154 at 0.00 0.8159 Query 100: 0.4484 at 0.10 0.6061 Query 101: 0.2135 at 0.20 0.5418 Precision Query 102: 0.5181 at 0.30 0.4890 0.4 Query 103: 0.5901 at 0.40 0.4322 Query 104: 0.1696 at 0.50 0.3731 Query 105: 0.5010 at 0.60 0.3106 Query 106: 0.1230 at 0.70 0.2509 0.2 Query 107: 0.1578 at 0.80 0.1916 Query 108: 0.4011 at 0.90 0.1110 Query 109: 0.1191 at 1.00 0.0102 Query 110: 0.6241 0.0 Query 111: 0.0814 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5816 for all rel. documents Query 113: 0.0069 0.3570 Recall Query 114: 0.2176 Precision: Query 115: 0.3373 At 5 docs: 0.5760 Run bky2muen2 Query 116: 0.5694 At 10 docs: 0.5540 Query 117: 0.2697 At 15 docs: 0.5600 Comparison to median by topic Query 118: 0.0379 At 20 docs: 0.5580 +1.0 Query 119: 0.7676 At 30 docs: 0.5387 Query 120: 0.1668 At 100 docs: 0.4044 Query 121: 0.5218 At 200 docs: 0.3148 Query 122: 0.1923 At 500 docs: 0.1915 Query 123: 0.7845 At 1000 docs: 0.1153 +0.5 Query 124: 0.0263 Query 125: 0.1055 R-Precision (prec. after R Query 126: 0.1748 (=num_rel) docs. retrieved): Query 127: 0.5701 Exact: 0.4055 Query 128: 0.0321 0.0 Query 129: 0.3330 Query 130: 0.4799 Query 131: 0.5198 Query 132: 0.3482 Query 133: 0.1085 Difference (average precision) Query 134: 0.8277 −0.5 Query 135: 0.6924 Query 136: 0.7821 Query 137: 0.2238 Query 138: 0.4248 Query 139: 0.3171 −1.0 Query 140: 0.2903 91 100 110 120 130 140 Topic number Eurospider IT AG, Multilingual English, Auto, TD Run EAN2MDF4 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.2240 Total number of documents Query 92: 0.3764 over all queries Query 93: 0.6842 Retrieved: 50000 0.8 Query 94: 0.5001 Relevant: 8068 Query 95: 0.1830 Rel_ret: 5698 Query 96: 0.2029 Query 97: 0.2717 Interpolated Recall - 0.6 Query 98: 0.4926 Precision Averages: Query 99: 0.1601 at 0.00 0.8629 Query 100: 0.4937 at 0.10 0.6314 Precision Query 101: 0.3323 at 0.20 0.5489 Query 102: 0.5128 at 0.30 0.4741 0.4 Query 103: 0.4731 at 0.40 0.4122 Query 104: 0.1513 at 0.50 0.3466 Query 105: 0.5629 at 0.60 0.2913 Query 106: 0.1669 at 0.70 0.2326 0.2 Query 107: 0.1734 at 0.80 0.1468 Query 108: 0.3487 at 0.90 0.0856 Query 109: 0.1563 at 1.00 0.0105 Query 110: 0.5965 0.0 Query 111: 0.2235 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5838 for all rel. documents Query 113: 0.0061 0.3480 Recall Query 114: 0.2488 Precision: Query 115: 0.2629 At 5 docs: 0.6480 Run EAN2MDF4 Query 116: 0.6334 At 10 docs: 0.6260 Comparison to median by topic Query 117: 0.2071 At 15 docs: 0.6067 +1.0 Query 118: 0.0214 At 20 docs: 0.5700 Query 119: 0.7728 At 30 docs: 0.5447 Query 120: 0.1728 At 100 docs: 0.3940 Query 121: 0.1982 At 200 docs: 0.2992 Query 122: 0.1219 At 500 docs: 0.1846 Query 123: 0.7282 At 1000 docs: 0.1140 +0.5 Query 124: 0.1566 Query 125: 0.3115 R-Precision (prec. after R Query 126: 0.4163 (=num_rel) docs. retrieved): Query 127: 0.4233 Exact: 0.3958 Query 128: 0.0903 0.0 Query 129: 0.3469 Query 130: 0.1722 Query 131: 0.1369 Query 132: 0.3890 Query 133: 0.2042 Difference (average precision) Query 134: 0.8435 −0.5 Query 135: 0.4988 Query 136: 0.8520 Query 137: 0.2787 Query 138: 0.3670 Query 139: 0.2943 −1.0 Query 140: 0.3754 91 100 110 120 130 140 Topic number Eurospider IT AG, Multilingual German, Auto, TD Run EIT2MDC3 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0818 Total number of documents Query 92: 0.3786 over all queries Query 93: 0.7509 Retrieved: 50000 0.8 Query 94: 0.6713 Relevant: 8068 Query 95: 0.0315 Rel_ret: 5368 Query 96: 0.4379 Query 97: 0.3165 Interpolated Recall - 0.6 Query 98: 0.4991 Precision Averages: Query 99: 0.0544 at 0.00 0.7893 Query 100: 0.4143 at 0.10 0.6033 Precision Query 101: 0.4902 at 0.20 0.5375 Query 102: 0.3781 at 0.30 0.4587 0.4 Query 103: 0.4825 at 0.40 0.4050 Query 104: 0.1027 at 0.50 0.3508 Query 105: 0.6118 at 0.60 0.2810 Query 106: 0.1426 at 0.70 0.2310 0.2 Query 107: 0.1767 at 0.80 0.1549 Query 108: 0.3006 at 0.90 0.0815 Query 109: 0.1408 at 1.00 0.0129 Query 110: 0.6960 0.0 Query 111: 0.0538 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5966 for all rel. documents Query 113: 0.0350 0.3400 Recall Query 114: 0.2745 Precision: Query 115: 0.1414 At 5 docs: 0.6160 Run EIT2MDC3 Query 116: 0.5209 At 10 docs: 0.5860 Comparison to median by topic Query 117: 0.1757 At 15 docs: 0.5813 +1.0 Query 118: 0.0602 At 20 docs: 0.5520 Query 119: 0.5612 At 30 docs: 0.5200 Query 120: 0.0470 At 100 docs: 0.3940 Query 121: 0.3871 At 200 docs: 0.2886 Query 122: 0.0761 At 500 docs: 0.1735 Query 123: 0.7615 At 1000 docs: 0.1074 +0.5 Query 124: 0.2050 Query 125: 0.4068 R-Precision (prec. after R Query 126: 0.3548 (=num_rel) docs. retrieved): Query 127: 0.5857 Exact: 0.3802 Query 128: 0.0562 0.0 Query 129: 0.3353 Query 130: 0.3197 Query 131: 0.1067 Query 132: 0.4175 Query 133: 0.4033 Difference (average precision) Query 134: 0.7047 −0.5 Query 135: 0.4059 Query 136: 0.9091 Query 137: 0.0303 Query 138: 0.5031 Query 139: 0.2298 −1.0 Query 140: 0.1761 91 100 110 120 130 140 Topic number Eurospider IT AG, Multilingual German, Auto, TD Run EIT2MDF3 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0820 Total number of documents Query 92: 0.3775 over all queries Query 93: 0.7410 Retrieved: 50000 0.8 Query 94: 0.6401 Relevant: 8068 Query 95: 0.0423 Rel_ret: 5411 Query 96: 0.4344 Query 97: 0.3248 Interpolated Recall - 0.6 Query 98: 0.4954 Precision Averages: Query 99: 0.0521 at 0.00 0.7882 Query 100: 0.4017 at 0.10 0.6047 Precision Query 101: 0.4538 at 0.20 0.5397 Query 102: 0.4011 at 0.30 0.4587 0.4 Query 103: 0.4521 at 0.40 0.4120 Query 104: 0.1017 at 0.50 0.3503 Query 105: 0.6143 at 0.60 0.2826 Query 106: 0.1569 at 0.70 0.2308 0.2 Query 107: 0.1793 at 0.80 0.1615 Query 108: 0.3092 at 0.90 0.0820 Query 109: 0.1387 at 1.00 0.0137 Query 110: 0.7302 0.0 Query 111: 0.0609 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6056 for all rel. documents Query 113: 0.0347 0.3409 Recall Query 114: 0.2866 Precision: Query 115: 0.1566 At 5 docs: 0.6160 Run EIT2MDF3 Query 116: 0.5285 At 10 docs: 0.6040 Comparison to median by topic Query 117: 0.1578 At 15 docs: 0.5853 +1.0 Query 118: 0.0591 At 20 docs: 0.5520 Query 119: 0.5937 At 30 docs: 0.5247 Query 120: 0.0418 At 100 docs: 0.3958 Query 121: 0.3411 At 200 docs: 0.2911 Query 122: 0.0807 At 500 docs: 0.1746 Query 123: 0.7649 At 1000 docs: 0.1082 +0.5 Query 124: 0.2238 Query 125: 0.3958 R-Precision (prec. after R Query 126: 0.3685 (=num_rel) docs. retrieved): Query 127: 0.5470 Exact: 0.3836 Query 128: 0.0532 0.0 Query 129: 0.3258 Query 130: 0.3432 Query 131: 0.1302 Query 132: 0.4066 Query 133: 0.4093 Difference (average precision) Query 134: 0.6917 −0.5 Query 135: 0.3865 Query 136: 0.9246 Query 137: 0.0283 Query 138: 0.5329 Query 139: 0.2370 −1.0 Query 140: 0.1999 91 100 110 120 130 140 Topic number Eurospider IT AG, Multilingual German, Auto, TDN Run EIT2MNF3 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1523 Total number of documents Query 92: 0.3837 over all queries Query 93: 0.7426 Retrieved: 50000 0.8 Query 94: 0.6015 Relevant: 8068 Query 95: 0.1727 Rel_ret: 5620 Query 96: 0.3906 Query 97: 0.6104 Interpolated Recall - 0.6 Query 98: 0.5064 Precision Averages: Query 99: 0.1594 at 0.00 0.8652 Query 100: 0.4017 at 0.10 0.6532 Precision Query 101: 0.4398 at 0.20 0.5573 Query 102: 0.3223 at 0.30 0.4966 0.4 Query 103: 0.4882 at 0.40 0.4081 Query 104: 0.0930 at 0.50 0.3532 Query 105: 0.5976 at 0.60 0.2893 Query 106: 0.2142 at 0.70 0.2212 0.2 Query 107: 0.1751 at 0.80 0.1523 Query 108: 0.3019 at 0.90 0.0896 Query 109: 0.1399 at 1.00 0.0128 Query 110: 0.6563 0.0 Query 111: 0.0681 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5486 for all rel. documents Query 113: 0.0436 0.3554 Recall Query 114: 0.5335 Precision: Query 115: 0.1085 At 5 docs: 0.6880 Run EIT2MNF3 Query 116: 0.4369 At 10 docs: 0.6520 Comparison to median by topic Query 117: 0.1359 At 15 docs: 0.6107 +1.0 Query 118: 0.1013 At 20 docs: 0.5950 Query 119: 0.6094 At 30 docs: 0.5593 Query 120: 0.0580 At 100 docs: 0.4160 Query 121: 0.2838 At 200 docs: 0.3084 Query 122: 0.1416 At 500 docs: 0.1849 Query 123: 0.6191 At 1000 docs: 0.1124 +0.5 Query 124: 0.2511 Query 125: 0.4001 R-Precision (prec. after R Query 126: 0.4946 (=num_rel) docs. retrieved): Query 127: 0.5298 Exact: 0.3934 Query 128: 0.0248 0.0 Query 129: 0.2733 Query 130: 0.5268 Query 131: 0.1900 Query 132: 0.3862 Query 133: 0.4671 Difference (average precision) Query 134: 0.6831 −0.5 Query 135: 0.4539 Query 136: 0.7005 Query 137: 0.0306 Query 138: 0.7074 Query 139: 0.1972 −1.0 Query 140: 0.2165 91 100 110 120 130 140 Topic number Eurospider IT AG, Multilingual German, Auto, TDN Run EIT2MNU1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0996 Total number of documents Query 92: 0.3913 over all queries Query 93: 0.7536 Retrieved: 50000 0.8 Query 94: 0.5366 Relevant: 8068 Query 95: 0.1531 Rel_ret: 5188 Query 96: 0.4614 Query 97: 0.6221 Interpolated Recall - 0.6 Query 98: 0.5187 Precision Averages: Query 99: 0.0862 at 0.00 0.8267 Query 100: 0.4378 at 0.10 0.6371 Precision Query 101: 0.4643 at 0.20 0.5431 Query 102: 0.2965 at 0.30 0.4725 0.4 Query 103: 0.5177 at 0.40 0.4071 Query 104: 0.0784 at 0.50 0.3524 Query 105: 0.6062 at 0.60 0.2995 Query 106: 0.1576 at 0.70 0.2402 0.2 Query 107: 0.1470 at 0.80 0.1620 Query 108: 0.2436 at 0.90 0.1031 Query 109: 0.0655 at 1.00 0.0081 Query 110: 0.6546 0.0 Query 111: 0.0352 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5817 for all rel. documents Query 113: 0.0391 0.3539 Recall Query 114: 0.4816 Precision: Query 115: 0.0159 At 5 docs: 0.6680 Run EIT2MNU1 Query 116: 0.3221 At 10 docs: 0.6560 Comparison to median by topic Query 117: 0.0773 At 15 docs: 0.6173 +1.0 Query 118: 0.0763 At 20 docs: 0.6010 Query 119: 0.4669 At 30 docs: 0.5607 Query 120: 0.0216 At 100 docs: 0.4066 Query 121: 0.5401 At 200 docs: 0.2994 Query 122: 0.1269 At 500 docs: 0.1720 Query 123: 0.8311 At 1000 docs: 0.1038 +0.5 Query 124: 0.1584 Query 125: 0.4183 R-Precision (prec. after R Query 126: 0.4415 (=num_rel) docs. retrieved): Query 127: 0.6387 Exact: 0.3906 Query 128: 0.0220 0.0 Query 129: 0.2774 Query 130: 0.6096 Query 131: 0.2074 Query 132: 0.3942 Query 133: 0.4417 Difference (average precision) Query 134: 0.7650 −0.5 Query 135: 0.3497 Query 136: 0.9119 Query 137: 0.0536 Query 138: 0.7498 Query 139: 0.1964 −1.0 Query 140: 0.1533 91 100 110 120 130 140 Topic number IRIT, Toulouse, Multilingual English, Auto, TD Run iritMEn2All Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0282 Total number of documents Query 92: 0.1103 over all queries Query 93: 0.0000 Retrieved: 50000 0.8 Query 94: 0.0956 Relevant: 8068 Query 95: 0.0244 Rel_ret: 1537 Query 96: 0.0005 Query 97: 0.0409 Interpolated Recall - 0.6 Query 98: 0.0000 Precision Averages: Query 99: 0.0371 at 0.00 0.6697 Query 100: 0.1560 at 0.10 0.2809 Precision Query 101: 0.0000 at 0.20 0.1402 Query 102: 0.1896 at 0.30 0.0542 0.4 Query 103: 0.0204 at 0.40 0.0326 Query 104: 0.2053 at 0.50 0.0197 Query 105: 0.4195 at 0.60 0.0067 Query 106: 0.0206 at 0.70 0.0000 0.2 Query 107: 0.0974 at 0.80 0.0000 Query 108: 0.0627 at 0.90 0.0000 Query 109: 0.0541 at 1.00 0.0000 Query 110: 0.0665 0.0 Query 111: 0.1842 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.1017 for all rel. documents Query 113: 0.0088 0.0756 Recall Query 114: 0.0280 Precision: Query 115: 0.0184 At 5 docs: 0.5000 Run iritMEn2All Query 116: 0.0480 At 10 docs: 0.4000 Comparison to median by topic Query 117: 0.0027 At 15 docs: 0.3400 +1.0 Query 118: 0.0000 At 20 docs: 0.3140 Query 119: 0.1446 At 30 docs: 0.2613 Query 120: 0.1153 At 100 docs: 0.1378 Query 121: 0.0452 At 200 docs: 0.0881 Query 122: 0.0407 At 500 docs: 0.0480 Query 123: 0.1617 At 1000 docs: 0.0307 +0.5 Query 124: 0.0450 Query 125: 0.0186 R-Precision (prec. after R Query 126: 0.1140 (=num_rel) docs. retrieved): Query 127: 0.0001 Exact: 0.1165 Query 128: 0.0015 0.0 Query 129: 0.0831 Query 130: 0.3751 Query 131: 0.0631 Query 132: 0.0000 Query 133: 0.0002 Difference (average precision) Query 134: 0.1574 −0.5 Query 135: 0.0042 Query 136: 0.1969 Query 137: 0.0313 Query 138: 0.0700 Query 139: 0.0234 −1.0 Query 140: 0.0667 91 100 110 120 130 140 Topic number National Taiwan Univ., Multilingual English, Auto, TD Run NTUmulti01 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0006 Total number of documents Query 92: 0.0415 over all queries Query 93: 0.0000 Retrieved: 50000 0.8 Query 94: 0.0358 Relevant: 8068 Query 95: 0.0293 Rel_ret: 1083 Query 96: 0.0004 Query 97: 0.0037 Interpolated Recall - 0.6 Query 98: 0.0005 Precision Averages: Query 99: 0.0140 at 0.00 0.3098 Query 100: 0.0044 at 0.10 0.0565 Precision Query 101: 0.0241 at 0.20 0.0161 Query 102: 0.0260 at 0.30 0.0068 0.4 Query 103: 0.0077 at 0.40 0.0015 Query 104: 0.0183 at 0.50 0.0013 Query 105: 0.0376 at 0.60 0.0000 Query 106: 0.0101 at 0.70 0.0000 0.2 Query 107: 0.0090 at 0.80 0.0000 Query 108: 0.0051 at 0.90 0.0000 Query 109: 0.0135 at 1.00 0.0000 Query 110: 0.0132 0.0 Query 111: 0.0287 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.0224 for all rel. documents Query 113: 0.0004 0.0173 Recall Query 114: 0.0002 Precision: Query 115: 0.0296 At 5 docs: 0.1440 Run NTUmulti01 Query 116: 0.0724 At 10 docs: 0.1440 Comparison to median by topic Query 117: 0.0026 At 15 docs: 0.1320 +1.0 Query 118: 0.0000 At 20 docs: 0.1280 Query 119: 0.0654 At 30 docs: 0.1027 Query 120: 0.0077 At 100 docs: 0.0664 Query 121: 0.0060 At 200 docs: 0.0488 Query 122: 0.0030 At 500 docs: 0.0309 Query 123: 0.0227 At 1000 docs: 0.0217 +0.5 Query 124: 0.0084 Query 125: 0.0027 R-Precision (prec. after R Query 126: 0.0316 (=num_rel) docs. retrieved): Query 127: 0.0010 Exact: 0.0535 Query 128: 0.0009 0.0 Query 129: 0.0369 Query 130: 0.0083 Query 131: 0.0198 Query 132: 0.0000 Query 133: 0.0072 Difference (average precision) Query 134: 0.0378 −0.5 Query 135: 0.0000 Query 136: 0.0046 Query 137: 0.0027 Query 138: 0.0915 Query 139: 0.0097 −1.0 Query 140: 0.0473 91 100 110 120 130 140 Topic number National Taiwan Univ., Multilingual English, Auto, TD Run NTUmulti02 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0022 Total number of documents Query 92: 0.0277 over all queries Query 93: 0.0000 Retrieved: 50000 0.8 Query 94: 0.0761 Relevant: 8068 Query 95: 0.0170 Rel_ret: 1135 Query 96: 0.0002 Query 97: 0.0061 Interpolated Recall - 0.6 Query 98: 0.0005 Precision Averages: Query 99: 0.0112 at 0.00 0.3584 Query 100: 0.0048 at 0.10 0.0989 Precision Query 101: 0.0395 at 0.20 0.0251 Query 102: 0.0745 at 0.30 0.0122 0.4 Query 103: 0.0101 at 0.40 0.0027 Query 104: 0.0107 at 0.50 0.0013 Query 105: 0.0579 at 0.60 0.0000 Query 106: 0.0073 at 0.70 0.0000 0.2 Query 107: 0.0121 at 0.80 0.0000 Query 108: 0.0192 at 0.90 0.0000 Query 109: 0.0280 at 1.00 0.0000 Query 110: 0.0120 0.0 Query 111: 0.0170 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.0479 for all rel. documents Query 113: 0.0002 0.0266 Recall Query 114: 0.0035 Precision: Query 115: 0.0656 At 5 docs: 0.2000 Run NTUmulti02 Query 116: 0.1534 At 10 docs: 0.1660 Comparison to median by topic Query 117: 0.0039 At 15 docs: 0.1507 +1.0 Query 118: 0.0000 At 20 docs: 0.1330 Query 119: 0.0818 At 30 docs: 0.1213 Query 120: 0.0368 At 100 docs: 0.0754 Query 121: 0.0215 At 200 docs: 0.0547 Query 122: 0.0025 At 500 docs: 0.0332 Query 123: 0.0274 At 1000 docs: 0.0227 +0.5 Query 124: 0.0064 Query 125: 0.0017 R-Precision (prec. after R Query 126: 0.0456 (=num_rel) docs. retrieved): Query 127: 0.0020 Exact: 0.0666 Query 128: 0.0008 0.0 Query 129: 0.0266 Query 130: 0.0477 Query 131: 0.0152 Query 132: 0.0000 Query 133: 0.0059 Difference (average precision) Query 134: 0.1413 −0.5 Query 135: 0.0004 Query 136: 0.0067 Query 137: 0.0030 Query 138: 0.1119 Query 139: 0.0112 −1.0 Query 140: 0.0237 91 100 110 120 130 140 Topic number National Taiwan Univ., Multilingual English, Auto, TD Run NTUmulti03 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0029 Total number of documents Query 92: 0.0858 over all queries Query 93: 0.0000 Retrieved: 50000 0.8 Query 94: 0.0770 Relevant: 8068 Query 95: 0.0426 Rel_ret: 1145 Query 96: 0.0001 Query 97: 0.0087 Interpolated Recall - 0.6 Query 98: 0.0003 Precision Averages: Query 99: 0.0129 at 0.00 0.4811 Query 100: 0.0105 at 0.10 0.1303 Precision Query 101: 0.0022 at 0.20 0.0301 Query 102: 0.1372 at 0.30 0.0097 0.4 Query 103: 0.0138 at 0.40 0.0012 Query 104: 0.0405 at 0.50 0.0000 Query 105: 0.1065 at 0.60 0.0000 Query 106: 0.0119 at 0.70 0.0000 0.2 Query 107: 0.0294 at 0.80 0.0000 Query 108: 0.0132 at 0.90 0.0000 Query 109: 0.0151 at 1.00 0.0000 Query 110: 0.0104 0.0 Query 111: 0.0809 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.0691 for all rel. documents Query 113: 0.0007 0.0336 Recall Query 114: 0.0022 Precision: Query 115: 0.0355 At 5 docs: 0.2880 Run NTUmulti03 Query 116: 0.1058 At 10 docs: 0.2340 Comparison to median by topic Query 117: 0.0008 At 15 docs: 0.2053 +1.0 Query 118: 0.0000 At 20 docs: 0.1880 Query 119: 0.1201 At 30 docs: 0.1587 Query 120: 0.0620 At 100 docs: 0.0892 Query 121: 0.0212 At 200 docs: 0.0601 Query 122: 0.0063 At 500 docs: 0.0349 Query 123: 0.0610 At 1000 docs: 0.0229 +0.5 Query 124: 0.0154 Query 125: 0.0081 R-Precision (prec. after R Query 126: 0.0302 (=num_rel) docs. retrieved): Query 127: 0.0003 Exact: 0.0724 Query 128: 0.0021 0.0 Query 129: 0.0391 Query 130: 0.0674 Query 131: 0.0334 Query 132: 0.0000 Query 133: 0.0148 Difference (average precision) Query 134: 0.1400 −0.5 Query 135: 0.0000 Query 136: 0.0048 Query 137: 0.0029 Query 138: 0.0820 Query 139: 0.0043 −1.0 Query 140: 0.0479 91 100 110 120 130 140 Topic number National Taiwan Univ., Multilingual English, Auto, TD Run NTUmulti04 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0045 Total number of documents Query 92: 0.0758 over all queries Query 93: 0.0000 Retrieved: 50000 0.8 Query 94: 0.0806 Relevant: 8068 Query 95: 0.0374 Rel_ret: 1195 Query 96: 0.0000 Query 97: 0.0065 Interpolated Recall - 0.6 Query 98: 0.0003 Precision Averages: Query 99: 0.0117 at 0.00 0.4930 Query 100: 0.0102 at 0.10 0.1452 Precision Query 101: 0.0200 at 0.20 0.0455 Query 102: 0.1640 at 0.30 0.0112 0.4 Query 103: 0.0154 at 0.40 0.0044 Query 104: 0.0390 at 0.50 0.0000 Query 105: 0.1224 at 0.60 0.0000 Query 106: 0.0105 at 0.70 0.0000 0.2 Query 107: 0.0365 at 0.80 0.0000 Query 108: 0.0206 at 0.90 0.0000 Query 109: 0.0250 at 1.00 0.0000 Query 110: 0.0078 0.0 Query 111: 0.0591 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.0689 for all rel. documents Query 113: 0.0005 0.0373 Recall Query 114: 0.0084 Precision: Query 115: 0.0581 At 5 docs: 0.2720 Run NTUmulti04 Query 116: 0.1561 At 10 docs: 0.2300 Comparison to median by topic Query 117: 0.0010 At 15 docs: 0.2080 +1.0 Query 118: 0.0000 At 20 docs: 0.1910 Query 119: 0.0983 At 30 docs: 0.1727 Query 120: 0.0849 At 100 docs: 0.0964 Query 121: 0.0301 At 200 docs: 0.0632 Query 122: 0.0054 At 500 docs: 0.0364 Query 123: 0.0598 At 1000 docs: 0.0239 +0.5 Query 124: 0.0136 Query 125: 0.0074 R-Precision (prec. after R Query 126: 0.0386 (=num_rel) docs. retrieved): Query 127: 0.0004 Exact: 0.0803 Query 128: 0.0021 0.0 Query 129: 0.0362 Query 130: 0.1277 Query 131: 0.0238 Query 132: 0.0000 Query 133: 0.0145 Difference (average precision) Query 134: 0.1486 −0.5 Query 135: 0.0001 Query 136: 0.0049 Query 137: 0.0024 Query 138: 0.0780 Query 139: 0.0046 −1.0 Query 140: 0.0415 91 100 110 120 130 140 Topic number National Taiwan Univ., Multilingual English, Auto, TD Run NTUmulti05 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0041 Total number of documents Query 92: 0.0715 over all queries Query 93: 0.0000 Retrieved: 50000 0.8 Query 94: 0.0793 Relevant: 8068 Query 95: 0.0366 Rel_ret: 1209 Query 96: 0.0002 Query 97: 0.0071 Interpolated Recall - 0.6 Query 98: 0.0004 Precision Averages: Query 99: 0.0095 at 0.00 0.4729 Query 100: 0.0086 at 0.10 0.1427 Precision Query 101: 0.0198 at 0.20 0.0403 Query 102: 0.1359 at 0.30 0.0142 0.4 Query 103: 0.0148 at 0.40 0.0046 Query 104: 0.0325 at 0.50 0.0000 Query 105: 0.1139 at 0.60 0.0000 Query 106: 0.0111 at 0.70 0.0000 0.2 Query 107: 0.0312 at 0.80 0.0000 Query 108: 0.0237 at 0.90 0.0000 Query 109: 0.0241 at 1.00 0.0000 Query 110: 0.0067 0.0 Query 111: 0.0571 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.0705 for all rel. documents Query 113: 0.0005 0.0361 Recall Query 114: 0.0086 Precision: Query 115: 0.0617 At 5 docs: 0.2720 Run NTUmulti05 Query 116: 0.1580 At 10 docs: 0.2240 Comparison to median by topic Query 117: 0.0008 At 15 docs: 0.2080 +1.0 Query 118: 0.0000 At 20 docs: 0.1920 Query 119: 0.1104 At 30 docs: 0.1713 Query 120: 0.0640 At 100 docs: 0.0992 Query 121: 0.0288 At 200 docs: 0.0637 Query 122: 0.0075 At 500 docs: 0.0371 Query 123: 0.0464 At 1000 docs: 0.0242 +0.5 Query 124: 0.0157 Query 125: 0.0041 R-Precision (prec. after R Query 126: 0.0423 (=num_rel) docs. retrieved): Query 127: 0.0004 Exact: 0.0820 Query 128: 0.0019 0.0 Query 129: 0.0433 Query 130: 0.1258 Query 131: 0.0280 Query 132: 0.0000 Query 133: 0.0140 Difference (average precision) Query 134: 0.1474 −0.5 Query 135: 0.0002 Query 136: 0.0048 Query 137: 0.0025 Query 138: 0.0856 Query 139: 0.0066 −1.0 Query 140: 0.0391 91 100 110 120 130 140 Topic number Océ, Multilingual English, Auto, TD Run oce02mulMSbf Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.2185 Total number of documents Query 92: 0.2740 over all queries Query 93: 0.5011 Retrieved: 50000 0.8 Query 94: 0.1004 Relevant: 8068 Query 95: 0.1020 Rel_ret: 4073 Query 96: 0.0889 Query 97: 0.1974 Interpolated Recall - 0.6 Query 98: 0.1501 Precision Averages: Query 99: 0.0526 at 0.00 0.6931 Query 100: 0.3835 at 0.10 0.4605 Precision Query 101: 0.0983 at 0.20 0.3551 Query 102: 0.4295 at 0.30 0.2842 0.4 Query 103: 0.2511 at 0.40 0.2444 Query 104: 0.0501 at 0.50 0.1897 Query 105: 0.3297 at 0.60 0.1334 Query 106: 0.0864 at 0.70 0.1046 0.2 Query 107: 0.0697 at 0.80 0.0660 Query 108: 0.0882 at 0.90 0.0292 Query 109: 0.0218 at 1.00 0.0077 Query 110: 0.5753 0.0 Query 111: 0.0572 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5215 for all rel. documents Query 113: 0.0080 0.2103 Recall Query 114: 0.1210 Precision: Query 115: 0.1322 At 5 docs: 0.4880 Run oce02mulMSbf Query 116: 0.4024 At 10 docs: 0.4820 Comparison to median by topic Query 117: 0.0812 At 15 docs: 0.4600 +1.0 Query 118: 0.0152 At 20 docs: 0.4280 Query 119: 0.6109 At 30 docs: 0.4020 Query 120: 0.1946 At 100 docs: 0.2820 Query 121: 0.4329 At 200 docs: 0.2132 Query 122: 0.0855 At 500 docs: 0.1308 Query 123: 0.7482 At 1000 docs: 0.0815 +0.5 Query 124: 0.0101 Query 125: 0.0166 R-Precision (prec. after R Query 126: 0.1613 (=num_rel) docs. retrieved): Query 127: 0.3039 Exact: 0.2717 Query 128: 0.0213 0.0 Query 129: 0.2002 Query 130: 0.3388 Query 131: 0.2273 Query 132: 0.0763 Query 133: 0.0370 Difference (average precision) Query 134: 0.7779 −0.5 Query 135: 0.1800 Query 136: 0.2928 Query 137: 0.0383 Query 138: 0.1781 Query 139: 0.0226 −1.0 Query 140: 0.1510 91 100 110 120 130 140 Topic number Océ, Multilingual English, Auto, TD Run oce02mulMSlo Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.2565 Total number of documents Query 92: 0.3234 over all queries Query 93: 0.4889 Retrieved: 50000 0.8 Query 94: 0.1164 Relevant: 8068 Query 95: 0.0436 Rel_ret: 4383 Query 96: 0.1001 Query 97: 0.1902 Interpolated Recall - 0.6 Query 98: 0.1612 Precision Averages: Query 99: 0.0653 at 0.00 0.8057 Query 100: 0.4693 at 0.10 0.4735 Precision Query 101: 0.0984 at 0.20 0.3713 Query 102: 0.4484 at 0.30 0.3091 0.4 Query 103: 0.2547 at 0.40 0.2636 Query 104: 0.0561 at 0.50 0.2274 Query 105: 0.3252 at 0.60 0.1588 Query 106: 0.1285 at 0.70 0.1266 0.2 Query 107: 0.0722 at 0.80 0.0981 Query 108: 0.2090 at 0.90 0.0411 Query 109: 0.0414 at 1.00 0.0116 Query 110: 0.5985 0.0 Query 111: 0.0761 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5437 for all rel. documents Query 113: 0.0053 0.2331 Recall Query 114: 0.0838 Precision: Query 115: 0.1500 At 5 docs: 0.5440 Run oce02mulMSlo Query 116: 0.3950 At 10 docs: 0.5280 Comparison to median by topic Query 117: 0.1274 At 15 docs: 0.4827 +1.0 Query 118: 0.0613 At 20 docs: 0.4600 Query 119: 0.5988 At 30 docs: 0.4227 Query 120: 0.2108 At 100 docs: 0.3002 Query 121: 0.3866 At 200 docs: 0.2269 Query 122: 0.0721 At 500 docs: 0.1374 Query 123: 0.7686 At 1000 docs: 0.0877 +0.5 Query 124: 0.0404 Query 125: 0.1779 R-Precision (prec. after R Query 126: 0.0752 (=num_rel) docs. retrieved): Query 127: 0.4086 Exact: 0.2914 Query 128: 0.0081 0.0 Query 129: 0.1595 Query 130: 0.3676 Query 131: 0.1808 Query 132: 0.0603 Query 133: 0.0804 Difference (average precision) Query 134: 0.6950 −0.5 Query 135: 0.2019 Query 136: 0.7603 Query 137: 0.0877 Query 138: 0.1359 Query 139: 0.0371 −1.0 Query 140: 0.2488 91 100 110 120 130 140 Topic number Océ, Multilingual English, Auto, TD Run oce02mulRRbf Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1319 Total number of documents Query 92: 0.2744 over all queries Query 93: 0.4112 Retrieved: 50000 0.8 Query 94: 0.1380 Relevant: 8068 Query 95: 0.0758 Rel_ret: 4073 Query 96: 0.0729 Query 97: 0.1943 Interpolated Recall - 0.6 Query 98: 0.2882 Precision Averages: Query 99: 0.0448 at 0.00 0.6783 Query 100: 0.3827 at 0.10 0.4447 Precision Query 101: 0.1397 at 0.20 0.3441 Query 102: 0.4212 at 0.30 0.2673 0.4 Query 103: 0.2343 at 0.40 0.2198 Query 104: 0.0481 at 0.50 0.1604 Query 105: 0.3131 at 0.60 0.1206 Query 106: 0.1192 at 0.70 0.0905 0.2 Query 107: 0.0611 at 0.80 0.0519 Query 108: 0.0996 at 0.90 0.0229 Query 109: 0.0196 at 1.00 0.0064 Query 110: 0.4448 0.0 Query 111: 0.0532 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5487 for all rel. documents Query 113: 0.0072 0.1973 Recall Query 114: 0.1706 Precision: Query 115: 0.1275 At 5 docs: 0.4840 Run oce02mulRRbf Query 116: 0.3484 At 10 docs: 0.4640 Comparison to median by topic Query 117: 0.0729 At 15 docs: 0.4373 +1.0 Query 118: 0.0139 At 20 docs: 0.4140 Query 119: 0.6330 At 30 docs: 0.3913 Query 120: 0.1504 At 100 docs: 0.2774 Query 121: 0.1772 At 200 docs: 0.2064 Query 122: 0.0893 At 500 docs: 0.1270 Query 123: 0.6377 At 1000 docs: 0.0815 +0.5 Query 124: 0.0163 Query 125: 0.0328 R-Precision (prec. after R Query 126: 0.1560 (=num_rel) docs. retrieved): Query 127: 0.1963 Exact: 0.2627 Query 128: 0.0163 0.0 Query 129: 0.2268 Query 130: 0.3157 Query 131: 0.2350 Query 132: 0.1055 Query 133: 0.0381 Difference (average precision) Query 134: 0.7517 −0.5 Query 135: 0.1652 Query 136: 0.2654 Query 137: 0.0420 Query 138: 0.1807 Query 139: 0.0224 −1.0 Query 140: 0.1530 91 100 110 120 130 140 Topic number Océ, Multilingual English, Auto, TD Run oce02mulRRlo Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1917 Total number of documents Query 92: 0.3089 over all queries Query 93: 0.3854 Retrieved: 50000 0.8 Query 94: 0.1316 Relevant: 8068 Query 95: 0.0447 Rel_ret: 4378 Query 96: 0.0866 Query 97: 0.1692 Interpolated Recall - 0.6 Query 98: 0.2658 Precision Averages: Query 99: 0.0603 at 0.00 0.7873 Query 100: 0.5020 at 0.10 0.4827 Precision Query 101: 0.1469 at 0.20 0.3736 Query 102: 0.4107 at 0.30 0.2966 0.4 Query 103: 0.2280 at 0.40 0.2412 Query 104: 0.0615 at 0.50 0.1929 Query 105: 0.3139 at 0.60 0.1331 Query 106: 0.1329 at 0.70 0.1046 0.2 Query 107: 0.0662 at 0.80 0.0747 Query 108: 0.2001 at 0.90 0.0380 Query 109: 0.0662 at 1.00 0.0099 Query 110: 0.4563 0.0 Query 111: 0.0534 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5787 for all rel. documents Query 113: 0.0067 0.2210 Recall Query 114: 0.0951 Precision: Query 115: 0.1634 At 5 docs: 0.5440 Run oce02mulRRlo Query 116: 0.3485 At 10 docs: 0.5060 Comparison to median by topic Query 117: 0.1146 At 15 docs: 0.4733 +1.0 Query 118: 0.0616 At 20 docs: 0.4560 Query 119: 0.6201 At 30 docs: 0.4140 Query 120: 0.1615 At 100 docs: 0.2988 Query 121: 0.2050 At 200 docs: 0.2242 Query 122: 0.0771 At 500 docs: 0.1364 Query 123: 0.6558 At 1000 docs: 0.0876 +0.5 Query 124: 0.0593 Query 125: 0.1648 R-Precision (prec. after R Query 126: 0.1324 (=num_rel) docs. retrieved): Query 127: 0.2409 Exact: 0.2897 Query 128: 0.0147 0.0 Query 129: 0.2195 Query 130: 0.3399 Query 131: 0.2082 Query 132: 0.1509 Query 133: 0.0886 Difference (average precision) Query 134: 0.7070 −0.5 Query 135: 0.1869 Query 136: 0.6201 Query 137: 0.0859 Query 138: 0.1224 Query 139: 0.0439 −1.0 Query 140: 0.2920 91 100 110 120 130 140 Topic number Océ, Multilingual English, Auto, T Run oce02mulRRloTO Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0344 Total number of documents Query 92: 0.2328 over all queries Query 93: 0.5934 Retrieved: 46431 0.8 Query 94: 0.0637 Relevant: 8068 Query 95: 0.0119 Rel_ret: 3455 Query 96: 0.0321 Query 97: 0.0956 Interpolated Recall - 0.6 Query 98: 0.0326 Precision Averages: Query 99: 0.0235 at 0.00 0.6694 Query 100: 0.3337 at 0.10 0.3865 Precision Query 101: 0.1814 at 0.20 0.2932 Query 102: 0.4728 at 0.30 0.2323 0.4 Query 103: 0.1507 at 0.40 0.1799 Query 104: 0.0707 at 0.50 0.1483 Query 105: 0.4627 at 0.60 0.1141 Query 106: 0.0979 at 0.70 0.0795 0.2 Query 107: 0.0637 at 0.80 0.0481 Query 108: 0.2256 at 0.90 0.0239 Query 109: 0.1601 at 1.00 0.0000 Query 110: 0.1701 0.0 Query 111: 0.0669 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.0266 for all rel. documents Query 113: 0.0010 0.1720 Recall Query 114: 0.0353 Precision: Query 115: 0.3260 At 5 docs: 0.4280 Run oce02mulRRloTO Query 116: 0.5121 At 10 docs: 0.4220 Comparison to median by topic Query 117: 0.0189 At 15 docs: 0.3880 +1.0 Query 118: 0.0509 At 20 docs: 0.3620 Query 119: 0.0810 At 30 docs: 0.3387 Query 120: 0.0339 At 100 docs: 0.2360 Query 121: 0.1620 At 200 docs: 0.1766 Query 122: 0.0306 At 500 docs: 0.1083 Query 123: 0.6591 At 1000 docs: 0.0691 +0.5 Query 124: 0.0418 Query 125: 0.1749 R-Precision (prec. after R Query 126: 0.1760 (=num_rel) docs. retrieved): Query 127: 0.0809 Exact: 0.2439 Query 128: 0.0590 0.0 Query 129: 0.0900 Query 130: 0.3808 Query 131: 0.2498 Query 132: 0.1313 Query 133: 0.0484 Difference (average precision) Query 134: 0.5494 −0.5 Query 135: 0.0096 Query 136: 0.6343 Query 137: 0.0639 Query 138: 0.2318 Query 139: 0.0297 −1.0 Query 140: 0.1359 91 100 110 120 130 140 Topic number RALI U Montreal, Multilingual English, Auto, TDN Run run1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0075 Total number of documents Query 92: 0.1020 over all queries Query 93: 0.0001 Retrieved: 50000 0.8 Query 94: 0.0916 Relevant: 8068 Query 95: 0.0178 Rel_ret: 2507 Query 96: 0.0092 Query 97: 0.1623 Interpolated Recall - 0.6 Query 98: 0.0015 Precision Averages: Query 99: 0.0110 at 0.00 0.6802 Query 100: 0.0413 at 0.10 0.3483 Precision Query 101: 0.0417 at 0.20 0.2077 Query 102: 0.2935 at 0.30 0.1351 0.4 Query 103: 0.0546 at 0.40 0.0811 Query 104: 0.0303 at 0.50 0.0530 Query 105: 0.4738 at 0.60 0.0394 Query 106: 0.0291 at 0.70 0.0260 0.2 Query 107: 0.0492 at 0.80 0.0082 Query 108: 0.0943 at 0.90 0.0000 Query 109: 0.0855 at 1.00 0.0000 Query 110: 0.2436 0.0 Query 111: 0.2778 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.1728 for all rel. documents Query 113: 0.0062 0.1155 Recall Query 114: 0.0447 Precision: Query 115: 0.0977 At 5 docs: 0.4640 Run run1 Query 116: 0.1154 At 10 docs: 0.4400 Comparison to median by topic Query 117: 0.0014 At 15 docs: 0.3947 +1.0 Query 118: 0.0143 At 20 docs: 0.3660 Query 119: 0.2268 At 30 docs: 0.3293 Query 120: 0.1604 At 100 docs: 0.1928 Query 121: 0.0377 At 200 docs: 0.1295 Query 122: 0.0853 At 500 docs: 0.0769 Query 123: 0.2334 At 1000 docs: 0.0501 +0.5 Query 124: 0.0972 Query 125: 0.2816 R-Precision (prec. after R Query 126: 0.0680 (=num_rel) docs. retrieved): Query 127: 0.0082 Exact: 0.1707 Query 128: 0.0254 0.0 Query 129: 0.0341 Query 130: 0.3150 Query 131: 0.1355 Query 132: 0.0001 Query 133: 0.0211 Difference (average precision) Query 134: 0.2808 −0.5 Query 135: 0.0506 Query 136: 0.6545 Query 137: 0.0583 Query 138: 0.1163 Query 139: 0.1265 −1.0 Query 140: 0.1895 91 100 110 120 130 140 Topic number RALI U Montreal, Multilingual English, Auto, TDN Run run2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0074 Total number of documents Query 92: 0.1080 over all queries Query 93: 0.0000 Retrieved: 50000 0.8 Query 94: 0.0902 Relevant: 8068 Query 95: 0.0313 Rel_ret: 2531 Query 96: 0.0093 Query 97: 0.1571 Interpolated Recall - 0.6 Query 98: 0.0061 Precision Averages: Query 99: 0.0110 at 0.00 0.6915 Query 100: 0.0347 at 0.10 0.3669 Precision Query 101: 0.0411 at 0.20 0.2245 Query 102: 0.2827 at 0.30 0.1389 0.4 Query 103: 0.0586 at 0.40 0.0802 Query 104: 0.0341 at 0.50 0.0518 Query 105: 0.4452 at 0.60 0.0361 Query 106: 0.0256 at 0.70 0.0231 0.2 Query 107: 0.0527 at 0.80 0.0069 Query 108: 0.0993 at 0.90 0.0000 Query 109: 0.0921 at 1.00 0.0000 Query 110: 0.2488 0.0 Query 111: 0.2802 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.1427 for all rel. documents Query 113: 0.0066 0.1182 Recall Query 114: 0.0423 Precision: Query 115: 0.1266 At 5 docs: 0.4840 Run run2 Query 116: 0.1245 At 10 docs: 0.4580 Comparison to median by topic Query 117: 0.0022 At 15 docs: 0.4067 +1.0 Query 118: 0.0162 At 20 docs: 0.3830 Query 119: 0.2429 At 30 docs: 0.3387 Query 120: 0.1575 At 100 docs: 0.1970 Query 121: 0.0393 At 200 docs: 0.1333 Query 122: 0.0819 At 500 docs: 0.0776 Query 123: 0.2692 At 1000 docs: 0.0506 +0.5 Query 124: 0.0949 Query 125: 0.2841 R-Precision (prec. after R Query 126: 0.0779 (=num_rel) docs. retrieved): Query 127: 0.0071 Exact: 0.1711 Query 128: 0.0218 0.0 Query 129: 0.0364 Query 130: 0.3248 Query 131: 0.1342 Query 132: 0.0002 Query 133: 0.0211 Difference (average precision) Query 134: 0.3065 −0.5 Query 135: 0.0487 Query 136: 0.6532 Query 137: 0.0583 Query 138: 0.1251 Query 139: 0.1254 −1.0 Query 140: 0.2227 91 100 110 120 130 140 Topic number RALI U Montreal, Multilingual English, Auto, TDN Run run3 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0135 Total number of documents Query 92: 0.1007 over all queries Query 93: 0.0000 Retrieved: 50000 0.8 Query 94: 0.0840 Relevant: 8068 Query 95: 0.0288 Rel_ret: 2129 Query 96: 0.0013 Query 97: 0.0692 Interpolated Recall - 0.6 Query 98: 0.0197 Precision Averages: Query 99: 0.0069 at 0.00 0.6530 Query 100: 0.0257 at 0.10 0.3157 Precision Query 101: 0.0289 at 0.20 0.1838 Query 102: 0.2257 at 0.30 0.0992 0.4 Query 103: 0.0244 at 0.40 0.0544 Query 104: 0.0358 at 0.50 0.0347 Query 105: 0.3385 at 0.60 0.0222 Query 106: 0.0082 at 0.70 0.0129 0.2 Query 107: 0.0491 at 0.80 0.0000 Query 108: 0.0996 at 0.90 0.0000 Query 109: 0.0967 at 1.00 0.0000 Query 110: 0.1414 0.0 Query 111: 0.2501 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.0985 for all rel. documents Query 113: 0.0066 0.0945 Recall Query 114: 0.0390 Precision: Query 115: 0.1042 At 5 docs: 0.4360 Run run3 Query 116: 0.1097 At 10 docs: 0.4000 Comparison to median by topic Query 117: 0.0000 At 15 docs: 0.3640 +1.0 Query 118: 0.0114 At 20 docs: 0.3340 Query 119: 0.1900 At 30 docs: 0.3000 Query 120: 0.1605 At 100 docs: 0.1734 Query 121: 0.0388 At 200 docs: 0.1172 Query 122: 0.0671 At 500 docs: 0.0653 Query 123: 0.2228 At 1000 docs: 0.0426 +0.5 Query 124: 0.0878 Query 125: 0.1936 R-Precision (prec. after R Query 126: 0.0319 (=num_rel) docs. retrieved): Query 127: 0.0018 Exact: 0.1484 Query 128: 0.0115 0.0 Query 129: 0.0374 Query 130: 0.3571 Query 131: 0.0943 Query 132: 0.0001 Query 133: 0.0160 Difference (average precision) Query 134: 0.2089 −0.5 Query 135: 0.0323 Query 136: 0.5483 Query 137: 0.0184 Query 138: 0.0860 Query 139: 0.0884 −1.0 Query 140: 0.2169 91 100 110 120 130 140 Topic number TLR Research, Multilingual English, Auto, TD Run tlren2multi Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0530 Total number of documents Query 92: 0.2557 over all queries Query 93: 0.4387 Retrieved: 50000 0.8 Query 94: 0.1622 Relevant: 8068 Query 95: 0.0511 Rel_ret: 4190 Query 96: 0.0665 Query 97: 0.1166 Interpolated Recall - 0.6 Query 98: 0.3067 Precision Averages: Query 99: 0.0142 at 0.00 0.7051 Query 100: 0.4408 at 0.10 0.4309 Precision Query 101: 0.1732 at 0.20 0.3625 Query 102: 0.3560 at 0.30 0.2883 0.4 Query 103: 0.4059 at 0.40 0.2265 Query 104: 0.1041 at 0.50 0.1891 Query 105: 0.4802 at 0.60 0.1454 Query 106: 0.1596 at 0.70 0.0974 0.2 Query 107: 0.1111 at 0.80 0.0567 Query 108: 0.0589 at 0.90 0.0235 Query 109: 0.0742 at 1.00 0.0000 Query 110: 0.4216 0.0 Query 111: 0.0409 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4052 for all rel. documents Query 113: 0.0084 0.2049 Recall Query 114: 0.1192 Precision: Query 115: 0.2706 At 5 docs: 0.4680 Run tlren2multi Query 116: 0.2921 At 10 docs: 0.4560 Comparison to median by topic Query 117: 0.1411 At 15 docs: 0.4413 +1.0 Query 118: 0.0325 At 20 docs: 0.4140 Query 119: 0.5910 At 30 docs: 0.3907 Query 120: 0.0870 At 100 docs: 0.2752 Query 121: 0.1085 At 200 docs: 0.2116 Query 122: 0.0766 At 500 docs: 0.1326 Query 123: 0.6074 At 1000 docs: 0.0838 +0.5 Query 124: 0.0491 Query 125: 0.0639 R-Precision (prec. after R Query 126: 0.2128 (=num_rel) docs. retrieved): Query 127: 0.1127 Exact: 0.2803 Query 128: 0.0678 0.0 Query 129: 0.1824 Query 130: 0.2593 Query 131: 0.3080 Query 132: 0.1187 Query 133: 0.0297 Difference (average precision) Query 134: 0.5389 −0.5 Query 135: 0.1511 Query 136: 0.5796 Query 137: 0.1796 Query 138: 0.2012 Query 139: 0.0119 −1.0 Query 140: 0.1461 91 100 110 120 130 140 Topic number Univ. Tampere, Dept. IS, Multilingual English, Auto, TD Run tremu1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0047 Total number of documents Query 92: 0.1111 over all queries Query 93: 0.3576 Retrieved: 50000 0.8 Query 94: 0.1837 Relevant: 8068 Query 95: 0.0413 Rel_ret: 2870 Query 96: 0.0003 Query 97: 0.0330 Interpolated Recall - 0.6 Query 98: 0.0507 Precision Averages: Query 99: 0.0272 at 0.00 0.7657 Query 100: 0.2186 at 0.10 0.3988 Precision Query 101: 0.0000 at 0.20 0.2875 Query 102: 0.5006 at 0.30 0.2155 0.4 Query 103: 0.2396 at 0.40 0.1548 Query 104: 0.1141 at 0.50 0.1128 Query 105: 0.2358 at 0.60 0.0938 Query 106: 0.0258 at 0.70 0.0629 0.2 Query 107: 0.0863 at 0.80 0.0410 Query 108: 0.1329 at 0.90 0.0154 Query 109: 0.1387 at 1.00 0.0000 Query 110: 0.2446 0.0 Query 111: 0.1921 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.3474 for all rel. documents Query 113: 0.0018 0.1637 Recall Query 114: 0.1118 Precision: Query 115: 0.1629 At 5 docs: 0.4920 Run tremu1 Query 116: 0.2287 At 10 docs: 0.4500 Comparison to median by topic Query 117: 0.0020 At 15 docs: 0.4280 +1.0 Query 118: 0.0041 At 20 docs: 0.3950 Query 119: 0.1864 At 30 docs: 0.3460 Query 120: 0.0893 At 100 docs: 0.2202 Query 121: 0.3633 At 200 docs: 0.1578 Query 122: 0.0594 At 500 docs: 0.0940 Query 123: 0.7505 At 1000 docs: 0.0574 +0.5 Query 124: 0.0142 Query 125: 0.0528 R-Precision (prec. after R Query 126: 0.0961 (=num_rel) docs. retrieved): Query 127: 0.2630 Exact: 0.2250 Query 128: 0.0087 0.0 Query 129: 0.1241 Query 130: 0.3200 Query 131: 0.0775 Query 132: 0.2980 Query 133: 0.0260 Difference (average precision) Query 134: 0.7924 −0.5 Query 135: 0.0498 Query 136: 0.6076 Query 137: 0.0187 Query 138: 0.0425 Query 139: 0.0120 −1.0 Query 140: 0.1360 91 100 110 120 130 140 Topic number Univ. Tampere, Dept. IS, Multilingual English, Auto, TD Run tremu2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0022 Total number of documents Query 92: 0.0666 over all queries Query 93: 0.2606 Retrieved: 50000 0.8 Query 94: 0.0800 Relevant: 8068 Query 95: 0.0104 Rel_ret: 2696 Query 96: 0.0071 Query 97: 0.0426 Interpolated Recall - 0.6 Query 98: 0.0600 Precision Averages: Query 99: 0.0187 at 0.00 0.7356 Query 100: 0.2180 at 0.10 0.2702 Precision Query 101: 0.0000 at 0.20 0.2142 Query 102: 0.2287 at 0.30 0.1597 0.4 Query 103: 0.2434 at 0.40 0.1186 Query 104: 0.0224 at 0.50 0.0805 Query 105: 0.1242 at 0.60 0.0637 Query 106: 0.0203 at 0.70 0.0438 0.2 Query 107: 0.0418 at 0.80 0.0236 Query 108: 0.0485 at 0.90 0.0050 Query 109: 0.0829 at 1.00 0.0000 Query 110: 0.1839 0.0 Query 111: 0.0733 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4331 for all rel. documents Query 113: 0.0007 0.1166 Recall Query 114: 0.0993 Precision: Query 115: 0.1716 At 5 docs: 0.3400 Run tremu2 Query 116: 0.1806 At 10 docs: 0.3240 Comparison to median by topic Query 117: 0.0037 At 15 docs: 0.2827 +1.0 Query 118: 0.0089 At 20 docs: 0.2700 Query 119: 0.1126 At 30 docs: 0.2433 Query 120: 0.0492 At 100 docs: 0.1748 Query 121: 0.0726 At 200 docs: 0.1301 Query 122: 0.0409 At 500 docs: 0.0812 Query 123: 0.5170 At 1000 docs: 0.0539 +0.5 Query 124: 0.0161 Query 125: 0.0655 R-Precision (prec. after R Query 126: 0.1001 (=num_rel) docs. retrieved): Query 127: 0.0320 Exact: 0.1848 Query 128: 0.0081 0.0 Query 129: 0.1360 Query 130: 0.2280 Query 131: 0.0686 Query 132: 0.2137 Query 133: 0.0105 Difference (average precision) Query 134: 0.7688 −0.5 Query 135: 0.0458 Query 136: 0.3963 Query 137: 0.0104 Query 138: 0.0196 Query 139: 0.0078 −1.0 Query 140: 0.1748 91 100 110 120 130 140 Topic number Univ. Jaen/SINAI, Multilingual English, Auto, TD Run UJAMLTD2RSV2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0234 Total number of documents Query 92: 0.3787 over all queries Query 93: 0.6437 Retrieved: 50000 0.8 Query 94: 0.2796 Relevant: 8068 Query 95: 0.1604 Rel_ret: 4532 Query 96: 0.1586 Query 97: 0.1326 Interpolated Recall - 0.6 Query 98: 0.4864 Precision Averages: Query 99: 0.0623 at 0.00 0.8224 Query 100: 0.5033 at 0.10 0.5431 Precision Query 101: 0.0354 at 0.20 0.4407 Query 102: 0.5434 at 0.30 0.3740 0.4 Query 103: 0.2406 at 0.40 0.3121 Query 104: 0.0935 at 0.50 0.2626 Query 105: 0.5900 at 0.60 0.1954 Query 106: 0.0777 at 0.70 0.1519 0.2 Query 107: 0.1153 at 0.80 0.1039 Query 108: 0.3325 at 0.90 0.0610 Query 109: 0.0810 at 1.00 0.0038 Query 110: 0.4143 0.0 Query 111: 0.1546 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.3125 for all rel. documents Query 113: 0.0004 0.2758 Recall Query 114: 0.2098 Precision: Query 115: 0.0805 At 5 docs: 0.6320 Run UJAMLTD2RSV2 Query 116: 0.5888 At 10 docs: 0.5860 Comparison to median by topic Query 117: 0.1695 At 15 docs: 0.5573 +1.0 Query 118: 0.0521 At 20 docs: 0.5260 Query 119: 0.5180 At 30 docs: 0.4920 Query 120: 0.0431 At 100 docs: 0.3346 Query 121: 0.6492 At 200 docs: 0.2458 Query 122: 0.1342 At 500 docs: 0.1454 Query 123: 0.8699 At 1000 docs: 0.0906 +0.5 Query 124: 0.0790 Query 125: 0.0176 R-Precision (prec. after R Query 126: 0.1575 (=num_rel) docs. retrieved): Query 127: 0.6545 Exact: 0.3265 Query 128: 0.0362 0.0 Query 129: 0.2961 Query 130: 0.3891 Query 131: 0.2371 Query 132: 0.3438 Query 133: 0.0073 Difference (average precision) Query 134: 0.7701 −0.5 Query 135: 0.1481 Query 136: 0.8813 Query 137: 0.0587 Query 138: 0.1030 Query 139: 0.1286 −1.0 Query 140: 0.3473 91 100 110 120 130 140 Topic number Univ. Jaen/SINAI, Multilingual English, Auto, TD Run UJAMLTDNORM Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0787 Total number of documents Query 92: 0.2863 over all queries Query 93: 0.0932 Retrieved: 50000 0.8 Query 94: 0.1180 Relevant: 8068 Query 95: 0.0898 Rel_ret: 4297 Query 96: 0.0507 Query 97: 0.1601 Interpolated Recall - 0.6 Query 98: 0.0661 Precision Averages: Query 99: 0.0815 at 0.00 0.7230 Query 100: 0.5374 at 0.10 0.3875 Precision Query 101: 0.0148 at 0.20 0.3290 Query 102: 0.4508 at 0.30 0.2874 0.4 Query 103: 0.1601 at 0.40 0.2409 Query 104: 0.0442 at 0.50 0.1967 Query 105: 0.4255 at 0.60 0.1495 Query 106: 0.0794 at 0.70 0.1184 0.2 Query 107: 0.1385 at 0.80 0.0754 Query 108: 0.1213 at 0.90 0.0454 Query 109: 0.0297 at 1.00 0.0000 Query 110: 0.4763 0.0 Query 111: 0.0328 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.3886 for all rel. documents Query 113: 0.0005 0.2068 Recall Query 114: 0.1214 Precision: Query 115: 0.0550 At 5 docs: 0.4840 Run UJAMLTDNORM Query 116: 0.5832 At 10 docs: 0.4260 Comparison to median by topic Query 117: 0.1570 At 15 docs: 0.4093 +1.0 Query 118: 0.0282 At 20 docs: 0.3850 Query 119: 0.5726 At 30 docs: 0.3647 Query 120: 0.0817 At 100 docs: 0.2726 Query 121: 0.4673 At 200 docs: 0.2105 Query 122: 0.0695 At 500 docs: 0.1337 Query 123: 0.7724 At 1000 docs: 0.0859 +0.5 Query 124: 0.0478 Query 125: 0.0497 R-Precision (prec. after R Query 126: 0.1261 (=num_rel) docs. retrieved): Query 127: 0.4476 Exact: 0.2647 Query 128: 0.0124 0.0 Query 129: 0.1020 Query 130: 0.2910 Query 131: 0.1859 Query 132: 0.0506 Query 133: 0.0189 Difference (average precision) Query 134: 0.7098 −0.5 Query 135: 0.0926 Query 136: 0.7347 Query 137: 0.0269 Query 138: 0.1583 Query 139: 0.1901 −1.0 Query 140: 0.2633 91 100 110 120 130 140 Topic number Univ. Jaen/SINAI, Multilingual English, Auto, TD Run UJAMLTDRR Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0499 Total number of documents Query 92: 0.2657 over all queries Query 93: 0.4026 Retrieved: 50000 0.8 Query 94: 0.1543 Relevant: 8068 Query 95: 0.1070 Rel_ret: 4246 Query 96: 0.0603 Query 97: 0.1673 Interpolated Recall - 0.6 Query 98: 0.1902 Precision Averages: Query 99: 0.1063 at 0.00 0.7200 Query 100: 0.4681 at 0.10 0.4239 Precision Query 101: 0.0175 at 0.20 0.3474 Query 102: 0.4931 at 0.30 0.2981 0.4 Query 103: 0.1933 at 0.40 0.2333 Query 104: 0.0475 at 0.50 0.1889 Query 105: 0.3782 at 0.60 0.1340 Query 106: 0.0676 at 0.70 0.0894 0.2 Query 107: 0.1429 at 0.80 0.0536 Query 108: 0.1796 at 0.90 0.0307 Query 109: 0.0219 at 1.00 0.0000 Query 110: 0.4404 0.0 Query 111: 0.0582 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4489 for all rel. documents Query 113: 0.0015 0.2038 Recall Query 114: 0.1077 Precision: Query 115: 0.0902 At 5 docs: 0.4840 Run UJAMLTDRR Query 116: 0.5303 At 10 docs: 0.4560 Comparison to median by topic Query 117: 0.1357 At 15 docs: 0.4333 +1.0 Query 118: 0.0417 At 20 docs: 0.4180 Query 119: 0.6267 At 30 docs: 0.3853 Query 120: 0.0992 At 100 docs: 0.2828 Query 121: 0.1842 At 200 docs: 0.2153 Query 122: 0.0822 At 500 docs: 0.1305 Query 123: 0.6638 At 1000 docs: 0.0849 +0.5 Query 124: 0.0356 Query 125: 0.0374 R-Precision (prec. after R Query 126: 0.1835 (=num_rel) docs. retrieved): Query 127: 0.1816 Exact: 0.2787 Query 128: 0.0141 0.0 Query 129: 0.1792 Query 130: 0.2774 Query 131: 0.2316 Query 132: 0.1216 Query 133: 0.0241 Difference (average precision) Query 134: 0.6395 −0.5 Query 135: 0.0970 Query 136: 0.5544 Query 137: 0.0311 Query 138: 0.1533 Query 139: 0.1552 −1.0 Query 140: 0.2477 91 100 110 120 130 140 Topic number Univ. Jaen/SINAI, Multilingual English, Auto, TD Run UJAMLTDRSV2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0188 Total number of documents Query 92: 0.3857 over all queries Query 93: 0.6444 Retrieved: 50000 0.8 Query 94: 0.2821 Relevant: 8068 Query 95: 0.1570 Rel_ret: 4551 Query 96: 0.1615 Query 97: 0.1363 Interpolated Recall - 0.6 Query 98: 0.4971 Precision Averages: Query 99: 0.0668 at 0.00 0.8080 Query 100: 0.5437 at 0.10 0.5375 Precision Query 101: 0.0373 at 0.20 0.4411 Query 102: 0.5229 at 0.30 0.3743 0.4 Query 103: 0.2535 at 0.40 0.3175 Query 104: 0.0758 at 0.50 0.2662 Query 105: 0.5913 at 0.60 0.1991 Query 106: 0.0822 at 0.70 0.1547 0.2 Query 107: 0.1051 at 0.80 0.1062 Query 108: 0.3229 at 0.90 0.0647 Query 109: 0.0746 at 1.00 0.0041 Query 110: 0.4288 0.0 Query 111: 0.1477 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.3282 for all rel. documents Query 113: 0.0003 0.2774 Recall Query 114: 0.2132 Precision: Query 115: 0.0773 At 5 docs: 0.6320 Run UJAMLTDRSV2 Query 116: 0.6051 At 10 docs: 0.5900 Comparison to median by topic Query 117: 0.1582 At 15 docs: 0.5533 +1.0 Query 118: 0.0491 At 20 docs: 0.5290 Query 119: 0.5062 At 30 docs: 0.4907 Query 120: 0.0423 At 100 docs: 0.3356 Query 121: 0.6519 At 200 docs: 0.2461 Query 122: 0.1493 At 500 docs: 0.1463 Query 123: 0.8622 At 1000 docs: 0.0910 +0.5 Query 124: 0.0789 Query 125: 0.0192 R-Precision (prec. after R Query 126: 0.1552 (=num_rel) docs. retrieved): Query 127: 0.6634 Exact: 0.3280 Query 128: 0.0369 0.0 Query 129: 0.2950 Query 130: 0.3797 Query 131: 0.2454 Query 132: 0.3534 Query 133: 0.0089 Difference (average precision) Query 134: 0.7823 −0.5 Query 135: 0.1572 Query 136: 0.8792 Query 137: 0.0470 Query 138: 0.1227 Query 139: 0.1216 −1.0 Query 140: 0.3475 91 100 110 120 130 140 Topic number Univ. Jaen/SINAI, Multilingual English, Auto, TD Run UJAMLTDRSV2RR Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0106 Total number of documents Query 92: 0.1814 over all queries Query 93: 0.0683 Retrieved: 25267 0.8 Query 94: 0.1019 Relevant: 8068 Query 95: 0.0786 Rel_ret: 3684 Query 96: 0.0548 Query 97: 0.0448 Interpolated Recall - 0.6 Query 98: 0.0774 Precision Averages: Query 99: 0.0270 at 0.00 0.4109 Query 100: 0.2228 at 0.10 0.2253 Precision Query 101: 0.0188 at 0.20 0.2007 Query 102: 0.1179 at 0.30 0.1738 0.4 Query 103: 0.1212 at 0.40 0.1390 Query 104: 0.0208 at 0.50 0.0977 Query 105: 0.1537 at 0.60 0.0617 Query 106: 0.0406 at 0.70 0.0520 0.2 Query 107: 0.0376 at 0.80 0.0191 Query 108: 0.1044 at 0.90 0.0165 Query 109: 0.0124 at 1.00 0.0000 Query 110: 0.3698 0.0 Query 111: 0.0136 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.1098 for all rel. documents Query 113: 0.0008 0.0966 Recall Query 114: 0.0741 Precision: Query 115: 0.0198 At 5 docs: 0.1520 Run UJAMLTDRSV2RR Query 116: 0.1430 At 10 docs: 0.1480 Comparison to median by topic Query 117: 0.1229 At 15 docs: 0.1427 +1.0 Query 118: 0.0136 At 20 docs: 0.1460 Query 119: 0.3829 At 30 docs: 0.1487 Query 120: 0.0341 At 100 docs: 0.1424 Query 121: 0.3355 At 200 docs: 0.1522 Query 122: 0.0464 At 500 docs: 0.1341 Query 123: 0.1454 At 1000 docs: 0.0737 +0.5 Query 124: 0.0199 Query 125: 0.0104 R-Precision (prec. after R Query 126: 0.0309 (=num_rel) docs. retrieved): Query 127: 0.3086 Exact: 0.1485 Query 128: 0.0138 0.0 Query 129: 0.0505 Query 130: 0.1082 Query 131: 0.1372 Query 132: 0.0288 Query 133: 0.0034 Difference (average precision) Query 134: 0.0322 −0.5 Query 135: 0.1040 Query 136: 0.1684 Query 137: 0.0094 Query 138: 0.0717 Query 139: 0.2489 −1.0 Query 140: 0.1750 91 100 110 120 130 140 Topic number Université de Neuchâtel, Multilingual English, Auto, TD Run UniNEm1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0968 Total number of documents Query 92: 0.3844 over all queries Query 93: 0.5100 Retrieved: 50000 0.8 Query 94: 0.5275 Relevant: 8068 Query 95: 0.1579 Rel_ret: 5507 Query 96: 0.2291 Query 97: 0.1475 Interpolated Recall - 0.6 Query 98: 0.4470 Precision Averages: Query 99: 0.1682 at 0.00 0.8196 Query 100: 0.5104 at 0.10 0.6078 Precision Query 101: 0.1677 at 0.20 0.5201 Query 102: 0.5105 at 0.30 0.4638 0.4 Query 103: 0.5268 at 0.40 0.4047 Query 104: 0.1349 at 0.50 0.3462 Query 105: 0.6043 at 0.60 0.3033 Query 106: 0.1868 at 0.70 0.2455 0.2 Query 107: 0.0998 at 0.80 0.1843 Query 108: 0.3605 at 0.90 0.1233 Query 109: 0.1046 at 1.00 0.0164 Query 110: 0.6326 0.0 Query 111: 0.1535 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5295 for all rel. documents Query 113: 0.0017 0.3488 Recall Query 114: 0.3601 Precision: Query 115: 0.3340 At 5 docs: 0.6240 Run UniNEm1 Query 116: 0.6550 At 10 docs: 0.6020 Comparison to median by topic Query 117: 0.1869 At 15 docs: 0.5800 +1.0 Query 118: 0.0277 At 20 docs: 0.5700 Query 119: 0.7652 At 30 docs: 0.5273 Query 120: 0.2262 At 100 docs: 0.3894 Query 121: 0.5418 At 200 docs: 0.2979 Query 122: 0.1143 At 500 docs: 0.1776 Query 123: 0.8598 At 1000 docs: 0.1101 +0.5 Query 124: 0.2481 Query 125: 0.3595 R-Precision (prec. after R Query 126: 0.2655 (=num_rel) docs. retrieved): Query 127: 0.5241 Exact: 0.3909 Query 128: 0.0670 0.0 Query 129: 0.3029 Query 130: 0.0000 Query 131: 0.0000 Query 132: 0.5585 Query 133: 0.1254 Difference (average precision) Query 134: 0.8883 −0.5 Query 135: 0.5668 Query 136: 0.9214 Query 137: 0.2654 Query 138: 0.3117 Query 139: 0.4250 −1.0 Query 140: 0.3456 91 100 110 120 130 140 Topic number Université de Neuchâtel, Multilingual English, Auto, TD Run UniNEm2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.3061 Total number of documents Query 92: 0.3940 over all queries Query 93: 0.7223 Retrieved: 50000 0.8 Query 94: 0.5416 Relevant: 8068 Query 95: 0.2018 Rel_ret: 5650 Query 96: 0.3469 Query 97: 0.1843 Interpolated Recall - 0.6 Query 98: 0.5620 Precision Averages: Query 99: 0.2037 at 0.00 0.8638 Query 100: 0.6002 at 0.10 0.6362 Precision Query 101: 0.5278 at 0.20 0.5571 Query 102: 0.4847 at 0.30 0.5009 0.4 Query 103: 0.6048 at 0.40 0.4535 Query 104: 0.1181 at 0.50 0.3894 Query 105: 0.5967 at 0.60 0.3334 Query 106: 0.2085 at 0.70 0.2681 0.2 Query 107: 0.0836 at 0.80 0.1967 Query 108: 0.3703 at 0.90 0.1187 Query 109: 0.0878 at 1.00 0.0193 Query 110: 0.5951 0.0 Query 111: 0.1167 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4977 for all rel. documents Query 113: 0.0013 0.3783 Recall Query 114: 0.3577 Precision: Query 115: 0.3158 At 5 docs: 0.7040 Run UniNEm2 Query 116: 0.6527 At 10 docs: 0.6640 Comparison to median by topic Query 117: 0.2561 At 15 docs: 0.6187 +1.0 Query 118: 0.0283 At 20 docs: 0.5960 Query 119: 0.7833 At 30 docs: 0.5600 Query 120: 0.1655 At 100 docs: 0.4154 Query 121: 0.4313 At 200 docs: 0.3145 Query 122: 0.1161 At 500 docs: 0.1882 Query 123: 0.8474 At 1000 docs: 0.1130 +0.5 Query 124: 0.2500 Query 125: 0.3887 R-Precision (prec. after R Query 126: 0.2721 (=num_rel) docs. retrieved): Query 127: 0.6995 Exact: 0.4141 Query 128: 0.0880 0.0 Query 129: 0.3694 Query 130: 0.0000 Query 131: 0.0000 Query 132: 0.6721 Query 133: 0.1727 Difference (average precision) Query 134: 0.8481 −0.5 Query 135: 0.6281 Query 136: 0.8519 Query 137: 0.2395 Query 138: 0.3058 Query 139: 0.4404 −1.0 Query 140: 0.3778 91 100 110 120 130 140 Topic number Université de Neuchâtel, Multilingual English, Auto, TD Run UniNEm3 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0726 Total number of documents Query 92: 0.4029 over all queries Query 93: 0.5171 Retrieved: 50000 0.8 Query 94: 0.5347 Relevant: 8068 Query 95: 0.1516 Rel_ret: 5511 Query 96: 0.2141 Query 97: 0.0990 Interpolated Recall - 0.6 Query 98: 0.4651 Precision Averages: Query 99: 0.1685 at 0.00 0.8190 Query 100: 0.5103 at 0.10 0.6027 Precision Query 101: 0.1907 at 0.20 0.5183 Query 102: 0.5236 at 0.30 0.4717 0.4 Query 103: 0.5218 at 0.40 0.4063 Query 104: 0.1470 at 0.50 0.3537 Query 105: 0.5973 at 0.60 0.3062 Query 106: 0.1887 at 0.70 0.2477 0.2 Query 107: 0.1015 at 0.80 0.1857 Query 108: 0.3485 at 0.90 0.1245 Query 109: 0.1414 at 1.00 0.0163 Query 110: 0.6093 0.0 Query 111: 0.1633 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5078 for all rel. documents Query 113: 0.0020 0.3512 Recall Query 114: 0.3520 Precision: Query 115: 0.2891 At 5 docs: 0.5960 Run UniNEm3 Query 116: 0.6574 At 10 docs: 0.6040 Comparison to median by topic Query 117: 0.1911 At 15 docs: 0.5800 +1.0 Query 118: 0.0229 At 20 docs: 0.5730 Query 119: 0.7759 At 30 docs: 0.5333 Query 120: 0.2485 At 100 docs: 0.3918 Query 121: 0.5721 At 200 docs: 0.2979 Query 122: 0.1173 At 500 docs: 0.1798 Query 123: 0.8546 At 1000 docs: 0.1102 +0.5 Query 124: 0.2516 Query 125: 0.4395 R-Precision (prec. after R Query 126: 0.2387 (=num_rel) docs. retrieved): Query 127: 0.5065 Exact: 0.3970 Query 128: 0.1169 0.0 Query 129: 0.3098 Query 130: 0.0000 Query 131: 0.0000 Query 132: 0.5577 Query 133: 0.1209 Difference (average precision) Query 134: 0.8784 −0.5 Query 135: 0.5522 Query 136: 0.9279 Query 137: 0.2555 Query 138: 0.3262 Query 139: 0.4760 −1.0 Query 140: 0.3452 91 100 110 120 130 140 Topic number Université de Neuchâtel, Multilingual English, Auto, TD Run UniNEm4 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.2170 Total number of documents Query 92: 0.4157 over all queries Query 93: 0.7120 Retrieved: 50000 0.8 Query 94: 0.5413 Relevant: 8068 Query 95: 0.1801 Rel_ret: 5633 Query 96: 0.2780 Query 97: 0.1107 Interpolated Recall - 0.6 Query 98: 0.5832 Precision Averages: Query 99: 0.2407 at 0.00 0.8375 Query 100: 0.5888 at 0.10 0.6287 Precision Query 101: 0.4765 at 0.20 0.5520 Query 102: 0.5032 at 0.30 0.4996 0.4 Query 103: 0.5827 at 0.40 0.4466 Query 104: 0.1245 at 0.50 0.3886 Query 105: 0.5707 at 0.60 0.3332 Query 106: 0.2209 at 0.70 0.2626 0.2 Query 107: 0.0810 at 0.80 0.1905 Query 108: 0.3557 at 0.90 0.1166 Query 109: 0.1032 at 1.00 0.0186 Query 110: 0.5992 0.0 Query 111: 0.1306 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4831 for all rel. documents Query 113: 0.0012 0.3756 Recall Query 114: 0.3503 Precision: Query 115: 0.3037 At 5 docs: 0.6760 Run UniNEm4 Query 116: 0.6737 At 10 docs: 0.6740 Comparison to median by topic Query 117: 0.2612 At 15 docs: 0.6067 +1.0 Query 118: 0.0264 At 20 docs: 0.5850 Query 119: 0.7821 At 30 docs: 0.5547 Query 120: 0.1641 At 100 docs: 0.4154 Query 121: 0.4506 At 200 docs: 0.3131 Query 122: 0.1144 At 500 docs: 0.1884 Query 123: 0.8562 At 1000 docs: 0.1127 +0.5 Query 124: 0.2676 Query 125: 0.4549 R-Precision (prec. after R Query 126: 0.2790 (=num_rel) docs. retrieved): Query 127: 0.6845 Exact: 0.4091 Query 128: 0.1079 0.0 Query 129: 0.3410 Query 130: 0.0000 Query 131: 0.0000 Query 132: 0.6491 Query 133: 0.1573 Difference (average precision) Query 134: 0.8293 −0.5 Query 135: 0.6328 Query 136: 0.8679 Query 137: 0.2278 Query 138: 0.2961 Query 139: 0.5293 −1.0 Query 140: 0.3723 91 100 110 120 130 140 Topic number Université de Neuchâtel, Multilingual English, Auto, TD Run UniNEm5 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1699 Total number of documents Query 92: 0.4133 over all queries Query 93: 0.4725 Retrieved: 50000 0.8 Query 94: 0.4265 Relevant: 8068 Query 95: 0.1729 Rel_ret: 5801 Query 96: 0.2252 Query 97: 0.3662 Interpolated Recall - 0.6 Query 98: 0.3514 Precision Averages: Query 99: 0.1680 at 0.00 0.8613 Query 100: 0.4974 at 0.10 0.6227 Precision Query 101: 0.2375 at 0.20 0.5483 Query 102: 0.4865 at 0.30 0.4745 0.4 Query 103: 0.5085 at 0.40 0.4094 Query 104: 0.1593 at 0.50 0.3564 Query 105: 0.6290 at 0.60 0.2963 Query 106: 0.1593 at 0.70 0.2494 0.2 Query 107: 0.1543 at 0.80 0.1841 Query 108: 0.4108 at 0.90 0.1101 Query 109: 0.1358 at 1.00 0.0135 Query 110: 0.6343 0.0 Query 111: 0.2199 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5725 for all rel. documents Query 113: 0.0043 0.3552 Recall Query 114: 0.2444 Precision: Query 115: 0.2398 At 5 docs: 0.6120 Run UniNEm5 Query 116: 0.6590 At 10 docs: 0.6200 Comparison to median by topic Query 117: 0.2325 At 15 docs: 0.5920 +1.0 Query 118: 0.0243 At 20 docs: 0.5740 Query 119: 0.7780 At 30 docs: 0.5360 Query 120: 0.1600 At 100 docs: 0.3992 Query 121: 0.5127 At 200 docs: 0.3089 Query 122: 0.1082 At 500 docs: 0.1893 Query 123: 0.8302 At 1000 docs: 0.1160 +0.5 Query 124: 0.1283 Query 125: 0.3151 R-Precision (prec. after R Query 126: 0.4050 (=num_rel) docs. retrieved): Query 127: 0.6154 Exact: 0.4031 Query 128: 0.1068 0.0 Query 129: 0.3434 Query 130: 0.1243 Query 131: 0.1412 Query 132: 0.3957 Query 133: 0.1745 Difference (average precision) Query 134: 0.8464 −0.5 Query 135: 0.5502 Query 136: 0.9094 Query 137: 0.2264 Query 138: 0.3887 Query 139: 0.4550 −1.0 Query 140: 0.2707 91 100 110 120 130 140 Topic number JHU/APL, Bilingual English to German, Auto, TD Run aplbiende Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1548 Total number of documents Query 92: 0.3229 over all queries Query 93: 0.0000 Retrieved: 50000 0.8 Query 94: 0.7985 Relevant: 1938 Query 95: 0.1881 Rel_ret: 1535 Query 96: 0.0044 Query 97: 0.0279 Interpolated Recall - Query 98: 0.0146 Precision Averages: 0.6 Query 99: 0.0681 at 0.00 0.5783 Query 100: 0.7147 at 0.10 0.5147 Query 101: 0.7386 at 0.20 0.4335 Precision Query 102: 0.2885 at 0.30 0.4033 0.4 Query 103: 0.2174 at 0.40 0.3845 Query 104: 0.1510 at 0.50 0.3377 Query 105: 0.6923 at 0.60 0.3049 Query 106: 0.0981 at 0.70 0.2368 0.2 Query 107: 0.0474 at 0.80 0.1827 Query 108: 0.3788 at 0.90 0.1408 Query 109: 0.5286 at 1.00 0.0783 Query 110: 0.5761 0.0 Query 111: 0.0431 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.0026 for all rel. documents Query 113: 0.0860 0.3137 Recall Query 114: 0.4198 Precision: Query 115: 0.2189 At 5 docs: 0.4160 Run aplbiende Query 116: 0.3118 At 10 docs: 0.3960 Query 117: 0.0156 At 15 docs: 0.3613 Comparison to median by topic Query 118: 0.0106 At 20 docs: 0.3370 +1.0 Query 119: 0.7254 At 30 docs: 0.3087 Query 120: 0.0421 At 100 docs: 0.1688 Query 121: 0.7778 At 200 docs: 0.1042 Query 122: 0.0697 At 500 docs: 0.0523 Query 123: 0.8955 At 1000 docs: 0.0307 +0.5 Query 124: 0.0810 Query 125: 0.3966 R-Precision (prec. after R Query 126: 0.0246 (=num_rel) docs. retrieved): Query 127: 0.1718 Exact: 0.3224 Query 128: 0.0108 0.0 Query 129: 0.5368 Query 130: 0.7446 Query 131: 0.6427 Query 132: 0.9152 Query 133: 0.0040 Difference (average precision) Query 134: 0.0693 −0.5 Query 135: 0.5230 Query 136: 0.6203 Query 137: 0.0014 Query 138: 0.6250 Query 139: 0.2924 −1.0 Query 140: 0.3955 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 1, Bilingual English to German, Auto, TDN Run BKBIEG1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.4179 Total number of documents Query 92: 0.2481 over all queries Query 93: 0.8737 Retrieved: 50000 0.8 Query 94: 0.0040 Relevant: 1938 Query 95: 0.1624 Rel_ret: 1628 Query 96: 0.5341 Query 97: 0.6633 Interpolated Recall - 0.6 Query 98: 0.6400 Precision Averages: Query 99: 0.0515 at 0.00 0.7108 Query 100: 0.5899 at 0.10 0.6190 Precision Query 101: 0.5730 at 0.20 0.5594 Query 102: 0.4682 at 0.30 0.5207 0.4 Query 103: 0.5153 at 0.40 0.4741 Query 104: 0.1166 at 0.50 0.4358 Query 105: 0.6316 at 0.60 0.4090 Query 106: 0.0496 at 0.70 0.3647 0.2 Query 107: 0.0350 at 0.80 0.2972 Query 108: 0.3834 at 0.90 0.2204 Query 109: 0.2370 at 1.00 0.1441 Query 110: 0.6333 0.0 Query 111: 0.2629 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6317 for all rel. documents Query 113: 0.2819 0.4150 Recall Query 114: 0.4456 Precision: Query 115: 0.3325 At 5 docs: 0.5320 Run BKBIEG1 Query 116: 0.7387 At 10 docs: 0.5000 Comparison to median by topic Query 117: 0.5594 At 15 docs: 0.4667 +1.0 Query 118: 0.0065 At 20 docs: 0.4260 Query 119: 0.8539 At 30 docs: 0.3753 Query 120: 0.0444 At 100 docs: 0.2074 Query 121: 0.2535 At 200 docs: 0.1268 Query 122: 0.0306 At 500 docs: 0.0602 Query 123: 0.9571 At 1000 docs: 0.0326 +0.5 Query 124: 0.2980 Query 125: 0.5580 R-Precision (prec. after R Query 126: 0.0479 (=num_rel) docs. retrieved): Query 127: 0.8892 Exact: 0.4137 Query 128: 0.1193 0.0 Query 129: 0.6476 Query 130: 0.6743 Query 131: 0.4329 Query 132: 0.7223 Query 133: 0.5709 Difference (average precision) Query 134: 0.6060 −0.5 Query 135: 0.0762 Query 136: 1.0000 Query 137: 0.0164 Query 138: 0.8533 Query 139: 0.0102 −1.0 Query 140: 0.0002 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 1, Bilingual English to German, Auto, TD Run BKBIEG2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.3662 Total number of documents Query 92: 0.2426 over all queries Query 93: 0.8805 Retrieved: 50000 0.8 Query 94: 0.0995 Relevant: 1938 Query 95: 0.0609 Rel_ret: 1661 Query 96: 0.5400 Query 97: 0.2730 Interpolated Recall - 0.6 Query 98: 0.6371 Precision Averages: Query 99: 0.2921 at 0.00 0.6625 Query 100: 0.6037 at 0.10 0.6011 Precision Query 101: 0.7378 at 0.20 0.5595 Query 102: 0.4680 at 0.30 0.5075 0.4 Query 103: 0.5313 at 0.40 0.4642 Query 104: 0.1500 at 0.50 0.4359 Query 105: 0.6717 at 0.60 0.4105 Query 106: 0.0907 at 0.70 0.3588 0.2 Query 107: 0.0959 at 0.80 0.3061 Query 108: 0.3545 at 0.90 0.2172 Query 109: 0.1352 at 1.00 0.1140 Query 110: 0.5794 0.0 Query 111: 0.0620 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6203 for all rel. documents Query 113: 0.0020 0.4060 Recall Query 114: 0.4315 Precision: Query 115: 0.2794 At 5 docs: 0.4800 Run BKBIEG2 Query 116: 0.7100 At 10 docs: 0.4680 Comparison to median by topic Query 117: 0.4698 At 15 docs: 0.4640 +1.0 Query 118: 0.0095 At 20 docs: 0.4330 Query 119: 0.8770 At 30 docs: 0.3860 Query 120: 0.0274 At 100 docs: 0.2168 Query 121: 0.4556 At 200 docs: 0.1331 Query 122: 0.0804 At 500 docs: 0.0624 Query 123: 0.8833 At 1000 docs: 0.0332 +0.5 Query 124: 0.2875 Query 125: 0.5283 R-Precision (prec. after R Query 126: 0.0435 (=num_rel) docs. retrieved): Query 127: 0.8157 Exact: 0.4021 Query 128: 0.0488 0.0 Query 129: 0.7195 Query 130: 0.5810 Query 131: 0.5522 Query 132: 0.7353 Query 133: 0.4913 Difference (average precision) Query 134: 0.5667 −0.5 Query 135: 0.2821 Query 136: 0.9683 Query 137: 0.0417 Query 138: 0.8514 Query 139: 0.0685 −1.0 Query 140: 0.0005 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 1, Bilingual French to German, Auto, TDN Run BKBIFG1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.3936 Total number of documents Query 92: 0.2950 over all queries Query 93: 0.8577 Retrieved: 50000 0.8 Query 94: 0.1738 Relevant: 1938 Query 95: 0.1861 Rel_ret: 1798 Query 96: 0.6107 Query 97: 0.7112 Interpolated Recall - 0.6 Query 98: 0.6184 Precision Averages: Query 99: 0.5770 at 0.00 0.8141 Query 100: 0.5416 at 0.10 0.7345 Precision Query 101: 0.6448 at 0.20 0.6959 Query 102: 0.3962 at 0.30 0.5947 0.4 Query 103: 0.4106 at 0.40 0.5490 Query 104: 0.1553 at 0.50 0.4851 Query 105: 0.6110 at 0.60 0.4465 Query 106: 0.1156 at 0.70 0.3833 0.2 Query 107: 0.0952 at 0.80 0.3084 Query 108: 0.4146 at 0.90 0.2289 Query 109: 0.4032 at 1.00 0.1271 Query 110: 0.6351 0.0 Query 111: 0.2526 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.7129 for all rel. documents Query 113: 0.2167 0.4722 Recall Query 114: 0.4932 Precision: Query 115: 0.4024 At 5 docs: 0.6480 Run BKBIFG1 Query 116: 0.7497 At 10 docs: 0.5860 Comparison to median by topic Query 117: 0.5248 At 15 docs: 0.5280 +1.0 Query 118: 0.2845 At 20 docs: 0.4880 Query 119: 0.8368 At 30 docs: 0.4320 Query 120: 0.0500 At 100 docs: 0.2418 Query 121: 0.3500 At 200 docs: 0.1466 Query 122: 0.0972 At 500 docs: 0.0683 Query 123: 0.8755 At 1000 docs: 0.0360 +0.5 Query 124: 0.5175 Query 125: 0.5445 R-Precision (prec. after R Query 126: 0.4758 (=num_rel) docs. retrieved): Query 127: 0.8864 Exact: 0.4683 Query 128: 0.2852 0.0 Query 129: 0.7183 Query 130: 0.6740 Query 131: 0.5366 Query 132: 0.4720 Query 133: 0.6386 Difference (average precision) Query 134: 0.6573 −0.5 Query 135: 0.3388 Query 136: 1.0000 Query 137: 0.0019 Query 138: 0.7679 Query 139: 0.3312 −1.0 Query 140: 0.0716 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 1, Bilingual French to German, Auto, TD Run BKBIFG2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.4342 Total number of documents Query 92: 0.2700 over all queries Query 93: 0.8806 Retrieved: 50000 0.8 Query 94: 0.4649 Relevant: 1938 Query 95: 0.1893 Rel_ret: 1760 Query 96: 0.5772 Query 97: 0.3412 Interpolated Recall - 0.6 Query 98: 0.6223 Precision Averages: Query 99: 0.5146 at 0.00 0.8122 Query 100: 0.5488 at 0.10 0.6959 Precision Query 101: 0.7378 at 0.20 0.6219 Query 102: 0.4018 at 0.30 0.5565 0.4 Query 103: 0.5301 at 0.40 0.5174 Query 104: 0.1771 at 0.50 0.4596 Query 105: 0.6681 at 0.60 0.4226 Query 106: 0.0730 at 0.70 0.3637 0.2 Query 107: 0.1338 at 0.80 0.3010 Query 108: 0.4459 at 0.90 0.2191 Query 109: 0.0202 at 1.00 0.1256 Query 110: 0.6194 0.0 Query 111: 0.2807 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6168 for all rel. documents Query 113: 0.0279 0.4448 Recall Query 114: 0.4330 Precision: Query 115: 0.4120 At 5 docs: 0.5760 Run BKBIFG2 Query 116: 0.6823 At 10 docs: 0.5500 Comparison to median by topic Query 117: 0.6471 At 15 docs: 0.5013 +1.0 Query 118: 0.0870 At 20 docs: 0.4760 Query 119: 0.8477 At 30 docs: 0.4253 Query 120: 0.0355 At 100 docs: 0.2346 Query 121: 0.3687 At 200 docs: 0.1436 Query 122: 0.1992 At 500 docs: 0.0669 Query 123: 0.8929 At 1000 docs: 0.0352 +0.5 Query 124: 0.4557 Query 125: 0.5511 R-Precision (prec. after R Query 126: 0.0438 (=num_rel) docs. retrieved): Query 127: 0.8913 Exact: 0.4382 Query 128: 0.1776 0.0 Query 129: 0.6855 Query 130: 0.5824 Query 131: 0.5514 Query 132: 0.5506 Query 133: 0.5993 Difference (average precision) Query 134: 0.5612 −0.5 Query 135: 0.2472 Query 136: 1.0000 Query 137: 0.0000 Query 138: 0.7024 Query 139: 0.4059 −1.0 Query 140: 0.0516 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 1, Bilingual Russian to German, Auto, TDN Run BKBIRG1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.2043 Total number of documents Query 92: 0.2434 over all queries Query 93: 0.8388 Retrieved: 50000 0.8 Query 94: 0.0198 Relevant: 1938 Query 95: 0.1652 Rel_ret: 1351 Query 96: 0.4131 Query 97: 0.7095 Interpolated Recall - 0.6 Query 98: 0.5794 Precision Averages: Query 99: 0.0036 at 0.00 0.5638 Query 100: 0.5963 at 0.10 0.5055 Precision Query 101: 0.4726 at 0.20 0.4565 Query 102: 0.4051 at 0.30 0.4141 0.4 Query 103: 0.2644 at 0.40 0.3761 Query 104: 0.0916 at 0.50 0.3408 Query 105: 0.5339 at 0.60 0.3122 Query 106: 0.0968 at 0.70 0.2687 0.2 Query 107: 0.0206 at 0.80 0.2253 Query 108: 0.0000 at 0.90 0.1659 Query 109: 0.3397 at 1.00 0.0927 Query 110: 0.6614 0.0 Query 111: 0.0726 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5901 for all rel. documents Query 113: 0.0002 0.3254 Recall Query 114: 0.5178 Precision: Query 115: 0.3109 At 5 docs: 0.4200 Run BKBIRG1 Query 116: 0.0169 At 10 docs: 0.3900 Comparison to median by topic Query 117: 0.3035 At 15 docs: 0.3760 +1.0 Query 118: 0.0059 At 20 docs: 0.3550 Query 119: 0.8707 At 30 docs: 0.3147 Query 120: 0.0188 At 100 docs: 0.1732 Query 121: 0.1682 At 200 docs: 0.1029 Query 122: 0.0195 At 500 docs: 0.0492 Query 123: 1.0000 At 1000 docs: 0.0270 +0.5 Query 124: 0.0202 Query 125: 0.6788 R-Precision (prec. after R Query 126: 0.4062 (=num_rel) docs. retrieved): Query 127: 0.0077 Exact: 0.3373 Query 128: 0.0076 0.0 Query 129: 0.5968 Query 130: 0.5862 Query 131: 0.7779 Query 132: 0.7904 Query 133: 0.0210 Difference (average precision) Query 134: 0.2952 −0.5 Query 135: 0.4585 Query 136: 0.0000 Query 137: 0.0175 Query 138: 0.8404 Query 139: 0.2110 −1.0 Query 140: 0.0020 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 1, Bilingual Russian to German, Auto, TD Run BKBIRG2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0437 Total number of documents Query 92: 0.2413 over all queries Query 93: 0.0000 Retrieved: 50000 0.8 Query 94: 0.3738 Relevant: 1938 Query 95: 0.1350 Rel_ret: 1260 Query 96: 0.4492 Query 97: 0.1571 Interpolated Recall - 0.6 Query 98: 0.1927 Precision Averages: Query 99: 0.0113 at 0.00 0.5051 Query 100: 0.7601 at 0.10 0.4029 Precision Query 101: 0.7216 at 0.20 0.3779 Query 102: 0.1371 at 0.30 0.3417 0.4 Query 103: 0.2290 at 0.40 0.3202 Query 104: 0.0471 at 0.50 0.2923 Query 105: 0.5826 at 0.60 0.2685 Query 106: 0.1290 at 0.70 0.2375 0.2 Query 107: 0.0186 at 0.80 0.1906 Query 108: 0.0000 at 0.90 0.1366 Query 109: 0.0056 at 1.00 0.0720 Query 110: 0.6535 0.0 Query 111: 0.0010 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6051 for all rel. documents Query 113: 0.0001 0.2691 Recall Query 114: 0.5273 Precision: Query 115: 0.2368 At 5 docs: 0.3440 Run BKBIRG2 Query 116: 0.0000 At 10 docs: 0.3260 Comparison to median by topic Query 117: 0.3834 At 15 docs: 0.3147 +1.0 Query 118: 0.0422 At 20 docs: 0.2970 Query 119: 0.8734 At 30 docs: 0.2707 Query 120: 0.0494 At 100 docs: 0.1530 Query 121: 0.0387 At 200 docs: 0.0938 Query 122: 0.0557 At 500 docs: 0.0453 Query 123: 0.9571 At 1000 docs: 0.0252 +0.5 Query 124: 0.0279 Query 125: 0.6069 R-Precision (prec. after R Query 126: 0.0435 (=num_rel) docs. retrieved): Query 127: 0.0010 Exact: 0.2694 Query 128: 0.0092 0.0 Query 129: 0.7132 Query 130: 0.5686 Query 131: 0.6080 Query 132: 0.7724 Query 133: 0.0004 Difference (average precision) Query 134: 0.0632 −0.5 Query 135: 0.2393 Query 136: 0.0000 Query 137: 0.0256 Query 138: 0.8607 Query 139: 0.2438 −1.0 Query 140: 0.0148 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 2, Bilingual English to German, Auto, TD Run bky2biende Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.3511 Total number of documents Query 92: 0.2645 over all queries Query 93: 0.7878 Retrieved: 50000 0.8 Query 94: 0.8560 Relevant: 1938 Query 95: 0.1395 Rel_ret: 1765 Query 96: 0.3860 Query 97: 0.3596 Interpolated Recall - Query 98: 0.4532 Precision Averages: 0.6 Query 99: 0.1020 at 0.00 0.7487 Query 100: 0.6470 at 0.10 0.6513 Query 101: 0.8043 at 0.20 0.5910 Precision Query 102: 0.4788 at 0.30 0.5558 0.4 Query 103: 0.6446 at 0.40 0.5242 Query 104: 0.0699 at 0.50 0.4838 Query 105: 0.5463 at 0.60 0.4420 Query 106: 0.1507 at 0.70 0.3967 0.2 Query 107: 0.1390 at 0.80 0.3322 Query 108: 0.4004 at 0.90 0.2436 Query 109: 0.3063 at 1.00 0.1564 Query 110: 0.6593 0.0 Query 111: 0.1033 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.8480 for all rel. documents Query 113: 0.0415 0.4479 Recall Query 114: 0.3269 Precision: Query 115: 0.5522 At 5 docs: 0.5760 Run bky2biende Query 116: 0.6110 At 10 docs: 0.5080 Query 117: 0.5818 At 15 docs: 0.4733 Comparison to median by topic Query 118: 0.0293 At 20 docs: 0.4480 +1.0 Query 119: 0.8017 At 30 docs: 0.4113 Query 120: 0.1047 At 100 docs: 0.2284 Query 121: 0.3167 At 200 docs: 0.1403 Query 122: 0.2874 At 500 docs: 0.0659 Query 123: 0.9821 At 1000 docs: 0.0353 +0.5 Query 124: 0.0268 Query 125: 0.3773 R-Precision (prec. after R Query 126: 0.3028 (=num_rel) docs. retrieved): Query 127: 0.9000 Exact: 0.4360 Query 128: 0.0537 0.0 Query 129: 0.6375 Query 130: 0.4588 Query 131: 0.5136 Query 132: 0.7530 Query 133: 0.4399 Difference (average precision) Query 134: 0.8095 −0.5 Query 135: 0.4596 Query 136: 1.0000 Query 137: 0.5000 Query 138: 0.5579 Query 139: 0.2713 −1.0 Query 140: 0.1990 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 2, Bilingual French to German, Auto, TD Run bky2bifrde Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.4163 Total number of documents Query 92: 0.2622 over all queries Query 93: 0.7767 Retrieved: 50000 0.8 Query 94: 0.9697 Relevant: 1938 Query 95: 0.0005 Rel_ret: 1609 Query 96: 0.3048 Query 97: 0.5069 Interpolated Recall - Query 98: 0.5592 Precision Averages: 0.6 Query 99: 0.6327 at 0.00 0.7835 Query 100: 0.6461 at 0.10 0.6801 Query 101: 0.8186 at 0.20 0.6484 Precision Query 102: 0.1861 at 0.30 0.6017 0.4 Query 103: 0.6437 at 0.40 0.5609 Query 104: 0.1624 at 0.50 0.5150 Query 105: 0.6365 at 0.60 0.4756 Query 106: 0.0746 at 0.70 0.4195 0.2 Query 107: 0.2412 at 0.80 0.3491 Query 108: 0.4238 at 0.90 0.2481 Query 109: 0.7795 at 1.00 0.1455 Query 110: 0.6641 0.0 Query 111: 0.0120 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.9006 for all rel. documents Query 113: 0.2640 0.4759 Recall Query 114: 0.3258 Precision: Query 115: 0.5461 At 5 docs: 0.5960 Run bky2bifrde Query 116: 0.6484 At 10 docs: 0.5380 Query 117: 0.4611 At 15 docs: 0.5133 Comparison to median by topic Query 118: 0.2521 At 20 docs: 0.4780 +1.0 Query 119: 0.8801 At 30 docs: 0.4227 Query 120: 0.0398 At 100 docs: 0.2344 Query 121: 0.2921 At 200 docs: 0.1373 Query 122: 0.1589 At 500 docs: 0.0615 Query 123: 0.9683 At 1000 docs: 0.0322 +0.5 Query 124: 0.5115 Query 125: 0.3781 R-Precision (prec. after R Query 126: 0.2715 (=num_rel) docs. retrieved): Query 127: 0.8632 Exact: 0.4790 Query 128: 0.0863 0.0 Query 129: 0.5714 Query 130: 0.5089 Query 131: 0.4508 Query 132: 0.6958 Query 133: 0.4032 Difference (average precision) Query 134: 0.7476 −0.5 Query 135: 0.1141 Query 136: 0.8260 Query 137: 1.0000 Query 138: 0.5534 Query 139: 0.3531 −1.0 Query 140: 0.0035 91 100 110 120 130 140 Topic number University of Amsterdam, Bilingual English to German, Auto, TD Run UAmsC02EnGeNGram Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0467 Total number of documents Query 92: 0.0138 over all queries Query 93: 0.7325 Retrieved: 50000 0.8 Query 94: 0.0399 Relevant: 1938 Query 95: 0.0459 Rel_ret: 1101 Query 96: 0.0063 Query 97: 0.0112 Interpolated Recall - 0.6 Query 98: 0.2719 Precision Averages: Query 99: 0.0491 at 0.00 0.5255 Query 100: 0.7077 at 0.10 0.4665 Precision Query 101: 0.3842 at 0.20 0.3834 Query 102: 0.3895 at 0.30 0.3316 0.4 Query 103: 0.5712 at 0.40 0.2936 Query 104: 0.4064 at 0.50 0.2674 Query 105: 0.4013 at 0.60 0.2327 Query 106: 0.1032 at 0.70 0.1997 0.2 Query 107: 0.1805 at 0.80 0.1604 Query 108: 0.0016 at 0.90 0.0969 Query 109: 0.2205 at 1.00 0.0537 Query 110: 0.6789 0.0 Query 111: 0.0258 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.1124 for all rel. documents Query 113: 0.0000 0.2614 Recall Query 114: 0.4032 Precision: Query 115: 0.2823 At 5 docs: 0.3960 Run UAmsC02EnGeNGram Query 116: 0.8573 At 10 docs: 0.3440 Comparison to median by topic Query 117: 0.1110 At 15 docs: 0.3133 +1.0 Query 118: 0.0392 At 20 docs: 0.2970 Query 119: 0.7923 At 30 docs: 0.2573 Query 120: 0.0184 At 100 docs: 0.1380 Query 121: 0.3936 At 200 docs: 0.0841 Query 122: 0.1076 At 500 docs: 0.0405 Query 123: 0.9821 At 1000 docs: 0.0220 +0.5 Query 124: 0.0022 Query 125: 0.0052 R-Precision (prec. after R Query 126: 0.0013 (=num_rel) docs. retrieved): Query 127: 0.0040 Exact: 0.2696 Query 128: 0.0095 0.0 Query 129: 0.4648 Query 130: 0.6954 Query 131: 0.1501 Query 132: 0.8325 Query 133: 0.0311 Difference (average precision) Query 134: 0.7917 −0.5 Query 135: 0.0294 Query 136: 0.0077 Query 137: 0.0000 Query 138: 0.0198 Query 139: 0.1093 −1.0 Query 140: 0.5262 91 100 110 120 130 140 Topic number Univ. Jaen/SINAI, Bilingual English to German, Auto, TD Run UJABIDE Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0417 Total number of documents Query 92: 0.2601 over all queries Query 93: 0.6380 Retrieved: 50000 0.8 Query 94: 0.0580 Relevant: 1938 Query 95: 0.2442 Rel_ret: 1293 Query 96: 0.0517 Query 97: 0.0584 Interpolated Recall - 0.6 Query 98: 0.3230 Precision Averages: Query 99: 0.3035 at 0.00 0.6445 Query 100: 0.7005 at 0.10 0.4962 Precision Query 101: 0.0123 at 0.20 0.4172 Query 102: 0.4218 at 0.30 0.3735 0.4 Query 103: 0.1915 at 0.40 0.3194 Query 104: 0.0463 at 0.50 0.2785 Query 105: 0.4886 at 0.60 0.2364 Query 106: 0.0673 at 0.70 0.1802 0.2 Query 107: 0.1341 at 0.80 0.1176 Query 108: 0.0298 at 0.90 0.0792 Query 109: 0.0118 at 1.00 0.0599 Query 110: 0.5320 0.0 Query 111: 0.0482 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.8117 for all rel. documents Query 113: 0.0002 0.2747 Recall Query 114: 0.0349 Precision: Query 115: 0.2424 At 5 docs: 0.4360 Run UJABIDE Query 116: 0.7853 At 10 docs: 0.3980 Comparison to median by topic Query 117: 0.1986 At 15 docs: 0.3560 +1.0 Query 118: 0.0094 At 20 docs: 0.3220 Query 119: 0.5859 At 30 docs: 0.2833 Query 120: 0.0555 At 100 docs: 0.1382 Query 121: 0.4389 At 200 docs: 0.0860 Query 122: 0.1823 At 500 docs: 0.0444 Query 123: 0.9683 At 1000 docs: 0.0259 +0.5 Query 124: 0.1445 Query 125: 0.1185 R-Precision (prec. after R Query 126: 0.3724 (=num_rel) docs. retrieved): Query 127: 0.2728 Exact: 0.3077 Query 128: 0.0069 0.0 Query 129: 0.3641 Query 130: 0.3380 Query 131: 0.1672 Query 132: 0.6403 Query 133: 0.0051 Difference (average precision) Query 134: 0.6429 −0.5 Query 135: 0.0397 Query 136: 1.0000 Query 137: 0.0333 Query 138: 0.3837 Query 139: 0.1715 −1.0 Query 140: 0.0594 91 100 110 120 130 140 Topic number Université de Neuchâtel, Bilingual English to German, Auto, TD Run UniNEdeBi Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1583 Total number of documents Query 92: 0.2413 over all queries Query 93: 0.8835 Retrieved: 50000 0.8 Query 94: 0.5447 Relevant: 1938 Query 95: 0.2085 Rel_ret: 1657 Query 96: 0.6349 Query 97: 0.2341 Interpolated Recall - 0.6 Query 98: 0.6372 Precision Averages: Query 99: 0.1271 at 0.00 0.7468 Query 100: 0.8596 at 0.10 0.6543 Precision Query 101: 0.7569 at 0.20 0.5651 Query 102: 0.3698 at 0.30 0.5213 0.4 Query 103: 0.5574 at 0.40 0.4612 Query 104: 0.0793 at 0.50 0.4201 Query 105: 0.6411 at 0.60 0.3813 Query 106: 0.0670 at 0.70 0.3297 0.2 Query 107: 0.0549 at 0.80 0.2711 Query 108: 0.3678 at 0.90 0.1976 Query 109: 0.4193 at 1.00 0.1243 Query 110: 0.6142 0.0 Query 111: 0.1387 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.8469 for all rel. documents Query 113: 0.0004 0.4129 Recall Query 114: 0.2829 Precision: Query 115: 0.1109 At 5 docs: 0.5480 Run UniNEdeBi Query 116: 0.7845 At 10 docs: 0.5060 Comparison to median by topic Query 117: 0.2983 At 15 docs: 0.4747 +1.0 Query 118: 0.0122 At 20 docs: 0.4470 Query 119: 0.7094 At 30 docs: 0.3887 Query 120: 0.0402 At 100 docs: 0.2010 Query 121: 0.4116 At 200 docs: 0.1278 Query 122: 0.1501 At 500 docs: 0.0611 Query 123: 1.0000 At 1000 docs: 0.0331 +0.5 Query 124: 0.3833 Query 125: 0.5633 R-Precision (prec. after R Query 126: 0.2532 (=num_rel) docs. retrieved): Query 127: 0.8841 Exact: 0.4128 Query 128: 0.1036 0.0 Query 129: 0.5586 Query 130: 0.0000 Query 131: 0.0000 Query 132: 0.8445 Query 133: 0.4005 Difference (average precision) Query 134: 0.9071 −0.5 Query 135: 0.4629 Query 136: 1.0000 Query 137: 0.0476 Query 138: 0.5441 Query 139: 0.2250 −1.0 Query 140: 0.2223 91 100 110 120 130 140 Topic number Université de Neuchâtel, Bilingual English to German, Auto, TD Run UniNEdeBi2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1155 Total number of documents Query 92: 0.2361 over all queries Query 93: 0.8830 Retrieved: 50000 0.8 Query 94: 0.5230 Relevant: 1938 Query 95: 0.1743 Rel_ret: 1655 Query 96: 0.6429 Query 97: 0.1560 Interpolated Recall - 0.6 Query 98: 0.6176 Precision Averages: Query 99: 0.1361 at 0.00 0.7070 Query 100: 0.8618 at 0.10 0.6276 Precision Query 101: 0.7426 at 0.20 0.5609 Query 102: 0.3533 at 0.30 0.5202 0.4 Query 103: 0.5319 at 0.40 0.4602 Query 104: 0.0700 at 0.50 0.4158 Query 105: 0.6105 at 0.60 0.3743 Query 106: 0.0664 at 0.70 0.3216 0.2 Query 107: 0.0559 at 0.80 0.2595 Query 108: 0.3786 at 0.90 0.1867 Query 109: 0.4075 at 1.00 0.1171 Query 110: 0.6184 0.0 Query 111: 0.0745 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.8430 for all rel. documents Query 113: 0.0008 0.4042 Recall Query 114: 0.2970 Precision: Query 115: 0.0644 At 5 docs: 0.5360 Run UniNEdeBi2 Query 116: 0.8082 At 10 docs: 0.4920 Comparison to median by topic Query 117: 0.3098 At 15 docs: 0.4667 +1.0 Query 118: 0.0155 At 20 docs: 0.4370 Query 119: 0.7095 At 30 docs: 0.3847 Query 120: 0.0487 At 100 docs: 0.2058 Query 121: 0.4060 At 200 docs: 0.1262 Query 122: 0.1674 At 500 docs: 0.0613 Query 123: 1.0000 At 1000 docs: 0.0331 +0.5 Query 124: 0.4051 Query 125: 0.5668 R-Precision (prec. after R Query 126: 0.1808 (=num_rel) docs. retrieved): Query 127: 0.8497 Exact: 0.4115 Query 128: 0.1609 0.0 Query 129: 0.5386 Query 130: 0.0000 Query 131: 0.0001 Query 132: 0.8397 Query 133: 0.4193 Difference (average precision) Query 134: 0.8661 −0.5 Query 135: 0.4819 Query 136: 1.0000 Query 137: 0.0172 Query 138: 0.4809 Query 139: 0.2213 −1.0 Query 140: 0.2556 91 100 110 120 130 140 Topic number JHU/APL, Bilingual Portuguese to English, Auto, TD Run aplbiptena Average precision Overall statistics Recall−Precision Values (individual queries): (for 42 queries): 1.0 Query 91: 0.5864 Total number of documents Query 92: 0.8273 over all queries Query 94: 0.8947 Retrieved: 42000 0.8 Query 95: 0.6089 Relevant: 821 Query 97: 1.0000 Rel_ret: 753 Query 98: 0.0104 Query 99: 0.1128 Interpolated Recall - Query 100: 0.5266 Precision Averages: 0.6 Query 102: 0.6535 at 0.00 0.6578 Query 103: 0.6113 at 0.10 0.6265 Query 104: 0.6630 at 0.20 0.5801 Precision Query 105: 0.6402 at 0.30 0.5338 0.4 Query 106: 0.1329 at 0.40 0.5034 Query 107: 0.6363 at 0.50 0.4615 Query 108: 0.4538 at 0.60 0.3866 Query 109: 0.0416 at 0.70 0.3401 0.2 Query 111: 0.0136 at 0.80 0.2690 Query 112: 0.5091 at 0.90 0.2189 Query 113: 0.1375 at 1.00 0.1512 Query 114: 0.5058 0.0 Query 115: 0.1309 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 116: 0.4217 for all rel. documents Query 119: 0.8829 0.4158 Recall Query 120: 0.3669 Precision: Query 121: 0.6639 At 5 docs: 0.4857 Run aplbiptena Query 122: 0.2152 At 10 docs: 0.4214 Query 123: 0.4710 At 15 docs: 0.3587 Comparison to median by topic Query 124: 0.2846 At 20 docs: 0.3226 +1.0 Query 125: 0.8875 At 30 docs: 0.2794 Query 126: 0.0000 At 100 docs: 0.1331 Query 128: 0.0040 At 200 docs: 0.0768 Query 129: 0.0369 At 500 docs: 0.0342 Query 130: 0.5850 At 1000 docs: 0.0179 +0.5 Query 131: 0.7474 Query 133: 0.3346 R-Precision (prec. after R Query 134: 0.7500 (=num_rel) docs. retrieved): Query 135: 0.2324 Exact: 0.3949 Query 136: 0.1000 0.0 Query 137: 0.0033 Query 138: 0.7214 Query 139: 0.0369 Query 140: 0.0204 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Clairvoyance Corp., Bilingual Chinese to English, Auto, TD Run clch2enc1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 42 queries): 1.0 Query 91: 0.0005 Total number of documents Query 92: 0.7954 over all queries Query 94: 0.0268 Retrieved: 42000 0.8 Query 95: 0.5068 Relevant: 821 Query 97: 0.0270 Rel_ret: 646 Query 98: 0.0000 Query 99: 0.0098 Interpolated Recall - 0.6 Query 100: 0.1946 Precision Averages: Query 102: 0.0040 at 0.00 0.3676 Query 103: 0.0067 at 0.10 0.3122 Precision Query 104: 0.2207 at 0.20 0.2893 Query 105: 0.0056 at 0.30 0.2225 0.4 Query 106: 0.0134 at 0.40 0.1940 Query 107: 0.0000 at 0.50 0.1745 Query 108: 0.4399 at 0.60 0.1546 Query 109: 0.3324 at 0.70 0.1404 0.2 Query 111: 0.0206 at 0.80 0.0998 Query 112: 0.4056 at 0.90 0.0736 Query 113: 0.1166 at 1.00 0.0388 Query 114: 0.5273 0.0 Query 115: 0.3620 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 116: 0.0365 for all rel. documents Query 119: 0.8813 0.1772 Recall Query 120: 0.0284 Precision: Query 121: 0.0000 At 5 docs: 0.2190 Run clch2enc1 Query 122: 0.1163 At 10 docs: 0.1952 Comparison to median by topic Query 123: 0.1054 At 15 docs: 0.1762 +1.0 Query 124: 0.1186 At 20 docs: 0.1726 Query 125: 0.0135 At 30 docs: 0.1500 Query 126: 0.0307 At 100 docs: 0.0902 Query 128: 0.0110 At 200 docs: 0.0564 Query 129: 0.0070 At 500 docs: 0.0283 Query 130: 0.2780 At 1000 docs: 0.0154 +0.5 Query 131: 0.2269 Query 133: 0.4847 R-Precision (prec. after R Query 134: 0.4333 (=num_rel) docs. retrieved): Query 135: 0.3950 Exact: 0.1712 Query 136: 0.0526 0.0 Query 137: 0.0000 Query 138: 0.0215 Query 139: 0.0333 Query 140: 0.1518 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Clairvoyance Corp., Bilingual Chinese to English, Auto, TD Run clch2ent1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 42 queries): 1.0 Query 91: 0.0186 Total number of documents Query 92: 0.7549 over all queries Query 94: 0.0212 Retrieved: 42000 0.8 Query 95: 0.4471 Relevant: 821 Query 97: 0.0182 Rel_ret: 558 Query 98: 0.0000 Query 99: 0.0043 Interpolated Recall - 0.6 Query 100: 0.0866 Precision Averages: Query 102: 0.0026 at 0.00 0.2302 Query 103: 0.0062 at 0.10 0.2022 Precision Query 104: 0.2935 at 0.20 0.1882 Query 105: 0.0007 at 0.30 0.1744 0.4 Query 106: 0.0317 at 0.40 0.1612 Query 107: 0.0000 at 0.50 0.1490 Query 108: 0.4348 at 0.60 0.1187 Query 109: 0.0515 at 0.70 0.1070 0.2 Query 111: 0.0005 at 0.80 0.0772 Query 112: 0.0052 at 0.90 0.0603 Query 113: 0.0137 at 1.00 0.0381 Query 114: 0.7255 0.0 Query 115: 0.1165 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 116: 0.0225 for all rel. documents Query 119: 0.6832 0.1322 Recall Query 120: 0.0008 Precision: Query 121: 0.0000 At 5 docs: 0.1429 Run clch2ent1 Query 122: 0.0935 At 10 docs: 0.1262 Comparison to median by topic Query 123: 0.0050 At 15 docs: 0.1095 +1.0 Query 124: 0.0017 At 20 docs: 0.1107 Query 125: 0.0542 At 30 docs: 0.1032 Query 126: 0.1394 At 100 docs: 0.0693 Query 128: 0.0002 At 200 docs: 0.0464 Query 129: 0.0615 At 500 docs: 0.0240 Query 130: 0.4957 At 1000 docs: 0.0133 +0.5 Query 131: 0.0248 Query 133: 0.0976 R-Precision (prec. after R Query 134: 0.6500 (=num_rel) docs. retrieved): Query 135: 0.0527 Exact: 0.1262 Query 136: 0.0588 0.0 Query 137: 0.0000 Query 138: 0.0083 Query 139: 0.0271 Query 140: 0.0434 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Clairvoyance Corp., Bilingual Chinese to English, Auto, TD Run clch2ent2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 42 queries): 1.0 Query 91: 0.0007 Total number of documents Query 92: 0.7979 over all queries Query 94: 0.0414 Retrieved: 42000 0.8 Query 95: 0.4512 Relevant: 821 Query 97: 0.0112 Rel_ret: 655 Query 98: 0.0000 Query 99: 0.0013 Interpolated Recall - 0.6 Query 100: 0.1060 Precision Averages: Query 102: 0.0590 at 0.00 0.3579 Query 103: 0.0066 at 0.10 0.2962 Precision Query 104: 0.1857 at 0.20 0.2747 Query 105: 0.5113 at 0.30 0.2531 0.4 Query 106: 0.0023 at 0.40 0.2350 Query 107: 0.0089 at 0.50 0.2129 Query 108: 0.4435 at 0.60 0.1743 Query 109: 0.1954 at 0.70 0.1541 0.2 Query 111: 0.0005 at 0.80 0.1172 Query 112: 0.2851 at 0.90 0.0860 Query 113: 0.0013 at 1.00 0.0509 Query 114: 0.6109 0.0 Query 115: 0.0210 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 116: 0.0202 for all rel. documents Query 119: 0.8643 0.1936 Recall Query 120: 0.0001 Precision: Query 121: 0.2021 At 5 docs: 0.2333 Run clch2ent2 Query 122: 0.0333 At 10 docs: 0.2310 Comparison to median by topic Query 123: 0.0840 At 15 docs: 0.2111 +1.0 Query 124: 0.0648 At 20 docs: 0.1988 Query 125: 0.0019 At 30 docs: 0.1833 Query 126: 0.0000 At 100 docs: 0.1021 Query 128: 0.0108 At 200 docs: 0.0614 Query 129: 0.2652 At 500 docs: 0.0293 Query 130: 0.6047 At 1000 docs: 0.0156 +0.5 Query 131: 0.6605 Query 133: 0.0026 R-Precision (prec. after R Query 134: 0.2808 (=num_rel) docs. retrieved): Query 135: 0.0000 Exact: 0.1853 Query 136: 0.0833 0.0 Query 137: 0.0000 Query 138: 0.6794 Query 139: 0.0006 Query 140: 0.5303 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Clairvoyance Corp., Bilingual Chinese to English, Auto, TD Run clch2enw Average precision Overall statistics Recall−Precision Values (individual queries): (for 42 queries): 1.0 Query 91: 0.0005 Total number of documents Query 92: 0.7729 over all queries Query 94: 0.0046 Retrieved: 42000 0.8 Query 95: 0.5273 Relevant: 821 Query 97: 0.0278 Rel_ret: 574 Query 98: 0.0000 Query 99: 0.0110 Interpolated Recall - 0.6 Query 100: 0.2019 Precision Averages: Query 102: 0.0050 at 0.00 0.3198 Query 103: 0.0426 at 0.10 0.2631 Precision Query 104: 0.2667 at 0.20 0.2072 Query 105: 0.0096 at 0.30 0.1779 0.4 Query 106: 0.0133 at 0.40 0.1615 Query 107: 0.0000 at 0.50 0.1453 Query 108: 0.4364 at 0.60 0.1177 Query 109: 0.3408 at 0.70 0.1061 0.2 Query 111: 0.0186 at 0.80 0.0872 Query 112: 0.3719 at 0.90 0.0655 Query 113: 0.1659 at 1.00 0.0376 Query 114: 0.2906 0.0 Query 115: 0.3715 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 116: 0.0373 for all rel. documents Query 119: 0.8823 0.1430 Recall Query 120: 0.0161 Precision: Query 121: 0.0001 At 5 docs: 0.1714 Run clch2enw Query 122: 0.0803 At 10 docs: 0.1619 Comparison to median by topic Query 123: 0.0543 At 15 docs: 0.1476 +1.0 Query 124: 0.0735 At 20 docs: 0.1464 Query 125: 0.0015 At 30 docs: 0.1389 Query 126: 0.0000 At 100 docs: 0.0764 Query 128: 0.0062 At 200 docs: 0.0500 Query 129: 0.0000 At 500 docs: 0.0252 Query 130: 0.2500 At 1000 docs: 0.0137 +0.5 Query 131: 0.0008 Query 133: 0.1034 R-Precision (prec. after R Query 134: 0.2458 (=num_rel) docs. retrieved): Query 135: 0.1492 Exact: 0.1412 Query 136: 0.0769 0.0 Query 137: 0.0000 Query 138: 0.0944 Query 139: 0.0270 Query 140: 0.0281 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Clairvoyance Corp., Bilingual Spanish to English, Auto, TD Run cles2enc1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 42 queries): 1.0 Query 91: 0.3988 Total number of documents Query 92: 0.7273 over all queries Query 94: 0.0193 Retrieved: 42000 0.8 Query 95: 0.4544 Relevant: 821 Query 97: 0.0128 Rel_ret: 750 Query 98: 1.0000 Query 99: 0.0419 Interpolated Recall - 0.6 Query 100: 0.5061 Precision Averages: Query 102: 0.8026 at 0.00 0.6128 Query 103: 0.4511 at 0.10 0.5670 Precision Query 104: 0.7256 at 0.20 0.5034 Query 105: 0.3857 at 0.30 0.4249 0.4 Query 106: 0.1156 at 0.40 0.3935 Query 107: 0.0069 at 0.50 0.3648 Query 108: 0.3817 at 0.60 0.3308 Query 109: 0.0248 at 0.70 0.2904 0.2 Query 111: 0.1622 at 0.80 0.2256 Query 112: 0.6583 at 0.90 0.1762 Query 113: 0.0147 at 1.00 0.1328 Query 114: 0.6531 0.0 Query 115: 0.3181 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 116: 0.3928 for all rel. documents Query 119: 0.8704 0.3478 Recall Query 120: 0.4060 Precision: Query 121: 0.5676 At 5 docs: 0.4000 Run cles2enc1 Query 122: 0.2440 At 10 docs: 0.3262 Comparison to median by topic Query 123: 0.5354 At 15 docs: 0.3048 +1.0 Query 124: 0.0214 At 20 docs: 0.2810 Query 125: 0.2383 At 30 docs: 0.2437 Query 126: 0.0000 At 100 docs: 0.1267 Query 128: 0.0361 At 200 docs: 0.0754 Query 129: 0.0308 At 500 docs: 0.0344 Query 130: 0.5150 At 1000 docs: 0.0179 +0.5 Query 131: 0.5476 Query 133: 0.0990 R-Precision (prec. after R Query 134: 0.6538 (=num_rel) docs. retrieved): Query 135: 0.1860 Exact: 0.3276 Query 136: 0.1429 0.0 Query 137: 0.0059 Query 138: 0.1439 Query 139: 0.6691 Query 140: 0.4399 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Clairvoyance Corp., Bilingual Spanish to English, Auto, TD Run cles2ent1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 42 queries): 1.0 Query 91: 0.1524 Total number of documents Query 92: 0.7402 over all queries Query 94: 0.0181 Retrieved: 42000 0.8 Query 95: 0.4436 Relevant: 821 Query 97: 0.0127 Rel_ret: 664 Query 98: 0.5000 Query 99: 0.0649 Interpolated Recall - 0.6 Query 100: 0.3162 Precision Averages: Query 102: 0.8751 at 0.00 0.5769 Query 103: 0.4491 at 0.10 0.5319 Precision Query 104: 0.8226 at 0.20 0.4514 Query 105: 0.3732 at 0.30 0.4062 0.4 Query 106: 0.0378 at 0.40 0.3441 Query 107: 0.0069 at 0.50 0.3168 Query 108: 0.3951 at 0.60 0.2846 Query 109: 0.0014 at 0.70 0.2548 0.2 Query 111: 0.1566 at 0.80 0.1966 Query 112: 0.6530 at 0.90 0.1449 Query 113: 0.3090 at 1.00 0.1104 Query 114: 0.6498 0.0 Query 115: 0.3058 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 116: 0.4141 for all rel. documents Query 119: 0.8774 0.3137 Recall Query 120: 0.4072 Precision: Query 121: 0.5719 At 5 docs: 0.3905 Run cles2ent1 Query 122: 0.2079 At 10 docs: 0.3119 Comparison to median by topic Query 123: 0.4987 At 15 docs: 0.2746 +1.0 Query 124: 0.0034 At 20 docs: 0.2560 Query 125: 0.0018 At 30 docs: 0.2270 Query 126: 0.0000 At 100 docs: 0.1121 Query 128: 0.0749 At 200 docs: 0.0663 Query 129: 0.7864 At 500 docs: 0.0306 Query 130: 0.5220 At 1000 docs: 0.0158 +0.5 Query 131: 0.0000 Query 133: 0.0000 R-Precision (prec. after R Query 134: 0.6308 (=num_rel) docs. retrieved): Query 135: 0.0075 Exact: 0.2871 Query 136: 0.1250 0.0 Query 137: 0.0000 Query 138: 0.0878 Query 139: 0.3688 Query 140: 0.3069 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Clairvoyance Corp., Bilingual Spanish to English, Auto, TD Run cles2ent2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 42 queries): 1.0 Query 91: 0.3237 Total number of documents Query 92: 0.7703 over all queries Query 94: 0.0171 Retrieved: 42000 0.8 Query 95: 0.4555 Relevant: 821 Query 97: 0.0128 Rel_ret: 706 Query 98: 0.5000 Query 99: 0.0678 Interpolated Recall - 0.6 Query 100: 0.3385 Precision Averages: Query 102: 0.8012 at 0.00 0.5733 Query 103: 0.4256 at 0.10 0.5311 Precision Query 104: 0.7251 at 0.20 0.4674 Query 105: 0.4135 at 0.30 0.4045 0.4 Query 106: 0.1191 at 0.40 0.3715 Query 107: 0.0069 at 0.50 0.3462 Query 108: 0.3995 at 0.60 0.3072 Query 109: 0.0018 at 0.70 0.2908 0.2 Query 111: 0.1977 at 0.80 0.2229 Query 112: 0.6560 at 0.90 0.1683 Query 113: 0.0000 at 1.00 0.1280 Query 114: 0.6188 0.0 Query 115: 0.3058 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 116: 0.3892 for all rel. documents Query 119: 0.8854 0.3310 Recall Query 120: 0.4211 Precision: Query 121: 0.5719 At 5 docs: 0.4095 Run cles2ent2 Query 122: 0.0123 At 10 docs: 0.3214 Comparison to median by topic Query 123: 0.5137 At 15 docs: 0.2921 +1.0 Query 124: 0.0023 At 20 docs: 0.2631 Query 125: 0.8750 At 30 docs: 0.2302 Query 126: 0.0000 At 100 docs: 0.1221 Query 128: 0.0282 At 200 docs: 0.0727 Query 129: 0.0402 At 500 docs: 0.0328 Query 130: 0.5286 At 1000 docs: 0.0168 +0.5 Query 131: 0.5140 Query 133: 0.1075 R-Precision (prec. after R Query 134: 0.6905 (=num_rel) docs. retrieved): Query 135: 0.1830 Exact: 0.3046 Query 136: 0.1000 0.0 Query 137: 0.0069 Query 138: 0.5072 Query 139: 0.0079 Query 140: 0.3602 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Clairvoyance Corp., Bilingual Spanish to English, Auto, TD Run cles2enw Average precision Overall statistics Recall−Precision Values (individual queries): (for 42 queries): 1.0 Query 91: 0.3307 Total number of documents Query 92: 0.7303 over all queries Query 94: 0.0197 Retrieved: 42000 0.8 Query 95: 0.4489 Relevant: 821 Query 97: 0.0128 Rel_ret: 720 Query 98: 1.0000 Query 99: 0.0395 Interpolated Recall - 0.6 Query 100: 0.5294 Precision Averages: Query 102: 0.8018 at 0.00 0.5882 Query 103: 0.4756 at 0.10 0.5524 Precision Query 104: 0.7249 at 0.20 0.5005 Query 105: 0.4063 at 0.30 0.4255 0.4 Query 106: 0.1168 at 0.40 0.3945 Query 107: 0.0068 at 0.50 0.3715 Query 108: 0.3918 at 0.60 0.3377 Query 109: 0.0147 at 0.70 0.3084 0.2 Query 111: 0.1656 at 0.80 0.2294 Query 112: 0.6583 at 0.90 0.1811 Query 113: 0.0083 at 1.00 0.1412 Query 114: 0.6602 0.0 Query 115: 0.3129 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 116: 0.3980 for all rel. documents Query 119: 0.8644 0.3502 Recall Query 120: 0.4130 Precision: Query 121: 0.5747 At 5 docs: 0.4143 Run cles2enw Query 122: 0.0147 At 10 docs: 0.3286 Comparison to median by topic Query 123: 0.5286 At 15 docs: 0.2984 +1.0 Query 124: 0.0030 At 20 docs: 0.2726 Query 125: 0.8929 At 30 docs: 0.2333 Query 126: 0.0000 At 100 docs: 0.1212 Query 128: 0.0178 At 200 docs: 0.0733 Query 129: 0.0232 At 500 docs: 0.0333 Query 130: 0.5181 At 1000 docs: 0.0171 +0.5 Query 131: 0.5578 Query 133: 0.1043 R-Precision (prec. after R Query 134: 0.6915 (=num_rel) docs. retrieved): Query 135: 0.1830 Exact: 0.3399 Query 136: 0.1000 0.0 Query 137: 0.0076 Query 138: 0.5837 Query 139: 0.0084 Query 140: 0.3676 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number IRIT, Toulouse, Bilingual French to English, Auto, TD Run iritBFr2En Average precision Overall statistics Recall−Precision Values (individual queries): (for 42 queries): 1.0 Query 91: 0.0143 Total number of documents Query 92: 0.8217 over all queries Query 94: 0.1534 Retrieved: 42000 0.8 Query 95: 0.2982 Relevant: 821 Query 97: 0.2000 Rel_ret: 623 Query 98: 0.0013 Query 99: 0.6143 Interpolated Recall - 0.6 Query 100: 0.1217 Precision Averages: Query 102: 0.3722 at 0.00 0.5280 Query 103: 0.5388 at 0.10 0.4530 Precision Query 104: 0.0668 at 0.20 0.3914 Query 105: 0.5195 at 0.30 0.3142 0.4 Query 106: 0.1493 at 0.40 0.2843 Query 107: 0.0034 at 0.50 0.2496 Query 108: 0.3993 at 0.60 0.2090 Query 109: 0.0095 at 0.70 0.1717 0.2 Query 111: 0.0002 at 0.80 0.1399 Query 112: 0.5502 at 0.90 0.0924 Query 113: 0.0781 at 1.00 0.0388 Query 114: 0.3528 0.0 Query 115: 0.2270 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 116: 0.3314 for all rel. documents Query 119: 0.7861 0.2449 Recall Query 120: 0.4218 Precision: Query 121: 0.5355 At 5 docs: 0.3048 Run iritBFr2En Query 122: 0.0935 At 10 docs: 0.2857 Comparison to median by topic Query 123: 0.6303 At 15 docs: 0.2540 +1.0 Query 124: 0.0043 At 20 docs: 0.2226 Query 125: 0.0000 At 30 docs: 0.1952 Query 126: 0.1284 At 100 docs: 0.0898 Query 128: 0.0060 At 200 docs: 0.0555 Query 129: 0.1363 At 500 docs: 0.0265 Query 130: 0.3302 At 1000 docs: 0.0148 +0.5 Query 131: 0.0153 Query 133: 0.5529 R-Precision (prec. after R Query 134: 0.4417 (=num_rel) docs. retrieved): Query 135: 0.0054 Exact: 0.2382 Query 136: 0.0000 0.0 Query 137: 0.0020 Query 138: 0.2735 Query 139: 0.0080 Query 140: 0.0895 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Middlesex Univ., Bilingual Portuguese to English, Manual, TDN Run MDXman Average precision Overall statistics Recall−Precision Values (individual queries): (for 42 queries): 1.0 Query 91: 0.0013 Total number of documents Query 92: 0.6080 over all queries Query 94: 0.8313 Retrieved: 42000 0.8 Query 95: 0.3352 Relevant: 821 Query 97: 0.0455 Rel_ret: 694 Query 98: 0.0000 Query 99: 0.3140 Interpolated Recall - 0.6 Query 100: 0.4137 Precision Averages: Query 102: 0.1499 at 0.00 0.4593 Query 103: 0.5494 at 0.10 0.4318 Precision Query 104: 0.1604 at 0.20 0.3279 Query 105: 0.0196 at 0.30 0.2942 0.4 Query 106: 0.2982 at 0.40 0.2446 Query 107: 0.3208 at 0.50 0.2169 Query 108: 0.0687 at 0.60 0.1672 Query 109: 0.0730 at 0.70 0.1354 0.2 Query 111: 0.1907 at 0.80 0.1037 Query 112: 0.1354 at 0.90 0.0726 Query 113: 0.0769 at 1.00 0.0398 Query 114: 0.0946 0.0 Query 115: 0.1087 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 116: 0.2977 for all rel. documents Query 119: 0.5279 0.2085 Recall Query 120: 0.0346 Precision: Query 121: 0.1771 At 5 docs: 0.2667 Run MDXman Query 122: 0.0471 At 10 docs: 0.2405 Comparison to median by topic Query 123: 0.0827 At 15 docs: 0.2238 +1.0 Query 124: 0.0124 At 20 docs: 0.2083 Query 125: 0.5794 At 30 docs: 0.1762 Query 126: 0.1667 At 100 docs: 0.0974 Query 128: 0.0066 At 200 docs: 0.0583 Query 129: 0.0292 At 500 docs: 0.0301 Query 130: 0.6497 At 1000 docs: 0.0165 +0.5 Query 131: 0.0053 Query 133: 0.2477 R-Precision (prec. after R Query 134: 0.1067 (=num_rel) docs. retrieved): Query 135: 0.0488 Exact: 0.2310 Query 136: 0.0014 0.0 Query 137: 0.0812 Query 138: 0.2194 Query 139: 0.1253 Query 140: 0.5162 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Middlesex Univ., Bilingual Portuguese to English, Auto, TD Run MDXtd Average precision Overall statistics Recall−Precision Values (individual queries): (for 42 queries): 1.0 Query 91: 0.0114 Total number of documents Query 92: 0.6455 over all queries Query 94: 0.7329 Retrieved: 42000 0.8 Query 95: 0.3713 Relevant: 821 Query 97: 0.0417 Rel_ret: 698 Query 98: 0.0000 Query 99: 0.4219 Interpolated Recall - 0.6 Query 100: 0.2710 Precision Averages: Query 102: 0.3142 at 0.00 0.5141 Query 103: 0.5650 at 0.10 0.4174 Precision Query 104: 0.1354 at 0.20 0.3379 Query 105: 0.0270 at 0.30 0.2782 0.4 Query 106: 0.3221 at 0.40 0.2384 Query 107: 0.1951 at 0.50 0.2057 Query 108: 0.0052 at 0.60 0.1677 Query 109: 0.1099 at 0.70 0.1474 0.2 Query 111: 0.1386 at 0.80 0.0959 Query 112: 0.2486 at 0.90 0.0683 Query 113: 0.1044 at 1.00 0.0438 Query 114: 0.2131 0.0 Query 115: 0.1352 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 116: 0.2507 for all rel. documents Query 119: 0.6854 0.2088 Recall Query 120: 0.1308 Precision: Query 121: 0.0166 At 5 docs: 0.2810 Run MDXtd Query 122: 0.1318 At 10 docs: 0.2452 Comparison to median by topic Query 123: 0.0949 At 15 docs: 0.2206 +1.0 Query 124: 0.1210 At 20 docs: 0.2024 Query 125: 0.5833 At 30 docs: 0.1786 Query 126: 0.0303 At 100 docs: 0.0948 Query 128: 0.0074 At 200 docs: 0.0611 Query 129: 0.0392 At 500 docs: 0.0302 Query 130: 0.2689 At 1000 docs: 0.0166 +0.5 Query 131: 0.0081 Query 133: 0.3020 R-Precision (prec. after R Query 134: 0.2313 (=num_rel) docs. retrieved): Query 135: 0.0503 Exact: 0.2168 Query 136: 0.0011 0.0 Query 137: 0.0205 Query 138: 0.1202 Query 139: 0.1789 Query 140: 0.4864 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Middlesex Univ., Bilingual Portuguese to English, Auto, T Run MDXtpc Average precision Overall statistics Recall−Precision Values (individual queries): (for 42 queries): 1.0 Query 91: 0.0000 Total number of documents Query 92: 0.6232 over all queries Query 94: 0.8202 Retrieved: 42000 0.8 Query 95: 0.2931 Relevant: 821 Query 97: 0.0455 Rel_ret: 662 Query 98: 0.0000 Query 99: 0.1139 Interpolated Recall - Query 100: 0.0564 Precision Averages: 0.6 Query 102: 0.3648 at 0.00 0.3723 Query 103: 0.0675 at 0.10 0.2997 Query 104: 0.1553 at 0.20 0.2392 Precision Query 105: 0.3181 at 0.30 0.2218 0.4 Query 106: 0.2170 at 0.40 0.1870 Query 107: 0.0804 at 0.50 0.1637 Query 108: 0.0687 at 0.60 0.1202 Query 109: 0.0072 at 0.70 0.0939 0.2 Query 111: 0.0055 at 0.80 0.0746 Query 112: 0.0136 at 0.90 0.0528 Query 113: 0.0308 at 1.00 0.0300 Query 114: 0.0946 0.0 Query 115: 0.1387 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 116: 0.2977 for all rel. documents Query 119: 0.5279 0.1551 Recall Query 120: 0.2529 Precision: Query 121: 0.1771 At 5 docs: 0.1952 Run MDXtpc Query 122: 0.0471 At 10 docs: 0.1714 Query 123: 0.0492 At 15 docs: 0.1571 Comparison to median by topic Query 124: 0.0124 At 20 docs: 0.1440 +1.0 Query 125: 0.5970 At 30 docs: 0.1405 Query 126: 0.0000 At 100 docs: 0.0838 Query 128: 0.0066 At 200 docs: 0.0529 Query 129: 0.0158 At 500 docs: 0.0279 Query 130: 0.0954 At 1000 docs: 0.0158 +0.5 Query 131: 0.0046 Query 133: 0.2477 R-Precision (prec. after R Query 134: 0.0062 (=num_rel) docs. retrieved): Query 135: 0.0488 Exact: 0.1676 Query 136: 0.0014 0.0 Query 137: 0.0226 Query 138: 0.1170 Query 139: 0.1253 Query 140: 0.3450 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Océ, Bilingual Spanish to English, Auto, TD Run oce02es2enBF Average precision Overall statistics Recall−Precision Values (individual queries): (for 42 queries): 1.0 Query 91: 0.4626 Total number of documents Query 92: 0.7071 over all queries Query 94: 0.0515 Retrieved: 42000 0.8 Query 95: 0.3178 Relevant: 821 Query 97: 0.0208 Rel_ret: 689 Query 98: 0.0909 Query 99: 0.1757 Interpolated Recall - 0.6 Query 100: 0.6454 Precision Averages: Query 102: 0.8862 at 0.00 0.6749 Query 103: 0.5853 at 0.10 0.5860 Precision Query 104: 0.2601 at 0.20 0.5038 Query 105: 0.3309 at 0.30 0.4295 0.4 Query 106: 0.0706 at 0.40 0.3767 Query 107: 0.1016 at 0.50 0.3297 Query 108: 0.3666 at 0.60 0.2783 Query 109: 0.1091 at 0.70 0.2306 0.2 Query 111: 0.2130 at 0.80 0.1616 Query 112: 0.6139 at 0.90 0.0987 Query 113: 0.1322 at 1.00 0.0636 Query 114: 0.4641 0.0 Query 115: 0.1978 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 116: 0.1778 for all rel. documents Query 119: 0.7624 0.3180 Recall Query 120: 0.5707 Precision: Query 121: 0.5038 At 5 docs: 0.4333 Run oce02es2enBF Query 122: 0.1930 At 10 docs: 0.3381 Comparison to median by topic Query 123: 0.6491 At 15 docs: 0.3000 +1.0 Query 124: 0.0068 At 20 docs: 0.2726 Query 125: 0.3117 At 30 docs: 0.2230 Query 126: 0.0000 At 100 docs: 0.1102 Query 128: 0.0084 At 200 docs: 0.0654 Query 129: 0.4110 At 500 docs: 0.0304 Query 130: 0.3853 At 1000 docs: 0.0164 +0.5 Query 131: 0.3750 Query 133: 0.3021 R-Precision (prec. after R Query 134: 0.7591 (=num_rel) docs. retrieved): Query 135: 0.2571 Exact: 0.3306 Query 136: 0.0000 0.0 Query 137: 0.0067 Query 138: 0.6044 Query 139: 0.2565 Query 140: 0.0140 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Océ, Bilingual Spanish to English, Auto, TD Run oce02es2enLO Average precision Overall statistics Recall−Precision Values (individual queries): (for 42 queries): 1.0 Query 91: 0.4626 Total number of documents Query 92: 0.7071 over all queries Query 94: 0.0515 Retrieved: 42000 0.8 Query 95: 0.3146 Relevant: 821 Query 97: 0.0208 Rel_ret: 700 Query 98: 0.3333 Query 99: 0.0232 Interpolated Recall - 0.6 Query 100: 0.6861 Precision Averages: Query 102: 0.8862 at 0.00 0.6713 Query 103: 0.5853 at 0.10 0.5603 Precision Query 104: 0.2601 at 0.20 0.5145 Query 105: 0.3309 at 0.30 0.4448 0.4 Query 106: 0.0706 at 0.40 0.3918 Query 107: 0.1016 at 0.50 0.3393 Query 108: 0.3666 at 0.60 0.2851 Query 109: 0.1091 at 0.70 0.2413 0.2 Query 111: 0.2130 at 0.80 0.1661 Query 112: 0.6139 at 0.90 0.1009 Query 113: 0.1322 at 1.00 0.0756 Query 114: 0.4641 0.0 Query 115: 0.1063 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 116: 0.1778 for all rel. documents Query 119: 0.7624 0.3235 Recall Query 120: 0.5707 Precision: Query 121: 0.4956 At 5 docs: 0.4381 Run oce02es2enLO Query 122: 0.1930 At 10 docs: 0.3452 Comparison to median by topic Query 123: 0.6621 At 15 docs: 0.3063 +1.0 Query 124: 0.0127 At 20 docs: 0.2762 Query 125: 0.3117 At 30 docs: 0.2365 Query 126: 0.0000 At 100 docs: 0.1157 Query 128: 0.0084 At 200 docs: 0.0685 Query 129: 0.4110 At 500 docs: 0.0317 Query 130: 0.3853 At 1000 docs: 0.0167 +0.5 Query 131: 0.3473 Query 133: 0.3021 R-Precision (prec. after R Query 134: 0.7578 (=num_rel) docs. retrieved): Query 135: 0.2571 Exact: 0.3338 Query 136: 0.0000 0.0 Query 137: 0.0067 Query 138: 0.6044 Query 139: 0.2565 Query 140: 0.2269 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Océ, Bilingual Dutch to English, Auto, TD Run oce02nl2enER Average precision Overall statistics Recall−Precision Values (individual queries): (for 42 queries): 1.0 Query 91: 0.3095 Total number of documents Query 92: 0.6803 over all queries Query 94: 0.0000 Retrieved: 42000 0.8 Query 95: 0.0081 Relevant: 821 Query 97: 0.0000 Rel_ret: 384 Query 98: 0.0222 Query 99: 0.0081 Interpolated Recall - 0.6 Query 100: 0.0208 Precision Averages: Query 102: 0.4734 at 0.00 0.3693 Query 103: 0.3438 at 0.10 0.3021 Precision Query 104: 0.2550 at 0.20 0.2550 Query 105: 0.1261 at 0.30 0.1989 0.4 Query 106: 0.0068 at 0.40 0.1677 Query 107: 0.0000 at 0.50 0.1407 Query 108: 0.0503 at 0.60 0.1156 Query 109: 0.0000 at 0.70 0.0929 0.2 Query 111: 0.0000 at 0.80 0.0527 Query 112: 0.6027 at 0.90 0.0368 Query 113: 0.0000 at 1.00 0.0214 Query 114: 0.0117 0.0 Query 115: 0.0025 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 116: 0.0000 for all rel. documents Query 119: 0.4600 0.1495 Recall Query 120: 0.4072 Precision: Query 121: 0.5017 At 5 docs: 0.2333 Run oce02nl2enER Query 122: 0.2521 At 10 docs: 0.1833 Comparison to median by topic Query 123: 0.4251 At 15 docs: 0.1556 +1.0 Query 124: 0.1530 At 20 docs: 0.1405 Query 125: 0.0000 At 30 docs: 0.1214 Query 126: 0.0000 At 100 docs: 0.0583 Query 128: 0.0253 At 200 docs: 0.0321 Query 129: 0.0037 At 500 docs: 0.0161 Query 130: 0.0681 At 1000 docs: 0.0091 +0.5 Query 131: 0.0220 Query 133: 0.0017 R-Precision (prec. after R Query 134: 0.7500 (=num_rel) docs. retrieved): Query 135: 0.0000 Exact: 0.1501 Query 136: 0.1000 0.0 Query 137: 0.0023 Query 138: 0.1803 Query 139: 0.0000 Query 140: 0.0033 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number JHU/APL, Bilingual English to Spanish, Auto, TD Run aplbienes Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.6676 Total number of documents Query 92: 0.5293 over all queries Query 93: 0.0022 Retrieved: 50000 0.8 Query 94: 0.8245 Relevant: 2854 Query 95: 0.3066 Rel_ret: 2326 Query 96: 0.0021 Query 97: 0.3928 Interpolated Recall - Query 98: 0.2257 Precision Averages: 0.6 Query 99: 0.3800 at 0.00 0.6452 Query 100: 0.4828 at 0.10 0.5729 Query 101: 0.4543 at 0.20 0.5053 Precision Query 102: 0.4768 at 0.30 0.4429 0.4 Query 103: 0.1710 at 0.40 0.4126 Query 104: 0.3756 at 0.50 0.3723 Query 105: 0.8126 at 0.60 0.3396 Query 106: 0.0831 at 0.70 0.2948 0.2 Query 107: 0.0709 at 0.80 0.2524 Query 108: 0.3519 at 0.90 0.1947 Query 109: 0.2318 at 1.00 0.1277 Query 110: 0.0452 0.0 Query 111: 0.0404 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.0098 for all rel. documents Query 113: 0.0079 0.3602 Recall Query 114: 0.8993 Precision: Query 115: 0.3391 At 5 docs: 0.4720 Run aplbienes Query 116: 0.8422 At 10 docs: 0.4320 Query 117: 0.2410 At 15 docs: 0.4160 Comparison to median by topic Query 118: 0.0082 At 20 docs: 0.3910 +1.0 Query 119: 0.8001 At 30 docs: 0.3513 Query 120: 0.0650 At 100 docs: 0.2188 Query 121: 0.3853 At 200 docs: 0.1501 Query 122: 0.0962 At 500 docs: 0.0812 Query 123: 0.8571 At 1000 docs: 0.0465 +0.5 Query 124: 0.3528 Query 125: 0.3399 R-Precision (prec. after R Query 126: 0.2120 (=num_rel) docs. retrieved): Query 127: 0.3688 Exact: 0.3494 Query 128: 0.1134 0.0 Query 129: 0.3734 Query 130: 0.8372 Query 131: 0.6305 Query 132: 0.7701 Query 133: 0.0009 Difference (average precision) Query 134: 0.0248 −0.5 Query 135: 0.5976 Query 136: 0.2322 Query 137: 0.3057 Query 138: 0.9048 Query 139: 0.0771 −1.0 Query 140: 0.3898 91 100 110 120 130 140 Topic number JHU/APL, Bilingual Portuguese to Spanish, Auto, TD Run aplbiptesa Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0036 Total number of documents Query 92: 0.4527 over all queries Query 93: 0.2532 Retrieved: 50000 0.8 Query 94: 0.8936 Relevant: 2854 Query 95: 0.0687 Rel_ret: 2071 Query 96: 0.1180 Query 97: 0.6664 Interpolated Recall - Query 98: 0.9429 Precision Averages: 0.6 Query 99: 0.1172 at 0.00 0.5377 Query 100: 0.3133 at 0.10 0.4779 Query 101: 0.0521 at 0.20 0.4606 Precision Query 102: 0.3832 at 0.30 0.4371 0.4 Query 103: 0.1134 at 0.40 0.3942 Query 104: 0.0217 at 0.50 0.3639 Query 105: 0.6156 at 0.60 0.3229 Query 106: 0.0496 at 0.70 0.2941 0.2 Query 107: 0.2332 at 0.80 0.2316 Query 108: 0.0000 at 0.90 0.1557 Query 109: 0.5160 at 1.00 0.1005 Query 110: 0.7003 0.0 Query 111: 0.0000 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6515 for all rel. documents Query 113: 0.0054 0.3325 Recall Query 114: 0.0005 Precision: Query 115: 0.0000 At 5 docs: 0.3920 Run aplbiptesa Query 116: 0.8721 At 10 docs: 0.3640 Query 117: 0.1407 At 15 docs: 0.3267 Comparison to median by topic Query 118: 0.2043 At 20 docs: 0.3000 +1.0 Query 119: 0.6573 At 30 docs: 0.2640 Query 120: 0.0494 At 100 docs: 0.1762 Query 121: 0.6488 At 200 docs: 0.1278 Query 122: 0.0412 At 500 docs: 0.0708 Query 123: 0.8545 At 1000 docs: 0.0414 +0.5 Query 124: 0.0012 Query 125: 0.6576 R-Precision (prec. after R Query 126: 0.0000 (=num_rel) docs. retrieved): Query 127: 0.5952 Exact: 0.3252 Query 128: 0.0653 0.0 Query 129: 0.0000 Query 130: 0.7729 Query 131: 0.3367 Query 132: 0.5978 Query 133: 0.0020 Difference (average precision) Query 134: 0.8500 −0.5 Query 135: 0.6125 Query 136: 0.7267 Query 137: 0.0125 Query 138: 0.0000 Query 139: 0.2416 −1.0 Query 140: 0.5134 91 100 110 120 130 140 Topic number JHU/APL, Bilingual Portuguese to Spanish, Auto, TD Run aplbiptesb Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.5223 Total number of documents Query 92: 0.4161 over all queries Query 93: 0.2806 Retrieved: 50000 0.8 Query 94: 0.0350 Relevant: 2854 Query 95: 0.0000 Rel_ret: 1589 Query 96: 0.1508 Query 97: 0.0000 Interpolated Recall - Query 98: 0.2824 Precision Averages: 0.6 Query 99: 0.0587 at 0.00 0.3457 Query 100: 0.2851 at 0.10 0.3145 Query 101: 0.0042 at 0.20 0.2866 Precision Query 102: 0.4195 at 0.30 0.2588 0.4 Query 103: 0.2088 at 0.40 0.2395 Query 104: 0.0031 at 0.50 0.2103 Query 105: 0.8565 at 0.60 0.1948 Query 106: 0.0000 at 0.70 0.1717 0.2 Query 107: 0.0186 at 0.80 0.1355 Query 108: 0.0000 at 0.90 0.0948 Query 109: 0.2757 at 1.00 0.0558 Query 110: 0.4982 0.0 Query 111: 0.0000 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6334 for all rel. documents Query 113: 0.0617 0.2000 Recall Query 114: 0.0036 Precision: Query 115: 0.0000 At 5 docs: 0.2160 Run aplbiptesb Query 116: 0.6266 At 10 docs: 0.2200 Query 117: 0.0631 At 15 docs: 0.2093 Comparison to median by topic Query 118: 0.0010 At 20 docs: 0.1890 +1.0 Query 119: 0.1792 At 30 docs: 0.1740 Query 120: 0.0428 At 100 docs: 0.1264 Query 121: 0.5790 At 200 docs: 0.0951 Query 122: 0.0000 At 500 docs: 0.0525 Query 123: 0.8710 At 1000 docs: 0.0318 +0.5 Query 124: 0.0002 Query 125: 0.0722 R-Precision (prec. after R Query 126: 0.0000 (=num_rel) docs. retrieved): Query 127: 0.6086 Exact: 0.2002 Query 128: 0.0202 0.0 Query 129: 0.0000 Query 130: 0.7673 Query 131: 0.1590 Query 132: 0.0022 Query 133: 0.0000 Difference (average precision) Query 134: 0.0274 −0.5 Query 135: 0.0000 Query 136: 0.6726 Query 137: 0.0008 Query 138: 0.0000 Query 139: 0.0148 −1.0 Query 140: 0.2776 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 2, Bilingual English to Spanish, Auto, TD Run bky2bienes Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.3371 Total number of documents Query 92: 0.4808 over all queries Query 93: 0.3041 Retrieved: 50000 0.8 Query 94: 0.1739 Relevant: 2854 Query 95: 0.4906 Rel_ret: 2375 Query 96: 0.3242 Query 97: 0.7183 Interpolated Recall - Query 98: 0.8769 Precision Averages: 0.6 Query 99: 0.6066 at 0.00 0.8082 Query 100: 0.8039 at 0.10 0.6708 Query 101: 0.0226 at 0.20 0.6273 Precision Query 102: 0.5628 at 0.30 0.5559 0.4 Query 103: 0.8525 at 0.40 0.5315 Query 104: 0.1208 at 0.50 0.4838 Query 105: 0.8126 at 0.60 0.4441 Query 106: 0.1379 at 0.70 0.3908 0.2 Query 107: 0.0582 at 0.80 0.3118 Query 108: 0.6221 at 0.90 0.2265 Query 109: 0.0576 at 1.00 0.1455 Query 110: 0.6135 0.0 Query 111: 0.0398 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5670 for all rel. documents Query 113: 0.0004 0.4567 Recall Query 114: 0.0791 Precision: Query 115: 0.2606 At 5 docs: 0.5960 Run bky2bienes Query 116: 0.7591 At 10 docs: 0.5200 Query 117: 0.1554 At 15 docs: 0.4720 Comparison to median by topic Query 118: 0.2209 At 20 docs: 0.4460 +1.0 Query 119: 0.8528 At 30 docs: 0.4087 Query 120: 0.3795 At 100 docs: 0.2616 Query 121: 0.7668 At 200 docs: 0.1774 Query 122: 0.2862 At 500 docs: 0.0874 Query 123: 0.9055 At 1000 docs: 0.0475 +0.5 Query 124: 0.0125 Query 125: 0.0023 R-Precision (prec. after R Query 126: 0.1102 (=num_rel) docs. retrieved): Query 127: 0.6373 Exact: 0.4271 Query 128: 0.1094 0.0 Query 129: 0.2325 Query 130: 0.7882 Query 131: 0.7152 Query 132: 0.7790 Query 133: 0.0016 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.8452 Query 136: 0.7635 Query 137: 0.4074 Query 138: 0.9571 Query 139: 0.5777 −1.0 Query 140: 0.6445 91 100 110 120 130 140 Topic number Univ. Exeter, Bilingual English to Spanish, Auto, TD Run exespengorgbi Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.6946 Total number of documents Query 92: 0.4892 over all queries Query 93: 0.2626 Retrieved: 50000 0.8 Query 94: 0.0558 Relevant: 2854 Query 95: 0.1951 Rel_ret: 2289 Query 96: 0.4266 Query 97: 0.0470 Interpolated Recall - Query 98: 1.0000 Precision Averages: 0.6 Query 99: 0.0028 at 0.00 0.7330 Query 100: 0.3964 at 0.10 0.6459 Query 101: 0.0235 at 0.20 0.5860 Precision Query 102: 0.5166 at 0.30 0.5226 0.4 Query 103: 0.7782 at 0.40 0.4815 Query 104: 0.2313 at 0.50 0.4279 Query 105: 0.7872 at 0.60 0.3795 Query 106: 0.3111 at 0.70 0.3431 0.2 Query 107: 0.3032 at 0.80 0.2647 Query 108: 0.7232 at 0.90 0.1900 Query 109: 0.3341 at 1.00 0.1011 Query 110: 0.6922 0.0 Query 111: 0.0063 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5358 for all rel. documents Query 113: 0.0001 0.4121 Recall Query 114: 0.0558 Precision: Query 115: 0.4370 At 5 docs: 0.5880 Run exespengorgbi Query 116: 0.8810 At 10 docs: 0.4840 Query 117: 0.1431 At 15 docs: 0.4307 Comparison to median by topic Query 118: 0.1877 At 20 docs: 0.3940 +1.0 Query 119: 0.8218 At 30 docs: 0.3633 Query 120: 0.1214 At 100 docs: 0.2390 Query 121: 0.7123 At 200 docs: 0.1666 Query 122: 0.0526 At 500 docs: 0.0825 Query 123: 0.8786 At 1000 docs: 0.0458 +0.5 Query 124: 0.5151 Query 125: 0.3480 R-Precision (prec. after R Query 126: 0.2580 (=num_rel) docs. retrieved): Query 127: 0.3896 Exact: 0.4128 Query 128: 0.1679 0.0 Query 129: 0.3648 Query 130: 0.8343 Query 131: 0.6793 Query 132: 0.0003 Query 133: 0.0529 Difference (average precision) Query 134: 0.8875 −0.5 Query 135: 0.1945 Query 136: 0.7332 Query 137: 0.1777 Query 138: 0.8784 Query 139: 0.7409 −1.0 Query 140: 0.2787 91 100 110 120 130 140 Topic number Univ. Exeter, Bilingual English to Spanish, Auto, TD Run exespengpiwgt Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.4337 Total number of documents Query 92: 0.4454 over all queries Query 93: 0.4732 Retrieved: 50000 0.8 Query 94: 0.3064 Relevant: 2854 Query 95: 0.3038 Rel_ret: 2372 Query 96: 0.4072 Query 97: 0.4738 Interpolated Recall - Query 98: 0.8262 Precision Averages: 0.6 Query 99: 0.1445 at 0.00 0.8256 Query 100: 0.4032 at 0.10 0.7179 Query 101: 0.2805 at 0.20 0.6417 Precision Query 102: 0.7718 at 0.30 0.5273 0.4 Query 103: 0.5820 at 0.40 0.4917 Query 104: 0.2420 at 0.50 0.4283 Query 105: 0.7509 at 0.60 0.3718 Query 106: 0.3527 at 0.70 0.3089 0.2 Query 107: 0.0799 at 0.80 0.2320 Query 108: 0.5099 at 0.90 0.1562 Query 109: 0.1908 at 1.00 0.0754 Query 110: 0.7598 0.0 Query 111: 0.0339 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5098 for all rel. documents Query 113: 0.0010 0.4202 Recall Query 114: 0.1656 Precision: Query 115: 0.3584 At 5 docs: 0.5920 Run exespengpiwgt Query 116: 0.8929 At 10 docs: 0.5280 Query 117: 0.1993 At 15 docs: 0.4987 Comparison to median by topic Query 118: 0.1920 At 20 docs: 0.4690 +1.0 Query 119: 0.7740 At 30 docs: 0.4153 Query 120: 0.0793 At 100 docs: 0.2598 Query 121: 0.7638 At 200 docs: 0.1746 Query 122: 0.2387 At 500 docs: 0.0858 Query 123: 0.8819 At 1000 docs: 0.0474 +0.5 Query 124: 0.2725 Query 125: 0.3741 R-Precision (prec. after R Query 126: 0.3396 (=num_rel) docs. retrieved): Query 127: 0.4946 Exact: 0.4117 Query 128: 0.1461 0.0 Query 129: 0.3110 Query 130: 0.7541 Query 131: 0.6028 Query 132: 0.0560 Query 133: 0.4495 Difference (average precision) Query 134: 0.5375 −0.5 Query 135: 0.8625 Query 136: 0.6015 Query 137: 0.1414 Query 138: 0.0046 Query 139: 0.7780 −1.0 Query 140: 0.4579 91 100 110 120 130 140 Topic number Univ. Exeter, Bilingual French to Spanish, Auto, TD Run exespfrmgcntbi Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.6499 Total number of documents Query 92: 0.4688 over all queries Query 93: 0.2550 Retrieved: 50000 0.8 Query 94: 0.0439 Relevant: 2854 Query 95: 0.2344 Rel_ret: 2210 Query 96: 0.2346 Query 97: 0.8430 Interpolated Recall - Query 98: 0.6693 Precision Averages: 0.6 Query 99: 0.0021 at 0.00 0.7526 Query 100: 0.2532 at 0.10 0.6501 Query 101: 0.4153 at 0.20 0.6054 Precision Query 102: 0.5364 at 0.30 0.5242 0.4 Query 103: 0.7746 at 0.40 0.4638 Query 104: 0.2229 at 0.50 0.4256 Query 105: 0.7955 at 0.60 0.3773 Query 106: 0.0457 at 0.70 0.3282 0.2 Query 107: 0.4052 at 0.80 0.2731 Query 108: 0.7547 at 0.90 0.2028 Query 109: 0.2867 at 1.00 0.0978 Query 110: 0.7326 0.0 Query 111: 0.0296 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5578 for all rel. documents Query 113: 0.0107 0.4141 Recall Query 114: 0.0294 Precision: Query 115: 0.4697 At 5 docs: 0.5320 Run exespfrmgcntbi Query 116: 0.8744 At 10 docs: 0.4560 Query 117: 0.2242 At 15 docs: 0.4093 Comparison to median by topic Query 118: 0.2160 At 20 docs: 0.3760 +1.0 Query 119: 0.8540 At 30 docs: 0.3407 Query 120: 0.1013 At 100 docs: 0.2218 Query 121: 0.7283 At 200 docs: 0.1540 Query 122: 0.0426 At 500 docs: 0.0795 Query 123: 0.8697 At 1000 docs: 0.0442 +0.5 Query 124: 0.6021 Query 125: 0.5928 R-Precision (prec. after R Query 126: 0.1879 (=num_rel) docs. retrieved): Query 127: 0.1337 Exact: 0.4000 Query 128: 0.0805 0.0 Query 129: 0.3779 Query 130: 0.7751 Query 131: 0.6524 Query 132: 0.0048 Query 133: 0.1436 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.1632 Query 136: 0.7332 Query 137: 0.1841 Query 138: 0.9821 Query 139: 0.1405 −1.0 Query 140: 0.3172 91 100 110 120 130 140 Topic number Univ. Exeter, Bilingual German to Spanish, Auto, TD Run exespgemgcntbi Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.7037 Total number of documents Query 92: 0.4525 over all queries Query 93: 0.2580 Retrieved: 50000 0.8 Query 94: 0.1348 Relevant: 2854 Query 95: 0.1434 Rel_ret: 2005 Query 96: 0.4066 Query 97: 0.7573 Interpolated Recall - Query 98: 0.4520 Precision Averages: 0.6 Query 99: 0.0005 at 0.00 0.7400 Query 100: 0.4563 at 0.10 0.5939 Query 101: 0.0543 at 0.20 0.5223 Precision Query 102: 0.6056 at 0.30 0.4254 0.4 Query 103: 0.6885 at 0.40 0.3941 Query 104: 0.2206 at 0.50 0.3536 Query 105: 0.7202 at 0.60 0.3199 Query 106: 0.2230 at 0.70 0.2911 0.2 Query 107: 0.1132 at 0.80 0.2165 Query 108: 0.5638 at 0.90 0.1608 Query 109: 0.3593 at 1.00 0.0769 Query 110: 0.7022 0.0 Query 111: 0.0069 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5789 for all rel. documents Query 113: 0.0085 0.3542 Recall Query 114: 0.0376 Precision: Query 115: 0.4219 At 5 docs: 0.5040 Run exespgemgcntbi Query 116: 0.1418 At 10 docs: 0.4360 Query 117: 0.4335 At 15 docs: 0.3813 Comparison to median by topic Query 118: 0.2244 At 20 docs: 0.3560 +1.0 Query 119: 0.7401 At 30 docs: 0.3180 Query 120: 0.1091 At 100 docs: 0.2082 Query 121: 0.6843 At 200 docs: 0.1428 Query 122: 0.1385 At 500 docs: 0.0718 Query 123: 0.8682 At 1000 docs: 0.0401 +0.5 Query 124: 0.1670 Query 125: 0.3746 R-Precision (prec. after R Query 126: 0.0281 (=num_rel) docs. retrieved): Query 127: 0.0482 Exact: 0.3570 Query 128: 0.1825 0.0 Query 129: 0.2089 Query 130: 0.7159 Query 131: 0.2021 Query 132: 0.0000 Query 133: 0.0852 Difference (average precision) Query 134: 0.7470 −0.5 Query 135: 0.7019 Query 136: 0.7500 Query 137: 0.0437 Query 138: 0.9405 Query 139: 0.0940 −1.0 Query 140: 0.0091 91 100 110 120 130 140 Topic number Univ. Exeter, Bilingual Italian to Spanish, Auto, TD Run exespitmgcntbi Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.2165 Total number of documents Query 92: 0.4582 over all queries Query 93: 0.2491 Retrieved: 50000 0.8 Query 94: 0.1335 Relevant: 2854 Query 95: 0.1665 Rel_ret: 2151 Query 96: 0.1361 Query 97: 0.7752 Interpolated Recall - Query 98: 1.0000 Precision Averages: 0.6 Query 99: 0.0138 at 0.00 0.6795 Query 100: 0.4382 at 0.10 0.5844 Query 101: 0.0265 at 0.20 0.5325 Precision Query 102: 0.6629 at 0.30 0.4701 0.4 Query 103: 0.7030 at 0.40 0.4373 Query 104: 0.2221 at 0.50 0.3934 Query 105: 0.8114 at 0.60 0.3492 Query 106: 0.0151 at 0.70 0.3103 0.2 Query 107: 0.1864 at 0.80 0.2485 Query 108: 0.7704 at 0.90 0.1873 Query 109: 0.1959 at 1.00 0.1178 Query 110: 0.7630 0.0 Query 111: 0.0030 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5324 for all rel. documents Query 113: 0.0009 0.3788 Recall Query 114: 0.0283 Precision: Query 115: 0.3505 At 5 docs: 0.5160 Run exespitmgcntbi Query 116: 0.7987 At 10 docs: 0.4320 Query 117: 0.1171 At 15 docs: 0.4040 Comparison to median by topic Query 118: 0.1548 At 20 docs: 0.3710 +1.0 Query 119: 0.6433 At 30 docs: 0.3233 Query 120: 0.1344 At 100 docs: 0.2038 Query 121: 0.6801 At 200 docs: 0.1397 Query 122: 0.0310 At 500 docs: 0.0747 Query 123: 0.8655 At 1000 docs: 0.0430 +0.5 Query 124: 0.3067 Query 125: 0.6113 R-Precision (prec. after R Query 126: 0.0553 (=num_rel) docs. retrieved): Query 127: 0.2813 Exact: 0.3811 Query 128: 0.0225 0.0 Query 129: 0.3462 Query 130: 0.7649 Query 131: 0.0054 Query 132: 0.0003 Query 133: 0.4946 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.1885 Query 136: 0.6065 Query 137: 0.1321 Query 138: 0.9196 Query 139: 0.4768 −1.0 Query 140: 0.4455 91 100 110 120 130 140 Topic number Océ, Bilingual English to Spanish, Auto, TD Run oce02en2esBF Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.6371 Total number of documents Query 92: 0.2914 over all queries Query 93: 0.2857 Retrieved: 50000 0.8 Query 94: 0.0929 Relevant: 2854 Query 95: 0.2641 Rel_ret: 2063 Query 96: 0.2760 Query 97: 0.1520 Interpolated Recall - 0.6 Query 98: 0.4008 Precision Averages: Query 99: 0.0422 at 0.00 0.7017 Query 100: 0.5525 at 0.10 0.5516 Precision Query 101: 0.0290 at 0.20 0.4759 Query 102: 0.7627 at 0.30 0.3933 0.4 Query 103: 0.5987 at 0.40 0.3621 Query 104: 0.2329 at 0.50 0.3160 Query 105: 0.5339 at 0.60 0.2707 Query 106: 0.1399 at 0.70 0.2327 0.2 Query 107: 0.0719 at 0.80 0.1541 Query 108: 0.3386 at 0.90 0.1135 Query 109: 0.0195 at 1.00 0.0624 Query 110: 0.6630 0.0 Query 111: 0.0140 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5401 for all rel. documents Query 113: 0.0026 0.3153 Recall Query 114: 0.0325 Precision: Query 115: 0.1144 At 5 docs: 0.4520 Run oce02en2esBF Query 116: 0.5021 At 10 docs: 0.3920 Comparison to median by topic Query 117: 0.1220 At 15 docs: 0.3587 +1.0 Query 118: 0.0126 At 20 docs: 0.3380 Query 119: 0.6880 At 30 docs: 0.3007 Query 120: 0.4963 At 100 docs: 0.1880 Query 121: 0.6266 At 200 docs: 0.1354 Query 122: 0.1140 At 500 docs: 0.0718 Query 123: 0.8315 At 1000 docs: 0.0413 +0.5 Query 124: 0.0009 Query 125: 0.0003 R-Precision (prec. after R Query 126: 0.1537 (=num_rel) docs. retrieved): Query 127: 0.6323 Exact: 0.3212 Query 128: 0.1408 0.0 Query 129: 0.2936 Query 130: 0.7091 Query 131: 0.2205 Query 132: 0.0026 Query 133: 0.0320 Difference (average precision) Query 134: 0.9167 −0.5 Query 135: 0.4803 Query 136: 0.7635 Query 137: 0.0350 Query 138: 0.6861 Query 139: 0.0436 −1.0 Query 140: 0.1725 91 100 110 120 130 140 Topic number Océ, Bilingual English to Spanish, Auto, TD Run oce02en2esLO Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.6371 Total number of documents Query 92: 0.4510 over all queries Query 93: 0.2951 Retrieved: 50000 0.8 Query 94: 0.0749 Relevant: 2854 Query 95: 0.1707 Rel_ret: 2218 Query 96: 0.2754 Query 97: 0.1915 Interpolated Recall - 0.6 Query 98: 0.4008 Precision Averages: Query 99: 0.0471 at 0.00 0.7988 Query 100: 0.7626 at 0.10 0.6617 Precision Query 101: 0.0391 at 0.20 0.5828 Query 102: 0.7416 at 0.30 0.4722 0.4 Query 103: 0.6384 at 0.40 0.4281 Query 104: 0.2318 at 0.50 0.3702 Query 105: 0.5473 at 0.60 0.3136 Query 106: 0.2342 at 0.70 0.2678 0.2 Query 107: 0.0616 at 0.80 0.1863 Query 108: 0.5660 at 0.90 0.1341 Query 109: 0.1493 at 1.00 0.0675 Query 110: 0.6850 0.0 Query 111: 0.0147 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5799 for all rel. documents Query 113: 0.0119 0.3699 Recall Query 114: 0.0086 Precision: Query 115: 0.2183 At 5 docs: 0.5560 Run oce02en2esLO Query 116: 0.6090 At 10 docs: 0.4640 Comparison to median by topic Query 117: 0.2754 At 15 docs: 0.4280 +1.0 Query 118: 0.2039 At 20 docs: 0.3900 Query 119: 0.7674 At 30 docs: 0.3480 Query 120: 0.5265 At 100 docs: 0.2186 Query 121: 0.6528 At 200 docs: 0.1518 Query 122: 0.0511 At 500 docs: 0.0786 Query 123: 0.8686 At 1000 docs: 0.0444 +0.5 Query 124: 0.0210 Query 125: 0.4214 R-Precision (prec. after R Query 126: 0.1909 (=num_rel) docs. retrieved): Query 127: 0.6753 Exact: 0.3635 Query 128: 0.1034 0.0 Query 129: 0.1949 Query 130: 0.7825 Query 131: 0.6106 Query 132: 0.0386 Query 133: 0.0674 Difference (average precision) Query 134: 0.7500 −0.5 Query 135: 0.4972 Query 136: 0.7648 Query 137: 0.1769 Query 138: 0.8057 Query 139: 0.0633 −1.0 Query 140: 0.3416 91 100 110 120 130 140 Topic number TLR Research, Bilingual English to Spanish, Auto, TD Run tlren2es Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.3149 Total number of documents Query 92: 0.2512 over all queries Query 93: 0.2536 Retrieved: 50000 0.8 Query 94: 0.0454 Relevant: 2854 Query 95: 0.2560 Rel_ret: 1958 Query 96: 0.0002 Query 97: 0.2747 Interpolated Recall - 0.6 Query 98: 0.5382 Precision Averages: Query 99: 0.0082 at 0.00 0.6436 Query 100: 0.2237 at 0.10 0.5192 Precision Query 101: 0.0254 at 0.20 0.4449 Query 102: 0.8328 at 0.30 0.3683 0.4 Query 103: 0.6278 at 0.40 0.3130 Query 104: 0.2245 at 0.50 0.2806 Query 105: 0.8420 at 0.60 0.2458 Query 106: 0.0754 at 0.70 0.2118 0.2 Query 107: 0.0605 at 0.80 0.1586 Query 108: 0.4973 at 0.90 0.0948 Query 109: 0.2291 at 1.00 0.0460 Query 110: 0.8656 0.0 Query 111: 0.0417 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4439 for all rel. documents Query 113: 0.0080 0.2873 Recall Query 114: 0.1140 Precision: Query 115: 0.2965 At 5 docs: 0.4240 Run tlren2es Query 116: 0.7003 At 10 docs: 0.3840 Comparison to median by topic Query 117: 0.0941 At 15 docs: 0.3400 +1.0 Query 118: 0.0330 At 20 docs: 0.3210 Query 119: 0.7965 At 30 docs: 0.2833 Query 120: 0.0008 At 100 docs: 0.1698 Query 121: 0.0202 At 200 docs: 0.1162 Query 122: 0.0231 At 500 docs: 0.0660 Query 123: 0.7603 At 1000 docs: 0.0392 +0.5 Query 124: 0.5219 Query 125: 0.4399 R-Precision (prec. after R Query 126: 0.1240 (=num_rel) docs. retrieved): Query 127: 0.3615 Exact: 0.2857 Query 128: 0.0373 0.0 Query 129: 0.0582 Query 130: 0.6198 Query 131: 0.6725 Query 132: 0.0000 Query 133: 0.0563 Difference (average precision) Query 134: 0.2794 −0.5 Query 135: 0.0864 Query 136: 0.8027 Query 137: 0.1768 Query 138: 0.0182 Query 139: 0.0279 −1.0 Query 140: 0.3031 91 100 110 120 130 140 Topic number Univ. Jaen/SINAI, Bilingual English to Spanish, Auto, TD Run UJABISP Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.3315 Total number of documents Query 92: 0.3148 over all queries Query 93: 0.3196 Retrieved: 50000 0.8 Query 94: 0.1238 Relevant: 2854 Query 95: 0.1655 Rel_ret: 2158 Query 96: 0.1617 Query 97: 0.1895 Interpolated Recall - 0.6 Query 98: 0.4410 Precision Averages: Query 99: 0.1054 at 0.00 0.6523 Query 100: 0.6608 at 0.10 0.5311 Precision Query 101: 0.0247 at 0.20 0.4669 Query 102: 0.6541 at 0.30 0.3973 0.4 Query 103: 0.2524 at 0.40 0.3611 Query 104: 0.0461 at 0.50 0.2973 Query 105: 0.7533 at 0.60 0.2426 Query 106: 0.0094 at 0.70 0.2029 0.2 Query 107: 0.2832 at 0.80 0.1501 Query 108: 0.6927 at 0.90 0.0999 Query 109: 0.0172 at 1.00 0.0574 Query 110: 0.5792 0.0 Query 111: 0.0000 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4685 for all rel. documents Query 113: 0.0016 0.2991 Recall Query 114: 0.0963 Precision: Query 115: 0.1991 At 5 docs: 0.4120 Run UJABISP Query 116: 0.7030 At 10 docs: 0.3700 Comparison to median by topic Query 117: 0.1997 At 15 docs: 0.3600 +1.0 Query 118: 0.3092 At 20 docs: 0.3360 Query 119: 0.6781 At 30 docs: 0.2987 Query 120: 0.2348 At 100 docs: 0.1904 Query 121: 0.5961 At 200 docs: 0.1376 Query 122: 0.0083 At 500 docs: 0.0751 Query 123: 0.8871 At 1000 docs: 0.0432 +0.5 Query 124: 0.0009 Query 125: 0.0251 R-Precision (prec. after R Query 126: 0.0001 (=num_rel) docs. retrieved): Query 127: 0.6595 Exact: 0.3141 Query 128: 0.0546 0.0 Query 129: 0.2188 Query 130: 0.5307 Query 131: 0.5281 Query 132: 0.0020 Query 133: 0.0054 Difference (average precision) Query 134: 0.3889 −0.5 Query 135: 0.1488 Query 136: 0.3301 Query 137: 0.1125 Query 138: 0.3562 Query 139: 0.5487 −1.0 Query 140: 0.5383 91 100 110 120 130 140 Topic number Université de Neuchâtel, Bilingual English to Spanish, Auto, TD Run UniNEesBi Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.5751 Total number of documents Query 92: 0.5209 over all queries Query 93: 0.3725 Retrieved: 50000 0.8 Query 94: 0.6421 Relevant: 2854 Query 95: 0.4044 Rel_ret: 2387 Query 96: 0.3274 Query 97: 0.2487 Interpolated Recall - 0.6 Query 98: 1.0000 Precision Averages: Query 99: 0.3639 at 0.00 0.8446 Query 100: 0.7595 at 0.10 0.7058 Precision Query 101: 0.3586 at 0.20 0.6527 Query 102: 0.6417 at 0.30 0.5857 0.4 Query 103: 0.8543 at 0.40 0.5348 Query 104: 0.1642 at 0.50 0.4999 Query 105: 0.8221 at 0.60 0.4453 Query 106: 0.2317 at 0.70 0.3949 0.2 Query 107: 0.0872 at 0.80 0.3323 Query 108: 0.7149 at 0.90 0.2369 Query 109: 0.0698 at 1.00 0.1488 Query 110: 0.7886 0.0 Query 111: 0.0210 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5914 for all rel. documents Query 113: 0.0019 0.4763 Recall Query 114: 0.3653 Precision: Query 115: 0.4368 At 5 docs: 0.6280 Run UniNEesBi Query 116: 0.8233 At 10 docs: 0.5420 Comparison to median by topic Query 117: 0.2658 At 15 docs: 0.5000 +1.0 Query 118: 0.0814 At 20 docs: 0.4650 Query 119: 0.8898 At 30 docs: 0.4173 Query 120: 0.4237 At 100 docs: 0.2706 Query 121: 0.6593 At 200 docs: 0.1809 Query 122: 0.0852 At 500 docs: 0.0887 Query 123: 0.9265 At 1000 docs: 0.0477 +0.5 Query 124: 0.2372 Query 125: 0.5121 R-Precision (prec. after R Query 126: 0.2048 (=num_rel) docs. retrieved): Query 127: 0.7356 Exact: 0.4664 Query 128: 0.2399 0.0 Query 129: 0.3930 Query 130: 0.0000 Query 131: 0.0000 Query 132: 0.8821 Query 133: 0.0916 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.8455 Query 136: 0.8273 Query 137: 0.3730 Query 138: 0.9617 Query 139: 0.5850 −1.0 Query 140: 0.4090 91 100 110 120 130 140 Topic number Université de Neuchâtel, Bilingual English to Spanish, Auto, TD Run UniNEesBi2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.4061 Total number of documents Query 92: 0.5130 over all queries Query 93: 0.3578 Retrieved: 50000 0.8 Query 94: 0.7358 Relevant: 2854 Query 95: 0.4071 Rel_ret: 2374 Query 96: 0.2635 Query 97: 0.1192 Interpolated Recall - 0.6 Query 98: 1.0000 Precision Averages: Query 99: 0.3870 at 0.00 0.8243 Query 100: 0.7572 at 0.10 0.6956 Precision Query 101: 0.5474 at 0.20 0.6472 Query 102: 0.6767 at 0.30 0.5949 0.4 Query 103: 0.8402 at 0.40 0.5328 Query 104: 0.2460 at 0.50 0.4957 Query 105: 0.8228 at 0.60 0.4489 Query 106: 0.2160 at 0.70 0.4002 0.2 Query 107: 0.0879 at 0.80 0.3400 Query 108: 0.6709 at 0.90 0.2393 Query 109: 0.0497 at 1.00 0.1532 Query 110: 0.7478 0.0 Query 111: 0.0208 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5844 for all rel. documents Query 113: 0.0015 0.4786 Recall Query 114: 0.3261 Precision: Query 115: 0.4114 At 5 docs: 0.6320 Run UniNEesBi2 Query 116: 0.8234 At 10 docs: 0.5460 Comparison to median by topic Query 117: 0.2829 At 15 docs: 0.4973 +1.0 Query 118: 0.0827 At 20 docs: 0.4640 Query 119: 0.9075 At 30 docs: 0.4160 Query 120: 0.3966 At 100 docs: 0.2694 Query 121: 0.6630 At 200 docs: 0.1814 Query 122: 0.0789 At 500 docs: 0.0881 Query 123: 0.9253 At 1000 docs: 0.0475 +0.5 Query 124: 0.2826 Query 125: 0.6878 R-Precision (prec. after R Query 126: 0.1988 (=num_rel) docs. retrieved): Query 127: 0.7363 Exact: 0.4633 Query 128: 0.3156 0.0 Query 129: 0.3924 Query 130: 0.0000 Query 131: 0.0000 Query 132: 0.8872 Query 133: 0.0159 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.8182 Query 136: 0.8282 Query 137: 0.3436 Query 138: 0.9478 Query 139: 0.7083 −1.0 Query 140: 0.4086 91 100 110 120 130 140 Topic number Université de Neuchâtel, Bilingual English to Spanish, Auto, TD Run UniNEesBi3 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.5035 Total number of documents Query 92: 0.5117 over all queries Query 93: 0.3780 Retrieved: 50000 0.8 Query 94: 0.6361 Relevant: 2854 Query 95: 0.3763 Rel_ret: 2354 Query 96: 0.2229 Query 97: 0.0203 Interpolated Recall - 0.6 Query 98: 1.0000 Precision Averages: Query 99: 0.4667 at 0.00 0.7874 Query 100: 0.7690 at 0.10 0.7013 Precision Query 101: 0.6000 at 0.20 0.6452 Query 102: 0.6343 at 0.30 0.6012 0.4 Query 103: 0.8185 at 0.40 0.5413 Query 104: 0.2468 at 0.50 0.4996 Query 105: 0.8232 at 0.60 0.4498 Query 106: 0.1615 at 0.70 0.3999 0.2 Query 107: 0.0938 at 0.80 0.3359 Query 108: 0.7040 at 0.90 0.2379 Query 109: 0.0663 at 1.00 0.1508 Query 110: 0.7460 0.0 Query 111: 0.0214 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5921 for all rel. documents Query 113: 0.0006 0.4784 Recall Query 114: 0.2930 Precision: Query 115: 0.4117 At 5 docs: 0.6360 Run UniNEesBi3 Query 116: 0.8391 At 10 docs: 0.5400 Comparison to median by topic Query 117: 0.2701 At 15 docs: 0.4907 +1.0 Query 118: 0.1017 At 20 docs: 0.4550 Query 119: 0.9145 At 30 docs: 0.4107 Query 120: 0.4046 At 100 docs: 0.2702 Query 121: 0.7097 At 200 docs: 0.1809 Query 122: 0.0626 At 500 docs: 0.0872 Query 123: 0.9232 At 1000 docs: 0.0471 +0.5 Query 124: 0.3393 Query 125: 0.4679 R-Precision (prec. after R Query 126: 0.1927 (=num_rel) docs. retrieved): Query 127: 0.7486 Exact: 0.4661 Query 128: 0.4949 0.0 Query 129: 0.4112 Query 130: 0.0000 Query 131: 0.0000 Query 132: 0.8786 Query 133: 0.0082 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.8087 Query 136: 0.8275 Query 137: 0.3382 Query 138: 0.9617 Query 139: 0.7384 −1.0 Query 140: 0.3816 91 100 110 120 130 140 Topic number JHU/APL, Bilingual English to Finnish, Auto, TD Run aplbienfi Average precision Overall statistics Recall−Precision Values (individual queries): (for 30 queries): 1.0 Query 92: 0.4844 Total number of documents Query 94: 0.2279 over all queries Query 95: 0.3692 Retrieved: 30000 0.8 Query 98: 0.0804 Relevant: 502 Query 100: 0.1882 Rel_ret: 388 Query 102: 0.9345 Query 103: 0.1908 Interpolated Recall - Query 105: 0.0711 Precision Averages: 0.6 Query 106: 0.0815 at 0.00 0.3742 Query 107: 0.0108 at 0.10 0.3088 Query 109: 0.0077 at 0.20 0.2762 Precision Query 111: 0.0003 at 0.30 0.2489 0.4 Query 114: 0.0435 at 0.40 0.2294 Query 115: 0.0052 at 0.50 0.2143 Query 116: 0.0005 at 0.60 0.1764 Query 118: 0.0080 at 0.70 0.1610 0.2 Query 119: 0.1435 at 0.80 0.1543 Query 122: 0.0869 at 0.90 0.1194 Query 123: 0.4500 at 1.00 0.0935 Query 124: 0.3997 0.0 Query 125: 0.0818 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 126: 0.5142 for all rel. documents Query 128: 0.0214 0.2003 Recall Query 130: 0.4063 Precision: Query 131: 0.0494 At 5 docs: 0.2400 Run aplbienfi Query 132: 1.0000 At 10 docs: 0.2167 Query 137: 0.0000 At 15 docs: 0.1756 Comparison to median by topic Query 138: 0.0115 At 20 docs: 0.1683 +1.0 Query 139: 0.0680 At 30 docs: 0.1556 Query 140: 0.0723 At 100 docs: 0.0740 At 200 docs: 0.0440 At 500 docs: 0.0215 At 1000 docs: 0.0129 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2134 0.0 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Univ. Tampere, Dept. IS, Bilingual English to Finnish, Auto, TD Run finbi2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 30 queries): 1.0 Query 92: 0.0086 Total number of documents Query 94: 0.2010 over all queries Query 95: 0.0002 Retrieved: 30000 0.8 Query 98: 0.6441 Relevant: 502 Query 100: 0.0427 Rel_ret: 348 Query 102: 0.0231 Query 103: 0.0314 Interpolated Recall - 0.6 Query 105: 0.5292 Precision Averages: Query 106: 0.0501 at 0.00 0.4022 Query 107: 0.0167 at 0.10 0.3558 Precision Query 109: 0.1784 at 0.20 0.2916 Query 111: 0.0544 at 0.30 0.2557 0.4 Query 114: 0.0045 at 0.40 0.2182 Query 115: 0.4461 at 0.50 0.2077 Query 116: 0.0000 at 0.60 0.1793 Query 118: 0.2302 at 0.70 0.1340 0.2 Query 119: 0.0067 at 0.80 0.1150 Query 122: 0.0245 at 0.90 0.1081 Query 123: 0.8333 at 1.00 0.0944 Query 124: 0.0549 0.0 Query 125: 0.1674 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 126: 0.1247 for all rel. documents Query 128: 0.0009 0.2016 Recall Query 130: 0.3263 Precision: Query 131: 0.0036 At 5 docs: 0.2267 Run finbi2 Query 132: 1.0000 At 10 docs: 0.1700 Comparison to median by topic Query 137: 0.0192 At 15 docs: 0.1667 +1.0 Query 138: 0.0524 At 20 docs: 0.1417 Query 139: 0.7228 At 30 docs: 0.1222 Query 140: 0.2500 At 100 docs: 0.0633 At 200 docs: 0.0412 At 500 docs: 0.0209 At 1000 docs: 0.0116 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.1790 0.0 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number JHU/APL, Bilingual English to French, Auto, TD Run aplbienfr Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0147 Total number of documents Query 92: 0.4854 over all queries Query 93: 0.0002 Retrieved: 50000 0.8 Query 94: 0.6918 Relevant: 1383 Query 95: 0.3503 Rel_ret: 1275 Query 96: 0.3116 Query 97: 0.1271 Interpolated Recall - Query 98: 0.4177 Precision Averages: 0.6 Query 99: 0.0762 at 0.00 0.6485 Query 100: 0.4398 at 0.10 0.5657 Query 101: 0.3891 at 0.20 0.5138 Precision Query 102: 0.7328 at 0.30 0.4601 0.4 Query 103: 0.1810 at 0.40 0.3938 Query 104: 0.5869 at 0.50 0.3700 Query 105: 0.8333 at 0.60 0.3170 Query 106: 0.1794 at 0.70 0.2710 0.2 Query 107: 0.3031 at 0.80 0.2218 Query 108: 0.4054 at 0.90 0.1595 Query 109: 0.0512 at 1.00 0.0963 Query 110: 0.3908 0.0 Query 111: 0.1564 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4156 for all rel. documents Query 113: 0.0293 0.3505 Recall Query 114: 0.3867 Precision: Query 115: 0.6444 At 5 docs: 0.4000 Run aplbienfr Query 116: 0.0146 At 10 docs: 0.3620 Query 117: 0.0691 At 15 docs: 0.3373 Comparison to median by topic Query 118: 0.0129 At 20 docs: 0.3130 +1.0 Query 119: 0.9371 At 30 docs: 0.2800 Query 120: 0.0069 At 100 docs: 0.1630 Query 121: 0.2500 At 200 docs: 0.0970 Query 122: 0.5799 At 500 docs: 0.0460 Query 123: 0.3667 At 1000 docs: 0.0255 +0.5 Query 124: 0.1593 Query 125: 0.4274 R-Precision (prec. after R Query 126: 0.7119 (=num_rel) docs. retrieved): Query 127: 0.1895 Exact: 0.3144 Query 128: 0.0516 0.0 Query 129: 0.4381 Query 130: 0.6850 Query 131: 0.6292 Query 132: 0.2472 Query 133: 0.0013 Difference (average precision) Query 134: 0.4038 −0.5 Query 135: 0.7455 Query 136: 0.0000 Query 137: 0.3568 Query 138: 0.7449 Query 139: 0.3121 −1.0 Query 140: 0.5821 91 100 110 120 130 140 Topic number Univ. Tampere, Dept. IS, Bilingual English to French, Auto, TD Run bifren Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0037 Total number of documents Query 92: 0.0004 over all queries Query 93: 1.0000 Retrieved: 50000 0.8 Query 94: 0.0012 Relevant: 1383 Query 95: 0.0053 Rel_ret: 733 Query 96: 0.1797 Query 97: 0.1190 Interpolated Recall - 0.6 Query 98: 0.1111 Precision Averages: Query 99: 0.0084 at 0.00 0.4483 Query 100: 0.2931 at 0.10 0.4038 Precision Query 101: 0.0000 at 0.20 0.3311 Query 102: 0.6405 at 0.30 0.2916 0.4 Query 103: 0.1733 at 0.40 0.2569 Query 104: 0.0095 at 0.50 0.2392 Query 105: 0.2429 at 0.60 0.2237 Query 106: 0.0118 at 0.70 0.1809 0.2 Query 107: 0.0011 at 0.80 0.1530 Query 108: 0.0965 at 0.90 0.1293 Query 109: 0.3889 at 1.00 0.1167 Query 110: 0.4081 0.0 Query 111: 0.0065 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.2839 for all rel. documents Query 113: 0.0066 0.2386 Recall Query 114: 0.3087 Precision: Query 115: 0.5316 At 5 docs: 0.2240 Run bifren Query 116: 0.5315 At 10 docs: 0.1900 Comparison to median by topic Query 117: 0.0093 At 15 docs: 0.1640 +1.0 Query 118: 0.0811 At 20 docs: 0.1490 Query 119: 0.0535 At 30 docs: 0.1213 Query 120: 0.0027 At 100 docs: 0.0630 Query 121: 1.0000 At 200 docs: 0.0428 Query 122: 0.0510 At 500 docs: 0.0245 Query 123: 1.0000 At 1000 docs: 0.0147 +0.5 Query 124: 0.0066 Query 125: 0.0563 R-Precision (prec. after R Query 126: 0.3775 (=num_rel) docs. retrieved): Query 127: 0.0057 Exact: 0.2452 Query 128: 0.0016 0.0 Query 129: 0.0265 Query 130: 0.5302 Query 131: 0.2183 Query 132: 0.5040 Query 133: 0.0006 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.0482 Query 136: 1.0000 Query 137: 0.0592 Query 138: 0.0001 Query 139: 0.0377 −1.0 Query 140: 0.4952 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 1, Bilingual English to French, Auto, TD Run BKBIEF1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0793 Total number of documents Query 92: 0.5275 over all queries Query 93: 1.0000 Retrieved: 50000 0.8 Query 94: 0.6184 Relevant: 1383 Query 95: 0.4030 Rel_ret: 1285 Query 96: 0.5485 Query 97: 0.5871 Interpolated Recall - 0.6 Query 98: 1.0000 Precision Averages: Query 99: 0.4073 at 0.00 0.6808 Query 100: 0.3016 at 0.10 0.6284 Precision Query 101: 0.1121 at 0.20 0.5642 Query 102: 0.5103 at 0.30 0.5210 0.4 Query 103: 0.4025 at 0.40 0.4962 Query 104: 0.0594 at 0.50 0.4702 Query 105: 0.3611 at 0.60 0.4260 Query 106: 0.2939 at 0.70 0.3713 0.2 Query 107: 0.0022 at 0.80 0.3302 Query 108: 0.3641 at 0.90 0.2626 Query 109: 0.0220 at 1.00 0.1868 Query 110: 0.6779 0.0 Query 111: 0.3815 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4588 for all rel. documents Query 113: 0.0033 0.4312 Recall Query 114: 0.5614 Precision: Query 115: 0.4917 At 5 docs: 0.4840 Run BKBIEF1 Query 116: 0.6177 At 10 docs: 0.4360 Comparison to median by topic Query 117: 0.1961 At 15 docs: 0.3827 +1.0 Query 118: 0.1498 At 20 docs: 0.3550 Query 119: 0.9126 At 30 docs: 0.3060 Query 120: 0.0379 At 100 docs: 0.1740 Query 121: 0.5000 At 200 docs: 0.1045 Query 122: 0.5236 At 500 docs: 0.0482 Query 123: 0.7000 At 1000 docs: 0.0257 +0.5 Query 124: 0.0601 Query 125: 0.2633 R-Precision (prec. after R Query 126: 0.3976 (=num_rel) docs. retrieved): Query 127: 0.7429 Exact: 0.4210 Query 128: 0.0209 0.0 Query 129: 0.5267 Query 130: 0.5233 Query 131: 0.5414 Query 132: 0.2294 Query 133: 0.0019 Difference (average precision) Query 134: 0.8095 −0.5 Query 135: 0.7787 Query 136: 1.0000 Query 137: 0.0034 Query 138: 0.5668 Query 139: 0.4784 −1.0 Query 140: 0.8008 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 1, Bilingual English to French, Auto, TDN Run BKBIEF2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0247 Total number of documents Query 92: 0.0001 over all queries Query 93: 0.0000 Retrieved: 50000 0.8 Query 94: 0.0000 Relevant: 1383 Query 95: 0.0001 Rel_ret: 162 Query 96: 0.0190 Query 97: 0.0000 Interpolated Recall - 0.6 Query 98: 0.4623 Precision Averages: Query 99: 0.0000 at 0.00 0.0840 Query 100: 0.0001 at 0.10 0.0795 Precision Query 101: 0.0000 at 0.20 0.0695 Query 102: 0.0075 at 0.30 0.0693 0.4 Query 103: 0.0000 at 0.40 0.0672 Query 104: 0.0000 at 0.50 0.0669 Query 105: 0.0000 at 0.60 0.0612 Query 106: 0.0000 at 0.70 0.0481 0.2 Query 107: 0.0003 at 0.80 0.0411 Query 108: 0.0000 at 0.90 0.0242 Query 109: 0.0006 at 1.00 0.0188 Query 110: 0.5964 0.0 Query 111: 0.0024 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.3502 for all rel. documents Query 113: 0.0000 0.0513 Recall Query 114: 0.0001 Precision: Query 115: 0.0231 At 5 docs: 0.0400 Run BKBIEF2 Query 116: 0.0020 At 10 docs: 0.0500 Comparison to median by topic Query 117: 0.0000 At 15 docs: 0.0533 +1.0 Query 118: 0.0000 At 20 docs: 0.0520 Query 119: 0.0000 At 30 docs: 0.0440 Query 120: 0.0000 At 100 docs: 0.0184 Query 121: 0.1250 At 200 docs: 0.0107 Query 122: 0.0000 At 500 docs: 0.0056 Query 123: 0.0409 At 1000 docs: 0.0032 +0.5 Query 124: 0.0000 Query 125: 0.0000 R-Precision (prec. after R Query 126: 0.0012 (=num_rel) docs. retrieved): Query 127: 0.4039 Exact: 0.0488 Query 128: 0.0000 0.0 Query 129: 0.0000 Query 130: 0.5002 Query 131: 0.0000 Query 132: 0.0009 Query 133: 0.0000 Difference (average precision) Query 134: 0.0015 −0.5 Query 135: 0.0000 Query 136: 0.0000 Query 137: 0.0000 Query 138: 0.0000 Query 139: 0.0000 −1.0 Query 140: 0.0001 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 1, Bilingual German to French, Auto, TD Run BKBIGF1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1870 Total number of documents Query 92: 0.4277 over all queries Query 93: 1.0000 Retrieved: 50000 0.8 Query 94: 0.6522 Relevant: 1383 Query 95: 0.2894 Rel_ret: 1303 Query 96: 0.5531 Query 97: 0.3971 Interpolated Recall - 0.6 Query 98: 1.0000 Precision Averages: Query 99: 0.0599 at 0.00 0.6759 Query 100: 0.3820 at 0.10 0.6271 Precision Query 101: 0.2519 at 0.20 0.5582 Query 102: 0.5407 at 0.30 0.4818 0.4 Query 103: 0.4030 at 0.40 0.4589 Query 104: 0.1743 at 0.50 0.4389 Query 105: 0.2101 at 0.60 0.3986 Query 106: 0.4205 at 0.70 0.3428 0.2 Query 107: 0.0000 at 0.80 0.2972 Query 108: 0.3559 at 0.90 0.2330 Query 109: 0.2614 at 1.00 0.1686 Query 110: 0.6653 0.0 Query 111: 0.2236 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4618 for all rel. documents Query 113: 0.4889 0.4100 Recall Query 114: 0.5133 Precision: Query 115: 0.2167 At 5 docs: 0.4560 Run BKBIGF1 Query 116: 0.5027 At 10 docs: 0.4180 Comparison to median by topic Query 117: 0.1586 At 15 docs: 0.3680 +1.0 Query 118: 0.1678 At 20 docs: 0.3330 Query 119: 0.8818 At 30 docs: 0.2953 Query 120: 0.0352 At 100 docs: 0.1704 Query 121: 0.1111 At 200 docs: 0.1044 Query 122: 0.4206 At 500 docs: 0.0490 Query 123: 0.8333 At 1000 docs: 0.0261 +0.5 Query 124: 0.4572 Query 125: 0.4137 R-Precision (prec. after R Query 126: 0.0129 (=num_rel) docs. retrieved): Query 127: 0.7410 Exact: 0.3854 Query 128: 0.0285 0.0 Query 129: 0.3167 Query 130: 0.6185 Query 131: 0.5420 Query 132: 0.0011 Query 133: 0.0096 Difference (average precision) Query 134: 0.7917 −0.5 Query 135: 0.7457 Query 136: 1.0000 Query 137: 0.0000 Query 138: 0.6550 Query 139: 0.2002 −1.0 Query 140: 0.7203 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 1, Bilingual Russian to French, Auto, TD Run BKBIRF1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1216 Total number of documents Query 92: 0.4667 over all queries Query 93: 0.0000 Retrieved: 50000 0.8 Query 94: 0.6266 Relevant: 1383 Query 95: 0.4330 Rel_ret: 1211 Query 96: 0.3932 Query 97: 0.0027 Interpolated Recall - 0.6 Query 98: 0.7178 Precision Averages: Query 99: 0.0067 at 0.00 0.5686 Query 100: 0.6117 at 0.10 0.5117 Precision Query 101: 0.4298 at 0.20 0.4726 Query 102: 0.2861 at 0.30 0.4312 0.4 Query 103: 0.5713 at 0.40 0.3841 Query 104: 0.0361 at 0.50 0.3312 Query 105: 0.4167 at 0.60 0.3022 Query 106: 0.3731 at 0.70 0.2656 0.2 Query 107: 0.0000 at 0.80 0.2283 Query 108: 0.3881 at 0.90 0.1674 Query 109: 0.0690 at 1.00 0.1093 Query 110: 0.6570 0.0 Query 111: 0.0186 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4114 for all rel. documents Query 113: 0.0463 0.3276 Recall Query 114: 0.2790 Precision: Query 115: 0.4623 At 5 docs: 0.3760 Run BKBIRF1 Query 116: 0.0082 At 10 docs: 0.3560 Comparison to median by topic Query 117: 0.2124 At 15 docs: 0.3200 +1.0 Query 118: 0.5494 At 20 docs: 0.3000 Query 119: 0.9153 At 30 docs: 0.2667 Query 120: 0.0040 At 100 docs: 0.1572 Query 121: 0.1111 At 200 docs: 0.0962 Query 122: 0.5396 At 500 docs: 0.0451 Query 123: 0.8333 At 1000 docs: 0.0242 +0.5 Query 124: 0.0150 Query 125: 0.0123 R-Precision (prec. after R Query 126: 0.0154 (=num_rel) docs. retrieved): Query 127: 0.3156 Exact: 0.2895 Query 128: 0.0306 0.0 Query 129: 0.4876 Query 130: 0.5920 Query 131: 0.5297 Query 132: 0.4299 Query 133: 0.0000 Difference (average precision) Query 134: 0.4189 −0.5 Query 135: 0.8255 Query 136: 0.0097 Query 137: 0.0014 Query 138: 0.6493 Query 139: 0.2523 −1.0 Query 140: 0.7976 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 2, Bilingual German to French, Auto, TD Run bky2bidefr Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.2053 Total number of documents Query 92: 0.3751 over all queries Query 93: 0.9267 Retrieved: 50000 0.8 Query 94: 0.0789 Relevant: 1383 Query 95: 0.0006 Rel_ret: 1160 Query 96: 0.4836 Query 97: 0.4286 Interpolated Recall - Query 98: 0.2377 Precision Averages: 0.6 Query 99: 0.0742 at 0.00 0.6765 Query 100: 0.3428 at 0.10 0.6096 Query 101: 0.0643 at 0.20 0.5740 Precision Query 102: 0.6378 at 0.30 0.5110 0.4 Query 103: 0.6392 at 0.40 0.4696 Query 104: 0.1214 at 0.50 0.4391 Query 105: 0.0742 at 0.60 0.4026 Query 106: 0.3903 at 0.70 0.3344 0.2 Query 107: 0.0981 at 0.80 0.2908 Query 108: 0.3686 at 0.90 0.2350 Query 109: 0.6970 at 1.00 0.1692 Query 110: 0.7846 0.0 Query 111: 0.1618 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6052 for all rel. documents Query 113: 0.4867 0.4124 Recall Query 114: 0.5556 Precision: Query 115: 0.0273 At 5 docs: 0.4440 Run bky2bidefr Query 116: 0.6827 At 10 docs: 0.4120 Query 117: 0.1770 At 15 docs: 0.3627 Comparison to median by topic Query 118: 0.1099 At 20 docs: 0.3320 +1.0 Query 119: 0.8636 At 30 docs: 0.2960 Query 120: 0.0332 At 100 docs: 0.1654 Query 121: 1.0000 At 200 docs: 0.0971 Query 122: 0.6504 At 500 docs: 0.0441 Query 123: 0.8333 At 1000 docs: 0.0232 +0.5 Query 124: 0.5744 Query 125: 0.3441 R-Precision (prec. after R Query 126: 0.0526 (=num_rel) docs. retrieved): Query 127: 0.6405 Exact: 0.4027 Query 128: 0.1039 0.0 Query 129: 0.0984 Query 130: 0.5558 Query 131: 0.5171 Query 132: 0.0000 Query 133: 0.0879 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.7927 Query 136: 1.0000 Query 137: 0.0567 Query 138: 0.6300 Query 139: 0.3313 −1.0 Query 140: 0.6205 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 2, Bilingual English to French, Auto, TD Run bky2bienfr Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0063 Total number of documents Query 92: 0.4362 over all queries Query 93: 0.9267 Retrieved: 50000 0.8 Query 94: 0.5993 Relevant: 1383 Query 95: 0.4321 Rel_ret: 1319 Query 96: 0.5522 Query 97: 0.5303 Interpolated Recall - Query 98: 0.9086 Precision Averages: 0.6 Query 99: 0.5227 at 0.00 0.7379 Query 100: 0.3223 at 0.10 0.6933 Query 101: 0.1093 at 0.20 0.6234 Precision Query 102: 0.6397 at 0.30 0.5782 0.4 Query 103: 0.6507 at 0.40 0.5475 Query 104: 0.1576 at 0.50 0.5090 Query 105: 0.3125 at 0.60 0.4731 Query 106: 0.3429 at 0.70 0.4200 0.2 Query 107: 0.1340 at 0.80 0.3730 Query 108: 0.3496 at 0.90 0.2898 Query 109: 0.0618 at 1.00 0.2088 Query 110: 0.7515 0.0 Query 111: 0.1209 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5983 for all rel. documents Query 113: 0.0006 0.4773 Recall Query 114: 0.5480 Precision: Query 115: 0.5733 At 5 docs: 0.5200 Run bky2bienfr Query 116: 0.6775 At 10 docs: 0.4560 Query 117: 0.3392 At 15 docs: 0.4173 Comparison to median by topic Query 118: 0.1237 At 20 docs: 0.3840 +1.0 Query 119: 0.8950 At 30 docs: 0.3373 Query 120: 0.0332 At 100 docs: 0.1810 Query 121: 1.0000 At 200 docs: 0.1103 Query 122: 0.6558 At 500 docs: 0.0501 Query 123: 0.8333 At 1000 docs: 0.0264 +0.5 Query 124: 0.3011 Query 125: 0.2420 R-Precision (prec. after R Query 126: 0.3941 (=num_rel) docs. retrieved): Query 127: 0.8043 Exact: 0.4610 Query 128: 0.0550 0.0 Query 129: 0.5081 Query 130: 0.5365 Query 131: 0.5807 Query 132: 0.3917 Query 133: 0.3389 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.8125 Query 136: 1.0000 Query 137: 0.0674 Query 138: 0.3521 Query 139: 0.5105 −1.0 Query 140: 0.8242 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 2, Bilingual English to French, Auto, TD Run bky2bienfr2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0050 Total number of documents Query 92: 0.4192 over all queries Query 93: 0.9267 Retrieved: 50000 0.8 Query 94: 0.6779 Relevant: 1383 Query 95: 0.4523 Rel_ret: 1311 Query 96: 0.5089 Query 97: 0.5143 Interpolated Recall - Query 98: 0.9086 Precision Averages: 0.6 Query 99: 0.6064 at 0.00 0.7290 Query 100: 0.2979 at 0.10 0.6764 Query 101: 0.3299 at 0.20 0.6173 Precision Query 102: 0.6591 at 0.30 0.5758 0.4 Query 103: 0.6796 at 0.40 0.5490 Query 104: 0.1195 at 0.50 0.5224 Query 105: 0.3125 at 0.60 0.4740 Query 106: 0.3469 at 0.70 0.4094 0.2 Query 107: 0.1574 at 0.80 0.3638 Query 108: 0.3744 at 0.90 0.2901 Query 109: 0.1201 at 1.00 0.1985 Query 110: 0.7063 0.0 Query 111: 0.1797 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6150 for all rel. documents Query 113: 0.0043 0.4744 Recall Query 114: 0.5542 Precision: Query 115: 0.6884 At 5 docs: 0.5160 Run bky2bienfr2 Query 116: 0.7961 At 10 docs: 0.4700 Query 117: 0.2720 At 15 docs: 0.4360 Comparison to median by topic Query 118: 0.1449 At 20 docs: 0.3910 +1.0 Query 119: 0.9072 At 30 docs: 0.3440 Query 120: 0.0377 At 100 docs: 0.1834 Query 121: 0.5000 At 200 docs: 0.1096 Query 122: 0.6513 At 500 docs: 0.0500 Query 123: 0.8333 At 1000 docs: 0.0262 +0.5 Query 124: 0.2594 Query 125: 0.3422 R-Precision (prec. after R Query 126: 0.3673 (=num_rel) docs. retrieved): Query 127: 0.8229 Exact: 0.4462 Query 128: 0.1242 0.0 Query 129: 0.4737 Query 130: 0.5129 Query 131: 0.5983 Query 132: 0.2453 Query 133: 0.1190 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.8277 Query 136: 1.0000 Query 137: 0.0344 Query 138: 0.3477 Query 139: 0.5063 −1.0 Query 140: 0.8307 91 100 110 120 130 140 Topic number TLR Research, Bilingual English to French, Auto, TD Run tlren2fr Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1037 Total number of documents Query 92: 0.1606 over all queries Query 93: 0.0000 Retrieved: 50000 0.8 Query 94: 0.0624 Relevant: 1383 Query 95: 0.1272 Rel_ret: 1125 Query 96: 0.0097 Query 97: 0.2575 Interpolated Recall - 0.6 Query 98: 0.6905 Precision Averages: Query 99: 0.3639 at 0.00 0.6422 Query 100: 0.1179 at 0.10 0.5340 Precision Query 101: 0.1546 at 0.20 0.4764 Query 102: 0.6619 at 0.30 0.4280 0.4 Query 103: 0.5452 at 0.40 0.3662 Query 104: 0.3567 at 0.50 0.3210 Query 105: 0.3088 at 0.60 0.2782 Query 106: 0.0439 at 0.70 0.2319 0.2 Query 107: 0.0886 at 0.80 0.1825 Query 108: 0.2067 at 0.90 0.1364 Query 109: 0.3908 at 1.00 0.1020 Query 110: 0.6097 0.0 Query 111: 0.2078 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.3773 for all rel. documents Query 113: 0.0025 0.3198 Recall Query 114: 0.4085 Precision: Query 115: 0.6690 At 5 docs: 0.3560 Run tlren2fr Query 116: 0.3772 At 10 docs: 0.3240 Comparison to median by topic Query 117: 0.1613 At 15 docs: 0.2800 +1.0 Query 118: 0.0025 At 20 docs: 0.2530 Query 119: 0.7556 At 30 docs: 0.2147 Query 120: 0.0003 At 100 docs: 0.1234 Query 121: 1.0000 At 200 docs: 0.0798 Query 122: 0.1207 At 500 docs: 0.0402 Query 123: 1.0000 At 1000 docs: 0.0225 +0.5 Query 124: 0.5758 Query 125: 0.3519 R-Precision (prec. after R Query 126: 0.4152 (=num_rel) docs. retrieved): Query 127: 0.5707 Exact: 0.3440 Query 128: 0.0146 0.0 Query 129: 0.0141 Query 130: 0.4319 Query 131: 0.6547 Query 132: 0.2681 Query 133: 0.0588 Difference (average precision) Query 134: 0.6250 −0.5 Query 135: 0.0678 Query 136: 1.0000 Query 137: 0.0120 Query 138: 0.1306 Query 139: 0.0284 −1.0 Query 140: 0.4260 91 100 110 120 130 140 Topic number Univ. Jaen/SINAI, Bilingual English to French, Auto, TD Run UJABIFR Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0115 Total number of documents Query 92: 0.3871 over all queries Query 93: 0.9267 Retrieved: 50000 0.8 Query 94: 0.0705 Relevant: 1383 Query 95: 0.3351 Rel_ret: 1214 Query 96: 0.3679 Query 97: 0.3537 Interpolated Recall - 0.6 Query 98: 0.8017 Precision Averages: Query 99: 0.0666 at 0.00 0.6684 Query 100: 0.5449 at 0.10 0.5586 Precision Query 101: 0.1756 at 0.20 0.4917 Query 102: 0.7175 at 0.30 0.4348 0.4 Query 103: 0.4591 at 0.40 0.4033 Query 104: 0.0229 at 0.50 0.3668 Query 105: 0.4167 at 0.60 0.3185 Query 106: 0.2135 at 0.70 0.2647 0.2 Query 107: 0.1217 at 0.80 0.2120 Query 108: 0.2653 at 0.90 0.1530 Query 109: 0.0859 at 1.00 0.1186 Query 110: 0.6831 0.0 Query 111: 0.0633 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.3668 for all rel. documents Query 113: 0.0000 0.3467 Recall Query 114: 0.3048 Precision: Query 115: 0.1106 At 5 docs: 0.4200 Run UJABIFR Query 116: 0.5537 At 10 docs: 0.3500 Comparison to median by topic Query 117: 0.3450 At 15 docs: 0.3227 +1.0 Query 118: 0.0942 At 20 docs: 0.2880 Query 119: 0.7593 At 30 docs: 0.2533 Query 120: 0.0112 At 100 docs: 0.1350 Query 121: 0.3333 At 200 docs: 0.0858 Query 122: 0.1932 At 500 docs: 0.0439 Query 123: 1.0000 At 1000 docs: 0.0243 +0.5 Query 124: 0.0893 Query 125: 0.1347 R-Precision (prec. after R Query 126: 0.5526 (=num_rel) docs. retrieved): Query 127: 0.4617 Exact: 0.3365 Query 128: 0.1391 0.0 Query 129: 0.1413 Query 130: 0.4948 Query 131: 0.3666 Query 132: 0.4491 Query 133: 0.0794 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.3977 Query 136: 1.0000 Query 137: 0.0056 Query 138: 0.0948 Query 139: 0.2001 −1.0 Query 140: 0.5673 91 100 110 120 130 140 Topic number Université de Neuchâtel, Bilingual English to French, Auto, TD Run UniNEfrBi Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0035 Total number of documents Query 92: 0.3965 over all queries Query 93: 1.0000 Retrieved: 50000 0.8 Query 94: 0.6254 Relevant: 1383 Query 95: 0.4008 Rel_ret: 1214 Query 96: 0.5946 Query 97: 0.1066 Interpolated Recall - 0.6 Query 98: 0.9361 Precision Averages: Query 99: 0.2644 at 0.00 0.7728 Query 100: 0.6066 at 0.10 0.7160 Precision Query 101: 0.6353 at 0.20 0.6611 Query 102: 0.6923 at 0.30 0.6065 0.4 Query 103: 0.6838 at 0.40 0.5657 Query 104: 0.0469 at 0.50 0.5393 Query 105: 0.6111 at 0.60 0.4800 Query 106: 0.4200 at 0.70 0.4011 0.2 Query 107: 0.0603 at 0.80 0.3489 Query 108: 0.4219 at 0.90 0.2533 Query 109: 0.6961 at 1.00 0.1871 Query 110: 0.6810 0.0 Query 111: 0.4687 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4060 for all rel. documents Query 113: 0.0012 0.4935 Recall Query 114: 0.5258 Precision: Query 115: 0.8154 At 5 docs: 0.5400 Run UniNEfrBi Query 116: 0.6441 At 10 docs: 0.4740 Comparison to median by topic Query 117: 0.3512 At 15 docs: 0.4160 +1.0 Query 118: 0.0978 At 20 docs: 0.3660 Query 119: 0.8632 At 30 docs: 0.3047 Query 120: 0.0222 At 100 docs: 0.1652 Query 121: 1.0000 At 200 docs: 0.0999 Query 122: 0.3171 At 500 docs: 0.0460 Query 123: 0.8333 At 1000 docs: 0.0243 +0.5 Query 124: 0.4849 Query 125: 0.3970 R-Precision (prec. after R Query 126: 0.7576 (=num_rel) docs. retrieved): Query 127: 0.9492 Exact: 0.4863 Query 128: 0.0779 0.0 Query 129: 0.4111 Query 130: 0.0000 Query 131: 0.0000 Query 132: 0.5405 Query 133: 0.2067 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.4048 Query 136: 1.0000 Query 137: 0.3042 Query 138: 0.3441 Query 139: 0.7578 −1.0 Query 140: 0.8096 91 100 110 120 130 140 Topic number Université de Neuchâtel, Bilingual English to French, Auto, TD Run UniNEfrBi2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0033 Total number of documents Query 92: 0.4189 over all queries Query 93: 1.0000 Retrieved: 50000 0.8 Query 94: 0.5394 Relevant: 1383 Query 95: 0.3862 Rel_ret: 1210 Query 96: 0.5993 Query 97: 0.1294 Interpolated Recall - 0.6 Query 98: 0.9460 Precision Averages: Query 99: 0.2404 at 0.00 0.7679 Query 100: 0.5932 at 0.10 0.7123 Precision Query 101: 0.5059 at 0.20 0.6446 Query 102: 0.6973 at 0.30 0.5939 0.4 Query 103: 0.6880 at 0.40 0.5575 Query 104: 0.0536 at 0.50 0.5318 Query 105: 0.5909 at 0.60 0.4723 Query 106: 0.4142 at 0.70 0.4028 0.2 Query 107: 0.0675 at 0.80 0.3473 Query 108: 0.4114 at 0.90 0.2435 Query 109: 0.6867 at 1.00 0.1826 Query 110: 0.6581 0.0 Query 111: 0.4659 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4366 for all rel. documents Query 113: 0.0020 0.4847 Recall Query 114: 0.5137 Precision: Query 115: 0.8145 At 5 docs: 0.5400 Run UniNEfrBi2 Query 116: 0.6321 At 10 docs: 0.4700 Comparison to median by topic Query 117: 0.3455 At 15 docs: 0.4067 +1.0 Query 118: 0.0736 At 20 docs: 0.3550 Query 119: 0.8522 At 30 docs: 0.2993 Query 120: 0.0198 At 100 docs: 0.1614 Query 121: 1.0000 At 200 docs: 0.0999 Query 122: 0.2983 At 500 docs: 0.0458 Query 123: 0.8333 At 1000 docs: 0.0242 +0.5 Query 124: 0.4745 Query 125: 0.3314 R-Precision (prec. after R Query 126: 0.7895 (=num_rel) docs. retrieved): Query 127: 0.9459 Exact: 0.4741 Query 128: 0.0364 0.0 Query 129: 0.3851 Query 130: 0.0000 Query 131: 0.0000 Query 132: 0.5405 Query 133: 0.2283 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.5054 Query 136: 1.0000 Query 137: 0.3023 Query 138: 0.2783 Query 139: 0.7079 −1.0 Query 140: 0.7905 91 100 110 120 130 140 Topic number Université de Neuchâtel, Bilingual English to French, Auto, TD Run UniNEfrBi3 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0035 Total number of documents Query 92: 0.4150 over all queries Query 93: 1.0000 Retrieved: 50000 0.8 Query 94: 0.5879 Relevant: 1383 Query 95: 0.3643 Rel_ret: 1211 Query 96: 0.6211 Query 97: 0.0791 Interpolated Recall - 0.6 Query 98: 0.9280 Precision Averages: Query 99: 0.2074 at 0.00 0.7422 Query 100: 0.5945 at 0.10 0.7033 Precision Query 101: 0.2982 at 0.20 0.6215 Query 102: 0.6962 at 0.30 0.5793 0.4 Query 103: 0.6712 at 0.40 0.5429 Query 104: 0.0653 at 0.50 0.5050 Query 105: 0.5833 at 0.60 0.4497 Query 106: 0.3804 at 0.70 0.3725 0.2 Query 107: 0.0746 at 0.80 0.3173 Query 108: 0.4147 at 0.90 0.2302 Query 109: 0.6899 at 1.00 0.1691 Query 110: 0.6676 0.0 Query 111: 0.5387 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4617 for all rel. documents Query 113: 0.0013 0.4620 Recall Query 114: 0.5094 Precision: Query 115: 0.8132 At 5 docs: 0.5320 Run UniNEfrBi3 Query 116: 0.6343 At 10 docs: 0.4380 Comparison to median by topic Query 117: 0.3239 At 15 docs: 0.3760 +1.0 Query 118: 0.1106 At 20 docs: 0.3400 Query 119: 0.8631 At 30 docs: 0.2893 Query 120: 0.0230 At 100 docs: 0.1588 Query 121: 1.0000 At 200 docs: 0.0983 Query 122: 0.2825 At 500 docs: 0.0450 Query 123: 0.8333 At 1000 docs: 0.0242 +0.5 Query 124: 0.3846 Query 125: 0.3170 R-Precision (prec. after R Query 126: 0.7844 (=num_rel) docs. retrieved): Query 127: 0.5441 Exact: 0.4547 Query 128: 0.0459 0.0 Query 129: 0.3731 Query 130: 0.0000 Query 131: 0.0000 Query 132: 0.5516 Query 133: 0.2493 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.4576 Query 136: 1.0000 Query 137: 0.2178 Query 138: 0.2122 Query 139: 0.4446 −1.0 Query 140: 0.7784 91 100 110 120 130 140 Topic number JHU/APL, Bilingual English to Italian, Auto, TD Run aplbienit Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.0078 Total number of documents Query 92: 0.3701 over all queries Query 93: 0.0027 Retrieved: 49000 0.8 Query 94: 0.6026 Relevant: 1072 Query 95: 0.0793 Rel_ret: 934 Query 96: 0.0971 Query 97: 0.8088 Interpolated Recall - Query 98: 0.1248 Precision Averages: 0.6 Query 99: 0.2582 at 0.00 0.6107 Query 100: 0.2559 at 0.10 0.5298 Query 101: 0.3742 at 0.20 0.4383 Precision Query 102: 0.2533 at 0.30 0.3720 0.4 Query 103: 0.0400 at 0.40 0.3039 Query 104: 0.0672 at 0.50 0.2568 Query 105: 0.5481 at 0.60 0.2329 Query 106: 0.1667 at 0.70 0.2051 0.2 Query 107: 0.1808 at 0.80 0.1524 Query 108: 0.1018 at 0.90 0.1167 Query 109: 0.2227 at 1.00 0.0966 Query 110: 0.3521 0.0 Query 111: 0.1729 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.0208 for all rel. documents Query 113: 0.1114 0.2794 Recall Query 114: 0.5429 Precision: Query 115: 0.3693 At 5 docs: 0.3347 Run aplbienit Query 116: 0.1594 At 10 docs: 0.3082 Query 117: 0.0847 At 15 docs: 0.2803 Comparison to median by topic Query 118: 0.0252 At 20 docs: 0.2602 +1.0 Query 119: 0.7003 At 30 docs: 0.2218 Query 121: 0.0479 At 100 docs: 0.1192 Query 122: 0.2417 At 200 docs: 0.0746 Query 123: 0.8548 At 500 docs: 0.0344 Query 124: 0.3250 At 1000 docs: 0.0191 +0.5 Query 125: 0.1297 Query 126: 0.4980 R-Precision (prec. after R Query 127: 0.5237 (=num_rel) docs. retrieved): Query 128: 0.0198 Exact: 0.2571 Query 129: 0.2429 0.0 Query 130: 0.4954 Query 131: 0.2579 Query 132: 0.5354 Query 133: 0.0019 Query 134: 0.6445 Difference (average precision) Query 135: 0.5382 −0.5 Query 136: 0.4591 Query 137: 0.0495 Query 138: 0.2442 Query 139: 0.3597 Query 140: 0.1204 −1.0 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 2, Bilingual English to Italian, Auto, TD Run bky2bienit Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.0157 Total number of documents Query 92: 0.2640 over all queries Query 93: 0.9046 Retrieved: 49000 0.8 Query 94: 0.5288 Relevant: 1072 Query 95: 0.0568 Rel_ret: 923 Query 96: 0.3227 Query 97: 1.0000 Interpolated Recall - Query 98: 0.6801 Precision Averages: 0.6 Query 99: 0.6144 at 0.00 0.6602 Query 100: 0.5508 at 0.10 0.5995 Query 101: 0.0187 at 0.20 0.5516 Precision Query 102: 0.3797 at 0.30 0.5177 0.4 Query 103: 0.5133 at 0.40 0.4836 Query 104: 0.0472 at 0.50 0.4507 Query 105: 0.5818 at 0.60 0.3804 Query 106: 0.2068 at 0.70 0.3378 0.2 Query 107: 0.3415 at 0.80 0.2974 Query 108: 0.1661 at 0.90 0.2362 Query 109: 0.0201 at 1.00 0.1774 Query 110: 0.9511 0.0 Query 111: 0.0313 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5993 for all rel. documents Query 113: 0.0006 0.4090 Recall Query 114: 0.4000 Precision: Query 115: 0.6532 At 5 docs: 0.4408 Run bky2bienit Query 116: 0.8342 At 10 docs: 0.4184 Query 117: 0.3511 At 15 docs: 0.3878 Comparison to median by topic Query 118: 0.0549 At 20 docs: 0.3500 +1.0 Query 119: 0.7340 At 30 docs: 0.2946 Query 121: 0.1101 At 100 docs: 0.1398 Query 122: 0.0400 At 200 docs: 0.0809 Query 123: 0.8987 At 500 docs: 0.0356 Query 124: 0.0050 At 1000 docs: 0.0188 +0.5 Query 125: 0.0352 Query 126: 0.6319 R-Precision (prec. after R Query 127: 0.4101 (=num_rel) docs. retrieved): Query 128: 0.0687 Exact: 0.3957 Query 129: 0.4722 0.0 Query 130: 0.4252 Query 131: 0.1362 Query 132: 0.3619 Query 133: 0.5675 Query 134: 0.8583 Difference (average precision) Query 135: 0.6782 −0.5 Query 136: 0.9483 Query 137: 0.5017 Query 138: 0.4298 Query 139: 0.2303 Query 140: 0.4067 −1.0 91 100 110 120 130 140 Topic number Univ. Exeter, Bilingual English to Italian, Auto, TD Run exeitengorgbi Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.2269 Total number of documents Query 92: 0.2935 over all queries Query 93: 0.8645 Retrieved: 49000 0.8 Query 94: 0.5187 Relevant: 1072 Query 95: 0.1118 Rel_ret: 938 Query 96: 0.0103 Query 97: 0.0744 Interpolated Recall - Query 98: 0.6672 Precision Averages: 0.6 Query 99: 0.5847 at 0.00 0.6506 Query 100: 0.3951 at 0.10 0.5813 Query 101: 0.2105 at 0.20 0.5331 Precision Query 102: 0.2546 at 0.30 0.4740 0.4 Query 103: 0.5122 at 0.40 0.4051 Query 104: 0.0572 at 0.50 0.3787 Query 105: 0.6591 at 0.60 0.3414 Query 106: 0.3644 at 0.70 0.3080 0.2 Query 107: 0.3119 at 0.80 0.2576 Query 108: 0.0953 at 0.90 0.2098 Query 109: 0.1362 at 1.00 0.1483 Query 110: 0.9435 0.0 Query 111: 0.2616 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6792 for all rel. documents Query 113: 0.0014 0.3742 Recall Query 114: 0.4094 Precision: Query 115: 0.7953 At 5 docs: 0.4408 Run exeitengorgbi Query 116: 0.7297 At 10 docs: 0.3735 Query 117: 0.3849 At 15 docs: 0.3537 Comparison to median by topic Query 118: 0.0725 At 20 docs: 0.3173 +1.0 Query 119: 0.1433 At 30 docs: 0.2728 Query 121: 0.3954 At 100 docs: 0.1333 Query 122: 0.0444 At 200 docs: 0.0786 Query 123: 0.9565 At 500 docs: 0.0362 Query 124: 0.0003 At 1000 docs: 0.0191 +0.5 Query 125: 0.4634 Query 126: 0.5260 R-Precision (prec. after R Query 127: 0.5638 (=num_rel) docs. retrieved): Query 128: 0.0349 Exact: 0.3578 Query 129: 0.3839 0.0 Query 130: 0.3759 Query 131: 0.1554 Query 132: 0.4755 Query 133: 0.4209 Query 134: 1.0000 Difference (average precision) Query 135: 0.5187 −0.5 Query 136: 1.0000 Query 137: 0.0003 Query 138: 0.0441 Query 139: 0.0216 Query 140: 0.1846 −1.0 91 100 110 120 130 140 Topic number Univ. Exeter, Bilingual English to Italian, Auto, TD Run exeitengpiwgt Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.0898 Total number of documents Query 92: 0.3508 over all queries Query 93: 0.9253 Retrieved: 49000 0.8 Query 94: 0.4767 Relevant: 1072 Query 95: 0.1675 Rel_ret: 968 Query 96: 0.1488 Query 97: 1.0000 Interpolated Recall - Query 98: 0.4775 Precision Averages: 0.6 Query 99: 0.3585 at 0.00 0.7289 Query 100: 0.4737 at 0.10 0.6610 Query 101: 0.3011 at 0.20 0.5817 Precision Query 102: 0.2591 at 0.30 0.4730 0.4 Query 103: 0.1648 at 0.40 0.4231 Query 104: 0.0871 at 0.50 0.4072 Query 105: 0.6369 at 0.60 0.3696 Query 106: 0.4082 at 0.70 0.3157 0.2 Query 107: 0.3504 at 0.80 0.2682 Query 108: 0.0681 at 0.90 0.2260 Query 109: 0.3354 at 1.00 0.1672 Query 110: 0.9453 0.0 Query 111: 0.3198 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5277 for all rel. documents Query 113: 0.0013 0.3994 Recall Query 114: 0.4274 Precision: Query 115: 0.6546 At 5 docs: 0.4694 Run exeitengpiwgt Query 116: 0.7465 At 10 docs: 0.4204 Query 117: 0.1736 At 15 docs: 0.3850 Comparison to median by topic Query 118: 0.1227 At 20 docs: 0.3500 +1.0 Query 119: 0.5511 At 30 docs: 0.2946 Query 121: 0.4244 At 100 docs: 0.1402 Query 122: 0.1008 At 200 docs: 0.0817 Query 123: 0.8345 At 500 docs: 0.0372 Query 124: 0.1305 At 1000 docs: 0.0198 +0.5 Query 125: 0.4161 Query 126: 0.3446 R-Precision (prec. after R Query 127: 0.5577 (=num_rel) docs. retrieved): Query 128: 0.0148 Exact: 0.3772 Query 129: 0.4648 0.0 Query 130: 0.3325 Query 131: 0.1775 Query 132: 0.3111 Query 133: 0.5548 Query 134: 0.7714 Difference (average precision) Query 135: 0.8853 −0.5 Query 136: 0.9833 Query 137: 0.0000 Query 138: 0.0041 Query 139: 0.2321 Query 140: 0.4817 −1.0 91 100 110 120 130 140 Topic number Univ. Exeter, Bilingual French to Italian, Auto, TD Run exeitfrmgbi Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.1943 Total number of documents Query 92: 0.2941 over all queries Query 93: 0.7699 Retrieved: 49000 0.8 Query 94: 0.2457 Relevant: 1072 Query 95: 0.1194 Rel_ret: 937 Query 96: 0.0458 Query 97: 0.7292 Interpolated Recall - Query 98: 0.0709 Precision Averages: 0.6 Query 99: 0.4545 at 0.00 0.6088 Query 100: 0.2681 at 0.10 0.5345 Query 101: 0.1082 at 0.20 0.4780 Precision Query 102: 0.2807 at 0.30 0.4424 0.4 Query 103: 0.5709 at 0.40 0.3777 Query 104: 0.0613 at 0.50 0.3612 Query 105: 0.5975 at 0.60 0.3295 Query 106: 0.3677 at 0.70 0.2832 0.2 Query 107: 0.2996 at 0.80 0.2448 Query 108: 0.0765 at 0.90 0.1956 Query 109: 0.1395 at 1.00 0.1390 Query 110: 0.8749 0.0 Query 111: 0.2266 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6918 for all rel. documents Query 113: 0.0000 0.3480 Recall Query 114: 0.4347 Precision: Query 115: 0.8238 At 5 docs: 0.3918 Run exeitfrmgbi Query 116: 0.6927 At 10 docs: 0.3633 Query 117: 0.4659 At 15 docs: 0.3293 Comparison to median by topic Query 118: 0.0591 At 20 docs: 0.3112 +1.0 Query 119: 0.0358 At 30 docs: 0.2619 Query 121: 0.0800 At 100 docs: 0.1271 Query 122: 0.0565 At 200 docs: 0.0762 Query 123: 0.9519 At 500 docs: 0.0354 Query 124: 0.0031 At 1000 docs: 0.0191 +0.5 Query 125: 0.1271 Query 126: 0.3072 R-Precision (prec. after R Query 127: 0.5321 (=num_rel) docs. retrieved): Query 128: 0.0162 Exact: 0.3356 Query 129: 0.2660 0.0 Query 130: 0.3830 Query 131: 0.1541 Query 132: 0.5556 Query 133: 0.1940 Query 134: 0.9429 Difference (average precision) Query 135: 0.7050 −0.5 Query 136: 0.9936 Query 137: 0.0000 Query 138: 0.4853 Query 139: 0.1037 Query 140: 0.1945 −1.0 91 100 110 120 130 140 Topic number Univ. Exeter, Bilingual German to Italian, Auto, TD Run exeitgemgcntbi Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.1576 Total number of documents Query 92: 0.3077 over all queries Query 93: 0.6516 Retrieved: 49000 0.8 Query 94: 0.4550 Relevant: 1072 Query 95: 0.0736 Rel_ret: 891 Query 96: 0.1433 Query 97: 0.7333 Interpolated Recall - Query 98: 0.2676 Precision Averages: 0.6 Query 99: 0.1347 at 0.00 0.6642 Query 100: 0.4263 at 0.10 0.6186 Query 101: 0.3485 at 0.20 0.5236 Precision Query 102: 0.2915 at 0.30 0.4657 0.4 Query 103: 0.5131 at 0.40 0.4162 Query 104: 0.0643 at 0.50 0.3844 Query 105: 0.5793 at 0.60 0.3335 Query 106: 0.4123 at 0.70 0.2727 0.2 Query 107: 0.2261 at 0.80 0.2231 Query 108: 0.0876 at 0.90 0.1850 Query 109: 0.1024 at 1.00 0.1387 Query 110: 0.9466 0.0 Query 111: 0.2879 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6085 for all rel. documents Query 113: 0.0040 0.3688 Recall Query 114: 0.4515 Precision: Query 115: 0.7660 At 5 docs: 0.4327 Run exeitgemgcntbi Query 116: 0.2134 At 10 docs: 0.3633 Query 117: 0.4695 At 15 docs: 0.3320 Comparison to median by topic Query 118: 0.0902 At 20 docs: 0.3031 +1.0 Query 119: 0.0627 At 30 docs: 0.2558 Query 121: 0.3826 At 100 docs: 0.1249 Query 122: 0.0350 At 200 docs: 0.0719 Query 123: 0.9379 At 500 docs: 0.0328 Query 124: 0.0016 At 1000 docs: 0.0182 +0.5 Query 125: 0.5768 Query 126: 0.4475 R-Precision (prec. after R Query 127: 0.5795 (=num_rel) docs. retrieved): Query 128: 0.0088 Exact: 0.3579 Query 129: 0.2590 0.0 Query 130: 0.3543 Query 131: 0.0931 Query 132: 0.5625 Query 133: 0.5072 Query 134: 1.0000 Difference (average precision) Query 135: 0.7314 −0.5 Query 136: 0.9076 Query 137: 0.0758 Query 138: 0.7331 Query 139: 0.0019 Query 140: 0.0020 −1.0 91 100 110 120 130 140 Topic number Univ. Exeter, Bilingual Spanish to Italian, Auto, TD Run exeitspmgbi Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.0990 Total number of documents Query 92: 0.2895 over all queries Query 93: 0.8573 Retrieved: 49000 0.8 Query 94: 0.4431 Relevant: 1072 Query 95: 0.1077 Rel_ret: 914 Query 96: 0.2389 Query 97: 1.0000 Interpolated Recall - Query 98: 0.6923 Precision Averages: 0.6 Query 99: 0.4699 at 0.00 0.6471 Query 100: 0.3239 at 0.10 0.5734 Query 101: 0.1993 at 0.20 0.5025 Precision Query 102: 0.3124 at 0.30 0.4446 0.4 Query 103: 0.5091 at 0.40 0.4235 Query 104: 0.0859 at 0.50 0.3936 Query 105: 0.6506 at 0.60 0.3465 Query 106: 0.3867 at 0.70 0.3024 0.2 Query 107: 0.3004 at 0.80 0.2592 Query 108: 0.0769 at 0.90 0.2104 Query 109: 0.1874 at 1.00 0.1607 Query 110: 0.9310 0.0 Query 111: 0.3262 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6986 for all rel. documents Query 113: 0.0036 0.3737 Recall Query 114: 0.4581 Precision: Query 115: 0.8031 At 5 docs: 0.4490 Run exeitspmgbi Query 116: 0.7083 At 10 docs: 0.3939 Query 117: 0.3689 At 15 docs: 0.3565 Comparison to median by topic Query 118: 0.0601 At 20 docs: 0.3276 +1.0 Query 119: 0.1434 At 30 docs: 0.2782 Query 121: 0.1195 At 100 docs: 0.1290 Query 122: 0.0265 At 200 docs: 0.0753 Query 123: 0.9769 At 500 docs: 0.0347 Query 124: 0.0034 At 1000 docs: 0.0187 +0.5 Query 125: 0.4448 Query 126: 0.2062 R-Precision (prec. after R Query 127: 0.4438 (=num_rel) docs. retrieved): Query 128: 0.0013 Exact: 0.3656 Query 129: 0.3322 0.0 Query 130: 0.3703 Query 131: 0.1654 Query 132: 0.0000 Query 133: 0.5079 Query 134: 1.0000 Difference (average precision) Query 135: 0.6551 −0.5 Query 136: 0.8686 Query 137: 0.0364 Query 138: 0.3205 Query 139: 0.0705 Query 140: 0.0279 −1.0 91 100 110 120 130 140 Topic number ITC-IRST, Bilingual English to Italian, Auto, TD Run IRSTen2it1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.0071 Total number of documents Query 92: 0.3345 over all queries Query 93: 0.9176 Retrieved: 47648 0.8 Query 94: 0.5176 Relevant: 1072 Query 95: 0.0720 Rel_ret: 887 Query 96: 0.1749 Query 97: 1.0000 Interpolated Recall - 0.6 Query 98: 0.3036 Precision Averages: Query 99: 0.1263 at 0.00 0.6211 Query 100: 0.5990 at 0.10 0.5190 Precision Query 101: 0.4554 at 0.20 0.4882 Query 102: 0.3126 at 0.30 0.4230 0.4 Query 103: 0.0551 at 0.40 0.3794 Query 104: 0.0437 at 0.50 0.3539 Query 105: 0.6025 at 0.60 0.3283 Query 106: 0.4608 at 0.70 0.2916 0.2 Query 107: 0.4086 at 0.80 0.2506 Query 108: 0.1574 at 0.90 0.2070 Query 109: 0.2613 at 1.00 0.1672 Query 110: 0.9670 0.0 Query 111: 0.2696 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.7546 for all rel. documents Query 113: 0.0158 0.3444 Recall Query 114: 0.4607 Precision: Query 115: 0.5252 At 5 docs: 0.3837 Run IRSTen2it1 Query 116: 0.7606 At 10 docs: 0.3327 Comparison to median by topic Query 117: 0.3464 At 15 docs: 0.2939 +1.0 Query 118: 0.2567 At 20 docs: 0.2714 Query 119: 0.5218 At 30 docs: 0.2388 Query 121: 0.0923 At 100 docs: 0.1202 Query 122: 0.2672 At 200 docs: 0.0730 Query 123: 0.9397 At 500 docs: 0.0331 Query 124: 0.0170 At 1000 docs: 0.0181 +0.5 Query 125: 0.0711 Query 126: 0.0000 R-Precision (prec. after R Query 127: 0.5436 (=num_rel) docs. retrieved): Query 128: 0.0209 Exact: 0.3275 Query 129: 0.2541 0.0 Query 130: 0.3877 Query 131: 0.1059 Query 132: 0.3778 Query 133: 0.0000 Query 134: 0.6588 Difference (average precision) Query 135: 0.1491 −0.5 Query 136: 0.9812 Query 137: 0.0005 Query 138: 0.0096 Query 139: 0.0863 Query 140: 0.2231 −1.0 91 100 110 120 130 140 Topic number ITC-IRST, Bilingual English to Italian, Auto, TD Run IRSTen2it2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.0068 Total number of documents Query 92: 0.3327 over all queries Query 93: 0.9079 Retrieved: 45487 0.8 Query 94: 0.5750 Relevant: 1072 Query 95: 0.0640 Rel_ret: 903 Query 96: 0.1786 Query 97: 1.0000 Interpolated Recall - 0.6 Query 98: 0.3944 Precision Averages: Query 99: 0.1293 at 0.00 0.6078 Query 100: 0.5990 at 0.10 0.5400 Precision Query 101: 0.4594 at 0.20 0.4853 Query 102: 0.3861 at 0.30 0.4268 0.4 Query 103: 0.0419 at 0.40 0.3922 Query 104: 0.0417 at 0.50 0.3662 Query 105: 0.6048 at 0.60 0.3324 Query 106: 0.4026 at 0.70 0.3020 0.2 Query 107: 0.4686 at 0.80 0.2543 Query 108: 0.1564 at 0.90 0.2118 Query 109: 0.4321 at 1.00 0.1718 Query 110: 0.9608 0.0 Query 111: 0.1001 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.7509 for all rel. documents Query 113: 0.0160 0.3531 Recall Query 114: 0.4352 Precision: Query 115: 0.5603 At 5 docs: 0.3796 Run IRSTen2it2 Query 116: 0.7688 At 10 docs: 0.3429 Comparison to median by topic Query 117: 0.3979 At 15 docs: 0.3211 +1.0 Query 118: 0.2565 At 20 docs: 0.2918 Query 119: 0.5147 At 30 docs: 0.2524 Query 121: 0.0631 At 100 docs: 0.1243 Query 122: 0.0954 At 200 docs: 0.0741 Query 123: 0.9527 At 500 docs: 0.0340 Query 124: 0.0265 At 1000 docs: 0.0184 +0.5 Query 125: 0.0675 Query 126: 0.0000 R-Precision (prec. after R Query 127: 0.5860 (=num_rel) docs. retrieved): Query 128: 0.0304 Exact: 0.3383 Query 129: 0.2482 0.0 Query 130: 0.3868 Query 131: 0.1235 Query 132: 0.3889 Query 133: 0.0127 Query 134: 0.6588 Difference (average precision) Query 135: 0.2733 −0.5 Query 136: 0.9812 Query 137: 0.0000 Query 138: 0.0661 Query 139: 0.1555 Query 140: 0.2426 −1.0 91 100 110 120 130 140 Topic number ITC-IRST, Bilingual English to Italian, Auto, TD Run IRSTen2it3 Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.0064 Total number of documents Query 92: 0.3300 over all queries Query 93: 0.9079 Retrieved: 47846 0.8 Query 94: 0.5385 Relevant: 1072 Query 95: 0.0796 Rel_ret: 893 Query 96: 0.1720 Query 97: 1.0000 Interpolated Recall - 0.6 Query 98: 0.4968 Precision Averages: Query 99: 0.1279 at 0.00 0.6371 Query 100: 0.6047 at 0.10 0.5337 Precision Query 101: 0.4683 at 0.20 0.4894 Query 102: 0.3852 at 0.30 0.4307 0.4 Query 103: 0.0287 at 0.40 0.3809 Query 104: 0.0413 at 0.50 0.3612 Query 105: 0.5970 at 0.60 0.3368 Query 106: 0.4053 at 0.70 0.3036 0.2 Query 107: 0.4782 at 0.80 0.2604 Query 108: 0.1419 at 0.90 0.2197 Query 109: 0.4361 at 1.00 0.1744 Query 110: 0.9619 0.0 Query 111: 0.1000 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.7509 for all rel. documents Query 113: 0.0185 0.3552 Recall Query 114: 0.5416 Precision: Query 115: 0.4250 At 5 docs: 0.3959 Run IRSTen2it3 Query 116: 0.7688 At 10 docs: 0.3347 Comparison to median by topic Query 117: 0.4039 At 15 docs: 0.3061 +1.0 Query 118: 0.2564 At 20 docs: 0.2867 Query 119: 0.5132 At 30 docs: 0.2429 Query 121: 0.1054 At 100 docs: 0.1204 Query 122: 0.0954 At 200 docs: 0.0736 Query 123: 0.9527 At 500 docs: 0.0335 Query 124: 0.0422 At 1000 docs: 0.0182 +0.5 Query 125: 0.0175 Query 126: 0.0000 R-Precision (prec. after R Query 127: 0.6418 (=num_rel) docs. retrieved): Query 128: 0.0206 Exact: 0.3313 Query 129: 0.2600 0.0 Query 130: 0.3843 Query 131: 0.1319 Query 132: 0.3889 Query 133: 0.0073 Query 134: 0.6622 Difference (average precision) Query 135: 0.1923 −0.5 Query 136: 0.9936 Query 137: 0.0004 Query 138: 0.1309 Query 139: 0.1259 Query 140: 0.2634 −1.0 91 100 110 120 130 140 Topic number Univ. Jaen/SINAI, Bilingual English to Italian, Auto, TD Run UJABIIT Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.0016 Total number of documents Query 92: 0.1852 over all queries Query 93: 0.7812 Retrieved: 49000 0.8 Query 94: 0.4311 Relevant: 1072 Query 95: 0.0701 Rel_ret: 759 Query 96: 0.0000 Query 97: 0.6679 Interpolated Recall - 0.6 Query 98: 0.0212 Precision Averages: Query 99: 0.0290 at 0.00 0.5235 Query 100: 0.3969 at 0.10 0.4137 Precision Query 101: 0.2126 at 0.20 0.3728 Query 102: 0.3005 at 0.30 0.3168 0.4 Query 103: 0.3112 at 0.40 0.2510 Query 104: 0.0368 at 0.50 0.2375 Query 105: 0.3222 at 0.60 0.2165 Query 106: 0.0768 at 0.70 0.1943 0.2 Query 107: 0.0443 at 0.80 0.1407 Query 108: 0.0869 at 0.90 0.0982 Query 109: 0.0004 at 1.00 0.0709 Query 110: 0.7533 0.0 Query 111: 0.0001 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.3392 for all rel. documents Query 113: 0.0071 0.2438 Recall Query 114: 0.2194 Precision: Query 115: 0.0060 At 5 docs: 0.3265 Run UJABIIT Query 116: 0.6520 At 10 docs: 0.2776 Comparison to median by topic Query 117: 0.1634 At 15 docs: 0.2408 +1.0 Query 118: 0.0749 At 20 docs: 0.2143 Query 119: 0.7205 At 30 docs: 0.1837 Query 121: 0.3742 At 100 docs: 0.0882 Query 122: 0.0498 At 200 docs: 0.0561 Query 123: 0.9820 At 500 docs: 0.0281 Query 124: 0.0265 At 1000 docs: 0.0155 +0.5 Query 125: 0.0045 Query 126: 0.3846 R-Precision (prec. after R Query 127: 0.1956 (=num_rel) docs. retrieved): Query 128: 0.0015 Exact: 0.2620 Query 129: 0.0000 0.0 Query 130: 0.2303 Query 131: 0.0564 Query 132: 0.2292 Query 133: 0.0585 Query 134: 0.8588 Difference (average precision) Query 135: 0.2796 −0.5 Query 136: 0.9722 Query 137: 0.0534 Query 138: 0.1780 Query 139: 0.0003 Query 140: 0.0987 −1.0 91 100 110 120 130 140 Topic number Université de Neuchâtel, Bilingual English to Italian, Auto, TD Run UniNEitBi Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.0243 Total number of documents Query 92: 0.3238 over all queries Query 93: 0.6712 Retrieved: 49000 0.8 Query 94: 0.4753 Relevant: 1072 Query 95: 0.0927 Rel_ret: 941 Query 96: 0.4103 Query 97: 0.2764 Interpolated Recall - 0.6 Query 98: 0.4161 Precision Averages: Query 99: 0.4003 at 0.00 0.6867 Query 100: 0.4885 at 0.10 0.6123 Precision Query 101: 0.3510 at 0.20 0.5469 Query 102: 0.2945 at 0.30 0.4774 0.4 Query 103: 0.6055 at 0.40 0.4208 Query 104: 0.0524 at 0.50 0.3927 Query 105: 0.5858 at 0.60 0.3445 Query 106: 0.2962 at 0.70 0.2696 0.2 Query 107: 0.1472 at 0.80 0.2292 Query 108: 0.1377 at 0.90 0.1904 Query 109: 0.0447 at 1.00 0.1356 Query 110: 0.9733 0.0 Query 111: 0.0405 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6608 for all rel. documents Query 113: 0.0081 0.3736 Recall Query 114: 0.3569 Precision: Query 115: 0.6379 At 5 docs: 0.4408 Run UniNEitBi Query 116: 0.7468 At 10 docs: 0.3939 Comparison to median by topic Query 117: 0.4261 At 15 docs: 0.3551 +1.0 Query 118: 0.0964 At 20 docs: 0.3296 Query 119: 0.7634 At 30 docs: 0.2769 Query 121: 0.3895 At 100 docs: 0.1320 Query 122: 0.1152 At 200 docs: 0.0788 Query 123: 0.9507 At 500 docs: 0.0367 Query 124: 0.0752 At 1000 docs: 0.0192 +0.5 Query 125: 0.1221 Query 126: 0.4269 R-Precision (prec. after R Query 127: 0.5029 (=num_rel) docs. retrieved): Query 128: 0.0517 Exact: 0.3524 Query 129: 0.3881 0.0 Query 130: 0.0000 Query 131: 0.0000 Query 132: 0.3125 Query 133: 0.3054 Query 134: 0.9029 Difference (average precision) Query 135: 0.7541 −0.5 Query 136: 0.9736 Query 137: 0.3243 Query 138: 0.1619 Query 139: 0.2405 Query 140: 0.5025 −1.0 91 100 110 120 130 140 Topic number Université de Neuchâtel, Bilingual English to Italian, Auto, TD Run UniNEitBi2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.0270 Total number of documents Query 92: 0.3095 over all queries Query 93: 0.6710 Retrieved: 49000 0.8 Query 94: 0.4711 Relevant: 1072 Query 95: 0.0819 Rel_ret: 940 Query 96: 0.4080 Query 97: 0.2874 Interpolated Recall - 0.6 Query 98: 0.4897 Precision Averages: Query 99: 0.4987 at 0.00 0.6775 Query 100: 0.4586 at 0.10 0.6136 Precision Query 101: 0.0852 at 0.20 0.5621 Query 102: 0.3157 at 0.30 0.4959 0.4 Query 103: 0.6017 at 0.40 0.4186 Query 104: 0.0494 at 0.50 0.3842 Query 105: 0.5765 at 0.60 0.3431 Query 106: 0.3026 at 0.70 0.2675 0.2 Query 107: 0.0944 at 0.80 0.2303 Query 108: 0.1365 at 0.90 0.1873 Query 109: 0.1447 at 1.00 0.1362 Query 110: 0.9753 0.0 Query 111: 0.1399 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4801 for all rel. documents Query 113: 0.0069 0.3756 Recall Query 114: 0.4960 Precision: Query 115: 0.6766 At 5 docs: 0.4531 Run UniNEitBi2 Query 116: 0.7661 At 10 docs: 0.4020 Comparison to median by topic Query 117: 0.4010 At 15 docs: 0.3714 +1.0 Query 118: 0.0797 At 20 docs: 0.3459 Query 119: 0.7608 At 30 docs: 0.2850 Query 121: 0.3950 At 100 docs: 0.1351 Query 122: 0.1132 At 200 docs: 0.0808 Query 123: 0.9483 At 500 docs: 0.0368 Query 124: 0.0601 At 1000 docs: 0.0192 +0.5 Query 125: 0.2710 Query 126: 0.4711 R-Precision (prec. after R Query 127: 0.5165 (=num_rel) docs. retrieved): Query 128: 0.0725 Exact: 0.3608 Query 129: 0.3842 0.0 Query 130: 0.0000 Query 131: 0.0000 Query 132: 0.2847 Query 133: 0.3162 Query 134: 0.9029 Difference (average precision) Query 135: 0.6965 −0.5 Query 136: 0.9812 Query 137: 0.2868 Query 138: 0.2359 Query 139: 0.2331 Query 140: 0.4429 −1.0 91 100 110 120 130 140 Topic number CWI/CNLP, Bilingual English to Dutch, Auto, T Run AAbiENNLt Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0000 Total number of documents Query 92: 0.0696 over all queries Query 93: 0.7315 Retrieved: 46858 0.8 Query 94: 0.0024 Relevant: 1862 Query 95: 0.0004 Rel_ret: 699 Query 96: 0.0000 Query 97: 0.0943 Interpolated Recall - 0.6 Query 98: 0.0014 Precision Averages: Query 99: 0.1349 at 0.00 0.3216 Query 100: 0.1153 at 0.10 0.2444 Precision Query 101: 0.6506 at 0.20 0.2021 Query 102: 0.4871 at 0.30 0.1668 0.4 Query 103: 0.0000 at 0.40 0.1424 Query 104: 0.0214 at 0.50 0.1292 Query 105: 0.3925 at 0.60 0.1177 Query 106: 0.0081 at 0.70 0.1092 0.2 Query 107: 0.0067 at 0.80 0.0807 Query 108: 0.3629 at 0.90 0.0473 Query 109: 0.0567 at 1.00 0.0294 Query 110: 0.0066 0.0 Query 111: 0.1721 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.1443 for all rel. documents Query 113: 0.0001 0.1326 Recall Query 114: 0.0252 Precision: Query 115: 0.2481 At 5 docs: 0.1800 Run AAbiENNLt Query 116: 0.5492 At 10 docs: 0.1760 Comparison to median by topic Query 117: 0.0009 At 15 docs: 0.1587 +1.0 Query 118: 0.0446 At 20 docs: 0.1540 Query 119: 0.0036 At 30 docs: 0.1287 Query 120: 0.0370 At 100 docs: 0.0720 Query 121: 0.4300 At 200 docs: 0.0452 Query 122: 0.0088 At 500 docs: 0.0244 Query 123: 0.0186 At 1000 docs: 0.0140 +0.5 Query 124: 0.0192 Query 125: 0.0005 R-Precision (prec. after R Query 126: 0.0000 (=num_rel) docs. retrieved): Query 127: 0.0064 Exact: 0.1429 Query 128: 0.0008 0.0 Query 129: 0.0018 Query 130: 0.0670 Query 131: 0.1270 Query 132: 0.4329 Query 133: 0.0094 Difference (average precision) Query 134: 0.7500 −0.5 Query 135: 0.0060 Query 136: 0.2182 Query 137: 0.0000 Query 138: 0.0000 Query 139: 0.0171 −1.0 Query 140: 0.1476 91 100 110 120 130 140 Topic number CWI/CNLP, Bilingual English to Dutch, Auto, TD Run AAbiENNLtd Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0001 Total number of documents Query 92: 0.1092 over all queries Query 93: 0.6391 Retrieved: 50000 0.8 Query 94: 0.0027 Relevant: 1862 Query 95: 0.0001 Rel_ret: 816 Query 96: 0.0000 Query 97: 0.1317 Interpolated Recall - 0.6 Query 98: 0.0044 Precision Averages: Query 99: 0.0006 at 0.00 0.3352 Query 100: 0.0492 at 0.10 0.2987 Precision Query 101: 0.5951 at 0.20 0.2274 Query 102: 0.4226 at 0.30 0.2130 0.4 Query 103: 0.0335 at 0.40 0.1787 Query 104: 0.0141 at 0.50 0.1678 Query 105: 0.3326 at 0.60 0.1471 Query 106: 0.0036 at 0.70 0.1272 0.2 Query 107: 0.0059 at 0.80 0.1079 Query 108: 0.3210 at 0.90 0.0811 Query 109: 0.2202 at 1.00 0.0672 Query 110: 0.3289 0.0 Query 111: 0.0145 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4493 for all rel. documents Query 113: 0.0011 0.1622 Recall Query 114: 0.1896 Precision: Query 115: 0.1413 At 5 docs: 0.1960 Run AAbiENNLtd Query 116: 0.5025 At 10 docs: 0.1800 Comparison to median by topic Query 117: 0.0002 At 15 docs: 0.1653 +1.0 Query 118: 0.0183 At 20 docs: 0.1480 Query 119: 0.0031 At 30 docs: 0.1353 Query 120: 0.0490 At 100 docs: 0.0772 Query 121: 0.2947 At 200 docs: 0.0519 Query 122: 0.0017 At 500 docs: 0.0278 Query 123: 0.9160 At 1000 docs: 0.0163 +0.5 Query 124: 0.0147 Query 125: 0.0015 R-Precision (prec. after R Query 126: 0.0000 (=num_rel) docs. retrieved): Query 127: 0.0476 Exact: 0.1745 Query 128: 0.0007 0.0 Query 129: 0.0004 Query 130: 0.1141 Query 131: 0.0130 Query 132: 0.3406 Query 133: 0.0251 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.1059 Query 136: 0.4867 Query 137: 0.0000 Query 138: 0.0000 Query 139: 0.0153 −1.0 Query 140: 0.1476 91 100 110 120 130 140 Topic number JHU/APL, Bilingual English to Dutch, Auto, TD Run aplbiennl Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0028 Total number of documents Query 92: 0.4238 over all queries Query 93: 0.0012 Retrieved: 50000 0.8 Query 94: 0.7476 Relevant: 1862 Query 95: 0.6341 Rel_ret: 1625 Query 96: 0.1376 Query 97: 0.1541 Interpolated Recall - Query 98: 0.5190 Precision Averages: 0.6 Query 99: 0.0535 at 0.00 0.6456 Query 100: 0.6386 at 0.10 0.5694 Query 101: 0.5855 at 0.20 0.5039 Precision Query 102: 0.7029 at 0.30 0.4423 0.4 Query 103: 0.2829 at 0.40 0.3980 Query 104: 0.5656 at 0.50 0.3734 Query 105: 0.4078 at 0.60 0.3229 Query 106: 0.0514 at 0.70 0.2692 0.2 Query 107: 0.1980 at 0.80 0.2429 Query 108: 0.3539 at 0.90 0.1818 Query 109: 0.2453 at 1.00 0.1111 Query 110: 0.0769 0.0 Query 111: 0.1324 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.0475 for all rel. documents Query 113: 0.0038 0.3516 Recall Query 114: 0.8430 Precision: Query 115: 0.3079 At 5 docs: 0.4440 Run aplbiennl Query 116: 0.0875 At 10 docs: 0.4140 Query 117: 0.1236 At 15 docs: 0.3813 Comparison to median by topic Query 118: 0.0235 At 20 docs: 0.3560 +1.0 Query 119: 0.8516 At 30 docs: 0.3147 Query 120: 0.0764 At 100 docs: 0.1772 Query 121: 0.2977 At 200 docs: 0.1136 Query 122: 0.2189 At 500 docs: 0.0576 Query 123: 0.8568 At 1000 docs: 0.0325 +0.5 Query 124: 0.2197 Query 125: 0.0590 R-Precision (prec. after R Query 126: 0.6795 (=num_rel) docs. retrieved): Query 127: 0.0319 Exact: 0.3559 Query 128: 0.0122 0.0 Query 129: 0.5334 Query 130: 0.6353 Query 131: 0.6180 Query 132: 0.9040 Query 133: 0.0105 Difference (average precision) Query 134: 0.2369 −0.5 Query 135: 0.8104 Query 136: 0.6862 Query 137: 0.0000 Query 138: 0.6025 Query 139: 0.1809 −1.0 Query 140: 0.7078 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 2, Bilingual English to Dutch, Auto, TD Run bky2biennl Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0011 Total number of documents Query 92: 0.1586 over all queries Query 93: 0.6790 Retrieved: 50000 0.8 Query 94: 0.0405 Relevant: 1862 Query 95: 0.6454 Rel_ret: 1524 Query 96: 0.1276 Query 97: 0.1024 Interpolated Recall - Query 98: 0.9167 Precision Averages: 0.6 Query 99: 0.0146 at 0.00 0.5415 Query 100: 0.7628 at 0.10 0.4731 Query 101: 0.7222 at 0.20 0.4326 Precision Query 102: 0.5815 at 0.30 0.4062 0.4 Query 103: 0.2651 at 0.40 0.3646 Query 104: 0.0987 at 0.50 0.3386 Query 105: 0.3053 at 0.60 0.3069 Query 106: 0.0716 at 0.70 0.2632 0.2 Query 107: 0.0169 at 0.80 0.2220 Query 108: 0.4186 at 0.90 0.1778 Query 109: 0.0559 at 1.00 0.1304 Query 110: 0.8750 0.0 Query 111: 0.0011 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.2256 for all rel. documents Query 113: 0.0316 0.3199 Recall Query 114: 0.7395 Precision: Query 115: 0.2806 At 5 docs: 0.3680 Run bky2biennl Query 116: 0.7758 At 10 docs: 0.3580 Query 117: 0.0322 At 15 docs: 0.3333 Comparison to median by topic Query 118: 0.0186 At 20 docs: 0.3090 +1.0 Query 119: 0.5778 At 30 docs: 0.2747 Query 120: 0.0260 At 100 docs: 0.1530 Query 121: 0.3415 At 200 docs: 0.0984 Query 122: 0.1420 At 500 docs: 0.0534 Query 123: 0.9273 At 1000 docs: 0.0305 +0.5 Query 124: 0.1983 Query 125: 0.2256 R-Precision (prec. after R Query 126: 0.3226 (=num_rel) docs. retrieved): Query 127: 0.8720 Exact: 0.3238 Query 128: 0.0886 0.0 Query 129: 0.0068 Query 130: 0.3347 Query 131: 0.0191 Query 132: 0.8625 Query 133: 0.2599 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.0199 Query 136: 0.0053 Query 137: 0.0000 Query 138: 0.0078 Query 139: 0.1918 −1.0 Query 140: 0.6026 91 100 110 120 130 140 Topic number Univ. Tampere, Dept. IS, Bilingual English to Dutch, Auto, TD Run dualge Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0000 Total number of documents Query 92: 0.0002 over all queries Query 93: 0.0000 Retrieved: 41009 0.8 Query 94: 0.0000 Relevant: 1862 Query 95: 0.0000 Rel_ret: 25 Query 96: 0.0000 Query 97: 0.0000 Interpolated Recall - Query 98: 0.0000 Precision Averages: 0.6 Query 99: 0.0000 at 0.00 0.0457 Query 100: 0.0000 at 0.10 0.0021 Query 101: 0.0000 at 0.20 0.0000 Precision Query 102: 0.0000 at 0.30 0.0000 0.4 Query 103: 0.0000 at 0.40 0.0000 Query 104: 0.0104 at 0.50 0.0000 Query 105: 0.0000 at 0.60 0.0000 Query 106: 0.0000 at 0.70 0.0000 0.2 Query 107: 0.0000 at 0.80 0.0000 Query 108: 0.0000 at 0.90 0.0000 Query 109: 0.0000 at 1.00 0.0000 Query 110: 0.0000 0.0 Query 111: 0.0101 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.0000 for all rel. documents Query 113: 0.0417 0.0023 Recall Query 114: 0.0000 Precision: Query 115: 0.0000 At 5 docs: 0.0280 Run dualge Query 116: 0.0002 At 10 docs: 0.0240 Query 117: 0.0037 At 15 docs: 0.0160 Comparison to median by topic Query 118: 0.0000 At 20 docs: 0.0120 +1.0 Query 119: 0.0000 At 30 docs: 0.0093 Query 120: 0.0000 At 100 docs: 0.0036 Query 121: 0.0001 At 200 docs: 0.0018 Query 122: 0.0000 At 500 docs: 0.0008 Query 123: 0.0000 At 1000 docs: 0.0005 +0.5 Query 124: 0.0000 Query 125: 0.0000 R-Precision (prec. after R Query 126: 0.0000 (=num_rel) docs. retrieved): Query 127: 0.0000 Exact: 0.0040 Query 128: 0.0000 0.0 Query 129: 0.0000 Query 130: 0.0000 Query 131: 0.0000 Query 132: 0.0000 Query 133: 0.0000 Difference (average precision) Query 134: 0.0000 −0.5 Query 135: 0.0000 Query 136: 0.0000 Query 137: 0.0000 Query 138: 0.0000 Query 139: 0.0000 −1.0 Query 140: 0.0463 91 100 110 120 130 140 Topic number Océ, Bilingual English to Dutch, Auto, TD Run oce02en2nlER Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0002 Total number of documents Query 92: 0.3796 over all queries Query 93: 0.0018 Retrieved: 50000 0.8 Query 94: 0.0215 Relevant: 1862 Query 95: 0.0153 Rel_ret: 902 Query 96: 0.0822 Query 97: 0.0011 Interpolated Recall - 0.6 Query 98: 0.8367 Precision Averages: Query 99: 0.0077 at 0.00 0.4767 Query 100: 0.3673 at 0.10 0.3133 Precision Query 101: 0.1201 at 0.20 0.2614 Query 102: 0.2330 at 0.30 0.2433 0.4 Query 103: 0.2336 at 0.40 0.2172 Query 104: 0.0985 at 0.50 0.1861 Query 105: 0.0480 at 0.60 0.1581 Query 106: 0.0021 at 0.70 0.1302 0.2 Query 107: 0.0005 at 0.80 0.1200 Query 108: 0.5068 at 0.90 0.0885 Query 109: 0.0012 at 1.00 0.0750 Query 110: 0.7679 0.0 Query 111: 0.0102 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6542 for all rel. documents Query 113: 0.0013 0.1912 Recall Query 114: 0.0540 Precision: Query 115: 0.0174 At 5 docs: 0.2800 Run oce02en2nlER Query 116: 0.0166 At 10 docs: 0.2160 Comparison to median by topic Query 117: 0.0040 At 15 docs: 0.1880 +1.0 Query 118: 0.0155 At 20 docs: 0.1640 Query 119: 0.4085 At 30 docs: 0.1433 Query 120: 0.0580 At 100 docs: 0.0844 Query 121: 0.1291 At 200 docs: 0.0531 Query 122: 0.0464 At 500 docs: 0.0288 Query 123: 0.8453 At 1000 docs: 0.0180 +0.5 Query 124: 0.2020 Query 125: 0.0547 R-Precision (prec. after R Query 126: 0.0004 (=num_rel) docs. retrieved): Query 127: 0.5813 Exact: 0.2107 Query 128: 0.0009 0.0 Query 129: 0.1012 Query 130: 0.3308 Query 131: 0.0241 Query 132: 0.0392 Query 133: 0.0415 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.1847 Query 136: 0.9267 Query 137: 0.0000 Query 138: 0.0179 Query 139: 0.0557 −1.0 Query 140: 0.0131 91 100 110 120 130 140 Topic number Univ. of Twente, Bilingual English to Dutch, Auto, TD Run tnoen1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0000 Total number of documents Query 92: 0.4278 over all queries Query 93: 0.7596 Retrieved: 49361 0.8 Query 94: 0.2169 Relevant: 1862 Query 95: 0.4290 Rel_ret: 1549 Query 96: 0.1767 Query 97: 0.1354 Interpolated Recall - 0.6 Query 98: 0.8413 Precision Averages: Query 99: 0.1109 at 0.00 0.7382 Query 100: 0.5993 at 0.10 0.5657 Precision Query 101: 0.3213 at 0.20 0.4565 Query 102: 0.4988 at 0.30 0.3958 0.4 Query 103: 0.3318 at 0.40 0.3636 Query 104: 0.2612 at 0.50 0.3285 Query 105: 0.3448 at 0.60 0.2980 Query 106: 0.1020 at 0.70 0.2730 0.2 Query 107: 0.1142 at 0.80 0.2330 Query 108: 0.0835 at 0.90 0.1688 Query 109: 0.0401 at 1.00 0.1371 Query 110: 1.0000 0.0 Query 111: 0.1938 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4775 for all rel. documents Query 113: 0.0015 0.3369 Recall Query 114: 0.3213 Precision: Query 115: 0.1368 At 5 docs: 0.4560 Run tnoen1 Query 116: 0.6608 At 10 docs: 0.3800 Comparison to median by topic Query 117: 0.1357 At 15 docs: 0.3360 +1.0 Query 118: 0.0932 At 20 docs: 0.3060 Query 119: 0.1855 At 30 docs: 0.2607 Query 120: 0.2232 At 100 docs: 0.1516 Query 121: 0.3899 At 200 docs: 0.1016 Query 122: 0.0886 At 500 docs: 0.0553 Query 123: 0.9414 At 1000 docs: 0.0310 +0.5 Query 124: 0.1399 Query 125: 0.2994 R-Precision (prec. after R Query 126: 0.2614 (=num_rel) docs. retrieved): Query 127: 0.8432 Exact: 0.3443 Query 128: 0.0764 0.0 Query 129: 0.3960 Query 130: 0.0662 Query 131: 0.2742 Query 132: 0.6222 Query 133: 0.0496 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.1430 Query 136: 1.0000 Query 137: 0.0000 Query 138: 0.1418 Query 139: 0.3483 −1.0 Query 140: 0.5400 91 100 110 120 130 140 Topic number University of Amsterdam, Bilingual English to Dutch, Auto, TD Run UAmsC02EnDuMorph Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0003 Total number of documents Query 92: 0.4495 over all queries Query 93: 0.2921 Retrieved: 50000 0.8 Query 94: 0.0232 Relevant: 1862 Query 95: 0.4873 Rel_ret: 1171 Query 96: 0.0000 Query 97: 0.0021 Interpolated Recall - Query 98: 0.0300 Precision Averages: 0.6 Query 99: 0.0054 at 0.00 0.4772 Query 100: 0.0018 at 0.10 0.4213 Query 101: 0.4911 at 0.20 0.3700 Precision Query 102: 0.6188 at 0.30 0.3238 0.4 Query 103: 0.4767 at 0.40 0.2933 Query 104: 0.7000 at 0.50 0.2569 Query 105: 0.4679 at 0.60 0.2331 Query 106: 0.0213 at 0.70 0.1946 0.2 Query 107: 0.0003 at 0.80 0.1627 Query 108: 0.3751 at 0.90 0.1204 Query 109: 0.0013 at 1.00 0.0941 Query 110: 0.3826 0.0 Query 111: 0.0314 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4559 for all rel. documents Query 113: 0.0013 0.2576 Recall Query 114: 0.6285 Precision: Query 115: 0.0008 At 5 docs: 0.3400 Run UAmsC02EnDuMorph Query 116: 0.7256 At 10 docs: 0.2940 Query 117: 0.0248 At 15 docs: 0.2720 Comparison to median by topic Query 118: 0.0105 At 20 docs: 0.2480 +1.0 Query 119: 0.1100 At 30 docs: 0.2153 Query 120: 0.0419 At 100 docs: 0.1232 Query 121: 0.3113 At 200 docs: 0.0807 Query 122: 0.0274 At 500 docs: 0.0417 Query 123: 0.8454 At 1000 docs: 0.0234 +0.5 Query 124: 0.0016 Query 125: 0.0014 R-Precision (prec. after R Query 126: 0.4569 (=num_rel) docs. retrieved): Query 127: 0.0094 Exact: 0.2621 Query 128: 0.0187 0.0 Query 129: 0.0169 Query 130: 0.5914 Query 131: 0.0003 Query 132: 0.5458 Query 133: 0.0266 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.0692 Query 136: 0.9267 Query 137: 0.0000 Query 138: 0.4446 Query 139: 0.1542 −1.0 Query 140: 0.5726 91 100 110 120 130 140 Topic number University of Amsterdam, Bilingual English to Dutch, Auto, TD Run UAmsC02EnDuNGiMO Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0080 Total number of documents Query 92: 0.3930 over all queries Query 93: 0.7893 Retrieved: 50000 0.8 Query 94: 0.4939 Relevant: 1862 Query 95: 0.4047 Rel_ret: 1206 Query 96: 0.0015 Query 97: 0.0015 Interpolated Recall - 0.6 Query 98: 0.1729 Precision Averages: Query 99: 0.1489 at 0.00 0.5652 Query 100: 0.6770 at 0.10 0.4744 Precision Query 101: 0.6211 at 0.20 0.4211 Query 102: 0.6760 at 0.30 0.3765 0.4 Query 103: 0.4779 at 0.40 0.3336 Query 104: 0.1777 at 0.50 0.3075 Query 105: 0.1002 at 0.60 0.2654 Query 106: 0.0069 at 0.70 0.2294 0.2 Query 107: 0.0110 at 0.80 0.1804 Query 108: 0.4452 at 0.90 0.1284 Query 109: 0.0225 at 1.00 0.0999 Query 110: 0.8875 0.0 Query 111: 0.1771 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4910 for all rel. documents Query 113: 0.0000 0.2933 Recall Query 114: 0.7581 Precision: Query 115: 0.5210 At 5 docs: 0.4080 Run UAmsC02EnDuNGiMO Query 116: 0.8945 At 10 docs: 0.3760 Comparison to median by topic Query 117: 0.0298 At 15 docs: 0.3413 +1.0 Query 118: 0.0127 At 20 docs: 0.3160 Query 119: 0.4670 At 30 docs: 0.2767 Query 120: 0.0476 At 100 docs: 0.1432 Query 121: 0.3136 At 200 docs: 0.0877 Query 122: 0.0702 At 500 docs: 0.0429 Query 123: 0.8393 At 1000 docs: 0.0241 +0.5 Query 124: 0.3809 Query 125: 0.0019 R-Precision (prec. after R Query 126: 0.0664 (=num_rel) docs. retrieved): Query 127: 0.0016 Exact: 0.3124 Query 128: 0.0002 0.0 Query 129: 0.0270 Query 130: 0.6070 Query 131: 0.0911 Query 132: 0.5841 Query 133: 0.0606 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.0276 Query 136: 0.0525 Query 137: 0.0000 Query 138: 0.1420 Query 139: 0.1619 −1.0 Query 140: 0.3197 91 100 110 120 130 140 Topic number University of Amsterdam, Bilingual English to Dutch, Auto, TD Run UAmsC02EnDuNGram Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0085 Total number of documents Query 92: 0.3430 over all queries Query 93: 0.7884 Retrieved: 50000 0.8 Query 94: 0.6506 Relevant: 1862 Query 95: 0.3564 Rel_ret: 1083 Query 96: 0.0019 Query 97: 0.0013 Interpolated Recall - 0.6 Query 98: 0.1195 Precision Averages: Query 99: 0.2326 at 0.00 0.5146 Query 100: 0.6980 at 0.10 0.4379 Precision Query 101: 0.6158 at 0.20 0.3943 Query 102: 0.6040 at 0.30 0.3533 0.4 Query 103: 0.4149 at 0.40 0.3230 Query 104: 0.0620 at 0.50 0.3035 Query 105: 0.0402 at 0.60 0.2593 Query 106: 0.0012 at 0.70 0.2184 0.2 Query 107: 0.0147 at 0.80 0.1823 Query 108: 0.4512 at 0.90 0.1233 Query 109: 0.0190 at 1.00 0.0952 Query 110: 1.0000 0.0 Query 111: 0.1751 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4821 for all rel. documents Query 113: 0.0000 0.2807 Recall Query 114: 0.7393 Precision: Query 115: 0.5459 At 5 docs: 0.3720 Run UAmsC02EnDuNGram Query 116: 0.9269 At 10 docs: 0.3480 Comparison to median by topic Query 117: 0.0138 At 15 docs: 0.3280 +1.0 Query 118: 0.0093 At 20 docs: 0.2890 Query 119: 0.5264 At 30 docs: 0.2473 Query 120: 0.0501 At 100 docs: 0.1294 Query 121: 0.3092 At 200 docs: 0.0801 Query 122: 0.0348 At 500 docs: 0.0398 Query 123: 0.7899 At 1000 docs: 0.0217 +0.5 Query 124: 0.3854 Query 125: 0.0018 R-Precision (prec. after R Query 126: 0.0012 (=num_rel) docs. retrieved): Query 127: 0.0010 Exact: 0.2972 Query 128: 0.0000 0.0 Query 129: 0.0245 Query 130: 0.5903 Query 131: 0.2250 Query 132: 0.5592 Query 133: 0.0372 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.0116 Query 136: 0.0057 Query 137: 0.0000 Query 138: 0.0003 Query 139: 0.1254 −1.0 Query 140: 0.0386 91 100 110 120 130 140 Topic number JHU/APL, Bilingual English to Swedish, Auto, TD Run aplbiensv Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.3998 Total number of documents Query 92: 0.5318 over all queries Query 93: 0.0014 Retrieved: 49000 0.8 Query 94: 0.2754 Relevant: 1196 Query 95: 0.2372 Rel_ret: 1052 Query 96: 0.0115 Query 97: 0.0277 Interpolated Recall - Query 98: 0.5833 Precision Averages: 0.6 Query 99: 0.2216 at 0.00 0.6244 Query 100: 0.3303 at 0.10 0.5183 Query 101: 0.6016 at 0.20 0.4095 Precision Query 102: 0.5509 at 0.30 0.3770 0.4 Query 103: 0.2045 at 0.40 0.3329 Query 104: 0.6371 at 0.50 0.2993 Query 105: 0.6753 at 0.60 0.2628 Query 106: 0.2456 at 0.70 0.2260 0.2 Query 107: 0.0134 at 0.80 0.1993 Query 108: 0.1463 at 0.90 0.1406 Query 109: 0.3544 at 1.00 0.0858 Query 111: 0.1178 0.0 Query 112: 0.1278 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 113: 0.0043 for all rel. documents Query 114: 0.4760 0.3003 Recall Query 115: 0.2206 Precision: Query 116: 0.0000 At 5 docs: 0.4082 Run aplbiensv Query 117: 0.1785 At 10 docs: 0.3551 Query 118: 0.0018 At 15 docs: 0.3184 Comparison to median by topic Query 119: 0.6268 At 20 docs: 0.2969 +1.0 Query 120: 0.3333 At 30 docs: 0.2578 Query 121: 0.0294 At 100 docs: 0.1490 Query 122: 0.6575 At 200 docs: 0.0881 Query 123: 0.8720 At 500 docs: 0.0390 Query 124: 0.3129 At 1000 docs: 0.0215 +0.5 Query 125: 0.5410 Query 126: 0.0238 R-Precision (prec. after R Query 127: 0.1087 (=num_rel) docs. retrieved): Query 128: 0.1155 Exact: 0.2832 Query 129: 0.0652 0.0 Query 130: 0.4910 Query 131: 0.5386 Query 132: 0.8252 Query 133: 0.0232 Query 134: 0.0061 Difference (average precision) Query 135: 0.2979 −0.5 Query 136: 0.5239 Query 137: 0.0000 Query 138: 0.0150 Query 139: 0.5099 Query 140: 0.6221 −1.0 91 100 110 120 130 140 Topic number JHU/APL, Monolingual German, Auto, TD Run aplmode Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.2640 Total number of documents Query 92: 0.3418 over all queries Query 93: 0.8639 Retrieved: 50000 0.8 Query 94: 0.9236 Relevant: 1938 Query 95: 0.2115 Rel_ret: 1792 Query 96: 0.6337 Query 97: 0.7802 Interpolated Recall - Query 98: 0.3304 Precision Averages: 0.6 Query 99: 0.5439 at 0.00 0.7179 Query 100: 0.6478 at 0.10 0.6479 Query 101: 0.5476 at 0.20 0.6121 Precision Query 102: 0.3986 at 0.30 0.5630 0.4 Query 103: 0.5205 at 0.40 0.5298 Query 104: 0.5083 at 0.50 0.4960 Query 105: 0.2154 at 0.60 0.4683 Query 106: 0.3054 at 0.70 0.4180 0.2 Query 107: 0.1399 at 0.80 0.3621 Query 108: 0.4255 at 0.90 0.2686 Query 109: 0.1675 at 1.00 0.1620 Query 110: 0.7455 0.0 Query 111: 0.0635 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.7913 for all rel. documents Query 113: 0.2421 0.4663 Recall Query 114: 0.3319 Precision: Query 115: 0.0247 At 5 docs: 0.5560 Run aplmode Query 116: 0.7678 At 10 docs: 0.5100 Query 117: 0.3307 At 15 docs: 0.4893 Comparison to median by topic Query 118: 0.1490 At 20 docs: 0.4580 +1.0 Query 119: 0.8526 At 30 docs: 0.4127 Query 120: 0.0220 At 100 docs: 0.2394 Query 121: 0.7576 At 200 docs: 0.1451 Query 122: 0.0485 At 500 docs: 0.0681 Query 123: 0.8881 At 1000 docs: 0.0358 +0.5 Query 124: 0.6096 Query 125: 0.6296 R-Precision (prec. after R Query 126: 0.0145 (=num_rel) docs. retrieved): Query 127: 0.8611 Exact: 0.4591 Query 128: 0.0478 0.0 Query 129: 0.7166 Query 130: 0.7645 Query 131: 0.0011 Query 132: 0.8105 Query 133: 0.6304 Difference (average precision) Query 134: 0.7871 −0.5 Query 135: 0.3590 Query 136: 1.0000 Query 137: 0.0185 Query 138: 0.7140 Query 139: 0.1706 −1.0 Query 140: 0.3934 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 1, Monolingual German, Auto, TDN Run BKMLGG1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.3080 Total number of documents Query 92: 0.3299 over all queries Query 93: 0.9098 Retrieved: 50000 0.8 Query 94: 0.8850 Relevant: 1938 Query 95: 0.1490 Rel_ret: 1705 Query 96: 0.6141 Query 97: 0.8464 Interpolated Recall - 0.6 Query 98: 0.6179 Precision Averages: Query 99: 0.6241 at 0.00 0.7686 Query 100: 0.6057 at 0.10 0.6750 Precision Query 101: 0.5057 at 0.20 0.6257 Query 102: 0.4255 at 0.30 0.5654 0.4 Query 103: 0.4666 at 0.40 0.5367 Query 104: 0.3366 at 0.50 0.5018 Query 105: 0.4665 at 0.60 0.4722 Query 106: 0.1309 at 0.70 0.4239 0.2 Query 107: 0.0719 at 0.80 0.3413 Query 108: 0.4175 at 0.90 0.2642 Query 109: 0.1793 at 1.00 0.1681 Query 110: 0.5800 0.0 Query 111: 0.0825 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6229 for all rel. documents Query 113: 0.1216 0.4696 Recall Query 114: 0.4547 Precision: Query 115: 0.3580 At 5 docs: 0.5960 Run BKMLGG1 Query 116: 0.7008 At 10 docs: 0.5460 Comparison to median by topic Query 117: 0.5523 At 15 docs: 0.5187 +1.0 Query 118: 0.4267 At 20 docs: 0.4800 Query 119: 0.8477 At 30 docs: 0.4333 Query 120: 0.0183 At 100 docs: 0.2376 Query 121: 0.4514 At 200 docs: 0.1423 Query 122: 0.0389 At 500 docs: 0.0650 Query 123: 0.8833 At 1000 docs: 0.0341 +0.5 Query 124: 0.5805 Query 125: 0.6196 R-Precision (prec. after R Query 126: 0.2315 (=num_rel) docs. retrieved): Query 127: 0.8973 Exact: 0.4639 Query 128: 0.0372 0.0 Query 129: 0.6964 Query 130: 0.8487 Query 131: 0.0295 Query 132: 0.6380 Query 133: 0.7521 Difference (average precision) Query 134: 0.5802 −0.5 Query 135: 0.6250 Query 136: 1.0000 Query 137: 0.0000 Query 138: 0.8772 Query 139: 0.0386 −1.0 Query 140: 0.0004 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 1, Monolingual German, Auto, TD Run BKMLGG2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.3434 Total number of documents Query 92: 0.3334 over all queries Query 93: 0.8868 Retrieved: 50000 0.8 Query 94: 0.9008 Relevant: 1938 Query 95: 0.1486 Rel_ret: 1734 Query 96: 0.7040 Query 97: 0.5706 Interpolated Recall - 0.6 Query 98: 0.6335 Precision Averages: Query 99: 0.4569 at 0.00 0.7670 Query 100: 0.5964 at 0.10 0.6161 Precision Query 101: 0.7420 at 0.20 0.5836 Query 102: 0.4258 at 0.30 0.5352 0.4 Query 103: 0.5045 at 0.40 0.4983 Query 104: 0.2810 at 0.50 0.4753 Query 105: 0.2042 at 0.60 0.4426 Query 106: 0.2375 at 0.70 0.4027 0.2 Query 107: 0.1126 at 0.80 0.3406 Query 108: 0.4233 at 0.90 0.2445 Query 109: 0.0006 at 1.00 0.1451 Query 110: 0.5573 0.0 Query 111: 0.0453 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6194 for all rel. documents Query 113: 0.2623 0.4404 Recall Query 114: 0.4261 Precision: Query 115: 0.2034 At 5 docs: 0.5560 Run BKMLGG2 Query 116: 0.7067 At 10 docs: 0.5220 Comparison to median by topic Query 117: 0.6375 At 15 docs: 0.4853 +1.0 Query 118: 0.1233 At 20 docs: 0.4600 Query 119: 0.8198 At 30 docs: 0.4127 Query 120: 0.0426 At 100 docs: 0.2236 Query 121: 0.3833 At 200 docs: 0.1382 Query 122: 0.0241 At 500 docs: 0.0650 Query 123: 0.8929 At 1000 docs: 0.0347 +0.5 Query 124: 0.4533 Query 125: 0.5425 R-Precision (prec. after R Query 126: 0.0437 (=num_rel) docs. retrieved): Query 127: 0.8996 Exact: 0.4223 Query 128: 0.1307 0.0 Query 129: 0.6890 Query 130: 0.7301 Query 131: 0.0114 Query 132: 0.7202 Query 133: 0.7082 Difference (average precision) Query 134: 0.6619 −0.5 Query 135: 0.3091 Query 136: 0.9481 Query 137: 0.0000 Query 138: 0.8052 Query 139: 0.0467 −1.0 Query 140: 0.0718 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 2, Monolingual German, Auto, TD Run bky2mode Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.3621 Total number of documents Query 92: 0.3485 over all queries Query 93: 0.8289 Retrieved: 50000 0.8 Query 94: 0.8517 Relevant: 1938 Query 95: 0.1451 Rel_ret: 1807 Query 96: 0.4631 Query 97: 0.6372 Interpolated Recall - Query 98: 0.6020 Precision Averages: 0.6 Query 99: 0.1539 at 0.00 0.8308 Query 100: 0.7521 at 0.10 0.7349 Query 101: 0.8186 at 0.20 0.6989 Precision Query 102: 0.4513 at 0.30 0.6489 0.4 Query 103: 0.6436 at 0.40 0.6043 Query 104: 0.1337 at 0.50 0.5675 Query 105: 0.6399 at 0.60 0.5202 Query 106: 0.3993 at 0.70 0.4705 0.2 Query 107: 0.6462 at 0.80 0.3898 Query 108: 0.4214 at 0.90 0.2884 Query 109: 0.5214 at 1.00 0.1735 Query 110: 0.4726 0.0 Query 111: 0.0428 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.9250 for all rel. documents Query 113: 0.1856 0.5234 Recall Query 114: 0.3313 Precision: Query 115: 0.5274 At 5 docs: 0.6480 Run bky2mode Query 116: 0.6150 At 10 docs: 0.5800 Query 117: 0.5123 At 15 docs: 0.5467 Comparison to median by topic Query 118: 0.1200 At 20 docs: 0.5130 +1.0 Query 119: 0.8392 At 30 docs: 0.4573 Query 120: 0.0382 At 100 docs: 0.2562 Query 121: 0.3131 At 200 docs: 0.1532 Query 122: 0.4119 At 500 docs: 0.0691 Query 123: 0.9821 At 1000 docs: 0.0361 +0.5 Query 124: 0.4723 Query 125: 0.4944 R-Precision (prec. after R Query 126: 0.2901 (=num_rel) docs. retrieved): Query 127: 0.9090 Exact: 0.5218 Query 128: 0.0745 0.0 Query 129: 0.6987 Query 130: 0.7174 Query 131: 0.0518 Query 132: 0.7160 Query 133: 0.5812 Difference (average precision) Query 134: 0.8231 −0.5 Query 135: 0.7314 Query 136: 1.0000 Query 137: 1.0000 Query 138: 0.5224 Query 139: 0.3670 −1.0 Query 140: 0.5857 91 100 110 120 130 140 Topic number Eurospider IT AG, Monolingual German, Auto, TD Run EIT2GDB1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1106 Total number of documents Query 92: 0.2981 over all queries Query 93: 0.8649 Retrieved: 50000 0.8 Query 94: 0.8726 Relevant: 1938 Query 95: 0.0098 Rel_ret: 1692 Query 96: 0.5354 Query 97: 0.1412 Interpolated Recall - 0.6 Query 98: 0.4342 Precision Averages: Query 99: 0.0868 at 0.00 0.7908 Query 100: 0.7434 at 0.10 0.6267 Precision Query 101: 0.7668 at 0.20 0.5772 Query 102: 0.4311 at 0.30 0.5441 0.4 Query 103: 0.6714 at 0.40 0.5139 Query 104: 0.1202 at 0.50 0.4800 Query 105: 0.6055 at 0.60 0.4425 Query 106: 0.2696 at 0.70 0.3974 0.2 Query 107: 0.3371 at 0.80 0.3384 Query 108: 0.3437 at 0.90 0.2311 Query 109: 0.3336 at 1.00 0.1291 Query 110: 0.7007 0.0 Query 111: 0.0314 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.9422 for all rel. documents Query 113: 0.2240 0.4482 Recall Query 114: 0.1408 Precision: Query 115: 0.1879 At 5 docs: 0.5640 Run EIT2GDB1 Query 116: 0.6403 At 10 docs: 0.5160 Comparison to median by topic Query 117: 0.3887 At 15 docs: 0.4747 +1.0 Query 118: 0.1464 At 20 docs: 0.4460 Query 119: 0.8102 At 30 docs: 0.4113 Query 120: 0.0943 At 100 docs: 0.2308 Query 121: 0.2186 At 200 docs: 0.1395 Query 122: 0.0616 At 500 docs: 0.0640 Query 123: 0.9325 At 1000 docs: 0.0338 +0.5 Query 124: 0.4478 Query 125: 0.5894 R-Precision (prec. after R Query 126: 0.5370 (=num_rel) docs. retrieved): Query 127: 0.8977 Exact: 0.4314 Query 128: 0.0593 0.0 Query 129: 0.5526 Query 130: 0.7423 Query 131: 0.0092 Query 132: 0.7940 Query 133: 0.6144 Difference (average precision) Query 134: 0.8485 −0.5 Query 135: 0.3744 Query 136: 1.0000 Query 137: 0.0588 Query 138: 0.5817 Query 139: 0.2761 −1.0 Query 140: 0.5300 91 100 110 120 130 140 Topic number Eurospider IT AG, Monolingual German, Auto, TD Run EIT2GDL1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1369 Total number of documents Query 92: 0.2673 over all queries Query 93: 0.9049 Retrieved: 50000 0.8 Query 94: 0.8866 Relevant: 1938 Query 95: 0.0139 Rel_ret: 1704 Query 96: 0.6233 Query 97: 0.1803 Interpolated Recall - 0.6 Query 98: 0.5652 Precision Averages: Query 99: 0.0409 at 0.00 0.7596 Query 100: 0.7611 at 0.10 0.6408 Precision Query 101: 0.7405 at 0.20 0.6027 Query 102: 0.4209 at 0.30 0.5657 0.4 Query 103: 0.6765 at 0.40 0.5283 Query 104: 0.1289 at 0.50 0.4828 Query 105: 0.6018 at 0.60 0.4439 Query 106: 0.2510 at 0.70 0.3974 0.2 Query 107: 0.3297 at 0.80 0.3258 Query 108: 0.3693 at 0.90 0.2406 Query 109: 0.3268 at 1.00 0.1412 Query 110: 0.6775 0.0 Query 111: 0.0498 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.9222 for all rel. documents Query 113: 0.1698 0.4561 Recall Query 114: 0.1247 Precision: Query 115: 0.2207 At 5 docs: 0.5920 Run EIT2GDL1 Query 116: 0.6515 At 10 docs: 0.5420 Comparison to median by topic Query 117: 0.3557 At 15 docs: 0.4893 +1.0 Query 118: 0.1055 At 20 docs: 0.4620 Query 119: 0.7713 At 30 docs: 0.4240 Query 120: 0.0628 At 100 docs: 0.2336 Query 121: 0.2087 At 200 docs: 0.1393 Query 122: 0.0872 At 500 docs: 0.0642 Query 123: 1.0000 At 1000 docs: 0.0341 +0.5 Query 124: 0.4421 Query 125: 0.6592 R-Precision (prec. after R Query 126: 0.4889 (=num_rel) docs. retrieved): Query 127: 0.9123 Exact: 0.4612 Query 128: 0.0880 0.0 Query 129: 0.5222 Query 130: 0.7162 Query 131: 0.0045 Query 132: 0.8599 Query 133: 0.6111 Difference (average precision) Query 134: 0.7844 −0.5 Query 135: 0.5492 Query 136: 1.0000 Query 137: 0.0063 Query 138: 0.5718 Query 139: 0.3531 −1.0 Query 140: 0.6045 91 100 110 120 130 140 Topic number Eurospider IT AG, Monolingual German, Auto, TD Run EIT2GDM1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1291 Total number of documents Query 92: 0.2841 over all queries Query 93: 0.8888 Retrieved: 50000 0.8 Query 94: 0.8919 Relevant: 1938 Query 95: 0.0095 Rel_ret: 1708 Query 96: 0.5850 Query 97: 0.1381 Interpolated Recall - 0.6 Query 98: 0.5032 Precision Averages: Query 99: 0.0651 at 0.00 0.7737 Query 100: 0.7507 at 0.10 0.6448 Precision Query 101: 0.7689 at 0.20 0.5928 Query 102: 0.4501 at 0.30 0.5604 0.4 Query 103: 0.6944 at 0.40 0.5236 Query 104: 0.1260 at 0.50 0.4865 Query 105: 0.6151 at 0.60 0.4494 Query 106: 0.2635 at 0.70 0.4049 0.2 Query 107: 0.3382 at 0.80 0.3398 Query 108: 0.3583 at 0.90 0.2372 Query 109: 0.3753 at 1.00 0.1357 Query 110: 0.6950 0.0 Query 111: 0.0432 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.9487 for all rel. documents Query 113: 0.1913 0.4577 Recall Query 114: 0.1262 Precision: Query 115: 0.2161 At 5 docs: 0.5960 Run EIT2GDM1 Query 116: 0.6421 At 10 docs: 0.5320 Comparison to median by topic Query 117: 0.3805 At 15 docs: 0.4907 +1.0 Query 118: 0.1236 At 20 docs: 0.4470 Query 119: 0.7967 At 30 docs: 0.4233 Query 120: 0.0701 At 100 docs: 0.2358 Query 121: 0.2152 At 200 docs: 0.1402 Query 122: 0.0760 At 500 docs: 0.0647 Query 123: 0.9683 At 1000 docs: 0.0342 +0.5 Query 124: 0.4514 Query 125: 0.6364 R-Precision (prec. after R Query 126: 0.5264 (=num_rel) docs. retrieved): Query 127: 0.9100 Exact: 0.4541 Query 128: 0.0749 0.0 Query 129: 0.5486 Query 130: 0.7470 Query 131: 0.0066 Query 132: 0.8239 Query 133: 0.6245 Difference (average precision) Query 134: 0.8120 −0.5 Query 135: 0.4675 Query 136: 1.0000 Query 137: 0.0213 Query 138: 0.5927 Query 139: 0.3338 −1.0 Query 140: 0.5795 91 100 110 120 130 140 Topic number Eurospider IT AG, Monolingual German, Auto, TDN Run EIT2GNM1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.2311 Total number of documents Query 92: 0.2933 over all queries Query 93: 0.9424 Retrieved: 50000 0.8 Query 94: 0.8929 Relevant: 1938 Query 95: 0.3791 Rel_ret: 1843 Query 96: 0.5480 Query 97: 0.7767 Interpolated Recall - 0.6 Query 98: 0.4905 Precision Averages: Query 99: 0.4454 at 0.00 0.8352 Query 100: 0.7627 at 0.10 0.7262 Precision Query 101: 0.6292 at 0.20 0.6908 Query 102: 0.4644 at 0.30 0.6289 0.4 Query 103: 0.7158 at 0.40 0.5854 Query 104: 0.1256 at 0.50 0.5388 Query 105: 0.6323 at 0.60 0.4966 Query 106: 0.3625 at 0.70 0.4463 0.2 Query 107: 0.3324 at 0.80 0.3808 Query 108: 0.3617 at 0.90 0.2660 Query 109: 0.4613 at 1.00 0.1574 Query 110: 0.6794 0.0 Query 111: 0.0521 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.9452 for all rel. documents Query 113: 0.2066 0.5148 Recall Query 114: 0.4153 Precision: Query 115: 0.1728 At 5 docs: 0.6760 Run EIT2GNM1 Query 116: 0.5222 At 10 docs: 0.5940 Comparison to median by topic Query 117: 0.2727 At 15 docs: 0.5440 +1.0 Query 118: 0.3329 At 20 docs: 0.5050 Query 119: 0.8156 At 30 docs: 0.4680 Query 120: 0.0576 At 100 docs: 0.2604 Query 121: 0.4556 At 200 docs: 0.1566 Query 122: 0.1406 At 500 docs: 0.0709 Query 123: 0.9683 At 1000 docs: 0.0369 +0.5 Query 124: 0.5000 Query 125: 0.7027 R-Precision (prec. after R Query 126: 0.6779 (=num_rel) docs. retrieved): Query 127: 0.9239 Exact: 0.5025 Query 128: 0.0406 0.0 Query 129: 0.4886 Query 130: 0.8268 Query 131: 0.0410 Query 132: 0.8025 Query 133: 0.6779 Difference (average precision) Query 134: 0.8072 −0.5 Query 135: 0.5437 Query 136: 1.0000 Query 137: 0.0141 Query 138: 0.8190 Query 139: 0.3218 −1.0 Query 140: 0.6682 91 100 110 120 130 140 Topic number Hummingbird, Monolingual German, Auto, TD Run humDE02 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.2582 Total number of documents Query 92: 0.2706 over all queries Query 93: 0.8614 Retrieved: 50000 0.8 Query 94: 0.8291 Relevant: 1938 Query 95: 0.0331 Rel_ret: 1707 Query 96: 0.3325 Query 97: 0.3809 Interpolated Recall - 0.6 Query 98: 0.3748 Precision Averages: Query 99: 0.1882 at 0.00 0.8087 Query 100: 0.7532 at 0.10 0.6996 Precision Query 101: 0.8040 at 0.20 0.6129 Query 102: 0.4237 at 0.30 0.5415 0.4 Query 103: 0.4728 at 0.40 0.4932 Query 104: 0.2736 at 0.50 0.4344 Query 105: 0.6278 at 0.60 0.4049 Query 106: 0.2700 at 0.70 0.3505 0.2 Query 107: 0.0924 at 0.80 0.2882 Query 108: 0.4700 at 0.90 0.2086 Query 109: 0.2642 at 1.00 0.1162 Query 110: 0.5360 0.0 Query 111: 0.0644 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.9309 for all rel. documents Query 113: 0.1130 0.4370 Recall Query 114: 0.1553 Precision: Query 115: 0.2908 At 5 docs: 0.6440 Run humDE02 Query 116: 0.5417 At 10 docs: 0.5520 Comparison to median by topic Query 117: 0.2999 At 15 docs: 0.4987 +1.0 Query 118: 0.1435 At 20 docs: 0.4680 Query 119: 0.8307 At 30 docs: 0.4073 Query 120: 0.0744 At 100 docs: 0.2202 Query 121: 0.1765 At 200 docs: 0.1363 Query 122: 0.2397 At 500 docs: 0.0630 Query 123: 1.0000 At 1000 docs: 0.0341 +0.5 Query 124: 0.5016 Query 125: 0.5081 R-Precision (prec. after R Query 126: 0.4020 (=num_rel) docs. retrieved): Query 127: 0.6645 Exact: 0.4462 Query 128: 0.1141 0.0 Query 129: 0.4423 Query 130: 0.6562 Query 131: 0.0265 Query 132: 0.8081 Query 133: 0.5544 Difference (average precision) Query 134: 0.8844 −0.5 Query 135: 0.3846 Query 136: 1.0000 Query 137: 0.2500 Query 138: 0.5587 Query 139: 0.2776 −1.0 Query 140: 0.4416 91 100 110 120 130 140 Topic number IMBIT Inst., Univ. Hildesheim, Monolingual German, Manual, TDN Run IMBIT Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1474 Total number of documents Query 92: 0.0101 over all queries Query 93: 0.4722 Retrieved: 653 0.8 Query 94: 0.2500 Relevant: 1938 Query 95: 0.0059 Rel_ret: 358 Query 96: 0.1468 Query 97: 0.1833 Interpolated Recall - 0.6 Query 98: 0.3600 Precision Averages: Query 99: 0.0438 at 0.00 0.7814 Query 100: 0.1111 at 0.10 0.5775 Precision Query 101: 0.1505 at 0.20 0.4014 Query 102: 0.4352 at 0.30 0.1982 0.4 Query 103: 0.0340 at 0.40 0.0727 Query 104: 0.6736 at 0.50 0.0218 Query 105: 0.0545 at 0.60 0.0218 Query 106: 0.0398 at 0.70 0.0182 0.2 Query 107: 0.1103 at 0.80 0.0000 Query 108: 0.0673 at 0.90 0.0000 Query 109: 0.0286 at 1.00 0.0000 Query 110: 0.0417 0.0 Query 111: 0.0208 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.2143 for all rel. documents Query 113: 0.1887 0.1645 Recall Query 114: 0.2184 Precision: Query 115: 0.0347 At 5 docs: 0.5840 Run IMBIT Query 116: 0.3871 At 10 docs: 0.4700 Comparison to median by topic Query 117: 0.1705 At 15 docs: 0.3747 +1.0 Query 118: 0.0694 At 20 docs: 0.3110 Query 119: 0.2761 At 30 docs: 0.2273 Query 120: 0.0000 At 100 docs: 0.0716 Query 121: 0.1162 At 200 docs: 0.0358 Query 122: 0.0737 At 500 docs: 0.0143 Query 123: 0.2143 At 1000 docs: 0.0072 +0.5 Query 124: 0.2413 Query 125: 0.0580 R-Precision (prec. after R Query 126: 0.0319 (=num_rel) docs. retrieved): Query 127: 0.1865 Exact: 0.1997 Query 128: 0.0000 0.0 Query 129: 0.0794 Query 130: 0.3333 Query 131: 0.1332 Query 132: 0.4407 Query 133: 0.1153 Difference (average precision) Query 134: 0.2857 −0.5 Query 135: 0.3281 Query 136: 0.0762 Query 137: 0.0000 Query 138: 0.3522 Query 139: 0.0691 −1.0 Query 140: 0.1454 91 100 110 120 130 140 Topic number Univ. Dortmund, Monolingual German, Auto, TD Run MLstem Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1843 Total number of documents Query 92: 0.2857 over all queries Query 93: 0.7490 Retrieved: 50000 0.8 Query 94: 0.7945 Relevant: 1938 Query 95: 0.1040 Rel_ret: 1385 Query 96: 0.1834 Query 97: 0.5434 Interpolated Recall - 0.6 Query 98: 0.3635 Precision Averages: Query 99: 0.1712 at 0.00 0.6688 Query 100: 0.2516 at 0.10 0.5055 Precision Query 101: 0.4487 at 0.20 0.4402 Query 102: 0.3681 at 0.30 0.3893 0.4 Query 103: 0.2805 at 0.40 0.3582 Query 104: 0.0958 at 0.50 0.3139 Query 105: 0.0218 at 0.60 0.2795 Query 106: 0.1165 at 0.70 0.2246 0.2 Query 107: 0.0446 at 0.80 0.1766 Query 108: 0.2314 at 0.90 0.1216 Query 109: 0.0033 at 1.00 0.0734 Query 110: 0.5223 0.0 Query 111: 0.0000 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.8678 for all rel. documents Query 113: 0.0438 0.3049 Recall Query 114: 0.0752 Precision: Query 115: 0.0382 At 5 docs: 0.4840 Run MLstem Query 116: 0.5646 At 10 docs: 0.4240 Comparison to median by topic Query 117: 0.1369 At 15 docs: 0.3773 +1.0 Query 118: 0.0700 At 20 docs: 0.3530 Query 119: 0.6439 At 30 docs: 0.3027 Query 120: 0.0831 At 100 docs: 0.1614 Query 121: 0.1667 At 200 docs: 0.0975 Query 122: 0.0113 At 500 docs: 0.0488 Query 123: 1.0000 At 1000 docs: 0.0277 +0.5 Query 124: 0.4682 Query 125: 0.1562 R-Precision (prec. after R Query 126: 0.0435 (=num_rel) docs. retrieved): Query 127: 0.6419 Exact: 0.3361 Query 128: 0.0784 0.0 Query 129: 0.2515 Query 130: 0.5261 Query 131: 0.0099 Query 132: 0.6372 Query 133: 0.5909 Difference (average precision) Query 134: 0.8687 −0.5 Query 135: 0.1104 Query 136: 1.0000 Query 137: 0.0000 Query 138: 0.3165 Query 139: 0.0158 −1.0 Query 140: 0.0639 91 100 110 120 130 140 Topic number Océ, Monolingual German, Auto, TD Run oce02monDE Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.2007 Total number of documents Query 92: 0.2862 over all queries Query 93: 0.9057 Retrieved: 50000 0.8 Query 94: 0.7876 Relevant: 1938 Query 95: 0.0707 Rel_ret: 1544 Query 96: 0.3032 Query 97: 0.5187 Interpolated Recall - 0.6 Query 98: 0.4247 Precision Averages: Query 99: 0.1133 at 0.00 0.7061 Query 100: 0.6355 at 0.10 0.6057 Precision Query 101: 0.6566 at 0.20 0.5324 Query 102: 0.3392 at 0.30 0.4657 0.4 Query 103: 0.3455 at 0.40 0.4179 Query 104: 0.0965 at 0.50 0.3863 Query 105: 0.0769 at 0.60 0.3438 Query 106: 0.1427 at 0.70 0.2932 0.2 Query 107: 0.2059 at 0.80 0.2337 Query 108: 0.2181 at 0.90 0.1607 Query 109: 0.1274 at 1.00 0.1055 Query 110: 0.5045 0.0 Query 111: 0.0168 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.9190 for all rel. documents Query 113: 0.1196 0.3710 Recall Query 114: 0.1295 Precision: Query 115: 0.1485 At 5 docs: 0.5520 Run oce02monDE Query 116: 0.7006 At 10 docs: 0.4820 Comparison to median by topic Query 117: 0.2474 At 15 docs: 0.4560 +1.0 Query 118: 0.2411 At 20 docs: 0.4180 Query 119: 0.7518 At 30 docs: 0.3507 Query 120: 0.0613 At 100 docs: 0.1912 Query 121: 0.2032 At 200 docs: 0.1150 Query 122: 0.0431 At 500 docs: 0.0555 Query 123: 0.9821 At 1000 docs: 0.0309 +0.5 Query 124: 0.5786 Query 125: 0.4395 R-Precision (prec. after R Query 126: 0.2898 (=num_rel) docs. retrieved): Query 127: 0.7112 Exact: 0.3938 Query 128: 0.1886 0.0 Query 129: 0.3576 Query 130: 0.5604 Query 131: 0.0051 Query 132: 0.6697 Query 133: 0.6310 Difference (average precision) Query 134: 0.8984 −0.5 Query 135: 0.1379 Query 136: 1.0000 Query 137: 0.0000 Query 138: 0.3961 Query 139: 0.0178 −1.0 Query 140: 0.1439 91 100 110 120 130 140 Topic number Océ, Monolingual German, Auto, T Run oce02monDEto Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1791 Total number of documents Query 92: 0.1649 over all queries Query 93: 0.7254 Retrieved: 39709 0.8 Query 94: 0.8222 Relevant: 1938 Query 95: 0.0042 Rel_ret: 1229 Query 96: 0.0014 Query 97: 0.6752 Interpolated Recall - 0.6 Query 98: 0.0000 Precision Averages: Query 99: 0.0414 at 0.00 0.6435 Query 100: 0.5395 at 0.10 0.4978 Precision Query 101: 0.6388 at 0.20 0.4317 Query 102: 0.4135 at 0.30 0.3968 0.4 Query 103: 0.1301 at 0.40 0.3469 Query 104: 0.1079 at 0.50 0.2962 Query 105: 0.0500 at 0.60 0.2634 Query 106: 0.0391 at 0.70 0.2266 0.2 Query 107: 0.2213 at 0.80 0.1918 Query 108: 0.3267 at 0.90 0.1263 Query 109: 0.0504 at 1.00 0.0839 Query 110: 0.1102 0.0 Query 111: 0.0160 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6737 for all rel. documents Query 113: 0.0000 0.3025 Recall Query 114: 0.0690 Precision: Query 115: 0.0417 At 5 docs: 0.4560 Run oce02monDEto Query 116: 0.8020 At 10 docs: 0.3900 Comparison to median by topic Query 117: 0.1505 At 15 docs: 0.3573 +1.0 Query 118: 0.2668 At 20 docs: 0.3240 Query 119: 0.6898 At 30 docs: 0.2867 Query 120: 0.0113 At 100 docs: 0.1474 Query 121: 0.3928 At 200 docs: 0.0926 Query 122: 0.0300 At 500 docs: 0.0447 Query 123: 0.9094 At 1000 docs: 0.0246 +0.5 Query 124: 0.3821 Query 125: 0.3464 R-Precision (prec. after R Query 126: 0.3732 (=num_rel) docs. retrieved): Query 127: 0.0033 Exact: 0.3137 Query 128: 0.1135 0.0 Query 129: 0.3091 Query 130: 0.5747 Query 131: 0.0100 Query 132: 0.8273 Query 133: 0.5518 Difference (average precision) Query 134: 0.7836 −0.5 Query 135: 0.0272 Query 136: 1.0000 Query 137: 0.0000 Query 138: 0.1903 Query 139: 0.0203 −1.0 Query 140: 0.3192 91 100 110 120 130 140 Topic number City University, Monolingual German, Auto, T Run plge02t Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0715 Total number of documents Query 92: 0.1856 over all queries Query 93: 0.0000 Retrieved: 50000 0.8 Query 94: 0.8599 Relevant: 1938 Query 95: 0.0000 Rel_ret: 748 Query 96: 0.0000 Query 97: 0.2002 Interpolated Recall - Query 98: 0.0013 Precision Averages: 0.6 Query 99: 0.0128 at 0.00 0.3576 Query 100: 0.0000 at 0.10 0.2540 Query 101: 0.0087 at 0.20 0.2233 Precision Query 102: 0.0000 at 0.30 0.1833 0.4 Query 103: 0.0854 at 0.40 0.1645 Query 104: 0.0021 at 0.50 0.1477 Query 105: 0.0000 at 0.60 0.1409 Query 106: 0.0764 at 0.70 0.1221 0.2 Query 107: 0.0000 at 0.80 0.0796 Query 108: 0.1244 at 0.90 0.0559 Query 109: 0.0000 at 1.00 0.0218 Query 110: 0.0430 0.0 Query 111: 0.0000 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5605 for all rel. documents Query 113: 0.0000 0.1470 Recall Query 114: 0.1359 Precision: Query 115: 0.0000 At 5 docs: 0.2280 Run plge02t Query 116: 0.5792 At 10 docs: 0.2100 Query 117: 0.0101 At 15 docs: 0.1987 Comparison to median by topic Query 118: 0.2534 At 20 docs: 0.1880 +1.0 Query 119: 0.7229 At 30 docs: 0.1687 Query 120: 0.0084 At 100 docs: 0.0860 Query 121: 0.0000 At 200 docs: 0.0524 Query 122: 0.0091 At 500 docs: 0.0257 Query 123: 0.5479 At 1000 docs: 0.0150 +0.5 Query 124: 0.0528 Query 125: 0.0000 R-Precision (prec. after R Query 126: 0.0440 (=num_rel) docs. retrieved): Query 127: 0.0006 Exact: 0.1721 Query 128: 0.0491 0.0 Query 129: 0.3264 Query 130: 0.0001 Query 131: 0.0055 Query 132: 0.5762 Query 133: 0.5584 Difference (average precision) Query 134: 0.0000 −0.5 Query 135: 0.0038 Query 136: 0.9683 Query 137: 0.0000 Query 138: 0.0049 Query 139: 0.0154 −1.0 Query 140: 0.2473 91 100 110 120 130 140 Topic number City University, Monolingual German, Auto, TD Run plge02td Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.2022 Total number of documents Query 92: 0.2505 over all queries Query 93: 0.0000 Retrieved: 50000 0.8 Query 94: 0.7761 Relevant: 1938 Query 95: 0.0000 Rel_ret: 918 Query 96: 0.1156 Query 97: 0.1695 Interpolated Recall - Query 98: 0.0000 Precision Averages: 0.6 Query 99: 0.0698 at 0.00 0.4347 Query 100: 0.0009 at 0.10 0.3226 Query 101: 0.0040 at 0.20 0.2638 Precision Query 102: 0.0001 at 0.30 0.2120 0.4 Query 103: 0.0729 at 0.40 0.1860 Query 104: 0.0315 at 0.50 0.1620 Query 105: 0.0025 at 0.60 0.1344 Query 106: 0.0982 at 0.70 0.1153 0.2 Query 107: 0.0000 at 0.80 0.0954 Query 108: 0.0360 at 0.90 0.0647 Query 109: 0.0000 at 1.00 0.0452 Query 110: 0.5409 0.0 Query 111: 0.0006 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.9328 for all rel. documents Query 113: 0.0008 0.1729 Recall Query 114: 0.0836 Precision: Query 115: 0.0022 At 5 docs: 0.2840 Run plge02td Query 116: 0.4405 At 10 docs: 0.2540 Query 117: 0.0230 At 15 docs: 0.2187 Comparison to median by topic Query 118: 0.0910 At 20 docs: 0.1980 +1.0 Query 119: 0.5351 At 30 docs: 0.1787 Query 120: 0.0236 At 100 docs: 0.1016 Query 121: 0.0000 At 200 docs: 0.0635 Query 122: 0.0005 At 500 docs: 0.0314 Query 123: 0.8736 At 1000 docs: 0.0184 +0.5 Query 124: 0.1813 Query 125: 0.0006 R-Precision (prec. after R Query 126: 0.0435 (=num_rel) docs. retrieved): Query 127: 0.4602 Exact: 0.1973 Query 128: 0.0855 0.0 Query 129: 0.2379 Query 130: 0.0438 Query 131: 0.0127 Query 132: 0.3806 Query 133: 0.5416 Difference (average precision) Query 134: 0.1894 −0.5 Query 135: 0.0525 Query 136: 0.9341 Query 137: 0.0000 Query 138: 0.0174 Query 139: 0.0126 −1.0 Query 140: 0.0708 91 100 110 120 130 140 Topic number TLR Research, Monolingual German, Auto, TD Run tlrde Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.3084 Total number of documents Query 92: 0.2604 over all queries Query 93: 0.8007 Retrieved: 50000 0.8 Query 94: 0.8060 Relevant: 1938 Query 95: 0.0356 Rel_ret: 1687 Query 96: 0.3092 Query 97: 0.1749 Interpolated Recall - 0.6 Query 98: 0.4879 Precision Averages: Query 99: 0.1515 at 0.00 0.8671 Query 100: 0.7187 at 0.10 0.7044 Precision Query 101: 0.8142 at 0.20 0.5970 Query 102: 0.5015 at 0.30 0.5052 0.4 Query 103: 0.4601 at 0.40 0.4714 Query 104: 0.4552 at 0.50 0.4187 Query 105: 0.5865 at 0.60 0.3816 Query 106: 0.1890 at 0.70 0.3345 0.2 Query 107: 0.0855 at 0.80 0.2793 Query 108: 0.3888 at 0.90 0.1887 Query 109: 0.3102 at 1.00 0.1095 Query 110: 0.5579 0.0 Query 111: 0.0658 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.8834 for all rel. documents Query 113: 0.3738 0.4221 Recall Query 114: 0.1547 Precision: Query 115: 0.2508 At 5 docs: 0.6240 Run tlrde Query 116: 0.5055 At 10 docs: 0.5340 Comparison to median by topic Query 117: 0.3397 At 15 docs: 0.4653 +1.0 Query 118: 0.1710 At 20 docs: 0.4310 Query 119: 0.8104 At 30 docs: 0.3800 Query 120: 0.0505 At 100 docs: 0.2144 Query 121: 0.1730 At 200 docs: 0.1330 Query 122: 0.2181 At 500 docs: 0.0629 Query 123: 1.0000 At 1000 docs: 0.0337 +0.5 Query 124: 0.5215 Query 125: 0.4084 R-Precision (prec. after R Query 126: 0.2809 (=num_rel) docs. retrieved): Query 127: 0.6916 Exact: 0.4294 Query 128: 0.1463 0.0 Query 129: 0.3090 Query 130: 0.6215 Query 131: 0.0170 Query 132: 0.6750 Query 133: 0.4935 Difference (average precision) Query 134: 0.8810 −0.5 Query 135: 0.3159 Query 136: 1.0000 Query 137: 0.0323 Query 138: 0.4898 Query 139: 0.3845 −1.0 Query 140: 0.4360 91 100 110 120 130 140 Topic number University of Amsterdam, Monolingual German, Auto, TD Run UAmsC02GeGeLC2F Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.2857 Total number of documents Query 92: 0.2726 over all queries Query 93: 0.8140 Retrieved: 50000 0.8 Query 94: 0.8975 Relevant: 1938 Query 95: 0.0631 Rel_ret: 1774 Query 96: 0.4892 Query 97: 0.1218 Interpolated Recall - 0.6 Query 98: 0.4200 Precision Averages: Query 99: 0.6378 at 0.00 0.7862 Query 100: 0.6685 at 0.10 0.6970 Precision Query 101: 0.4733 at 0.20 0.6171 Query 102: 0.6837 at 0.30 0.5637 0.4 Query 103: 0.5387 at 0.40 0.5084 Query 104: 0.3448 at 0.50 0.4643 Query 105: 0.3313 at 0.60 0.4125 Query 106: 0.2995 at 0.70 0.3625 0.2 Query 107: 0.1894 at 0.80 0.3161 Query 108: 0.3807 at 0.90 0.2193 Query 109: 0.0647 at 1.00 0.1235 Query 110: 0.6472 0.0 Query 111: 0.0669 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6475 for all rel. documents Query 113: 0.0079 0.4476 Recall Query 114: 0.3228 Precision: Query 115: 0.2841 At 5 docs: 0.5560 Run UAmsC02GeGeLC2F Query 116: 0.6160 At 10 docs: 0.5520 Comparison to median by topic Query 117: 0.1509 At 15 docs: 0.5067 +1.0 Query 118: 0.4499 At 20 docs: 0.4650 Query 119: 0.8126 At 30 docs: 0.4033 Query 120: 0.1853 At 100 docs: 0.2290 Query 121: 0.5286 At 200 docs: 0.1428 Query 122: 0.3282 At 500 docs: 0.0666 Query 123: 0.9478 At 1000 docs: 0.0355 +0.5 Query 124: 0.3449 Query 125: 0.4636 R-Precision (prec. after R Query 126: 0.5214 (=num_rel) docs. retrieved): Query 127: 0.9230 Exact: 0.4435 Query 128: 0.1365 0.0 Query 129: 0.6176 Query 130: 0.7510 Query 131: 0.0511 Query 132: 0.7571 Query 133: 0.4497 Difference (average precision) Query 134: 0.3663 −0.5 Query 135: 0.2119 Query 136: 1.0000 Query 137: 0.0094 Query 138: 0.8064 Query 139: 0.3633 −1.0 Query 140: 0.6347 91 100 110 120 130 140 Topic number University of Amsterdam, Monolingual German, Auto, TD Run UAmsC02GeGeNGiMO Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.3350 Total number of documents Query 92: 0.2770 over all queries Query 93: 0.8446 Retrieved: 50000 0.8 Query 94: 0.9124 Relevant: 1938 Query 95: 0.0928 Rel_ret: 1789 Query 96: 0.5228 Query 97: 0.3864 Interpolated Recall - 0.6 Query 98: 0.5495 Precision Averages: Query 99: 0.7001 at 0.00 0.8226 Query 100: 0.7104 at 0.10 0.7224 Precision Query 101: 0.4816 at 0.20 0.6471 Query 102: 0.6558 at 0.30 0.6026 0.4 Query 103: 0.6032 at 0.40 0.5441 Query 104: 0.3561 at 0.50 0.4994 Query 105: 0.3838 at 0.60 0.4496 Query 106: 0.2782 at 0.70 0.3997 0.2 Query 107: 0.1919 at 0.80 0.3472 Query 108: 0.3807 at 0.90 0.2474 Query 109: 0.1306 at 1.00 0.1442 Query 110: 0.6459 0.0 Query 111: 0.0626 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.7097 for all rel. documents Query 113: 0.0113 0.4802 Recall Query 114: 0.3620 Precision: Query 115: 0.2833 At 5 docs: 0.6040 Run UAmsC02GeGeNGiMO Query 116: 0.7707 At 10 docs: 0.5560 Comparison to median by topic Query 117: 0.1643 At 15 docs: 0.5160 +1.0 Query 118: 0.5283 At 20 docs: 0.4770 Query 119: 0.8422 At 30 docs: 0.4247 Query 120: 0.1814 At 100 docs: 0.2398 Query 121: 0.4909 At 200 docs: 0.1463 Query 122: 0.2827 At 500 docs: 0.0677 Query 123: 0.9821 At 1000 docs: 0.0358 +0.5 Query 124: 0.3502 Query 125: 0.4685 R-Precision (prec. after R Query 126: 0.4901 (=num_rel) docs. retrieved): Query 127: 0.9318 Exact: 0.4690 Query 128: 0.2201 0.0 Query 129: 0.6097 Query 130: 0.7579 Query 131: 0.0455 Query 132: 0.7906 Query 133: 0.4640 Difference (average precision) Query 134: 0.6957 −0.5 Query 135: 0.2144 Query 136: 1.0000 Query 137: 0.0029 Query 138: 0.8055 Query 139: 0.3744 −1.0 Query 140: 0.6774 91 100 110 120 130 140 Topic number University of Amsterdam, Monolingual German, Auto, TD Run UAmsC02GeGeNGram Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.2401 Total number of documents Query 92: 0.2875 over all queries Query 93: 0.7205 Retrieved: 50000 0.8 Query 94: 0.9108 Relevant: 1938 Query 95: 0.1139 Rel_ret: 1619 Query 96: 0.4140 Query 97: 0.6092 Interpolated Recall - 0.6 Query 98: 0.7476 Precision Averages: Query 99: 0.6599 at 0.00 0.7215 Query 100: 0.6740 at 0.10 0.6366 Precision Query 101: 0.3773 at 0.20 0.5507 Query 102: 0.2016 at 0.30 0.5183 0.4 Query 103: 0.6135 at 0.40 0.4705 Query 104: 0.2518 at 0.50 0.4449 Query 105: 0.4393 at 0.60 0.3952 Query 106: 0.1792 at 0.70 0.3536 0.2 Query 107: 0.1716 at 0.80 0.3005 Query 108: 0.3563 at 0.90 0.2166 Query 109: 0.1734 at 1.00 0.1341 Query 110: 0.6734 0.0 Query 111: 0.0258 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.8737 for all rel. documents Query 113: 0.0100 0.4177 Recall Query 114: 0.3910 Precision: Query 115: 0.3131 At 5 docs: 0.5360 Run UAmsC02GeGeNGram Query 116: 0.8189 At 10 docs: 0.4800 Comparison to median by topic Query 117: 0.1555 At 15 docs: 0.4547 +1.0 Query 118: 0.4556 At 20 docs: 0.4170 Query 119: 0.7991 At 30 docs: 0.3767 Query 120: 0.0151 At 100 docs: 0.2094 Query 121: 0.4304 At 200 docs: 0.1292 Query 122: 0.1788 At 500 docs: 0.0608 Query 123: 0.9617 At 1000 docs: 0.0324 +0.5 Query 124: 0.1763 Query 125: 0.1731 R-Precision (prec. after R Query 126: 0.0147 (=num_rel) docs. retrieved): Query 127: 0.9203 Exact: 0.4216 Query 128: 0.1503 0.0 Query 129: 0.2009 Query 130: 0.7200 Query 131: 0.0070 Query 132: 0.8467 Query 133: 0.3945 Difference (average precision) Query 134: 0.9683 −0.5 Query 135: 0.0979 Query 136: 1.0000 Query 137: 0.0000 Query 138: 0.1118 Query 139: 0.2767 −1.0 Query 140: 0.5848 91 100 110 120 130 140 Topic number Université de Neuchâtel, Monolingual German, Auto, TD Run UniNEde Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.3652 Total number of documents Query 92: 0.3182 over all queries Query 93: 0.8750 Retrieved: 50000 0.8 Query 94: 0.8349 Relevant: 1938 Query 95: 0.2091 Rel_ret: 1777 Query 96: 0.6086 Query 97: 0.4998 Interpolated Recall - 0.6 Query 98: 0.6240 Precision Averages: Query 99: 0.2540 at 0.00 0.7685 Query 100: 0.8028 at 0.10 0.6735 Precision Query 101: 0.7931 at 0.20 0.6192 Query 102: 0.4553 at 0.30 0.5716 0.4 Query 103: 0.5502 at 0.40 0.5259 Query 104: 0.2522 at 0.50 0.4787 Query 105: 0.2088 at 0.60 0.4444 Query 106: 0.2127 at 0.70 0.4091 0.2 Query 107: 0.1538 at 0.80 0.3554 Query 108: 0.4465 at 0.90 0.2633 Query 109: 0.0913 at 1.00 0.1592 Query 110: 0.6265 0.0 Query 111: 0.0477 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.9495 for all rel. documents Query 113: 0.0789 0.4672 Recall Query 114: 0.3230 Precision: Query 115: 0.1952 At 5 docs: 0.6000 Run UniNEde Query 116: 0.7582 At 10 docs: 0.5520 Comparison to median by topic Query 117: 0.3553 At 15 docs: 0.5080 +1.0 Query 118: 0.2066 At 20 docs: 0.4610 Query 119: 0.8357 At 30 docs: 0.4187 Query 120: 0.0491 At 100 docs: 0.2344 Query 121: 0.4667 At 200 docs: 0.1440 Query 122: 0.1433 At 500 docs: 0.0674 Query 123: 1.0000 At 1000 docs: 0.0355 +0.5 Query 124: 0.5448 Query 125: 0.6269 R-Precision (prec. after R Query 126: 0.0146 (=num_rel) docs. retrieved): Query 127: 0.9311 Exact: 0.4571 Query 128: 0.2016 0.0 Query 129: 0.5690 Query 130: 0.6210 Query 131: 0.0018 Query 132: 0.8220 Query 133: 0.6916 Difference (average precision) Query 134: 0.9137 −0.5 Query 135: 0.3587 Query 136: 1.0000 Query 137: 0.0227 Query 138: 0.7577 Query 139: 0.1103 −1.0 Query 140: 0.5831 91 100 110 120 130 140 Topic number Université de Neuchâtel, Monolingual German, Manual, TDN Run UniNEdetdn Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.4205 Total number of documents Query 92: 0.2803 over all queries Query 93: 0.9004 Retrieved: 50000 0.8 Query 94: 0.8574 Relevant: 1938 Query 95: 0.2295 Rel_ret: 1799 Query 96: 0.5580 Query 97: 0.8074 Interpolated Recall - 0.6 Query 98: 0.7256 Precision Averages: Query 99: 0.3798 at 0.00 0.7900 Query 100: 0.7985 at 0.10 0.7128 Precision Query 101: 0.5579 at 0.20 0.6575 Query 102: 0.3282 at 0.30 0.6074 0.4 Query 103: 0.6072 at 0.40 0.5521 Query 104: 0.2330 at 0.50 0.5094 Query 105: 0.2760 at 0.60 0.4690 Query 106: 0.2440 at 0.70 0.4203 0.2 Query 107: 0.1229 at 0.80 0.3659 Query 108: 0.4459 at 0.90 0.2797 Query 109: 0.4225 at 1.00 0.1584 Query 110: 0.6040 0.0 Query 111: 0.0626 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.9470 for all rel. documents Query 113: 0.0362 0.4911 Recall Query 114: 0.5787 Precision: Query 115: 0.3580 At 5 docs: 0.6000 Run UniNEdetdn Query 116: 0.7757 At 10 docs: 0.5940 Comparison to median by topic Query 117: 0.3635 At 15 docs: 0.5387 +1.0 Query 118: 0.2898 At 20 docs: 0.4890 Query 119: 0.8384 At 30 docs: 0.4393 Query 120: 0.0325 At 100 docs: 0.2410 Query 121: 0.4574 At 200 docs: 0.1466 Query 122: 0.1124 At 500 docs: 0.0680 Query 123: 1.0000 At 1000 docs: 0.0360 +0.5 Query 124: 0.5698 Query 125: 0.6482 R-Precision (prec. after R Query 126: 0.1698 (=num_rel) docs. retrieved): Query 127: 0.9374 Exact: 0.4849 Query 128: 0.0423 0.0 Query 129: 0.5498 Query 130: 0.7889 Query 131: 0.0009 Query 132: 0.6498 Query 133: 0.7171 Difference (average precision) Query 134: 0.8588 −0.5 Query 135: 0.4876 Query 136: 1.0000 Query 137: 0.0128 Query 138: 0.8359 Query 139: 0.1224 −1.0 Query 140: 0.5125 91 100 110 120 130 140 Topic number JHU/APL, Monolingual Spanish, Auto, TD Run aplmoes Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.6455 Total number of documents Query 92: 0.4669 over all queries Query 93: 0.2916 Retrieved: 50000 0.8 Query 94: 0.8455 Relevant: 2854 Query 95: 0.3673 Rel_ret: 2659 Query 96: 0.3847 Query 97: 0.8642 Interpolated Recall - Query 98: 0.6810 Precision Averages: 0.6 Query 99: 0.4831 at 0.00 0.8039 Query 100: 0.9082 at 0.10 0.7398 Query 101: 0.5857 at 0.20 0.7150 Precision Query 102: 0.4375 at 0.30 0.6539 0.4 Query 103: 0.6162 at 0.40 0.6103 Query 104: 0.0493 at 0.50 0.5606 Query 105: 0.7560 at 0.60 0.4928 Query 106: 0.1859 at 0.70 0.4359 0.2 Query 107: 0.0906 at 0.80 0.3713 Query 108: 0.7438 at 0.90 0.2727 Query 109: 0.5935 at 1.00 0.1547 Query 110: 0.6051 0.0 Query 111: 0.0156 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6867 for all rel. documents Query 113: 0.6983 0.5192 Recall Query 114: 0.0826 Precision: Query 115: 0.3597 At 5 docs: 0.6120 Run aplmoes Query 116: 0.9127 At 10 docs: 0.5920 Query 117: 0.2976 At 15 docs: 0.5480 Comparison to median by topic Query 118: 0.3809 At 20 docs: 0.5090 +1.0 Query 119: 0.9080 At 30 docs: 0.4700 Query 120: 0.1245 At 100 docs: 0.3070 Query 121: 0.5908 At 200 docs: 0.2056 Query 122: 0.0897 At 500 docs: 0.0991 Query 123: 0.8803 At 1000 docs: 0.0532 +0.5 Query 124: 0.5564 Query 125: 0.7336 R-Precision (prec. after R Query 126: 0.0245 (=num_rel) docs. retrieved): Query 127: 0.7786 Exact: 0.4901 Query 128: 0.3636 0.0 Query 129: 0.4450 Query 130: 0.8008 Query 131: 0.8122 Query 132: 0.7908 Query 133: 0.0062 Difference (average precision) Query 134: 0.4900 −0.5 Query 135: 0.8735 Query 136: 0.7253 Query 137: 0.2745 Query 138: 0.7420 Query 139: 0.7434 −1.0 Query 140: 0.1687 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 2, Monolingual Spanish, Auto, TD Run bky2moes Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.6703 Total number of documents Query 92: 0.4633 over all queries Query 93: 0.2997 Retrieved: 50000 0.8 Query 94: 0.8726 Relevant: 2854 Query 95: 0.4559 Rel_ret: 2673 Query 96: 0.2799 Query 97: 0.8554 Interpolated Recall - Query 98: 0.9000 Precision Averages: 0.6 Query 99: 0.4140 at 0.00 0.8822 Query 100: 0.7434 at 0.10 0.7646 Query 101: 0.3491 at 0.20 0.7343 Precision Query 102: 0.6314 at 0.30 0.6650 0.4 Query 103: 0.8304 at 0.40 0.6242 Query 104: 0.2386 at 0.50 0.5679 Query 105: 0.7937 at 0.60 0.5157 Query 106: 0.2374 at 0.70 0.4643 0.2 Query 107: 0.0495 at 0.80 0.3642 Query 108: 0.6780 at 0.90 0.2862 Query 109: 0.3702 at 1.00 0.1723 Query 110: 0.6311 0.0 Query 111: 0.0277 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5535 for all rel. documents Query 113: 0.7725 0.5338 Recall Query 114: 0.1091 Precision: Query 115: 0.2699 At 5 docs: 0.6600 Run bky2moes Query 116: 0.7119 At 10 docs: 0.6020 Query 117: 0.2672 At 15 docs: 0.5480 Comparison to median by topic Query 118: 0.2678 At 20 docs: 0.5200 +1.0 Query 119: 0.8623 At 30 docs: 0.4780 Query 120: 0.3795 At 100 docs: 0.3176 Query 121: 0.8036 At 200 docs: 0.2096 Query 122: 0.1376 At 500 docs: 0.1018 Query 123: 0.9062 At 1000 docs: 0.0535 +0.5 Query 124: 0.4980 Query 125: 0.7411 R-Precision (prec. after R Query 126: 0.0278 (=num_rel) docs. retrieved): Query 127: 0.7436 Exact: 0.5100 Query 128: 0.1462 0.0 Query 129: 0.2977 Query 130: 0.7932 Query 131: 0.8000 Query 132: 0.6861 Query 133: 0.0007 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.8572 Query 136: 0.7637 Query 137: 0.5160 Query 138: 0.7546 Query 139: 0.8316 −1.0 Query 140: 0.3973 91 100 110 120 130 140 Topic number COLE Group Univ. La Coruna, Monolingual Spanish, Auto, TD Run coleTDlem02 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.4066 Total number of documents Query 92: 0.3544 over all queries Query 93: 0.2125 Retrieved: 50000 0.8 Query 94: 0.6461 Relevant: 2854 Query 95: 0.1839 Rel_ret: 2495 Query 96: 0.2868 Query 97: 0.6441 Interpolated Recall - 0.6 Query 98: 0.2726 Precision Averages: Query 99: 0.2039 at 0.00 0.7173 Query 100: 0.6594 at 0.10 0.6226 Precision Query 101: 0.1130 at 0.20 0.5587 Query 102: 0.6181 at 0.30 0.4786 0.4 Query 103: 0.3669 at 0.40 0.4375 Query 104: 0.0197 at 0.50 0.3877 Query 105: 0.7554 at 0.60 0.3154 Query 106: 0.2396 at 0.70 0.2673 0.2 Query 107: 0.0917 at 0.80 0.2061 Query 108: 0.4879 at 0.90 0.1480 Query 109: 0.3543 at 1.00 0.0627 Query 110: 0.6503 0.0 Query 111: 0.0035 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4053 for all rel. documents Query 113: 0.2901 0.3608 Recall Query 114: 0.0362 Precision: Query 115: 0.2775 At 5 docs: 0.4600 Run coleTDlem02 Query 116: 0.5875 At 10 docs: 0.4540 Comparison to median by topic Query 117: 0.2076 At 15 docs: 0.4133 +1.0 Query 118: 0.2339 At 20 docs: 0.4050 Query 119: 0.7690 At 30 docs: 0.3693 Query 120: 0.1229 At 100 docs: 0.2348 Query 121: 0.5385 At 200 docs: 0.1645 Query 122: 0.0380 At 500 docs: 0.0892 Query 123: 0.8091 At 1000 docs: 0.0499 +0.5 Query 124: 0.3273 Query 125: 0.2070 R-Precision (prec. after R Query 126: 0.0249 (=num_rel) docs. retrieved): Query 127: 0.5186 Exact: 0.3697 Query 128: 0.2135 0.0 Query 129: 0.3250 Query 130: 0.6574 Query 131: 0.7524 Query 132: 0.7314 Query 133: 0.0049 Difference (average precision) Query 134: 0.3302 −0.5 Query 135: 0.4762 Query 136: 0.6365 Query 137: 0.1126 Query 138: 0.1665 Query 139: 0.5029 −1.0 Query 140: 0.1683 91 100 110 120 130 140 Topic number COLE Group Univ. La Coruna, Monolingual Spanish, Auto, TDN Run coleTDNlem02 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.4765 Total number of documents Query 92: 0.3542 over all queries Query 93: 0.4282 Retrieved: 50000 0.8 Query 94: 0.6847 Relevant: 2854 Query 95: 0.3395 Rel_ret: 2634 Query 96: 0.3122 Query 97: 0.7619 Interpolated Recall - 0.6 Query 98: 0.3123 Precision Averages: Query 99: 0.2866 at 0.00 0.8065 Query 100: 0.7255 at 0.10 0.7311 Precision Query 101: 0.4742 at 0.20 0.6699 Query 102: 0.6320 at 0.30 0.5788 0.4 Query 103: 0.4435 at 0.40 0.5360 Query 104: 0.0426 at 0.50 0.4645 Query 105: 0.7268 at 0.60 0.3971 Query 106: 0.2498 at 0.70 0.3560 0.2 Query 107: 0.0566 at 0.80 0.2769 Query 108: 0.4604 at 0.90 0.1883 Query 109: 0.2927 at 1.00 0.0877 Query 110: 0.6853 0.0 Query 111: 0.0055 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4238 for all rel. documents Query 113: 0.3113 0.4448 Recall Query 114: 0.7244 Precision: Query 115: 0.3127 At 5 docs: 0.5480 Run coleTDNlem02 Query 116: 0.6217 At 10 docs: 0.5280 Comparison to median by topic Query 117: 0.1494 At 15 docs: 0.5000 +1.0 Query 118: 0.2320 At 20 docs: 0.4760 Query 119: 0.7812 At 30 docs: 0.4327 Query 120: 0.1268 At 100 docs: 0.2636 Query 121: 0.5539 At 200 docs: 0.1831 Query 122: 0.0573 At 500 docs: 0.0952 Query 123: 0.8513 At 1000 docs: 0.0527 +0.5 Query 124: 0.3769 Query 125: 0.3301 R-Precision (prec. after R Query 126: 0.5947 (=num_rel) docs. retrieved): Query 127: 0.6280 Exact: 0.4466 Query 128: 0.2244 0.0 Query 129: 0.2854 Query 130: 0.6884 Query 131: 0.7358 Query 132: 0.7231 Query 133: 0.2634 Difference (average precision) Query 134: 0.5952 −0.5 Query 135: 0.4917 Query 136: 0.7917 Query 137: 0.1464 Query 138: 0.6471 Query 139: 0.4987 −1.0 Query 140: 0.3240 91 100 110 120 130 140 Topic number COLE Group Univ. La Coruna, Monolingual Spanish, Auto, TDN Run coleTDNpds02 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.4172 Total number of documents Query 92: 0.3382 over all queries Query 93: 0.2995 Retrieved: 50000 0.8 Query 94: 0.6641 Relevant: 2854 Query 95: 0.3132 Rel_ret: 2624 Query 96: 0.3215 Query 97: 0.7175 Interpolated Recall - Query 98: 0.1308 Precision Averages: 0.6 Query 99: 0.4919 at 0.00 0.7732 Query 100: 0.7147 at 0.10 0.6785 Query 101: 0.3097 at 0.20 0.6158 Precision Query 102: 0.6935 at 0.30 0.5278 0.4 Query 103: 0.1687 at 0.40 0.4816 Query 104: 0.0233 at 0.50 0.4263 Query 105: 0.6474 at 0.60 0.3598 Query 106: 0.1878 at 0.70 0.3009 0.2 Query 107: 0.0919 at 0.80 0.2307 Query 108: 0.5774 at 0.90 0.1624 Query 109: 0.3361 at 1.00 0.0689 Query 110: 0.6839 0.0 Query 111: 0.0008 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4668 for all rel. documents Query 113: 0.3354 0.4043 Recall Query 114: 0.4524 Precision: Query 115: 0.3555 At 5 docs: 0.5680 Run coleTDNpds02 Query 116: 0.5566 At 10 docs: 0.5280 Query 117: 0.1490 At 15 docs: 0.4880 Comparison to median by topic Query 118: 0.2690 At 20 docs: 0.4510 +1.0 Query 119: 0.6212 At 30 docs: 0.3960 Query 120: 0.1120 At 100 docs: 0.2474 Query 121: 0.5669 At 200 docs: 0.1780 Query 122: 0.0260 At 500 docs: 0.0933 Query 123: 0.8030 At 1000 docs: 0.0525 +0.5 Query 124: 0.3736 Query 125: 0.3528 R-Precision (prec. after R Query 126: 0.4440 (=num_rel) docs. retrieved): Query 127: 0.6525 Exact: 0.3983 Query 128: 0.0708 0.0 Query 129: 0.1776 Query 130: 0.6503 Query 131: 0.7248 Query 132: 0.6089 Query 133: 0.0512 Difference (average precision) Query 134: 0.4083 −0.5 Query 135: 0.4997 Query 136: 0.7493 Query 137: 0.1069 Query 138: 0.6025 Query 139: 0.5665 −1.0 Query 140: 0.3303 91 100 110 120 130 140 Topic number COLE Group Univ. La Coruna, Monolingual Spanish, Auto, TDN Run coleTDNsyn02 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.4656 Total number of documents Query 92: 0.3570 over all queries Query 93: 0.4282 Retrieved: 50000 0.8 Query 94: 0.6847 Relevant: 2854 Query 95: 0.3395 Rel_ret: 2632 Query 96: 0.2923 Query 97: 0.7619 Interpolated Recall - Query 98: 0.3123 Precision Averages: 0.6 Query 99: 0.2814 at 0.00 0.8052 Query 100: 0.7255 at 0.10 0.7199 Query 101: 0.4742 at 0.20 0.6637 Precision Query 102: 0.6320 at 0.30 0.5812 0.4 Query 103: 0.4435 at 0.40 0.5355 Query 104: 0.0376 at 0.50 0.4619 Query 105: 0.7296 at 0.60 0.4004 Query 106: 0.2525 at 0.70 0.3558 0.2 Query 107: 0.0592 at 0.80 0.2704 Query 108: 0.4604 at 0.90 0.1908 Query 109: 0.2927 at 1.00 0.0837 Query 110: 0.6877 0.0 Query 111: 0.0055 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4238 for all rel. documents Query 113: 0.3113 0.4423 Recall Query 114: 0.7171 Precision: Query 115: 0.3259 At 5 docs: 0.5440 Run coleTDNsyn02 Query 116: 0.6217 At 10 docs: 0.5280 Query 117: 0.1494 At 15 docs: 0.4933 Comparison to median by topic Query 118: 0.2297 At 20 docs: 0.4690 +1.0 Query 119: 0.7818 At 30 docs: 0.4280 Query 120: 0.1348 At 100 docs: 0.2642 Query 121: 0.5539 At 200 docs: 0.1825 Query 122: 0.0822 At 500 docs: 0.0952 Query 123: 0.8465 At 1000 docs: 0.0526 +0.5 Query 124: 0.2951 Query 125: 0.3301 R-Precision (prec. after R Query 126: 0.5947 (=num_rel) docs. retrieved): Query 127: 0.6118 Exact: 0.4438 Query 128: 0.2244 0.0 Query 129: 0.2811 Query 130: 0.6884 Query 131: 0.7266 Query 132: 0.7183 Query 133: 0.2535 Difference (average precision) Query 134: 0.6071 −0.5 Query 135: 0.4917 Query 136: 0.7917 Query 137: 0.1418 Query 138: 0.6374 Query 139: 0.4987 −1.0 Query 140: 0.3238 91 100 110 120 130 140 Topic number Univ. Exeter, Monolingual Spanish, Auto, TD Run exespmgcntmn Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.5143 Total number of documents Query 92: 0.4469 over all queries Query 93: 0.2640 Retrieved: 50000 0.8 Query 94: 0.7357 Relevant: 2854 Query 95: 0.1823 Rel_ret: 2460 Query 96: 0.2946 Query 97: 0.8294 Interpolated Recall - Query 98: 0.9667 Precision Averages: 0.6 Query 99: 0.3588 at 0.00 0.8038 Query 100: 0.5819 at 0.10 0.7267 Query 101: 0.1489 at 0.20 0.6566 Precision Query 102: 0.6141 at 0.30 0.5755 0.4 Query 103: 0.7593 at 0.40 0.5298 Query 104: 0.2242 at 0.50 0.4913 Query 105: 0.8155 at 0.60 0.4481 Query 106: 0.2170 at 0.70 0.4081 0.2 Query 107: 0.0983 at 0.80 0.3268 Query 108: 0.7479 at 0.90 0.2253 Query 109: 0.3481 at 1.00 0.1164 Query 110: 0.7682 0.0 Query 111: 0.0072 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5342 for all rel. documents Query 113: 0.3381 0.4684 Recall Query 114: 0.0303 Precision: Query 115: 0.4047 At 5 docs: 0.5880 Run exespmgcntmn Query 116: 0.8733 At 10 docs: 0.5400 Query 117: 0.2883 At 15 docs: 0.5000 Comparison to median by topic Query 118: 0.2189 At 20 docs: 0.4730 +1.0 Query 119: 0.7759 At 30 docs: 0.4173 Query 120: 0.1446 At 100 docs: 0.2766 Query 121: 0.6649 At 200 docs: 0.1878 Query 122: 0.1062 At 500 docs: 0.0912 Query 123: 0.8587 At 1000 docs: 0.0492 +0.5 Query 124: 0.1150 Query 125: 0.6415 R-Precision (prec. after R Query 126: 0.0325 (=num_rel) docs. retrieved): Query 127: 0.6333 Exact: 0.4555 Query 128: 0.0815 0.0 Query 129: 0.3779 Query 130: 0.7953 Query 131: 0.7232 Query 132: 0.7563 Query 133: 0.0952 Difference (average precision) Query 134: 0.9500 −0.5 Query 135: 0.3955 Query 136: 0.7514 Query 137: 0.1189 Query 138: 0.6252 Query 139: 0.8202 −1.0 Query 140: 0.3448 91 100 110 120 130 140 Topic number Univ. Exeter, Monolingual Spanish, Auto, TD Run exespmgmn Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.5483 Total number of documents Query 92: 0.4583 over all queries Query 93: 0.3322 Retrieved: 50000 0.8 Query 94: 0.7581 Relevant: 2854 Query 95: 0.3337 Rel_ret: 2479 Query 96: 0.2060 Query 97: 0.8489 Interpolated Recall - Query 98: 0.8278 Precision Averages: 0.6 Query 99: 0.4020 at 0.00 0.7942 Query 100: 0.5186 at 0.10 0.7220 Query 101: 0.1248 at 0.20 0.6613 Precision Query 102: 0.5618 at 0.30 0.5787 0.4 Query 103: 0.7964 at 0.40 0.5454 Query 104: 0.1520 at 0.50 0.5113 Query 105: 0.8104 at 0.60 0.4534 Query 106: 0.2018 at 0.70 0.3968 0.2 Query 107: 0.1014 at 0.80 0.3327 Query 108: 0.6559 at 0.90 0.2161 Query 109: 0.3387 at 1.00 0.1015 Query 110: 0.6671 0.0 Query 111: 0.0089 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5233 for all rel. documents Query 113: 0.3614 0.4701 Recall Query 114: 0.0295 Precision: Query 115: 0.4370 At 5 docs: 0.6040 Run exespmgmn Query 116: 0.8753 At 10 docs: 0.5440 Query 117: 0.3345 At 15 docs: 0.5040 Comparison to median by topic Query 118: 0.2352 At 20 docs: 0.4770 +1.0 Query 119: 0.7436 At 30 docs: 0.4213 Query 120: 0.1577 At 100 docs: 0.2834 Query 121: 0.6816 At 200 docs: 0.1895 Query 122: 0.1064 At 500 docs: 0.0920 Query 123: 0.8608 At 1000 docs: 0.0496 +0.5 Query 124: 0.0168 Query 125: 0.7289 R-Precision (prec. after R Query 126: 0.0298 (=num_rel) docs. retrieved): Query 127: 0.7451 Exact: 0.4508 Query 128: 0.1619 0.0 Query 129: 0.3392 Query 130: 0.8507 Query 131: 0.7508 Query 132: 0.7712 Query 133: 0.0884 Difference (average precision) Query 134: 0.8875 −0.5 Query 135: 0.3955 Query 136: 0.7598 Query 137: 0.2501 Query 138: 0.5334 Query 139: 0.8252 −1.0 Query 140: 0.3688 91 100 110 120 130 140 Topic number Univ. Exeter, Monolingual Spanish, Auto, TD Run exesporgmn Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.5690 Total number of documents Query 92: 0.4039 over all queries Query 93: 0.3196 Retrieved: 50000 0.8 Query 94: 0.7525 Relevant: 2854 Query 95: 0.3442 Rel_ret: 2517 Query 96: 0.1050 Query 97: 0.8781 Interpolated Recall - Query 98: 1.0000 Precision Averages: 0.6 Query 99: 0.3958 at 0.00 0.7785 Query 100: 0.5642 at 0.10 0.7268 Query 101: 0.1278 at 0.20 0.6689 Precision Query 102: 0.5866 at 0.30 0.5793 0.4 Query 103: 0.7782 at 0.40 0.5486 Query 104: 0.1495 at 0.50 0.5068 Query 105: 0.8133 at 0.60 0.4587 Query 106: 0.2263 at 0.70 0.4050 0.2 Query 107: 0.1132 at 0.80 0.3341 Query 108: 0.6559 at 0.90 0.2322 Query 109: 0.4015 at 1.00 0.1149 Query 110: 0.7307 0.0 Query 111: 0.0042 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5351 for all rel. documents Query 113: 0.3602 0.4745 Recall Query 114: 0.0322 Precision: Query 115: 0.4370 At 5 docs: 0.6320 Run exesporgmn Query 116: 0.8753 At 10 docs: 0.5680 Query 117: 0.3094 At 15 docs: 0.5160 Comparison to median by topic Query 118: 0.2348 At 20 docs: 0.4960 +1.0 Query 119: 0.7925 At 30 docs: 0.4387 Query 120: 0.1113 At 100 docs: 0.2858 Query 121: 0.6832 At 200 docs: 0.1872 Query 122: 0.1238 At 500 docs: 0.0932 Query 123: 0.8728 At 1000 docs: 0.0503 +0.5 Query 124: 0.2729 Query 125: 0.7289 R-Precision (prec. after R Query 126: 0.0299 (=num_rel) docs. retrieved): Query 127: 0.7211 Exact: 0.4526 Query 128: 0.0928 0.0 Query 129: 0.4289 Query 130: 0.8267 Query 131: 0.7455 Query 132: 0.7689 Query 133: 0.0884 Difference (average precision) Query 134: 0.9500 −0.5 Query 135: 0.3955 Query 136: 0.7566 Query 137: 0.1985 Query 138: 0.2486 Query 139: 0.7900 −1.0 Query 140: 0.3962 91 100 110 120 130 140 Topic number Univ. Exeter, Monolingual Spanish, Auto, TD Run exespwtpimn Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.6783 Total number of documents Query 92: 0.3800 over all queries Query 93: 0.4453 Retrieved: 50000 0.8 Query 94: 0.6552 Relevant: 2854 Query 95: 0.2324 Rel_ret: 2538 Query 96: 0.2115 Query 97: 0.7483 Interpolated Recall - Query 98: 1.0000 Precision Averages: 0.6 Query 99: 0.3502 at 0.00 0.8148 Query 100: 0.5344 at 0.10 0.7429 Query 101: 0.4216 at 0.20 0.6894 Precision Query 102: 0.7205 at 0.30 0.5822 0.4 Query 103: 0.7542 at 0.40 0.5499 Query 104: 0.1185 at 0.50 0.5104 Query 105: 0.8191 at 0.60 0.4455 Query 106: 0.2663 at 0.70 0.3919 0.2 Query 107: 0.0414 at 0.80 0.3190 Query 108: 0.6528 at 0.90 0.2103 Query 109: 0.4362 at 1.00 0.0899 Query 110: 0.7409 0.0 Query 111: 0.0125 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5252 for all rel. documents Query 113: 0.3448 0.4726 Recall Query 114: 0.1002 Precision: Query 115: 0.3684 At 5 docs: 0.6240 Run exespwtpimn Query 116: 0.8916 At 10 docs: 0.5800 Query 117: 0.2465 At 15 docs: 0.5253 Comparison to median by topic Query 118: 0.2218 At 20 docs: 0.4900 +1.0 Query 119: 0.7532 At 30 docs: 0.4427 Query 120: 0.1002 At 100 docs: 0.2880 Query 121: 0.7578 At 200 docs: 0.1953 Query 122: 0.1542 At 500 docs: 0.0933 Query 123: 0.8579 At 1000 docs: 0.0508 +0.5 Query 124: 0.3770 Query 125: 0.4761 R-Precision (prec. after R Query 126: 0.0317 (=num_rel) docs. retrieved): Query 127: 0.7257 Exact: 0.4724 Query 128: 0.1359 0.0 Query 129: 0.4265 Query 130: 0.7359 Query 131: 0.7208 Query 132: 0.4937 Query 133: 0.3297 Difference (average precision) Query 134: 0.7333 −0.5 Query 135: 0.8598 Query 136: 0.7719 Query 137: 0.1262 Query 138: 0.0459 Query 139: 0.7603 −1.0 Query 140: 0.3378 91 100 110 120 130 140 Topic number Hummingbird, Monolingual Spanish, Auto, TD Run humES02 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.6381 Total number of documents Query 92: 0.4404 over all queries Query 93: 0.3427 Retrieved: 50000 0.8 Query 94: 0.7402 Relevant: 2854 Query 95: 0.2135 Rel_ret: 2561 Query 96: 0.4172 Query 97: 0.8222 Interpolated Recall - 0.6 Query 98: 0.4525 Precision Averages: Query 99: 0.3870 at 0.00 0.8684 Query 100: 0.7695 at 0.10 0.7887 Precision Query 101: 0.4478 at 0.20 0.7206 Query 102: 0.7288 at 0.30 0.6124 0.4 Query 103: 0.8312 at 0.40 0.5820 Query 104: 0.2381 at 0.50 0.5106 Query 105: 0.7852 at 0.60 0.4512 Query 106: 0.2167 at 0.70 0.4088 0.2 Query 107: 0.0714 at 0.80 0.3158 Query 108: 0.6533 at 0.90 0.2170 Query 109: 0.3013 at 1.00 0.1092 Query 110: 0.5765 0.0 Query 111: 0.0207 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5319 for all rel. documents Query 113: 0.7153 0.4909 Recall Query 114: 0.0473 Precision: Query 115: 0.2474 At 5 docs: 0.6760 Run humES02 Query 116: 0.7619 At 10 docs: 0.5900 Comparison to median by topic Query 117: 0.3201 At 15 docs: 0.5413 +1.0 Query 118: 0.4033 At 20 docs: 0.5150 Query 119: 0.8449 At 30 docs: 0.4687 Query 120: 0.3932 At 100 docs: 0.2944 Query 121: 0.6746 At 200 docs: 0.1988 Query 122: 0.1190 At 500 docs: 0.0955 Query 123: 0.8722 At 1000 docs: 0.0512 +0.5 Query 124: 0.5319 Query 125: 0.5956 R-Precision (prec. after R Query 126: 0.0769 (=num_rel) docs. retrieved): Query 127: 0.6990 Exact: 0.4713 Query 128: 0.4282 0.0 Query 129: 0.3942 Query 130: 0.7941 Query 131: 0.7781 Query 132: 0.8453 Query 133: 0.0098 Difference (average precision) Query 134: 0.5226 −0.5 Query 135: 0.6440 Query 136: 0.6845 Query 137: 0.3312 Query 138: 0.2075 Query 139: 0.7723 −1.0 Query 140: 0.2025 91 100 110 120 130 140 Topic number University of Alicante, Monolingual Spanish, Auto, TD Run irn1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.6346 Total number of documents Query 92: 0.5571 over all queries Query 93: 0.2766 Retrieved: 50000 0.8 Query 94: 0.8835 Relevant: 2854 Query 95: 0.2619 Rel_ret: 2571 Query 96: 0.4153 Query 97: 0.7800 Interpolated Recall - 0.6 Query 98: 0.3341 Precision Averages: Query 99: 0.0879 at 0.00 0.8891 Query 100: 0.5623 at 0.10 0.7720 Precision Query 101: 0.4686 at 0.20 0.7038 Query 102: 0.8183 at 0.30 0.6078 0.4 Query 103: 0.6046 at 0.40 0.5447 Query 104: 0.1725 at 0.50 0.4688 Query 105: 0.6807 at 0.60 0.4148 Query 106: 0.1904 at 0.70 0.3568 0.2 Query 107: 0.0693 at 0.80 0.2799 Query 108: 0.6674 at 0.90 0.1971 Query 109: 0.3314 at 1.00 0.1018 Query 110: 0.6222 0.0 Query 111: 0.0144 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6341 for all rel. documents Query 113: 0.4552 0.4684 Recall Query 114: 0.1477 Precision: Query 115: 0.3210 At 5 docs: 0.6800 Run irn1 Query 116: 0.8571 At 10 docs: 0.5820 Comparison to median by topic Query 117: 0.2170 At 15 docs: 0.5453 +1.0 Query 118: 0.2098 At 20 docs: 0.5140 Query 119: 0.8107 At 30 docs: 0.4620 Query 120: 0.3062 At 100 docs: 0.2752 Query 121: 0.7361 At 200 docs: 0.1837 Query 122: 0.0719 At 500 docs: 0.0932 Query 123: 0.9234 At 1000 docs: 0.0514 +0.5 Query 124: 0.4900 Query 125: 0.5743 R-Precision (prec. after R Query 126: 0.0861 (=num_rel) docs. retrieved): Query 127: 0.6989 Exact: 0.4637 Query 128: 0.2761 0.0 Query 129: 0.3599 Query 130: 0.6655 Query 131: 0.6768 Query 132: 0.9116 Query 133: 0.0601 Difference (average precision) Query 134: 0.8269 −0.5 Query 135: 0.5638 Query 136: 0.7595 Query 137: 0.2245 Query 138: 0.3964 Query 139: 0.4784 −1.0 Query 140: 0.2500 91 100 110 120 130 140 Topic number University of Alicante, Monolingual Spanish, Auto, TDN Run irn2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.6219 Total number of documents Query 92: 0.5513 over all queries Query 93: 0.3259 Retrieved: 50000 0.8 Query 94: 0.8818 Relevant: 2854 Query 95: 0.4110 Rel_ret: 2644 Query 96: 0.3643 Query 97: 0.8149 Interpolated Recall - 0.6 Query 98: 0.4115 Precision Averages: Query 99: 0.1796 at 0.00 0.9240 Query 100: 0.5640 at 0.10 0.8209 Precision Query 101: 0.5238 at 0.20 0.7539 Query 102: 0.7339 at 0.30 0.6496 0.4 Query 103: 0.6513 at 0.40 0.5861 Query 104: 0.1834 at 0.50 0.5106 Query 105: 0.6492 at 0.60 0.4454 Query 106: 0.1925 at 0.70 0.3986 0.2 Query 107: 0.0803 at 0.80 0.3064 Query 108: 0.6180 at 0.90 0.2206 Query 109: 0.3634 at 1.00 0.0977 Query 110: 0.5555 0.0 Query 111: 0.0341 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5411 for all rel. documents Query 113: 0.5018 0.5067 Recall Query 114: 0.8852 Precision: Query 115: 0.4233 At 5 docs: 0.7200 Run irn2 Query 116: 0.5918 At 10 docs: 0.6380 Comparison to median by topic Query 117: 0.2051 At 15 docs: 0.5933 +1.0 Query 118: 0.2096 At 20 docs: 0.5600 Query 119: 0.8050 At 30 docs: 0.4813 Query 120: 0.5226 At 100 docs: 0.2880 Query 121: 0.7384 At 200 docs: 0.1898 Query 122: 0.0500 At 500 docs: 0.0958 Query 123: 0.8873 At 1000 docs: 0.0529 +0.5 Query 124: 0.4520 Query 125: 0.5729 R-Precision (prec. after R Query 126: 0.6482 (=num_rel) docs. retrieved): Query 127: 0.6797 Exact: 0.4958 Query 128: 0.2171 0.0 Query 129: 0.3015 Query 130: 0.6716 Query 131: 0.5983 Query 132: 0.7385 Query 133: 0.4367 Difference (average precision) Query 134: 0.8214 −0.5 Query 135: 0.5184 Query 136: 0.7640 Query 137: 0.1668 Query 138: 0.8753 Query 139: 0.4020 −1.0 Query 140: 0.3983 91 100 110 120 130 140 Topic number University of Alicante, Monolingual Spanish, Auto, TD Run irn3 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.6650 Total number of documents Query 92: 0.5516 over all queries Query 93: 0.4493 Retrieved: 50000 0.8 Query 94: 0.9127 Relevant: 2854 Query 95: 0.4365 Rel_ret: 2669 Query 96: 0.4428 Query 97: 0.7805 Interpolated Recall - 0.6 Query 98: 0.3572 Precision Averages: Query 99: 0.1752 at 0.00 0.8549 Query 100: 0.5371 at 0.10 0.7597 Precision Query 101: 0.5476 at 0.20 0.7091 Query 102: 0.7987 at 0.30 0.6391 0.4 Query 103: 0.6184 at 0.40 0.5891 Query 104: 0.1969 at 0.50 0.5145 Query 105: 0.7754 at 0.60 0.4592 Query 106: 0.1514 at 0.70 0.4126 0.2 Query 107: 0.0453 at 0.80 0.3119 Query 108: 0.6209 at 0.90 0.2342 Query 109: 0.4063 at 1.00 0.1181 Query 110: 0.5574 0.0 Query 111: 0.0147 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6587 for all rel. documents Query 113: 0.5209 0.4980 Recall Query 114: 0.1225 Precision: Query 115: 0.3217 At 5 docs: 0.6920 Run irn3 Query 116: 0.6903 At 10 docs: 0.5920 Comparison to median by topic Query 117: 0.1853 At 15 docs: 0.5560 +1.0 Query 118: 0.1308 At 20 docs: 0.5190 Query 119: 0.8230 At 30 docs: 0.4667 Query 120: 0.1316 At 100 docs: 0.2944 Query 121: 0.7222 At 200 docs: 0.2018 Query 122: 0.0642 At 500 docs: 0.0999 Query 123: 0.9246 At 1000 docs: 0.0534 +0.5 Query 124: 0.5687 Query 125: 0.6917 R-Precision (prec. after R Query 126: 0.2431 (=num_rel) docs. retrieved): Query 127: 0.7184 Exact: 0.4771 Query 128: 0.2024 0.0 Query 129: 0.4354 Query 130: 0.6938 Query 131: 0.7141 Query 132: 0.8534 Query 133: 0.1310 Difference (average precision) Query 134: 0.8500 −0.5 Query 135: 0.8292 Query 136: 0.7903 Query 137: 0.2667 Query 138: 0.6115 Query 139: 0.5930 −1.0 Query 140: 0.3700 91 100 110 120 130 140 Topic number University of Alicante, Monolingual Spanish, Auto, TDN Run irn4 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.6344 Total number of documents Query 92: 0.5588 over all queries Query 93: 0.3317 Retrieved: 50000 0.8 Query 94: 0.8837 Relevant: 2854 Query 95: 0.3155 Rel_ret: 2621 Query 96: 0.4030 Query 97: 0.7846 Interpolated Recall - 0.6 Query 98: 0.3918 Precision Averages: Query 99: 0.1683 at 0.00 0.9261 Query 100: 0.5432 at 0.10 0.8022 Precision Query 101: 0.5784 at 0.20 0.7485 Query 102: 0.8121 at 0.30 0.6458 0.4 Query 103: 0.6099 at 0.40 0.5793 Query 104: 0.1780 at 0.50 0.5050 Query 105: 0.7353 at 0.60 0.4406 Query 106: 0.1993 at 0.70 0.3888 0.2 Query 107: 0.0835 at 0.80 0.2982 Query 108: 0.6493 at 0.90 0.2230 Query 109: 0.4008 at 1.00 0.1006 Query 110: 0.5310 0.0 Query 111: 0.0236 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5524 for all rel. documents Query 113: 0.6208 0.4976 Recall Query 114: 0.4587 Precision: Query 115: 0.4010 At 5 docs: 0.7120 Run irn4 Query 116: 0.7523 At 10 docs: 0.6120 Comparison to median by topic Query 117: 0.2319 At 15 docs: 0.5640 +1.0 Query 118: 0.2078 At 20 docs: 0.5380 Query 119: 0.8022 At 30 docs: 0.4867 Query 120: 0.3225 At 100 docs: 0.2890 Query 121: 0.7433 At 200 docs: 0.1936 Query 122: 0.0909 At 500 docs: 0.0964 Query 123: 0.9109 At 1000 docs: 0.0524 +0.5 Query 124: 0.4833 Query 125: 0.5897 R-Precision (prec. after R Query 126: 0.2237 (=num_rel) docs. retrieved): Query 127: 0.7020 Exact: 0.4881 Query 128: 0.3109 0.0 Query 129: 0.3652 Query 130: 0.7049 Query 131: 0.6667 Query 132: 0.8771 Query 133: 0.2226 Difference (average precision) Query 134: 0.8269 −0.5 Query 135: 0.5626 Query 136: 0.7746 Query 137: 0.2450 Query 138: 0.6202 Query 139: 0.4674 −1.0 Query 140: 0.3290 91 100 110 120 130 140 Topic number Océ, Monolingual Spanish, Auto, TD Run oce02monES Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.5800 Total number of documents Query 92: 0.4424 over all queries Query 93: 0.3006 Retrieved: 50000 0.8 Query 94: 0.7177 Relevant: 2854 Query 95: 0.2846 Rel_ret: 2528 Query 96: 0.4526 Query 97: 0.8097 Interpolated Recall - 0.6 Query 98: 0.3803 Precision Averages: Query 99: 0.3507 at 0.00 0.8745 Query 100: 0.7999 at 0.10 0.7531 Precision Query 101: 0.3568 at 0.20 0.6728 Query 102: 0.6609 at 0.30 0.5956 0.4 Query 103: 0.6295 at 0.40 0.5422 Query 104: 0.2322 at 0.50 0.4749 Query 105: 0.6238 at 0.60 0.4031 Query 106: 0.2028 at 0.70 0.3476 0.2 Query 107: 0.0688 at 0.80 0.2721 Query 108: 0.6518 at 0.90 0.1813 Query 109: 0.1675 at 1.00 0.0844 Query 110: 0.7383 0.0 Query 111: 0.0094 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5381 for all rel. documents Query 113: 0.6015 0.4557 Recall Query 114: 0.0452 Precision: Query 115: 0.1810 At 5 docs: 0.6520 Run oce02monES Query 116: 0.6090 At 10 docs: 0.5760 Comparison to median by topic Query 117: 0.2650 At 15 docs: 0.5400 +1.0 Query 118: 0.2923 At 20 docs: 0.5050 Query 119: 0.7544 At 30 docs: 0.4367 Query 120: 0.4877 At 100 docs: 0.2760 Query 121: 0.6880 At 200 docs: 0.1905 Query 122: 0.0894 At 500 docs: 0.0936 Query 123: 0.8856 At 1000 docs: 0.0506 +0.5 Query 124: 0.3441 Query 125: 0.5502 R-Precision (prec. after R Query 126: 0.0014 (=num_rel) docs. retrieved): Query 127: 0.7628 Exact: 0.4508 Query 128: 0.4530 0.0 Query 129: 0.2643 Query 130: 0.7605 Query 131: 0.7027 Query 132: 0.6374 Query 133: 0.0085 Difference (average precision) Query 134: 0.7784 −0.5 Query 135: 0.5649 Query 136: 0.6152 Query 137: 0.1769 Query 138: 0.3402 Query 139: 0.7432 −1.0 Query 140: 0.1813 91 100 110 120 130 140 Topic number Océ, Monolingual Spanish, Auto, T Run oce02monESto Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.4120 Total number of documents Query 92: 0.3637 over all queries Query 93: 0.2355 Retrieved: 45024 0.8 Query 94: 0.8637 Relevant: 2854 Query 95: 0.0705 Rel_ret: 2385 Query 96: 0.2175 Query 97: 0.8510 Interpolated Recall - 0.6 Query 98: 0.0067 Precision Averages: Query 99: 0.2598 at 0.00 0.7667 Query 100: 0.4879 at 0.10 0.6452 Precision Query 101: 0.4845 at 0.20 0.5770 Query 102: 0.7732 at 0.30 0.5032 0.4 Query 103: 0.5113 at 0.40 0.4472 Query 104: 0.0462 at 0.50 0.3942 Query 105: 0.7370 at 0.60 0.3495 Query 106: 0.1353 at 0.70 0.2951 0.2 Query 107: 0.1376 at 0.80 0.2138 Query 108: 0.6715 at 0.90 0.1486 Query 109: 0.2983 at 1.00 0.0595 Query 110: 0.3316 0.0 Query 111: 0.0058 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4279 for all rel. documents Query 113: 0.6016 0.3866 Recall Query 114: 0.0299 Precision: Query 115: 0.3247 At 5 docs: 0.5680 Run oce02monESto Query 116: 0.7143 At 10 docs: 0.5100 Comparison to median by topic Query 117: 0.0711 At 15 docs: 0.4520 +1.0 Query 118: 0.3890 At 20 docs: 0.4190 Query 119: 0.2278 At 30 docs: 0.3647 Query 120: 0.0636 At 100 docs: 0.2428 Query 121: 0.6241 At 200 docs: 0.1694 Query 122: 0.1093 At 500 docs: 0.0858 Query 123: 0.6897 At 1000 docs: 0.0477 +0.5 Query 124: 0.3824 Query 125: 0.5349 R-Precision (prec. after R Query 126: 0.0069 (=num_rel) docs. retrieved): Query 127: 0.5874 Exact: 0.3910 Query 128: 0.5305 0.0 Query 129: 0.1185 Query 130: 0.7880 Query 131: 0.7353 Query 132: 0.6992 Query 133: 0.0025 Difference (average precision) Query 134: 0.2602 −0.5 Query 135: 0.6573 Query 136: 0.6370 Query 137: 0.1667 Query 138: 0.0479 Query 139: 0.7910 −1.0 Query 140: 0.2128 91 100 110 120 130 140 Topic number City University, Monolingual Spanish, Auto, T Run plsp02t Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.4007 Total number of documents Query 92: 0.3723 over all queries Query 93: 0.0000 Retrieved: 50000 0.8 Query 94: 0.8589 Relevant: 2854 Query 95: 0.0305 Rel_ret: 1761 Query 96: 0.0576 Query 97: 0.0068 Interpolated Recall - Query 98: 0.0000 Precision Averages: 0.6 Query 99: 0.2129 at 0.00 0.4553 Query 100: 0.2693 at 0.10 0.3755 Query 101: 0.3001 at 0.20 0.3170 Precision Query 102: 0.6932 at 0.30 0.2787 0.4 Query 103: 0.1261 at 0.40 0.2574 Query 104: 0.0064 at 0.50 0.2311 Query 105: 0.0306 at 0.60 0.1917 Query 106: 0.0050 at 0.70 0.1704 0.2 Query 107: 0.0000 at 0.80 0.1397 Query 108: 0.6634 at 0.90 0.0724 Query 109: 0.0000 at 1.00 0.0267 Query 110: 0.3661 0.0 Query 111: 0.0000 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.2221 for all rel. documents Query 113: 0.5598 0.2173 Recall Query 114: 0.0188 Precision: Query 115: 0.1139 At 5 docs: 0.3200 Run plsp02t Query 116: 0.8571 At 10 docs: 0.2940 Query 117: 0.0353 At 15 docs: 0.2680 Comparison to median by topic Query 118: 0.0435 At 20 docs: 0.2540 +1.0 Query 119: 0.6807 At 30 docs: 0.2320 Query 120: 0.0661 At 100 docs: 0.1578 Query 121: 0.0000 At 200 docs: 0.1129 Query 122: 0.0151 At 500 docs: 0.0605 Query 123: 0.6616 At 1000 docs: 0.0352 +0.5 Query 124: 0.0266 Query 125: 0.1874 R-Precision (prec. after R Query 126: 0.0170 (=num_rel) docs. retrieved): Query 127: 0.5582 Exact: 0.2252 Query 128: 0.0000 0.0 Query 129: 0.0000 Query 130: 0.4863 Query 131: 0.0020 Query 132: 0.0000 Query 133: 0.0479 Difference (average precision) Query 134: 0.5395 −0.5 Query 135: 0.3231 Query 136: 0.1303 Query 137: 0.0832 Query 138: 0.0037 Query 139: 0.6833 −1.0 Query 140: 0.1003 91 100 110 120 130 140 Topic number City University, Monolingual Spanish, Auto, TD Run plsp02td Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.4167 Total number of documents Query 92: 0.4757 over all queries Query 93: 0.0000 Retrieved: 50000 0.8 Query 94: 0.6381 Relevant: 2854 Query 95: 0.0314 Rel_ret: 1952 Query 96: 0.2732 Query 97: 0.0003 Interpolated Recall - Query 98: 0.0000 Precision Averages: 0.6 Query 99: 0.0461 at 0.00 0.5612 Query 100: 0.4825 at 0.10 0.4570 Query 101: 0.2686 at 0.20 0.3854 Precision Query 102: 0.6628 at 0.30 0.3380 0.4 Query 103: 0.3387 at 0.40 0.3069 Query 104: 0.0481 at 0.50 0.2766 Query 105: 0.0008 at 0.60 0.2254 Query 106: 0.1037 at 0.70 0.1956 0.2 Query 107: 0.0000 at 0.80 0.1466 Query 108: 0.6527 at 0.90 0.0711 Query 109: 0.0005 at 1.00 0.0248 Query 110: 0.8571 0.0 Query 111: 0.0000 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4859 for all rel. documents Query 113: 0.4500 0.2552 Recall Query 114: 0.0018 Precision: Query 115: 0.1427 At 5 docs: 0.3760 Run plsp02td Query 116: 0.4404 At 10 docs: 0.3540 Query 117: 0.1198 At 15 docs: 0.3400 Comparison to median by topic Query 118: 0.0118 At 20 docs: 0.3130 +1.0 Query 119: 0.5163 At 30 docs: 0.2820 Query 120: 0.3429 At 100 docs: 0.1818 Query 121: 0.4160 At 200 docs: 0.1280 Query 122: 0.0132 At 500 docs: 0.0691 Query 123: 0.8312 At 1000 docs: 0.0390 +0.5 Query 124: 0.0837 Query 125: 0.2209 R-Precision (prec. after R Query 126: 0.0030 (=num_rel) docs. retrieved): Query 127: 0.7099 Exact: 0.2791 Query 128: 0.0036 0.0 Query 129: 0.0963 Query 130: 0.5964 Query 131: 0.0256 Query 132: 0.0005 Query 133: 0.0085 Difference (average precision) Query 134: 0.4667 −0.5 Query 135: 0.3346 Query 136: 0.0556 Query 137: 0.0377 Query 138: 0.4222 Query 139: 0.5389 −1.0 Query 140: 0.0853 91 100 110 120 130 140 Topic number IRIT, Toulouse, Monolingual Spanish, Auto, TD Run RunMonoSp Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1181 Total number of documents Query 92: 0.4917 over all queries Query 93: 0.2285 Retrieved: 50000 0.8 Query 94: 0.7668 Relevant: 2854 Query 95: 0.0770 Rel_ret: 2060 Query 96: 0.0946 Query 97: 0.7439 Interpolated Recall - Query 98: 0.0214 Precision Averages: 0.6 Query 99: 0.0360 at 0.00 0.6673 Query 100: 0.7389 at 0.10 0.5699 Query 101: 0.4689 at 0.20 0.5170 Precision Query 102: 0.7466 at 0.30 0.4120 0.4 Query 103: 0.4059 at 0.40 0.3731 Query 104: 0.0492 at 0.50 0.3383 Query 105: 0.7652 at 0.60 0.2802 Query 106: 0.0306 at 0.70 0.2461 0.2 Query 107: 0.0010 at 0.80 0.1908 Query 108: 0.3369 at 0.90 0.1428 Query 109: 0.0446 at 1.00 0.0575 Query 110: 0.2926 0.0 Query 111: 0.0025 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4675 for all rel. documents Query 113: 0.7396 0.3305 Recall Query 114: 0.0068 Precision: Query 115: 0.2057 At 5 docs: 0.4360 Run RunMonoSp Query 116: 0.7283 At 10 docs: 0.4100 Query 117: 0.0494 At 15 docs: 0.3907 Comparison to median by topic Query 118: 0.2148 At 20 docs: 0.3730 +1.0 Query 119: 0.1189 At 30 docs: 0.3380 Query 120: 0.1053 At 100 docs: 0.2092 Query 121: 0.5812 At 200 docs: 0.1424 Query 122: 0.0026 At 500 docs: 0.0736 Query 123: 0.8724 At 1000 docs: 0.0412 +0.5 Query 124: 0.2027 Query 125: 0.3434 R-Precision (prec. after R Query 126: 0.0117 (=num_rel) docs. retrieved): Query 127: 0.6237 Exact: 0.3363 Query 128: 0.2563 0.0 Query 129: 0.1839 Query 130: 0.7187 Query 131: 0.5989 Query 132: 0.7346 Query 133: 0.0265 Difference (average precision) Query 134: 0.4232 −0.5 Query 135: 0.3788 Query 136: 0.6001 Query 137: 0.2155 Query 138: 0.0441 Query 139: 0.6080 −1.0 Query 140: 0.0000 91 100 110 120 130 140 Topic number TLR Research, Monolingual Spanish, Auto, TD Run tlres Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.5768 Total number of documents Query 92: 0.4649 over all queries Query 93: 0.3282 Retrieved: 50000 0.8 Query 94: 0.7189 Relevant: 2854 Query 95: 0.2278 Rel_ret: 2559 Query 96: 0.4275 Query 97: 0.8116 Interpolated Recall - 0.6 Query 98: 0.5109 Precision Averages: Query 99: 0.4889 at 0.00 0.8606 Query 100: 0.7422 at 0.10 0.7773 Precision Query 101: 0.5505 at 0.20 0.7113 Query 102: 0.7938 at 0.30 0.6271 0.4 Query 103: 0.5160 at 0.40 0.5853 Query 104: 0.2291 at 0.50 0.5274 Query 105: 0.8004 at 0.60 0.4774 Query 106: 0.1689 at 0.70 0.4116 0.2 Query 107: 0.0607 at 0.80 0.3391 Query 108: 0.6145 at 0.90 0.2249 Query 109: 0.2729 at 1.00 0.1242 Query 110: 0.7538 0.0 Query 111: 0.0239 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6100 for all rel. documents Query 113: 0.5147 0.4993 Recall Query 114: 0.1151 Precision: Query 115: 0.3155 At 5 docs: 0.6600 Run tlres Query 116: 0.7571 At 10 docs: 0.5960 Comparison to median by topic Query 117: 0.3904 At 15 docs: 0.5507 +1.0 Query 118: 0.7042 At 20 docs: 0.5130 Query 119: 0.8463 At 30 docs: 0.4593 Query 120: 0.4314 At 100 docs: 0.2864 Query 121: 0.5899 At 200 docs: 0.1966 Query 122: 0.0524 At 500 docs: 0.0968 Query 123: 0.8772 At 1000 docs: 0.0512 +0.5 Query 124: 0.7525 Query 125: 0.6271 R-Precision (prec. after R Query 126: 0.0779 (=num_rel) docs. retrieved): Query 127: 0.6616 Exact: 0.4816 Query 128: 0.4292 0.0 Query 129: 0.3396 Query 130: 0.7994 Query 131: 0.7318 Query 132: 0.8973 Query 133: 0.0083 Difference (average precision) Query 134: 0.6319 −0.5 Query 135: 0.6092 Query 136: 0.5627 Query 137: 0.3934 Query 138: 0.1634 Query 139: 0.8077 −1.0 Query 140: 0.1861 91 100 110 120 130 140 Topic number University of Amsterdam, Monolingual Spanish, Auto, TD Run UAmsC02SpSpNGiSt Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1239 Total number of documents Query 92: 0.5088 over all queries Query 93: 0.2510 Retrieved: 50000 0.8 Query 94: 0.9127 Relevant: 2854 Query 95: 0.2236 Rel_ret: 2461 Query 96: 0.3701 Query 97: 0.8474 Interpolated Recall - Query 98: 0.4319 Precision Averages: 0.6 Query 99: 0.2165 at 0.00 0.8393 Query 100: 0.5513 at 0.10 0.7520 Query 101: 0.3947 at 0.20 0.7053 Precision Query 102: 0.6510 at 0.30 0.6199 0.4 Query 103: 0.8349 at 0.40 0.5408 Query 104: 0.2554 at 0.50 0.4763 Query 105: 0.6676 at 0.60 0.4162 Query 106: 0.2021 at 0.70 0.3689 0.2 Query 107: 0.1346 at 0.80 0.2917 Query 108: 0.6581 at 0.90 0.2065 Query 109: 0.5521 at 1.00 0.1114 Query 110: 0.6066 0.0 Query 111: 0.0187 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6249 for all rel. documents Query 113: 0.6533 0.4734 Recall Query 114: 0.0948 Precision: Query 115: 0.4166 At 5 docs: 0.6440 Run UAmsC02SpSpNGiSt Query 116: 0.9201 At 10 docs: 0.5480 Query 117: 0.2829 At 15 docs: 0.5120 Comparison to median by topic Query 118: 0.2469 At 20 docs: 0.4720 +1.0 Query 119: 0.8567 At 30 docs: 0.4207 Query 120: 0.0906 At 100 docs: 0.2678 Query 121: 0.6818 At 200 docs: 0.1812 Query 122: 0.0922 At 500 docs: 0.0920 Query 123: 0.8693 At 1000 docs: 0.0492 +0.5 Query 124: 0.3720 Query 125: 0.4901 R-Precision (prec. after R Query 126: 0.0340 (=num_rel) docs. retrieved): Query 127: 0.6298 Exact: 0.4448 Query 128: 0.3357 0.0 Query 129: 0.3986 Query 130: 0.7695 Query 131: 0.7037 Query 132: 0.8374 Query 133: 0.0362 Difference (average precision) Query 134: 0.9500 −0.5 Query 135: 0.4576 Query 136: 0.7871 Query 137: 0.1704 Query 138: 0.7476 Query 139: 0.5680 −1.0 Query 140: 0.1401 91 100 110 120 130 140 Topic number Université de Neuchâtel, Monolingual Spanish, Auto, TD Run UniNEes Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.7244 Total number of documents Query 92: 0.5534 over all queries Query 93: 0.5019 Retrieved: 50000 0.8 Query 94: 0.9094 Relevant: 2854 Query 95: 0.4090 Rel_ret: 2658 Query 96: 0.4030 Query 97: 0.8640 Interpolated Recall - 0.6 Query 98: 0.9111 Precision Averages: Query 99: 0.1228 at 0.00 0.8702 Query 100: 0.7572 at 0.10 0.7892 Precision Query 101: 0.6941 at 0.20 0.7263 Query 102: 0.6469 at 0.30 0.6695 0.4 Query 103: 0.8464 at 0.40 0.6300 Query 104: 0.1545 at 0.50 0.5799 Query 105: 0.8147 at 0.60 0.5232 Query 106: 0.1370 at 0.70 0.4616 0.2 Query 107: 0.1212 at 0.80 0.3966 Query 108: 0.7132 at 0.90 0.2658 Query 109: 0.3754 at 1.00 0.1473 Query 110: 0.7899 0.0 Query 111: 0.0186 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5602 for all rel. documents Query 113: 0.3787 0.5441 Recall Query 114: 0.2145 Precision: Query 115: 0.4411 At 5 docs: 0.6920 Run UniNEes Query 116: 0.8395 At 10 docs: 0.6200 Comparison to median by topic Query 117: 0.3546 At 15 docs: 0.5773 +1.0 Query 118: 0.3711 At 20 docs: 0.5350 Query 119: 0.8938 At 30 docs: 0.4787 Query 120: 0.4453 At 100 docs: 0.3084 Query 121: 0.7631 At 200 docs: 0.2056 Query 122: 0.1171 At 500 docs: 0.0999 Query 123: 0.8988 At 1000 docs: 0.0532 +0.5 Query 124: 0.4289 Query 125: 0.7758 R-Precision (prec. after R Query 126: 0.0616 (=num_rel) docs. retrieved): Query 127: 0.6959 Exact: 0.5021 Query 128: 0.4029 0.0 Query 129: 0.4079 Query 130: 0.8774 Query 131: 0.8009 Query 132: 0.8180 Query 133: 0.0088 Difference (average precision) Query 134: 0.8929 −0.5 Query 135: 0.8640 Query 136: 0.7739 Query 137: 0.3703 Query 138: 0.2891 Query 139: 0.7223 −1.0 Query 140: 0.2662 91 100 110 120 130 140 Topic number Université de Neuchâtel, Monolingual Spanish, Auto, TDN Run UniNEestdn Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.7412 Total number of documents Query 92: 0.5489 over all queries Query 93: 0.5331 Retrieved: 50000 0.8 Query 94: 0.9571 Relevant: 2854 Query 95: 0.3384 Rel_ret: 2666 Query 96: 0.4207 Query 97: 0.8335 Interpolated Recall - 0.6 Query 98: 1.0000 Precision Averages: Query 99: 0.2748 at 0.00 0.9404 Query 100: 0.8086 at 0.10 0.8576 Precision Query 101: 0.6716 at 0.20 0.7990 Query 102: 0.6686 at 0.30 0.7368 0.4 Query 103: 0.8487 at 0.40 0.6872 Query 104: 0.2493 at 0.50 0.6465 Query 105: 0.7767 at 0.60 0.5878 Query 106: 0.1502 at 0.70 0.5321 0.2 Query 107: 0.0495 at 0.80 0.4297 Query 108: 0.6996 at 0.90 0.3116 Query 109: 0.3921 at 1.00 0.1695 Query 110: 0.7113 0.0 Query 111: 0.0287 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6473 for all rel. documents Query 113: 0.6891 0.6051 Recall Query 114: 0.9636 Precision: Query 115: 0.4981 At 5 docs: 0.7760 Run UniNEestdn Query 116: 0.8323 At 10 docs: 0.6920 Comparison to median by topic Query 117: 0.3666 At 15 docs: 0.6413 +1.0 Query 118: 0.6175 At 20 docs: 0.5960 Query 119: 0.8858 At 30 docs: 0.5307 Query 120: 0.6414 At 100 docs: 0.3218 Query 121: 0.7481 At 200 docs: 0.2128 Query 122: 0.1331 At 500 docs: 0.1002 Query 123: 0.9036 At 1000 docs: 0.0533 +0.5 Query 124: 0.4270 Query 125: 0.7211 R-Precision (prec. after R Query 126: 0.8063 (=num_rel) docs. retrieved): Query 127: 0.7322 Exact: 0.5559 Query 128: 0.1325 0.0 Query 129: 0.3609 Query 130: 0.8264 Query 131: 0.7557 Query 132: 0.7667 Query 133: 0.3343 Difference (average precision) Query 134: 0.8929 −0.5 Query 135: 0.8409 Query 136: 0.8195 Query 137: 0.3010 Query 138: 0.8048 Query 139: 0.6913 −1.0 Query 140: 0.4134 91 100 110 120 130 140 Topic number University of Salamanca, Monolingual Spanish, Auto, TDN Run usalFNTDN Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.5873 Total number of documents Query 92: 0.4898 over all queries Query 93: 0.2666 Retrieved: 50000 0.8 Query 94: 0.8537 Relevant: 2854 Query 95: 0.1680 Rel_ret: 2544 Query 96: 0.1970 Query 97: 0.6482 Interpolated Recall - 0.6 Query 98: 0.4972 Precision Averages: Query 99: 0.0971 at 0.00 0.7743 Query 100: 0.3904 at 0.10 0.6570 Precision Query 101: 0.1744 at 0.20 0.5850 Query 102: 0.6756 at 0.30 0.5139 0.4 Query 103: 0.5570 at 0.40 0.4798 Query 104: 0.0254 at 0.50 0.4134 Query 105: 0.6701 at 0.60 0.3615 Query 106: 0.2465 at 0.70 0.3259 0.2 Query 107: 0.0380 at 0.80 0.2438 Query 108: 0.4694 at 0.90 0.1824 Query 109: 0.2680 at 1.00 0.0789 Query 110: 0.4773 0.0 Query 111: 0.0036 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4956 for all rel. documents Query 113: 0.6211 0.4051 Recall Query 114: 0.6113 Precision: Query 115: 0.3092 At 5 docs: 0.5360 Run usalFNTDN Query 116: 0.6835 At 10 docs: 0.4900 Comparison to median by topic Query 117: 0.1824 At 15 docs: 0.4573 +1.0 Query 118: 0.2375 At 20 docs: 0.4310 Query 119: 0.7891 At 30 docs: 0.3967 Query 120: 0.1045 At 100 docs: 0.2504 Query 121: 0.6387 At 200 docs: 0.1697 Query 122: 0.0332 At 500 docs: 0.0900 Query 123: 0.8381 At 1000 docs: 0.0509 +0.5 Query 124: 0.3012 Query 125: 0.1560 R-Precision (prec. after R Query 126: 0.4355 (=num_rel) docs. retrieved): Query 127: 0.6935 Exact: 0.4076 Query 128: 0.0797 0.0 Query 129: 0.1914 Query 130: 0.6236 Query 131: 0.4673 Query 132: 0.7390 Query 133: 0.1515 Difference (average precision) Query 134: 0.4182 −0.5 Query 135: 0.2237 Query 136: 0.7543 Query 137: 0.1057 Query 138: 0.9405 Query 139: 0.3416 −1.0 Query 140: 0.2864 91 100 110 120 130 140 Topic number University of Salamanca, Monolingual Spanish, Auto, T Run usalNAT Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.3882 Total number of documents Query 92: 0.4314 over all queries Query 93: 0.5185 Retrieved: 50000 0.8 Query 94: 0.8591 Relevant: 2854 Query 95: 0.0412 Rel_ret: 2248 Query 96: 0.1627 Query 97: 0.6117 Interpolated Recall - 0.6 Query 98: 0.5954 Precision Averages: Query 99: 0.1141 at 0.00 0.6121 Query 100: 0.3369 at 0.10 0.4652 Precision Query 101: 0.0924 at 0.20 0.4245 Query 102: 0.6587 at 0.30 0.3615 0.4 Query 103: 0.1487 at 0.40 0.3351 Query 104: 0.0046 at 0.50 0.2976 Query 105: 0.7869 at 0.60 0.2747 Query 106: 0.0599 at 0.70 0.2473 0.2 Query 107: 0.0487 at 0.80 0.1802 Query 108: 0.5388 at 0.90 0.1278 Query 109: 0.1589 at 1.00 0.0612 Query 110: 0.0482 0.0 Query 111: 0.0078 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4889 for all rel. documents Query 113: 0.6043 0.2879 Recall Query 114: 0.1382 Precision: Query 115: 0.1520 At 5 docs: 0.4120 Run usalNAT Query 116: 0.6788 At 10 docs: 0.3540 Comparison to median by topic Query 117: 0.0630 At 15 docs: 0.3307 +1.0 Query 118: 0.2248 At 20 docs: 0.3120 Query 119: 0.1067 At 30 docs: 0.2913 Query 120: 0.0728 At 100 docs: 0.1884 Query 121: 0.6227 At 200 docs: 0.1386 Query 122: 0.0053 At 500 docs: 0.0767 Query 123: 0.3574 At 1000 docs: 0.0450 +0.5 Query 124: 0.0846 Query 125: 0.2390 R-Precision (prec. after R Query 126: 0.0083 (=num_rel) docs. retrieved): Query 127: 0.6211 Exact: 0.2952 Query 128: 0.0766 0.0 Query 129: 0.0369 Query 130: 0.5895 Query 131: 0.4174 Query 132: 0.7312 Query 133: 0.0004 Difference (average precision) Query 134: 0.0628 −0.5 Query 135: 0.1358 Query 136: 0.7371 Query 137: 0.0656 Query 138: 0.0308 Query 139: 0.3314 −1.0 Query 140: 0.0969 91 100 110 120 130 140 Topic number University of Salamanca, Monolingual Spanish, Auto, TDN Run usalNNTDN Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.5376 Total number of documents Query 92: 0.4742 over all queries Query 93: 0.2926 Retrieved: 50000 0.8 Query 94: 0.8438 Relevant: 2854 Query 95: 0.2396 Rel_ret: 2525 Query 96: 0.2014 Query 97: 0.6685 Interpolated Recall - 0.6 Query 98: 0.4505 Precision Averages: Query 99: 0.0797 at 0.00 0.7345 Query 100: 0.3723 at 0.10 0.6462 Precision Query 101: 0.2291 at 0.20 0.5753 Query 102: 0.6742 at 0.30 0.5052 0.4 Query 103: 0.5899 at 0.40 0.4760 Query 104: 0.0156 at 0.50 0.4152 Query 105: 0.6038 at 0.60 0.3565 Query 106: 0.1553 at 0.70 0.3083 0.2 Query 107: 0.0399 at 0.80 0.2282 Query 108: 0.4566 at 0.90 0.1688 Query 109: 0.2172 at 1.00 0.0758 Query 110: 0.5696 0.0 Query 111: 0.0036 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4608 for all rel. documents Query 113: 0.6362 0.3908 Recall Query 114: 0.6493 Precision: Query 115: 0.2611 At 5 docs: 0.5200 Run usalNNTDN Query 116: 0.6945 At 10 docs: 0.4840 Comparison to median by topic Query 117: 0.1322 At 15 docs: 0.4613 +1.0 Query 118: 0.2278 At 20 docs: 0.4320 Query 119: 0.6592 At 30 docs: 0.3840 Query 120: 0.1025 At 100 docs: 0.2402 Query 121: 0.6453 At 200 docs: 0.1658 Query 122: 0.0073 At 500 docs: 0.0889 Query 123: 0.8458 At 1000 docs: 0.0505 +0.5 Query 124: 0.2957 Query 125: 0.1862 R-Precision (prec. after R Query 126: 0.4347 (=num_rel) docs. retrieved): Query 127: 0.6962 Exact: 0.3844 Query 128: 0.0963 0.0 Query 129: 0.1136 Query 130: 0.6193 Query 131: 0.5646 Query 132: 0.7528 Query 133: 0.0078 Difference (average precision) Query 134: 0.4413 −0.5 Query 135: 0.1303 Query 136: 0.7545 Query 137: 0.0882 Query 138: 0.7913 Query 139: 0.2926 −1.0 Query 140: 0.2389 91 100 110 120 130 140 Topic number University of Salamanca, Monolingual Spanish, Auto, T Run usalNST Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.4106 Total number of documents Query 92: 0.4958 over all queries Query 93: 0.4921 Retrieved: 50000 0.8 Query 94: 0.8699 Relevant: 2854 Query 95: 0.0599 Rel_ret: 2326 Query 96: 0.2062 Query 97: 0.7025 Interpolated Recall - 0.6 Query 98: 0.8097 Precision Averages: Query 99: 0.1356 at 0.00 0.6623 Query 100: 0.3439 at 0.10 0.5206 Precision Query 101: 0.1804 at 0.20 0.4438 Query 102: 0.6897 at 0.30 0.3792 0.4 Query 103: 0.2373 at 0.40 0.3496 Query 104: 0.0068 at 0.50 0.3283 Query 105: 0.7930 at 0.60 0.3009 Query 106: 0.0648 at 0.70 0.2731 0.2 Query 107: 0.0474 at 0.80 0.2087 Query 108: 0.6111 at 0.90 0.1454 Query 109: 0.2091 at 1.00 0.0700 Query 110: 0.0533 0.0 Query 111: 0.0078 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4668 for all rel. documents Query 113: 0.6382 0.3143 Recall Query 114: 0.2001 Precision: Query 115: 0.1678 At 5 docs: 0.4400 Run usalNST Query 116: 0.6235 At 10 docs: 0.3940 Comparison to median by topic Query 117: 0.1098 At 15 docs: 0.3653 +1.0 Query 118: 0.2212 At 20 docs: 0.3460 Query 119: 0.1491 At 30 docs: 0.3100 Query 120: 0.0671 At 100 docs: 0.2044 Query 121: 0.6358 At 200 docs: 0.1494 Query 122: 0.0084 At 500 docs: 0.0801 Query 123: 0.4085 At 1000 docs: 0.0465 +0.5 Query 124: 0.0617 Query 125: 0.3211 R-Precision (prec. after R Query 126: 0.0060 (=num_rel) docs. retrieved): Query 127: 0.6266 Exact: 0.3185 Query 128: 0.0904 0.0 Query 129: 0.0368 Query 130: 0.6110 Query 131: 0.4036 Query 132: 0.7688 Query 133: 0.0007 Difference (average precision) Query 134: 0.0969 −0.5 Query 135: 0.2223 Query 136: 0.7374 Query 137: 0.0806 Query 138: 0.0455 Query 139: 0.3707 −1.0 Query 140: 0.1127 91 100 110 120 130 140 Topic number JHU/APL, Monolingual Finnish, Auto, TD Run aplmofi Average precision Overall statistics Recall−Precision Values (individual queries): (for 30 queries): 1.0 Query 92: 0.4036 Total number of documents Query 94: 0.3912 over all queries Query 95: 0.1597 Retrieved: 30000 0.8 Query 98: 0.7800 Relevant: 502 Query 100: 0.7317 Rel_ret: 483 Query 102: 0.9450 Query 103: 0.1630 Interpolated Recall - Query 105: 0.3820 Precision Averages: 0.6 Query 106: 0.0235 at 0.00 0.5427 Query 107: 0.0119 at 0.10 0.5012 Query 109: 0.1372 at 0.20 0.4375 Precision Query 111: 0.0459 at 0.30 0.4023 0.4 Query 114: 0.0769 at 0.40 0.3740 Query 115: 0.0473 at 0.50 0.3532 Query 116: 0.0652 at 0.60 0.3159 Query 118: 0.6636 at 0.70 0.2691 0.2 Query 119: 0.3126 at 0.80 0.2383 Query 122: 0.0702 at 0.90 0.2145 Query 123: 0.3409 at 1.00 0.1727 Query 124: 0.4156 0.0 Query 125: 0.2265 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 126: 0.0094 for all rel. documents Query 128: 0.0586 0.3280 Recall Query 130: 0.6117 Precision: Query 131: 0.4088 At 5 docs: 0.3333 Run aplmofi Query 132: 1.0000 At 10 docs: 0.2933 Query 137: 0.0074 At 15 docs: 0.2644 Comparison to median by topic Query 138: 0.1055 At 20 docs: 0.2450 +1.0 Query 139: 0.7655 At 30 docs: 0.2311 Query 140: 0.4792 At 100 docs: 0.1160 At 200 docs: 0.0717 At 500 docs: 0.0311 At 1000 docs: 0.0161 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3118 0.0 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Univ. Tampere, Dept. IS, Monolingual Finnish, Auto, TD Run finmo1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 30 queries): 1.0 Query 92: 0.3710 Total number of documents Query 94: 0.3518 over all queries Query 95: 0.0367 Retrieved: 30000 0.8 Query 98: 0.4190 Relevant: 502 Query 100: 0.0003 Rel_ret: 453 Query 102: 0.8121 Query 103: 0.2522 Interpolated Recall - 0.6 Query 105: 0.4834 Precision Averages: Query 106: 0.0121 at 0.00 0.5811 Query 107: 0.0205 at 0.10 0.4772 Precision Query 109: 0.0642 at 0.20 0.4330 Query 111: 0.0276 at 0.30 0.3258 0.4 Query 114: 0.1250 at 0.40 0.2978 Query 115: 0.0738 at 0.50 0.2667 Query 116: 0.0500 at 0.60 0.2271 Query 118: 0.2462 at 0.70 0.1975 0.2 Query 119: 0.2534 at 0.80 0.1614 Query 122: 0.1500 at 0.90 0.1269 Query 123: 0.7500 at 1.00 0.1123 Query 124: 0.2025 0.0 Query 125: 0.2890 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 126: 0.0164 for all rel. documents Query 128: 0.4822 0.2700 Recall Query 130: 0.2539 Precision: Query 131: 0.3301 At 5 docs: 0.2800 Run finmo1 Query 132: 1.0000 At 10 docs: 0.2500 Comparison to median by topic Query 137: 0.0008 At 15 docs: 0.2378 +1.0 Query 138: 0.4112 At 20 docs: 0.2083 Query 139: 0.4652 At 30 docs: 0.1844 Query 140: 0.1500 At 100 docs: 0.0940 At 200 docs: 0.0552 At 500 docs: 0.0269 At 1000 docs: 0.0151 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2556 0.0 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Univ. Tampere, Dept. IS, Monolingual Finnish, Auto, TD Run finmo2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 30 queries): 1.0 Query 92: 0.3710 Total number of documents Query 94: 0.3518 over all queries Query 95: 0.0367 Retrieved: 30000 0.8 Query 98: 0.4510 Relevant: 502 Query 100: 0.0003 Rel_ret: 433 Query 102: 0.0566 Query 103: 0.1227 Interpolated Recall - 0.6 Query 105: 0.4834 Precision Averages: Query 106: 0.0161 at 0.00 0.5749 Query 107: 0.0205 at 0.10 0.4452 Precision Query 109: 0.0642 at 0.20 0.3976 Query 111: 0.0276 at 0.30 0.2837 0.4 Query 114: 0.1250 at 0.40 0.2552 Query 115: 0.0738 at 0.50 0.2321 Query 116: 0.0500 at 0.60 0.1925 Query 118: 0.2462 at 0.70 0.1646 0.2 Query 119: 0.2534 at 0.80 0.1359 Query 122: 0.1500 at 0.90 0.1063 Query 123: 0.7500 at 1.00 0.0945 Query 124: 0.2025 0.0 Query 125: 0.2890 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 126: 0.0164 for all rel. documents Query 128: 0.4822 0.2400 Recall Query 130: 0.2034 Precision: Query 131: 0.3301 At 5 docs: 0.2467 Run finmo2 Query 132: 1.0000 At 10 docs: 0.2167 Comparison to median by topic Query 137: 0.0008 At 15 docs: 0.2022 +1.0 Query 138: 0.4112 At 20 docs: 0.1783 Query 139: 0.4652 At 30 docs: 0.1600 Query 140: 0.1500 At 100 docs: 0.0847 At 200 docs: 0.0505 At 500 docs: 0.0256 At 1000 docs: 0.0144 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2199 0.0 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Hummingbird, Monolingual Finnish, Auto, TD Run humFI02 Average precision Overall statistics Recall−Precision Values (individual queries): (for 30 queries): 1.0 Query 92: 0.2947 Total number of documents Query 94: 0.3310 over all queries Query 95: 0.0834 Retrieved: 30000 0.8 Query 98: 0.8302 Relevant: 502 Query 100: 0.5000 Rel_ret: 477 Query 102: 0.9280 Query 103: 0.2607 Interpolated Recall - 0.6 Query 105: 0.5082 Precision Averages: Query 106: 0.1617 at 0.00 0.7034 Query 107: 0.0327 at 0.10 0.6532 Precision Query 109: 0.3529 at 0.20 0.5703 Query 111: 0.3363 at 0.30 0.5060 0.4 Query 114: 0.2500 at 0.40 0.4724 Query 115: 0.7530 at 0.50 0.4337 Query 116: 0.1130 at 0.60 0.3436 Query 118: 0.3698 at 0.70 0.2867 0.2 Query 119: 0.5759 at 0.80 0.2624 Query 122: 0.5141 at 0.90 0.2206 Query 123: 0.5625 at 1.00 0.1880 Query 124: 0.3325 0.0 Query 125: 0.2996 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 126: 0.3279 for all rel. documents Query 128: 0.3833 0.3956 Recall Query 130: 0.3327 Precision: Query 131: 0.0260 At 5 docs: 0.4267 Run humFI02 Query 132: 1.0000 At 10 docs: 0.3600 Comparison to median by topic Query 137: 0.0029 At 15 docs: 0.3067 +1.0 Query 138: 0.1659 At 20 docs: 0.2800 Query 139: 0.8866 At 30 docs: 0.2389 Query 140: 0.3521 At 100 docs: 0.1210 At 200 docs: 0.0712 At 500 docs: 0.0311 At 1000 docs: 0.0159 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3621 0.0 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Hummingbird, Monolingual Finnish, Auto, TDN Run humFI02n Average precision Overall statistics Recall−Precision Values (individual queries): (for 30 queries): 1.0 Query 92: 0.2848 Total number of documents Query 94: 0.3624 over all queries Query 95: 0.1553 Retrieved: 30000 0.8 Query 98: 0.8394 Relevant: 502 Query 100: 0.4583 Rel_ret: 473 Query 102: 0.9271 Query 103: 0.2514 Interpolated Recall - 0.6 Query 105: 0.6110 Precision Averages: Query 106: 0.0607 at 0.00 0.6993 Query 107: 0.0453 at 0.10 0.6309 Precision Query 109: 0.3435 at 0.20 0.5575 Query 111: 0.5235 at 0.30 0.5178 0.4 Query 114: 0.3333 at 0.40 0.4628 Query 115: 0.5963 at 0.50 0.4222 Query 116: 0.3570 at 0.60 0.3203 Query 118: 0.3153 at 0.70 0.2889 0.2 Query 119: 0.5558 at 0.80 0.2490 Query 122: 0.1134 at 0.90 0.2236 Query 123: 0.6429 at 1.00 0.1725 Query 124: 0.3981 0.0 Query 125: 0.2457 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 126: 0.6424 for all rel. documents Query 128: 0.1257 0.3873 Recall Query 130: 0.3767 Precision: Query 131: 0.0200 At 5 docs: 0.4267 Run humFI02n Query 132: 0.8056 At 10 docs: 0.3567 Comparison to median by topic Query 137: 0.0020 At 15 docs: 0.3356 +1.0 Query 138: 0.2132 At 20 docs: 0.3067 Query 139: 0.5540 At 30 docs: 0.2533 Query 140: 0.4588 At 100 docs: 0.1260 At 200 docs: 0.0717 At 500 docs: 0.0304 At 1000 docs: 0.0158 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3528 0.0 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number City University, Monolingual Finnish, Auto, T Run plfn02t Average precision Overall statistics Recall−Precision Values (individual queries): (for 30 queries): 1.0 Query 92: 0.1894 Total number of documents Query 94: 0.1026 over all queries Query 95: 0.0425 Retrieved: 18699 0.8 Query 98: 0.0289 Relevant: 502 Query 100: 0.3787 Rel_ret: 240 Query 102: 0.4453 Query 103: 0.0004 Interpolated Recall - Query 105: 0.1036 Precision Averages: 0.6 Query 106: 0.0000 at 0.00 0.4066 Query 107: 0.0000 at 0.10 0.2892 Query 109: 0.0001 at 0.20 0.2296 Precision Query 111: 0.0000 at 0.30 0.1595 0.4 Query 114: 0.0204 at 0.40 0.1051 Query 115: 0.0000 at 0.50 0.0831 Query 116: 0.0093 at 0.60 0.0616 Query 118: 0.1942 at 0.70 0.0566 0.2 Query 119: 0.2496 at 0.80 0.0452 Query 122: 0.0072 at 0.90 0.0425 Query 123: 0.2534 at 1.00 0.0407 Query 124: 0.1432 0.0 Query 125: 0.1546 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 126: 0.0045 for all rel. documents Query 128: 0.0007 0.1229 Recall Query 130: 0.1095 Precision: Query 131: 0.0192 At 5 docs: 0.2000 Run plfn02t Query 132: 1.0000 At 10 docs: 0.1433 Query 137: 0.0000 At 15 docs: 0.1222 Comparison to median by topic Query 138: 0.0159 At 20 docs: 0.1000 +1.0 Query 139: 0.1136 At 30 docs: 0.0800 Query 140: 0.0994 At 100 docs: 0.0393 At 200 docs: 0.0288 At 500 docs: 0.0144 At 1000 docs: 0.0080 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.1504 0.0 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number City University, Monolingual Finnish, Auto, TD Run plfn02td Average precision Overall statistics Recall−Precision Values (individual queries): (for 30 queries): 1.0 Query 92: 0.4081 Total number of documents Query 94: 0.0517 over all queries Query 95: 0.0173 Retrieved: 29891 0.8 Query 98: 0.3578 Relevant: 502 Query 100: 0.2737 Rel_ret: 308 Query 102: 0.8167 Query 103: 0.2115 Interpolated Recall - Query 105: 0.1189 Precision Averages: 0.6 Query 106: 0.0014 at 0.00 0.4673 Query 107: 0.0006 at 0.10 0.3732 Query 109: 0.0038 at 0.20 0.2926 Precision Query 111: 0.0002 at 0.30 0.2229 0.4 Query 114: 0.1111 at 0.40 0.1737 Query 115: 0.0000 at 0.50 0.1571 Query 116: 0.0535 at 0.60 0.1049 Query 118: 0.2433 at 0.70 0.0899 0.2 Query 119: 0.1142 at 0.80 0.0765 Query 122: 0.0372 at 0.90 0.0541 Query 123: 0.6429 at 1.00 0.0451 Query 124: 0.2415 0.0 Query 125: 0.0869 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 126: 0.0011 for all rel. documents Query 128: 0.0095 0.1709 Recall Query 130: 0.2309 Precision: Query 131: 0.1171 At 5 docs: 0.2400 Run plfn02td Query 132: 0.4306 At 10 docs: 0.1900 Query 137: 0.0000 At 15 docs: 0.1667 Comparison to median by topic Query 138: 0.0198 At 20 docs: 0.1567 +1.0 Query 139: 0.4486 At 30 docs: 0.1356 Query 140: 0.0764 At 100 docs: 0.0657 At 200 docs: 0.0385 At 500 docs: 0.0185 At 1000 docs: 0.0103 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.1755 0.0 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Univ. of Twente, Monolingual Finnish, Auto, TD Run tnofifi1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 30 queries): 1.0 Query 92: 0.3144 Total number of documents Query 94: 0.5969 over all queries Query 95: 0.1314 Retrieved: 30000 0.8 Query 98: 0.8371 Relevant: 502 Query 100: 0.2215 Rel_ret: 478 Query 102: 0.9158 Query 103: 0.2834 Interpolated Recall - 0.6 Query 105: 0.3352 Precision Averages: Query 106: 0.0559 at 0.00 0.7752 Query 107: 0.0097 at 0.10 0.6874 Precision Query 109: 0.3146 at 0.20 0.5671 Query 111: 0.3464 at 0.30 0.4981 0.4 Query 114: 0.2500 at 0.40 0.4221 Query 115: 0.5534 at 0.50 0.4045 Query 116: 0.0878 at 0.60 0.3387 Query 118: 0.3918 at 0.70 0.3080 0.2 Query 119: 0.7415 at 0.80 0.2712 Query 122: 0.2687 at 0.90 0.2275 Query 123: 0.6250 at 1.00 0.2039 Query 124: 0.3878 0.0 Query 125: 0.2985 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 126: 0.1022 for all rel. documents Query 128: 0.2361 0.4056 Recall Query 130: 0.3633 Precision: Query 131: 0.7092 At 5 docs: 0.4467 Run tnofifi1 Query 132: 1.0000 At 10 docs: 0.3700 Comparison to median by topic Query 137: 0.1667 At 15 docs: 0.3178 +1.0 Query 138: 0.4635 At 20 docs: 0.2850 Query 139: 0.7480 At 30 docs: 0.2456 Query 140: 0.4118 At 100 docs: 0.1253 At 200 docs: 0.0732 At 500 docs: 0.0313 At 1000 docs: 0.0159 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3420 0.0 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number University of Amsterdam, Monolingual Finnish, Auto, TD Run UAmsC02FiFiNGram Average precision Overall statistics Recall−Precision Values (individual queries): (for 30 queries): 1.0 Query 92: 0.2655 Total number of documents Query 94: 0.4747 over all queries Query 95: 0.2018 Retrieved: 30000 0.8 Query 98: 0.6688 Relevant: 502 Query 100: 0.2300 Rel_ret: 437 Query 102: 0.5363 Query 103: 0.2118 Interpolated Recall - 0.6 Query 105: 0.2112 Precision Averages: Query 106: 0.0753 at 0.00 0.5933 Query 107: 0.0097 at 0.10 0.5569 Precision Query 109: 0.3481 at 0.20 0.4836 Query 111: 0.2495 at 0.30 0.4088 0.4 Query 114: 0.1000 at 0.40 0.3712 Query 115: 0.1116 at 0.50 0.3544 Query 116: 0.1333 at 0.60 0.2892 Query 118: 0.5201 at 0.70 0.2057 0.2 Query 119: 0.4535 at 0.80 0.1508 Query 122: 0.2474 at 0.90 0.0779 Query 123: 0.5000 at 1.00 0.0479 Query 124: 0.3382 0.0 Query 125: 0.1405 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 126: 0.0241 for all rel. documents Query 128: 0.0777 0.3034 Recall Query 130: 0.1526 Precision: Query 131: 0.6664 At 5 docs: 0.3667 Run UAmsC02FiFiNGram Query 132: 0.6667 At 10 docs: 0.3033 Comparison to median by topic Query 137: 0.0000 At 15 docs: 0.2756 +1.0 Query 138: 0.1533 At 20 docs: 0.2567 Query 139: 0.8542 At 30 docs: 0.2300 Query 140: 0.4816 At 100 docs: 0.1123 At 200 docs: 0.0642 At 500 docs: 0.0280 At 1000 docs: 0.0146 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3046 0.0 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Université de Neuchâtel, Monolingual Finnish, Auto, TD Run UniNEfi1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 30 queries): 1.0 Query 92: 0.3268 Total number of documents Query 94: 0.4857 over all queries Query 95: 0.1610 Retrieved: 30000 0.8 Query 98: 0.8523 Relevant: 502 Query 100: 0.1311 Rel_ret: 480 Query 102: 0.9413 Query 103: 0.1710 Interpolated Recall - 0.6 Query 105: 0.2633 Precision Averages: Query 106: 0.0726 at 0.00 0.6864 Query 107: 0.0056 at 0.10 0.6301 Precision Query 109: 0.3857 at 0.20 0.5564 Query 111: 0.3877 at 0.30 0.5068 0.4 Query 114: 0.2500 at 0.40 0.4608 Query 115: 0.6073 at 0.50 0.4445 Query 116: 0.0984 at 0.60 0.3796 Query 118: 0.7153 at 0.70 0.3260 0.2 Query 119: 0.8803 at 0.80 0.2846 Query 122: 0.3601 at 0.90 0.2464 Query 123: 0.5667 at 1.00 0.2022 Query 124: 0.3477 0.0 Query 125: 0.2635 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 126: 0.0639 for all rel. documents Query 128: 0.2110 0.4090 Recall Query 130: 0.4236 Precision: Query 131: 0.7633 At 5 docs: 0.4400 Run UniNEfi1 Query 132: 1.0000 At 10 docs: 0.3767 Comparison to median by topic Query 137: 0.1263 At 15 docs: 0.3267 +1.0 Query 138: 0.0419 At 20 docs: 0.2867 Query 139: 0.8785 At 30 docs: 0.2533 Query 140: 0.4871 At 100 docs: 0.1280 At 200 docs: 0.0733 At 500 docs: 0.0306 At 1000 docs: 0.0160 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3735 0.0 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number Université de Neuchâtel, Monolingual Finnish, Auto, TD Run UniNEfi2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 30 queries): 1.0 Query 92: 0.3971 Total number of documents Query 94: 0.5900 over all queries Query 95: 0.1431 Retrieved: 30000 0.8 Query 98: 0.8326 Relevant: 502 Query 100: 0.5993 Rel_ret: 471 Query 102: 0.9522 Query 103: 0.1629 Interpolated Recall - 0.6 Query 105: 0.5317 Precision Averages: Query 106: 0.0522 at 0.00 0.6704 Query 107: 0.0057 at 0.10 0.6007 Precision Query 109: 0.2572 at 0.20 0.5471 Query 111: 0.2778 at 0.30 0.4819 0.4 Query 114: 0.1667 at 0.40 0.4482 Query 115: 0.4027 at 0.50 0.4292 Query 116: 0.1140 at 0.60 0.3889 Query 118: 0.5531 at 0.70 0.3327 0.2 Query 119: 0.7621 at 0.80 0.2609 Query 122: 0.3112 at 0.90 0.2263 Query 123: 0.2976 at 1.00 0.1815 Query 124: 0.4303 0.0 Query 125: 0.3141 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 126: 0.0621 for all rel. documents Query 128: 0.1533 0.3933 Recall Query 130: 0.4978 Precision: Query 131: 0.6383 At 5 docs: 0.4600 Run UniNEfi2 Query 132: 0.9167 At 10 docs: 0.3767 Comparison to median by topic Query 137: 0.0625 At 15 docs: 0.3267 +1.0 Query 138: 0.0404 At 20 docs: 0.2967 Query 139: 0.8580 At 30 docs: 0.2667 Query 140: 0.4156 At 100 docs: 0.1293 At 200 docs: 0.0740 At 500 docs: 0.0311 At 1000 docs: 0.0157 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3875 0.0 Difference (average precision) −0.5 −1.0 91 100 110 120 130 140 Topic number JHU/APL, Monolingual French, Auto, TD Run aplmofr Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0234 Total number of documents Query 92: 0.5255 over all queries Query 93: 0.9667 Retrieved: 50000 0.8 Query 94: 0.7242 Relevant: 1383 Query 95: 0.4640 Rel_ret: 1364 Query 96: 0.4970 Query 97: 0.5318 Interpolated Recall - Query 98: 0.8412 Precision Averages: 0.6 Query 99: 0.1882 at 0.00 0.7175 Query 100: 0.2443 at 0.10 0.6286 Query 101: 0.2531 at 0.20 0.5951 Precision Query 102: 0.7967 at 0.30 0.5564 0.4 Query 103: 0.2934 at 0.40 0.5259 Query 104: 0.1369 at 0.50 0.4921 Query 105: 0.7000 at 0.60 0.4280 Query 106: 0.4295 at 0.70 0.3970 0.2 Query 107: 0.0568 at 0.80 0.3525 Query 108: 0.3912 at 0.90 0.2740 Query 109: 0.0913 at 1.00 0.1897 Query 110: 0.6764 0.0 Query 111: 0.0191 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6371 for all rel. documents Query 113: 0.0310 0.4509 Recall Query 114: 0.6585 Precision: Query 115: 0.2234 At 5 docs: 0.4800 Run aplmofr Query 116: 0.5199 At 10 docs: 0.4440 Query 117: 0.1542 At 15 docs: 0.4000 Comparison to median by topic Query 118: 0.0599 At 20 docs: 0.3740 +1.0 Query 119: 0.9315 At 30 docs: 0.3260 Query 120: 0.0091 At 100 docs: 0.1850 Query 121: 0.0769 At 200 docs: 0.1117 Query 122: 0.3958 At 500 docs: 0.0514 Query 123: 0.3667 At 1000 docs: 0.0273 +0.5 Query 124: 0.4402 Query 125: 0.5276 R-Precision (prec. after R Query 126: 0.7137 (=num_rel) docs. retrieved): Query 127: 0.9281 Exact: 0.4070 Query 128: 0.3922 0.0 Query 129: 0.4325 Query 130: 0.6468 Query 131: 0.5972 Query 132: 0.3200 Query 133: 0.0576 Difference (average precision) Query 134: 0.8095 −0.5 Query 135: 0.7857 Query 136: 1.0000 Query 137: 0.1169 Query 138: 0.4628 Query 139: 0.6730 −1.0 Query 140: 0.7281 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 1, Monolingual French, Auto, TDN Run BKMLFF1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1618 Total number of documents Query 92: 0.4657 over all queries Query 93: 0.8393 Retrieved: 50000 0.8 Query 94: 0.5813 Relevant: 1383 Query 95: 0.4307 Rel_ret: 1337 Query 96: 0.4934 Query 97: 0.6535 Interpolated Recall - 0.6 Query 98: 0.9798 Precision Averages: Query 99: 0.5159 at 0.00 0.8125 Query 100: 0.3607 at 0.10 0.7747 Precision Query 101: 0.3506 at 0.20 0.6718 Query 102: 0.4733 at 0.30 0.5718 0.4 Query 103: 0.4293 at 0.40 0.5461 Query 104: 0.3215 at 0.50 0.5017 Query 105: 0.8333 at 0.60 0.4647 Query 106: 0.3656 at 0.70 0.3938 0.2 Query 107: 0.0024 at 0.80 0.3440 Query 108: 0.3387 at 0.90 0.2720 Query 109: 0.0449 at 1.00 0.1945 Query 110: 0.6216 0.0 Query 111: 0.1457 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5913 for all rel. documents Query 113: 0.3840 0.4884 Recall Query 114: 0.5204 Precision: Query 115: 0.4851 At 5 docs: 0.5200 Run BKMLFF1 Query 116: 0.5628 At 10 docs: 0.4640 Comparison to median by topic Query 117: 0.7460 At 15 docs: 0.4333 +1.0 Query 118: 0.1424 At 20 docs: 0.3890 Query 119: 0.9136 At 30 docs: 0.3453 Query 120: 0.0157 At 100 docs: 0.1926 Query 121: 0.0769 At 200 docs: 0.1132 Query 122: 0.4052 At 500 docs: 0.0512 Query 123: 0.5833 At 1000 docs: 0.0267 +0.5 Query 124: 0.5523 Query 125: 0.5145 R-Precision (prec. after R Query 126: 0.6931 (=num_rel) docs. retrieved): Query 127: 0.7727 Exact: 0.4445 Query 128: 0.2611 0.0 Query 129: 0.3782 Query 130: 0.6859 Query 131: 0.5466 Query 132: 0.3716 Query 133: 0.3872 Difference (average precision) Query 134: 0.7917 −0.5 Query 135: 0.8095 Query 136: 1.0000 Query 137: 0.0080 Query 138: 0.6813 Query 139: 0.3364 −1.0 Query 140: 0.7920 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 1, Monolingual French, Auto, TD Run BKMLFF2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1574 Total number of documents Query 92: 0.4705 over all queries Query 93: 1.0000 Retrieved: 50000 0.8 Query 94: 0.6211 Relevant: 1383 Query 95: 0.5053 Rel_ret: 1313 Query 96: 0.5050 Query 97: 0.4606 Interpolated Recall - 0.6 Query 98: 1.0000 Precision Averages: Query 99: 0.3912 at 0.00 0.7475 Query 100: 0.3670 at 0.10 0.6990 Precision Query 101: 0.0307 at 0.20 0.6363 Query 102: 0.5958 at 0.30 0.5358 0.4 Query 103: 0.3466 at 0.40 0.5068 Query 104: 0.3106 at 0.50 0.4717 Query 105: 0.2361 at 0.60 0.4332 Query 106: 0.3860 at 0.70 0.3752 0.2 Query 107: 0.0017 at 0.80 0.3301 Query 108: 0.3190 at 0.90 0.2666 Query 109: 0.0316 at 1.00 0.1904 Query 110: 0.6182 0.0 Query 111: 0.0352 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4847 for all rel. documents Query 113: 0.0394 0.4558 Recall Query 114: 0.5279 Precision: Query 115: 0.4917 At 5 docs: 0.4880 Run BKMLFF2 Query 116: 0.8369 At 10 docs: 0.4620 Comparison to median by topic Query 117: 0.2705 At 15 docs: 0.3947 +1.0 Query 118: 0.0500 At 20 docs: 0.3620 Query 119: 0.9193 At 30 docs: 0.3113 Query 120: 0.0059 At 100 docs: 0.1762 Query 121: 1.0000 At 200 docs: 0.1086 Query 122: 0.5250 At 500 docs: 0.0503 Query 123: 0.7000 At 1000 docs: 0.0263 +0.5 Query 124: 0.3019 Query 125: 0.5024 R-Precision (prec. after R Query 126: 0.5964 (=num_rel) docs. retrieved): Query 127: 0.7419 Exact: 0.4348 Query 128: 0.2621 0.0 Query 129: 0.4638 Query 130: 0.6353 Query 131: 0.5667 Query 132: 0.3214 Query 133: 0.0074 Difference (average precision) Query 134: 0.7778 −0.5 Query 135: 0.8134 Query 136: 1.0000 Query 137: 0.0006 Query 138: 0.3968 Query 139: 0.3924 −1.0 Query 140: 0.7683 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 2, Monolingual French, Auto, TD Run bky2mofr Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.2224 Total number of documents Query 92: 0.4334 over all queries Query 93: 0.9267 Retrieved: 50000 0.8 Query 94: 0.5928 Relevant: 1383 Query 95: 0.5886 Rel_ret: 1354 Query 96: 0.5370 Query 97: 0.4999 Interpolated Recall - Query 98: 0.9086 Precision Averages: 0.6 Query 99: 0.6372 at 0.00 0.7915 Query 100: 0.3899 at 0.10 0.7270 Query 101: 0.0643 at 0.20 0.6835 Precision Query 102: 0.6971 at 0.30 0.6353 0.4 Query 103: 0.5443 at 0.40 0.5985 Query 104: 0.0997 at 0.50 0.5661 Query 105: 0.3250 at 0.60 0.5106 Query 106: 0.4773 at 0.70 0.4696 0.2 Query 107: 0.4787 at 0.80 0.4047 Query 108: 0.3371 at 0.90 0.3197 Query 109: 0.0944 at 1.00 0.2019 Query 110: 0.6143 0.0 Query 111: 0.0178 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6019 for all rel. documents Query 113: 0.0376 0.5191 Recall Query 114: 0.4982 Precision: Query 115: 0.5733 At 5 docs: 0.5640 Run bky2mofr Query 116: 0.8363 At 10 docs: 0.4980 Query 117: 0.3124 At 15 docs: 0.4520 Comparison to median by topic Query 118: 0.0266 At 20 docs: 0.4170 +1.0 Query 119: 0.9151 At 30 docs: 0.3653 Query 120: 0.0182 At 100 docs: 0.1986 Query 121: 1.0000 At 200 docs: 0.1158 Query 122: 0.7194 At 500 docs: 0.0524 Query 123: 0.8333 At 1000 docs: 0.0271 +0.5 Query 124: 0.5278 Query 125: 0.4500 R-Precision (prec. after R Query 126: 0.4911 (=num_rel) docs. retrieved): Query 127: 0.9187 Exact: 0.4996 Query 128: 0.4994 0.0 Query 129: 0.4725 Query 130: 0.7017 Query 131: 0.5307 Query 132: 0.3735 Query 133: 0.4996 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.7846 Query 136: 1.0000 Query 137: 0.0063 Query 138: 0.3593 Query 139: 0.6812 −1.0 Query 140: 0.8008 91 100 110 120 130 140 Topic number Hummingbird, Monolingual French, Auto, TD Run humFR02 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1156 Total number of documents Query 92: 0.3714 over all queries Query 93: 1.0000 Retrieved: 50000 0.8 Query 94: 0.3073 Relevant: 1383 Query 95: 0.2459 Rel_ret: 1265 Query 96: 0.4298 Query 97: 0.2700 Interpolated Recall - 0.6 Query 98: 0.6171 Precision Averages: Query 99: 0.3456 at 0.00 0.7831 Query 100: 0.4415 at 0.10 0.7263 Precision Query 101: 0.0678 at 0.20 0.6536 Query 102: 0.7137 at 0.30 0.5561 0.4 Query 103: 0.6265 at 0.40 0.5056 Query 104: 0.3093 at 0.50 0.4414 Query 105: 0.2361 at 0.60 0.3826 Query 106: 0.3786 at 0.70 0.3279 0.2 Query 107: 0.2101 at 0.80 0.2581 Query 108: 0.3166 at 0.90 0.1815 Query 109: 0.0731 at 1.00 0.1385 Query 110: 0.6324 0.0 Query 111: 0.0343 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5321 for all rel. documents Query 113: 0.0225 0.4359 Recall Query 114: 0.4090 Precision: Query 115: 0.6389 At 5 docs: 0.5400 Run humFR02 Query 116: 0.3953 At 10 docs: 0.4540 Comparison to median by topic Query 117: 0.2540 At 15 docs: 0.3960 +1.0 Query 118: 0.1009 At 20 docs: 0.3580 Query 119: 0.8520 At 30 docs: 0.3007 Query 120: 0.0097 At 100 docs: 0.1644 Query 121: 1.0000 At 200 docs: 0.0995 Query 122: 0.2784 At 500 docs: 0.0468 Query 123: 1.0000 At 1000 docs: 0.0253 +0.5 Query 124: 0.4605 Query 125: 0.4774 R-Precision (prec. after R Query 126: 0.5500 (=num_rel) docs. retrieved): Query 127: 0.8673 Exact: 0.4226 Query 128: 0.5275 0.0 Query 129: 0.1757 Query 130: 0.6820 Query 131: 0.5734 Query 132: 0.3708 Query 133: 0.1839 Difference (average precision) Query 134: 0.7255 −0.5 Query 135: 0.4354 Query 136: 1.0000 Query 137: 0.0360 Query 138: 0.2988 Query 139: 0.5466 −1.0 Query 140: 0.6502 91 100 110 120 130 140 Topic number Océ, Monolingual French, Auto, TD Run oce02monFR Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1592 Total number of documents Query 92: 0.4108 over all queries Query 93: 1.0000 Retrieved: 50000 0.8 Query 94: 0.2673 Relevant: 1383 Query 95: 0.2647 Rel_ret: 1247 Query 96: 0.5526 Query 97: 0.2526 Interpolated Recall - 0.6 Query 98: 0.6551 Precision Averages: Query 99: 0.3686 at 0.00 0.7318 Query 100: 0.3619 at 0.10 0.6615 Precision Query 101: 0.0414 at 0.20 0.5792 Query 102: 0.4878 at 0.30 0.5098 0.4 Query 103: 0.5232 at 0.40 0.4533 Query 104: 0.1396 at 0.50 0.3800 Query 105: 0.2429 at 0.60 0.3364 Query 106: 0.3882 at 0.70 0.2872 0.2 Query 107: 0.1577 at 0.80 0.2278 Query 108: 0.2739 at 0.90 0.1696 Query 109: 0.0639 at 1.00 0.1437 Query 110: 0.5632 0.0 Query 111: 0.0343 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4222 for all rel. documents Query 113: 0.0395 0.3920 Recall Query 114: 0.4564 Precision: Query 115: 0.4373 At 5 docs: 0.5240 Run oce02monFR Query 116: 0.3542 At 10 docs: 0.4160 Comparison to median by topic Query 117: 0.1998 At 15 docs: 0.3667 +1.0 Query 118: 0.0226 At 20 docs: 0.3340 Query 119: 0.7149 At 30 docs: 0.2840 Query 120: 0.0161 At 100 docs: 0.1546 Query 121: 1.0000 At 200 docs: 0.0951 Query 122: 0.2715 At 500 docs: 0.0459 Query 123: 0.5833 At 1000 docs: 0.0249 +0.5 Query 124: 0.4860 Query 125: 0.4408 R-Precision (prec. after R Query 126: 0.0864 (=num_rel) docs. retrieved): Query 127: 0.9327 Exact: 0.3832 Query 128: 0.5680 0.0 Query 129: 0.0427 Query 130: 0.6485 Query 131: 0.4315 Query 132: 0.3196 Query 133: 0.1960 Difference (average precision) Query 134: 0.9167 −0.5 Query 135: 0.4580 Query 136: 1.0000 Query 137: 0.0386 Query 138: 0.3569 Query 139: 0.5292 −1.0 Query 140: 0.4195 91 100 110 120 130 140 Topic number Océ, Monolingual French, Auto, T Run oce02monFRto Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0018 Total number of documents Query 92: 0.2759 over all queries Query 93: 0.9667 Retrieved: 44062 0.8 Query 94: 0.0029 Relevant: 1383 Query 95: 0.0454 Rel_ret: 1083 Query 96: 0.0899 Query 97: 0.2423 Interpolated Recall - 0.6 Query 98: 0.1379 Precision Averages: Query 99: 0.3479 at 0.00 0.6466 Query 100: 0.2639 at 0.10 0.5610 Precision Query 101: 0.1085 at 0.20 0.5092 Query 102: 0.4682 at 0.30 0.4435 0.4 Query 103: 0.3697 at 0.40 0.3537 Query 104: 0.1281 at 0.50 0.3221 Query 105: 0.4167 at 0.60 0.2927 Query 106: 0.2221 at 0.70 0.2515 0.2 Query 107: 0.0939 at 0.80 0.2074 Query 108: 0.2944 at 0.90 0.1366 Query 109: 0.0624 at 1.00 0.1025 Query 110: 0.3035 0.0 Query 111: 0.0033 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6583 for all rel. documents Query 113: 0.0147 0.3344 Recall Query 114: 0.0978 Precision: Query 115: 0.8263 At 5 docs: 0.3760 Run oce02monFRto Query 116: 0.3364 At 10 docs: 0.3100 Comparison to median by topic Query 117: 0.0265 At 15 docs: 0.2787 +1.0 Query 118: 0.0706 At 20 docs: 0.2550 Query 119: 0.7183 At 30 docs: 0.2227 Query 120: 0.0032 At 100 docs: 0.1280 Query 121: 0.5000 At 200 docs: 0.0808 Query 122: 0.3313 At 500 docs: 0.0394 Query 123: 1.0000 At 1000 docs: 0.0217 +0.5 Query 124: 0.5358 Query 125: 0.4974 R-Precision (prec. after R Query 126: 0.7303 (=num_rel) docs. retrieved): Query 127: 0.8494 Exact: 0.3272 Query 128: 0.2693 0.0 Query 129: 0.0159 Query 130: 0.6246 Query 131: 0.5371 Query 132: 0.3207 Query 133: 0.0626 Difference (average precision) Query 134: 0.4286 −0.5 Query 135: 0.0634 Query 136: 1.0000 Query 137: 0.3525 Query 138: 0.1012 Query 139: 0.5395 −1.0 Query 140: 0.3631 91 100 110 120 130 140 Topic number City University, Monolingual French, Auto, TD Run plfr02td Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0163 Total number of documents Query 92: 0.2283 over all queries Query 93: 0.0000 Retrieved: 50000 0.8 Query 94: 0.0207 Relevant: 1383 Query 95: 0.3105 Rel_ret: 1106 Query 96: 0.3242 Query 97: 0.1267 Interpolated Recall - Query 98: 0.0412 Precision Averages: 0.6 Query 99: 0.1343 at 0.00 0.5196 Query 100: 0.1222 at 0.10 0.4445 Query 101: 0.0851 at 0.20 0.3761 Precision Query 102: 0.1657 at 0.30 0.3184 0.4 Query 103: 0.1479 at 0.40 0.2900 Query 104: 0.0385 at 0.50 0.2428 Query 105: 0.0032 at 0.60 0.2038 Query 106: 0.3446 at 0.70 0.1854 0.2 Query 107: 0.0022 at 0.80 0.1486 Query 108: 0.0023 at 0.90 0.1038 Query 109: 0.0056 at 1.00 0.0590 Query 110: 0.6404 0.0 Query 111: 0.0058 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.3923 for all rel. documents Query 113: 0.0027 0.2480 Recall Query 114: 0.4931 Precision: Query 115: 0.2494 At 5 docs: 0.3400 Run plfr02td Query 116: 0.5268 At 10 docs: 0.2980 Query 117: 0.2705 At 15 docs: 0.2627 Comparison to median by topic Query 118: 0.0196 At 20 docs: 0.2380 +1.0 Query 119: 0.8168 At 30 docs: 0.2100 Query 120: 0.0183 At 100 docs: 0.1202 Query 121: 0.2000 At 200 docs: 0.0754 Query 122: 0.2154 At 500 docs: 0.0388 Query 123: 0.4500 At 1000 docs: 0.0221 +0.5 Query 124: 0.1191 Query 125: 0.4083 R-Precision (prec. after R Query 126: 0.4544 (=num_rel) docs. retrieved): Query 127: 0.9470 Exact: 0.2407 Query 128: 0.1288 0.0 Query 129: 0.0044 Query 130: 0.5955 Query 131: 0.0819 Query 132: 0.2684 Query 133: 0.1225 Difference (average precision) Query 134: 0.5889 −0.5 Query 135: 0.0425 Query 136: 0.8333 Query 137: 0.0000 Query 138: 0.4088 Query 139: 0.4759 −1.0 Query 140: 0.5014 91 100 110 120 130 140 Topic number Tagmatica, Monolingual French, Manual, DN Run runA Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0769 Total number of documents Query 92: 0.3025 over all queries Query 93: 0.2000 Retrieved: 641 0.8 Query 94: 0.3022 Relevant: 1383 Query 95: 0.1546 Rel_ret: 286 Query 96: 0.0714 Query 97: 0.1762 Interpolated Recall - 0.6 Query 98: 0.6198 Precision Averages: Query 99: 0.1971 at 0.00 0.7548 Query 100: 0.2667 at 0.10 0.6036 Precision Query 101: 0.1429 at 0.20 0.4864 Query 102: 0.3664 at 0.30 0.3215 0.4 Query 103: 0.3077 at 0.40 0.2626 Query 104: 0.0000 at 0.50 0.2216 Query 105: 0.1667 at 0.60 0.1330 Query 106: 0.0502 at 0.70 0.0761 0.2 Query 107: 0.0000 at 0.80 0.0393 Query 108: 0.0391 at 0.90 0.0333 Query 109: 0.0000 at 1.00 0.0333 Query 110: 0.3304 0.0 Query 111: 0.5000 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.1278 for all rel. documents Query 113: 0.0556 0.2391 Recall Query 114: 0.1333 Precision: Query 115: 0.6000 At 5 docs: 0.4520 Run runA Query 116: 0.4054 At 10 docs: 0.3400 Comparison to median by topic Query 117: 0.0000 At 15 docs: 0.2693 +1.0 Query 118: 0.3500 At 20 docs: 0.2310 Query 119: 0.2598 At 30 docs: 0.1720 Query 120: 0.0000 At 100 docs: 0.0558 Query 121: 0.0000 At 200 docs: 0.0286 Query 122: 0.2500 At 500 docs: 0.0114 Query 123: 0.5833 At 1000 docs: 0.0057 +0.5 Query 124: 0.0448 Query 125: 0.0233 R-Precision (prec. after R Query 126: 0.2500 (=num_rel) docs. retrieved): Query 127: 0.7276 Exact: 0.2789 Query 128: 0.2500 0.0 Query 129: 0.2021 Query 130: 0.5154 Query 131: 0.0122 Query 132: 0.5000 Query 133: 0.0000 Difference (average precision) Query 134: 0.3333 −0.5 Query 135: 0.0000 Query 136: 1.0000 Query 137: 0.6667 Query 138: 0.0327 Query 139: 0.2222 −1.0 Query 140: 0.1406 91 100 110 120 130 140 Topic number IRIT, Toulouse, Monolingual French, Auto, TD Run RunMonFrench Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0292 Total number of documents Query 92: 0.2887 over all queries Query 93: 1.0000 Retrieved: 50000 0.8 Query 94: 0.6058 Relevant: 1383 Query 95: 0.3151 Rel_ret: 1120 Query 96: 0.6191 Query 97: 0.1677 Interpolated Recall - 0.6 Query 98: 0.8094 Precision Averages: Query 99: 0.2029 at 0.00 0.6617 Query 100: 0.3794 at 0.10 0.5785 Precision Query 101: 0.0915 at 0.20 0.5347 Query 102: 0.7104 at 0.30 0.4851 0.4 Query 103: 0.4514 at 0.40 0.4433 Query 104: 0.1108 at 0.50 0.4042 Query 105: 0.2381 at 0.60 0.3530 Query 106: 0.4417 at 0.70 0.2740 0.2 Query 107: 0.0338 at 0.80 0.2178 Query 108: 0.1064 at 0.90 0.1563 Query 109: 0.0000 at 1.00 0.1066 Query 110: 0.6893 0.0 Query 111: 0.0168 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5310 for all rel. documents Query 113: 0.0182 0.3733 Recall Query 114: 0.2959 Precision: Query 115: 0.6370 At 5 docs: 0.4440 Run RunMonFrench Query 116: 0.5483 At 10 docs: 0.3720 Comparison to median by topic Query 117: 0.0238 At 15 docs: 0.3453 +1.0 Query 118: 0.0088 At 20 docs: 0.3250 Query 119: 0.7612 At 30 docs: 0.2753 Query 120: 0.0115 At 100 docs: 0.1456 Query 121: 0.5000 At 200 docs: 0.0857 Query 122: 0.1989 At 500 docs: 0.0405 Query 123: 1.0000 At 1000 docs: 0.0224 +0.5 Query 124: 0.4932 Query 125: 0.3788 R-Precision (prec. after R Query 126: 0.6152 (=num_rel) docs. retrieved): Query 127: 0.6391 Exact: 0.3610 Query 128: 0.0572 0.0 Query 129: 0.1267 Query 130: 0.6628 Query 131: 0.4235 Query 132: 0.5506 Query 133: 0.0999 Difference (average precision) Query 134: 0.7381 −0.5 Query 135: 0.0102 Query 136: 1.0000 Query 137: 0.0012 Query 138: 0.1694 Query 139: 0.2873 −1.0 Query 140: 0.5683 91 100 110 120 130 140 Topic number SICS/Conexor, Monolingual French, Auto, TD Run sicsFRFR0 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0226 Total number of documents Query 92: 0.3558 over all queries Query 93: 0.8254 Retrieved: 50000 0.8 Query 94: 0.1622 Relevant: 1383 Query 95: 0.0634 Rel_ret: 1117 Query 96: 0.1200 Query 97: 0.1797 Interpolated Recall - 0.6 Query 98: 0.2322 Precision Averages: Query 99: 0.3034 at 0.00 0.4945 Query 100: 0.1615 at 0.10 0.3948 Precision Query 101: 0.0316 at 0.20 0.3350 Query 102: 0.3989 at 0.30 0.2691 0.4 Query 103: 0.1910 at 0.40 0.2437 Query 104: 0.0311 at 0.50 0.2152 Query 105: 0.2193 at 0.60 0.1774 Query 106: 0.1127 at 0.70 0.1310 0.2 Query 107: 0.1072 at 0.80 0.1064 Query 108: 0.1135 at 0.90 0.0776 Query 109: 0.0099 at 1.00 0.0539 Query 110: 0.4329 0.0 Query 111: 0.0034 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4360 for all rel. documents Query 113: 0.0227 0.2118 Recall Query 114: 0.1110 Precision: Query 115: 0.0081 At 5 docs: 0.2480 Run sicsFRFR0 Query 116: 0.2401 At 10 docs: 0.2180 Comparison to median by topic Query 117: 0.1327 At 15 docs: 0.1960 +1.0 Query 118: 0.0069 At 20 docs: 0.1860 Query 119: 0.5239 At 30 docs: 0.1693 Query 120: 0.0028 At 100 docs: 0.1048 Query 121: 0.0270 At 200 docs: 0.0697 Query 122: 0.0409 At 500 docs: 0.0387 Query 123: 1.0000 At 1000 docs: 0.0223 +0.5 Query 124: 0.1100 Query 125: 0.2370 R-Precision (prec. after R Query 126: 0.4534 (=num_rel) docs. retrieved): Query 127: 0.1954 Exact: 0.2155 Query 128: 0.2722 0.0 Query 129: 0.0460 Query 130: 0.4127 Query 131: 0.2675 Query 132: 0.2981 Query 133: 0.0130 Difference (average precision) Query 134: 0.7292 −0.5 Query 135: 0.0738 Query 136: 0.1964 Query 137: 0.0027 Query 138: 0.0911 Query 139: 0.2197 −1.0 Query 140: 0.3415 91 100 110 120 130 140 Topic number SICS/Conexor, Monolingual French, Auto, TD Run sicsFRFRX0 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0154 Total number of documents Query 92: 0.4701 over all queries Query 93: 0.6195 Retrieved: 50000 0.8 Query 94: 0.0572 Relevant: 1383 Query 95: 0.0461 Rel_ret: 1072 Query 96: 0.1839 Query 97: 0.1115 Interpolated Recall - 0.6 Query 98: 0.3177 Precision Averages: Query 99: 0.1179 at 0.00 0.4722 Query 100: 0.1936 at 0.10 0.3360 Precision Query 101: 0.2502 at 0.20 0.2691 Query 102: 0.1725 at 0.30 0.2314 0.4 Query 103: 0.1787 at 0.40 0.2071 Query 104: 0.0222 at 0.50 0.1856 Query 105: 0.0995 at 0.60 0.1507 Query 106: 0.1108 at 0.70 0.1051 0.2 Query 107: 0.0421 at 0.80 0.0785 Query 108: 0.1215 at 0.90 0.0562 Query 109: 0.0104 at 1.00 0.0349 Query 110: 0.4671 0.0 Query 111: 0.0003 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.3073 for all rel. documents Query 113: 0.0472 0.1775 Recall Query 114: 0.1317 Precision: Query 115: 0.0087 At 5 docs: 0.2200 Run sicsFRFRX0 Query 116: 0.2174 At 10 docs: 0.1920 Comparison to median by topic Query 117: 0.1480 At 15 docs: 0.1773 +1.0 Query 118: 0.0047 At 20 docs: 0.1600 Query 119: 0.4825 At 30 docs: 0.1460 Query 120: 0.0027 At 100 docs: 0.1004 Query 121: 0.0312 At 200 docs: 0.0695 Query 122: 0.0156 At 500 docs: 0.0372 Query 123: 1.0000 At 1000 docs: 0.0214 +0.5 Query 124: 0.1220 Query 125: 0.1485 R-Precision (prec. after R Query 126: 0.1817 (=num_rel) docs. retrieved): Query 127: 0.1490 Exact: 0.2030 Query 128: 0.0835 0.0 Query 129: 0.0299 Query 130: 0.4293 Query 131: 0.0798 Query 132: 0.3392 Query 133: 0.0096 Difference (average precision) Query 134: 0.2126 −0.5 Query 135: 0.0396 Query 136: 0.1024 Query 137: 0.0014 Query 138: 0.0767 Query 139: 0.1562 −1.0 Query 140: 0.7105 91 100 110 120 130 140 Topic number TLR Research, Monolingual French, Auto, TD Run tlrfr Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0801 Total number of documents Query 92: 0.3484 over all queries Query 93: 0.9667 Retrieved: 50000 0.8 Query 94: 0.2348 Relevant: 1383 Query 95: 0.2389 Rel_ret: 1269 Query 96: 0.3390 Query 97: 0.2087 Interpolated Recall - 0.6 Query 98: 0.6845 Precision Averages: Query 99: 0.3438 at 0.00 0.7839 Query 100: 0.4101 at 0.10 0.6749 Precision Query 101: 0.1001 at 0.20 0.6010 Query 102: 0.6592 at 0.30 0.5373 0.4 Query 103: 0.6027 at 0.40 0.4791 Query 104: 0.4101 at 0.50 0.4439 Query 105: 0.1917 at 0.60 0.3840 Query 106: 0.3327 at 0.70 0.3374 0.2 Query 107: 0.2014 at 0.80 0.2674 Query 108: 0.3084 at 0.90 0.1997 Query 109: 0.1237 at 1.00 0.1493 Query 110: 0.6292 0.0 Query 111: 0.2598 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6024 for all rel. documents Query 113: 0.0257 0.4232 Recall Query 114: 0.3562 Precision: Query 115: 0.7390 At 5 docs: 0.4800 Run tlrfr Query 116: 0.3836 At 10 docs: 0.4360 Comparison to median by topic Query 117: 0.2174 At 15 docs: 0.3813 +1.0 Query 118: 0.0898 At 20 docs: 0.3420 Query 119: 0.8164 At 30 docs: 0.2947 Query 120: 0.0091 At 100 docs: 0.1614 Query 121: 1.0000 At 200 docs: 0.0990 Query 122: 0.2361 At 500 docs: 0.0466 Query 123: 0.8333 At 1000 docs: 0.0254 +0.5 Query 124: 0.6291 Query 125: 0.2275 R-Precision (prec. after R Query 126: 0.3002 (=num_rel) docs. retrieved): Query 127: 0.8357 Exact: 0.4134 Query 128: 0.3628 0.0 Query 129: 0.1638 Query 130: 0.7899 Query 131: 0.6424 Query 132: 0.2535 Query 133: 0.1114 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.4288 Query 136: 1.0000 Query 137: 0.0887 Query 138: 0.2105 Query 139: 0.5069 −1.0 Query 140: 0.6266 91 100 110 120 130 140 Topic number University of Amsterdam, Monolingual French, Auto, TD Run UAmsC02FrFrNGiMO Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.2649 Total number of documents Query 92: 0.4938 over all queries Query 93: 0.9667 Retrieved: 50000 0.8 Query 94: 0.7425 Relevant: 1383 Query 95: 0.3460 Rel_ret: 1254 Query 96: 0.5443 Query 97: 0.4583 Interpolated Recall - 0.6 Query 98: 0.9500 Precision Averages: Query 99: 0.1486 at 0.00 0.7671 Query 100: 0.3021 at 0.10 0.6717 Precision Query 101: 0.2493 at 0.20 0.6408 Query 102: 0.4830 at 0.30 0.5716 0.4 Query 103: 0.4409 at 0.40 0.5244 Query 104: 0.1630 at 0.50 0.4734 Query 105: 0.8333 at 0.60 0.4200 Query 106: 0.4659 at 0.70 0.3642 0.2 Query 107: 0.1413 at 0.80 0.3034 Query 108: 0.3803 at 0.90 0.2334 Query 109: 0.1383 at 1.00 0.1730 Query 110: 0.6479 0.0 Query 111: 0.0245 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4755 for all rel. documents Query 113: 0.0211 0.4535 Recall Query 114: 0.6039 Precision: Query 115: 0.5729 At 5 docs: 0.5120 Run UAmsC02FrFrNGiMO Query 116: 0.5816 At 10 docs: 0.4580 Comparison to median by topic Query 117: 0.3744 At 15 docs: 0.4067 +1.0 Query 118: 0.0247 At 20 docs: 0.3720 Query 119: 0.8948 At 30 docs: 0.3213 Query 120: 0.0032 At 100 docs: 0.1692 Query 121: 0.1667 At 200 docs: 0.1028 Query 122: 0.2341 At 500 docs: 0.0478 Query 123: 1.0000 At 1000 docs: 0.0251 +0.5 Query 124: 0.3435 Query 125: 0.4128 R-Precision (prec. after R Query 126: 0.7764 (=num_rel) docs. retrieved): Query 127: 0.8929 Exact: 0.4224 Query 128: 0.3538 0.0 Query 129: 0.4827 Query 130: 0.5868 Query 131: 0.4092 Query 132: 0.3811 Query 133: 0.0093 Difference (average precision) Query 134: 0.8095 −0.5 Query 135: 0.6212 Query 136: 1.0000 Query 137: 0.0117 Query 138: 0.2989 Query 139: 0.4705 −1.0 Query 140: 0.6773 91 100 110 120 130 140 Topic number Université de Neuchâtel, Monolingual French, Auto, TD Run UniNEfr Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0443 Total number of documents Query 92: 0.3878 over all queries Query 93: 1.0000 Retrieved: 50000 0.8 Query 94: 0.6619 Relevant: 1383 Query 95: 0.2255 Rel_ret: 1283 Query 96: 0.4272 Query 97: 0.3047 Interpolated Recall - 0.6 Query 98: 0.9361 Precision Averages: Query 99: 0.2132 at 0.00 0.8179 Query 100: 0.4147 at 0.10 0.7563 Precision Query 101: 0.1472 at 0.20 0.6657 Query 102: 0.6961 at 0.30 0.5924 0.4 Query 103: 0.6553 at 0.40 0.5379 Query 104: 0.1662 at 0.50 0.5110 Query 105: 0.8333 at 0.60 0.4443 Query 106: 0.4578 at 0.70 0.4008 0.2 Query 107: 0.2373 at 0.80 0.3289 Query 108: 0.3265 at 0.90 0.2369 Query 109: 0.0914 at 1.00 0.1832 Query 110: 0.6421 0.0 Query 111: 0.2199 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5090 for all rel. documents Query 113: 0.0333 0.4841 Recall Query 114: 0.4609 Precision: Query 115: 0.7669 At 5 docs: 0.5760 Run UniNEfr Query 116: 0.5683 At 10 docs: 0.4740 Comparison to median by topic Query 117: 0.2405 At 15 docs: 0.4160 +1.0 Query 118: 0.0842 At 20 docs: 0.3770 Query 119: 0.8730 At 30 docs: 0.3180 Query 120: 0.0148 At 100 docs: 0.1676 Query 121: 1.0000 At 200 docs: 0.1021 Query 122: 0.2770 At 500 docs: 0.0478 Query 123: 0.8333 At 1000 docs: 0.0257 +0.5 Query 124: 0.5192 Query 125: 0.4949 R-Precision (prec. after R Query 126: 0.7688 (=num_rel) docs. retrieved): Query 127: 0.8277 Exact: 0.4530 Query 128: 0.5568 0.0 Query 129: 0.2469 Query 130: 0.6325 Query 131: 0.5904 Query 132: 0.5314 Query 133: 0.1378 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.4301 Query 136: 1.0000 Query 137: 0.1241 Query 138: 0.2982 Query 139: 0.6178 −1.0 Query 140: 0.6795 91 100 110 120 130 140 Topic number Université de Neuchâtel, Monolingual French, Manual, TDN Run UniNEfrtdn Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.2406 Total number of documents Query 92: 0.4104 over all queries Query 93: 1.0000 Retrieved: 50000 0.8 Query 94: 0.7353 Relevant: 1383 Query 95: 0.4106 Rel_ret: 1368 Query 96: 0.4754 Query 97: 0.5889 Interpolated Recall - 0.6 Query 98: 0.9599 Precision Averages: Query 99: 0.4631 at 0.00 0.8916 Query 100: 0.4622 at 0.10 0.8377 Precision Query 101: 0.4440 at 0.20 0.7959 Query 102: 0.7928 at 0.30 0.7059 0.4 Query 103: 0.7282 at 0.40 0.6457 Query 104: 0.3354 at 0.50 0.6107 Query 105: 1.0000 at 0.60 0.5734 Query 106: 0.4599 at 0.70 0.5131 0.2 Query 107: 0.2666 at 0.80 0.4454 Query 108: 0.3515 at 0.90 0.3489 Query 109: 0.7436 at 1.00 0.2602 Query 110: 0.6740 0.0 Query 111: 0.4726 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5450 for all rel. documents Query 113: 0.0954 0.5919 Recall Query 114: 0.7689 Precision: Query 115: 0.6909 At 5 docs: 0.6400 Run UniNEfrtdn Query 116: 0.6037 At 10 docs: 0.5560 Comparison to median by topic Query 117: 0.5259 At 15 docs: 0.4907 +1.0 Query 118: 0.1444 At 20 docs: 0.4470 Query 119: 0.8948 At 30 docs: 0.3713 Query 120: 0.0154 At 100 docs: 0.2008 Query 121: 1.0000 At 200 docs: 0.1158 Query 122: 0.3777 At 500 docs: 0.0519 Query 123: 0.5833 At 1000 docs: 0.0274 +0.5 Query 124: 0.6548 Query 125: 0.4537 R-Precision (prec. after R Query 126: 0.8460 (=num_rel) docs. retrieved): Query 127: 0.9626 Exact: 0.5616 Query 128: 0.5214 0.0 Query 129: 0.3875 Query 130: 0.7616 Query 131: 0.6633 Query 132: 0.4894 Query 133: 0.4128 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.7358 Query 136: 1.0000 Query 137: 0.2657 Query 138: 0.7295 Query 139: 0.6565 −1.0 Query 140: 0.7922 91 100 110 120 130 140 Topic number JHU/APL, Monolingual Italian, Auto, TD Run aplmoit Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.1064 Total number of documents Query 92: 0.4206 over all queries Query 93: 1.0000 Retrieved: 49000 0.8 Query 94: 0.6402 Relevant: 1072 Query 95: 0.1619 Rel_ret: 1039 Query 96: 0.4859 Query 97: 0.8304 Interpolated Recall - Query 98: 0.7450 Precision Averages: 0.6 Query 99: 0.7079 at 0.00 0.6890 Query 100: 0.4633 at 0.10 0.6614 Query 101: 0.1192 at 0.20 0.5992 Precision Query 102: 0.3659 at 0.30 0.5548 0.4 Query 103: 0.5649 at 0.40 0.5219 Query 104: 0.0606 at 0.50 0.4993 Query 105: 0.5113 at 0.60 0.4420 Query 106: 0.2934 at 0.70 0.3921 0.2 Query 107: 0.0703 at 0.80 0.3499 Query 108: 0.1707 at 0.90 0.2915 Query 109: 0.2763 at 1.00 0.2207 Query 110: 0.9062 0.0 Query 111: 0.0387 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6269 for all rel. documents Query 113: 0.1011 0.4599 Recall Query 114: 0.5688 Precision: Query 115: 0.7126 At 5 docs: 0.5224 Run aplmoit Query 116: 0.8204 At 10 docs: 0.4939 Query 117: 0.5416 At 15 docs: 0.4463 Comparison to median by topic Query 118: 0.0462 At 20 docs: 0.4071 +1.0 Query 119: 0.8984 At 30 docs: 0.3476 Query 121: 0.0940 At 100 docs: 0.1647 Query 122: 0.0645 At 200 docs: 0.0939 Query 123: 0.9490 At 500 docs: 0.0411 Query 124: 0.3617 At 1000 docs: 0.0212 +0.5 Query 125: 0.7759 Query 126: 0.5017 R-Precision (prec. after R Query 127: 0.6344 (=num_rel) docs. retrieved): Query 128: 0.0531 Exact: 0.4270 Query 129: 0.3102 0.0 Query 130: 0.3868 Query 131: 0.4021 Query 132: 0.3452 Query 133: 0.4232 Query 134: 0.8711 Difference (average precision) Query 135: 0.9055 −0.5 Query 136: 0.9462 Query 137: 0.0414 Query 138: 0.7910 Query 139: 0.1925 Query 140: 0.2299 −1.0 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 2, Monolingual Italian, Auto, TD Run bky2moit Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.4281 Total number of documents Query 92: 0.3792 over all queries Query 93: 0.8250 Retrieved: 49000 0.8 Query 94: 0.5717 Relevant: 1072 Query 95: 0.0892 Rel_ret: 1024 Query 96: 0.2927 Query 97: 0.7875 Interpolated Recall - Query 98: 0.5611 Precision Averages: 0.6 Query 99: 0.7468 at 0.00 0.8008 Query 100: 0.5821 at 0.10 0.7185 Query 101: 0.0185 at 0.20 0.6413 Precision Query 102: 0.4769 at 0.30 0.5938 0.4 Query 103: 0.5317 at 0.40 0.5655 Query 104: 0.0292 at 0.50 0.5356 Query 105: 0.5753 at 0.60 0.4489 Query 106: 0.2539 at 0.70 0.3976 0.2 Query 107: 0.1773 at 0.80 0.3189 Query 108: 0.1652 at 0.90 0.2587 Query 109: 0.5105 at 1.00 0.1885 Query 110: 0.9308 0.0 Query 111: 0.2343 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5998 for all rel. documents Query 113: 0.1439 0.4750 Recall Query 114: 0.3635 Precision: Query 115: 0.5995 At 5 docs: 0.5429 Run bky2moit Query 116: 0.5880 At 10 docs: 0.5061 Query 117: 0.4959 At 15 docs: 0.4599 Comparison to median by topic Query 118: 0.2594 At 20 docs: 0.4112 +1.0 Query 119: 0.8294 At 30 docs: 0.3456 Query 121: 0.0994 At 100 docs: 0.1616 Query 122: 0.1485 At 200 docs: 0.0939 Query 123: 0.9051 At 500 docs: 0.0407 Query 124: 0.1450 At 1000 docs: 0.0209 +0.5 Query 125: 0.7757 Query 126: 0.2031 R-Precision (prec. after R Query 127: 0.4769 (=num_rel) docs. retrieved): Query 128: 0.1561 Exact: 0.4572 Query 129: 0.5056 0.0 Query 130: 0.4349 Query 131: 0.4116 Query 132: 0.5556 Query 133: 0.7171 Query 134: 0.9267 Difference (average precision) Query 135: 0.8042 −0.5 Query 136: 0.9516 Query 137: 0.5667 Query 138: 0.6309 Query 139: 0.5138 Query 140: 0.3022 −1.0 91 100 110 120 130 140 Topic number Univ. Exeter, Monolingual Italian, Auto, TD Run exeitmgcntmn Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.5111 Total number of documents Query 92: 0.3705 over all queries Query 93: 0.9500 Retrieved: 49000 0.8 Query 94: 0.5320 Relevant: 1072 Query 95: 0.0757 Rel_ret: 978 Query 96: 0.4255 Query 97: 0.4854 Interpolated Recall - Query 98: 0.6015 Precision Averages: 0.6 Query 99: 0.6412 at 0.00 0.7440 Query 100: 0.4947 at 0.10 0.6569 Query 101: 0.1134 at 0.20 0.5672 Precision Query 102: 0.3893 at 0.30 0.5104 0.4 Query 103: 0.4857 at 0.40 0.4553 Query 104: 0.0654 at 0.50 0.4283 Query 105: 0.6476 at 0.60 0.3648 Query 106: 0.3416 at 0.70 0.3248 0.2 Query 107: 0.2317 at 0.80 0.2753 Query 108: 0.1076 at 0.90 0.2284 Query 109: 0.2089 at 1.00 0.1765 Query 110: 0.9608 0.0 Query 111: 0.2730 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6915 for all rel. documents Query 113: 0.0302 0.4107 Recall Query 114: 0.4317 Precision: Query 115: 0.7619 At 5 docs: 0.4735 Run exeitmgcntmn Query 116: 0.2065 At 10 docs: 0.4306 Query 117: 0.4986 At 15 docs: 0.3946 Comparison to median by topic Query 118: 0.0951 At 20 docs: 0.3561 +1.0 Query 119: 0.6251 At 30 docs: 0.3014 Query 121: 0.1073 At 100 docs: 0.1410 Query 122: 0.0340 At 200 docs: 0.0827 Query 123: 0.9747 At 500 docs: 0.0377 Query 124: 0.2091 At 1000 docs: 0.0200 +0.5 Query 125: 0.2395 Query 126: 0.5589 R-Precision (prec. after R Query 127: 0.5193 (=num_rel) docs. retrieved): Query 128: 0.0021 Exact: 0.3851 Query 129: 0.3258 0.0 Query 130: 0.3691 Query 131: 0.0069 Query 132: 0.4874 Query 133: 0.7721 Query 134: 1.0000 Difference (average precision) Query 135: 0.5306 −0.5 Query 136: 1.0000 Query 137: 0.0642 Query 138: 0.2900 Query 139: 0.0467 Query 140: 0.3327 −1.0 91 100 110 120 130 140 Topic number Univ. Exeter, Monolingual Italian, Auto, TD Run exeitmgmn Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.5132 Total number of documents Query 92: 0.4063 over all queries Query 93: 0.8935 Retrieved: 49000 0.8 Query 94: 0.5321 Relevant: 1072 Query 95: 0.1044 Rel_ret: 1002 Query 96: 0.4927 Query 97: 0.5099 Interpolated Recall - Query 98: 0.5896 Precision Averages: 0.6 Query 99: 0.6445 at 0.00 0.7354 Query 100: 0.4855 at 0.10 0.6333 Query 101: 0.0979 at 0.20 0.5650 Precision Query 102: 0.4029 at 0.30 0.5355 0.4 Query 103: 0.5124 at 0.40 0.4765 Query 104: 0.0606 at 0.50 0.4475 Query 105: 0.6345 at 0.60 0.3831 Query 106: 0.2334 at 0.70 0.3321 0.2 Query 107: 0.3374 at 0.80 0.2893 Query 108: 0.1382 at 0.90 0.2354 Query 109: 0.2485 at 1.00 0.1700 Query 110: 0.9524 0.0 Query 111: 0.2681 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6822 for all rel. documents Query 113: 0.0312 0.4206 Recall Query 114: 0.4172 Precision: Query 115: 0.8178 At 5 docs: 0.4980 Run exeitmgmn Query 116: 0.2252 At 10 docs: 0.4347 Query 117: 0.3889 At 15 docs: 0.3973 Comparison to median by topic Query 118: 0.0994 At 20 docs: 0.3653 +1.0 Query 119: 0.8352 At 30 docs: 0.3109 Query 121: 0.1474 At 100 docs: 0.1459 Query 122: 0.0975 At 200 docs: 0.0837 Query 123: 0.9806 At 500 docs: 0.0381 Query 124: 0.2441 At 1000 docs: 0.0204 +0.5 Query 125: 0.0946 Query 126: 0.3360 R-Precision (prec. after R Query 127: 0.5447 (=num_rel) docs. retrieved): Query 128: 0.0030 Exact: 0.3983 Query 129: 0.3242 0.0 Query 130: 0.3651 Query 131: 0.0158 Query 132: 0.4755 Query 133: 0.6414 Query 134: 0.9250 Difference (average precision) Query 135: 0.7078 −0.5 Query 136: 1.0000 Query 137: 0.0097 Query 138: 0.7396 Query 139: 0.0566 Query 140: 0.3440 −1.0 91 100 110 120 130 140 Topic number Univ. Exeter, Monolingual Italian, Auto, TD Run exeitorgmn Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.5129 Total number of documents Query 92: 0.3802 over all queries Query 93: 0.9576 Retrieved: 49000 0.8 Query 94: 0.5356 Relevant: 1072 Query 95: 0.1777 Rel_ret: 1004 Query 96: 0.4348 Query 97: 0.5099 Interpolated Recall - Query 98: 0.5391 Precision Averages: 0.6 Query 99: 0.6861 at 0.00 0.7474 Query 100: 0.4159 at 0.10 0.6828 Query 101: 0.2105 at 0.20 0.6187 Precision Query 102: 0.3502 at 0.30 0.5650 0.4 Query 103: 0.5749 at 0.40 0.5166 Query 104: 0.0567 at 0.50 0.4954 Query 105: 0.6321 at 0.60 0.4201 Query 106: 0.4123 at 0.70 0.3594 0.2 Query 107: 0.2690 at 0.80 0.3089 Query 108: 0.1382 at 0.90 0.2635 Query 109: 0.3532 at 1.00 0.1957 Query 110: 0.9443 0.0 Query 111: 0.3496 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6899 for all rel. documents Query 113: 0.0275 0.4531 Recall Query 114: 0.4176 Precision: Query 115: 0.8161 At 5 docs: 0.5224 Run exeitorgmn Query 116: 0.6577 At 10 docs: 0.4796 Query 117: 0.4602 At 15 docs: 0.4340 Comparison to median by topic Query 118: 0.0941 At 20 docs: 0.3837 +1.0 Query 119: 0.8107 At 30 docs: 0.3279 Query 121: 0.1200 At 100 docs: 0.1531 Query 122: 0.0471 At 200 docs: 0.0876 Query 123: 0.9773 At 500 docs: 0.0392 Query 124: 0.1626 At 1000 docs: 0.0205 +0.5 Query 125: 0.5979 Query 126: 0.4963 R-Precision (prec. after R Query 127: 0.4639 (=num_rel) docs. retrieved): Query 128: 0.0033 Exact: 0.4305 Query 129: 0.2920 0.0 Query 130: 0.3585 Query 131: 0.0118 Query 132: 0.4755 Query 133: 0.7985 Query 134: 0.9667 Difference (average precision) Query 135: 0.7079 −0.5 Query 136: 1.0000 Query 137: 0.1876 Query 138: 0.7439 Query 139: 0.0584 Query 140: 0.3158 −1.0 91 100 110 120 130 140 Topic number Univ. Exeter, Monolingual Italian, Auto, TD Run exeitwtpimn Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.5189 Total number of documents Query 92: 0.3483 over all queries Query 93: 0.9008 Retrieved: 49000 0.8 Query 94: 0.5181 Relevant: 1072 Query 95: 0.1401 Rel_ret: 993 Query 96: 0.4120 Query 97: 0.4702 Interpolated Recall - Query 98: 0.2949 Precision Averages: 0.6 Query 99: 0.4699 at 0.00 0.7364 Query 100: 0.4968 at 0.10 0.6732 Query 101: 0.1652 at 0.20 0.5748 Precision Query 102: 0.2800 at 0.30 0.5070 0.4 Query 103: 0.4996 at 0.40 0.4590 Query 104: 0.0850 at 0.50 0.4308 Query 105: 0.6909 at 0.60 0.3684 Query 106: 0.2627 at 0.70 0.3301 0.2 Query 107: 0.2716 at 0.80 0.2836 Query 108: 0.0829 at 0.90 0.2203 Query 109: 0.2165 at 1.00 0.1641 Query 110: 0.9299 0.0 Query 111: 0.4274 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6194 for all rel. documents Query 113: 0.0199 0.4141 Recall Query 114: 0.3872 Precision: Query 115: 0.6925 At 5 docs: 0.4939 Run exeitwtpimn Query 116: 0.7614 At 10 docs: 0.4347 Query 117: 0.3918 At 15 docs: 0.3850 Comparison to median by topic Query 118: 0.1365 At 20 docs: 0.3561 +1.0 Query 119: 0.7605 At 30 docs: 0.3102 Query 121: 0.2381 At 100 docs: 0.1455 Query 122: 0.0868 At 200 docs: 0.0850 Query 123: 0.9642 At 500 docs: 0.0385 Query 124: 0.1219 At 1000 docs: 0.0203 +0.5 Query 125: 0.3837 Query 126: 0.4325 R-Precision (prec. after R Query 127: 0.4476 (=num_rel) docs. retrieved): Query 128: 0.0056 Exact: 0.3963 Query 129: 0.3708 0.0 Query 130: 0.2994 Query 131: 0.0126 Query 132: 0.5111 Query 133: 0.7631 Query 134: 0.9250 Difference (average precision) Query 135: 0.8445 −0.5 Query 136: 0.8420 Query 137: 0.0865 Query 138: 0.2250 Query 139: 0.0615 Query 140: 0.4156 −1.0 91 100 110 120 130 140 Topic number Fondazione Ugo Bordini, Monolingual Italian, Auto, TD Run fub02 Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.2987 Total number of documents Query 92: 0.4191 over all queries Query 93: 0.9576 Retrieved: 49000 0.8 Query 94: 0.5216 Relevant: 1072 Query 95: 0.0870 Rel_ret: 1036 Query 96: 0.5162 Query 97: 0.9167 Interpolated Recall - 0.6 Query 98: 0.7548 Precision Averages: Query 99: 0.6478 at 0.00 0.7893 Query 100: 0.4358 at 0.10 0.7077 Precision Query 101: 0.0437 at 0.20 0.6640 Query 102: 0.3369 at 0.30 0.6271 0.4 Query 103: 0.7359 at 0.40 0.5732 Query 104: 0.4149 at 0.50 0.5384 Query 105: 0.5523 at 0.60 0.4749 Query 106: 0.2576 at 0.70 0.4269 0.2 Query 107: 0.1844 at 0.80 0.3726 Query 108: 0.1527 at 0.90 0.3034 Query 109: 0.5263 at 1.00 0.2249 Query 110: 0.6705 0.0 Query 111: 0.2726 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.7228 for all rel. documents Query 113: 0.2337 0.5019 Recall Query 114: 0.4726 Precision: Query 115: 0.7345 At 5 docs: 0.5347 Run fub02 Query 116: 0.8711 At 10 docs: 0.4959 Comparison to median by topic Query 117: 0.4272 At 15 docs: 0.4435 +1.0 Query 118: 0.1667 At 20 docs: 0.4122 Query 119: 0.8336 At 30 docs: 0.3537 Query 121: 0.1407 At 100 docs: 0.1739 Query 122: 0.1251 At 200 docs: 0.0965 Query 123: 0.9563 At 500 docs: 0.0412 Query 124: 0.4502 At 1000 docs: 0.0211 +0.5 Query 125: 0.7310 Query 126: 0.6264 R-Precision (prec. after R Query 127: 0.4762 (=num_rel) docs. retrieved): Query 128: 0.0675 Exact: 0.4659 Query 129: 0.3042 0.0 Query 130: 0.3822 Query 131: 0.2438 Query 132: 0.8095 Query 133: 0.7179 Query 134: 0.8000 Difference (average precision) Query 135: 0.9018 −0.5 Query 136: 0.9881 Query 137: 0.5576 Query 138: 0.4400 Query 139: 0.3610 Query 140: 0.3493 −1.0 91 100 110 120 130 140 Topic number Fondazione Ugo Bordini, Monolingual Italian, Auto, TD Run fub02b Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.2987 Total number of documents Query 92: 0.3649 over all queries Query 93: 0.9576 Retrieved: 49000 0.8 Query 94: 0.5216 Relevant: 1072 Query 95: 0.0915 Rel_ret: 1033 Query 96: 0.5162 Query 97: 0.9167 Interpolated Recall - 0.6 Query 98: 0.7548 Precision Averages: Query 99: 0.6478 at 0.00 0.7631 Query 100: 0.4648 at 0.10 0.6816 Precision Query 101: 0.0354 at 0.20 0.6433 Query 102: 0.3369 at 0.30 0.6113 0.4 Query 103: 0.7405 at 0.40 0.5568 Query 104: 0.4821 at 0.50 0.5225 Query 105: 0.5523 at 0.60 0.4621 Query 106: 0.2576 at 0.70 0.4172 0.2 Query 107: 0.1844 at 0.80 0.3631 Query 108: 0.1527 at 0.90 0.2946 Query 109: 0.5263 at 1.00 0.2211 Query 110: 0.6705 0.0 Query 111: 0.2818 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.7228 for all rel. documents Query 113: 0.2253 0.4872 Recall Query 114: 0.3713 Precision: Query 115: 0.7345 At 5 docs: 0.5102 Run fub02b Query 116: 0.8711 At 10 docs: 0.4878 Comparison to median by topic Query 117: 0.4272 At 15 docs: 0.4313 +1.0 Query 118: 0.1667 At 20 docs: 0.4020 Query 119: 0.8423 At 30 docs: 0.3415 Query 121: 0.1407 At 100 docs: 0.1706 Query 122: 0.1251 At 200 docs: 0.0951 Query 123: 0.9563 At 500 docs: 0.0410 Query 124: 0.2587 At 1000 docs: 0.0211 +0.5 Query 125: 0.7310 Query 126: 0.6264 R-Precision (prec. after R Query 127: 0.4762 (=num_rel) docs. retrieved): Query 128: 0.0675 Exact: 0.4548 Query 129: 0.3113 0.0 Query 130: 0.4110 Query 131: 0.1679 Query 132: 0.8095 Query 133: 0.7179 Query 134: 0.8000 Difference (average precision) Query 135: 0.9018 −0.5 Query 136: 0.9881 Query 137: 0.5573 Query 138: 0.0290 Query 139: 0.3306 Query 140: 0.3493 −1.0 91 100 110 120 130 140 Topic number Fondazione Ugo Bordini, Monolingual Italian, Auto, TD Run fub02bl Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.5144 Total number of documents Query 92: 0.3593 over all queries Query 93: 0.9576 Retrieved: 49000 0.8 Query 94: 0.5216 Relevant: 1072 Query 95: 0.0915 Rel_ret: 1034 Query 96: 0.5438 Query 97: 0.9167 Interpolated Recall - 0.6 Query 98: 0.7548 Precision Averages: Query 99: 0.6450 at 0.00 0.7810 Query 100: 0.4784 at 0.10 0.7076 Precision Query 101: 0.0354 at 0.20 0.6721 Query 102: 0.3521 at 0.30 0.6210 0.4 Query 103: 0.7487 at 0.40 0.5634 Query 104: 0.4821 at 0.50 0.5293 Query 105: 0.5523 at 0.60 0.4554 Query 106: 0.2630 at 0.70 0.4120 0.2 Query 107: 0.1614 at 0.80 0.3652 Query 108: 0.1533 at 0.90 0.2929 Query 109: 0.4970 at 1.00 0.2194 Query 110: 0.6705 0.0 Query 111: 0.2818 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.7228 for all rel. documents Query 113: 0.2253 0.4947 Recall Query 114: 0.3027 Precision: Query 115: 0.7345 At 5 docs: 0.5224 Run fub02bl Query 116: 0.8711 At 10 docs: 0.4939 Comparison to median by topic Query 117: 0.3684 At 15 docs: 0.4313 +1.0 Query 118: 0.2631 At 20 docs: 0.4000 Query 119: 0.8462 At 30 docs: 0.3422 Query 121: 0.1685 At 100 docs: 0.1714 Query 122: 0.1251 At 200 docs: 0.0951 Query 123: 0.9669 At 500 docs: 0.0410 Query 124: 0.2583 At 1000 docs: 0.0211 +0.5 Query 125: 0.7404 Query 126: 0.6264 R-Precision (prec. after R Query 127: 0.4762 (=num_rel) docs. retrieved): Query 128: 0.0675 Exact: 0.4683 Query 129: 0.3113 0.0 Query 130: 0.4110 Query 131: 0.1679 Query 132: 0.8095 Query 133: 0.7179 Query 134: 0.7422 Difference (average precision) Query 135: 0.9304 −0.5 Query 136: 0.9812 Query 137: 0.5573 Query 138: 0.0290 Query 139: 0.4818 Query 140: 0.3589 −1.0 91 100 110 120 130 140 Topic number Fondazione Ugo Bordini, Monolingual Italian, Auto, TD Run fub02l Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.5144 Total number of documents Query 92: 0.4058 over all queries Query 93: 0.9576 Retrieved: 49000 0.8 Query 94: 0.5216 Relevant: 1072 Query 95: 0.0870 Rel_ret: 1036 Query 96: 0.5438 Query 97: 0.9167 Interpolated Recall - 0.6 Query 98: 0.7548 Precision Averages: Query 99: 0.6450 at 0.00 0.8082 Query 100: 0.4392 at 0.10 0.7350 Precision Query 101: 0.0437 at 0.20 0.6970 Query 102: 0.3521 at 0.30 0.6378 0.4 Query 103: 0.7431 at 0.40 0.5767 Query 104: 0.4149 at 0.50 0.5398 Query 105: 0.5523 at 0.60 0.4619 Query 106: 0.2630 at 0.70 0.4247 0.2 Query 107: 0.1614 at 0.80 0.3745 Query 108: 0.1533 at 0.90 0.3016 Query 109: 0.4970 at 1.00 0.2242 Query 110: 0.6705 0.0 Query 111: 0.2726 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.7228 for all rel. documents Query 113: 0.2337 0.5088 Recall Query 114: 0.4563 Precision: Query 115: 0.7345 At 5 docs: 0.5510 Run fub02l Query 116: 0.8711 At 10 docs: 0.5061 Comparison to median by topic Query 117: 0.3684 At 15 docs: 0.4422 +1.0 Query 118: 0.2631 At 20 docs: 0.4082 Query 119: 0.8505 At 30 docs: 0.3503 Query 121: 0.1685 At 100 docs: 0.1745 Query 122: 0.1251 At 200 docs: 0.0965 Query 123: 0.9669 At 500 docs: 0.0412 Query 124: 0.4275 At 1000 docs: 0.0211 +0.5 Query 125: 0.7404 Query 126: 0.6264 R-Precision (prec. after R Query 127: 0.4762 (=num_rel) docs. retrieved): Query 128: 0.0675 Exact: 0.4836 Query 129: 0.3042 0.0 Query 130: 0.3814 Query 131: 0.2438 Query 132: 0.8095 Query 133: 0.7179 Query 134: 0.7422 Difference (average precision) Query 135: 0.9304 −0.5 Query 136: 0.9812 Query 137: 0.5576 Query 138: 0.4479 Query 139: 0.4469 Query 140: 0.3589 −1.0 91 100 110 120 130 140 Topic number Hummingbird, Monolingual Italian, Auto, TD Run humIT02 Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.3250 Total number of documents Query 92: 0.3639 over all queries Query 93: 0.8937 Retrieved: 49000 0.8 Query 94: 0.4614 Relevant: 1072 Query 95: 0.0606 Rel_ret: 1004 Query 96: 0.6006 Query 97: 0.5932 Interpolated Recall - 0.6 Query 98: 0.7007 Precision Averages: Query 99: 0.3650 at 0.00 0.7209 Query 100: 0.4133 at 0.10 0.6892 Precision Query 101: 0.0638 at 0.20 0.6258 Query 102: 0.3302 at 0.30 0.5289 0.4 Query 103: 0.6139 at 0.40 0.4775 Query 104: 0.0528 at 0.50 0.4361 Query 105: 0.6185 at 0.60 0.3548 Query 106: 0.3080 at 0.70 0.2943 0.2 Query 107: 0.1166 at 0.80 0.2516 Query 108: 0.1122 at 0.90 0.1886 Query 109: 0.2655 at 1.00 0.1285 Query 110: 0.8291 0.0 Query 111: 0.1224 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.7568 for all rel. documents Query 113: 0.0878 0.4075 Recall Query 114: 0.4014 Precision: Query 115: 0.4872 At 5 docs: 0.5020 Run humIT02 Query 116: 0.6152 At 10 docs: 0.4490 Comparison to median by topic Query 117: 0.4037 At 15 docs: 0.3918 +1.0 Query 118: 0.1428 At 20 docs: 0.3582 Query 119: 0.7004 At 30 docs: 0.3068 Query 121: 0.1032 At 100 docs: 0.1496 Query 122: 0.0518 At 200 docs: 0.0866 Query 123: 0.9350 At 500 docs: 0.0394 Query 124: 0.3137 At 1000 docs: 0.0205 +0.5 Query 125: 0.6763 Query 126: 0.4659 R-Precision (prec. after R Query 127: 0.4407 (=num_rel) docs. retrieved): Query 128: 0.0733 Exact: 0.4100 Query 129: 0.2611 0.0 Query 130: 0.3308 Query 131: 0.3624 Query 132: 0.2688 Query 133: 0.4572 Query 134: 0.8600 Difference (average precision) Query 135: 0.4600 −0.5 Query 136: 0.8216 Query 137: 0.4277 Query 138: 0.3085 Query 139: 0.2486 Query 140: 0.2969 −1.0 91 100 110 120 130 140 Topic number ITC-IRST, Monolingual Italian, Auto, TD Run IRSTit1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.2668 Total number of documents Query 92: 0.4086 over all queries Query 93: 0.9176 Retrieved: 47599 0.8 Query 94: 0.5213 Relevant: 1072 Query 95: 0.0687 Rel_ret: 1028 Query 96: 0.4485 Query 97: 0.8929 Interpolated Recall - 0.6 Query 98: 0.2952 Precision Averages: Query 99: 0.7797 at 0.00 0.7824 Query 100: 0.4966 at 0.10 0.7415 Precision Query 101: 0.4554 at 0.20 0.6803 Query 102: 0.3741 at 0.30 0.5998 0.4 Query 103: 0.6692 at 0.40 0.5604 Query 104: 0.0458 at 0.50 0.5192 Query 105: 0.6614 at 0.60 0.4671 Query 106: 0.3085 at 0.70 0.4259 0.2 Query 107: 0.3596 at 0.80 0.3355 Query 108: 0.1257 at 0.90 0.2726 Query 109: 0.4392 at 1.00 0.2070 Query 110: 0.9240 0.0 Query 111: 0.3542 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.7524 for all rel. documents Query 113: 0.0690 0.4920 Recall Query 114: 0.5011 Precision: Query 115: 0.6266 At 5 docs: 0.5469 Run IRSTit1 Query 116: 0.5792 At 10 docs: 0.4878 Comparison to median by topic Query 117: 0.4675 At 15 docs: 0.4490 +1.0 Query 118: 0.2580 At 20 docs: 0.4082 Query 119: 0.8216 At 30 docs: 0.3435 Query 121: 0.1333 At 100 docs: 0.1694 Query 122: 0.0611 At 200 docs: 0.0954 Query 123: 0.9552 At 500 docs: 0.0409 Query 124: 0.4706 At 1000 docs: 0.0210 +0.5 Query 125: 0.7600 Query 126: 0.6942 R-Precision (prec. after R Query 127: 0.4915 (=num_rel) docs. retrieved): Query 128: 0.1325 Exact: 0.4470 Query 129: 0.2359 0.0 Query 130: 0.3891 Query 131: 0.3391 Query 132: 0.4028 Query 133: 0.7419 Query 134: 0.5544 Difference (average precision) Query 135: 0.8734 −0.5 Query 136: 0.9936 Query 137: 0.6823 Query 138: 0.8824 Query 139: 0.1526 Query 140: 0.2713 −1.0 91 100 110 120 130 140 Topic number Océ, Monolingual Italian, Auto, TD Run oce02monIT Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.1455 Total number of documents Query 92: 0.3744 over all queries Query 93: 0.8982 Retrieved: 49000 0.8 Query 94: 0.5159 Relevant: 1072 Query 95: 0.1478 Rel_ret: 995 Query 96: 0.3567 Query 97: 0.5417 Interpolated Recall - 0.6 Query 98: 0.2618 Precision Averages: Query 99: 0.4935 at 0.00 0.6834 Query 100: 0.4610 at 0.10 0.6006 Precision Query 101: 0.0747 at 0.20 0.5577 Query 102: 0.2892 at 0.30 0.5176 0.4 Query 103: 0.3851 at 0.40 0.4571 Query 104: 0.0456 at 0.50 0.4062 Query 105: 0.4518 at 0.60 0.3326 Query 106: 0.3047 at 0.70 0.2719 0.2 Query 107: 0.1572 at 0.80 0.2216 Query 108: 0.0892 at 0.90 0.1579 Query 109: 0.2276 at 1.00 0.1151 Query 110: 0.8397 0.0 Query 111: 0.1236 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.7412 for all rel. documents Query 113: 0.0471 0.3682 Recall Query 114: 0.5308 Precision: Query 115: 0.5192 At 5 docs: 0.4898 Run oce02monIT Query 116: 0.6213 At 10 docs: 0.4347 Comparison to median by topic Query 117: 0.3160 At 15 docs: 0.3905 +1.0 Query 118: 0.0882 At 20 docs: 0.3469 Query 119: 0.7167 At 30 docs: 0.2932 Query 121: 0.1024 At 100 docs: 0.1427 Query 122: 0.0381 At 200 docs: 0.0840 Query 123: 0.9269 At 500 docs: 0.0383 Query 124: 0.2082 At 1000 docs: 0.0203 +0.5 Query 125: 0.5842 Query 126: 0.3778 R-Precision (prec. after R Query 127: 0.4477 (=num_rel) docs. retrieved): Query 128: 0.0244 Exact: 0.3899 Query 129: 0.3365 0.0 Query 130: 0.2535 Query 131: 0.2996 Query 132: 0.1428 Query 133: 0.4537 Query 134: 0.8769 Difference (average precision) Query 135: 0.4330 −0.5 Query 136: 0.6913 Query 137: 0.1703 Query 138: 0.3831 Query 139: 0.2805 Query 140: 0.2446 −1.0 91 100 110 120 130 140 Topic number Océ, Monolingual Italian, Auto, T Run oce02monITto Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.0019 Total number of documents Query 92: 0.2638 over all queries Query 93: 0.6667 Retrieved: 43290 0.8 Query 94: 0.3062 Relevant: 1072 Query 95: 0.0162 Rel_ret: 900 Query 96: 0.5022 Query 97: 0.9500 Interpolated Recall - 0.6 Query 98: 0.0089 Precision Averages: Query 99: 0.3103 at 0.00 0.7325 Query 100: 0.4114 at 0.10 0.5766 Precision Query 101: 0.4321 at 0.20 0.5210 Query 102: 0.3219 at 0.30 0.4294 0.4 Query 103: 0.3111 at 0.40 0.3858 Query 104: 0.0414 at 0.50 0.3595 Query 105: 0.4583 at 0.60 0.2983 Query 106: 0.0740 at 0.70 0.2145 0.2 Query 107: 0.0766 at 0.80 0.1782 Query 108: 0.1067 at 0.90 0.1407 Query 109: 0.2235 at 1.00 0.1057 Query 110: 0.3078 0.0 Query 111: 0.1105 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6312 for all rel. documents Query 113: 0.0527 0.3404 Recall Query 114: 0.1261 Precision: Query 115: 0.6194 At 5 docs: 0.4653 Run oce02monITto Query 116: 0.6389 At 10 docs: 0.3837 Comparison to median by topic Query 117: 0.1424 At 15 docs: 0.3361 +1.0 Query 118: 0.1306 At 20 docs: 0.2949 Query 119: 0.2679 At 30 docs: 0.2476 Query 121: 0.2173 At 100 docs: 0.1167 Query 122: 0.0497 At 200 docs: 0.0720 Query 123: 0.9588 At 500 docs: 0.0343 Query 124: 0.3950 At 1000 docs: 0.0184 +0.5 Query 125: 0.5822 Query 126: 0.5120 R-Precision (prec. after R Query 127: 0.6582 (=num_rel) docs. retrieved): Query 128: 0.0315 Exact: 0.3479 Query 129: 0.0679 0.0 Query 130: 0.4092 Query 131: 0.1723 Query 132: 0.3833 Query 133: 0.4254 Query 134: 0.7196 Difference (average precision) Query 135: 0.5043 −0.5 Query 136: 0.9936 Query 137: 0.3900 Query 138: 0.1409 Query 139: 0.3745 Query 140: 0.1846 −1.0 91 100 110 120 130 140 Topic number IMS Univ. Padova, Monolingual Italian, Auto, TD Run PDDN Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.2704 Total number of documents Query 92: 0.3841 over all queries Query 93: 0.7981 Retrieved: 49000 0.8 Query 94: 0.4679 Relevant: 1072 Query 95: 0.0576 Rel_ret: 887 Query 96: 0.2125 Query 97: 0.5655 Interpolated Recall - 0.6 Query 98: 0.3074 Precision Averages: Query 99: 0.2298 at 0.00 0.7122 Query 100: 0.4383 at 0.10 0.5809 Precision Query 101: 0.1427 at 0.20 0.5182 Query 102: 0.2686 at 0.30 0.4089 0.4 Query 103: 0.4841 at 0.40 0.3465 Query 104: 0.0609 at 0.50 0.3059 Query 105: 0.4519 at 0.60 0.2697 Query 106: 0.2007 at 0.70 0.2146 0.2 Query 107: 0.0530 at 0.80 0.1621 Query 108: 0.0896 at 0.90 0.1298 Query 109: 0.2682 at 1.00 0.0848 Query 110: 0.8771 0.0 Query 111: 0.0838 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.7142 for all rel. documents Query 113: 0.1361 0.3193 Recall Query 114: 0.1834 Precision: Query 115: 0.3112 At 5 docs: 0.4367 Run PDDN Query 116: 0.8122 At 10 docs: 0.3959 Comparison to median by topic Query 117: 0.4284 At 15 docs: 0.3456 +1.0 Query 118: 0.0136 At 20 docs: 0.3020 Query 119: 0.7398 At 30 docs: 0.2429 Query 121: 0.0670 At 100 docs: 0.1194 Query 122: 0.0012 At 200 docs: 0.0715 Query 123: 0.0000 At 500 docs: 0.0338 Query 124: 0.2589 At 1000 docs: 0.0181 +0.5 Query 125: 0.6216 Query 126: 0.3529 R-Precision (prec. after R Query 127: 0.4061 (=num_rel) docs. retrieved): Query 128: 0.0373 Exact: 0.3367 Query 129: 0.3013 0.0 Query 130: 0.2598 Query 131: 0.2188 Query 132: 0.2444 Query 133: 0.0800 Query 134: 0.8711 Difference (average precision) Query 135: 0.3403 −0.5 Query 136: 0.6501 Query 137: 0.1250 Query 138: 0.4417 Query 139: 0.2261 Query 140: 0.0905 −1.0 91 100 110 120 130 140 Topic number IMS Univ. Padova, Monolingual Italian, Auto, TD Run PDDP Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.3037 Total number of documents Query 92: 0.3803 over all queries Query 93: 0.8151 Retrieved: 49000 0.8 Query 94: 0.4574 Relevant: 1072 Query 95: 0.0740 Rel_ret: 914 Query 96: 0.3033 Query 97: 0.3765 Interpolated Recall - 0.6 Query 98: 0.1159 Precision Averages: Query 99: 0.4224 at 0.00 0.7047 Query 100: 0.3654 at 0.10 0.6178 Precision Query 101: 0.3196 at 0.20 0.5704 Query 102: 0.3031 at 0.30 0.4728 0.4 Query 103: 0.3946 at 0.40 0.3907 Query 104: 0.0373 at 0.50 0.3445 Query 105: 0.5105 at 0.60 0.2931 Query 106: 0.2278 at 0.70 0.2316 0.2 Query 107: 0.2070 at 0.80 0.1738 Query 108: 0.1248 at 0.90 0.1219 Query 109: 0.2641 at 1.00 0.0698 Query 110: 0.8834 0.0 Query 111: 0.1115 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.7184 for all rel. documents Query 113: 0.0551 0.3419 Recall Query 114: 0.2878 Precision: Query 115: 0.4295 At 5 docs: 0.4571 Run PDDP Query 116: 0.7969 At 10 docs: 0.3837 Comparison to median by topic Query 117: 0.3929 At 15 docs: 0.3333 +1.0 Query 118: 0.0294 At 20 docs: 0.3031 Query 119: 0.6758 At 30 docs: 0.2565 Query 121: 0.2401 At 100 docs: 0.1263 Query 122: 0.0002 At 200 docs: 0.0750 Query 123: 0.0000 At 500 docs: 0.0344 Query 124: 0.1212 At 1000 docs: 0.0187 +0.5 Query 125: 0.6736 Query 126: 0.3380 R-Precision (prec. after R Query 127: 0.4774 (=num_rel) docs. retrieved): Query 128: 0.0254 Exact: 0.3579 Query 129: 0.2879 0.0 Query 130: 0.2470 Query 131: 0.1984 Query 132: 0.5588 Query 133: 0.1822 Query 134: 0.7455 Difference (average precision) Query 135: 0.2362 −0.5 Query 136: 0.6376 Query 137: 0.4337 Query 138: 0.5197 Query 139: 0.2914 Query 140: 0.1537 −1.0 91 100 110 120 130 140 Topic number IMS Univ. Padova, Monolingual Italian, Auto, TD Run PDDS2PL Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.2688 Total number of documents Query 92: 0.3533 over all queries Query 93: 0.7900 Retrieved: 49000 0.8 Query 94: 0.5723 Relevant: 1072 Query 95: 0.0553 Rel_ret: 913 Query 96: 0.2493 Query 97: 0.7321 Interpolated Recall - 0.6 Query 98: 0.1275 Precision Averages: Query 99: 0.2859 at 0.00 0.7296 Query 100: 0.2942 at 0.10 0.6053 Precision Query 101: 0.1935 at 0.20 0.5146 Query 102: 0.2225 at 0.30 0.4265 0.4 Query 103: 0.4008 at 0.40 0.3397 Query 104: 0.1660 at 0.50 0.3042 Query 105: 0.5106 at 0.60 0.2504 Query 106: 0.1716 at 0.70 0.1939 0.2 Query 107: 0.1734 at 0.80 0.1505 Query 108: 0.0798 at 0.90 0.1084 Query 109: 0.2648 at 1.00 0.0686 Query 110: 0.9021 0.0 Query 111: 0.0769 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6791 for all rel. documents Query 113: 0.0826 0.3173 Recall Query 114: 0.2008 Precision: Query 115: 0.4916 At 5 docs: 0.4367 Run PDDS2PL Query 116: 0.2654 At 10 docs: 0.3776 Comparison to median by topic Query 117: 0.3736 At 15 docs: 0.3320 +1.0 Query 118: 0.0140 At 20 docs: 0.3000 Query 119: 0.6461 At 30 docs: 0.2449 Query 121: 0.0976 At 100 docs: 0.1214 Query 122: 0.0000 At 200 docs: 0.0720 Query 123: 0.0000 At 500 docs: 0.0349 Query 124: 0.3473 At 1000 docs: 0.0186 +0.5 Query 125: 0.6647 Query 126: 0.3434 R-Precision (prec. after R Query 127: 0.4915 (=num_rel) docs. retrieved): Query 128: 0.1192 Exact: 0.3310 Query 129: 0.0514 0.0 Query 130: 0.2493 Query 131: 0.3411 Query 132: 0.5667 Query 133: 0.1884 Query 134: 0.7619 Difference (average precision) Query 135: 0.2902 −0.5 Query 136: 0.5773 Query 137: 0.0841 Query 138: 0.3321 Query 139: 0.2240 Query 140: 0.1739 −1.0 91 100 110 120 130 140 Topic number IMS Univ. Padova, Monolingual Italian, Auto, TD Run PDDS2PLL3 Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.2688 Total number of documents Query 92: 0.3533 over all queries Query 93: 0.7900 Retrieved: 49000 0.8 Query 94: 0.5723 Relevant: 1072 Query 95: 0.0563 Rel_ret: 911 Query 96: 0.2397 Query 97: 0.7321 Interpolated Recall - 0.6 Query 98: 0.0589 Precision Averages: Query 99: 0.2806 at 0.00 0.7137 Query 100: 0.2942 at 0.10 0.5967 Precision Query 101: 0.0595 at 0.20 0.5241 Query 102: 0.2252 at 0.30 0.4349 0.4 Query 103: 0.4105 at 0.40 0.3499 Query 104: 0.1648 at 0.50 0.3099 Query 105: 0.5103 at 0.60 0.2632 Query 106: 0.1830 at 0.70 0.2064 0.2 Query 107: 0.1743 at 0.80 0.1605 Query 108: 0.0816 at 0.90 0.1053 Query 109: 0.2648 at 1.00 0.0638 Query 110: 0.9021 0.0 Query 111: 0.0569 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.7271 for all rel. documents Query 113: 0.0849 0.3200 Recall Query 114: 0.2519 Precision: Query 115: 0.4916 At 5 docs: 0.4286 Run PDDS2PLL3 Query 116: 0.4120 At 10 docs: 0.3796 Comparison to median by topic Query 117: 0.3479 At 15 docs: 0.3238 +1.0 Query 118: 0.0275 At 20 docs: 0.2980 Query 119: 0.6319 At 30 docs: 0.2456 Query 121: 0.0976 At 100 docs: 0.1224 Query 122: 0.0000 At 200 docs: 0.0732 Query 123: 0.0000 At 500 docs: 0.0350 Query 124: 0.1778 At 1000 docs: 0.0186 +0.5 Query 125: 0.6858 Query 126: 0.3434 R-Precision (prec. after R Query 127: 0.4876 (=num_rel) docs. retrieved): Query 128: 0.1192 Exact: 0.3254 Query 129: 0.4143 0.0 Query 130: 0.2527 Query 131: 0.3260 Query 132: 0.5667 Query 133: 0.1901 Query 134: 0.7619 Difference (average precision) Query 135: 0.3412 −0.5 Query 136: 0.4318 Query 137: 0.0842 Query 138: 0.4000 Query 139: 0.1722 Query 140: 0.1736 −1.0 91 100 110 120 130 140 Topic number City University, Monolingual Italian, Auto, TD Run plit02td Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.0237 Total number of documents Query 92: 0.3381 over all queries Query 93: 0.4250 Retrieved: 49000 0.8 Query 94: 0.4157 Relevant: 1072 Query 95: 0.1025 Rel_ret: 778 Query 96: 0.3320 Query 97: 0.4688 Interpolated Recall - Query 98: 0.0237 Precision Averages: 0.6 Query 99: 0.3667 at 0.00 0.6262 Query 100: 0.0000 at 0.10 0.5419 Query 101: 0.3508 at 0.20 0.4693 Precision Query 102: 0.3319 at 0.30 0.3715 0.4 Query 103: 0.0008 at 0.40 0.3123 Query 104: 0.0754 at 0.50 0.2370 Query 105: 0.3311 at 0.60 0.1859 Query 106: 0.0518 at 0.70 0.1463 0.2 Query 107: 0.0949 at 0.80 0.1049 Query 108: 0.0103 at 0.90 0.0803 Query 109: 0.0909 at 1.00 0.0518 Query 110: 0.7007 0.0 Query 111: 0.2244 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5375 for all rel. documents Query 113: 0.0033 0.2662 Recall Query 114: 0.4775 Precision: Query 115: 0.2825 At 5 docs: 0.3837 Run plit02td Query 116: 0.5246 At 10 docs: 0.3184 Query 117: 0.2682 At 15 docs: 0.2762 Comparison to median by topic Query 118: 0.3559 At 20 docs: 0.2561 +1.0 Query 119: 0.7778 At 30 docs: 0.2184 Query 121: 0.0130 At 100 docs: 0.0996 Query 122: 0.0167 At 200 docs: 0.0600 Query 123: 0.9176 At 500 docs: 0.0286 Query 124: 0.0872 At 1000 docs: 0.0159 +0.5 Query 125: 0.6012 Query 126: 0.4270 R-Precision (prec. after R Query 127: 0.4735 (=num_rel) docs. retrieved): Query 128: 0.0000 Exact: 0.2672 Query 129: 0.0000 0.0 Query 130: 0.2619 Query 131: 0.1692 Query 132: 0.2587 Query 133: 0.4388 Query 134: 0.0432 Difference (average precision) Query 135: 0.1606 −0.5 Query 136: 0.1874 Query 137: 0.2822 Query 138: 0.3644 Query 139: 0.0173 Query 140: 0.3386 −1.0 91 100 110 120 130 140 Topic number IRIT, Toulouse, Monolingual Italian, Auto, TD Run RunMonoIt Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.0867 Total number of documents Query 92: 0.3454 over all queries Query 93: 0.6955 Retrieved: 49000 0.8 Query 94: 0.3684 Relevant: 1072 Query 95: 0.0326 Rel_ret: 934 Query 96: 0.0958 Query 97: 0.6543 Interpolated Recall - 0.6 Query 98: 0.3962 Precision Averages: Query 99: 0.2697 at 0.00 0.7199 Query 100: 0.4131 at 0.10 0.6127 Precision Query 101: 0.3539 at 0.20 0.5404 Query 102: 0.2528 at 0.30 0.4683 0.4 Query 103: 0.5630 at 0.40 0.4405 Query 104: 0.0436 at 0.50 0.4147 Query 105: 0.4359 at 0.60 0.3506 Query 106: 0.2471 at 0.70 0.2581 0.2 Query 107: 0.0390 at 0.80 0.1930 Query 108: 0.0851 at 0.90 0.1149 Query 109: 0.2593 at 1.00 0.0674 Query 110: 0.6356 0.0 Query 111: 0.1790 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6610 for all rel. documents Query 113: 0.1631 0.3618 Recall Query 114: 0.2090 Precision: Query 115: 0.6745 At 5 docs: 0.4490 Run RunMonoIt Query 116: 0.5204 At 10 docs: 0.3878 Comparison to median by topic Query 117: 0.3737 At 15 docs: 0.3660 +1.0 Query 118: 0.0619 At 20 docs: 0.3357 Query 119: 0.5590 At 30 docs: 0.2782 Query 121: 0.0659 At 100 docs: 0.1357 Query 122: 0.1440 At 200 docs: 0.0796 Query 123: 0.9473 At 500 docs: 0.0358 Query 124: 0.3119 At 1000 docs: 0.0191 +0.5 Query 125: 0.5575 Query 126: 0.5088 R-Precision (prec. after R Query 127: 0.4398 (=num_rel) docs. retrieved): Query 128: 0.0039 Exact: 0.3752 Query 129: 0.4261 0.0 Query 130: 0.2855 Query 131: 0.2111 Query 132: 0.6748 Query 133: 0.3430 Query 134: 0.8263 Difference (average precision) Query 135: 0.2681 −0.5 Query 136: 0.7164 Query 137: 0.1305 Query 138: 0.7861 Query 139: 0.2520 Query 140: 0.1552 −1.0 91 100 110 120 130 140 Topic number SICS/Conexor, Monolingual Italian, Auto, TD Run sicsITIT Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.0195 Total number of documents Query 92: 0.1786 over all queries Query 93: 0.7673 Retrieved: 49000 0.8 Query 94: 0.3923 Relevant: 1072 Query 95: 0.0255 Rel_ret: 871 Query 96: 0.1387 Query 97: 0.7986 Interpolated Recall - 0.6 Query 98: 0.0289 Precision Averages: Query 99: 0.2660 at 0.00 0.4859 Query 100: 0.3677 at 0.10 0.4203 Precision Query 101: 0.0531 at 0.20 0.3494 Query 102: 0.1391 at 0.30 0.2960 0.4 Query 103: 0.0983 at 0.40 0.2588 Query 104: 0.0136 at 0.50 0.2344 Query 105: 0.4997 at 0.60 0.1869 Query 106: 0.1294 at 0.70 0.1514 0.2 Query 107: 0.0326 at 0.80 0.0967 Query 108: 0.0139 at 0.90 0.0690 Query 109: 0.0639 at 1.00 0.0484 Query 110: 0.5131 0.0 Query 111: 0.0096 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5349 for all rel. documents Query 113: 0.0187 0.2239 Recall Query 114: 0.1784 Precision: Query 115: 0.1096 At 5 docs: 0.3061 Run sicsITIT Query 116: 0.4490 At 10 docs: 0.2714 Comparison to median by topic Query 117: 0.1036 At 15 docs: 0.2408 +1.0 Query 118: 0.0314 At 20 docs: 0.2163 Query 119: 0.4703 At 30 docs: 0.1755 Query 121: 0.0356 At 100 docs: 0.0943 Query 122: 0.0086 At 200 docs: 0.0609 Query 123: 0.8680 At 500 docs: 0.0315 Query 124: 0.0498 At 1000 docs: 0.0178 +0.5 Query 125: 0.2736 Query 126: 0.0666 R-Precision (prec. after R Query 127: 0.3231 (=num_rel) docs. retrieved): Query 128: 0.0377 Exact: 0.2344 Query 129: 0.2080 0.0 Query 130: 0.2170 Query 131: 0.2025 Query 132: 0.2051 Query 133: 0.0339 Query 134: 0.5478 Difference (average precision) Query 135: 0.0890 −0.5 Query 136: 0.6987 Query 137: 0.0356 Query 138: 0.3961 Query 139: 0.0309 Query 140: 0.2001 −1.0 91 100 110 120 130 140 Topic number SICS/Conexor, Monolingual Italian, Auto, TD Run sicsITITX Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.0121 Total number of documents Query 92: 0.2009 over all queries Query 93: 0.6597 Retrieved: 49000 0.8 Query 94: 0.4733 Relevant: 1072 Query 95: 0.0553 Rel_ret: 844 Query 96: 0.0896 Query 97: 0.5256 Interpolated Recall - 0.6 Query 98: 0.0177 Precision Averages: Query 99: 0.0554 at 0.00 0.4421 Query 100: 0.3614 at 0.10 0.3502 Precision Query 101: 0.0454 at 0.20 0.3025 Query 102: 0.0845 at 0.30 0.2423 0.4 Query 103: 0.0781 at 0.40 0.2103 Query 104: 0.0181 at 0.50 0.1854 Query 105: 0.2801 at 0.60 0.1439 Query 106: 0.0954 at 0.70 0.1137 0.2 Query 107: 0.0204 at 0.80 0.0774 Query 108: 0.0010 at 0.90 0.0465 Query 109: 0.0705 at 1.00 0.0278 Query 110: 0.5025 0.0 Query 111: 0.0215 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4399 for all rel. documents Query 113: 0.0172 0.1836 Recall Query 114: 0.1162 Precision: Query 115: 0.2244 At 5 docs: 0.2449 Run sicsITITX Query 116: 0.1353 At 10 docs: 0.2224 Comparison to median by topic Query 117: 0.0554 At 15 docs: 0.1850 +1.0 Query 118: 0.0173 At 20 docs: 0.1684 Query 119: 0.5131 At 30 docs: 0.1571 Query 121: 0.0191 At 100 docs: 0.0880 Query 122: 0.0071 At 200 docs: 0.0562 Query 123: 0.6470 At 500 docs: 0.0298 Query 124: 0.0148 At 1000 docs: 0.0172 +0.5 Query 125: 0.2972 Query 126: 0.2225 R-Precision (prec. after R Query 127: 0.1936 (=num_rel) docs. retrieved): Query 128: 0.0551 Exact: 0.1799 Query 129: 0.0669 0.0 Query 130: 0.1715 Query 131: 0.0902 Query 132: 0.1238 Query 133: 0.0147 Query 134: 0.8833 Difference (average precision) Query 135: 0.0911 −0.5 Query 136: 0.6397 Query 137: 0.0437 Query 138: 0.0442 Query 139: 0.0219 Query 140: 0.1624 −1.0 91 100 110 120 130 140 Topic number TLR Research, Monolingual Italian, Auto, TD Run tlrit Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.2179 Total number of documents Query 92: 0.3888 over all queries Query 93: 0.9675 Retrieved: 48302 0.8 Query 94: 0.4627 Relevant: 1072 Query 95: 0.0529 Rel_ret: 1007 Query 96: 0.3083 Query 97: 0.7986 Interpolated Recall - 0.6 Query 98: 0.4470 Precision Averages: Query 99: 0.5126 at 0.00 0.7855 Query 100: 0.6016 at 0.10 0.6923 Precision Query 101: 0.1035 at 0.20 0.6150 Query 102: 0.3439 at 0.30 0.5304 0.4 Query 103: 0.6808 at 0.40 0.4855 Query 104: 0.0435 at 0.50 0.4481 Query 105: 0.5652 at 0.60 0.3558 Query 106: 0.3350 at 0.70 0.3122 0.2 Query 107: 0.2564 at 0.80 0.2420 Query 108: 0.1129 at 0.90 0.2033 Query 109: 0.2737 at 1.00 0.1438 Query 110: 0.8548 0.0 Query 111: 0.1190 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.7281 for all rel. documents Query 113: 0.1287 0.4159 Recall Query 114: 0.3309 Precision: Query 115: 0.5972 At 5 docs: 0.5469 Run tlrit Query 116: 0.6840 At 10 docs: 0.4551 Comparison to median by topic Query 117: 0.4960 At 15 docs: 0.4014 +1.0 Query 118: 0.1355 At 20 docs: 0.3622 Query 119: 0.7386 At 30 docs: 0.3027 Query 121: 0.2074 At 100 docs: 0.1473 Query 122: 0.0529 At 200 docs: 0.0868 Query 123: 0.9386 At 500 docs: 0.0395 Query 124: 0.3728 At 1000 docs: 0.0206 +0.5 Query 125: 0.6370 Query 126: 0.4680 R-Precision (prec. after R Query 127: 0.4346 (=num_rel) docs. retrieved): Query 128: 0.1037 Exact: 0.4072 Query 129: 0.1922 0.0 Query 130: 0.4334 Query 131: 0.3008 Query 132: 0.0926 Query 133: 0.4468 Query 134: 0.9267 Difference (average precision) Query 135: 0.4370 −0.5 Query 136: 0.7561 Query 137: 0.5125 Query 138: 0.3229 Query 139: 0.2487 Query 140: 0.2108 −1.0 91 100 110 120 130 140 Topic number University of Amsterdam, Monolingual Italian, Auto, TD Run UAmsC02ItItNGiMO Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.3020 Total number of documents Query 92: 0.3483 over all queries Query 93: 0.7683 Retrieved: 49000 0.8 Query 94: 0.5760 Relevant: 1072 Query 95: 0.1295 Rel_ret: 975 Query 96: 0.6832 Query 97: 0.9500 Interpolated Recall - 0.6 Query 98: 0.6323 Precision Averages: Query 99: 0.7076 at 0.00 0.7752 Query 100: 0.3692 at 0.10 0.6944 Precision Query 101: 0.1330 at 0.20 0.6331 Query 102: 0.3054 at 0.30 0.5437 0.4 Query 103: 0.4569 at 0.40 0.5109 Query 104: 0.0507 at 0.50 0.4802 Query 105: 0.5847 at 0.60 0.4214 Query 106: 0.3369 at 0.70 0.3452 0.2 Query 107: 0.1415 at 0.80 0.2707 Query 108: 0.1745 at 0.90 0.1964 Query 109: 0.1376 at 1.00 0.1426 Query 110: 0.8779 0.0 Query 111: 0.2280 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5681 for all rel. documents Query 113: 0.0149 0.4407 Recall Query 114: 0.3558 Precision: Query 115: 0.4068 At 5 docs: 0.5224 Run UAmsC02ItItNGiMO Query 116: 0.5491 At 10 docs: 0.4449 Comparison to median by topic Query 117: 0.4682 At 15 docs: 0.3959 +1.0 Query 118: 0.2641 At 20 docs: 0.3612 Query 119: 0.8204 At 30 docs: 0.3116 Query 121: 0.1021 At 100 docs: 0.1537 Query 122: 0.0116 At 200 docs: 0.0873 Query 123: 0.9033 At 500 docs: 0.0387 Query 124: 0.2417 At 1000 docs: 0.0199 +0.5 Query 125: 0.6632 Query 126: 0.5939 R-Precision (prec. after R Query 127: 0.5013 (=num_rel) docs. retrieved): Query 128: 0.0711 Exact: 0.4163 Query 129: 0.2315 0.0 Query 130: 0.3479 Query 131: 0.1702 Query 132: 0.7917 Query 133: 0.6299 Query 134: 0.7092 Difference (average precision) Query 135: 0.7853 −0.5 Query 136: 0.9516 Query 137: 0.4176 Query 138: 0.7145 Query 139: 0.0717 Query 140: 0.3428 −1.0 91 100 110 120 130 140 Topic number Université de Neuchâtel, Monolingual Italian, Auto, TD Run UniNEit Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.5192 Total number of documents Query 92: 0.4124 over all queries Query 93: 0.7853 Retrieved: 49000 0.8 Query 94: 0.5283 Relevant: 1072 Query 95: 0.0800 Rel_ret: 1022 Query 96: 0.6166 Query 97: 0.8750 Interpolated Recall - 0.6 Query 98: 0.7319 Precision Averages: Query 99: 0.6303 at 0.00 0.8077 Query 100: 0.4779 at 0.10 0.7277 Precision Query 101: 0.0872 at 0.20 0.6413 Query 102: 0.3439 at 0.30 0.5978 0.4 Query 103: 0.6343 at 0.40 0.5330 Query 104: 0.0517 at 0.50 0.4951 Query 105: 0.6158 at 0.60 0.4289 Query 106: 0.3167 at 0.70 0.3765 0.2 Query 107: 0.2522 at 0.80 0.2928 Query 108: 0.1460 at 0.90 0.2334 Query 109: 0.3038 at 1.00 0.1690 Query 110: 0.8023 0.0 Query 111: 0.3252 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6745 for all rel. documents Query 113: 0.0425 0.4618 Recall Query 114: 0.5216 Precision: Query 115: 0.6618 At 5 docs: 0.5224 Run UniNEit Query 116: 0.5216 At 10 docs: 0.4816 Comparison to median by topic Query 117: 0.3918 At 15 docs: 0.4463 +1.0 Query 118: 0.0614 At 20 docs: 0.4010 Query 119: 0.8188 At 30 docs: 0.3415 Query 121: 0.1896 At 100 docs: 0.1627 Query 122: 0.0942 At 200 docs: 0.0934 Query 123: 0.9319 At 500 docs: 0.0404 Query 124: 0.4265 At 1000 docs: 0.0209 +0.5 Query 125: 0.6925 Query 126: 0.5215 R-Precision (prec. after R Query 127: 0.4774 (=num_rel) docs. retrieved): Query 128: 0.2440 Exact: 0.4391 Query 129: 0.3436 0.0 Query 130: 0.3820 Query 131: 0.4114 Query 132: 0.6095 Query 133: 0.4888 Query 134: 0.7533 Difference (average precision) Query 135: 0.7179 −0.5 Query 136: 0.9453 Query 137: 0.1871 Query 138: 0.3923 Query 139: 0.2483 Query 140: 0.3394 −1.0 91 100 110 120 130 140 Topic number CWI/CNLP, Monolingual Dutch, Auto, T Run AAmoNLt Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0006 Total number of documents Query 92: 0.3669 over all queries Query 93: 0.7248 Retrieved: 38605 0.8 Query 94: 0.7346 Relevant: 1862 Query 95: 0.5428 Rel_ret: 1428 Query 96: 0.3924 Query 97: 0.4686 Interpolated Recall - 0.6 Query 98: 0.7885 Precision Averages: Query 99: 0.1897 at 0.00 0.6770 Query 100: 0.4834 at 0.10 0.6046 Precision Query 101: 0.6525 at 0.20 0.5254 Query 102: 0.6261 at 0.30 0.4678 0.4 Query 103: 0.3270 at 0.40 0.3909 Query 104: 0.4570 at 0.50 0.3449 Query 105: 0.4555 at 0.60 0.2891 Query 106: 0.0666 at 0.70 0.2521 0.2 Query 107: 0.0250 at 0.80 0.2155 Query 108: 0.4992 at 0.90 0.1539 Query 109: 0.0500 at 1.00 0.0964 Query 110: 0.0081 0.0 Query 111: 0.0000 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5256 for all rel. documents Query 113: 0.0013 0.3480 Recall Query 114: 0.1380 Precision: Query 115: 0.0000 At 5 docs: 0.4960 Run AAmoNLt Query 116: 0.1253 At 10 docs: 0.4280 Comparison to median by topic Query 117: 0.0083 At 15 docs: 0.4053 +1.0 Query 118: 0.0293 At 20 docs: 0.3660 Query 119: 0.2202 At 30 docs: 0.3080 Query 120: 0.0373 At 100 docs: 0.1652 Query 121: 0.4388 At 200 docs: 0.1052 Query 122: 0.0528 At 500 docs: 0.0528 Query 123: 0.7770 At 1000 docs: 0.0286 +0.5 Query 124: 0.7069 Query 125: 0.4073 R-Precision (prec. after R Query 126: 0.0000 (=num_rel) docs. retrieved): Query 127: 0.9455 Exact: 0.3480 Query 128: 0.0217 0.0 Query 129: 0.1160 Query 130: 0.7121 Query 131: 0.2163 Query 132: 0.6225 Query 133: 0.1004 Difference (average precision) Query 134: 0.7917 −0.5 Query 135: 0.5176 Query 136: 1.0000 Query 137: 0.1686 Query 138: 0.1600 Query 139: 0.1059 −1.0 Query 140: 0.5958 91 100 110 120 130 140 Topic number CWI/CNLP, Monolingual Dutch, Auto, TD Run AAmoNLtd Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0497 Total number of documents Query 92: 0.4097 over all queries Query 93: 0.7846 Retrieved: 50000 0.8 Query 94: 0.7632 Relevant: 1862 Query 95: 0.5941 Rel_ret: 1550 Query 96: 0.4146 Query 97: 0.4819 Interpolated Recall - 0.6 Query 98: 0.8413 Precision Averages: Query 99: 0.1099 at 0.00 0.7446 Query 100: 0.6056 at 0.10 0.6647 Precision Query 101: 0.6520 at 0.20 0.5564 Query 102: 0.7046 at 0.30 0.5074 0.4 Query 103: 0.6521 at 0.40 0.4387 Query 104: 0.3772 at 0.50 0.3983 Query 105: 0.5289 at 0.60 0.3473 Query 106: 0.0648 at 0.70 0.3063 0.2 Query 107: 0.0270 at 0.80 0.2765 Query 108: 0.5342 at 0.90 0.2084 Query 109: 0.0333 at 1.00 0.1350 Query 110: 1.0000 0.0 Query 111: 0.0016 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5093 for all rel. documents Query 113: 0.0007 0.3988 Recall Query 114: 0.2362 Precision: Query 115: 0.1098 At 5 docs: 0.5520 Run AAmoNLtd Query 116: 0.1544 At 10 docs: 0.4800 Comparison to median by topic Query 117: 0.0208 At 15 docs: 0.4253 +1.0 Query 118: 0.1170 At 20 docs: 0.3880 Query 119: 0.5904 At 30 docs: 0.3360 Query 120: 0.0974 At 100 docs: 0.1866 Query 121: 0.3459 At 200 docs: 0.1163 Query 122: 0.0829 At 500 docs: 0.0571 Query 123: 0.8390 At 1000 docs: 0.0310 +0.5 Query 124: 0.7384 Query 125: 0.4989 R-Precision (prec. after R Query 126: 0.0000 (=num_rel) docs. retrieved): Query 127: 0.6030 Exact: 0.3909 Query 128: 0.1006 0.0 Query 129: 0.3646 Query 130: 0.4387 Query 131: 0.1756 Query 132: 0.6136 Query 133: 0.1047 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.3703 Query 136: 0.9667 Query 137: 0.2357 Query 138: 0.2820 Query 139: 0.1019 −1.0 Query 140: 0.6120 91 100 110 120 130 140 Topic number JHU/APL, Monolingual Dutch, Auto, TD Run aplmonl Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0079 Total number of documents Query 92: 0.4341 over all queries Query 93: 0.7525 Retrieved: 50000 0.8 Query 94: 0.7897 Relevant: 1862 Query 95: 0.6983 Rel_ret: 1773 Query 96: 0.4431 Query 97: 0.6634 Interpolated Recall - Query 98: 0.8417 Precision Averages: 0.6 Query 99: 0.1666 at 0.00 0.7624 Query 100: 0.7414 at 0.10 0.6895 Query 101: 0.9040 at 0.20 0.6383 Precision Query 102: 0.8546 at 0.30 0.5909 0.4 Query 103: 0.7169 at 0.40 0.5609 Query 104: 0.4724 at 0.50 0.5294 Query 105: 0.5088 at 0.60 0.4870 Query 106: 0.1145 at 0.70 0.4495 0.2 Query 107: 0.0239 at 0.80 0.4018 Query 108: 0.5569 at 0.90 0.3333 Query 109: 0.0674 at 1.00 0.2055 Query 110: 1.0000 0.0 Query 111: 0.0701 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.3668 for all rel. documents Query 113: 0.0388 0.5028 Recall Query 114: 0.5735 Precision: Query 115: 0.3071 At 5 docs: 0.5960 Run aplmonl Query 116: 0.5372 At 10 docs: 0.5260 Query 117: 0.0795 At 15 docs: 0.4933 Comparison to median by topic Query 118: 0.2076 At 20 docs: 0.4630 +1.0 Query 119: 0.8740 At 30 docs: 0.4020 Query 120: 0.0382 At 100 docs: 0.2366 Query 121: 0.3778 At 200 docs: 0.1441 Query 122: 0.2049 At 500 docs: 0.0669 Query 123: 0.8934 At 1000 docs: 0.0355 +0.5 Query 124: 0.8980 Query 125: 0.5756 R-Precision (prec. after R Query 126: 0.0000 (=num_rel) docs. retrieved): Query 127: 0.6421 Exact: 0.4842 Query 128: 0.0281 0.0 Query 129: 0.7133 Query 130: 0.7008 Query 131: 0.6315 Query 132: 0.6825 Query 133: 0.2231 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.8488 Query 136: 0.9667 Query 137: 0.0408 Query 138: 0.6494 Query 139: 0.3860 −1.0 Query 140: 0.8279 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 2, Monolingual Dutch, Auto, TD Run bky2monl Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1135 Total number of documents Query 92: 0.4730 over all queries Query 93: 0.7771 Retrieved: 50000 0.8 Query 94: 0.7214 Relevant: 1862 Query 95: 0.6982 Rel_ret: 1734 Query 96: 0.4031 Query 97: 0.5953 Interpolated Recall - Query 98: 0.9167 Precision Averages: 0.6 Query 99: 0.3017 at 0.00 0.8181 Query 100: 0.7248 at 0.10 0.7722 Query 101: 0.7222 at 0.20 0.6584 Precision Query 102: 0.8440 at 0.30 0.5786 0.4 Query 103: 0.6730 at 0.40 0.5359 Query 104: 0.4160 at 0.50 0.4895 Query 105: 0.5136 at 0.60 0.4506 Query 106: 0.1036 at 0.70 0.4012 0.2 Query 107: 0.0193 at 0.80 0.3483 Query 108: 0.4431 at 0.90 0.2708 Query 109: 0.4012 at 1.00 0.1741 Query 110: 0.9500 0.0 Query 111: 0.2353 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5298 for all rel. documents Query 113: 0.0006 0.4847 Recall Query 114: 0.6605 Precision: Query 115: 0.1763 At 5 docs: 0.6080 Run bky2monl Query 116: 0.4388 At 10 docs: 0.5200 Query 117: 0.0843 At 15 docs: 0.4760 Comparison to median by topic Query 118: 0.2238 At 20 docs: 0.4400 +1.0 Query 119: 0.7116 At 30 docs: 0.3767 Query 120: 0.3296 At 100 docs: 0.2262 Query 121: 0.3888 At 200 docs: 0.1406 Query 122: 0.3098 At 500 docs: 0.0657 Query 123: 0.9342 At 1000 docs: 0.0347 +0.5 Query 124: 0.7458 Query 125: 0.4204 R-Precision (prec. after R Query 126: 0.0333 (=num_rel) docs. retrieved): Query 127: 0.6367 Exact: 0.4658 Query 128: 0.0797 0.0 Query 129: 0.5404 Query 130: 0.5996 Query 131: 0.2822 Query 132: 0.7527 Query 133: 0.3412 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.8221 Query 136: 1.0000 Query 137: 0.1071 Query 138: 0.1186 Query 139: 0.1602 −1.0 Query 140: 0.7630 91 100 110 120 130 140 Topic number Hummingbird, Monolingual Dutch, Auto, TD Run humNL02 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1222 Total number of documents Query 92: 0.3994 over all queries Query 93: 0.7360 Retrieved: 50000 0.8 Query 94: 0.7383 Relevant: 1862 Query 95: 0.5545 Rel_ret: 1722 Query 96: 0.3754 Query 97: 0.4719 Interpolated Recall - 0.6 Query 98: 0.8819 Precision Averages: Query 99: 0.1864 at 0.00 0.8253 Query 100: 0.7559 at 0.10 0.7364 Precision Query 101: 0.6546 at 0.20 0.6160 Query 102: 0.4661 at 0.30 0.5353 0.4 Query 103: 0.6774 at 0.40 0.4871 Query 104: 0.3504 at 0.50 0.4495 Query 105: 0.3954 at 0.60 0.4073 Query 106: 0.0802 at 0.70 0.3546 0.2 Query 107: 0.0409 at 0.80 0.3111 Query 108: 0.6140 at 0.90 0.2351 Query 109: 0.3697 at 1.00 0.1376 Query 110: 1.0000 0.0 Query 111: 0.2155 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5633 for all rel. documents Query 113: 0.0232 0.4447 Recall Query 114: 0.2332 Precision: Query 115: 0.2968 At 5 docs: 0.5840 Run humNL02 Query 116: 0.6069 At 10 docs: 0.5040 Comparison to median by topic Query 117: 0.1158 At 15 docs: 0.4547 +1.0 Query 118: 0.2075 At 20 docs: 0.4170 Query 119: 0.5328 At 30 docs: 0.3627 Query 120: 0.2254 At 100 docs: 0.2084 Query 121: 0.4011 At 200 docs: 0.1335 Query 122: 0.1608 At 500 docs: 0.0636 Query 123: 0.8719 At 1000 docs: 0.0344 +0.5 Query 124: 0.6420 Query 125: 0.5156 R-Precision (prec. after R Query 126: 0.0588 (=num_rel) docs. retrieved): Query 127: 0.7969 Exact: 0.4342 Query 128: 0.0592 0.0 Query 129: 0.4179 Query 130: 0.5359 Query 131: 0.2183 Query 132: 0.5438 Query 133: 0.1902 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.6959 Query 136: 1.0000 Query 137: 0.0201 Query 138: 0.1340 Query 139: 0.3200 −1.0 Query 140: 0.7594 91 100 110 120 130 140 Topic number Hummingbird, Monolingual Dutch, Auto, TDN Run humNL02n Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1359 Total number of documents Query 92: 0.4021 over all queries Query 93: 0.5840 Retrieved: 50000 0.8 Query 94: 0.7571 Relevant: 1862 Query 95: 0.6357 Rel_ret: 1743 Query 96: 0.3579 Query 97: 0.5374 Interpolated Recall - 0.6 Query 98: 0.8819 Precision Averages: Query 99: 0.2042 at 0.00 0.8757 Query 100: 0.7913 at 0.10 0.7678 Precision Query 101: 0.7950 at 0.20 0.6847 Query 102: 0.5691 at 0.30 0.6166 0.4 Query 103: 0.6883 at 0.40 0.5524 Query 104: 0.2572 at 0.50 0.4914 Query 105: 0.4778 at 0.60 0.4325 Query 106: 0.1260 at 0.70 0.3704 0.2 Query 107: 0.1106 at 0.80 0.3264 Query 108: 0.5943 at 0.90 0.2498 Query 109: 0.3812 at 1.00 0.1538 Query 110: 1.0000 0.0 Query 111: 0.2254 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5586 for all rel. documents Query 113: 0.0027 0.4857 Recall Query 114: 0.4720 Precision: Query 115: 0.4000 At 5 docs: 0.6320 Run humNL02n Query 116: 0.6270 At 10 docs: 0.5460 Comparison to median by topic Query 117: 0.3401 At 15 docs: 0.4867 +1.0 Query 118: 0.3104 At 20 docs: 0.4340 Query 119: 0.5423 At 30 docs: 0.3793 Query 120: 0.2399 At 100 docs: 0.2174 Query 121: 0.4078 At 200 docs: 0.1371 Query 122: 0.1669 At 500 docs: 0.0656 Query 123: 0.9201 At 1000 docs: 0.0349 +0.5 Query 124: 0.6605 Query 125: 0.5320 R-Precision (prec. after R Query 126: 0.2494 (=num_rel) docs. retrieved): Query 127: 0.8909 Exact: 0.4722 Query 128: 0.0475 0.0 Query 129: 0.5059 Query 130: 0.7576 Query 131: 0.3996 Query 132: 0.3175 Query 133: 0.2648 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.5415 Query 136: 1.0000 Query 137: 0.0105 Query 138: 0.5261 Query 139: 0.3067 −1.0 Query 140: 0.7743 91 100 110 120 130 140 Topic number MediaLab BV, Monolingual Dutch, Auto, TD Run medialab2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0263 Total number of documents Query 92: 0.3739 over all queries Query 93: 0.3772 Retrieved: 50000 0.8 Query 94: 0.6144 Relevant: 1862 Query 95: 0.2450 Rel_ret: 1425 Query 96: 0.2917 Query 97: 0.3560 Interpolated Recall - 0.6 Query 98: 0.9167 Precision Averages: Query 99: 0.1647 at 0.00 0.6784 Query 100: 0.2968 at 0.10 0.5603 Precision Query 101: 0.4123 at 0.20 0.4486 Query 102: 0.7083 at 0.30 0.4020 0.4 Query 103: 0.5735 at 0.40 0.3372 Query 104: 0.1121 at 0.50 0.2905 Query 105: 0.4244 at 0.60 0.2635 Query 106: 0.0648 at 0.70 0.2326 0.2 Query 107: 0.0256 at 0.80 0.2056 Query 108: 0.0402 at 0.90 0.1602 Query 109: 0.0333 at 1.00 0.1145 Query 110: 1.0000 0.0 Query 111: 0.0008 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.3675 for all rel. documents Query 113: 0.0084 0.3167 Recall Query 114: 0.1073 Precision: Query 115: 0.0006 At 5 docs: 0.4560 Run medialab2 Query 116: 0.2103 At 10 docs: 0.4120 Comparison to median by topic Query 117: 0.1472 At 15 docs: 0.3587 +1.0 Query 118: 0.1833 At 20 docs: 0.3250 Query 119: 0.4975 At 30 docs: 0.2800 Query 120: 0.1701 At 100 docs: 0.1542 Query 121: 0.3835 At 200 docs: 0.1026 Query 122: 0.0678 At 500 docs: 0.0509 Query 123: 0.7706 At 1000 docs: 0.0285 +0.5 Query 124: 0.4664 Query 125: 0.3989 R-Precision (prec. after R Query 126: 0.0000 (=num_rel) docs. retrieved): Query 127: 0.6395 Exact: 0.3552 Query 128: 0.0160 0.0 Query 129: 0.1111 Query 130: 0.5890 Query 131: 0.2475 Query 132: 0.1484 Query 133: 0.1289 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.6086 Query 136: 0.7862 Query 137: 0.0029 Query 138: 0.0093 Query 139: 0.0696 −1.0 Query 140: 0.6415 91 100 110 120 130 140 Topic number MediaLab BV, Monolingual Dutch, Auto, TD Run medialab3 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0096 Total number of documents Query 92: 0.3920 over all queries Query 93: 0.5536 Retrieved: 50000 0.8 Query 94: 0.6526 Relevant: 1862 Query 95: 0.3573 Rel_ret: 1453 Query 96: 0.3049 Query 97: 0.3288 Interpolated Recall - 0.6 Query 98: 0.9103 Precision Averages: Query 99: 0.2782 at 0.00 0.7048 Query 100: 0.4426 at 0.10 0.5817 Precision Query 101: 0.4482 at 0.20 0.4711 Query 102: 0.6987 at 0.30 0.4108 0.4 Query 103: 0.5539 at 0.40 0.3574 Query 104: 0.1192 at 0.50 0.3244 Query 105: 0.5094 at 0.60 0.2894 Query 106: 0.0758 at 0.70 0.2624 0.2 Query 107: 0.0274 at 0.80 0.2206 Query 108: 0.1545 at 0.90 0.1555 Query 109: 0.0333 at 1.00 0.1067 Query 110: 0.9167 0.0 Query 111: 0.0006 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.3557 for all rel. documents Query 113: 0.0173 0.3344 Recall Query 114: 0.1409 Precision: Query 115: 0.0007 At 5 docs: 0.5120 Run medialab3 Query 116: 0.2310 At 10 docs: 0.4180 Comparison to median by topic Query 117: 0.0715 At 15 docs: 0.3787 +1.0 Query 118: 0.1624 At 20 docs: 0.3440 Query 119: 0.5027 At 30 docs: 0.2913 Query 120: 0.1192 At 100 docs: 0.1638 Query 121: 0.3634 At 200 docs: 0.1085 Query 122: 0.1068 At 500 docs: 0.0527 Query 123: 0.7381 At 1000 docs: 0.0291 +0.5 Query 124: 0.3869 Query 125: 0.3764 R-Precision (prec. after R Query 126: 0.0000 (=num_rel) docs. retrieved): Query 127: 0.6452 Exact: 0.3529 Query 128: 0.0479 0.0 Query 129: 0.1220 Query 130: 0.6112 Query 131: 0.3423 Query 132: 0.2370 Query 133: 0.1398 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.6952 Query 136: 0.8100 Query 137: 0.0098 Query 138: 0.0069 Query 139: 0.0581 −1.0 Query 140: 0.6549 91 100 110 120 130 140 Topic number MediaLab BV, Monolingual Dutch, Auto, TD Run medialab4 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0010 Total number of documents Query 92: 0.3476 over all queries Query 93: 0.6614 Retrieved: 49811 0.8 Query 94: 0.6929 Relevant: 1862 Query 95: 0.3564 Rel_ret: 1391 Query 96: 0.3111 Query 97: 0.2970 Interpolated Recall - 0.6 Query 98: 0.7046 Precision Averages: Query 99: 0.3841 at 0.00 0.6813 Query 100: 0.1884 at 0.10 0.5680 Precision Query 101: 0.4566 at 0.20 0.4512 Query 102: 0.6705 at 0.30 0.3859 0.4 Query 103: 0.4787 at 0.40 0.3329 Query 104: 0.1313 at 0.50 0.3118 Query 105: 0.5142 at 0.60 0.2763 Query 106: 0.0855 at 0.70 0.2271 0.2 Query 107: 0.0288 at 0.80 0.1871 Query 108: 0.2773 at 0.90 0.1214 Query 109: 0.0333 at 1.00 0.0603 Query 110: 0.3883 0.0 Query 111: 0.0027 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.3182 for all rel. documents Query 113: 0.0131 0.3035 Recall Query 114: 0.0852 Precision: Query 115: 0.0001 At 5 docs: 0.4560 Run medialab4 Query 116: 0.4716 At 10 docs: 0.3980 Comparison to median by topic Query 117: 0.0309 At 15 docs: 0.3573 +1.0 Query 118: 0.0966 At 20 docs: 0.3200 Query 119: 0.3710 At 30 docs: 0.2867 Query 120: 0.0434 At 100 docs: 0.1614 Query 121: 0.2899 At 200 docs: 0.1002 Query 122: 0.0962 At 500 docs: 0.0513 Query 123: 0.6799 At 1000 docs: 0.0278 +0.5 Query 124: 0.1782 Query 125: 0.3246 R-Precision (prec. after R Query 126: 0.0000 (=num_rel) docs. retrieved): Query 127: 0.5900 Exact: 0.3138 Query 128: 0.0951 0.0 Query 129: 0.1170 Query 130: 0.6118 Query 131: 0.2646 Query 132: 0.2458 Query 133: 0.0952 Difference (average precision) Query 134: 0.8750 −0.5 Query 135: 0.7385 Query 136: 0.8100 Query 137: 0.0337 Query 138: 0.0026 Query 139: 0.0509 −1.0 Query 140: 0.6367 91 100 110 120 130 140 Topic number MediaLab BV, Monolingual Dutch, Auto, TD Run medialab5 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0096 Total number of documents Query 92: 0.3974 over all queries Query 93: 0.5528 Retrieved: 50000 0.8 Query 94: 0.6569 Relevant: 1862 Query 95: 0.3556 Rel_ret: 1454 Query 96: 0.3222 Query 97: 0.3286 Interpolated Recall - 0.6 Query 98: 0.8825 Precision Averages: Query 99: 0.3064 at 0.00 0.7058 Query 100: 0.3939 at 0.10 0.5791 Precision Query 101: 0.4465 at 0.20 0.4707 Query 102: 0.6959 at 0.30 0.4050 0.4 Query 103: 0.5521 at 0.40 0.3613 Query 104: 0.1206 at 0.50 0.3269 Query 105: 0.5072 at 0.60 0.2899 Query 106: 0.0757 at 0.70 0.2602 0.2 Query 107: 0.0275 at 0.80 0.2167 Query 108: 0.1545 at 0.90 0.1567 Query 109: 0.0333 at 1.00 0.1063 Query 110: 0.9167 0.0 Query 111: 0.0006 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.3616 for all rel. documents Query 113: 0.0198 0.3341 Recall Query 114: 0.1449 Precision: Query 115: 0.0007 At 5 docs: 0.5000 Run medialab5 Query 116: 0.2365 At 10 docs: 0.4180 Comparison to median by topic Query 117: 0.0719 At 15 docs: 0.3827 +1.0 Query 118: 0.1635 At 20 docs: 0.3440 Query 119: 0.5066 At 30 docs: 0.2927 Query 120: 0.1178 At 100 docs: 0.1638 Query 121: 0.3624 At 200 docs: 0.1084 Query 122: 0.0998 At 500 docs: 0.0529 Query 123: 0.7438 At 1000 docs: 0.0291 +0.5 Query 124: 0.3816 Query 125: 0.3805 R-Precision (prec. after R Query 126: 0.0000 (=num_rel) docs. retrieved): Query 127: 0.6428 Exact: 0.3538 Query 128: 0.0479 0.0 Query 129: 0.1253 Query 130: 0.6112 Query 131: 0.3416 Query 132: 0.2353 Query 133: 0.1384 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.6939 Query 136: 0.8100 Query 137: 0.0099 Query 138: 0.0069 Query 139: 0.0578 −1.0 Query 140: 0.6550 91 100 110 120 130 140 Topic number Océ, Monolingual Dutch, Auto, TD Run oce02monNL Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1776 Total number of documents Query 92: 0.3688 over all queries Query 93: 0.7182 Retrieved: 50000 0.8 Query 94: 0.5059 Relevant: 1862 Query 95: 0.1883 Rel_ret: 1538 Query 96: 0.2415 Query 97: 0.4858 Interpolated Recall - 0.6 Query 98: 0.8819 Precision Averages: Query 99: 0.1796 at 0.00 0.7852 Query 100: 0.7166 at 0.10 0.6649 Precision Query 101: 0.5918 at 0.20 0.5454 Query 102: 0.4521 at 0.30 0.4913 0.4 Query 103: 0.5443 at 0.40 0.4306 Query 104: 0.1999 at 0.50 0.3841 Query 105: 0.2089 at 0.60 0.3442 Query 106: 0.0807 at 0.70 0.2980 0.2 Query 107: 0.0441 at 0.80 0.2579 Query 108: 0.5301 at 0.90 0.1936 Query 109: 0.3382 at 1.00 0.1299 Query 110: 1.0000 0.0 Query 111: 0.0014 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5090 for all rel. documents Query 113: 0.0076 0.3919 Recall Query 114: 0.2155 Precision: Query 115: 0.4457 At 5 docs: 0.5480 Run oce02monNL Query 116: 0.0706 At 10 docs: 0.4640 Comparison to median by topic Query 117: 0.0434 At 15 docs: 0.4187 +1.0 Query 118: 0.0938 At 20 docs: 0.3790 Query 119: 0.6461 At 30 docs: 0.3220 Query 120: 0.3073 At 100 docs: 0.1790 Query 121: 0.3526 At 200 docs: 0.1141 Query 122: 0.1056 At 500 docs: 0.0561 Query 123: 0.8328 At 1000 docs: 0.0308 +0.5 Query 124: 0.6711 Query 125: 0.5206 R-Precision (prec. after R Query 126: 0.0697 (=num_rel) docs. retrieved): Query 127: 0.8395 Exact: 0.3957 Query 128: 0.0206 0.0 Query 129: 0.4672 Query 130: 0.5561 Query 131: 0.1603 Query 132: 0.2284 Query 133: 0.1717 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.6382 Query 136: 1.0000 Query 137: 0.0019 Query 138: 0.2139 Query 139: 0.3639 −1.0 Query 140: 0.5844 91 100 110 120 130 140 Topic number Océ, Monolingual Dutch, Auto, T Run oce02monNLto Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0003 Total number of documents Query 92: 0.3625 over all queries Query 93: 0.7446 Retrieved: 41619 0.8 Query 94: 0.7780 Relevant: 1862 Query 95: 0.5489 Rel_ret: 1551 Query 96: 0.4353 Query 97: 0.4827 Interpolated Recall - 0.6 Query 98: 0.8542 Precision Averages: Query 99: 0.2419 at 0.00 0.7291 Query 100: 0.5346 at 0.10 0.6460 Precision Query 101: 0.6677 at 0.20 0.5356 Query 102: 0.6517 at 0.30 0.4829 0.4 Query 103: 0.2710 at 0.40 0.4297 Query 104: 0.5135 at 0.50 0.3742 Query 105: 0.4797 at 0.60 0.3274 Query 106: 0.0547 at 0.70 0.2895 0.2 Query 107: 0.0332 at 0.80 0.2587 Query 108: 0.5354 at 0.90 0.1932 Query 109: 0.2373 at 1.00 0.1143 Query 110: 0.0429 0.0 Query 111: 0.0000 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.3987 for all rel. documents Query 113: 0.0011 0.3808 Recall Query 114: 0.1326 Precision: Query 115: 0.4155 At 5 docs: 0.5120 Run oce02monNLto Query 116: 0.1771 At 10 docs: 0.4480 Comparison to median by topic Query 117: 0.0100 At 15 docs: 0.4013 +1.0 Query 118: 0.0396 At 20 docs: 0.3690 Query 119: 0.1681 At 30 docs: 0.3287 Query 120: 0.0390 At 100 docs: 0.1828 Query 121: 0.3885 At 200 docs: 0.1157 Query 122: 0.0591 At 500 docs: 0.0565 Query 123: 0.8044 At 1000 docs: 0.0310 +0.5 Query 124: 0.5839 Query 125: 0.3674 R-Precision (prec. after R Query 126: 0.1133 (=num_rel) docs. retrieved): Query 127: 0.9849 Exact: 0.3782 Query 128: 0.0395 0.0 Query 129: 0.0933 Query 130: 0.7087 Query 131: 0.3927 Query 132: 0.4635 Query 133: 0.1530 Difference (average precision) Query 134: 0.9500 −0.5 Query 135: 0.7551 Query 136: 1.0000 Query 137: 0.1295 Query 138: 0.1146 Query 139: 0.3689 −1.0 Query 140: 0.7164 91 100 110 120 130 140 Topic number City University, Monolingual Dutch, Auto, T Run pldu02t Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0000 Total number of documents Query 92: 0.2868 over all queries Query 93: 0.3857 Retrieved: 50000 0.8 Query 94: 0.7848 Relevant: 1862 Query 95: 0.3043 Rel_ret: 987 Query 96: 0.4454 Query 97: 0.4495 Interpolated Recall - Query 98: 0.0022 Precision Averages: 0.6 Query 99: 0.0362 at 0.00 0.4235 Query 100: 0.0000 at 0.10 0.3607 Query 101: 0.6337 at 0.20 0.3063 Precision Query 102: 0.0000 at 0.30 0.2655 0.4 Query 103: 0.1058 at 0.40 0.2213 Query 104: 0.4067 at 0.50 0.1899 Query 105: 0.0000 at 0.60 0.1628 Query 106: 0.0000 at 0.70 0.1382 0.2 Query 107: 0.3261 at 0.80 0.0971 Query 108: 0.5545 at 0.90 0.0684 Query 109: 0.0000 at 1.00 0.0384 Query 110: 0.0568 0.0 Query 111: 0.0000 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.1961 for all rel. documents Query 113: 0.0006 0.1934 Recall Query 114: 0.1288 Precision: Query 115: 0.0000 At 5 docs: 0.3080 Run pldu02t Query 116: 0.6030 At 10 docs: 0.2820 Query 117: 0.0000 At 15 docs: 0.2547 Comparison to median by topic Query 118: 0.0224 At 20 docs: 0.2250 +1.0 Query 119: 0.0049 At 30 docs: 0.1853 Query 120: 0.0329 At 100 docs: 0.1022 Query 121: 0.0010 At 200 docs: 0.0639 Query 122: 0.0000 At 500 docs: 0.0336 Query 123: 0.7905 At 1000 docs: 0.0197 +0.5 Query 124: 0.2013 Query 125: 0.1441 R-Precision (prec. after R Query 126: 0.0000 (=num_rel) docs. retrieved): Query 127: 0.9783 Exact: 0.2073 Query 128: 0.0197 0.0 Query 129: 0.0000 Query 130: 0.0005 Query 131: 0.0859 Query 132: 0.2567 Query 133: 0.0110 Difference (average precision) Query 134: 0.0000 −0.5 Query 135: 0.1181 Query 136: 0.4520 Query 137: 0.0243 Query 138: 0.3954 Query 139: 0.0000 −1.0 Query 140: 0.4222 91 100 110 120 130 140 Topic number City University, Monolingual Dutch, Auto, TD Run pldu02td Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0001 Total number of documents Query 92: 0.3770 over all queries Query 93: 0.1772 Retrieved: 50000 0.8 Query 94: 0.7734 Relevant: 1862 Query 95: 0.4906 Rel_ret: 1193 Query 96: 0.2660 Query 97: 0.6642 Interpolated Recall - Query 98: 0.0054 Precision Averages: 0.6 Query 99: 0.0611 at 0.00 0.6082 Query 100: 0.0061 at 0.10 0.4805 Query 101: 0.8258 at 0.20 0.4010 Precision Query 102: 0.0000 at 0.30 0.3343 0.4 Query 103: 0.0686 at 0.40 0.2765 Query 104: 0.1940 at 0.50 0.2512 Query 105: 0.0509 at 0.60 0.2178 Query 106: 0.0000 at 0.70 0.1816 0.2 Query 107: 0.0094 at 0.80 0.1381 Query 108: 0.4261 at 0.90 0.0987 Query 109: 0.0187 at 1.00 0.0555 Query 110: 1.0000 0.0 Query 111: 0.0001 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.3606 for all rel. documents Query 113: 0.0077 0.2593 Recall Query 114: 0.4132 Precision: Query 115: 0.0255 At 5 docs: 0.3960 Run pldu02td Query 116: 0.3705 At 10 docs: 0.3460 Query 117: 0.0088 At 15 docs: 0.3040 Comparison to median by topic Query 118: 0.0264 At 20 docs: 0.2810 +1.0 Query 119: 0.5186 At 30 docs: 0.2420 Query 120: 0.1164 At 100 docs: 0.1248 Query 121: 0.0631 At 200 docs: 0.0769 Query 122: 0.0584 At 500 docs: 0.0404 Query 123: 0.7490 At 1000 docs: 0.0239 +0.5 Query 124: 0.2743 Query 125: 0.4048 R-Precision (prec. after R Query 126: 0.2070 (=num_rel) docs. retrieved): Query 127: 0.8655 Exact: 0.2801 Query 128: 0.0333 0.0 Query 129: 0.3664 Query 130: 0.4617 Query 131: 0.0758 Query 132: 0.3441 Query 133: 0.0191 Difference (average precision) Query 134: 0.3375 −0.5 Query 135: 0.0571 Query 136: 0.4456 Query 137: 0.0167 Query 138: 0.7430 Query 139: 0.0000 −1.0 Query 140: 0.1784 91 100 110 120 130 140 Topic number TLR Research, Monolingual Dutch, Auto, TD Run tlrnl Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.1171 Total number of documents Query 92: 0.3303 over all queries Query 93: 0.8039 Retrieved: 50000 0.8 Query 94: 0.7123 Relevant: 1862 Query 95: 0.5095 Rel_ret: 1691 Query 96: 0.3148 Query 97: 0.4251 Interpolated Recall - 0.6 Query 98: 0.6799 Precision Averages: Query 99: 0.1416 at 0.00 0.7341 Query 100: 0.7238 at 0.10 0.6606 Precision Query 101: 0.5886 at 0.20 0.5746 Query 102: 0.4895 at 0.30 0.5116 0.4 Query 103: 0.5780 at 0.40 0.4641 Query 104: 0.5298 at 0.50 0.4411 Query 105: 0.3784 at 0.60 0.3954 Query 106: 0.0285 at 0.70 0.3471 0.2 Query 107: 0.0294 at 0.80 0.3081 Query 108: 0.5126 at 0.90 0.2133 Query 109: 0.3857 at 1.00 0.1257 Query 110: 1.0000 0.0 Query 111: 0.1118 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.5298 for all rel. documents Query 113: 0.0234 0.4141 Recall Query 114: 0.1667 Precision: Query 115: 0.2514 At 5 docs: 0.4960 Run tlrnl Query 116: 0.4989 At 10 docs: 0.4440 Comparison to median by topic Query 117: 0.0853 At 15 docs: 0.4093 +1.0 Query 118: 0.1586 At 20 docs: 0.3900 Query 119: 0.5295 At 30 docs: 0.3420 Query 120: 0.2401 At 100 docs: 0.2022 Query 121: 0.3582 At 200 docs: 0.1280 Query 122: 0.1070 At 500 docs: 0.0618 Query 123: 0.8160 At 1000 docs: 0.0338 +0.5 Query 124: 0.8200 Query 125: 0.4816 R-Precision (prec. after R Query 126: 0.0471 (=num_rel) docs. retrieved): Query 127: 0.6023 Exact: 0.4211 Query 128: 0.0124 0.0 Query 129: 0.5201 Query 130: 0.3506 Query 131: 0.0882 Query 132: 0.6050 Query 133: 0.1517 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.6072 Query 136: 1.0000 Query 137: 0.0108 Query 138: 0.2244 Query 139: 0.2909 −1.0 Query 140: 0.7363 91 100 110 120 130 140 Topic number Univ. of Twente, Monolingual Dutch, Manual, TDN Run tnoutn1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.6087 Total number of documents Query 92: 0.3937 over all queries Query 93: 0.3913 Retrieved: 47510 0.8 Query 94: 0.8055 Relevant: 1862 Query 95: 0.4963 Rel_ret: 1602 Query 96: 0.3887 Query 97: 0.4748 Interpolated Recall - 0.6 Query 98: 0.7300 Precision Averages: Query 99: 0.2637 at 0.00 0.8535 Query 100: 0.5479 at 0.10 0.7594 Precision Query 101: 0.4675 at 0.20 0.6562 Query 102: 0.7800 at 0.30 0.5866 0.4 Query 103: 0.5004 at 0.40 0.5290 Query 104: 0.4637 at 0.50 0.4686 Query 105: 0.3311 at 0.60 0.3862 Query 106: 0.0891 at 0.70 0.3245 0.2 Query 107: 0.0523 at 0.80 0.2443 Query 108: 0.4860 at 0.90 0.1846 Query 109: 0.1826 at 1.00 0.1176 Query 110: 0.8500 0.0 Query 111: 0.1441 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.6477 for all rel. documents Query 113: 0.6937 0.4471 Recall Query 114: 0.6667 Precision: Query 115: 0.0809 At 5 docs: 0.6240 Run tnoutn1 Query 116: 0.6134 At 10 docs: 0.5440 Comparison to median by topic Query 117: 0.1955 At 15 docs: 0.4853 +1.0 Query 118: 0.5820 At 20 docs: 0.4380 Query 119: 0.6023 At 30 docs: 0.3807 Query 120: 0.0387 At 100 docs: 0.2014 Query 121: 0.2908 At 200 docs: 0.1218 Query 122: 0.2787 At 500 docs: 0.0590 Query 123: 0.6475 At 1000 docs: 0.0320 +0.5 Query 124: 0.6469 Query 125: 0.4074 R-Precision (prec. after R Query 126: 0.3001 (=num_rel) docs. retrieved): Query 127: 0.7063 Exact: 0.4515 Query 128: 0.0138 0.0 Query 129: 0.3173 Query 130: 0.8714 Query 131: 0.2272 Query 132: 0.2016 Query 133: 0.1564 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.1963 Query 136: 0.9250 Query 137: 0.3143 Query 138: 0.3016 Query 139: 0.3373 −1.0 Query 140: 0.6490 91 100 110 120 130 140 Topic number University of Amsterdam, Monolingual Dutch, Auto, TD Run UAmsC02DuDuNGiMO Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0407 Total number of documents Query 92: 0.4167 over all queries Query 93: 0.7991 Retrieved: 50000 0.8 Query 94: 0.7994 Relevant: 1862 Query 95: 0.6718 Rel_ret: 1697 Query 96: 0.4199 Query 97: 0.4999 Interpolated Recall - 0.6 Query 98: 0.9333 Precision Averages: Query 99: 0.2418 at 0.00 0.7502 Query 100: 0.6162 at 0.10 0.7049 Precision Query 101: 0.6704 at 0.20 0.6269 Query 102: 0.6964 at 0.30 0.5757 0.4 Query 103: 0.7070 at 0.40 0.5178 Query 104: 0.1275 at 0.50 0.4807 Query 105: 0.2679 at 0.60 0.4348 Query 106: 0.0831 at 0.70 0.3868 0.2 Query 107: 0.0186 at 0.80 0.3375 Query 108: 0.4947 at 0.90 0.2522 Query 109: 0.5390 at 1.00 0.1468 Query 110: 0.4304 0.0 Query 111: 0.0912 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.3053 for all rel. documents Query 113: 0.0029 0.4598 Recall Query 114: 0.6349 Precision: Query 115: 0.3875 At 5 docs: 0.5960 Run UAmsC02DuDuNGiMO Query 116: 0.6194 At 10 docs: 0.5160 Comparison to median by topic Query 117: 0.1270 At 15 docs: 0.4720 +1.0 Query 118: 0.1366 At 20 docs: 0.4330 Query 119: 0.7295 At 30 docs: 0.3827 Query 120: 0.0517 At 100 docs: 0.2274 Query 121: 0.3592 At 200 docs: 0.1392 Query 122: 0.3483 At 500 docs: 0.0640 Query 123: 0.8049 At 1000 docs: 0.0339 +0.5 Query 124: 0.7875 Query 125: 0.4776 R-Precision (prec. after R Query 126: 0.0000 (=num_rel) docs. retrieved): Query 127: 0.7955 Exact: 0.4415 Query 128: 0.0364 0.0 Query 129: 0.7079 Query 130: 0.6095 Query 131: 0.7403 Query 132: 0.6194 Query 133: 0.1176 Difference (average precision) Query 134: 0.8929 −0.5 Query 135: 0.8214 Query 136: 1.0000 Query 137: 0.0035 Query 138: 0.2334 Query 139: 0.2426 −1.0 Query 140: 0.8334 91 100 110 120 130 140 Topic number University of Amsterdam, Monolingual Dutch, Auto, TD Run UAmsC02DuDuNGram Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0399 Total number of documents Query 92: 0.3834 over all queries Query 93: 0.8048 Retrieved: 50000 0.8 Query 94: 0.8332 Relevant: 1862 Query 95: 0.6612 Rel_ret: 1683 Query 96: 0.4245 Query 97: 0.6043 Interpolated Recall - 0.6 Query 98: 0.9444 Precision Averages: Query 99: 0.2523 at 0.00 0.7516 Query 100: 0.6207 at 0.10 0.6949 Precision Query 101: 0.6726 at 0.20 0.6093 Query 102: 0.6779 at 0.30 0.5683 0.4 Query 103: 0.7264 at 0.40 0.5135 Query 104: 0.1406 at 0.50 0.4762 Query 105: 0.1418 at 0.60 0.4195 Query 106: 0.0774 at 0.70 0.3794 0.2 Query 107: 0.0190 at 0.80 0.3245 Query 108: 0.4755 at 0.90 0.2403 Query 109: 0.5486 at 1.00 0.1437 Query 110: 0.3808 0.0 Query 111: 0.1157 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.3210 for all rel. documents Query 113: 0.0037 0.4542 Recall Query 114: 0.6332 Precision: Query 115: 0.4183 At 5 docs: 0.5840 Run UAmsC02DuDuNGram Query 116: 0.7487 At 10 docs: 0.5060 Comparison to median by topic Query 117: 0.1235 At 15 docs: 0.4707 +1.0 Query 118: 0.1402 At 20 docs: 0.4380 Query 119: 0.7297 At 30 docs: 0.3840 Query 120: 0.0543 At 100 docs: 0.2222 Query 121: 0.3169 At 200 docs: 0.1371 Query 122: 0.3888 At 500 docs: 0.0636 Query 123: 0.7799 At 1000 docs: 0.0337 +0.5 Query 124: 0.7655 Query 125: 0.4012 R-Precision (prec. after R Query 126: 0.0001 (=num_rel) docs. retrieved): Query 127: 0.8026 Exact: 0.4326 Query 128: 0.0354 0.0 Query 129: 0.6950 Query 130: 0.5729 Query 131: 0.6079 Query 132: 0.5845 Query 133: 0.1439 Difference (average precision) Query 134: 0.8929 −0.5 Query 135: 0.7943 Query 136: 1.0000 Query 137: 0.0000 Query 138: 0.1287 Query 139: 0.2814 −1.0 Query 140: 0.8017 91 100 110 120 130 140 Topic number Université de Neuchâtel, Monolingual Dutch, Auto, TD Run UniNEnl Average precision Overall statistics Recall−Precision Values (individual queries): (for 50 queries): 1.0 Query 91: 0.0083 Total number of documents Query 92: 0.4085 over all queries Query 93: 0.8462 Retrieved: 50000 0.8 Query 94: 0.7545 Relevant: 1862 Query 95: 0.6097 Rel_ret: 1745 Query 96: 0.5120 Query 97: 0.6179 Interpolated Recall - 0.6 Query 98: 0.8819 Precision Averages: Query 99: 0.2722 at 0.00 0.7793 Query 100: 0.7956 at 0.10 0.7036 Precision Query 101: 0.7279 at 0.20 0.6364 Query 102: 0.7434 at 0.30 0.5819 0.4 Query 103: 0.6776 at 0.40 0.5450 Query 104: 0.1816 at 0.50 0.5120 Query 105: 0.3785 at 0.60 0.4660 Query 106: 0.0641 at 0.70 0.4105 0.2 Query 107: 0.0233 at 0.80 0.3697 Query 108: 0.5776 at 0.90 0.2938 Query 109: 0.3220 at 1.00 0.1734 Query 110: 1.0000 0.0 Query 111: 0.1025 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 112: 0.4931 for all rel. documents Query 113: 0.0108 0.4878 Recall Query 114: 0.3664 Precision: Query 115: 0.4260 At 5 docs: 0.6080 Run UniNEnl Query 116: 0.6476 At 10 docs: 0.5380 Comparison to median by topic Query 117: 0.1107 At 15 docs: 0.4880 +1.0 Query 118: 0.1710 At 20 docs: 0.4530 Query 119: 0.7509 At 30 docs: 0.3960 Query 120: 0.2022 At 100 docs: 0.2260 Query 121: 0.3953 At 200 docs: 0.1369 Query 122: 0.2415 At 500 docs: 0.0656 Query 123: 0.8618 At 1000 docs: 0.0349 +0.5 Query 124: 0.8678 Query 125: 0.5466 R-Precision (prec. after R Query 126: 0.0000 (=num_rel) docs. retrieved): Query 127: 0.8349 Exact: 0.4776 Query 128: 0.0431 0.0 Query 129: 0.6747 Query 130: 0.6686 Query 131: 0.5535 Query 132: 0.6716 Query 133: 0.2662 Difference (average precision) Query 134: 1.0000 −0.5 Query 135: 0.7804 Query 136: 1.0000 Query 137: 0.0049 Query 138: 0.0426 Query 139: 0.4188 −1.0 Query 140: 0.8324 91 100 110 120 130 140 Topic number JHU/APL, Monolingual Swedish, Auto, TD Run aplmosv Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.6054 Total number of documents Query 92: 0.4416 over all queries Query 93: 0.6926 Retrieved: 49000 0.8 Query 94: 0.9412 Relevant: 1196 Query 95: 0.1476 Rel_ret: 1155 Query 96: 0.4536 Query 97: 0.8368 Interpolated Recall - Query 98: 0.8333 Precision Averages: 0.6 Query 99: 0.2230 at 0.00 0.7357 Query 100: 0.5220 at 0.10 0.6645 Query 101: 0.7562 at 0.20 0.5977 Precision Query 102: 0.6882 at 0.30 0.5265 0.4 Query 103: 0.1445 at 0.40 0.5027 Query 104: 0.4008 at 0.50 0.4574 Query 105: 0.0323 at 0.60 0.3936 Query 106: 0.2648 at 0.70 0.3565 0.2 Query 107: 0.4726 at 0.80 0.3270 Query 108: 0.3014 at 0.90 0.2606 Query 109: 0.1840 at 1.00 0.2001 Query 111: 0.0772 0.0 Query 112: 0.5655 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 113: 0.0477 for all rel. documents Query 114: 0.3313 0.4317 Recall Query 115: 0.2200 Precision: Query 116: 0.1263 At 5 docs: 0.4857 Run aplmosv Query 117: 0.0185 At 10 docs: 0.4306 Query 118: 0.4973 At 15 docs: 0.3728 Comparison to median by topic Query 119: 0.8428 At 20 docs: 0.3439 +1.0 Query 120: 0.0333 At 30 docs: 0.2980 Query 121: 0.0227 At 100 docs: 0.1682 Query 122: 0.1257 At 200 docs: 0.0983 Query 123: 0.9327 At 500 docs: 0.0444 Query 124: 0.5464 At 1000 docs: 0.0236 +0.5 Query 125: 0.4669 Query 126: 0.0311 R-Precision (prec. after R Query 127: 0.8659 (=num_rel) docs. retrieved): Query 128: 0.3034 Exact: 0.3986 Query 129: 0.8144 0.0 Query 130: 0.5930 Query 131: 0.2568 Query 132: 0.8229 Query 133: 0.0044 Query 134: 0.6429 Difference (average precision) Query 135: 0.9166 −0.5 Query 136: 0.5304 Query 137: 0.0653 Query 138: 0.1583 Query 139: 0.7326 Query 140: 0.6212 −1.0 91 100 110 120 130 140 Topic number Hummingbird, Monolingual Swedish, Auto, TD Run humSV02 Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.3441 Total number of documents Query 92: 0.4115 over all queries Query 93: 0.6862 Retrieved: 48993 0.8 Query 94: 0.8161 Relevant: 1196 Query 95: 0.0715 Rel_ret: 911 Query 96: 0.3662 Query 97: 0.5890 Interpolated Recall - 0.6 Query 98: 1.0000 Precision Averages: Query 99: 0.1737 at 0.00 0.7518 Query 100: 0.3353 at 0.10 0.6139 Precision Query 101: 0.6658 at 0.20 0.4776 Query 102: 0.6629 at 0.30 0.4351 0.4 Query 103: 0.1824 at 0.40 0.3777 Query 104: 0.2671 at 0.50 0.3470 Query 105: 0.0000 at 0.60 0.3023 Query 106: 0.1790 at 0.70 0.2561 0.2 Query 107: 0.0882 at 0.80 0.2190 Query 108: 0.2572 at 0.90 0.1614 Query 109: 0.0829 at 1.00 0.1351 Query 111: 0.0960 0.0 Query 112: 0.8103 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 113: 0.0275 for all rel. documents Query 114: 0.1670 0.3441 Recall Query 115: 0.1131 Precision: Query 116: 0.3172 At 5 docs: 0.4408 Run humSV02 Query 117: 0.0349 At 10 docs: 0.3735 Comparison to median by topic Query 118: 0.2071 At 15 docs: 0.3320 +1.0 Query 119: 0.7097 At 20 docs: 0.2898 Query 120: 0.0111 At 30 docs: 0.2503 Query 121: 0.0833 At 100 docs: 0.1288 Query 122: 0.0489 At 200 docs: 0.0764 Query 123: 0.9587 At 500 docs: 0.0347 Query 124: 0.4823 At 1000 docs: 0.0186 +0.5 Query 125: 0.3141 Query 126: 0.0000 R-Precision (prec. after R Query 127: 0.6946 (=num_rel) docs. retrieved): Query 128: 0.1453 Exact: 0.3310 Query 129: 0.6462 0.0 Query 130: 0.5432 Query 131: 0.1440 Query 132: 0.2304 Query 133: 0.0539 Query 134: 0.8333 Difference (average precision) Query 135: 0.3669 −0.5 Query 136: 0.7024 Query 137: 0.0000 Query 138: 0.1771 Query 139: 0.2989 Query 140: 0.4646 −1.0 91 100 110 120 130 140 Topic number Hummingbird, Monolingual Swedish, Auto, TDN Run humSV02n Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.2694 Total number of documents Query 92: 0.4028 over all queries Query 93: 0.7595 Retrieved: 49000 0.8 Query 94: 0.8257 Relevant: 1196 Query 95: 0.2382 Rel_ret: 966 Query 96: 0.4902 Query 97: 0.5939 Interpolated Recall - 0.6 Query 98: 1.0000 Precision Averages: Query 99: 0.1574 at 0.00 0.7546 Query 100: 0.3045 at 0.10 0.5862 Precision Query 101: 0.8384 at 0.20 0.5368 Query 102: 0.5486 at 0.30 0.4703 0.4 Query 103: 0.1528 at 0.40 0.4234 Query 104: 0.2745 at 0.50 0.3819 Query 105: 0.0017 at 0.60 0.3421 Query 106: 0.0986 at 0.70 0.2890 0.2 Query 107: 0.0598 at 0.80 0.2388 Query 108: 0.2250 at 0.90 0.1822 Query 109: 0.1750 at 1.00 0.1318 Query 111: 0.1847 0.0 Query 112: 0.8225 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 113: 0.0069 for all rel. documents Query 114: 0.4724 0.3753 Recall Query 115: 0.0295 Precision: Query 116: 0.2503 At 5 docs: 0.4449 Run humSV02n Query 117: 0.0162 At 10 docs: 0.3980 Comparison to median by topic Query 118: 0.3359 At 15 docs: 0.3510 +1.0 Query 119: 0.6958 At 20 docs: 0.3082 Query 120: 0.0100 At 30 docs: 0.2578 Query 121: 0.1667 At 100 docs: 0.1329 Query 122: 0.0718 At 200 docs: 0.0811 Query 123: 0.9809 At 500 docs: 0.0371 Query 124: 0.5489 At 1000 docs: 0.0197 +0.5 Query 125: 0.5322 Query 126: 0.0014 R-Precision (prec. after R Query 127: 0.6043 (=num_rel) docs. retrieved): Query 128: 0.2002 Exact: 0.3683 Query 129: 0.6185 0.0 Query 130: 0.7752 Query 131: 0.0935 Query 132: 0.1725 Query 133: 0.1538 Query 134: 1.0000 Difference (average precision) Query 135: 0.4359 −0.5 Query 136: 0.6854 Query 137: 0.0016 Query 138: 0.3987 Query 139: 0.2849 Query 140: 0.4226 −1.0 91 100 110 120 130 140 Topic number Lexware Labs Ltd., Monolingual Swedish, Auto, TDN Run lexware2002 Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.0000 Total number of documents Query 92: 0.0593 over all queries Query 93: 0.0000 Retrieved: 470 0.8 Query 94: 0.4591 Relevant: 1196 Query 95: 0.0117 Rel_ret: 201 Query 96: 0.5064 Query 97: 0.1369 Interpolated Recall - 0.6 Query 98: 1.0000 Precision Averages: Query 99: 0.0000 at 0.00 0.6709 Query 100: 0.4623 at 0.10 0.4587 Precision Query 101: 0.4372 at 0.20 0.3521 Query 102: 0.1845 at 0.30 0.2191 0.4 Query 103: 0.0250 at 0.40 0.1857 Query 104: 0.0526 at 0.50 0.1691 Query 105: 0.0000 at 0.60 0.1052 Query 106: 0.0788 at 0.70 0.0608 0.2 Query 107: 0.0143 at 0.80 0.0582 Query 108: 0.0526 at 0.90 0.0204 Query 109: 0.1053 at 1.00 0.0204 Query 111: 0.0000 0.0 Query 112: 0.2000 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 113: 0.0078 for all rel. documents Query 114: 0.0667 0.1866 Recall Query 115: 0.1343 Precision: Query 116: 0.2197 At 5 docs: 0.4000 Run lexware2002 Query 117: 0.0000 At 10 docs: 0.2898 Comparison to median by topic Query 118: 0.0347 At 15 docs: 0.2190 +1.0 Query 119: 0.3748 At 20 docs: 0.1806 Query 120: 0.0000 At 30 docs: 0.1327 Query 121: 0.0000 At 100 docs: 0.0410 Query 122: 0.0656 At 200 docs: 0.0205 Query 123: 0.2583 At 500 docs: 0.0082 Query 124: 0.0132 At 1000 docs: 0.0041 +0.5 Query 125: 0.1337 Query 126: 0.2778 R-Precision (prec. after R Query 127: 0.8594 (=num_rel) docs. retrieved): Query 128: 0.2353 Exact: 0.2254 Query 129: 0.0000 0.0 Query 130: 0.4028 Query 131: 0.0000 Query 132: 0.1960 Query 133: 0.2577 Query 134: 0.5000 Difference (average precision) Query 135: 0.0364 −0.5 Query 136: 0.7844 Query 137: 0.0556 Query 138: 0.1250 Query 139: 0.2642 Query 140: 0.0562 −1.0 91 100 110 120 130 140 Topic number SICS/Conexor, Monolingual Swedish, Auto, TD Run sicsSVSV Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.1710 Total number of documents Query 92: 0.1946 over all queries Query 93: 0.5692 Retrieved: 49000 0.8 Query 94: 0.0306 Relevant: 1196 Query 95: 0.0333 Rel_ret: 836 Query 96: 0.0584 Query 97: 0.2670 Interpolated Recall - 0.6 Query 98: 0.0618 Precision Averages: Query 99: 0.2315 at 0.00 0.3513 Query 100: 0.1805 at 0.10 0.3008 Precision Query 101: 0.0360 at 0.20 0.2343 Query 102: 0.3511 at 0.30 0.1698 0.4 Query 103: 0.1169 at 0.40 0.1482 Query 104: 0.1134 at 0.50 0.1342 Query 105: 0.0002 at 0.60 0.1134 Query 106: 0.0602 at 0.70 0.0904 0.2 Query 107: 0.0862 at 0.80 0.0737 Query 108: 0.1330 at 0.90 0.0439 Query 109: 0.1485 at 1.00 0.0354 Query 111: 0.0014 0.0 Query 112: 0.1057 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 113: 0.0031 for all rel. documents Query 114: 0.0621 0.1347 Recall Query 115: 0.0194 Precision: Query 116: 0.0957 At 5 docs: 0.1714 Run sicsSVSV Query 117: 0.0010 At 10 docs: 0.1633 Comparison to median by topic Query 118: 0.0065 At 15 docs: 0.1646 +1.0 Query 119: 0.5697 At 20 docs: 0.1500 Query 120: 0.0069 At 30 docs: 0.1395 Query 121: 0.0455 At 100 docs: 0.0796 Query 122: 0.0218 At 200 docs: 0.0552 Query 123: 0.6251 At 500 docs: 0.0295 Query 124: 0.1125 At 1000 docs: 0.0171 +0.5 Query 125: 0.0137 Query 126: 0.0000 R-Precision (prec. after R Query 127: 0.1028 (=num_rel) docs. retrieved): Query 128: 0.0298 Exact: 0.1432 Query 129: 0.1057 0.0 Query 130: 0.3248 Query 131: 0.0180 Query 132: 0.2216 Query 133: 0.0120 Query 134: 0.1825 Difference (average precision) Query 135: 0.0877 −0.5 Query 136: 0.3410 Query 137: 0.0607 Query 138: 0.1306 Query 139: 0.1871 Query 140: 0.2638 −1.0 91 100 110 120 130 140 Topic number SICS/Conexor, Monolingual Swedish, Auto, TD Run sicsSVSVX Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.1804 Total number of documents Query 92: 0.2015 over all queries Query 93: 0.2998 Retrieved: 49000 0.8 Query 94: 0.0048 Relevant: 1196 Query 95: 0.1775 Rel_ret: 850 Query 96: 0.0490 Query 97: 0.1566 Interpolated Recall - 0.6 Query 98: 0.0063 Precision Averages: Query 99: 0.0121 at 0.00 0.3119 Query 100: 0.2454 at 0.10 0.2635 Precision Query 101: 0.0295 at 0.20 0.1905 Query 102: 0.1949 at 0.30 0.1274 0.4 Query 103: 0.0282 at 0.40 0.1051 Query 104: 0.0778 at 0.50 0.0919 Query 105: 0.0048 at 0.60 0.0800 Query 106: 0.0233 at 0.70 0.0641 0.2 Query 107: 0.0161 at 0.80 0.0515 Query 108: 0.0373 at 0.90 0.0297 Query 109: 0.1214 at 1.00 0.0193 Query 111: 0.0025 0.0 Query 112: 0.0979 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 113: 0.0041 for all rel. documents Query 114: 0.0896 0.1053 Recall Query 115: 0.0251 Precision: Query 116: 0.0603 At 5 docs: 0.1306 Run sicsSVSVX Query 117: 0.0004 At 10 docs: 0.1510 Comparison to median by topic Query 118: 0.0022 At 15 docs: 0.1415 +1.0 Query 119: 0.1006 At 20 docs: 0.1367 Query 120: 0.0089 At 30 docs: 0.1204 Query 121: 0.0400 At 100 docs: 0.0757 Query 122: 0.0193 At 200 docs: 0.0509 Query 123: 0.5114 At 500 docs: 0.0288 Query 124: 0.1053 At 1000 docs: 0.0173 +0.5 Query 125: 0.0937 Query 126: 0.0000 R-Precision (prec. after R Query 127: 0.0159 (=num_rel) docs. retrieved): Query 128: 0.0029 Exact: 0.1170 Query 129: 0.1423 0.0 Query 130: 0.4696 Query 131: 0.0137 Query 132: 0.2391 Query 133: 0.0057 Query 134: 0.0418 Difference (average precision) Query 135: 0.1852 −0.5 Query 136: 0.2970 Query 137: 0.0041 Query 138: 0.0706 Query 139: 0.1928 Query 140: 0.4492 −1.0 91 100 110 120 130 140 Topic number SICS/Conexor, Monolingual Swedish, Auto, T Run siteseeker Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.0909 Total number of documents Query 92: 0.0240 over all queries Query 93: 0.6593 Retrieved: 1796 0.8 Query 94: 0.5610 Relevant: 1196 Query 95: 0.0144 Rel_ret: 190 Query 96: 0.0556 Query 97: 0.1429 Interpolated Recall - 0.6 Query 98: 0.5000 Precision Averages: Query 99: 0.0833 at 0.00 0.5394 Query 100: 0.0385 at 0.10 0.3046 Precision Query 101: 0.0083 at 0.20 0.1873 Query 102: 0.0761 at 0.30 0.1154 0.4 Query 103: 0.0500 at 0.40 0.1102 Query 104: 0.0787 at 0.50 0.1029 Query 105: 0.0000 at 0.60 0.0364 Query 106: 0.0707 at 0.70 0.0350 0.2 Query 107: 0.0333 at 0.80 0.0340 Query 108: 0.0357 at 0.90 0.0290 Query 109: 0.1930 at 1.00 0.0290 Query 111: 0.2364 0.0 Query 112: 0.1345 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 113: 0.0166 for all rel. documents Query 114: 0.0383 0.1143 Recall Query 115: 0.0556 Precision: Query 116: 0.1757 At 5 docs: 0.2408 Run siteseeker Query 117: 0.0108 At 10 docs: 0.1816 Comparison to median by topic Query 118: 0.2000 At 15 docs: 0.1293 +1.0 Query 119: 0.0556 At 20 docs: 0.1051 Query 120: 0.0086 At 30 docs: 0.0762 Query 121: 0.0000 At 100 docs: 0.0290 Query 122: 0.0000 At 200 docs: 0.0160 Query 123: 0.0000 At 500 docs: 0.0076 Query 124: 0.0263 At 1000 docs: 0.0039 +0.5 Query 125: 0.0290 Query 126: 0.0000 R-Precision (prec. after R Query 127: 0.0000 (=num_rel) docs. retrieved): Query 128: 0.0075 Exact: 0.1331 Query 129: 0.0000 0.0 Query 130: 0.0000 Query 131: 0.0000 Query 132: 0.0222 Query 133: 0.0000 Query 134: 0.5000 Difference (average precision) Query 135: 0.0364 −0.5 Query 136: 0.5423 Query 137: 0.1667 Query 138: 0.1250 Query 139: 0.0000 Query 140: 0.4976 −1.0 91 100 110 120 130 140 Topic number TLR Research, Monolingual Swedish, Auto, TD Run tlrsv Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.1588 Total number of documents Query 92: 0.3121 over all queries Query 93: 0.3657 Retrieved: 48458 0.8 Query 94: 0.8248 Relevant: 1196 Query 95: 0.0464 Rel_ret: 599 Query 96: 0.1806 Query 97: 0.5589 Interpolated Recall - 0.6 Query 98: 0.5000 Precision Averages: Query 99: 0.2716 at 0.00 0.6682 Query 100: 0.2727 at 0.10 0.5441 Precision Query 101: 0.0075 at 0.20 0.4121 Query 102: 0.3455 at 0.30 0.3196 0.4 Query 103: 0.1430 at 0.40 0.2569 Query 104: 0.3735 at 0.50 0.2351 Query 105: 0.0007 at 0.60 0.1831 Query 106: 0.1711 at 0.70 0.1384 0.2 Query 107: 0.0071 at 0.80 0.1020 Query 108: 0.3231 at 0.90 0.0398 Query 109: 0.1305 at 1.00 0.0321 Query 111: 0.1364 0.0 Query 112: 0.7100 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 113: 0.0399 for all rel. documents Query 114: 0.0507 0.2439 Recall Query 115: 0.0556 Precision: Query 116: 0.0227 At 5 docs: 0.4000 Run tlrsv Query 117: 0.0549 At 10 docs: 0.3245 Comparison to median by topic Query 118: 0.2986 At 15 docs: 0.2803 +1.0 Query 119: 0.7396 At 20 docs: 0.2551 Query 120: 0.0000 At 30 docs: 0.2054 Query 121: 0.0455 At 100 docs: 0.0943 Query 122: 0.0367 At 200 docs: 0.0536 Query 123: 0.6875 At 500 docs: 0.0233 Query 124: 0.2609 At 1000 docs: 0.0122 +0.5 Query 125: 0.1261 Query 126: 0.0005 R-Precision (prec. after R Query 127: 0.1482 (=num_rel) docs. retrieved): Query 128: 0.0551 Exact: 0.2700 Query 129: 0.2131 0.0 Query 130: 0.3583 Query 131: 0.0369 Query 132: 0.2743 Query 133: 0.0197 Query 134: 0.8333 Difference (average precision) Query 135: 0.3142 −0.5 Query 136: 0.8571 Query 137: 0.0000 Query 138: 0.1912 Query 139: 0.1756 Query 140: 0.2142 −1.0 91 100 110 120 130 140 Topic number University of Amsterdam, Monolingual Swedish, Auto, TD Run UAmsC02SwSwNGram Average precision Overall statistics Recall−Precision Values (individual queries): (for 49 queries): 1.0 Query 91: 0.4894 Total number of documents Query 92: 0.4932 over all queries Query 93: 0.6833 Retrieved: 49000 0.8 Query 94: 0.9494 Relevant: 1196 Query 95: 0.1483 Rel_ret: 1102 Query 96: 0.5089 Query 97: 0.7793 Interpolated Recall - 0.6 Query 98: 0.8333 Precision Averages: Query 99: 0.2561 at 0.00 0.7323 Query 100: 0.3793 at 0.10 0.6510 Precision Query 101: 0.0769 at 0.20 0.5797 Query 102: 0.5755 at 0.30 0.5435 0.4 Query 103: 0.6328 at 0.40 0.4836 Query 104: 0.0896 at 0.50 0.4478 Query 105: 0.2881 at 0.60 0.3841 Query 106: 0.3283 at 0.70 0.3403 0.2 Query 107: 0.2653 at 0.80 0.3002 Query 108: 0.2619 at 0.90 0.2063 Query 109: 0.1388 at 1.00 0.1361 Query 111: 0.4085 0.0 Query 112: 0.7485 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 113: 0.1696 for all rel. documents Query 114: 0.4773 0.4187 Recall Query 115: 0.6048 Precision: Query 116: 0.6314 At 5 docs: 0.4980 Run UAmsC02SwSwNGram Query 117: 0.0196 At 10 docs: 0.4163 Comparison to median by topic Query 118: 0.5130 At 15 docs: 0.3728 +1.0 Query 119: 0.9096 At 20 docs: 0.3561 Query 120: 0.0217 At 30 docs: 0.3129 Query 121: 0.0294 At 100 docs: 0.1686 Query 122: 0.1152 At 200 docs: 0.0991 Query 123: 0.9452 At 500 docs: 0.0432 Query 124: 0.4585 At 1000 docs: 0.0225 +0.5 Query 125: 0.5064 Query 126: 0.0000 R-Precision (prec. after R Query 127: 0.3836 (=num_rel) docs. retrieved): Query 128: 0.0913 Exact: 0.4022 Query 129: 0.6753 0.0 Query 130: 0.5229 Query 131: 0.0706 Query 132: 0.6901 Query 133: 0.0244 Query 134: 0.2500 Difference (average precision) Query 135: 0.8182 −0.5 Query 136: 0.5482 Query 137: 0.1777 Query 138: 0.0796 Query 139: 0.7802 Query 140: 0.6674 −1.0 91 100 110 120 130 140 Topic number Univ. of Ca. at Berkeley 1, Amaryllis English, Auto, TD Run BKAMEF1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 25 queries): 1.0 Query 1: 0.5818 Total number of documents Query 2: 0.4202 over all queries Query 3: 0.0666 Retrieved: 25000 0.8 Query 4: 0.2801 Relevant: 2018 Query 5: 0.2079 Rel_ret: 1583 Query 6: 0.2449 Query 7: 0.0084 Interpolated Recall - 0.6 Query 8: 0.1812 Precision Averages: Query 9: 0.0657 at 0.00 0.6665 Query 10: 0.3584 at 0.10 0.5198 Precision Query 11: 0.2214 at 0.20 0.4370 Query 12: 0.6355 at 0.30 0.3791 0.4 Query 13: 0.0121 at 0.40 0.3346 Query 14: 0.1714 at 0.50 0.2942 Query 15: 0.5194 at 0.60 0.2481 Query 16: 0.1982 at 0.70 0.1874 0.2 Query 17: 0.5176 at 0.80 0.1251 Query 18: 0.2580 at 0.90 0.0501 Query 19: 0.6395 at 1.00 0.0074 Query 20: 0.2188 0.0 Query 21: 0.3353 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 22: 0.0016 for all rel. documents Query 23: 0.0485 0.2792 Recall Query 24: 0.5392 Precision: Query 25: 0.2473 At 5 docs: 0.4560 Run BKAMEF1 At 10 docs: 0.4960 Comparison to median by topic At 15 docs: 0.4747 +1.0 At 20 docs: 0.4600 At 30 docs: 0.4427 At 100 docs: 0.2976 At 200 docs: 0.2040 At 500 docs: 0.1080 At 1000 docs: 0.0633 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3194 0.0 Difference (average precision) −0.5 −1.0 1 5 10 15 20 25 Topic number Univ. of Ca. at Berkeley 1, Amaryllis English, Auto, TD Run BKAMEF2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 25 queries): 1.0 Query 1: 0.6355 Total number of documents Query 2: 0.3809 over all queries Query 3: 0.2761 Retrieved: 25000 0.8 Query 4: 0.2249 Relevant: 2018 Query 5: 0.3165 Rel_ret: 1897 Query 6: 0.7252 Query 7: 0.4125 Interpolated Recall - 0.6 Query 8: 0.3529 Precision Averages: Query 9: 0.2154 at 0.00 0.8079 Query 10: 0.4688 at 0.10 0.7027 Precision Query 11: 0.2875 at 0.20 0.6114 Query 12: 0.1211 at 0.30 0.5612 0.4 Query 13: 0.0044 at 0.40 0.5033 Query 14: 0.8069 at 0.50 0.4489 Query 15: 0.5196 at 0.60 0.3825 Query 16: 0.3614 at 0.70 0.3381 0.2 Query 17: 0.6907 at 0.80 0.2664 Query 18: 0.5444 at 0.90 0.1888 Query 19: 0.7647 at 1.00 0.0300 Query 20: 0.5611 0.0 Query 21: 0.6332 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 22: 0.0493 for all rel. documents Query 23: 0.3378 0.4272 Recall Query 24: 0.5458 Precision: Query 25: 0.4426 At 5 docs: 0.6480 Run BKAMEF2 At 10 docs: 0.6280 Comparison to median by topic At 15 docs: 0.6053 +1.0 At 20 docs: 0.5880 At 30 docs: 0.5493 At 100 docs: 0.4036 At 200 docs: 0.2748 At 500 docs: 0.1398 At 1000 docs: 0.0759 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.4282 0.0 Difference (average precision) −0.5 −1.0 1 5 10 15 20 25 Topic number Univ. of Ca. at Berkeley 1, Amaryllis English, Auto, TD Run BKAMEF3 Average precision Overall statistics Recall−Precision Values (individual queries): (for 25 queries): 1.0 Query 1: 0.6468 Total number of documents Query 2: 0.0587 over all queries Query 3: 0.2359 Retrieved: 25000 0.8 Query 4: 0.3773 Relevant: 2018 Query 5: 0.4624 Rel_ret: 1729 Query 6: 0.5720 Query 7: 0.0155 Interpolated Recall - 0.6 Query 8: 0.5003 Precision Averages: Query 9: 0.2815 at 0.00 0.6806 Query 10: 0.5265 at 0.10 0.6497 Precision Query 11: 0.5897 at 0.20 0.5874 Query 12: 0.4623 at 0.30 0.5337 0.4 Query 13: 0.0068 at 0.40 0.4983 Query 14: 0.7954 at 0.50 0.4452 Query 15: 0.5550 at 0.60 0.3848 Query 16: 0.2426 at 0.70 0.3114 0.2 Query 17: 0.0589 at 0.80 0.2414 Query 18: 0.2634 at 0.90 0.1570 Query 19: 0.6856 at 1.00 0.0504 Query 20: 0.6190 0.0 Query 21: 0.6100 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 22: 0.0494 for all rel. documents Query 23: 0.5698 0.4038 Recall Query 24: 0.8013 Precision: Query 25: 0.1095 At 5 docs: 0.5200 Run BKAMEF3 At 10 docs: 0.5320 Comparison to median by topic At 15 docs: 0.5173 +1.0 At 20 docs: 0.5280 At 30 docs: 0.5027 At 100 docs: 0.3584 At 200 docs: 0.2340 At 500 docs: 0.1243 At 1000 docs: 0.0692 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.4078 0.0 Difference (average precision) −0.5 −1.0 1 5 10 15 20 25 Topic number Univ. of Ca. at Berkeley 1, Amaryllis French, Auto, TDN Run BKAMFF1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 25 queries): 1.0 Query 1: 0.5170 Total number of documents Query 2: 0.6931 over all queries Query 3: 0.4797 Retrieved: 25000 0.8 Query 4: 0.5442 Relevant: 2018 Query 5: 0.4912 Rel_ret: 1935 Query 6: 0.4562 Query 7: 0.5933 Interpolated Recall - 0.6 Query 8: 0.4490 Precision Averages: Query 9: 0.2886 at 0.00 0.9242 Query 10: 0.2113 at 0.10 0.8011 Precision Query 11: 0.6341 at 0.20 0.7300 Query 12: 0.6931 at 0.30 0.6802 0.4 Query 13: 0.0603 at 0.40 0.6089 Query 14: 0.7797 at 0.50 0.5458 Query 15: 0.6319 at 0.60 0.4784 Query 16: 0.4788 at 0.70 0.4242 0.2 Query 17: 0.7256 at 0.80 0.3326 Query 18: 0.5512 at 0.90 0.2193 Query 19: 0.8350 at 1.00 0.0596 Query 20: 0.4104 0.0 Query 21: 0.7111 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 22: 0.1220 for all rel. documents Query 23: 0.4710 0.5218 Recall Query 24: 0.7792 Precision: Query 25: 0.4378 At 5 docs: 0.7680 Run BKAMFF1 At 10 docs: 0.7360 Comparison to median by topic At 15 docs: 0.7147 +1.0 At 20 docs: 0.6940 At 30 docs: 0.6627 At 100 docs: 0.4408 At 200 docs: 0.2932 At 500 docs: 0.1473 At 1000 docs: 0.0774 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.4980 0.0 Difference (average precision) −0.5 −1.0 1 5 10 15 20 25 Topic number Univ. of Ca. at Berkeley 1, Amaryllis French, Auto, TD Run BKAMFF2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 25 queries): 1.0 Query 1: 0.5781 Total number of documents Query 2: 0.2908 over all queries Query 3: 0.0643 Retrieved: 25000 0.8 Query 4: 0.2873 Relevant: 2018 Query 5: 0.4217 Rel_ret: 1863 Query 6: 0.5780 Query 7: 0.5942 Interpolated Recall - 0.6 Query 8: 0.2415 Precision Averages: Query 9: 0.1441 at 0.00 0.8175 Query 10: 0.2375 at 0.10 0.7284 Precision Query 11: 0.6601 at 0.20 0.6296 Query 12: 0.5400 at 0.30 0.5677 0.4 Query 13: 0.0088 at 0.40 0.5159 Query 14: 0.7932 at 0.50 0.4722 Query 15: 0.6445 at 0.60 0.4035 Query 16: 0.4154 at 0.70 0.3315 0.2 Query 17: 0.7464 at 0.80 0.2682 Query 18: 0.4599 at 0.90 0.1788 Query 19: 0.7697 at 1.00 0.0396 Query 20: 0.5538 0.0 Query 21: 0.6478 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 22: 0.1610 for all rel. documents Query 23: 0.4616 0.4396 Recall Query 24: 0.6271 Precision: Query 25: 0.0644 At 5 docs: 0.6480 Run BKAMFF2 At 10 docs: 0.6480 Comparison to median by topic At 15 docs: 0.6107 +1.0 At 20 docs: 0.5920 At 30 docs: 0.5533 At 100 docs: 0.3968 At 200 docs: 0.2692 At 500 docs: 0.1367 At 1000 docs: 0.0745 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.4338 0.0 Difference (average precision) −0.5 −1.0 1 5 10 15 20 25 Topic number University of Amsterdam, Amaryllis English, Auto, TD Run UAmsC02EnAmTTiKW Average precision Overall statistics Recall−Precision Values (individual queries): (for 25 queries): 1.0 Query 1: 0.3947 Total number of documents Query 2: 0.0335 over all queries Query 3: 0.1055 Retrieved: 25000 0.8 Query 4: 0.0510 Relevant: 2018 Query 5: 0.4395 Rel_ret: 1420 Query 6: 0.3984 Query 7: 0.5206 Interpolated Recall - 0.6 Query 8: 0.2209 Precision Averages: Query 9: 0.1825 at 0.00 0.7380 Query 10: 0.2680 at 0.10 0.5646 Precision Query 11: 0.1940 at 0.20 0.4634 Query 12: 0.0259 at 0.30 0.3956 0.4 Query 13: 0.0088 at 0.40 0.3145 Query 14: 0.6332 at 0.50 0.2393 Query 15: 0.4825 at 0.60 0.1876 Query 16: 0.2711 at 0.70 0.1331 0.2 Query 17: 0.3794 at 0.80 0.0807 Query 18: 0.5626 at 0.90 0.0172 Query 19: 0.5202 at 1.00 0.0034 Query 20: 0.0555 0.0 Query 21: 0.0758 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 22: 0.0761 for all rel. documents Query 23: 0.0140 0.2660 Recall Query 24: 0.5196 Precision: Query 25: 0.2172 At 5 docs: 0.5440 Run UAmsC02EnAmTTiKW At 10 docs: 0.5040 Comparison to median by topic At 15 docs: 0.5040 +1.0 At 20 docs: 0.4860 At 30 docs: 0.4467 At 100 docs: 0.2800 At 200 docs: 0.1882 At 500 docs: 0.0976 At 1000 docs: 0.0568 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3060 0.0 Difference (average precision) −0.5 −1.0 1 5 10 15 20 25 Topic number University of Amsterdam, Amaryllis English, Auto, TD Run UAmsC02EnAmTTiRR Average precision Overall statistics Recall−Precision Values (individual queries): (for 25 queries): 1.0 Query 1: 0.3229 Total number of documents Query 2: 0.0506 over all queries Query 3: 0.1220 Retrieved: 25000 0.8 Query 4: 0.0577 Relevant: 2018 Query 5: 0.4617 Rel_ret: 1337 Query 6: 0.3853 Query 7: 0.4836 Interpolated Recall - 0.6 Query 8: 0.1755 Precision Averages: Query 9: 0.2240 at 0.00 0.7535 Query 10: 0.2255 at 0.10 0.5604 Precision Query 11: 0.1798 at 0.20 0.4444 Query 12: 0.0285 at 0.30 0.3609 0.4 Query 13: 0.0165 at 0.40 0.2961 Query 14: 0.5418 at 0.50 0.2350 Query 15: 0.4963 at 0.60 0.1766 Query 16: 0.2987 at 0.70 0.1079 0.2 Query 17: 0.3306 at 0.80 0.0570 Query 18: 0.5227 at 0.90 0.0060 Query 19: 0.4495 at 1.00 0.0000 Query 20: 0.0439 0.0 Query 21: 0.0622 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 22: 0.0607 for all rel. documents Query 23: 0.0164 0.2537 Recall Query 24: 0.4227 Precision: Query 25: 0.3635 At 5 docs: 0.5920 Run UAmsC02EnAmTTiRR At 10 docs: 0.5400 Comparison to median by topic At 15 docs: 0.4827 +1.0 At 20 docs: 0.4580 At 30 docs: 0.4213 At 100 docs: 0.2740 At 200 docs: 0.1816 At 500 docs: 0.0973 At 1000 docs: 0.0535 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2957 0.0 Difference (average precision) −0.5 −1.0 1 5 10 15 20 25 Topic number University of Amsterdam, Amaryllis French, Auto, N Run UAmsC02FrAmKW Average precision Overall statistics Recall−Precision Values (individual queries): (for 25 queries): 1.0 Query 1: 0.0000 Total number of documents Query 2: 0.4131 over all queries Query 3: 0.1817 Retrieved: 24001 0.8 Query 4: 0.0295 Relevant: 2018 Query 5: 0.0226 Rel_ret: 1377 Query 6: 0.1353 Query 7: 0.1923 Interpolated Recall - 0.6 Query 8: 0.0441 Precision Averages: Query 9: 0.1095 at 0.00 0.7129 Query 10: 0.0050 at 0.10 0.5029 Precision Query 11: 0.1242 at 0.20 0.4594 Query 12: 0.1267 at 0.30 0.4143 0.4 Query 13: 0.5863 at 0.40 0.3232 Query 14: 0.4527 at 0.50 0.2719 Query 15: 0.1969 at 0.60 0.2054 Query 16: 0.1851 at 0.70 0.1529 0.2 Query 17: 0.5227 at 0.80 0.0714 Query 18: 0.7027 at 0.90 0.0382 Query 19: 0.6159 at 1.00 0.0096 Query 20: 0.6926 0.0 Query 21: 0.2663 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 22: 0.0063 for all rel. documents Query 23: 0.4813 0.2684 Recall Query 24: 0.0554 Precision: Query 25: 0.5612 At 5 docs: 0.5440 Run UAmsC02FrAmKW At 10 docs: 0.5000 Comparison to median by topic At 15 docs: 0.4640 +1.0 At 20 docs: 0.4520 At 30 docs: 0.4187 At 100 docs: 0.2820 At 200 docs: 0.1856 At 500 docs: 0.0997 At 1000 docs: 0.0551 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2908 0.0 Difference (average precision) −0.5 −1.0 1 5 10 15 20 25 Topic number University of Amsterdam, Amaryllis French, Auto, TD Run UAmsC02FrAmTT Average precision Overall statistics Recall−Precision Values (individual queries): (for 25 queries): 1.0 Query 1: 0.5640 Total number of documents Query 2: 0.0738 over all queries Query 3: 0.0309 Retrieved: 25000 0.8 Query 4: 0.1649 Relevant: 2018 Query 5: 0.4790 Rel_ret: 1423 Query 6: 0.3455 Query 7: 0.5088 Interpolated Recall - 0.6 Query 8: 0.1045 Precision Averages: Query 9: 0.2122 at 0.00 0.7464 Query 10: 0.2109 at 0.10 0.5504 Precision Query 11: 0.2212 at 0.20 0.4724 Query 12: 0.1673 at 0.30 0.3875 0.4 Query 13: 0.1040 at 0.40 0.3357 Query 14: 0.5554 at 0.50 0.2631 Query 15: 0.5244 at 0.60 0.1935 Query 16: 0.3371 at 0.70 0.1148 0.2 Query 17: 0.3245 at 0.80 0.0714 Query 18: 0.4244 at 0.90 0.0213 Query 19: 0.2675 at 1.00 0.0000 Query 20: 0.3664 0.0 Query 21: 0.0397 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 22: 0.1470 for all rel. documents Query 23: 0.0104 0.2681 Recall Query 24: 0.4368 Precision: Query 25: 0.0806 At 5 docs: 0.5280 Run UAmsC02FrAmTT At 10 docs: 0.5200 Comparison to median by topic At 15 docs: 0.5013 +1.0 At 20 docs: 0.4800 At 30 docs: 0.4453 At 100 docs: 0.2920 At 200 docs: 0.1936 At 500 docs: 0.1000 At 1000 docs: 0.0569 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3211 0.0 Difference (average precision) −0.5 −1.0 1 5 10 15 20 25 Topic number University of Amsterdam, Amaryllis French, Auto, TDN Run UAmsC02FrAmTTiKW Average precision Overall statistics Recall−Precision Values (individual queries): (for 25 queries): 1.0 Query 1: 0.5640 Total number of documents Query 2: 0.1006 over all queries Query 3: 0.0872 Retrieved: 25000 0.8 Query 4: 0.2535 Relevant: 2018 Query 5: 0.4954 Rel_ret: 1606 Query 6: 0.3542 Query 7: 0.5413 Interpolated Recall - 0.6 Query 8: 0.1259 Precision Averages: Query 9: 0.2529 at 0.00 0.8460 Query 10: 0.1993 at 0.10 0.6878 Precision Query 11: 0.3343 at 0.20 0.5517 Query 12: 0.1989 at 0.30 0.4649 0.4 Query 13: 0.2239 at 0.40 0.4100 Query 14: 0.7168 at 0.50 0.3511 Query 15: 0.5811 at 0.60 0.2631 Query 16: 0.3532 at 0.70 0.1729 0.2 Query 17: 0.4554 at 0.80 0.1129 Query 18: 0.5277 at 0.90 0.0532 Query 19: 0.4847 at 1.00 0.0108 Query 20: 0.4974 0.0 Query 21: 0.0502 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 22: 0.1367 for all rel. documents Query 23: 0.1488 0.3401 Recall Query 24: 0.4482 Precision: Query 25: 0.3710 At 5 docs: 0.6640 Run UAmsC02FrAmTTiKW At 10 docs: 0.6120 Comparison to median by topic At 15 docs: 0.5867 +1.0 At 20 docs: 0.5460 At 30 docs: 0.5027 At 100 docs: 0.3412 At 200 docs: 0.2308 At 500 docs: 0.1150 At 1000 docs: 0.0642 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3837 0.0 Difference (average precision) −0.5 −1.0 1 5 10 15 20 25 Topic number Université de Neuchâtel, Amaryllis French, Auto, TD Run UniNEama1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 25 queries): 1.0 Query 1: 0.6743 Total number of documents Query 2: 0.0960 over all queries Query 3: 0.0705 Retrieved: 25000 0.8 Query 4: 0.3713 Relevant: 2018 Query 5: 0.5924 Rel_ret: 1830 Query 6: 0.5163 Query 7: 0.5660 Interpolated Recall - 0.6 Query 8: 0.2615 Precision Averages: Query 9: 0.4157 at 0.00 0.9050 Query 10: 0.4187 at 0.10 0.7576 Precision Query 11: 0.3534 at 0.20 0.6637 Query 12: 0.6930 at 0.30 0.6076 0.4 Query 13: 0.4670 at 0.40 0.5608 Query 14: 0.7743 at 0.50 0.5034 Query 15: 0.8673 at 0.60 0.4443 Query 16: 0.6670 at 0.70 0.3894 0.2 Query 17: 0.6463 at 0.80 0.3165 Query 18: 0.5277 at 0.90 0.1961 Query 19: 0.6887 at 1.00 0.0425 Query 20: 0.7668 0.0 Query 21: 0.3333 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 22: 0.1924 for all rel. documents Query 23: 0.2343 0.4802 Recall Query 24: 0.7571 Precision: Query 25: 0.0534 At 5 docs: 0.7120 Run UniNEama1 At 10 docs: 0.6720 Comparison to median by topic At 15 docs: 0.6507 +1.0 At 20 docs: 0.6200 At 30 docs: 0.6000 At 100 docs: 0.4212 At 200 docs: 0.2748 At 500 docs: 0.1351 At 1000 docs: 0.0732 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.4696 0.0 Difference (average precision) −0.5 −1.0 1 5 10 15 20 25 Topic number Université de Neuchâtel, Amaryllis French, Auto, TD Run UniNEama2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 25 queries): 1.0 Query 1: 0.3510 Total number of documents Query 2: 0.0628 over all queries Query 3: 0.0997 Retrieved: 25000 0.8 Query 4: 0.5041 Relevant: 2018 Query 5: 0.4872 Rel_ret: 1877 Query 6: 0.3415 Query 7: 0.6348 Interpolated Recall - 0.6 Query 8: 0.4323 Precision Averages: Query 9: 0.4568 at 0.00 0.8646 Query 10: 0.4005 at 0.10 0.7250 Precision Query 11: 0.3471 at 0.20 0.6859 Query 12: 0.7531 at 0.30 0.6272 0.4 Query 13: 0.6871 at 0.40 0.5764 Query 14: 0.8184 at 0.50 0.5191 Query 15: 0.7416 at 0.60 0.4358 Query 16: 0.5036 at 0.70 0.3680 0.2 Query 17: 0.6939 at 0.80 0.2769 Query 18: 0.6112 at 0.90 0.1760 Query 19: 0.6812 at 1.00 0.0320 Query 20: 0.6841 0.0 Query 21: 0.5179 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 22: 0.1202 for all rel. documents Query 23: 0.0123 0.4734 Recall Query 24: 0.7509 Precision: Query 25: 0.1407 At 5 docs: 0.6880 Run UniNEama2 At 10 docs: 0.6800 Comparison to median by topic At 15 docs: 0.6640 +1.0 At 20 docs: 0.6360 At 30 docs: 0.5933 At 100 docs: 0.4156 At 200 docs: 0.2702 At 500 docs: 0.1377 At 1000 docs: 0.0751 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.4631 0.0 Difference (average precision) −0.5 −1.0 1 5 10 15 20 25 Topic number Université de Neuchâtel, Amaryllis French, Auto, TD Run UniNEama3 Average precision Overall statistics Recall−Precision Values (individual queries): (for 25 queries): 1.0 Query 1: 0.7347 Total number of documents Query 2: 0.0972 over all queries Query 3: 0.0701 Retrieved: 25000 0.8 Query 4: 0.2937 Relevant: 2018 Query 5: 0.4420 Rel_ret: 1808 Query 6: 0.5057 Query 7: 0.6198 Interpolated Recall - 0.6 Query 8: 0.3314 Precision Averages: Query 9: 0.4558 at 0.00 0.9144 Query 10: 0.3975 at 0.10 0.7224 Precision Query 11: 0.3332 at 0.20 0.6519 Query 12: 0.7131 at 0.30 0.5866 0.4 Query 13: 0.4649 at 0.40 0.5542 Query 14: 0.7788 at 0.50 0.5047 Query 15: 0.8657 at 0.60 0.4510 Query 16: 0.6549 at 0.70 0.3892 0.2 Query 17: 0.6977 at 0.80 0.2968 Query 18: 0.4845 at 0.90 0.1891 Query 19: 0.6577 at 1.00 0.0365 Query 20: 0.7507 0.0 Query 21: 0.2816 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 22: 0.1568 for all rel. documents Query 23: 0.2109 0.4723 Recall Query 24: 0.7731 Precision: Query 25: 0.0368 At 5 docs: 0.6880 Run UniNEama3 At 10 docs: 0.6720 Comparison to median by topic At 15 docs: 0.6533 +1.0 At 20 docs: 0.6200 At 30 docs: 0.5893 At 100 docs: 0.4184 At 200 docs: 0.2732 At 500 docs: 0.1319 At 1000 docs: 0.0723 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.4625 0.0 Difference (average precision) −0.5 −1.0 1 5 10 15 20 25 Topic number Université de Neuchâtel, Amaryllis French, Auto, TD Run UniNEama4 Average precision Overall statistics Recall−Precision Values (individual queries): (for 25 queries): 1.0 Query 1: 0.4261 Total number of documents Query 2: 0.0819 over all queries Query 3: 0.0646 Retrieved: 25000 0.8 Query 4: 0.4720 Relevant: 2018 Query 5: 0.4656 Rel_ret: 1875 Query 6: 0.3416 Query 7: 0.6311 Interpolated Recall - 0.6 Query 8: 0.4698 Precision Averages: Query 9: 0.5083 at 0.00 0.8728 Query 10: 0.4184 at 0.10 0.7436 Precision Query 11: 0.3790 at 0.20 0.6950 Query 12: 0.7415 at 0.30 0.6285 0.4 Query 13: 0.6919 at 0.40 0.5882 Query 14: 0.8178 at 0.50 0.5286 Query 15: 0.7227 at 0.60 0.4400 Query 16: 0.4821 at 0.70 0.3705 0.2 Query 17: 0.7197 at 0.80 0.2806 Query 18: 0.6190 at 0.90 0.1755 Query 19: 0.6693 at 1.00 0.0294 Query 20: 0.6919 0.0 Query 21: 0.5117 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 22: 0.1172 for all rel. documents Query 23: 0.0103 0.4778 Recall Query 24: 0.7678 Precision: Query 25: 0.1244 At 5 docs: 0.6880 Run UniNEama4 At 10 docs: 0.6880 Comparison to median by topic At 15 docs: 0.6773 +1.0 At 20 docs: 0.6420 At 30 docs: 0.5947 At 100 docs: 0.4120 At 200 docs: 0.2730 At 500 docs: 0.1378 At 1000 docs: 0.0750 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.4677 0.0 Difference (average precision) −0.5 −1.0 1 5 10 15 20 25 Topic number Université de Neuchâtel, Amaryllis French, Auto, TDN Run UniNEamaN1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 25 queries): 1.0 Query 1: 0.5633 Total number of documents Query 2: 0.2238 over all queries Query 3: 0.4300 Retrieved: 25000 0.8 Query 4: 0.6027 Relevant: 2018 Query 5: 0.7350 Rel_ret: 1942 Query 6: 0.4652 Query 7: 0.6164 Interpolated Recall - 0.6 Query 8: 0.4624 Precision Averages: Query 9: 0.4196 at 0.00 0.9697 Query 10: 0.3816 at 0.10 0.8358 Precision Query 11: 0.3618 at 0.20 0.7829 Query 12: 0.6586 at 0.30 0.7017 0.4 Query 13: 0.7168 at 0.40 0.6343 Query 14: 0.8057 at 0.50 0.5788 Query 15: 0.8386 at 0.60 0.5234 Query 16: 0.6330 at 0.70 0.4499 0.2 Query 17: 0.7636 at 0.80 0.3749 Query 18: 0.6485 at 0.90 0.2549 Query 19: 0.8011 at 1.00 0.0565 Query 20: 0.8037 0.0 Query 21: 0.2488 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 22: 0.1596 for all rel. documents Query 23: 0.2842 0.5582 Recall Query 24: 0.7249 Precision: Query 25: 0.6068 At 5 docs: 0.8160 Run UniNEamaN1 At 10 docs: 0.7880 Comparison to median by topic At 15 docs: 0.7627 +1.0 At 20 docs: 0.7280 At 30 docs: 0.6787 At 100 docs: 0.4664 At 200 docs: 0.3086 At 500 docs: 0.1476 At 1000 docs: 0.0777 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.5323 0.0 Difference (average precision) −0.5 −1.0 1 5 10 15 20 25 Topic number Univ. of Ca. at Berkeley 1, GIRT English, Auto, TD Run BKGREG1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 24 queries): 1.0 Query 51: 0.1771 Total number of documents Query 52: 0.6197 over all queries Query 53: 0.0309 Retrieved: 24000 0.8 Query 54: 0.3269 Relevant: 961 Query 55: 0.0007 Rel_ret: 735 Query 56: 0.1247 Query 57: 0.0086 Interpolated Recall - 0.6 Query 58: 0.0182 Precision Averages: Query 59: 0.3757 at 0.00 0.5257 Query 60: 0.0874 at 0.10 0.3888 Precision Query 61: 0.1341 at 0.20 0.3544 Query 62: 0.0991 at 0.30 0.3258 0.4 Query 63: 0.0585 at 0.40 0.2907 Query 64: 0.2504 at 0.50 0.2345 Query 65: 0.0418 at 0.60 0.2042 Query 67: 0.6095 at 0.70 0.1432 0.2 Query 68: 0.4620 at 0.80 0.1188 Query 69: 0.0816 at 0.90 0.0713 Query 70: 0.0592 at 1.00 0.0454 Query 71: 0.1762 0.0 Query 72: 0.0000 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 73: 0.7683 for all rel. documents Query 74: 0.7917 0.2330 Recall Query 75: 0.2907 Precision: At 5 docs: 0.3167 Run BKGREG1 At 10 docs: 0.2833 Comparison to median by topic At 15 docs: 0.2639 +1.0 At 20 docs: 0.2500 At 30 docs: 0.2222 At 100 docs: 0.1504 At 200 docs: 0.1004 At 500 docs: 0.0543 At 1000 docs: 0.0306 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2355 0.0 Difference (average precision) −0.5 −1.0 51 55 60 65 70 75 Topic number Univ. of Ca. at Berkeley 1, GIRT German, Auto, TDN Run BKGRGG1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 24 queries): 1.0 Query 51: 0.1035 Total number of documents Query 52: 0.6009 over all queries Query 53: 0.1965 Retrieved: 24000 0.8 Query 54: 0.3795 Relevant: 961 Query 55: 0.6291 Rel_ret: 853 Query 56: 0.1457 Query 57: 0.2062 Interpolated Recall - 0.6 Query 58: 0.0840 Precision Averages: Query 59: 0.2429 at 0.00 0.7450 Query 60: 0.0517 at 0.10 0.6316 Precision Query 61: 0.2154 at 0.20 0.5529 Query 62: 0.1457 at 0.30 0.5112 0.4 Query 63: 0.3640 at 0.40 0.4569 Query 64: 0.4406 at 0.50 0.4034 Query 65: 0.0744 at 0.60 0.3249 Query 67: 0.8000 at 0.70 0.2753 0.2 Query 68: 0.4293 at 0.80 0.2129 Query 69: 0.7500 at 0.90 0.1293 Query 70: 0.1320 at 1.00 0.0826 Query 71: 0.5644 0.0 Query 72: 0.4176 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 73: 0.8320 for all rel. documents Query 74: 0.7797 0.3771 Recall Query 75: 0.4657 Precision: At 5 docs: 0.5083 Run BKGRGG1 At 10 docs: 0.4375 Comparison to median by topic At 15 docs: 0.4167 +1.0 At 20 docs: 0.3854 At 30 docs: 0.3361 At 100 docs: 0.2137 At 200 docs: 0.1319 At 500 docs: 0.0644 At 1000 docs: 0.0355 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.3862 0.0 Difference (average precision) −0.5 −1.0 51 55 60 65 70 75 Topic number Univ. of Ca. at Berkeley 1, GIRT German, Auto, TD Run BKGRGG2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 24 queries): 1.0 Query 51: 0.1313 Total number of documents Query 52: 0.4433 over all queries Query 53: 0.3325 Retrieved: 24000 0.8 Query 54: 0.3613 Relevant: 961 Query 55: 0.0306 Rel_ret: 665 Query 56: 0.1685 Query 57: 0.2463 Interpolated Recall - 0.6 Query 58: 0.0603 Precision Averages: Query 59: 0.0106 at 0.00 0.6227 Query 60: 0.0038 at 0.10 0.4928 Precision Query 61: 0.1822 at 0.20 0.4554 Query 62: 0.0427 at 0.30 0.3551 0.4 Query 63: 0.1705 at 0.40 0.3095 Query 64: 0.3209 at 0.50 0.2462 Query 65: 0.0447 at 0.60 0.1953 Query 67: 0.8000 at 0.70 0.1663 0.2 Query 68: 0.0511 at 0.80 0.1323 Query 69: 0.0476 at 0.90 0.0497 Query 70: 0.1901 at 1.00 0.0216 Query 71: 0.4246 0.0 Query 72: 0.3706 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 73: 0.5996 for all rel. documents Query 74: 0.6921 0.2587 Recall Query 75: 0.4833 Precision: At 5 docs: 0.4333 Run BKGRGG2 At 10 docs: 0.3792 Comparison to median by topic At 15 docs: 0.3111 +1.0 At 20 docs: 0.2917 At 30 docs: 0.2597 At 100 docs: 0.1500 At 200 docs: 0.0954 At 500 docs: 0.0493 At 1000 docs: 0.0277 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2744 0.0 Difference (average precision) −0.5 −1.0 51 55 60 65 70 75 Topic number Univ. of Ca. at Berkeley 1, GIRT Russian, Auto, TDN Run BKGRRG1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 24 queries): 1.0 Query 51: 0.1220 Total number of documents Query 52: 0.5782 over all queries Query 53: 0.0256 Retrieved: 24000 0.8 Query 54: 0.3588 Relevant: 961 Query 55: 0.0465 Rel_ret: 710 Query 56: 0.1323 Query 57: 0.0035 Interpolated Recall - 0.6 Query 58: 0.0089 Precision Averages: Query 59: 0.0683 at 0.00 0.5617 Query 60: 0.1883 at 0.10 0.3595 Precision Query 61: 0.1846 at 0.20 0.3200 Query 62: 0.0616 at 0.30 0.2705 0.4 Query 63: 0.1143 at 0.40 0.2275 Query 64: 0.2344 at 0.50 0.1793 Query 65: 0.1111 at 0.60 0.1451 Query 67: 0.1748 at 0.70 0.0945 0.2 Query 68: 0.3981 at 0.80 0.0679 Query 69: 0.0057 at 0.90 0.0413 Query 70: 0.0300 at 1.00 0.0256 Query 71: 0.2293 0.0 Query 72: 0.0421 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 73: 0.4323 for all rel. documents Query 74: 0.8012 0.1903 Recall Query 75: 0.2156 Precision: At 5 docs: 0.3333 Run BKGRRG1 At 10 docs: 0.2708 Comparison to median by topic At 15 docs: 0.2556 +1.0 At 20 docs: 0.2333 At 30 docs: 0.2181 At 100 docs: 0.1417 At 200 docs: 0.0904 At 500 docs: 0.0495 At 1000 docs: 0.0296 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2427 0.0 Difference (average precision) −0.5 −1.0 51 55 60 65 70 75 Topic number Univ. of Ca. at Berkeley 1, GIRT Russian, Auto, TD Run BKGRRG2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 24 queries): 1.0 Query 51: 0.0224 Total number of documents Query 52: 0.6808 over all queries Query 53: 0.0908 Retrieved: 24000 0.8 Query 54: 0.4439 Relevant: 961 Query 55: 0.0074 Rel_ret: 719 Query 56: 0.0606 Query 57: 0.0018 Interpolated Recall - 0.6 Query 58: 0.0010 Precision Averages: Query 59: 0.0840 at 0.00 0.5179 Query 60: 0.0652 at 0.10 0.3603 Precision Query 61: 0.1905 at 0.20 0.3233 Query 62: 0.0890 at 0.30 0.2867 0.4 Query 63: 0.0002 at 0.40 0.2263 Query 64: 0.1894 at 0.50 0.1932 Query 65: 0.0476 at 0.60 0.1553 Query 67: 0.1879 at 0.70 0.1105 0.2 Query 68: 0.5231 at 0.80 0.0858 Query 69: 0.0069 at 0.90 0.0606 Query 70: 0.0526 at 1.00 0.0310 Query 71: 0.2266 0.0 Query 72: 0.1546 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 73: 0.6673 for all rel. documents Query 74: 0.5682 0.1973 Recall Query 75: 0.3742 Precision: At 5 docs: 0.2917 Run BKGRRG2 At 10 docs: 0.2875 Comparison to median by topic At 15 docs: 0.2500 +1.0 At 20 docs: 0.2354 At 30 docs: 0.2167 At 100 docs: 0.1471 At 200 docs: 0.1008 At 500 docs: 0.0518 At 1000 docs: 0.0300 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2112 0.0 Difference (average precision) −0.5 −1.0 51 55 60 65 70 75 Topic number Univ. Dortmund, GIRT German, Auto, TD Run GIRTsppc Average precision Overall statistics Recall−Precision Values (individual queries): (for 24 queries): 1.0 Query 51: 0.2302 Total number of documents Query 52: 0.0849 over all queries Query 53: 0.1135 Retrieved: 24000 0.8 Query 54: 0.2055 Relevant: 961 Query 55: 0.0050 Rel_ret: 625 Query 56: 0.1015 Query 57: 0.3270 Interpolated Recall - Query 58: 0.0047 Precision Averages: 0.6 Query 59: 0.6436 at 0.00 0.5650 Query 60: 0.0919 at 0.10 0.3470 Query 61: 0.0047 at 0.20 0.2849 Precision Query 62: 0.0468 at 0.30 0.2451 0.4 Query 63: 0.0005 at 0.40 0.1735 Query 64: 0.1499 at 0.50 0.1562 Query 65: 0.0016 at 0.60 0.1172 Query 67: 0.2397 at 0.70 0.0789 0.2 Query 68: 0.4239 at 0.80 0.0548 Query 69: 0.0006 at 0.90 0.0255 Query 70: 0.0968 at 1.00 0.0107 Query 71: 0.0736 0.0 Query 72: 0.4489 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 73: 0.3946 for all rel. documents Query 74: 0.1960 0.1654 Recall Query 75: 0.0853 Precision: At 5 docs: 0.3083 Run GIRTsppc At 10 docs: 0.2583 At 15 docs: 0.2389 Comparison to median by topic At 20 docs: 0.2188 +1.0 At 30 docs: 0.2014 At 100 docs: 0.1208 At 200 docs: 0.0833 At 500 docs: 0.0443 At 1000 docs: 0.0260 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.1932 0.0 Difference (average precision) −0.5 −1.0 51 55 60 65 70 75 Topic number Univ. Dortmund, GIRT German, Auto, TD Run GIRTstem Average precision Overall statistics Recall−Precision Values (individual queries): (for 24 queries): 1.0 Query 51: 0.0963 Total number of documents Query 52: 0.2224 over all queries Query 53: 0.0654 Retrieved: 24000 0.8 Query 54: 0.0570 Relevant: 961 Query 55: 0.0187 Rel_ret: 510 Query 56: 0.0688 Query 57: 0.2902 Interpolated Recall - 0.6 Query 58: 0.0044 Precision Averages: Query 59: 0.0276 at 0.00 0.5118 Query 60: 0.0043 at 0.10 0.3646 Precision Query 61: 0.0096 at 0.20 0.3174 Query 62: 0.0100 at 0.30 0.2729 0.4 Query 63: 0.0877 at 0.40 0.2272 Query 64: 0.1275 at 0.50 0.1948 Query 65: 0.0059 at 0.60 0.1583 Query 67: 0.8000 at 0.70 0.0977 0.2 Query 68: 0.0180 at 0.80 0.0839 Query 69: 0.2500 at 0.90 0.0200 Query 70: 0.1778 at 1.00 0.0041 Query 71: 0.2331 0.0 Query 72: 0.3292 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 73: 0.7093 for all rel. documents Query 74: 0.6788 0.1890 Recall Query 75: 0.2435 Precision: At 5 docs: 0.3250 Run GIRTstem At 10 docs: 0.2542 Comparison to median by topic At 15 docs: 0.2250 +1.0 At 20 docs: 0.2062 At 30 docs: 0.1681 At 100 docs: 0.0958 At 200 docs: 0.0637 At 500 docs: 0.0343 At 1000 docs: 0.0213 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2382 0.0 Difference (average precision) −0.5 −1.0 51 55 60 65 70 75 Topic number University of Amsterdam, GIRT English, Auto, TD Run UAmsC02EnGiTTiKW Average precision Overall statistics Recall−Precision Values (individual queries): (for 24 queries): 1.0 Query 51: 0.1443 Total number of documents Query 52: 0.0346 over all queries Query 53: 0.0002 Retrieved: 24000 0.8 Query 54: 0.0929 Relevant: 961 Query 55: 0.0001 Rel_ret: 253 Query 56: 0.0009 Query 57: 0.0069 Interpolated Recall - 0.6 Query 58: 0.0027 Precision Averages: Query 59: 0.0126 at 0.00 0.2694 Query 60: 0.0333 at 0.10 0.1868 Precision Query 61: 0.0019 at 0.20 0.1182 Query 62: 0.0339 at 0.30 0.0890 0.4 Query 63: 0.0000 at 0.40 0.0485 Query 64: 0.0375 at 0.50 0.0416 Query 65: 0.0005 at 0.60 0.0366 Query 67: 0.0922 at 0.70 0.0264 0.2 Query 68: 0.0164 at 0.80 0.0146 Query 69: 0.0005 at 0.90 0.0043 Query 70: 0.1746 at 1.00 0.0037 Query 71: 0.0034 0.0 Query 72: 0.1588 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 73: 0.4145 for all rel. documents Query 74: 0.0640 0.0620 Recall Query 75: 0.1618 Precision: At 5 docs: 0.1417 Run UAmsC02EnGiTTiKW At 10 docs: 0.1417 Comparison to median by topic At 15 docs: 0.1472 +1.0 At 20 docs: 0.1271 At 30 docs: 0.1056 At 100 docs: 0.0542 At 200 docs: 0.0354 At 500 docs: 0.0187 At 1000 docs: 0.0105 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.1052 0.0 Difference (average precision) −0.5 −1.0 51 55 60 65 70 75 Topic number University of Amsterdam, GIRT English, Auto, TD Run UAmsC02EnGiTTiRR Average precision Overall statistics Recall−Precision Values (individual queries): (for 24 queries): 1.0 Query 51: 0.1648 Total number of documents Query 52: 0.0413 over all queries Query 53: 0.0001 Retrieved: 24000 0.8 Query 54: 0.0715 Relevant: 961 Query 55: 0.0001 Rel_ret: 251 Query 56: 0.0004 Query 57: 0.0042 Interpolated Recall - 0.6 Query 58: 0.0159 Precision Averages: Query 59: 0.0114 at 0.00 0.3022 Query 60: 0.0196 at 0.10 0.1921 Precision Query 61: 0.0017 at 0.20 0.1489 Query 62: 0.0335 at 0.30 0.1129 0.4 Query 63: 0.0000 at 0.40 0.0554 Query 64: 0.0536 at 0.50 0.0446 Query 65: 0.0006 at 0.60 0.0317 Query 67: 0.0854 at 0.70 0.0228 0.2 Query 68: 0.0180 at 0.80 0.0143 Query 69: 0.0007 at 0.90 0.0077 Query 70: 0.1961 at 1.00 0.0008 Query 71: 0.0040 0.0 Query 72: 0.1966 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 73: 0.4268 for all rel. documents Query 74: 0.1428 0.0704 Recall Query 75: 0.2016 Precision: At 5 docs: 0.1750 Run UAmsC02EnGiTTiRR At 10 docs: 0.1583 Comparison to median by topic At 15 docs: 0.1500 +1.0 At 20 docs: 0.1437 At 30 docs: 0.1208 At 100 docs: 0.0617 At 200 docs: 0.0371 At 500 docs: 0.0189 At 1000 docs: 0.0105 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.1204 0.0 Difference (average precision) −0.5 −1.0 51 55 60 65 70 75 Topic number University of Amsterdam, GIRT German, Auto, TD Run UAmsC02GeGiTT Average precision Overall statistics Recall−Precision Values (individual queries): (for 24 queries): 1.0 Query 51: 0.2411 Total number of documents Query 52: 0.1203 over all queries Query 53: 0.1549 Retrieved: 24000 0.8 Query 54: 0.1915 Relevant: 961 Query 55: 0.0276 Rel_ret: 603 Query 56: 0.1144 Query 57: 0.1484 Interpolated Recall - 0.6 Query 58: 0.0097 Precision Averages: Query 59: 0.2451 at 0.00 0.6310 Query 60: 0.0033 at 0.10 0.4342 Precision Query 61: 0.0983 at 0.20 0.3105 Query 62: 0.1725 at 0.30 0.2396 0.4 Query 63: 0.0094 at 0.40 0.1555 Query 64: 0.1725 at 0.50 0.1306 Query 65: 0.0050 at 0.60 0.0809 Query 67: 0.3756 at 0.70 0.0567 0.2 Query 68: 0.0354 at 0.80 0.0397 Query 69: 0.1667 at 0.90 0.0099 Query 70: 0.3191 at 1.00 0.0035 Query 71: 0.1886 0.0 Query 72: 0.2125 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 73: 0.4064 for all rel. documents Query 74: 0.3500 0.1639 Recall Query 75: 0.1658 Precision: At 5 docs: 0.3500 Run UAmsC02GeGiTT At 10 docs: 0.2792 Comparison to median by topic At 15 docs: 0.2278 +1.0 At 20 docs: 0.2104 At 30 docs: 0.1708 At 100 docs: 0.1112 At 200 docs: 0.0760 At 500 docs: 0.0420 At 1000 docs: 0.0251 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2071 0.0 Difference (average precision) −0.5 −1.0 51 55 60 65 70 75 Topic number University of Amsterdam, GIRT German, Auto, TD Run UAmsC02GeGiTTiKW Average precision Overall statistics Recall−Precision Values (individual queries): (for 24 queries): 1.0 Query 51: 0.1650 Total number of documents Query 52: 0.1230 over all queries Query 53: 0.2297 Retrieved: 24000 0.8 Query 54: 0.2186 Relevant: 961 Query 55: 0.0274 Rel_ret: 636 Query 56: 0.2009 Query 57: 0.1607 Interpolated Recall - 0.6 Query 58: 0.0073 Precision Averages: Query 59: 0.3162 at 0.00 0.6339 Query 60: 0.0032 at 0.10 0.4176 Precision Query 61: 0.0905 at 0.20 0.3071 Query 62: 0.0636 at 0.30 0.2244 0.4 Query 63: 0.0269 at 0.40 0.1741 Query 64: 0.1649 at 0.50 0.1346 Query 65: 0.0068 at 0.60 0.0951 Query 67: 0.6511 at 0.70 0.0687 0.2 Query 68: 0.0706 at 0.80 0.0517 Query 69: 0.0876 at 0.90 0.0193 Query 70: 0.3255 at 1.00 0.0108 Query 71: 0.1960 0.0 Query 72: 0.2091 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 73: 0.3343 for all rel. documents Query 74: 0.2180 0.1687 Recall Query 75: 0.1527 Precision: At 5 docs: 0.3250 Run UAmsC02GeGiTTiKW At 10 docs: 0.2750 Comparison to median by topic At 15 docs: 0.2389 +1.0 At 20 docs: 0.2083 At 30 docs: 0.1847 At 100 docs: 0.1221 At 200 docs: 0.0823 At 500 docs: 0.0438 At 1000 docs: 0.0265 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2048 0.0 Difference (average precision) −0.5 −1.0 51 55 60 65 70 75 Topic number University of Amsterdam, GIRT German, Auto, TD Run UAmsC02GeGiTTiRR Average precision Overall statistics Recall−Precision Values (individual queries): (for 24 queries): 1.0 Query 51: 0.2323 Total number of documents Query 52: 0.1492 over all queries Query 53: 0.3242 Retrieved: 24000 0.8 Query 54: 0.2287 Relevant: 961 Query 55: 0.0377 Rel_ret: 603 Query 56: 0.1660 Query 57: 0.1691 Interpolated Recall - 0.6 Query 58: 0.0087 Precision Averages: Query 59: 0.3511 at 0.00 0.6685 Query 60: 0.0025 at 0.10 0.4222 Precision Query 61: 0.0843 at 0.20 0.3417 Query 62: 0.0726 at 0.30 0.2709 0.4 Query 63: 0.0249 at 0.40 0.1913 Query 64: 0.1685 at 0.50 0.1657 Query 65: 0.0055 at 0.60 0.1014 Query 67: 0.4000 at 0.70 0.0748 0.2 Query 68: 0.0643 at 0.80 0.0508 Query 69: 0.1667 at 0.90 0.0177 Query 70: 0.3676 at 1.00 0.0065 Query 71: 0.2936 0.0 Query 72: 0.2344 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 73: 0.4642 for all rel. documents Query 74: 0.3665 0.1906 Recall Query 75: 0.1908 Precision: At 5 docs: 0.3583 Run UAmsC02GeGiTTiRR At 10 docs: 0.3167 Comparison to median by topic At 15 docs: 0.2639 +1.0 At 20 docs: 0.2333 At 30 docs: 0.1986 At 100 docs: 0.1262 At 200 docs: 0.0887 At 500 docs: 0.0465 At 1000 docs: 0.0251 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.2138 0.0 Difference (average precision) −0.5 −1.0 51 55 60 65 70 75 Topic number ISU, Univ. Hildesheim, GIRT German, Auto, TD Run UHi02r1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 24 queries): 1.0 Query 51: 0.0815 Total number of documents Query 52: 0.0773 over all queries Query 53: 0.0875 Retrieved: 23751 0.8 Query 54: 0.1516 Relevant: 961 Query 55: 0.0600 Rel_ret: 387 Query 56: 0.0640 Query 57: 0.2197 Interpolated Recall - 0.6 Query 58: 0.0025 Precision Averages: Query 59: 0.2614 at 0.00 0.5582 Query 60: 0.0016 at 0.10 0.3472 Precision Query 61: 0.1040 at 0.20 0.1920 Query 62: 0.0199 at 0.30 0.1420 0.4 Query 63: 0.0079 at 0.40 0.0804 Query 64: 0.1136 at 0.50 0.0617 Query 65: 0.0299 at 0.60 0.0229 Query 67: 0.0000 at 0.70 0.0124 0.2 Query 68: 0.0057 at 0.80 0.0033 Query 69: 0.1667 at 0.90 0.0020 Query 70: 0.0601 at 1.00 0.0017 Query 71: 0.2760 0.0 Query 72: 0.1139 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 73: 0.4618 for all rel. documents Query 74: 0.2479 0.1093 Recall Query 75: 0.0097 Precision: At 5 docs: 0.2667 Run UHi02r1 At 10 docs: 0.2250 Comparison to median by topic At 15 docs: 0.2028 +1.0 At 20 docs: 0.1875 At 30 docs: 0.1542 At 100 docs: 0.0783 At 200 docs: 0.0508 At 500 docs: 0.0277 At 1000 docs: 0.0161 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.1363 0.0 Difference (average precision) −0.5 −1.0 51 55 60 65 70 75 Topic number ISU, Univ. Hildesheim, GIRT German, Auto, TD Run UHi02r2 Average precision Overall statistics Recall−Precision Values (individual queries): (for 24 queries): 1.0 Query 51: 0.0826 Total number of documents Query 52: 0.0761 over all queries Query 53: 0.0880 Retrieved: 23751 0.8 Query 54: 0.1481 Relevant: 961 Query 55: 0.0589 Rel_ret: 397 Query 56: 0.0659 Query 57: 0.2353 Interpolated Recall - 0.6 Query 58: 0.0023 Precision Averages: Query 59: 0.2587 at 0.00 0.5554 Query 60: 0.0016 at 0.10 0.3385 Precision Query 61: 0.1014 at 0.20 0.2134 Query 62: 0.0193 at 0.30 0.1502 0.4 Query 63: 0.0077 at 0.40 0.0818 Query 64: 0.1129 at 0.50 0.0602 Query 65: 0.0356 at 0.60 0.0224 Query 67: 0.0000 at 0.70 0.0149 0.2 Query 68: 0.0053 at 0.80 0.0034 Query 69: 0.1667 at 0.90 0.0022 Query 70: 0.0613 at 1.00 0.0017 Query 71: 0.2455 0.0 Query 72: 0.1082 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 73: 0.4624 for all rel. documents Query 74: 0.2752 0.1097 Recall Query 75: 0.0141 Precision: At 5 docs: 0.2750 Run UHi02r2 At 10 docs: 0.2167 Comparison to median by topic At 15 docs: 0.2000 +1.0 At 20 docs: 0.1854 At 30 docs: 0.1542 At 100 docs: 0.0771 At 200 docs: 0.0508 At 500 docs: 0.0272 At 1000 docs: 0.0165 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.1336 0.0 Difference (average precision) −0.5 −1.0 51 55 60 65 70 75 Topic number XRCE Xerox, GIRT English, Auto, T Run xrcegirt1 Average precision Overall statistics Recall−Precision Values (individual queries): (for 24 queries): 1.0 Query 51: 0.0395 Total number of documents Query 52: 0.0572 over all queries Query 53: 0.0150 Retrieved: 24000 0.8 Query 54: 0.1692 Relevant: 961 Query 55: 0.1140 Rel_ret: 663 Query 56: 0.0250 Query 57: 0.0192 Interpolated Recall - Query 58: 0.0389 Precision Averages: 0.6 Query 59: 0.4142 at 0.00 0.3978 Query 60: 0.1159 at 0.10 0.2474 Query 61: 0.1436 at 0.20 0.2095 Precision Query 62: 0.0529 at 0.30 0.1925 0.4 Query 63: 0.1087 at 0.40 0.1518 Query 64: 0.0597 at 0.50 0.1298 Query 65: 0.0122 at 0.60 0.0975 Query 67: 0.0302 at 0.70 0.0661 0.2 Query 68: 0.4370 at 0.80 0.0405 Query 69: 0.0182 at 0.90 0.0198 Query 70: 0.0572 at 1.00 0.0039 Query 71: 0.1050 0.0 Query 72: 0.0444 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 73: 0.5613 for all rel. documents Query 74: 0.3632 0.1255 Recall Query 75: 0.0103 Precision: At 5 docs: 0.1667 Run xrcegirt1 At 10 docs: 0.1917 At 15 docs: 0.1750 Comparison to median by topic At 20 docs: 0.1646 +1.0 At 30 docs: 0.1528 At 100 docs: 0.1154 At 200 docs: 0.0827 At 500 docs: 0.0467 At 1000 docs: 0.0276 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.1621 0.0 Difference (average precision) −0.5 −1.0 51 55 60 65 70 75 Topic number XRCE Xerox, GIRT English, Auto, T Run xrcegirt3 Average precision Overall statistics Recall−Precision Values (individual queries): (for 24 queries): 1.0 Query 51: 0.0395 Total number of documents Query 52: 0.0332 over all queries Query 53: 0.0197 Retrieved: 24000 0.8 Query 54: 0.1692 Relevant: 961 Query 55: 0.1140 Rel_ret: 658 Query 56: 0.0250 Query 57: 0.0179 Interpolated Recall - Query 58: 0.0204 Precision Averages: 0.6 Query 59: 0.4167 at 0.00 0.3848 Query 60: 0.0900 at 0.10 0.2380 Query 61: 0.1436 at 0.20 0.1869 Precision Query 62: 0.0191 at 0.30 0.1714 0.4 Query 63: 0.1490 at 0.40 0.1489 Query 64: 0.0637 at 0.50 0.1275 Query 65: 0.0098 at 0.60 0.0930 Query 67: 0.0302 at 0.70 0.0631 0.2 Query 68: 0.4370 at 0.80 0.0383 Query 69: 0.0118 at 0.90 0.0141 Query 70: 0.0572 at 1.00 0.0038 Query 71: 0.1003 0.0 Query 72: 0.0438 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 73: 0.4641 for all rel. documents Query 74: 0.3716 0.1191 Recall Query 75: 0.0103 Precision: At 5 docs: 0.1500 Run xrcegirt3 At 10 docs: 0.1750 At 15 docs: 0.1667 Comparison to median by topic At 20 docs: 0.1688 +1.0 At 30 docs: 0.1514 At 100 docs: 0.1129 At 200 docs: 0.0819 At 500 docs: 0.0463 At 1000 docs: 0.0274 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.1459 0.0 Difference (average precision) −0.5 −1.0 51 55 60 65 70 75 Topic number XRCE Xerox, GIRT English, Auto, T Run xrcegirt4 Average precision Overall statistics Recall−Precision Values (individual queries): (for 24 queries): 1.0 Query 51: 0.0369 Total number of documents Query 52: 0.0585 over all queries Query 53: 0.0150 Retrieved: 24000 0.8 Query 54: 0.1422 Relevant: 961 Query 55: 0.0574 Rel_ret: 653 Query 56: 0.0217 Query 57: 0.0174 Interpolated Recall - Query 58: 0.0288 Precision Averages: 0.6 Query 59: 0.3812 at 0.00 0.3322 Query 60: 0.1211 at 0.10 0.2235 Query 61: 0.1299 at 0.20 0.1800 Precision Query 62: 0.0423 at 0.30 0.1660 0.4 Query 63: 0.0822 at 0.40 0.1282 Query 64: 0.0450 at 0.50 0.1118 Query 65: 0.0109 at 0.60 0.0788 Query 67: 0.0353 at 0.70 0.0579 0.2 Query 68: 0.4174 at 0.80 0.0407 Query 69: 0.0172 at 0.90 0.0181 Query 70: 0.0440 at 1.00 0.0032 Query 71: 0.0867 0.0 Query 72: 0.0350 Avg. prec. (non-interpolated) 0.0 0.2 0.4 0.6 0.8 1.0 Query 73: 0.5309 for all rel. documents Query 74: 0.2604 0.1095 Recall Query 75: 0.0105 Precision: At 5 docs: 0.1333 Run xrcegirt4 At 10 docs: 0.1500 At 15 docs: 0.1417 Comparison to median by topic At 20 docs: 0.1417 +1.0 At 30 docs: 0.1389 At 100 docs: 0.1058 At 200 docs: 0.0773 At 500 docs: 0.0445 At 1000 docs: 0.0272 +0.5 R-Precision (prec. after R (=num_rel) docs. retrieved): Exact: 0.1453 0.0 Difference (average precision) −0.5 −1.0 51 55 60 65 70 75 Topic number