Germany iaidetd01 CMUbll25e3tdn25 Bi CMUbnn25e2tdn15 Bi DE TDN Auto Y
APL/JHU APL/JHU APL/JHU APL/JHU APL/JHU APL/JHU APL/JHU APL/JHU APL/JHU APL/JHU APL/JHU APL/JHU APL/JHU APL/JHU APL/JHU APL/JHU APL/JHU APL/JHU CMU CMU CMU CMU CMU CMU Eidetica Eidetica Eidetica Eidetica Eidetica Eidetica Eurospider Eurospider Eurospider Eurospider Eurospider Eurospider Eurospider Eurospider Eurospider Greenwich U Greenwich U Greenwich U HKUST Hummingbird Hummingbird Hummingbird Hummingbird Hummingbird Hummingbird Hummingbird Hummingbird Hummingbird Hummingbird USA USA USA UK UK UK Netherlands EidDE2001A Netherlands EidDE2001B Netherlands EidDE2001C Netherlands EidNL2001A Netherlands EidNL2001B Netherlands EidNL2001C Switzerland EIT01FFFN Multi DE TDN Auto Multi DE TDN Auto Mono FR T Auto Bi France France France France France France Italy Italy Italy Italy Thailand Thailand Thailand Thailand Canada Canada Canada Canada Canada Canada Canada Canada Canada iritmonFR iritmonGE iritmonIT iritmonSP thaiexp thaiexpa thaiseg thaithes kcslbilint kcslbilintd kcslbilintdn kcslmonot kcslmonotd kcslmonotdn kcslmultit kcslmultitd kcslmultitdn naraDOM naraFEED naraFR naraORG naraTR Nara Inst. of Tech. NAIST Japan Nara Inst. of Tech. NAIST Japan Nara Inst. of Tech. NAIST Japan Nara Inst. of Tech. NAIST Japan Nara Inst. of Tech. NAIST Japan National Taiwan U National Taiwan U National Taiwan U National Taiwan U
Netherlands medialab Taiwan ROC NTUa1wco Taiwan ROC NTUaswtw Taiwan ROC NTUco
Taiwan ROC NTUtpwn IRIT IRIT IRIT IRIT IRIT IRIT IRST IRST IRST IRST Kasetsart U Kasetsart U Kasetsart U Kasetsart U KCSL Inc.
KCSL Inc.
KCSL Inc.
KCSL Inc.
KCSL Inc.
KCSL Inc.
KCSL Inc.
KCSL Inc.
KCSL Inc.
Medialab SICS/Conexor SICS/Conexor SICS/Conexor SICS/Conexor SINAI/U Jaen SINAI/U Jaen SINAI/U Jaen Thomson L&R Thomson L&R Thomson L&R Thomson L&R Thomson L&R Thomson L&R Thomson L&R Thomson L&R Thomson L&R Thomson L&R Thomson L&R IT TD Auto TH D Auto Y FR TD Auto Y FR TDN Auto
Auto
Auto Bi Bi Bi Bi Bi Bi Bi Bi Bi Bi Bi Bi Bi Bi Bi Bi Bi Bi Bi Bi Bi Bi Bi Bi Bi Bi Bi Bi Bi Thomson L&R TNO TNO TNO TNO TNO TNO TNO TNO TNO TNO TNO TNO TNO TNO TNO TNO TNO U Alicante U Alicante U Alicante U Alicante U Alicante U Alicante U Alicante U Amsterdam U Amsterdam U Amsterdam U Amsterdam U Amsterdam U Amsterdam U Amsterdam U Amsterdam U Amsterdam U Exeter U Exeter U Exeter U Exeter U Glasgow U Montreal U Montreal U Montreal U Montreal U Neuchatel U Neuchatel U Neuchatel U Neuchatel U Neuchatel U Neuchatel U Neuchatel U Neuchatel U Neuchatel U Salamanca U Salamanca U Salamanca tlrittd tlrnltd Netherlands tnodd1 Canada Canada Canada Canada
ClefSPAutEI ClefSPAutM ClefSPAutPR ClefSPENAutEI Bi ClefSPENAutEXP Bi ClefSPENAutM Bi ClefSPENAutPR Bi glaenl RaliMidfF2E RaliP01D2E RaliP01F2E
RaliP01I2E Bi Bi Bi Mono ES TD Auto Y Mono ES TD Auto Bi Bi Bi Bi Bi Bi Bi U Tampere U Tampere U Tampere U Twente UC Berkeley 1 UC Berkeley 1 UC Berkeley 1 UC Berkeley 1 UC Berkeley 1 UC Berkeley 1 UC Berkeley 1 UC Berkeley 1 UC Berkeley 1 UC Berkeley 1 UC Berkeley 1 UC Berkeley 2 UC Berkeley 2 UC Berkeley 2 UC Berkeley 2 UC Berkeley 2 UC Berkeley 2 UC Berkeley 2 UC Berkeley 2 UC Berkeley 2 UC Berkeley 2 Explanations: Runtag Task Topic Language Topic Fields Type Pool
Finland Finland Finland Finland USA USA USA TAYgershort TAYgerstr TAYswestr BKGRGGA BKGRRGA1 BK2MUCAA1 BK2MUEAA1 BK2SSA1 Bi Bi Bi Bi Bi Bi Bi Bi Experiment identifier. Matches the identifiers on the individual result pages. Y = run was used for formation of the pools judged by relevance assessors. U E U A U (A ievB sk ,nF laauT itirsceo p euN rsopu eCB /JPLH reCB − tom F E L C 8 . 0 6 . 0 4 . 0 2 . 0 0 . 1 8 . 0 R & (skA tseeBG aT ,ivF h n
o is ti lg ir
p nE secD to le+ la ti
T u ,c g it in a l m i to B uA − 1 0 0 2 2 . 0 T A U sak ievB T ,nF
o ch ti
p tu irc D se ag sp r u ev ro
G (A tse irbgd LR& adm aug itirop iln sce rag sp (A tseG k B sa iev lT ,nF
TNO iunHmm sohonTm /JPLAHU trseAUm ono ltie+D M,T
c tch tia
m u to −D uA 0 . 1 8 . 0 6 . 0 4 . 0 2 . 0 0 . 1 8 . 0 (A tsG sk eeB laauT ,itinopvF ng irc li se o D n + o lte MiT ) n o i s i c e r P e g ra s e up v ro cnh ,tiac
m re to F uA F E L C 8 . 0 6 . 0 4 . 0 2 . 0 0 . 1 8 . 0 (A tseG k B s e a iv 8 . 0 6 . 0 4 . 0 2 . 0 2 R ey L& l ek on r e s (A tsG ) n o i s i c e r P e g ra s lsaangukT ,itiireenovFB p c liono lsee+D MitT h ,c is ti n a
m ap tou S A 0 . 1 8 . 0 6 . 0 4 . 0 2 . 0 2 . 0 eg sp a uo rev trs
G ITR ,tiacT 1−G tuoAm 0 . 1 8 . 0 6 . 0 4 . 0 2 . 0 R 1 8 . 0 ) noisi cer p egareva( ecnereffi D .50+ .00 .50− 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T tit sta que Au, evrO (for lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor P 8 . 0 c 2 ip
o At
y Ab Can
i Ud
e Mm 2 to 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T tit sta que Au, evrO (for lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor P R 8 . 0 c 1 ip
o At
y Ab Ean
i Ud
e Mm 2 to 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T tit sta que Au, evrO (for lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor P R 8 . 0 c 2 ip
o At
y Ab Ean
i Ud
e Mm 2 to 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T tit sta que Au, evrO (for lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor P 0 . 1 8 . 0 uRpom 8 .
0 0 . 1 2 .
0 2 . 0
C 0 7 r e b m u n c i p o
T itl sta que ,totAu lalrvOe 50r(fo , l a u g n : i ) l s i lt on rie u i e M si uq lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor ion(prs l)docse t:c 0 . 1 8 . 0 uRpom 8 .
0 0 . 1 2 .
0 2 . 0
C 0 7 r e b m u n c i p o
T itl sta que ,totAu lalrvOe 50r(fo , l a u g n : i ) l s i lt on rie u i e M si uq lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor ion(prs l)docse t:c 1 8 . 0 GnV ) noisi cer p egareva( ecnereffi D .50+ .00 .05− 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T tit sta que , l a u g n : i ) l s lt on rie i u i e M si uq lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor P R 8 . 0 An C ll− ) noisi cer p egareva( ecnereffi D .50+ .00 .50− 8 .
0 0 . 1 2 .
0 2 . 0
C 0 7 r e b m u n c i p o
T , l a u g n i l : o ) n s M on rie o , i e RP si uq ut ec l PA pr uad S ef ge vi l a i C r d lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor P 8 . 0 R R inNirsce ) noisi cer p egareva( ecnereffi D .05+ .00 .05− 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T itl sta que ,totAu lalrvOe 49r(fo , l a u g n : i ) l s on on rie o M i e , is qu es ec l mo pr uad E iN ge vi n a i U r d lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor terR ievd)e 556 R s 8 . 0 o n unR llace− 4 .
0 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T tle sta que , l a u g : n ) i s l no on rie o i e M si uq a, ec l lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor terR ievd)e 097 s 8 . 0 o n ap tno 6 . 0 4 .
0 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T 0 8 0 6 0 5 1 4 tle sta que , l a u g : n ) i s l no on rie o i e M si uq b, ec l lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor terR ievd)e 851 s 8 . 0 01 laeu hu recP ) noisi cer p egareva( ecnereffi D .50+ .00 .50− 6 . 0 4 .
0 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T 0 8 0 6 0 5 1 4 , l a u : g ) n s i ol on rie n i e oM si uq s t n e m u c o lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor terR ievd)e 331 8 . 0 c i p o x t 1 by 0 n Sia
Eed 6 . 0 4 .
0 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T 0 8 0 6 0 5 1 4 tle sta que tito,Au allerOv 49(for , l a u g : n ) i s no on rie l o i e M si uq x, ec l s t n e m u c o lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor terR ievd)e 296 8 . 0 ir reP
s SP laeu iir tno 6 . 0 4 .
0 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T 0 8 0 6 0 5 1 4 itl sta que ,totAu lalrvOe 49r(fo , l a u g n : i ) l s o on on rie M i e , is qu SP ec l s t n e m u c o
49000 2694 2176 erofd ueiers ved: nt: t: lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor terR ievd)e 459 8 . 0 c to k n n iso u ra Rp m o C c i p t to 6 . 0 4 .
0 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T 0 8 0 6 0 5 1 4 tlei tats quer ,totAu lalrvOe 49r(fo , l a u g n : i ) l s on on rie o M i e , is qu s t n e m u c o
1571 9426 728 erofd ueiers ved: nt: t: lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor terR ievd)e 514 8 . 0 tond lseauV k no n irs u ap Rm o C c i p d to t y o b me l m sc to 6 . 0 4 .
0 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T 0 8 0 6 0 5 1 4 tit sta que o,t lal 49 , l a u g n i : l ) o s n Mo on rie
i e ,d si uq s t n e m u c o
5936 9426 897 erofd ueiers ved: nt: t: lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor terR ievd)e 792 8 . 0 on noV
i k ll− n ca uR eR lscmirsceP c n k so
i n ra u p Rom
C tnod ianyb o d
e 6 . 0 4 .
0 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T 0 8 0 6 0 5 1 4 to,Auh, allreOv 49(for , l a u g n i l : o ) n s o M on rie , i e nd si uq s t n e m u c o
2239 9426 517 erofd ueiers ved: nt: t: lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor terR ievd)e 723 8 . 0
e n rP u l− R lac e
R un ton
s Rir a p m o
C trsed iaend tl om c i p o t y b 6 . 0 4 .
0 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T 0 8 0 6 0 5 1 4 tit lal 49 to,u verO (for A , l a u : g ) n s i ol on rie n i e oM si uq td pr uad s re ge vi l a i t r d s t n e m u c o lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor terR ievd)e 132 8 . 0 R eR noisi cer P
.60 .04 tlr tno 6 . 0 4 .
0 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T 0 8 0 6 0 5 1 4 tit sta que o,t lal 49 , l a u g n i : l ) o s n Mo on rie
i e ,c si uq td pr uad s re ge vi l a i t r d s t n e m u c o
49000 2694 2621 erofd ueiers ved: nt: t: lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor terR ievd)e 280 8 . 0 R ec
R se edm r tl to un irson R ap m o C c i p o t 6 . 0 4 .
0 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T 0 8 0 6 0 5 1 4 itl sta que ,totAu lalrvOe 49r(fo , l a u g n : i ) l s o on on rie M i e , is qu td pr uad s re ge vi l a i t r d s t n e m u c o
49000 2694 2586 erofd ueiers ved: nt: t: lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor terR ievd)e 095 8 . 0 6 . 0 4 .
0 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T 0 8 0 6 0 5 1 4 esc tis rie e+d sta que itltto, lalrOve 49r(fo u A , l a : u ) g s n li on rie o i e no si uq s t n e m u c o
49000 2694 2542 erofd ueiers ved: nt: t: lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor terR ievd)e 058 8 . 0 6 . 0 4 .
0 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T 0 8 0 6 0 5 1 4 esc tis rie e+d sta que itltto, lalrOve 49r(fo u A , l a : u ) g s li on rie n o i e no si uq 3, pr uad s os ge vi n a i t r d s t n e m u c o
49000 2694 2545 erofd ueiers ved: nt: t: lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor terR ievd)e 213 8 . 0 sau iisceo 6 . 0 4 .
0 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T 0 8 0 6 0 5 1 4 arrn csi :) + t s c s e s i i e t r e+d sta que itltto, lalrOve 49r(fo u A , l a : u ) g s li on rie n o i e no si uq 1, pr uad 0 al ge vi s a i u r d s t n e m u c o
49000 2694 2460 erofd ueiers ved: nt: t: lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor terR ievd)e 088 8 . 0 sau iisceo 6 . 0 4 .
0 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T 0 8 0 6 0 5 1 4 arrn csi :) + t s c s e s i i e t r e+d sta que itltto, lalrOve 49r(fo u A , l a : u ) g s li on rie n o i e no si uq 2, pr uad 0 al ge vi s a i u r d s t n e m u c o
49000 2694 2489 erofd ueiers ved: nt: t: lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor terR ievd)e 383 8 . 0 sau iisceo 6 . 0 4 .
0 8 .
0 0 . 1 2 .
0 2 . 0 0 7 r e b m u n c i p o
T 0 8 0 6 0 5 1 4 arrn csi :) + t s c s e s i i e t r e+d sta que itltto, lalrOve 49r(fo u A , l a : u ) g s n li on rie o i e no si uq 3, pr uad 0 al ge vi s a i u r d s t n e m u c o
49000 2694 2432 erofd ueiers ved: nt: t: lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor s: s: s: s: s: s: s: s: s: c c c c c c c c c terR ievd)e 073 8 . 0 R R c i p o At
y Gb
n Ga
i Red Gm
o Kt Bon
s n ir u ap Rm o
C 6 . 0 4 .
0 8 .
0 0 . 1 2 .
0 2 . 0 40 re b m u n c i p o
T 5 4 5 3 0 3 arrn tats quer : ) s T, on rie R i e IG si uq , ec l s t n e m u c o erofd ueiers ved: nt: t: lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor s: s: s: s: s: s: s: s: s: c c c c c c c c c terR ievd)e 839 26: 27: 28: 29: 30: 31: 32: 33: 34: 35: 36: 37: 38: 39: 40: 41: 42: 43: 44: 45: 46: 47: 48: 49: 50: R 8 . 0 1 s Aeu
l B ll− un ecaR c i 1 op
t Ay
b
Gn 6 . 0 4 .
0 8 .
0 0 . 1 2 .
0 2 . 0 40 re b m u n c i p o
T 5 4 5 3 0 3 arr tis rie c+n sta que des+ all 25 itle Over (for , n a i s s u R : ) s 1, ec l s t n e m u c o
00250 1111 747 erofd ueiers ved: nt: t: lnumaTot allrove etrR Rele Rel_ prec.(nonAvg. lalre.ldofor s: s: s: s: s: s: s: s: s: c c c c c c c c c ion(prs l)docse t:c i r a cer _mu xE 26: 27: 28: 29: 30: 31: 32: 33: 34: 35: 36: 37: 38: 39: 40: 41: 42: 43: 44: 45: 46: 47: 48: 49: 50: R 8 . 0 2 s Aeu
l B ll− un ecaR c i 2 op
t Ay
b
Gn 0 . 1 + 6 . 0 4 .
0 8 .
0 0 . 1 2 .
0 2 . 0 40 re b m u n c i p o
T 5 4 5 3 0 3 arr tis rie c+n sta que des+ all 25 itle Over (for , n a i s s u R : ) s 2, ec l s t n e m u c o
00250 1111 817 erofd ueiers ved: nt: t: lnumaTot allrove etrR Rele Rel_
032 018 441 257 790 341 993 712 502 232 013 prec.(nonAvg. lalre.ldofor 320 280 293 160 973 304 926 514 312 s: s: s: s: s: s: s: s: s: c c c c c c c c c ec.af .retr 0.1 ion(prs l)docse t:c i r a cer _mu xE 26: 27: 28: 29: 30: 31: 32: 33: 34: 35: 36: 37: 38: 39: 40: 41: 42: 43: 44: 45: 46: 47: 48: 49: 50: RGanV c i 3 op
t Ay
b
Gn 8 .
0 0 . 1 2 .
0 0 3 , n a i s s u R s t n e m u c o
00250 1111 138 erofd ueiers ved: nt: t:
793 640 381 984 782 609 323 125 765 273 000