STATISTICS FOR THE QA@CLEF TRACK
QA@CLEF: Test set breakdown according to question type.
F (150) D (40) L (10) T (40) NIL (20)
BG 145 43 12 26 17
DE 152 38 10 39 20
EN 150 40 10 40 18
ES 148 42 10 40 21
FR 148 42 10 40 20
IT 147 41 12 38 20
NL 147 40 13 30 20
PT 142 47 11 23 20
QA@CLEF: Participants in QA@CLEF from 20031006.
Number of New Veterans Absent
America Europe Asia Australia TOTAL registered groups veterans
participants
CLEF
3 5 8
2003
CLEF 18
1 17
2004 (+125% )
CLEF 24 9 15 4
1 22 1 27
2005 (+33% )
CLEF 30 10 20 4
4 24 2 36
2006 (+25% )
QA@CLEF: Participants in QA@CLEF from 20031006.
# of # of # of
Submitted Monolingual Crosslingual
runs # Runs Runs
CLEF 2003 17 6 11
CLEF 2004 48 20 28
CLEF 2005 67 43 24
CLEF 2006 77 42 35
QA@CLEF 2006: Source and Target Languages and Activated Tasks (in green)
TARGET LANGUAGES (corpus and answers)
BG DE EN ES FR IT NL PT
BG
S
O DE
U
R EN
C
ES
E
FR
L
A IN
N
G IT
U
NL
A
G PT
E
S PL
(questions)
RO
QA@CLEF 2006: Tasks chosen by at least 1 participant.
MONOLINGUAL BILINGUAL
CLEF 2003
CLEF 2004 6 13
CLEF 2005 8 15
CLEF 2006 7 17
QA@CLEF 2006: Results.
FILE NAME OVERALL K1 MRR CWS P@N for “L”
ACCURACY Questions
1. alia061enes.txt 21.58% 0.4158 0.2132 0.09283 0
2. alia061eses.txt 37.89% 0.1232 0.3763 0.23630 0
3. aliv061esen.txt 20.00% 0.4210 0.2000 0.06122 0.0411
4. aliv061eses.txt 29.47% 0.2612 0.2947 0.13970 0.0100
5. aliv062esen.txt 15.26% 0.4892 0.1526 0.05650 0.0200
6. aliv062eses.txt 29.47% 0.2715 0.2947 0.13862 0
7. aske061enes.txt 5.79% 0.2967 0.0579 0.00533 0.0100
8. aske061enfr.txt 10.00% 0.2797 0.1445 0.01662 0.0633
9. aske061esen.txt 5.26% 0.3540 0.0526 0.00826 0
10. aske061eses.txt 13.68% 0.2409 0.1368 0.03511 0.0300
11. aske061fren.txt 3.68% 0.3647 0.0368 0.00341 0.0100
76. aske061fres.txt 11.58% 0.2251 0.1158 0.03652 0.0200
12. aske061fr fr.txt 14.21% Not available 0.1974 0.14211 0.0900
13. btb061bgbg.txt 26.60% Not available 0.2660 0.10828 0.0833
14. dfki061dede.txt 42.33% Not available 0.4568 Not available 0.2593
15. dfki061deen.txt 17.89% Not available 0.1789 Not available 0.2000
16. dfki061ende.txt 32.98% Not available 0.3555 Not available 0.1000
17. dfki062dede.txt 33.33% Not available 0.3783 Not available 0.3333
18. dfki062ende.txt 26.60% Not available 0.2961 Not available 0.1000
19. dltg061fren.txt 18.95% Not available 0.1895 0.18947 0.2000
20. esfg061enpt.txt 14.89% Not available 0.1489 0.14894 0.0833
21. esfg061ptpt.txt 25.53% Not available 0.2553 0.25532 0.1667
22. esfg062ptpt.txt 23.40% Not available 0.2353 0.23404 0.1667
23. FUHA061dede.txt 32.28% 0.1799 0.3211 0.30923 0.1111
24. FUHA062dede.txt 33.86% 0.1895 0.3386 0.32194 0.1111
25. gron061ennl.txt 20.32% 0.2540 0.2239 0.12028 0.0615
26. gron061nlnl.txt 31.02% 0.1432 0.3460 0.22032 0.2308
27. ims061dede.txt 13.23% 0.3386 0.1429 0.07275 0.2542
28. ims062dede.txt 12.17% 0.3333 0.1332 0.06581 0.2643
29. inao061eses.txt 52.63% 0.0716 0.5263 0.43387 0.2000
31. ir st061iten.txt 12.63% 0.4379 0.1263 0.12189 0.1600
30. ir st06itit.txt 22.87% 0.4000 0.2764 0.19602 0.1528
32. isla061nlnl.txt 21.39% 0.5722 0.2341 0.21390 0
33. isla062nlnl.txt 21.93% 0.5615 0.2357 0.21925 0
34. jrc061bgbg.txt 11.70% Not available 0.1170 0.11702 0.0833
35. jrc062bgbg.txt 11.70% Not available 0.1170 0.11702 0.0833
36. lcc_061enes.txt 20.00% 0.1811 0.2000 0.04916 0
37. lcc_061enfr.txt 21.05% 0.1816 0.2623 0.04856 0.3967
38. lcc_061enpt.txt 8.51% 0.1943 0.1328 0.01494 0.1914
39. lcea061fr fr.txt 15.79% Not available 0.1907 0.15789 0.1633
40. lina061fr fr.txt 29.47% 0.3777 0.2947 0.25517 0.3651
41. lire061fren.txt 22.63% 0.1782 0.2263 0.08556 0.0900
42. lire062fren.txt 25.26% 0.1004 0.2526 0.15447 0.0800
43. mira061eses.txt 18.95% 0.3405 0.1895 0.05708 0.0300
44. mira062eses.txt 21.05% 0.3977 0.2105 0.07896 0.0100
45. nilc061ptpt.txt 0.00% 0.6445 0.0000 0.00000 0
46. nilc062ptpt.txt 1.60% 0.5134 0.0160 0.00020 0
47. prib061espt.txt 34.57% Not available 0.3457 0.34574 0.1667
48. prib061ptpt.txt 65.96% Not available 0.6596 0.65957 0.8333
49. pribe061eses.txt 53.16% Not available 0.5316 0.53158 0.4000
50. pribe061ptes.txt 36.84% Not available 0.3684 0.36842 0.2000
51. Roma061enit.txt 17.02% 0.4205 0.1801 0.08433 0.1000
52. Roma061ennl.txt 13.37% 0.4704 0.1430 0.04375 0.1795
53. Roma062enit.txt 17.02% 0.4205 0.2035 0.08433 0.1500
54. Roma062ennl.txt 13.37% 0.4704 0.1529 0.04375 0.1538
55. sinaiBruja06eses.txt 20.53% 0.5730 0.2063 0.16384 0
56. syna061enfr.txt 45.79% Not available 0.4579 0.45789 0.2000
57. syna061fr fr.txt 68.95% 0.2832 0.6895 0.56724 0.5000
75. syna061ptfr.txt 49.47% Not available 0.4947 0.49474 0
58. syna062enfr.txt 33.16% Not available 0.3316 0.33158 0.1000
59. uaic061roen.txt 13.16% 0.5474 0.1316 0.08666 0.1131
60. uaic062roen.txt 9.47% 0.8105 0.0947 0.09474 0.0800
77. uind061inen.txt 7.37% 0.1249 0.0737 0.00886 0
61. ulia061enfr.txt 35.26% 0.1684 0.3526 0.34017 0.1000
62. ulia061fr fr.txt 46.32% 0.0684 0.4632 0.46075 0.5000
63. ulia062enfr.txt 34.74% 0.1789 0.3474 0.33478 0.2000
64. ulia062fr fr.txt 45.79% 0.0579 0.4579 0.45546 0.3000
65. uporto061ptpt.txt 11.17% 0.0747 0.1141 0.06875 0.1667
66. uporto062ptpt.txt 12.77% 0.2099 0.1326 0.09633 0.1667
67. upv_061eses.txt 36.84% 0.0014 0.3684 0.22530 0
68. upv_061fr fr.txt 31.58% 0.0047 0.3158 0.16389 0.3000
69. upv_061itit.txt 28.19% 0.1569 0.2819 0.12330 0.0833
70. upv_062eses.txt 30.00% 0.0333 0.3000 0.14854 0
71. upv_062fr fr.txt 24.74% 0.0931 0.2474 0.10883 0.2000
72. upv_062itit.txt 28.19% 0.1307 0.2819 0.13209 0.1667
73. utjp061plen.txt 86.32% 0.5838 0.8632 0.79790 0.6500
74. vein061eses.txt 42.11% 0.0657 0.4211 0.33582 0.1000
QA@CLEF 2006: Best and Average Results in the QA@CLEF campaigns.
80
68,95
70 64,5
60
49,47
50 45,5
41,5 39,5
40 35 35
29 29,36 27,94
30 24,83
23,7
18,48
20 17 14,7
10
0
Mono
Bilingual
Mono
Bilingual
Mono
Bilingual
Mono
Bilingual
Best
CLEF03 CLEF04 CLEF05 CLEF06 Average
QA@CLEF 2006: Best Results in each language in the last QA@CLEF campaigns
80
68,95
65,96
64, 5
70 Best
64
2004
53,16
60
49, 5
45,5
42,33
Best
43, 5
50
2005
42
34,01
40 32,5
31,2
28,64
28,19
27, 5
27, 5
26,6
25, 5
28
22,63
24,5
Best
23,5
30
2006
20 23
10
0
Bulg arian
German
Eng lish
Spanish
Finnish
French
Italian
Dutch
Portug uese