STATISTICS FOR THE QA@CLEF TRACK QA@CLEF: Test set breakdown according to question type. F (150) D (40) L (10) T (40) NIL (20) BG 145 43 12 26 17 DE 152 38 10 39 20 EN 150 40 10 40 18 ES 148 42 10 40 21 FR 148 42 10 40 20 IT 147 41 12 38 20 NL 147 40 13 30 20 PT 142 47 11 23 20 QA@CLEF: Participants in QA@CLEF from 2003­1006. Number of New Veterans Absent America Europe Asia Australia TOTAL registered groups veterans participants CLEF 3 5 ­ ­ 8 2003 CLEF 18 1 17 ­ ­ 2004 (+125% ) CLEF 24 9 15 4 1 22 1 ­ 27 2005 (+33% ) CLEF 30 10 20 4 4 24 2 ­ 36 2006 (+25% ) QA@CLEF: Participants in QA@CLEF from 2003­1006. # of # of # of Submitted Monolingual Cross­lingual runs # Runs Runs CLEF 2003 17 6 11 CLEF 2004 48 20 28 CLEF 2005 67 43 24 CLEF 2006 77 42 35 QA@CLEF 2006: Source and Target Languages and Activated Tasks (in green) TARGET LANGUAGES (corpus and answers) BG DE EN ES FR IT NL PT BG S O DE U R EN C ES E FR L A IN N G IT U NL A G PT E S PL (questions) RO QA@CLEF 2006: Tasks chosen by at least 1 participant. MONOLINGUAL BILINGUAL CLEF 2003 CLEF 2004 6 13 CLEF 2005 8 15 CLEF 2006 7 17 QA@CLEF 2006: Results. FILE NAME OVERALL K1 MRR CWS P@N for “L” ACCURACY Questions 1. alia061enes.txt 21.58% ­0.4158 0.2132 0.09283 0 2. alia061eses.txt 37.89% ­0.1232 0.3763 0.23630 0 3. aliv061esen.txt 20.00% ­0.4210 0.2000 0.06122 0.0411 4. aliv061eses.txt 29.47% ­0.2612 0.2947 0.13970 0.0100 5. aliv062esen.txt 15.26% ­0.4892 0.1526 0.05650 0.0200 6. aliv062eses.txt 29.47% ­0.2715 0.2947 0.13862 0 7. aske061enes.txt 5.79% ­0.2967 0.0579 0.00533 0.0100 8. aske061enfr.txt 10.00% ­0.2797 0.1445 0.01662 0.0633 9. aske061esen.txt 5.26% ­0.3540 0.0526 0.00826 0 10. aske061eses.txt 13.68% ­0.2409 0.1368 0.03511 0.0300 11. aske061fren.txt 3.68% ­0.3647 0.0368 0.00341 0.0100 76. aske061fres.txt 11.58% ­0.2251 0.1158 0.03652 0.0200 12. aske061fr fr.txt 14.21% Not available 0.1974 0.14211 0.0900 13. btb061bgbg.txt 26.60% Not available 0.2660 0.10828 0.0833 14. dfki061dede.txt 42.33% Not available 0.4568 Not available 0.2593 15. dfki061deen.txt 17.89% Not available 0.1789 Not available 0.2000 16. dfki061ende.txt 32.98% Not available 0.3555 Not available 0.1000 17. dfki062dede.txt 33.33% Not available 0.3783 Not available 0.3333 18. dfki062ende.txt 26.60% Not available 0.2961 Not available 0.1000 19. dltg061fren.txt 18.95% Not available 0.1895 0.18947 0.2000 20. esfg061enpt.txt 14.89% Not available 0.1489 0.14894 0.0833 21. esfg061ptpt.txt 25.53% Not available 0.2553 0.25532 0.1667 22. esfg062ptpt.txt 23.40% Not available 0.2353 0.23404 0.1667 23. FUHA061dede.txt 32.28% 0.1799 0.3211 0.30923 0.1111 24. FUHA062dede.txt 33.86% 0.1895 0.3386 0.32194 0.1111 25. gron061ennl.txt 20.32% ­0.2540 0.2239 0.12028 0.0615 26. gron061nlnl.txt 31.02% ­0.1432 0.3460 0.22032 0.2308 27. ims061dede.txt 13.23% ­0.3386 0.1429 0.07275 0.2542 28. ims062dede.txt 12.17% ­0.3333 0.1332 0.06581 0.2643 29. inao061eses.txt 52.63% 0.0716 0.5263 0.43387 0.2000 31. ir st061iten.txt 12.63% ­0.4379 0.1263 0.12189 0.1600 30. ir st06itit.txt 22.87% ­0.4000 0.2764 0.19602 0.1528 32. isla061nlnl.txt 21.39% ­0.5722 0.2341 0.21390 0 33. isla062nlnl.txt 21.93% ­0.5615 0.2357 0.21925 0 34. jrc061bgbg.txt 11.70% Not available 0.1170 0.11702 0.0833 35. jrc062bgbg.txt 11.70% Not available 0.1170 0.11702 0.0833 36. lcc_061enes.txt 20.00% ­0.1811 0.2000 0.04916 0 37. lcc_061enfr.txt 21.05% ­0.1816 0.2623 0.04856 0.3967 38. lcc_061enpt.txt 8.51% ­0.1943 0.1328 0.01494 0.1914 39. lcea061fr fr.txt 15.79% Not available 0.1907 0.15789 0.1633 40. lina061fr fr.txt 29.47% ­0.3777 0.2947 0.25517 0.3651 41. lire061fren.txt 22.63% ­0.1782 0.2263 0.08556 0.0900 42. lire062fren.txt 25.26% ­0.1004 0.2526 0.15447 0.0800 43. mira061eses.txt 18.95% ­0.3405 0.1895 0.05708 0.0300 44. mira062eses.txt 21.05% ­0.3977 0.2105 0.07896 0.0100 45. nilc061ptpt.txt 0.00% ­0.6445 0.0000 0.00000 0 46. nilc062ptpt.txt 1.60% ­0.5134 0.0160 0.00020 0 47. prib061espt.txt 34.57% Not available 0.3457 0.34574 0.1667 48. prib061ptpt.txt 65.96% Not available 0.6596 0.65957 0.8333 49. pribe061eses.txt 53.16% Not available 0.5316 0.53158 0.4000 50. pribe061ptes.txt 36.84% Not available 0.3684 0.36842 0.2000 51. Roma061enit.txt 17.02% ­0.4205 0.1801 0.08433 0.1000 52. Roma061ennl.txt 13.37% ­0.4704 0.1430 0.04375 0.1795 53. Roma062enit.txt 17.02% ­0.4205 0.2035 0.08433 0.1500 54. Roma062ennl.txt 13.37% ­0.4704 0.1529 0.04375 0.1538 55. sinaiBruja06eses.txt 20.53% ­0.5730 0.2063 0.16384 0 56. syna061enfr.txt 45.79% Not available 0.4579 0.45789 0.2000 57. syna061fr fr.txt 68.95% 0.2832 0.6895 0.56724 0.5000 75. syna061ptfr.txt 49.47% Not available 0.4947 0.49474 0 58. syna062enfr.txt 33.16% Not available 0.3316 0.33158 0.1000 59. uaic061roen.txt 13.16% ­0.5474 0.1316 0.08666 0.1131 60. uaic062roen.txt 9.47% ­0.8105 0.0947 0.09474 0.0800 77. uind061inen.txt 7.37% ­0.1249 0.0737 0.00886 0 61. ulia061enfr.txt 35.26% ­0.1684 0.3526 0.34017 0.1000 62. ulia061fr fr.txt 46.32% 0.0684 0.4632 0.46075 0.5000 63. ulia062enfr.txt 34.74% ­0.1789 0.3474 0.33478 0.2000 64. ulia062fr fr.txt 45.79% 0.0579 0.4579 0.45546 0.3000 65. uporto061ptpt.txt 11.17% ­0.0747 0.1141 0.06875 0.1667 66. uporto062ptpt.txt 12.77% ­0.2099 0.1326 0.09633 0.1667 67. upv_061eses.txt 36.84% 0.0014 0.3684 0.22530 0 68. upv_061fr fr.txt 31.58% ­0.0047 0.3158 0.16389 0.3000 69. upv_061itit.txt 28.19% ­0.1569 0.2819 0.12330 0.0833 70. upv_062eses.txt 30.00% ­0.0333 0.3000 0.14854 0 71. upv_062fr fr.txt 24.74% ­0.0931 0.2474 0.10883 0.2000 72. upv_062itit.txt 28.19% ­0.1307 0.2819 0.13209 0.1667 73. utjp061plen.txt 86.32% 0.5838 0.8632 0.79790 0.6500 74. vein061eses.txt 42.11% ­0.0657 0.4211 0.33582 0.1000 QA@CLEF 2006: Best and Average Results in the QA@CLEF campaigns. 80 68,95 70 64,5 60 49,47 50 45,5 41,5 39,5 40 35 35 29 29,36 27,94 30 24,83 23,7 18,48 20 17 14,7 10 0 Mono Bilingual Mono Bilingual Mono Bilingual Mono Bilingual Best CLEF03 CLEF04 CLEF05 CLEF06 Average QA@CLEF 2006: Best Results in each language in the last QA@CLEF campaigns 80 68,95 65,96 64, 5 70 Best 64 2004 53,16 60 49, 5 45,5 42,33 Best 43, 5 50 2005 42 34,01 40 32,5 31,2 28,64 28,19 27, 5 27, 5 26,6 25, 5 28 22,63 24,5 Best 23,5 30 2006 20 23 10 0 Bulg arian German Eng lish Spanish Finnish French Italian Dutch Portug uese