between questions and passages ontaining the answers and to validate the type senten es (module D1) and the answer type, when spe ied in the question, is Our approa h onsists in he king if all the hara teristi s of a question The main obje tive of FIDJI is to produ e answers whi h are fully validated of the potential answer in this passage or in another do ument. validated by several do uments. or several do uments. In this ontext, FIDJI has to dete t synta ti impli ations (namely the dependen y relations and the answer type) may be retrieved in one is that an answer (or some pie es of information omposing an answer) may be validated (module E). by a supporting text (or passage) with respe t to a given question. The di ult y
details on erning the system an be found in [1℄. Sin e the last evaluation ampaign in 2009, FIDJI has been adapted to English. Spe i rules have been developped for question analysis (module A) and glish and Fren h. do ument pro essing (module B). The other modules are ommon to both EnFig. 1. Ar hite ture of FIDJI resented by the ‘ANSWER’ lemma. The following senten e is sele ted be ause The question is turned into a de larative senten e where the answer is repit ontains the highest number of dependen y relations: tions, as well as the question type and the expe ted answer type. For example: Question analysis provides lemmatisation, POS tagging and dependen y rela
Dependen ies: attribut(sky, blue) validate the andidate answer, the system sear hes the missing dependen ies (Prime Minister Pierre BØrØgovoy) and the answer will be validated. a single senten e of the whole do ument olle tion. These dependen ies will be Pierre BØrØgovoy instantiates the ANSWER slot of the question dependen ies three dependen ies of the question are validated in this senten e. In order to fully found in any senten e speaking about le premier ministre Pierre BØrØgovoy (attribut(Pierre BØrØgovoy, ministre) and attribut(ministre, premier) ) in and be omes a andidate answer. The named entity type (person) and the rst of questions, the system behaves as a lassi al passage retrieval system, ex ept dis ourse markers (about 100 nouns, verbs, prepositions and adje tives, manually ompiled) instead of keywords only. Here is an example of a omplex question: For omplex questions, it is obvious that answers are not always short phrases. that andidate passages are retrieved through synta ti relations and relevant For this reason, FIDJI provides a full passage as an answer. On these kinds
SUBJ(se sui ider, Pierre BØrØgovoy) (Pierre BØrØgovoy ommitted sui ide in 1993.) DATE(1993) PERSON(Pierre BØrØgovoy) Pierre BØrØgovoy s’est sui idØ en 1993. (Whi h Prime Minister ommitted sui ide in 1993? ) PERSON(ANSWER) attribut(ministre, premier) attribut(ANSWER, ministre) Expe ted answer type: person (spe i Question: Quel premier ministre s’est sui idØ en 1993 ? SUBJ(se sui ider, ANSWER) Dependen ies: DATE(1993) Question type: fa toid
lations of the question and a ausal marker:
taining a maximum of question signi an t keywords (module C2), and answer exIn 2009, ResPubliQA results learned us a lot about the behavior of our system.
Other evaluations (former CLEF and Quaero ampaigns) had shown that C1 and D1 in Figure 1. uation set, passage extra tion turned out to be mu h better by repla ing syntax tra tion is a hieved without slot instantiation within dependen ies (module D2). by traditional bag-of-words te hniques [3℄. This is done by turning o modules answers signi an tly improved the results [1℄. However, with ResPubliQA evalusing synta ti analysis modules for retrieving do uments and extra ting the Expe ted answer type: reason2 Question type: omplex (why) modules on the same task. an obtain a PS run with modules turned on, whi h allows us to evaluate 2. AS task, synta ti modules turned on, in order to test whether answer expassage retrieval, that had the best results of the system last year. tra tion was ee tiv e or not on this olle tion. Moreover, by adding answers 1. PS task, synta ti modules turned o , leading to an approa h loser to with INEXA CT, MISSED and RIGHT status from our AS run, we PS task with synta ti modules turned on (exa t answers of judged as ➁: ➀ PS task with synta ti modules turned o. ➂: RIGHT), RIGHT, INEXA CT, MISSED), AS task with synta ti modules turned on (exa t answers judged as ➀: We present the results of 5 experiments for both Fren h and English. The rst three ome from o ial ResPubliQA runs: this issue3. answer would be odd with the English question. We identied 7 questions raising
6 A knowledgements This work has been partially nan ed by OSEO under the Quaero program. Referen es