This paper presents the evaluation of a Temporal QA system for the treatment of temporal complex questions. The system was implemented as a multilayered architecture where complex temporal questions are ¯rst decomposed into simple questions, according to the temporal relations expressed in the original question. These simple questions are then processed independently by our standard Question Answering engine and their respective answers are ¯ltered to accomplish each simple question temporal restrictions. Finally, answers to simple decomposed questions are integrated following the temporal relations extracted from the original complex question in order to compose the ¯nal answer. This evaluation was performed as a pilot task at the Spanish QA Track from the Cross Language Evaluation Forum 2004.
Although current operational Question Answering systems deal with simple
factual questions, more and more systems dealing with complex questions are
needed in order to reach more complex information. One of this kind of questions
are temporal questions, that is, questions requiring a date as answer (\When did
Bob Marley die?") or questions that use temporal expressions in their
formulation (\Who won the U.S. Open in 1999?"). Processing this kind of questions is
usually accomplished by identifying explicit temporal expressions in questions
and relevant documents in order to detect temporal expressions that are
necessary to answer the queries. Even though, it seems necessary to emphasize the
system described in [
This paper shows the participation of the University of Alicante in the Spanish CLEF 2004 Pilot Task, that aims at investigating how Question Answering systems answer complex questions with temporal restriction, with more than a correct answer, or whose answer is a list of items. Our participation is focused in treating these complex temporal questions.
The treatment is based on the decomposition of complex questions into simple questions to be resolved in conventional Question Answering systems as describe above. 2
The study of two corpora [
The temporal analyzer used in this task is based on the TERSEO temporal
resolution system [
In order to perform these tasks, the system is able to classify among the following types of question: { No temporal Questions: Type 0 { Simple Temporal Questions ² Type 1: Single event temporal questions without temporal expression. Resolved by a Question Answering System directly without pre or postprocessing of the question. When did Jordan close the port of Aqaba to Kuwait? ² Type 2: Single event temporal questions with temporal expression. There are one or more temporal expressions that need to be recognized, resolved and annotated. Who won the 1988 New Hampshire republican primary?. { Complex Temporal Questions ² Type 3: Multiple events temporal questions with temporal expression.
Questions contain two or more events, related by a temporal signal. The temporal expressions need to be recognized, resolved and annotated. What did George Bush do after the U.N. Security Council ordered a global embargo on trade with Iraq in August 90? ² Type 4: Multiple events temporal questions without temporal expression.
Questions consist of two or more events, related by a temporal signal. What happened to world oil prices after the Iraqi \annexation" of Kuwait?.
The processing of these questions will be explained in detail in the following sections. 3
Our Temporal Question-Answering system is based on a multilayered architecture increasing the functionality of actual Question-Answering systems to solve any type of temporal questions. This architecture superposes an additional processing layer, one by each type of complex question, to the General Purpose Question Answering system.
Our system is focused on the Temporal Q.A. processing module, however other kinds of question could be solved according this architecture (script questions, template questions, ...) 3.1
Temporal Question Answering Module { Question Decomposition Unit, { General purpose Question-Answering system, and { Answer Recomposition Unit.
These components work all together in order to obtain a ¯nal answer. The Question Decomposition Unit and the Answer Recomposition Unit are the units that conform the Temporal Question-Answering layer which process the temporal questions, before and after using the General Purpose Question Answering system.
{ Question Decomposition Unit is a preprocessing unit which has three main
tasks. First of all, because we are dealing with questions related to
temporality, the recognition and resolution of temporal expressions of the question
is needed, if there is any. Regarding the taxonomy of the questions shown
before, there are di®erent types of questions and every type has to be treated
in a di®erent way from the others. For this reason, type identi¯cation needs
to be done. After that, complex questions, are split in simple ones. These
simple questions are the input of the General Purpose Question-Answering
system.
{ General Purpose Question-Answering system. Any generic Question-Answering
system could be used here. In this case, we use the QA System for Spanish
developed in the University of Alicante [
Temporal Tags
Question Decomposition
Unit Q1 Q2
. . . Qm General purpose
QA System A1 A2
. . . An
Answer Recomposition
Unit Answer
TE Recognition and
Resolution Type Identification Question Splitter Individual Answer
filtering Answer Comparison and composition Participants in the CLEF Spanish QA Pilot Task have to answer 100 questions, that are equally distributed throughout the following types: { Factoid questions with temporal restriction ² Date restriction: A precise date contextualises the object of the question, that can refer either to that particular moment, or before or after that date. \>Qui¶en gobernaba en Francia en 1988?". \>Qu¶e pa¶³s visit¶o Berlusconi antes de junio de 1994?". This kind of questions is recognized by our system as type 2 questions. ² Period restriction: In this case, questions refer explicitly to a whole period. \>Qui¶en gobernaba en Irak entre 1985 y 1987?". \>Cu¶antos coches se vendieron en Espan~a en la d¶ecada de los ochenta?". These questions also correspond to type 2, although TERSEO solves the temporary expression as a period of time instead of a concrete date. ² Event restriction (embedded question): Temporal restriction refers here to the moment in which a second event occurred. \>Qui¶en gobernaba en Argentina durante la guerra de las Malvinas?". \>Qu¶e le ocurri¶o al precio del crudo tras la invasi¶on iraqu¶³ de Kuwait?". These questions are recognized by our system as type 3 and 4. { Lists: Participating groups are given questions whose answer is a list of items, persons, organisations, etc. \Enumere los pa¶³ses que pertenecen a la UE". \>Qui¶en ha presidido el gobierno espan~ol desde 1992?". As our system is focused in temporal questions, most of them are not able to be solved. However, taking into account the multilayered architecture, a layer specialized in lists could be integrated to the system to confront this new challenge.
Results will be known during the CLEF 2004 Conference. 5
This paper presents the participation of the University of Alicante in the Spanish CLEF QA Pilot Task for complex questions with a new method for answering complex temporal questions using current factual-based Question Answering systems. The method is based on a multilayered architecture increasing the functionality of actual Question-Answering systems to solve any type of temporal questions. The architecture superposes an additional processing layer to the General Purpose Question Answering system.
The complex temporal QA layer is based on a new proposal for the decomposition of temporal questions where complex questions are divided into simpler ones by means of the detection of temporal signals. The TERSEO system, a temporal information extraction system applied to event ordering, has been used to detect and resolve temporal expressions in questions and answers. This information is needed in the following steps: { the decomposition of the question into simple events to generate simple questions (sub-questions), { the ordering of the sub-questions, { the ¯ltering of the sub-answers, { and ¯nally, the comparison between sub-answers to build the ¯nal complex answer.
Moreover, as TERSEO system is a multilingual system, the layer used to complex temporal questions in Spanish could be easily extended to other languages, obtaining a multilingual Temporal QA system.
Although our participation has been focussed on solving complex temporal questions, the same approach can be applied to process other kinds of complex questions that allow question decomposition such as script questions, or template-like questions.