An Automatic System for Modality and Negation Detection

Abstract

The article presents the experiments carried out as part of the participation in the pilot task (Modality and Negation)1 of QA4MRE@CLEF 2012. Modality and Negation are two main grammatical devices that allow to express extra-propositional aspects of meaning. Modality is a grammatical category that allows to express aspects related to the attitude of the speaker towards statements. Negation is a grammatical category that allows to change the truth value of a proposition. The input for the systems is a text where all events expressed by verbs are identified and numbered the output should be a label per event. The possible values are: mod, neg, neg-mod, none. In the developed system, we first build a database for modal verbs of two categories: epistemic and deontic. Also, we used a negative verb list of 1877 verbs. This negative verb list has been used to identify negative modality. We extract the each tagged events from each sentences. Then our system check modal verbs by that database from each sentences. If any modal verbs is found before that an event then that event should be modal verb and tagged as mod. If modal verb is there and also negeted words is found before that evet then that event should negeted mod and tagged as neg-mod. If no modal verb is found before that an event but negeted word are found before that event then that event should be negeted and tagged as neg. Otherwise the event should tagged as none. We trained our system by traing data (sample data) that was provided by QA4MRE organizer. Then we are tested our system on test dataset. In test data set there are eight documents, two per each of the four topics such as Alzheimer, music and society, AIDs and climate change. Our system overall accuracy is 0.6262 (779 out of 1244).

Keywords

• QA4MRE Data Sets
• Modal Verbs List

1 Introduction

Corpus Statistics

The organizer provided a test set consisting of 8 documents, 2 per topic. Documents annotated as shown in Table 1 in ( 1 ), where events are marked in the text, assigned an identification number and label per event with the format shown in Table 1 in ( 2 ). The possible values are: mod, neg, neg-mod, none. There are eight documents, two per each of the four topics, such as: Alzheimer, music and society, AIDs and climate change. For each document organizer provided the text version in the directory "txt" and the version with marked events in the directory "events" is shown in Table 2. This task was defined as an annotation task where systems have to determine whether an event mentioned in a text is presented as negated, modalised (i.e. affected by an expression of modality), or both. i. Sentence Extractor ii. Event Tag Identifier and Event Generator iii. Modality and Negation Processing iv. Decision Maker Further, Modality and Negation Processing module divides in two-sub modules: (i) Modality Processor and (ii) Negation Processor Database: The modal lists contain the following lists: modal verbs, epistemic adjectives, epistemic adverbs, epistemic nouns, propositional attitude verbs and adjectives, epistemic judgment verbs, epistemic evidential verbs, epistemic deductive verbs.

Explicit negation lists have been also prepared manually to handle explicit negations. Those lists include negative nouns, negative verbs, negative prepositions, negative determiners, negative pronouns, and negative conjunctions. Sentence Extractor Module: The input to this module is single document and output is a list of sentences S = {S1, S2, S3 …Sn-1, Sn}. The objective of this module is to identify each sentence and make list S for next level. Event Tag Identifier and Event Generator: This module takes list of sentences S as input and processes each sentence to extract individual event. This module has the ability to identify event tag. For each sentence in sentence list S individual events have been identified using event tag and a list of events E= {e1, e2, e3…en-1, en} has been generated.

Modality and Negation Processing: This module is the core module of the system. This module has two submodules- modality processor and negation processor.

Modality Processor module is responsible for identifying an event is modalised or not. The manually prepared lists described at database section are applied to the processing event and a pair {event, modality} has been generated for each event. Next, the Negation Processor module uses the negative lists to check whether it appears before the event. If that do not occur then marks it as negative. So, for each event a new pair has been generated by this moduleeventi = {modality, negation}; i.e e1 = {yes, no} , e1€ E ={e1, e2, e3…en-1, en}.

Decision Maker: This modules takes event list E= {e1, e2, e3…en-1, en} and decides one of the four output based on the table is shown in Table 3. Where, NONE: The event is presented as certain and it happened NEG: The event is presented as certain and did not happen MOD: The event is not presented as certain and is not negated

NEGMOD: The event is not presented as certain and is negated

4 Evaluation

We have trained our system by train data (sample data) and tested on test data set. Experiments result is shown in Table 4.

Dataset name eval.JUCSENLP-aids-all-colors-of-the-brainbowr1.txt eval.JUCSENLP-aids-darc-continent-r1.txt eval.JUCSENLP-alz-barking-up-wrong-trip-r1.txt eval.JUCSENLP-alz-have-have-not-r1.txt eval.JUCSENLP-climate-a-record-making-effortr1.txt eval.JUCSENLP-music-can-hiphop-change-theworld-r1.txt eval.JUCSENLP-music-how-to-sink-piratesr1.txt

Overall MOD NONE MOD NONE MOD NONE

MOD NEGMOD

NONE MOD NONE

MOD NEGMOD

NONE MOD NONE MOD NONE

MOD NEGMOD

NONE Acknowledgements. We acknowledge the support of the IFCPAR funded Indo-French project “An Advanced Platform for Question Answering Systems”, partial support of the DST India—CONACYT Mexico project “Answer Validation through Textual Entailment” and the DIT, Government of India funded project “Development of Cross Lingual Information Access (CLIA) System Phase II”.

References

1. Anselmo Peñas , Eduard Hovy, Pamela Forner, Álvaro Rodrigo, Richard Sutcliffe, Caroline Sporleder, Corina Forascu, Yassine Benajiba, Petya Osenova . Overview of QA4MRE at CLEF 2012 : Question Answering for Machine Reading Evaluation . CLEF 2012 Evaluation Labs and Workshop Working Notes Papers, 17 - 20 September, 2012 , Rome, Italy. [2012]

2. Partha Pakray , Pinaki Bhaskar, Santanu Pal, Dipankar Das , Sivaji Bandyopadhyay and Alexander Gelbukh. JU_CSE_TE: System Description QA@CLEF 2010 - ResPubliQA . CLEF 2010 Workshop on Multiple Language Question Answering (MLQA 2010 ). [2010]

3. Partha Pakray , Pinaki Bhaskar, Somnath Banerjee, Bidhan Chandra Pal, Sivaji Bandyopadhyay and Alexander Gelbukh . A Hybrid Question Answering System based on Information Retrieval and Answer Validation . CLEF 2011 Workshop on Question Answering For Machine Reading Evaluation (QA4MRE) . CLEF 2011 Labs and Workshop. Notebook Papers. 19-22 September, Amsterdam. ISBN 978-88-904810-1-7; ISSN 2038 - 4726 . 2011 , 16 pp. [ 2011 ]

4. Pinaki Bhaskar , Partha Pakray, Somnath Banerjee, Samadrita Banerjee, Sivaji Bandyopadhyay and Alexander Gelbukh . Question Answering System for QA4MRE@CLEF 2012 . CLEF 2012 Workshop on Question Answering For Machine Reading Evaluation (QA4MRE) . CLEF 2012 Labs and Workshop . Notebook Papers. 17 - 20 September 2012 , Rome, Italy. [2012]