The objective of NEGES Task 1 is to establish a standard for the annotation of negation in Spanish-language corpora. Speci cally, the task analyzes guidelines used for such annotation in ve projects over three domains: news (Sandoval and Salazar, 2013), clinical reports (Oronoz et al., 2015; Cruz et al., 2017; Marimon, Vivaldi, and Bel, 2017), and product reviews (Jimenez-Zafra et al., 2018b). Here, an assessment of these various guidelines is presented, with the goals of helping establish a standard set of guidelines for annotating negation in Spanish across domains, and of contributing to the workshop's overall conversation about the treatment of negation in computational linguistics.
Negation is fundamental to sentence
meaning, bearing on the key questions of \what
happened (and what did not)", \who did (or
did not do) what to whom," and \what
existed (or did not)." In other words, negation
helps establish what is fact and what is not,
due to its ability to a ect the truth value of
a sentence
The annotation of negation is not a
trivial task. Negation acts at the syntactic,
semantic, and pragmatic levels
(1) Ella no dijo nada. `She did not say anything.'
(2) Nadie le hac a caso. `Nobody paid attention to him/her.'
(3) Ni de una forma ni de otra consiguieron convencerla. `They couldn't convince her one way or another.'
(4) En mi vida he visto cosa igual. `I haven't seen anything similar in my life.' (5) No hables tanto. `Don't talk so much.' (6) >No son ya las dos? `Isn't it already two o'clock?'
Example (1) negates the action of speaking; no functions as a negation cue, and nada as a negative polarity item (NPI) within the scope of the cue in the form of a negative inde nite. (2) a rms the matter that no one, signaled by the inde nite pronoun nadie, participated in an attention-giving activity. (3) exhibits two negative conjunctions, ni, that describe an ine ective manner of convincing. (4) exhibits an adverbial phrase, en mi vida, that equates to the negative temporal adverb nunca `never'. Finally, (5) and (6) utilize negation to support speech acts: (5) as a negative command, and (6) as a leading question.
Given this complexity in mind, the process of annotating negation in Spanish is daunting. Nevertheless, it is essential to annotate corpora with such information in order to train algorithms to perform to human capacity. Ideally, an annotation framework could be designed to capture negation patterns cross-linguistically. However, since such patterns are quite varied and distinct, the task here focuses on annotating negation in Spanish as a rst step towards potentially broader research. 2
NEGES task 1 seeks to reach an agreement
on the guidelines to follow for the
annotation of negation in Spanish building o
previous, domain-speci c guidelines used to
annotate corpora built around news, clinical
reports and product reviews
Given that the UAM Treebank is in xml format, the authors adapt their annotation scheme to be compatible. The authors distinguish two levels of annotation for negation: sentence-level and lexical. The former is further divided into sentential and phrase-level negation; the latter is divided into pronouns and adverbs. Annotations thus mark both negation cues and the scope of negation.
The use of a syntactically annotated corpus is helpful for the overall annotation scheme; this is especially true given the widespread knowledge of the Penn Treebank, as well as the concurrent annotation of lexical features that specify POS. Theoretically, such annotation will help identify the lexical category of negation marker as well as its syntactic scope of negation. Algorithms trained on such data will then be able to recognize patterns of use of negation for varying parts of speech and compare how syntactic scope relates to semantic and pragmatic scope.
As the authors themselves note, there is
much room for improvement in the
annotation scheme presented. First, the type of
negation is not speci ed apart from <YES>
and <NO> (to mark scope), and <NEG>
(to mark negation cues). This fails to
capture gradations of negation (i.e. whether it
is an assertion or a speech act that is
negated
The authors collected 142,154 anonymous discharge reports from the outpatient consultations of the Galdake-Usansolo Hospital from 2008 to 2012. Negation (and speculation) was annotated as a modi er of a disorder or drug, and individual cues were left unmarked. As such, the text span sin otras alergias medicamentosas \without other drug allergies" would possess negation on alergias medicamentosas while sin would be left bare. This practice was used to maintain consistency in the domain: a disease-entity such as afebril \afebrile" was also marked as negative in the absence of surrounding negative lexical material.
Four entity types were annotated: diseases, allergies, drugs, and procedures. For diseases and allergies, a distinction was made between negated entity, speculated entity and entity (for non-speculative and non-negated entities). 2,362 diseases were annotated, out of which 490 (20.75 %) were tagged as negated diseases and 40 (1.69 %) as speculated diseases. 404 allergy entities were identi ed, of which 273 (67.57 %) were negated and 13 (3.22 %), speculated. The quality of the annotation process was assessed by measuring the inter-annotator agreement (IAA), which was 90.53 % for entities and 82.86 % for events.
This annotation scheme needs to be adapted in order to extend beyond the clinical domain. First and foremost, negation needs to be treated linguistically and broken down into its components apart from the disorders and drugs it acts on. Nevertheless, the recognition that entities and events may be marked for negation in distinct ways (i.e. syntactically versus morphologically, as in the above examples), as well as the intuition that some entities and events may possess qualities of negation without being explicitly marked for it, is an important contribution to developing a comprehensive annotation scheme. Additionally, the distinction between negation and speculation is important to consider, as the linguistic interaction between negation and modality is complicated, yet merits attention. 3.3
Jimenez-Zafra et al. (2018b) present the SFU ReviewSP-NEG corpus, the rst Spanish corpus that includes event negation as part of the annotation scheme as well as the annotation of discontinuous negation markers. The corpus was also the rst to have de ned a typology of patterns involving negation speci c to Spanish. In this SFU ReviewSP-NEG corpus, syntactic negation, scope, focus, and event were annotated. Yet, annotations on the event and on how negation a ects the polarity of the words within its scope were included for whether there is a complete change in the polarity of the span in question, or an increment or reduction of its value.
The Spanish SFU Review corpus,
originally intended for work on sentiment
analysis, consists of 400 reviews extracted from
the website ciao.es. The reviews span 8 di
erent produce areas: cars, hotels, washing
machines, books, cell phones, music, computers,
and movies. For each product area there are
50 positive and 50 negative reviews, which
provides an informative context for how to
interpret the e ects of negation at the discourse
level
For the SFU ReviewSP-NEG corpus, each review was automatically annotated at the token level with POS-tags and lemmas; negation cues and their corresponding scopes and events were manually annotated at the sentence level. The annotations were performed by two senior researchers with in-depth experience in corpus annotation who supervised the two trained annotators who carried out the annotation task. The nal corpus is composed of 9,455 sentences, out of which 3,022 sentences (31.97 %) contain at least one negation marker. The Kappa coe cient for IAA was of 0.97 for negation cues, 0.95 for negated events and 0.94 for scopes.
Similar to the UAM Spanish Treebank,
the annotation scheme for SFU
ReviewSPNEG ought to be broadened to include
morphological, lexical, and discourse-oriented
negation. Nevertheless, the inclusion of
gradient interpretations of negation within its
scope captures the subtle meaning di
erences that negation markers, and their
combinations, may produce. In fact, this gradation
of meaning may be expanded to include even
ner-grained distinctions of meaning in
future work.
3.4
In this corpus, all types of negation
were annotated: syntactic, morphological (a
xal negation), and lexical. Negation was
marked both linguistically and as a modi er of a
disorder of a drug, i.e. whether or not the
drug was e ective. Similar to
Either due to the domain of application or presentation format, the annotation guidelines presented for the UHU-HUVR corpus often seemed unclear. For example, non-clinical experts may have trouble di erentiated negative test results from negative clinical events, as the annotation scheme does. Additionally, while some negation a xes are marked as negating symptoms (a-febril \afebrile") others are not, considered positive symptoms unto themselves (in-continencia urinaria \urinary incontinence"). Finally, the authors' treatment of coordination as a single unit of negation ought to be revised. 3.5
The IULA Spanish Clinical Record corpus
Annotators did not include the negation
cue nor the subject in its scope as part of
annotation, unless the subject was located
after the verb. This practice seems to be based
on linear order alone, and does not take into
account the semantics of scope nor the
possibility of backwards scope
Similar to
With any linguistic annotation task, striking a balance between linguistic precision and annotation feasibility is an inevitable and essential question. For the annotation of negation in Spanish, several components of the proposals discussed above may be combined into a set of complex guidelines that is both linguistically accurate and domain neutral. Here I summarize the main components I nd to be worth annotating.
Most basically, the semantics of negation
is represented (and ought to be annotated)
through (i) the identi cation of the negation
cue (the lexical element expressing
negation); (ii) its scope (the text section that
is negated); (iii) its focus (that part of the
scope that is prominently or explicitly
negated); and, if present, (iv) its reinforcement
(an auxiliary negation or NPI)
The negation cue (no) is represented in bold; the scope (come carne sino verduras ) is in italics; the focus (carner) is in brackets; and the NPI (sino) in bold and italics. Negation markers that do not carry negative polarity semantic information (nada mas \as soon as") can be marked as such (for example, as <noneg > instead of <neg > (JimenezZafra et al., 2018b).
Following
This could be accomplished with both the
annotation of the cues themselves and a
linking to some sort of lexical de nition or
modal e ect of the cue, as some combination
of
As a closing point, the Brat annotation
tool
This paper has presented an analysis and evaluation of existing guidelines for the annotation of negation in several domain-speci c Spanish corpora. A preliminary proposal is given for how to combine linguistically accurate and precise annotation with more practical concerns regarding domain of application and ease of annotation. Future work points in particular towards re ning the subtle meaning e ects negation can have on words and phrase meaning, as well as its interaction with modality for ultimate interpretation of event factuality.
Many thanks to Elena Herburger, one of my doctoral advisors and an expert on negation. Thanks as well to Claire Bonial and the team at Army Research Lab (ARL) for supporting the work behind this paper.