=Paper=
{{Paper
|id=None
|storemode=property
|title=Linguistic scope-based and biological event-based speculation and negation annotations in the Genia Event and BioScope corpora
|pdfUrl=https://ceur-ws.org/Vol-714/Paper10_Vincze.pdf
|volume=Vol-714
|dblpUrl=https://dblp.org/rec/conf/smbm/VinczeSMOF10
}}
==Linguistic scope-based and biological event-based speculation and negation annotations in the Genia Event and BioScope corpora==
https://ceur-ws.org/Vol-714/Paper10_Vincze.pdf
Linguistic scope-based and biological event-based specula-
tion and negation annotations in the Genia Event and Bio-
Scope corpora
Veronika Vincze∗1 , György Szarvas2 , György Móra1 , Tomoko Ohta3 and Richárd Farkas∗1,4
1 University of Szeged, Department of Informatics, Szeged, Hungary
2 Technische Unversität Darmstadt, UKP Lab, Darmstadt, Germany
3 University of Tokyo, Tsujii Laboratory, Tokyo, Japan
4 Hungarian Academy of Sciences, Research Group on Artificial Intelligence, Szeged, Hungary
Email: vinczev@inf.u-szeged.hu; szarvas@tk.informatik.tu-darmstadt.de; gymora@inf.u-szeged.hu; okap@is.s.u-tokyo.ac.jp;
rfarkas@inf.u-szeged.hu;
∗ Corresponding author
Abstract
Background: The treatment of negation and hedging in natural language processing has received much interest
recently, especially in the biomedical domain. However, open access corpora annotated for negation and/or
speculation are hardly available for training and testing applications, and even if they are, they sometimes follow
different design principles. In this paper, the annotation principles of the two largest corpora containing annotation
for negation and speculation – BioScope and Genia Event – are compared. BioScope marks linguistic cues and
their scopes for negation and speculation while in Genia biological events are marked for uncertainty and/or
negation.
Results: Differences among the annotations of the two corpora are thematically categorized and the frequency of
each category is estimated. We found that the largest amount of differences is due to the issue that scopes –
which cover text spans – deal with the key events and each argument (including events within events) of these
events is under the scope as well. In contrast, Genia deals with the modality of events within events independently.
Conclusions: We think that the useful information for the biologist can be acquired from the key events, thus if we
aim to detect ”new knowledge”, an automatic scope-detector trained on BioScope can contribute to biomedical
information extraction. However, for detecting the negation and speculation status of events (within events)
syntax-based rules investigating the dependency path between the modality cue and the event cue may be em-
ployed.
Background plications seek to extract factual information from
text. In order to distinguish assertions from un-
In natural language processing (NLP) – and in par- reliable/uncertain information and negated state-
ticular, in information extraction (IE) – many ap-
81
ments, linguistic devices of negation or hedges have lations are annotated for negation (1100 sen-
to be identified. Applications should handle detected tences in size).
modified parts in a different manner. A typical ex-
ample is protein-protein interaction extraction from • The BioScope corpus [8], which includes three
biological texts, where the aim is to mine text evi- types of texts from the biomedical domain –
dence for biological entities that are in a particular namely, radiological reports, biological full pa-
relation with each other. Here, while an uncertain pers and abstracts from the Genia corpus – an-
relation might be of some interest for an end-user notated for both negation and hedge keywords
as well, such information must not be confused with and their linguistic scopes (20924 sentences).
factual textual evidence (reliable information).
• The system developed by Medlock & Briscoe
There are several available negation and hedge
[6] made use of a corpus consisting of six papers
detection systems (usually for the clinical and bio-
from genomics literature in which 1537 sen-
logical domains). The first systems were fully hand-
tences were annotated for speculation. These
crafted [1–3] without any empirical evaluation on a
texts – with re-annotation – are also included
dedicated corpus. Recently, there have been sev-
in BioScope.
eral corpora published with manual annotation and
several rule-based systems have been developed and • Shatkay et al. [16] describe a database where
evaluated on them [4, 5]. 10000 biomedical sentences are annotated for
Recent approaches exploit machine learning polarity and three levels of certainty.
models. Medlock & Briscoe [6] used single words
as input features in order to classify sentences In the corpora Genia Event and BioInfer, biolog-
from biological articles (FlyBase) as speculative or ical concepts (relations and events) have been anno-
non-speculative based on semi-automatically col- tated for negation and – in the case of Genia Event
lected training examples. Szarvas [7] extended their – for hedging as well, but linguistic cues (i.e. which
methodology to use n-gram features and a semi- keyword modifies the semantics of the statement)
supervised selection of the keyword features. Using have not been annotated for them. In the last two
BioScope [8] for training and evaluation, Morante corpora, speculative annotation can be found on the
et al. [9] developed in-sentence scope detectors for sentence level.
negation and speculation following a supervised se- In contrast to those, BioScope was not fine-tuned
quence labeling approach, while Özgür and Radev for information extraction tasks but it contains lin-
[10] constructed a rule-based system that exploits guistic annotation for hedge and negative cues and
syntactic patterns. BioScope is also the source their in-sentence scope as well. Its chief objective
of training and evaluation datasets of the CoNLL- is to investigate these language phenomena in a
2010 Shared Task [11]. Several related works have general, task-independent and linguistically-oriented
also been published within the framework of The way. Automatically recognized in-sentence scopes
BioNLP’09 Shared Task on Event Extraction [12], (i.e. the negated or hedged text spans) are impor-
where a separate subtask was dedicated to predict- tant for many natural language processing applica-
ing whether the recognized biological events are un- tions. For instance
der negation or speculation [4].
In this paper we focus on corpora annotated for • in clinical document classification tasks [17,
negation and speculation. There are several avail- 18], the goal is to assign labels to medical doc-
able corpora outside the biomedical domain (e.g. uments according to factual statements about
FactBank [13], Wikipedia weasels [11]) as well. How- the patient in question. Here the removal (or
ever, we deal here with biological information extrac- separate handling) of hedged or negated text
tion and to our best knowledge, the following related spans has a great contribution in the training
corpora have been constructed for this domain: and prediction phases as well.
• The Genia Event corpus [14] which annotates • In information retrieval the query mentions
biological events with negation and two types under hedging can be ranked lower,
of uncertainty (9372 sentences).
• in machine translation the extension of nega-
• The BioInfer corpus [15] where biological re- tion or speculation scopes has to be precisely
82
known in order to translate meaning ade- and 50 sentences where at least one of the arguments
quately. of an event was under a BioScope negation scope and
marked as existing by Genia (50+50 sentences were
Although the BioScope corpus consists of clini- selected for speculation analogously). By manual in-
cal and biological documents, its annotation guide- spection of this sample we thematically categorized
lines do not contain any domain-specific instruction. these differences.
Councill et al. [19] employed BioScope as training
corpus for detecting negated scopes for opinion min-
ing from product reviews, which proves its task- and
Results
domain-independence.
Annotation principles
In the following sections, the hedge and nega-
tion annotation principles of BioScope and Genia BioScope annotation
Event are compared, resolution strategies for the dif- When annotating keywords and their scopes in the
ferences are offered and we discuss how BioScope BioScope corpus [8], the corpus builders followed a
can contribute to identifying ”new knowledge” in min-max strategy. When marking the keywords, a
biomedical papers. minimalist strategy was followed: the minimal unit
that expresses hedging or negation is marked as a
keyword. Special attention is paid to the case of
complex keywords, that is, words that express un-
Methods certainty or negation together, but not on their own
In this paper we quantitatively compare the nega- (either the semantic interpretation or the hedging
tion and speculation annotations of the BioScope strength of its subcomponents are significantly dif-
and Genia Event corpora. We investigated sentences ferent from those of the whole phrase).
that occur in both corpora, i.e. the intersection of The scopes of negative and speculative keywords
the two corpora containing 958 abstracts and 8942 are extended to the largest syntactic unit possible.
sentences (abstracts that were not segmented in the Thus, annotated scopes always have the maximal
same way on the sentence level in the two corpora length. In the next example, however is not affected
were neglected) was used. This corpus contains 1287 by the hedge cue but it should be included within the
negation and 1980 speculation BioScope scopes scope, otherwise the keyword and its target phrase
(376 nested scopes) while 2123 non-exist and 1475 would be separated (scopes are marked by brackets
probable Genia events (200 events have both la- and keywords are bold):
bels).
As for negation, events with at least one clue [Atelectasis in the right mid zone is, how-
occurring within a negative scope in BioScope and ever, possible].
being annotated as non-exist in Genia Event were
considered as cases of agreement. As regards to That is why the corpus builders preferred to include
speculation, events with at least one clue within a every possible element within the scope rather than
speculative scope in BioScope and being marked as exclude elements that should probably be included.
probable in Genia Event were accepted as cases of As for annotating, the most important thing to con-
agreement. Mismatches included events with differ- sider is that hedging or negation is determined not
ent labels in the two corpora (e.g. an event labeled just by the presence of an apparent cue: it is rather
as negative in Genia Event and speculative in Bio- an issue of the keyword, the context and the syntac-
Scope) on the one hand, and events annotated only tic structure of the sentence taken together.
in one of the corpora on the other hand. The scope of a keyword can be determined on the
In order to understand the differences between basis of constituency grammar. The scope of verbs,
the annotation principles and to investigate the pos- auxiliaries, adjectives and adverbs usually extends
sible contribution of the BioScope annotation to Ge- to the right of the keyword. In the case of verbal el-
nia event modality detectors, we randomly sampled ements, i.e. verbs and auxiliaries, it ends at the end
200 sentences from the intersection of the two cor- of the clause (if the verbal element is within a rela-
pora. This sampling consists of 50 sentences where tive clause or a coordinated clause) or the sentence,
events are marked to be negated by Genia and none hence all complements and adjuncts are included,
of its arguments was included in a negation scope in accordance with the principle of maximal scope
83
size. In the case of elliptic sentences, the scope of In the corpus, no explicit marking of either the
the negative keyword may be deleted as in: keywords or the scope of negation and hedging can
be found.
This decrease was seen in patients who
responded to the therapy as well as in
those who did [not]. Number of disagreements
Table 1a shows the number of cases of agreement
In these cases, the scope contains only the keyword.
and disagreement between the two corpora (agree-
ment rate: 48%). The numbers in column TP (true
positive) denote instances which are considered in
Genia Event modality annotation
the same way in both corpora. The numbers in col-
The Genia Event corpus was primarily designed for
umn BPGN refer to cases where in BioScope any
(biological) event annotation [14] and the database
clue of a Genia event is under a negative / specu-
contains annotation for uncertainty and negation
lative scope, however, in Genia Event, it is not. As
on the level of events. The annotation scheme fo-
opposed to this, in column GPBN, the numbers show
cuses on events, and arguments of events can occa-
cases where Genia contains some speculative / neg-
sionally be found across clause boundaries, typically
ative annotation for any argument of the event but
due to anaphora or coreference (out of 35419 Ge-
BioScope does not.
nia events used in our experiment, 1127 referred to
an external event and 2076 clues are arguments of
an event expressed in another sentence (mostly clue-
Categorization of differences
types theme (1447 instances, 70%) and cause (619
In this section, mismatches in annotation between
instances, 29.8%)).
the Genia Event and the BioScope corpora are pre-
As for uncertainty, events can have three labels
sented. Systematic differences are categorized on the
in the corpus: certain, probable and doubtful.
basis of a possible solution aiming to resolve the
Events are marked as doubtful if they are under in-
mismatch, and subtypes of these categories are il-
vestigation or they form part of a hypothesis, etc.
lustrated with examples along with their estimated
An example (event arguments are underlined in our
frequencies based on a random sample of 200 anno-
examples) for a doubtful event is provided here:
tation differences (see Table 1b).
We then investigated if HCMV binding Event-centered vs. linguistic annotation An essential
also resulted in the translation and se- difference in annotation principles between the two
cretion of cytokines. corpora is that Genia Event follows the principles of
Event-centered annotation [14] while BioScope an-
Events are considered probable if their existence can- notation does not put special emphasis on events
not be stated for certain. An example of a probable as it aims a task-independent modeling of specula-
event is shown here: tion and negation. Event-centered annotation means
that annotators are required to identify as many bi-
Together, this evidence strongly impli-
ological events as possible within the sentence then
cates BSAP in the regulation of the
label each separately for negation and speculation.
CD19 gene.
Events are usually expressed by verbs, however, (de-
The attribute certain is chosen by default if none of verbal) adjectives and nouns can also refer to events.
the two others hold: an event the existence of which Consider the following example:
cannot be questioned in any way. Calcineurin acts in synergy with PMA to
As for negation, events are marked with the la- inactivate I kappa B/MAD3, an inhibitor
bels exist or non-exist. An example for a negated of NF-kappa B.
event is shown below:
This sentence describes two events, the inactivation
Analysis of Tax mutants showed that of I kappa B/MAD3 by Calcineurin and the inhibi-
two mutants, IEXC29S and IEXL320G, tion of NF-kappa B by I kappa B/MAD3.
were unable to significantly transactivate From a linguistic point of view, an event is un-
the c-sis/PDGF-B promoter. derstood as a predicate together with its arguments
84
BPGN GPBN
event within event 68% –
TP BPGN GPBN syntax 7% 3%
negation 1554 1484 569 lexical semantics 20% 72%
probable 1295 3761 180 morphological negation – 14%
annotation error 5% 11%
TOTAL 100% 100%
Table 1: Numbers of agreement and disagreement between BioScope and Genia Event and the frequency of
mismatch categories.
and the role of the predicate can be fulfilled by a Syntactic issues Some of the mismatches in annota-
verb, a noun, or an adjective in the text. In contrast tion can be traced back to syntax. For instance, the
to this, BioScope is not event-oriented in the above treatment of subjects remains problematic since in
sense. Instead, verbs play a central role, i.e. a verb BioScope it is only the complements that are usually
and its arguments form one event in BioScope as included within the scope of a keyword (that is, sub-
well. Accordingly, the above sentence refers to one jects are not with the exception of passive construc-
event in BioScope and inhibitor is not considered as tions and raising verbs) in contrast to Genia where
a predicate. events are argument-centered (i.e. complements and
As a consequence, there are much more events in subject are considered) as in:
Genia than in BioScope. The multiplicity of events
Both c-Rel and RelA induced
in Genia Event and the maximum scope principle ex-
jagged1 gene expression, whereas
ploited in BioScope taken together often yields that
a mutant defective for transactivation
a Genia event falls within the scope of a BioScope
did [not].
keyword, however, it should not be seen as a specu-
lated or negated event on its own. Here we provide In this example, no argument of the event denoted
an illustrative example: by induced is under the BioScope scope, which yields
a case of disagreement.
In summary, our data [suggest With regards to the problem concerning the
that changes in the composition of treatment of subjects, the dependency parse of the
transcription factor AP-1 is a key sentence/clause might help the correct identification
molecular mechanism for increasing of the modality of the events. We can apply the
IL-2 transcription and may underlie the following rule: if a verb that functions as the trig-
phenomenon of costimulation by EC]. ger word for an event is negated or hedged, all its
children in the dependency tree (including the sub-
According to the BioScope analysis of the sentence, ject as well) are to be included in the scope of the
the scope of suggest extends to the end of the sen- modifier. In this way, instances of mismatch when
tence. It entails that in Genia it is only the events is it is only the subject that is within the scope of the
a key molecular mechanism and underlie the phe- modifier (e.g. in the case of elliptic sentences) can
nomenon that are marked as probable, neverthe- be eliminated from the GPBN set.
less, the events changes, increasing, transcription Semantic issues There are some cases where the dif-
and costimulation are also included in the BioScope ference in annotations originates from conceptual
speculative scope. Thus, within this sentence, there discrepancies. These differences can hardly be re-
are six Genia events out of which two are labeled solved without harmonizing the annotation princi-
as probable, however, in BioScope, all six are within ples behind the corpora and re-annotating the data,
a speculative scope, resulting in two cases of agree- however, the most typical cases are presented here.
ment and four cases of disagreement. Concerning Events labeled as doubtful in Genia Event are
the whole corpora, the large number of BPGN cases rarely annotated as speculative in BioScope. In Ge-
(see Tables 1a and 1b) can be explained in a similar nia Event, the investigation, examination, study, etc.
way. of a phenomenon does not necessarily mean that the
85
phenomenon exists. However, in BioScope this as- why it is marked as a negative keyword in Bio-
pect is neglected and phenomena being under inves- Scope. However, in Genia, ’lack of something’ is
tigation, examination, etc. are only marked as in- understood as negation of status, not negation of
stances of speculation if they are within the scope of an event. Hence here the class type of the event
a speculative keyword (e.g. whether ). As only 17% is negative regulation but the event itself is as-
of doubtful Genia event clues is under speculation sertive (out of 4347 negative regulations in Genia
scope, we focus just on the probable class during 4164 are assertive, some of which are annotated as
our comparison. negative in BioScope due to semantically negative
There are some examples of mismatch where a keywords).
generalization or a widely accepted claim is stated. Another case of conceptual discrepancy is mor-
Grammatically, these sentences usually occur in the phological negation, i.e. on the morphological level,
passive voice without explicitly marking the agent the clueword contains a negative prefix such as in-
(i.e. the one whom the claim originates from). Such or un-. Here is a typical example:
sentences are instances of weaseling [20], and are an-
notated as probable events in Genia, however, in In monocytic cells, IL-1beta treatment
BioScope they are not as they express a different led to a production of ROIs which is
type of uncertainty: it is the exact source of the independent of the 5-LOX enzyme but
opinion that is missing rather than the factuality of requires the NADPH oxidase activity.
the event (it is known that some hold this opinion
but it is unknown who they are). It is a kind of un- The event denoted by led is not triggered by the
certainty expressed at the discourse level as opposed presence of the 5-LOX enzyme, thus, there is no
to uncertainty on the semantic level. An example regulation event here and this is expressed in Genia
for a weasel sentence is shown below: by marking the regulation event with the attribute
non-exist while in BioScope its meaning is consid-
Receptors for leukocyte chemoattrac- ered to be lexicalized and not necessarily negative.
tants, including chemokines, are tradi- Mismatches originating from morphological
tionally considered to be responsible for negation mostly include the adjective independent.
the activation of special leukocyte func- We argue that although this word contains a nega-
tions such as chemotaxis, degranulation, tive prefix at the level of morphology, its meaning
and the release of superoxide anions. is lexicalized and not necessarily negative: it rather
describes a state or a lack of relation between its
Weasel sentences and cue phrases can be automat- arguments. In this way, it could be treated simi-
ically detected by employing machine learnt mod- larly to lack, that is, not the event itself but its state
els. For instance, the CoNLL-2010 Shared Task should be negated. On the other hand, cluewords
dataset [11] includes a corpus dedicated to weasel including morphological negation can be easily iden-
detection in Wikipedia articles. We suppose that the tified by automatic methods (segmenting the word
phenomenon of weasel is domain-independent hence into a negative prefix and an existing (adjectival)
the model trained on Wikipedia could be adequately morpheme) and these can be automatically tagged
applied for (biological) scientific publications as well. as negative cues.
Sometimes an event is marked as negation in Bio- The interpretation of some speculative keywords
Scope but not in Genia: too seems to vary in BioScope and Genia Event. The
most striking example is the case of events modified
[Lack of full activation of NF-AT] could by other words or phrases expressing ability (e.g. be
be correlated to a dramatically reduced able to, ability etc.), which are annotated for proba-
capacity to induce calcium flux and bility in Genia but not in BioScope. An example is
could be complemented with a calcium offered here:
ionophore.
NF-kappa B activation correlated with
As lack is understood as ’the state of not hav- the ability of CD40 to induce Ab
ing something’, it denotes negation, i.e. the non- secretion and the up-regulation of
existence of the following NP complement, that is ICAM-1 and LFA-1.
86
A highly interesting subclass of words expressing Similar considerations implied the design of
ability is when the derivational suffix conveys the the ”Meta-Knowledge Annotation Scheme for Bio-
’ability’ meaning as in inducible or inhibitable. Take Events” [21]. It introduces dedicated labeling di-
the following sentence: mensions of events about
Despite stimulation with LPS, disrup- • New Knowledge (yes/no), the motivation of
tion of the NF-kappaB signaling path- which is that events ”. . . could correspond to
way in precursor B cells led to the loss new knowledge, but only if they represent ob-
of inducible Oct-2 DNA binding activ- servations from the current study, rather than
ity in vitro and the suppression of Oct- observations cited from elsewhere. In a simi-
2-directed transcription in vivo. lar way, an analysis drawn from experimental
results in the current study could be treated
as new knowledge, but generally only if it
The event described by inducible can be paraphrased
represents a straightforward interpretation of
as Oct-2 DNA binding activity can be induced in
results, rather than something more specula-
vitro, which is an ’ability’ usage of the auxiliary can,
tive.”
thus, it is annotated for probability in Genia but not
in BioScope. • Knowledge type (investigation / observation
The lexical semantic-related differences originate / analysis / general) whose ”. . . purpose is to
from conceptual discrepancies of the two corpora. form the basis of distinguishing between the
These mismatches can hardly be resolved without most critical types of rhetorical/pragmatic in-
harmonizing the annotation principles behind the tent, according to the needs of biologists.”
corpora and re-annotating the data. As one of the
chief design goals of BioScope annotation was to Krallinger [22] also argues that from a biologist
be task-independent and the modality annotation of point of view only the events supported by experi-
Genia is fine-tuned to biological event extraction, bi- mental evidence are interesting.
ological information extractors may incorporate the In conclusion, as the BioScope corpus is designed
modality principles of Genia while BioScope annota- to be task-independent its scopes could not be ap-
tions may be followed when the target domain differs plied directly for the deep and detailed (sub)event
from the biomedical one. annotation of Genia, however, it can recognize the
Lastly we note that few differences (about 5.7%) negation and hedge state of chief statements (new
in annotation can be obviously traced back to anno- knowledge). Note that there are in-sentence scope
tation errors. detectors published (which achieve 58% strict F-
measure) [11] for this task.
Discussion BioScope for event modality detection
Detailed event annotations We discussed in the previous section that the scopes
Table 1a and 1b reveal that the biggest subset of of BioScope are not useful directly to the detec-
the differences (60%) came from the issue that Genia tion of assertion and certainty state of Genia events,
handles events within events as individual informa- however, we believe that using cue phrases in event
tion sources while BioScope deals with constituent- modality detection can yield significant contribution.
based text spans. An interesting question for consid- For instance, Kilicoglu and Bergler [4] constructed
eration is whether the expected output of an infor- lexicons for speculation and negation keywords and
mation extraction system consists of facts solely on introduced rules for recognizing the modality state of
the basis of this textual evidence, where the trigger an event by utilizing the dependency path between
for the event does not belong to the main statement the event clue phrase and the speculation/negation
of the sentence/document. Note that the informa- cue.
tion content of these events within events is usually Kilicoglu and Bergler employed hand-crafted lex-
introduced and discussed in detail in other parts of icons for cue recognition, however, keywords are am-
the document or in other publications or belongs to biguous, i.e. they express speculation and nega-
the trivial domain knowledge. tion just in certain contexts. Hence a cue phrase
87
detection system is needed which classifies tokens mains is concerned, the annotation system in Bio-
based on their local context then the dependency Scope seems to be more easily adaptable to non-
paths between these predicted speculation/negation biomedical applications because of the high level of
evidences and event triggers should be analyzed. domain specificity in the Genia Event annotation
The BioScope corpus can be employed as a training system.
dataset for general speculation/negation cue clas- As regards to the frequency of mismatch cate-
sifiers. The state-of-the-art modifier cue detectors gories, we found that the largest amount of differ-
achieve strict phrase-level F-measures over 80% [11]. ences is due to the issue that scopes aim to iden-
Dependency-based rules defined for each (sub)type tify the negation/certainty status of the key event in
of keywords can be also added to the system in order the sentence and each argument of these key events
to determine the negative/speculative status of the (including arguments that are events themselves) is
event. As future work, we plan to develop an event under scope as well in BioScope. In contrast, Ge-
modality detector which uses BioScope as a training nia deals with the modality of events within events
database for identifying speculation/negation cues independently. We think that the useful informa-
and is enhanced by hand-crafted dependency-based tion for the biologist can be acquired from the key
rules for determining the modality of the event. events, thus if we aim to detect ”new knowledge”,
an automatic scope-detector trained on BioScope
can contribute to biomedical information extraction.
The usability of different annotation schemes On the other hand, BioScope cue phrases could be
also employed to identify the assertion and certainty
As discussed earlier, the annotation scheme of Bio-
status of events. To reach this goal, we plan to
Scope relies on linguistic principles while Genia
develop a procedure which makes use of automat-
Event is based on a more detailed annotation system
ically recognized negation/speculation cues and em-
specifically tailored to biological event annotation,
ploys syntax-based rules (investigating the depen-
where several complex relations are encoded be-
dency path between the modality cue and the event
tween participants of the events – often across clause
cue) to classify the status of the event.
boundaries. In this way, the annotation scheme of
Genia Event is highly domain-specific and the corpus
can be fruitfully utilized in biomedical information
extraction, resulting in a deep and precise analysis of Authors contributions
biological events though it might require a lot of ad- Veronika Vincze and Tomoko Ohta categorized the
ditional work to adapt the system to other domains. mismatches between the two corpora. György Móra
On the other hand, as the BioScope annotation implemented the software tools for gathering and vi-
scheme is linguistic-based, scope- and cue-marking sualizing the differences. Richárd Farkas and György
rules extracted from the corpus data can be more Szarvas carried out the statistical analysis of mis-
easily exploited when developing negation/hedge de- matches.
tectors in other domains as well.
Acknowledgements
This work was supported in part by the NKTH
Conclusions
grants (project codenames MASZEKER and TEX-
In this paper, we discussed the differences between TREND) of the Hungarian government. The au-
the linguistic-based and event-oriented annotation of thors would like to thank the annotators of the two
negation and speculation in biological documents. corpora for their devoted efforts.
We defined categories for the differences between
the linguistic scope-based BioScope and the event-
oriented Genia Event corpora. They have an in-
tersection of documents (biological abstracts) which
was randomly sampled, frequencies of mismatch cat-
egories were estimated and resolution strategies were
also offered for them.
As far as information extraction in different do-
88
References Volume for Shared Task 2009:1–9, [http://www.aclweb.
1. Light M, Qiu XY, Srinivasan P: The language of Bio- org/anthology/W09-1401].
science: Facts, speculations, and statements in
13. Saurı́ R, Pustejovsky J: FactBank: a corpus an-
between. In Proc. of Biolink 2004, Linking Biologi-
notated with event factuality. Language Resources
cal Literature, Ontologies and Databases (HLT-NAACL
and Evaluation 2009, 43:227–268, [http://dx.doi.org/10.
Workshop:) 2004:17–24.
1007/s10579-009-9089-9]. [10.1007/s10579-009-9089-9].
2. Friedman C, Alderson PO, Austin JHM, Cimino JJ,
Johnson SB: A General Natural-language Text Pro- 14. Kim JD, Ohta T, Tsujii J: Corpus annotation for
cessor for Clinical Radiology. Journal of the Ameri- mining biomedical events from literature. BMC
can Medical Informatics Association 1994, 1(2):161–174, Bioinformatics 2008, 9:10, [http://www.biomedcentral.
[http://jamia.bmj.com/content/1/2/161.abstract]. com/1471-2105/9/10].
3. Chapman WW, Chu D, Dowling JN: ConText: an al- 15. Pyysalo S, Ginter F, Heimonen J, Björne J, Boberg J,
gorithm for identifying contextual features from Järvinen J, Salakoski T: BioInfer: a corpus for infor-
clinical text. In Proceedings of the ACL Workshop on mation extraction in the biomedical domain. BMC
BioNLP 2007 2007:81–88. Bioinformatics 2007, 8.
4. Kilicoglu H, Bergler S: Syntactic Dependency Based 16. Shatkay H, Pan F, Rzhetsky A, Wilbur WJ: Multi-
Heuristics for Biological Event Extraction. In Pro- dimensional classification of biomedical text: To-
ceedings of the BioNLP Workshop Companion Volume ward automated, practical provision of high-
for Shared Task 2009:119–127, [http://www.aclweb.org/ utility text to diverse users. Bioinformatics 2008,
anthology/W09-1418]. 24(18):2086–2093, [http://bioinformatics.oxfordjournals.
org/cgi/content/abstract/24/18/2086].
5. Aramaki E, Miura Y, Tonoike M, Ohkuma T, Mashuichi
H, Ohe K: TEXT2TABLE: Medical Text Summa- 17. Pestian JP, Brew C, Matykiewicz P, Hovermale D, John-
rization System Based on Named Entity Recogni- son N, Cohen KB, Duch W: A shared task in-
tion and Modality Identification. In Proceedings of volving multi-label classification of clinical free
the BioNLP 2009 Workshop 2009:185–192, [http://www. text. In Proceedings of the ACL Workshop on BioNLP
aclweb.org/anthology/W09-1324]. 2007 2007:97–104, [http://www.aclweb.org/anthology/
6. Medlock B, Briscoe T: Weakly Supervised Learn- W/W07/W07-1013].
ing for Hedge Classification in Scientific Liter- 18. Uzuner O: Recognizing obesity and comor-
ature. In Proceedings of the ACL 2007:992–999, [http: bidities in sparse data. Journal of Amer-
//www.aclweb.org/anthology/P/P07/P07-1125]. ican Medical Informatics Association 2009,
7. Szarvas Gy: Hedge Classification in Biomedical 16(4):561–70, [http://www.biomedsearch.com/nih/
Texts with a Weakly Supervised Selection of Key- Recognizing-obesity-comorbidities-in-sparse/19390096.
words. In Proceedings of ACL-08 2008:281–289, [http: html].
//www.aclweb.org/anthology/P/P08/P08-1033]. 19. Councill I, McDonald R, Velikovich L: What’s great
8. Vincze V, Szarvas Gy, Farkas R, Móra Gy, Csirik J: and what’s not: learning to classify the scope
The BioScope Corpus: Biomedical Texts Anno- of negation for improved sentiment analysis. In
tated for Uncertainty, Negation and their Scopes. Proceedings of the Workshop on Negation and Specu-
BMC Bioinformatics 2008, 9(Suppl 11):S9, [http://www. lation in Natural Language Processing, Uppsala, Swe-
biomedcentral.com/1471-2105/9/S11/S9]. den 2010:51–59, [http://www.aclweb.org/anthology/W/
W10/W10-3110].
9. Morante R, van Asch V, van den Bosch A: Joint
memory-based learning of syntactic and semantic 20. Ganter V, Strube M: Finding Hedges by Chas-
dependencies in multiple languages. In Proceedings ing Weasels: Hedge Detection Using Wikipedia
of CoNLL 2009:25–30. Tags and Shallow Linguistic Features. In Proceed-
10. Özgür A, Radev DR: Detecting Speculations and ings of the ACL-IJCNLP 2009 Conference Short Pa-
their Scopes in Scientific Text. In Proceedings of the pers 2009:173–176, [http://www.aclweb.org/anthology/
2009 Conference on Empirical Methods in Natural Lan- P/P09/P09-2044].
guage Processing 2009:1398–1407, [http://www.aclweb. 21. Nawaz R, Thompson P, Ananiadou S: Evaluating
org/anthology/D/D09/D09-1145]. a meta-knowledge annotation scheme for bio-
11. Farkas R, Vincze V, Móra Gy, Csirik J, Szarvas Gy: events. In Proceedings of the Workshop on Negation and
The CoNLL-2010 Shared Task: Learning to De- Speculation in Natural Language Processing, Uppsala,
tect Hedges and their Scope in Natural Lan- Sweden 2010:69–77, [http://www.aclweb.org/anthology/
guage Text. In Proceedings of the Fourteenth Con- W/W10/W10-3112].
ference on Computational Natural Language Learning 22. Krallinger M: Importance of negations and ex-
(CoNLL-2010): Shared Task 2010:1–12, [http://www. perimental qualifiers in biomedical literature. In
aclweb.org/anthology/W/W05/W05-0201]. Proceedings of the Workshop on Negation and Specu-
12. Kim JD, Ohta T, Pyysalo S, Kano Y, Tsujii J: Overview lation in Natural Language Processing, Uppsala, Swe-
of BioNLP’09 Shared Task on Event Extraction. In den 2010:46–49, [http://www.aclweb.org/anthology/W/
Proceedings of the BioNLP 2009 Workshop Companion W10/W10-3108].
89