English. This paper presents work in progress for the development of IFrameNet, a large-scale, computationally oriented, lexical resource based on Fillmore's frame semantics for Italian. For the development of IFrameNet linguistic analysis, corpusprocessing and machine learning techniques are combined in order to support the semiautomatic development and annotation of the resource. Italiano. Questo articolo presenta un work in progress per lo sviluppo di IFrameNet, una risorsa lessicale ad ampia copertura, computazionalmente orientata, basata sulle teorie di Semantica dei Frame proposte da Fillmore. Per lo sviluppo di IFrameNet sono combinate analisi linguistica, corpusprocessing e tecniche di machine learning al fine di semi-automatizzare lo sviluppo della risorsa e il processo di annotazione.
Firstly developed at the University of Berkeley
(California) in 1997, FrameNet adopts theories
from Frame Semantics
The initial American project has since been extended to other languages: French, Chinese, Brazilian Portuguese, German, Spanish, Japanese, Swedish and Korean.
All these projects are based on the idea that
most of the Frames are the same among languages
and that, thanks to this, it is possible to adopt
Berkeley’s Frames and FEs and their relations,
with few changes, once all the language-specific
information has been cut away
With regard to Italian, over the past ten years
several research projects have been carried out at
different universities and Research Centres. In
particular, the ILC-CNR in Pisa
As a lexical resource of this kind is useful in
many computational applications (such as
HumanRobot interaction), a new effort is currently being
jointly made at the universities of Bologna and
Roma, Tor Vergata. The IFrameNet project aims to
develop a large-coverage FrameNet-like resource
for Italian, relying on robust and scalable methods,
in which the automatic corpus processing is
consistently integrated with manual lexical analysis. It
builds upon the achievements of previous projects
that automatically harvested FrameNet LUs
exploiting both distributional and WordNet based
models
The aim is also to provide Sample Sentences for
LUs with the highest corpus frequency. On the one
side, they will be derived from already existing
resources such as the HuRIC corpus
The need for a large-scale resource cannot be satisfied without resorting to a semi-automatic process for the gathering of linguistic evidence, selection of lexical examples as well as the annotation of the targeted texts. This work is thus at the cross roads of linguistic theoretical investigation, corpus analysis and natural language processing.
On the one hand, the matching between LUs and frames is always granted through manual linguistic validation applied to the data in the development stage. For every Frame the correctness of the inducted LUs is analysed and the ‘missing’ LUs, that is the BFN LUs’ translations, which are absent in the inducted LU’s list, are detected.
On the other hand, most choices rely on large sets of corpus examples, as made available by CORIS. Finally, the scaling to large sets of textual examples is supported by automatically searching candidate items through semantic pre-filtering over the corpus: frame phenomena are here used as queries while intelligent retrieval and ranking methods are applied to the corpus material to minimize the manual effort involved.
In the following section, we will sketch the main stages of the process that integrate the above paradigms. 2.1
The beneficial contribution of the interaction
between corpus processing techniques and lexical
analysis for the semi-automatic expansion of the
FrameNet resource has been discussed since
In order to rely on comparable representations
for LUs and sentences for transferring semantic
information from the former to the latter, we
exploit Distributional Models (DM) of Lexical
Semantics, in line with
In the following paragraphs, we outline the different stages in the development process. Each stage corresponds to specific computational processes. Validation of existing resources. At this stage, the existing resources, dating back to previous work, are analysed and manually pruned of errors such as lexical units wrongly assigned to frames (e.g. ‘asta’ or ‘colmo’ to the Frame ‘BODY_PARTS’), or words never assigned to their correct frame, for instances the LU ‘piede’ or ‘mano’ for the Frame ‘BODY_PARTS’.
All the acquired Italian LUs have been compared, frame by frame, to BFN’s ones, using bilingual dictionaries (e.g. Oxford bilingual dictionary) and WordNet in order to verify the correctness of matching between lexical and frames. Over the 15,134 automatically acquired ⟨LU, frame⟩ pairs (6,670 nouns and 8,464 verbs and adjectives), 7,377 LUs have been considered correctly assigned (2,506 verb and adjective and 4,871 noun pairs).
In addition, bilingual dictionaries, ItalWordNet and MultiWordNet have been used to manually insert a list of missing lexical entries for each frame. At the end of the process, the resulting validated and refined ⟨LU,frame⟩ amount to 7,902 (5,128 nouns and 2,774 verbs and adjectives).
Corpus processing and lexical modeling. At
this stage, the LUs made available from manual
validation are used to model distributionally the
individual frames. Firstly, distributional corpus
analysis is applied to map individual LUs into
distributional vectors. A distributional model will be
acquired from the CORIS corpus by applying the
neural method presented in
Lexical clustering is important here as specific
space regions enclosing the instance vectors of
some considered LUs correspond to semantically
coherent lexical subsets. This is a priming function
for mapping unseen word vectors to frames, as
applied in
The distributional information has been acquired for the considered 7,902 LUs from CORIS and used to support the LU mapping and the sentence validation. In fact, given a sentence s containing a target LU l, a specific geometrical representation for s can be derived by linearly combining all vectors representing words w surrounding l in sentence s. This duality property allows the embedding space to represent sentences s, lexical units l as well as generic words w. This enables to model the relevance of a frame f for an incoming sentence s through the distance d(f,s) between vectors f related to a centroid for a frame f and the vector s of the sentence s. It corresponds to a confidence measure computed for a rule such as:
“s is a valid example of the usage of frame f ” The open aspects of the above semi-automatic process are the following:
How to design a suitable representation (centroid or model) for a frame f How to define the vector for a sentence s
How to compute the distance function d(f,s) The current research activity is focusing on the best solution for these issues and part of our experimental activity is devoted to assess these design choices, as discussed in Section 3.
First Lexical Analysis and Validation. A further stage for the resource development focuses on the selection of a significant sample of LUs, chosen on the basis of their high semantic salience and for their high number of occurrences in the corpus (primary LUs). By relying on the method described above, we use the distributional representation of words, lexical units and sentences, to gather CORIS sentences s where a LU occurs and evaluate its suitability as an example for the evoked f. This decision function is based on the geometric distance d(f,s) that can be computed over a large number of sentences s. When this step is carried out in CORIS, the validation of the acquired candidate sentences allows for positive examples of a frame f to develop quickly: this is used to trigger supervised learning of f.
The manually validation in fact confirms the proper correspondence between automatically selected sentences and LUs that evoke a targeted frame f. It produces novel seed examples for f: these will serve as a training set for a semi-automatic stage of resource expansion.
Semi-automatic resource expansion. The
acquired distributional model will support the
semiautomatic expansion of the seed set, by selecting
the most semantically similar word to the seed set
and assigning them to frames by applying the
methodologies suggested in
tences will be ordered by decreasing probability, according to their distributional collocation, and a list of 15 to 20 candidates per LU will be provided. This list will be manually validated. The aim is to provide at least 4 sample sentences for each of the primary LUs.
Although the general software architecture for the project progress is available, the overall process described above has not been fully accomplished.
Current material covers a set of 554 frames and 7,902 lexical units, of which 2,604 verbs, 5,128 nouns and 170 adjectives. The average number of occurrences for each of these selected words is higher than 9,400, although there are still 508 words not present in CORIS.
All these occurrences correspond to a number of about 70 millions non validated and unsorted sentences. In the rest of the paper, we describe the outcome of the First Lexical Analysis and Validation stage: its aim is to trigger the semi-automatic learning and tagging of the whole corpus, according to the methods suggested in section 2.2. 3.1
The stage First Lexical Analysis and Validation has been currently accomplished. The three research questions posed above: (I) the modelling of a frame f, (II) the sentence representation and (III) the definition of a distance function able to model similarity between sentences.
About the problem (I) two approaches are
possible. We can model a frame via clustering its
lexical units and applying the method described in
“f is the frame evoked by l in s” is true. Notice that the pair ⟨s,l⟩ can be expressed as an instance by combining the embedding vector l of its lexical unit l with a vector s for s.
As a solution for the problem (II) we define s as the linear combination of vectors w, for each word w in s, i.e. s = Σw∈s w .
The above formulation allows to define the classification task as follows:
Given a sentence s including a word l as a potential frame evoking LU, Find the frame f that characterizes l in s.
The solution of the above problem over a ⟨s,l⟩ pair would also be a useful solution for the problem (III), as the confidence h(s,l,f) in the classification of a sentence s in a frame f for l can be retained as the inverse of the target distance function d(f,l) local to the sentence.
The major problem with the above formulation is that the training of the statistical classifier is not possible without the availability of useful examples of different frame f. The idea is thus to develop ways to derive from CORIS the proper candidates s for f through the knowledge of some of its LUs. In the bootstrapping stage, we define as virtual examples the pairs ⟨l,{l}⟩ that are retained as positive examples for the frame f, for every l that is a known lexical unit for f. In our approach, an example is thus obtained by modelling the sentence s as a singleton {l}, i.e. the lexical unit l.
A statistical classifier considers every known LU as an individual (positive) example and can be applied to every LU in our initial resource (i.e. 7,902 for the 554 frames).
In synthesis, the method works as follows. First,
for every lemma w in the corpus, an n-dimensional
embedding vector w is derived, according to
A multiclass-statistical categorizer is trained for every frame f for which at least 5 examples (i.e. 5 different LUs) where available.
When applied to an incoming sentence s including a LU l, the classifier outcome h(l,s, f) is said to accept the frame f if: • f belongs to the set of frames evoked by l • f = argmaxf’ { h(l,s,f’) } For every sentence s including a frame evoking lexical unit l, the above function suggests one candidate frame among the possibly multiple ones. When the scoring function h is negative everywhere (e.g. with the SVM formulation of a classification task), the sentence is rejected and is not considered a valid example for future iterations.
The application of this method to the CORIS corpus has been carried out applying a multiclassifier SVM with linear kernel to the 2ndimensional vectors of each pair ⟨l, {l}⟩. Starting from the lexicon validated in the first stages, the SVM has been able to label over 2 million sentences. 3.2
In order to evaluate the proposed supervised classification method for the stage “First Lexical Analysis and Validation” we run and experimental evaluation over a set of 3261 frames, the ones with more than 5 lexical units in the initial lexicon. In this way, we selected 1,095 different LUs, represented as an embedding vector in the wordspace. On average, we have 12 LU per frame, and every individual lexical entry l appears in about 1.88 frames. The baseline of a classification task that maps a sentence s including a lexical unit into its own frame is about 35%, as for the ambiguity characterizing most frequent entries.
We asked three annotators to evaluate individual triples ⟨l, s, f⟩ validating the system proposal. Four main cases where possible: • MISSING FRAME. The sentence s is not manifesting any of the frames f evoked by the lexical unit l, but corresponds to a frame not yet present in the lexicon for l. In this case the algorithm cannot provide the suitable frame, as it cannot generate a novel frame. • NOT APPLICABLE. The sentence s does not contain an occurrence of the lexical unit l in one of its proper senses: this case is typical for phraseological uses of a verb such as morire di freddo, andare di fretta, … that do not directly correspond to lexical predicates and thus cannot be treated through the lexical embedding vectors. • CORRECT/INCORRECT, when the outcome argmaxf’ { h(l,s,f’) } is correct (or incorrect) as the frame evoked by l in s is exactly (or not) f. According to the above method annotators validated 667 sentences for 113 frames and 212 different verbal lexical units. The analysis resulted into a precision (i.e. the number of correct candidate frames emitted by the algorithm w.r.t. the number of valid cases, that is all but the MISSING FRAME or NOT APPLICABLE cases) is 75,2%, well beyond the 35% baseline. The method could be applied onto the 74,5% of the sentences, including CORRECT cases and MISSING FRAME cases. We neglected in this coverage score the NOT APPLICABLE cases that amount to 44 sentences, i.e. about 6,4%.
Examples of the correct assignment of the algorithm on quite ambiguous verbs, such as finire (i.e. to end, in frames ACTIVITY_FINISH, CAUSE_TO_END and KILLING) or rivelare (i.e. to reveal, in frames REVEAL_SECRET, OMEN, EVIDENCE) are the following: La vicenda avrebbe potuto [finire]ACTIVITY_FINISH lì , ma il prefetto di Nuoro fece presentare ...
In prova si è [rivelato]EVIDENCE ad altissimo livello sia sull' asciutto sia sul ... 1 By keeping the frames that include at least 4 lexical units the number of targeted frames grows to 371.
An example of Missing Frame is BEAT_OPPONENT for the verb battere in ... impegnato a fornire quante più informazioni possibili, anche per [battere]BEAT_OPPONENT la concorrenza dei siti Ipsoa e il ... as the lexicon of the verb battere only includes the frames CAUSE_HARM, CORPORAL_PUNISHMENT and EXPERIENCE_BODI-LY_HARM.
The experiments only run over verbal lexical units will be extended soon to nouns and adjectives. However, the encouraging precision reached by the method allows for direct use it in an iterative active learning schema, where the more ambiguous sentences found and annotated within a specific training stage are used to train the system at the next stage. We expect this to speed up the lexicon development process and to allow bootstrapping with fewer resources. The lexicon will be made available for crowdsourcing further annotations and delivered incrementally in the next few months.