English In this paper we introduce the task of interpreting verbal neologism (VNeo) for the Italian language making use of a highly context-sensitive distributional semantic model (DSM). The task is commonly performed manually by lexicographers verifying the contexts in which the VNeo appear. Developing such a task is likely to be of use from a cognitive, social and linguistic perspective. In the following, we first outline the motivation for our study and our goal, then focus on the construction of the dataset and the definition of the task.
Studying neologisms can tell us several things.
From a lexicographic point of view, neologisms
can show trends that a language is following. In
our opinion, they can also shed light on various
aspects related to linguistic creativity; when
speakers use new words (coined by themselves,
or recently coined by someone else), they expect
that the hearer can understand what they have
1 This is not the case of neologisms created for
advertising, brand names or marketing purposes in
general
The interpretation of neologisms presents two
main challenges: a) analysing verbs using vectors
built only upon co-occurrences (thus excluding
argument structures) is notoriously a difficult
task for DSM;3 b) neologisms are, by definition,
words whose frequency is (very) low, because
their use is (still) not widespread. Thus, it
represents a challenge for DSM models exactly
because the vectors for most VNeo will rely
upon few occurrences. In order to evaluate our
results, we will compare them with the ones
obtained using the Word2Vec model
DSM is a technique for building up measurable,
computationally tractable lexical semantic
representations based on observations of the way
that words co-occur with one another across
large-scale corpora. This methodology is
grounded in the distributional hypothesis, which
maintains that words that are observed to have
similar co-occurrence profiles are likely to be
semantically related
Standard DSM techniques present two problems for the task of interpreting neologisms. First, distributional representations are predicated on many observations of a word across a large-scale corpus: it is the plurality of context which gives these representations their semantic nuance. Second, the spaces generated by standard approaches like matrix factorisation and neural networks are abstract, in the sense that their dimensions are not interpretable; as such, typical distributional semantic models are not sensitive to the context specific way in which meaning arises in the course of language use. McGregor et al. (2015) have proposed a contextsensitive approach to distributional semantic modelling that seeks to overcome this second problem by using contextual information to project semantic representations into lower dimensional conceptual perspectives in an online way.
This methodology entails the selection of sets
of dimensions from a base space of
cooccurrence statistics that are in some sense
conceptually salient to the context being
modelled. The selection of salient features
facilitates the projection of subspaces in which
the geometric situation of and relationship
between word-vectors are expected to map to a
specific conceptual context. This technique has
been applied to tasks involving context sensitive
semantic phenomena such as metaphor rating
With regard to the first problem of data
sparsity, we propose that the facility of the
dynamically contextual approach for handling
the ad hoc emergence of concepts
We will now explain the methodology we use in our analysis, and describe the resources we exploit highlighting their main features.
To select the VNeo to be analysed, we extract
data from pre-existing lists of Italian neologisms.
These lists come from three websites: a)
treccani.it4 b) iliesi.cnr.it/ONLI/5 c)
accademiadellacrusca.it.6 (a) and (b) are
manually compiled and validated: they contain
words manually found in some widely read
newspapers but not (yet) included in Italian
dictionaries, coherently with the lexicographical
definition of neologisms
Within the lists, we select only the verbs, obtaining a set of 504 VNeo. Of these VNeo, we check their presence in the itTenTen16 corpus, which we will also use to create the distributional vector space. 340 VNeo are attested in the corpus: 108 have between 10 and 99 occurrences; 79 between 100 and 999 occurrences; and 26 have more than 1000 occurrences.
Instead of using heuristic techniques that might have identified neologisms within the corpus (e.g. computing less frequent words and manually checking their presence in dictionaries),7 we chose to rely on lists because we intend to study words whose use is wider and not restricted only to the web domain.
We conduct an analysis of the itTenTen16 corpus (Jakubíček et al. 2013) because it is the most up-to-date corpus available for Italian. It is also a web-based corpus, and so particularly well fitted to examine neologisms: in fact, the web and IT domain is a notable source of new words and, especially, of new loanwords. As the corpus dimensions are sizeable (4.9 billion tokens), we will use a random sample of the full corpus for purposes of computability. This sample will correspond to ⅕ of the original corpus. 4 http://www.treccani.it/magazine/lingua_italiana/ neologismi (last consulted 10/04/2018) 5 http://www.iliesi.cnr.it/ONLI/BD.php (last consulted 02/05/2018) 6 http://www.accademiadellacrusca.it/it/linguaitaliana/parole-nuove (last consulted 02/05/2018) 7 We are aware that this might correspond to the loss of some other neologisms contained in the corpus.
Starting from the corpus, the base DSM is built based on observations of the most frequent 200,000 words (defined as vocabulary) and their contextual information, considering a cooccurrence window of 5 words on either side of a target word. For the purposes of this study, we consider the VNeos included in the vocabulary. In this way we obtain the base space.
In order to project a subspace contextualised by a VNeo, we consider the co-occurrence features with the highest mutual information statistics associate with that particular VNeo. So, for instance, we find the following salient features:
customizzare 'to customise' [city; modellazione; illustrato; type; batch; editare; nastro; segmentare; preferenza; iconico; ...] resettare 'to reset' [reset; password; formattare; bios; clempad; clementoni; fonera; resettare; centralina; router; ...]
googlare 'to google' [telespettatore; pdf; tecnologia; informazione; addirittura; vi; chiave; invito; risposta; sapere; ...].
These features are associated with the maximum mutual information values in terms of their co-occurrence with each of the corresponding input neologisms.
Some other VNeos represented in the vocabulary are: postare ‘to post’, taggare ‘to tag’, twittare ‘to tweet’, spammare ‘to spam’, attenzionare ‘to warn’, spoilerare ‘share information that reveals plot of a book or film’, bloggare ‘to blog’, loggare ‘to log’, switchare ‘to switch’.
It is worth noting that we create vectors
starting from lemmas (not tokens). Our analysis
highlighted the presence of some inaccuracies in
the automatic lemmatisation of neologisms,8
which was already present in the original
corpus.9 In a future investigation we are planning
to compare the results produced with the original
lemmatised corpus against the results obtained
from a corpus version, where the lemmatisation
will be corrected. This correction process might
be performed using regular expressions, in order
8 Neologisms are not stored in common word-lists,
and they are (usually) rare words, thus presenting
difficulties for machine learning techniques.
9 The lemmatisation is obtained using the TreeTagger
tool
VNeos token.10 As referenced in §1, our goal is to verify whether the meaning of a neologism can be induced from its context through distributional techniques, in particular by discovering verbs with salient geometric features in a contextualised subspace.
To this end, we organize the task as follows. Starting from a subset of the most frequent VNeos found in the corpus (§3), we first build subspaces for VNeos using the DSM model presented in §2. Subspaces are created by selecting the sets of dimensions that are conceptually salient to the context being modelled: each dimension in a subspace corresponds to a specific co-occurrence feature (i.e. a word). By finding a whole set of cooccurrences and using these to generate a relatively high-dimensional projection, we hope to establish a general contextualised conceptual profile and to overcome the peculiarities associated with low-frequency targets. For example, if the model finds that googlare ‘to google’ co-occurs with words like nome ‘name’, indirizzo ‘address’, and sito ‘website’, we use those co-occurrences as a basis for a projection of a subspace in which one could predict to find 10 Regular expressions might be useful, within the corpus, to find an inflected form of a verb (lemmatised as it is) and replace it with the correct lemma: e.g. find lemma googlav. (meaning googlavo, googlavi, etc.) and replace it with googlare. terms like cercare ‘search’ using geometric techniques.
Context can be defined in an open ended way
in these models. For instance, the salient
cooccurrence features of a single word can be used
to generate a subspace. Small sets of words,
either components of observed compositions
Contextualised subspaces can be explored in terms of the geometric features of word-vectors projected into those subspaces. So, for instance, McGregor et al. (2015) propose a norm method, by which word-vectors salient in a particular context will emerge as being far from the origin. This phenomenon is observed with appropriate interpretations percolating into the salient regions even in the low-dimensional toy examples illustrated in Figure 1, which involves a dynamically contextual DSM built from English language Wikipedia. Choices about context selection techniques, geometric characteristics of subspaces to be explored, and modelling parameters including dimensionality of projections will be the subject of our forthcoming experiments.
In order to evaluate the model, we will
compare our results against the results obtained
applying the Word2Vec model to the same
corpus
With further investigations we will also test this model using a gold standard consisting of human judgments on VNeos interpretations collected for this purpose. Similarity judgments will be provided by two native speakers with significant background in linguistics. Specifically, the dataset will consist of verb pairs in which VNeo are grouped with more common verbs (googlare and cercare) based on human ratings collected in the form of a TOEFL-like multiple-choice synonymy test.11 11 Here the task is to determine, for a number of target words, the closest synonym from a choice of four alternatives.
The aim of the task presented here is to investigate the importance of linguistic context for the interpretation of neologisms, grounding the analysis in a context-sensitive DSM. With this task we intend to tackle issues connected with creativity processes and the environmental (contextual) sensibility typical of human cognition. In addition, we apply, for the first time, this DSM to Italian, providing a new semantic resource for the analysis of the language. Further studies may compare our results with other DSMs, and/or study what the semantic relations found with this specific approach reveal about other phenomena belonging to different linguistic levels (e.g. syntax). of distributional
Linguistics,