=Paper=
{{Paper
|id=None
|storemode=property
|title=
Modeling Emergent Associations of Nominal Compound: Ongoing Research and Preliminary Results
|pdfUrl=https://ceur-ws.org/Vol-704/33.pdf
|volume=Vol-704
|dblpUrl=https://dblp.org/rec/conf/iir/MelucciS11
}}
==
Modeling Emergent Associations of Nominal Compound: Ongoing Research and Preliminary Results
==
https://ceur-ws.org/Vol-704/33.pdf
Modeling Emergent Associations of Nominal
Compounds: Ongoing Research and Preliminary
Results?
Massimo Melucci1 and Laurianne Sitbon2
1
University of Padua, Italy
2
Queensland University of Technology, Australia
Abstract. This paper presents a roadmap to deliver tools able to model
and predict the behaviour of neologisms in the form of nominal com-
pounds. Quite often these compounds yield meanings (in our case, word
associations) that are not related to the components of the compounds
taken individually. Classical probabilities cannot handle this effect, thus
a framework based on quantum probability to model the phenomenon is
proposed.
1 Introduction
Most people agree that the combination pet tree is likely a bonzai and
that a pet human is probably a slave, although none of these combina-
tions exist in the English language, there is no connection between pet
and the interpretations of the compounds, bonzai is a quite rare type of
tree and slave is a rather rare type of human. Words that are connected
to a combination but are not connected to the single constituent words
of the combination are termed “emergent associates”. This capacity that
humans have to share a creative understanding of novel combinations has
been at the centre of several studies in psychology and cognitive science
trying to understand what are the processes conducting to their inter-
pretation. In particular, we have evidence that some novel combinations
yield to associates that cannot be predicted from the associates of the
words taken separately nor from a collection of documents containing
both words together.
Information Retrieval (IR) systems can deal with combinations only if
they occur in the documents, but they are still largely ignoring novel
combinations and emergent associates. In IR, novel combinations not
only can be encountered within queries and documents, but also are es-
sential to capture the intentions of authors or users. Multilingual users
may express queries using novel combinations assuming a consensus on
their interpretation when either they don’t know the correct words (para-
phrasing) or they translate literally from their native language a concept
expressed as a combination. For example, a Chinese user may search
?
The research leading to these results has received funding from the European Union
Seventh Framework Programme (FP7/2007-2013) under grant agreement N. 247590.
documents relevant to hippopotamus after literally translating it river
horse, and lobster would be literally translated dragon shrimp. This strat-
egy could work in human to human communication, but automatic sys-
tems are to date not able to infer such new meanings as early work on
cross-lingual information retrieval using dictionary based translation has
proven [1]
Additionally, native speakers of a language may voluntarily create novel
combinations that cannot be classically interpreted (e.g. from a com-
bination of the associates of the words taken individually) in order to
emphasize the importance of the emerging associates and attract the at-
tention of the reader, in the same way neologisms are intended for [2].
News headlines are typical examples of application of such strategies,
with headlines such as Oklahoma Surprise: Islam as an Election Issue
or Brother’s transplant gift carries unbearable cost, as well as video titles
with for example fish wish or a Wallmart adventure that attracted a lot
of attention on a single day. More generally, it was also found “that no
less than 39% of the neologisms had idiomatic meanings at their very
birth” [3].
The previous examples of usage of novel combinations reflect the main
problems (among others) QONTEXT (EU funded contract N. 247590) is
striving to address: (i) can we predict whether a combination will yield
to an emergent associate? (ii) can we predict what will be the meaning
of an novel combination?
2 Research Question and Contribution
The key to solve these problems is to find a representation that allows
for the modelling of the emergence of unexpected associates from novel
combinations. Classical representations of words and their meaning are
either based on semantic networks (formal relations) or semantic spaces
(based on co-occurrences). Composition is then expressed as a function of
the neighbours or the dimensions of each word taken individually within
a single space [4] which doesn’t allow for the modelling of emergent as-
sociates. QONTEXT will explore the suitability of quantum probability
to model the combined representations as well as its ability to act as a
predictive model.
Quantum Theory (QT) has been previously shown to be the only way
of going beyond the classical interpretation of compositionality to solve
the problem of typicality of combinations [5]. This problem is classically
known as the guppy effect because a guppy is found not to be a typical
example of pet nor fish, but highly typical of “pet fish”. The goal is not
only to model existing combination and examples, but also to derive a
computational model that can be used to interpret novel combinations
enough to generate query expansion terms or accurate translations. To
this end, a quantum probability space has to be defined. In this paper,
it is shown that a single quantum probability space in one vector space
can be defined.
3 Words, Vectors and Probability
In the classical probabilistic model, events (e.g., associates or combina-
tions) are represented as sets and the probability measure is based on
a set measure, e.g., set cardinality. In contrast, in quantum probability,
events are represented as subspaces and probabilities are generated by
density matrices3 , the most simple case being vectors. Suppose that the
vectors |wi, |bi represent an event and a density, respectively.4 The prob-
ability that w occurs given b is provided by the quantum probability rule,
that is, |hw|bi|2 = |awb |2 where awb is termed amplitude and |hx|xi|2 = 1
for every |xi.5 . In general, the probability measure is given by the trace
of the product between the density matrix which corresponds to the
probability distribution, and the matrix representing an event. A quan-
tum probability space is then given by a set of subspaces and a density
matrix. [6]
4 Combination and Quantum Probability
Suppose that b = b1 b2 is a combination (e.g. pet tree) and v is an associate
(e.g., slave or bonzai). Using classical probability, these events are sets
and then b ⊆ b1 , b2 (if b occurs, then both b1 and b2 occur but the
viceversa does not hold). Suppose also that b1 and b2 are quite common
words, thus their probability is not negligible. Moreover,
P (v, b1 ) ≥ P (v, b1 , b2 ) ≥ P (v, b) P (v, b2 ) ≥ P (v, b1 , b2 ) ≥ P (v, b)
If v (e.g., bonzai) rarely occurs with b1 (e.g., pet) or with b2 (e.g., tree), it
is also rare within b and thus it cannot be an emergent associate, which
cannot be detected by classical probability.
Using quantum probability, instead,
|bi = aib |bi i + āib |b̄i i |aib |2 + |āib |2 = 1 hbi |b̄i i = 0 i = 1, 2
and
|hv|bi|2 = |aib hv|bi i + āi bhv|b̄i i|2
= |aib |2 |aiv |2 + |āib |2 |āiv |2 + 2I i = 1, 2 (1)
where the sum of the first two terms of the right-hand side is the classical
probability P (v|b) and
I = |aib ||aiv ||āib ||āiv | cos θ (2)
is the interference term. The conditions for emergence can be studied
using (1) and (2). As −1 ≤ I ≤ +1, (1) can be lower or higher than
P (v|b) and then can be used for predicting emergent associates. The
estimation of I is thus crucial. In particular, whereas the the |aij |2 ’s are
3
In general, density operators.
4
The Dirac notation is used in QT.
5
hx| is the transpose of |xi.
empirical probabilities, the estimation of cos θ is the most difficult step
because it is mainly due to the interpretation of θ, which is the angle of
the complex number aib aiv āib āiv – further investigation are then needed.
The vector formalism allows us to build the quantum probability space.
Suppose that the empirical probabilities |aij |2 are estimated for n as-
sociates w1 , . . . , wn with respect to the bi ’s, i = 1, 2. If the |aij |2 ’s are
arranged in a 2 × n matrix,
„ «
` ´ ai1 · · · ain
(|w1 i · · · |wn i) = |bi i |b̄i i i = 1, 2 (3)
āi1 · · · āin
Through SVD of the 2 × n matrix, the |wj i’s and the |bi i’s can be built,
thus obtaining the space and the probability |hb1 |b2 i|2 which measures
the distance between the constituent words of the combination b.
5 Concluding Remarks and Future Work
A single representation of the events is obtained so that the empirical
probabilities can be reproduced by the quantum probability rule. The
modeling results are still quite preliminary, however, we are confident
that the ongoing research may provide the useful theoretical framework
for detecting emergent associates.
References
1. Kwok, K.: Evaluation of an english-chinese cross-lingual retrieval ex-
periment. In: Working Notes of AAAI-97 Spring Symposiums on
Cross-Language Text and Speech Retrieval. (1997) 110–114
2. Lehrer, A.: Understanding trendy neologisms. Italian Journal of Lin-
guistics (2003)
3. Schmid, H.J.: New words in the mind: Concept-formation and en-
trenchment of neologisms. Anglia-Zeitschrift Fur Englische Philologie
126 (2008) 1–36
4. Lapata, M., Mitchell, J.: Composition in distributional models of
semantics. Cognitive Science 34 (2010) 1388–1429
5. Aerts, D., Gabora, L.: Diederik aerts and liane m. gabora, a theory
of concepts and their combinations ii: A hilbert space representation.
Kibernetes 34 (2004) 192–221
6. Melucci, M., van Rijsbergen, C.J.: Quantum mechanics and infor-
mation retrieval: a probabilistic overview. In: Advanced Topics in
Information Retrieval. Springer (Forthcoming)