=Paper=
{{Paper
|id=Vol-2765/145
|storemode=property
|title=KonKretiKa @ CONcreTEXT: Computing Concreteness Indexes with Sigmoid Transformation and Adjustment for Context
|pdfUrl=https://ceur-ws.org/Vol-2765/paper145.pdf
|volume=Vol-2765
|authors=Yulia Badryzlova
|dblpUrl=https://dblp.org/rec/conf/evalita/Badryzlova20
}}
==KonKretiKa @ CONcreTEXT: Computing Concreteness Indexes with Sigmoid Transformation and Adjustment for Context==
https://ceur-ws.org/Vol-2765/paper145.pdf
KonKretiKa @ CONcreTEXT: Computing Concreteness Indexes with
Sigmoid Transformation and Adjustment for Context
Yulia Badryzlova
HSE University
Moscow, Russia
yuliya.badryzlova@gmail.com
(Lakoff and Johnson, 1980). These theories claim
Abstract that human thinking is intrinsically metaphoric,
since the conceptual representations underlying
The present paper is a technical report of knowledge are grounded in sensory and motor
KonKretiKa, a system for computation of systems, and conceptual metaphor is the primary
concreteness indexes of words in context, mechanism for transferring conventional mental
submitted to the English track of the imagery from sensorimotor domains to the do-
CONcreTEXT shared task. We treat con- mains of subjective experience.
creteness as a bimodal problem and com- An established method to compute the con-
pute the concreteness indexes using para- creteness index of a word is to collect two sets of
digms of concrete and abstract seed words lexemes (‘seed lists’, or ‘paradigms’) consisting
and distributional semantic similarity. We of abstract and concrete words – and to measure
also conduct sigmoid transformation to the lexical similarity between each word in the
achieve greater similarity to the psycho- lexicon and each of the paradigm words.
linguistically attested data, and apply dy- Turney et al. (2011) use concreteness indexes
namic adjustment of static indexes for to identify linguistic metaphor in the TroFi dataset
sentential context. One of the modifica- (Birke and Sarkar, 2006). They compute the con-
tions of the presented system ranked third creteness index of a word by comparing its distri-
in the task, with rs = .6634 and r = .6685 butional semantic embedding to the vector repre-
against the gold standard. sentations of 20 abstract and 20 concrete words.
The paradigm words are automatically selected
1 Introduction from the MRC Psycholinguistic Database Ma-
chine Usable Dictionary (Coltheart, 1981), a col-
This paper is a description of the system with the lection of 4,295 English words rated with degrees
working title KonKretiKa, which was submitted of abstractness by human subjects in psycholin-
to the English track of CONcreTEXT, the shared guistic experiments.
task on evaluation of concreteness in context Tsvetkov et al. (2013) also compute the con-
(Gregori et al., 2020) offered at EVALITA 2020, creteness indexes of English words by using a dis-
the 7th evaluation campaign of Natural Language tributional semantic model and the MRC data-
Processing and speech tools for the Italian lan- base. They train a logistic regression classifier on
guage (Basile et al., 2020). 1,225 most abstract and 1,225 most concrete
KonKretiKa stems from our previous work on words from MRC; the degree of concreteness of a
computation of such indexes for the purposes of word is the posterior probability produced by the
metaphor identification. classifier. The Tsvetkov et al. system for meta-
Computationally obtained indexes of concrete- phor identification with concreteness indexes is
ness are extensively explored in experiments for based on cross-lingual model transfer, when the
automated metaphor identification. Application of model is trained on English data, and then the
concreteness indexes to metaphor identification classification features are translated into other lan-
relies on the assumptions made by the theories of guages by means of an electronic dictionary.
embodied and grounded cognition (Barsalou,
2008), and primary and conceptual metaphor
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0
International (CC BY 4.0).
albatross, balloon, bench, bridge, catfish, cauliflower, chicken, clown, corkscrew, crab,
Concrete
daisy, deer, eagle, egg, frog, garlic, goat, harpsichord, lion, mattress, mussel, nightgown,
nightingale, owl, ox, pants, peach, piano, pig, potato, quilt, rabbit, saxophone, sheep,
shrimp, skyscraper, sofa, stoat, tulip, turtle
affirmation, animosity, demeanour, derivation, determination, detestation, devotion, enuncia-
Abstract
tion, etiquette, fallacy, forethought, gratitude, harm, hatred, ignorance, illiteracy, impatience,
independence, indolence, inefficiency, insufficiency, integrity, intellect, interposition, justifi-
cation, malice, mediocrity, obedience, oblivion, optimism, prestige, pretence, reputation, re-
sentment, tendency, unanimity, uneasiness, unhappiness, unreality, value
Table 1. The concrete and the abstract paradigm lists.
∀ 𝑣𝑖 , ∀𝑠𝑗 ∃ 𝐷𝑖 = {𝑆𝑖𝑚(𝑣𝑖 , 𝑠1 ), 𝑆𝑖𝑚(𝑣𝑖 , 𝑠2 ), … , 𝑆𝑖𝑚(𝑣𝑖 , 𝑠𝑗 ), … , 𝑆𝑖𝑚(𝑣𝑖 , 𝑠𝑘 )} , (1)
where 𝑉 is the set of words in the vocabulary,
𝑆 is the set of words in the seed list, k is the number of elements in S
𝑁𝑁 = {𝑑𝑖 1′ , 𝑑𝑖 ′2 , … , 𝑑𝑖 10
′
}, (2)
where 𝐷𝑖 ′ is a linearly ordered set of 𝐷𝑖 (in ascending order)
𝐼 = 𝑀𝑒𝑎𝑛{𝑁𝑁} (3)
Equations 1-3. Computation of indexes with paradigm lists.
Badryzlova (2020) explores concreteness and The present work develops and extends the
abstractness indexes for linguistic metaphor iden- method of Badryzlova (2020) in two directions:
tification in Russian and English. The paradigm (a) we apply sigmoid transformation to fit the
words are selected in a semi-automatic fashion: curve comprised of the computed concreteness
the Russian paradigm is derived from the Open and abstractness indexes to the distribution of in-
Semantics of the Russian Language, the semanti- dexes in psycholinguistic data; and (b) we suggest
cally annotated dataset of the KartaSlov database a method for dynamic adjustment of the obtained
(Kulagin, 2019); the English paradigm is selected indexes for sentential context, according to the re-
from the MRC database (Coltheart, 1981). The in- quirements of the CONcreTEXT shared task
dexes of concreteness and abstractness are com- (Gregori et al., 2020). The working title of the pro-
puted for large sets of Russian and English words posed system is KonKretiKa.
(about 18,000 and 17,000 lexemes, respectively).
The metaphor identification in Russian is con- 2 Description of the system
ducted on the RusMet corpus (Badryzlova, 2019;
We demonstrate a method for evaluating con-
Badryzlova and Panicheva, 2018), and the Eng-
creteness on English data; however, it can be
lish on the TroFi dataset. The author shows that
transferred to any other language provided that the
the distributions of concreteness and abstractness
following types of resources are available:
indexes in the two languages follow the same pat-
(1) a lexicon with semantic (e.g. Fellbaum, 1998;
tern: in the lexicon, there is a distinct group of
Kulagin, 2019) or psycholinguistic (e.g. Brysbaert
highly concrete words, which have very high con-
et al., 2014; Coltheart, 1981) annotation to select
creteness and very low abstractness indexes; sim-
the paradigm words from; (2) a pre-trained distri-
ilarly, there is a group of distinctly abstract vocab-
butional semantic model; and (3) a relatively
ulary, with low concreteness and high abstract-
large wordlist containing lexemes with different
ness scores. Moreover, there is a general trend for
frequencies of occurrence (ipm) in order to ensure
abstractness indexes to increase as the corre-
the maximum possible variation in concreteness
sponding concreteness indexes decrease. The au-
across the lexicon.
thor also observes statistical correlation between
When analyzing the distribution of psycholin-
two Russian abstractness ratings, which may indi-
guistic concreteness ratings, Brysbaert et al.
cate that the category of abstractness is more se-
(2014) observe that “concreteness and abstract-
mantically homogeneous than the category of
ness may be not the two extremes of a quantitative
concreteness.
continuum […], but two qualitatively different
Figure 1. Distribution of computational (raw KonKretiKa) and psycholinguistic (Brysba-
ert et al.) indexes
Figure 2. Sigmoid transformations
characteristics.” of a word. Following this obser- pre-trained on the lemmatized Gigaword 5th Edi-
vation and the previous work in (Badryzlova, tion corpus (Parker et al., 2011).
2020), we treat concreteness as a bimodal prop- As shown in Equations 1-3, to compute a con-
erty investing the word with two characteristics: creteness or an abstractness index (𝐼) of a word,
the rate of concreteness and the rate of abstract- we measured semantic similarity (cosine distance)
ness. Thus, we start by computing the standalone Sim between the vectors of this word and each
indexes of concreteness and of abstractness; then, word in the paradigm (concrete or abstract, re-
the single aggregate index is computed as a func- spectively), and took the mean of the ten nearest
tion of these two indexes. semantic neighbors (NN).
In total, we computed concreteness and ab-
2.1 Computation of raw indexes with para- stractness indexes for approximately 23,000 Eng-
digm words and distributional semantic lish words (nouns, verbs, adjectives, and adverbs);
similarity this lexicon was taken from the Brysbaert et al.
Computation of the standalone concreteness and (2014) ranking, which allowed us to analyze the
abstractness indexes is based on paradigm lists of correlation between the computational and the
concrete and abstract words; we use the English large-scale psycholinguistic data at the subse-
concrete and abstract paradigms from Badryzlova quent stages of the present study (see Section 3).
(2020). These paradigms were compiled from the The obtained computational sets of concrete-
MRC Psycholinguistic Database: nouns from the ness and abstractness indexes were normalized to
top and from the end of the MRC concreteness rat- the range [1, 7]1 in order to comply with the scale
ing were drawn to populate the concrete and the set by the CONcreTEXT shared task. In order to
abstract paradigms, respectively. The paradigm obtain an aggregate single-value index of a word,
lists are presented in Table 1. which would be representative of both its con-
The indexes of concreteness and abstractness creteness and abstractness, we subtracted the ab-
were computed using a Continuous Skip-Gram stractness indexes from the concreteness indexes.
model (Kutuzov et al., 2017) which had been
1
Scikit-learn’s MinMax Scaler (Pedregosa et al.,
2011)
this fitting, only the values of the indexes are ad-
justed, while their initial ranks remain intact –
Transformation
adjustment (c)
Contextual
System thus, there is no data leakage from the psycholin-
guistic ranking. 2
type
Result (rs) Result (r)
2.3 Contextual adjustment
Since the CONcreTEXT shared task requires that
the concreteness indexes of target words be dy-
Leader-1 0.83313 0.83406 namically adjusted to their sentential context, the
Leader-2 0.78541 0.78682 following heuristic was applied in the submitted
KonKretiKa-3 2 0.5 0.6634 0.6685 KonKretiKa models. We computed the mean con-
creteness of all content words in the sentence
KonKretiKa-1 1 0.5 0.65102 0.66652
(with the target word excluded) and adjusted the
Baseline-2 0.55449 0.56742 concreteness value of the target word accordingly.
KonKretiKa-4 2 0.8 0.54216 0.54465 The adjusted index 𝐴 was computed as follows:
KonKretiKa-2 1 0.8 0.54089 0.54479 𝐴𝑡 = 𝑅𝑡 − (𝑀 ∗ 𝑐)
Baseline-1 0.3825 0.37743
where 𝑡 is the target word, 𝑅 is the raw index
Table 2. Modifications of KonKretiKa and from the KonKretiKa ranking, 𝑀 is the mean con-
their results in the shared task.
creteness of the sentence, and 𝑐 is the adjustment
coefficient. In the models submitted to the CON-
2.2 Sigmoid transformation of raw indexes creTEXT shared task, we applied two heuristi-
cally defined 𝑐 coefficients: 𝑐 = 0.5 and 𝑐 = 0.8.
Figure 1 shows distributions of our raw aggregate Thus, the four modifications of KonKretiKa
indexes and the indexes attested in psycholinguis- submitted to the shared task were differentiated by
tic research (Brysbaert et al., 2014). It is noticea- the two parameters: the type of transformation and
ble that the curve of computational indexes has a the contextual adjustment coefficient.
much steeper slope, resulting in lower variance;
consequently, the discriminative power of such 3 Results and discussion
indexes will also be lower.
The raw KonKretiKa curve has the shape of a The parameters of the four modifications and their
sigmoid; in generic form, the sigmoid function is results are presented in Table 2 (along with the
described by the equation: Baselines and the Leaders). The results indicate
that systems with the lower coefficient of senten-
1
𝑆(𝑥) = tial adjustment (0.5) perform better than systems
1 + exp(−𝑎𝑥 + 𝑎 ∗ 𝑏) with the higher adjustment coefficient (0.8) irre-
where 𝑎 defines the slope of the function and 𝑏 spective of the type of sigmoid transformation;
defines the inflection point. Consequently, we can yet, the system with Type 2 (fitted to the psycho-
transform the sigmoid by changing the 𝑎 and 𝑏 linguistic data) transformation somewhat outper-
coefficients. forms the system with Type 1 (S-shaped) transfor-
In the submissions to the CONcreTEXT shared mation.
task, we experimented with two transformations The best of our modifications, KonKretiKa-3,
of the raw KonKretiKa curve (Figure 2). In the demonstrated Spearman correlation with the gold
first transformation, we applied a heuristically standard rs = .6634 and Pearson correlation
chosen combination of 𝑎 and 𝑏 which was in- r = .6685, ranking our system third in the track,
tended to increase the slope and the curvature yet by a substantial margin behind the two win-
while preserving the S-shape of the sigmoid. The ning system (with rs = .83313 and r = .83406 and
second transformation was intended to attain rs = .78541 and r = .78682, respectively).
maximum resemblance of its shape to the Brysba-
ert et al. curve. We used grid search with different
combinations of coefficients 𝑎 and 𝑏 to maximize
the correlation between the two curves. During
2
The KonKretiKa ranking is available at:
https://github.com/yubadryzlova/CONcreTEXT-2020
Gold BRY word BRY KKK Diff
Dataset
(dynamic) (static)
handmaiden (N) 6.45 1.54 4.91
rs =.743
KKK (static) tire (V) 7 2.18 4.82
r =.751
rs = .663 bedrock (N) 6.18 1.55 4.63
KKK (dynamic)
r = .669
alarm (N) 6.19 1.58 4.61
rs = .755
BRY (static) text (N) 6.89 2.31 4.58
r = .761
Table 3. Pairwise correlations: nonreactive (ADJ) 2.25 6.82 -4.57
KKK – KonKretiKa, BRY – Brysba- temptingly (ADV) 1.72 6.26 -4.55
ert et al., Gold – CONcreTEXT gold stand-
ard. hail (V) 5.96 1.5 4.47
stance (N) 5.53 1.11 4.42
3.1 Analysis of contextual adjust- nudge (N) 6.19 1.8 4.39
ment
chasm (N) 5.84 1.45 4.39
We carried out a post hoc analysis of the contextal Table 4. Top residuals: Brysbaert et al.
adjustment coefficient (c) by using grid search to (BRY) vs. KonKretiKa (KKK).
maximize the correlation between KonKretiKa
(Type 2 transformation) and the gold standard. (BRY vs. Gold) is rs = .755, r = .761, which is
Moreover, we altered the scope of the context close to the correlation between KKK and BRY.
words for which the mean sentential concreteness We undertook closer pairwise comparative
(M) was computed – by taking 2-3 nearest seman- analysis between two pairs of rankings:
tic neighbors (either of any part of speech, or only
nouns, or only verbs); this was done in order to 1. Static KonKretiKa indexes (the indexes af-
reduce the possible noise from the words that are ter Type 2 sigmoid transformation, without
not semantically related to the target in the sen- contextual adjustment) vs. the Brysba-
tence. The change of the contextual scope did not ert et al. ranking (which is also static): ap-
lead to a substantial difference in the result. As for proximately 23,000 words – nouns, verbs,
the contextual adjustment coefficient, the grid adjectives, and adverbs (the two wordlists
search showed that c = 0.32 – which is lower than are identical).
the most efficient coefficient from our earlier sub- 2. Indexes of the target words from the CON-
missions (c = 0.5 in KonKretiKa-3) – results in a creTEXT test data as presented in the dy-
slight increase of correlations: rs = .678 and namic version of KonKretiKa (the sig-
r = .688. moid-transformed Type 2 indexes with
A closer analysis of the test sentences suggests contextual adjustment coefficient c = 0.32)
that contribution of contextual adjustment pre- vs. the Gold standard (where the target
sumably may be increased by considering a words are also ranked dynamically in con-
broader context of a sentence – for instance, span- text): 436 words – verbs and nouns.
ning over 1-3 adjacent sentences from the left and The top residuals between the KonKretiKa and
the right contexts; this option constitutes a possi- the Brysbaert et al. indexes are presented in Ta-
ble direction for future work. ble 4. Analysis of these discrepancies suggests
3.1 Comparison of computational and psy- that most of them stem from polysemy and the
cholinguistic data differences between its representation in distribu-
tional semantic models and in psycholinguistic re-
Pairwise correlations between the computational ality. Thus, distributional semantic models do not
(KonKretiKa, KKK) and the psycholinguistic discriminate between various meanings of words;
rankings (Brysbaert et al., BRY and the gold if occurrences of one of the meanings substan-
standard) are shown in Table 3. It can be seen that tially outnumber the other meanings in discourse
KKK better correlates with the BRY data than and, as a consequence, in the training corpus, the
with the gold standard (rs = .743, r = .751 vs. resulting vector reflects the more frequent mean-
rs = .663, r = .669, respectively). Presumably, ing.
such difference in the two correlations is due to
the much larger size of the BRY lexicon. The cor-
relation between the two psycholinguistic datasets
Sentence
Target
KKK
word
Gold
Diff
TEXT
399 vision (N) 6.03 1.86 4.17 Check your < vision > to see if you are seeing blurry or double.
With retinal migraine, you may experience loss of < vision > in one
353 vision (N) 5.97 1.82 4.15
eye and a headache that starts behind your eyes.
155 spirit (N) 6 2.33 3.67 Gin is an alcoholic < spirit > made from distilled grain or malt.
324 pain (N) 5.2 1.59 3.61 See your doctor if you are experiencing < pain > or discomfort.
61 answer (N) 5.45 1.91 3.54 Be sure to write your final < answer > without the negative sign.
385 war (N) 5.57 2.06 3.51 They have escaped from civil < war > in Liberia or Zimbabwe.
Final < answers > for equations are considered wrong unless you
81 answer (N) 5.32 1.92 3.4
have broken them down to their simplest form.
237 heart (N) 6.32 2.98 3.34 The < heart > pumps blood due to an internal electrical system.
163 pain (N) 4.97 1.63 3.34 Take your medications to ease your physical < pain >.
After signing the indemnification < agreement >, you can sign the le-
176 agreement (N) 5.16 1.85 3.31
gally binding bond agreement.
Table 5. Top residuals: KonKretiKa (KKK) vs. Gold standard.
For example, the nearest semantic neighbors of tic data. Meanwhile, the nearest semantic neigh-
the noun handmaiden in the distributional seman- bors of temptingly in the distributional semantic
tic model3 are: embodiment, personification, epit- model are: strappy sandal, capelet, knee-length
ome, and paragon – associating this word with its skirt, enticingly, floral-print, high-heeled sandal,
abstract, metaphoric meaning ‘something that lace-trimmed, harem pants, and puffed sleeve – all
supports something else that is more important’4, rather concrete objects (or the properties of such
whereas for speakers of English the other, con- objects).
crete meaning ‘a woman who is someone’s serv- The top residuals between KonKretiKa and the
ant’ apparently stands out as being more salient. gold standard are shown in Table 5. The discrep-
Similarly, among the nearest semantic neighbors ancy between the abstract meaning of vision (‘the
of the noun chasm in the distributional semantic ability to think about and plan for the future, using
model are: disparity, schism, rich-poor divide, intelligence and imagination, especially in politics
mistrust, (the) haves, divergence, antagonism, and and business’) and its concrete meaning (‘the abil-
inequality – indicating that the distributional vec- ity to see’) can also be attributed to the differences
tor of chasm is biased towards the abstract mean- between representation of meanings in distribu-
ing of this word (‘a very big difference that sepa- tional semantic models and in psycholinguistic re-
rates one person or group from another’) rather ality – the reason already discussed above. Thus,
than the concrete one (‘a very deep crack in rock the nearest distributional semantic neighbors of
or ice’), while human subjects see the latter mean- vision are: worldview, ideal, visionary, thinking,
ing as more salient or prevalent. perspective, idea, dream, and blueprint – rather
As for nonreactive and temptingly, which are than terms related to eyesight.
more concrete in the computational data, this The noun spirit in Table 5 (Sentence 155) is
could be explained by their perceived vagueness used in the sense of ‘strong alcoholic drink’. How-
to human subjects, since these words do not have ever, its nearest neighbors in the distributional se-
meanings that would be markedly juxtaposed to mantic model are ethos, ideal, idealism, tradition,
each other in terms of concreteness-abstractness – essence, enthusiasm, passion, faith, chivalric,
thus ranking them rather low in the psycholinguis- zeal, credo, and compassion – indicating that the
meaning ‘your attitude to life or to other people’
3 4
Continuous Skip-Gram model (Kutuzov et al., Definitions are cited according to Macmillan Dic-
2017), pre-trained on Gigaword 5th Edition corpus tionary (n.d.)
is dominant in the model, and the contextual ad- contradiction revealed that it stems from the vul-
justment we apply is not sufficient for overcoming nerability in the semantic composition of the con-
the abstractness of the dominant meaning. crete paradigm which was used to compute the
As for the noun war, its nearest neighbors in the raw indexes (see Table 1). The words of this par-
distributional semantic model are conflict, war- adigm belong to the two major semantic classes –
fare, invasion, 1991-95 Serbo-Croatian, Israel- living organisms (animals and plants) and man-
Hezbollah, genocide, Bosnia war, Jehad, civil- made artifacts. The class of words denoting hu-
war, Croatia war, Cold War, Iran-Iraq, wartime, man beings was intentionally excluded when the
Vietnam-like, etc. – that is, rather abstract con- paradigm was compiled on the grounds that such
cepts. The only more concrete words referring to nouns tend to indicate abstract social roles rather
physical combat action that occur in the distribu- than physical humans. As a consequence, physical
tional semantic neighborhood of war are battle- organic objects such as body parts and organs, or
field and bloodshed, but this is not enough to out- physical sensations and physiological conditions
weigh the abstract terms. Thus, the distributional received non-uniform indexes in KonKretiKa:
semantic model models warfare in terms of ab- those that refer to humans as well as to animals
stract rather than concrete (such as names of (e.g. in veterinary or gastronomic discourse)
weapons, military equipment, military personnel, ranked rather high in concreteness: e.g. liver (6.6),
etc.) concepts. As a result, military action is not pancreas (6.4), foot (6.3), encephalitis (6.25),
sufficiently juxtaposed to the metaphoric meaning kidney (6.25), entrails (6.05), tummy (5.92),
of war as ‘a situation in which two people or womb (5.6) – whereas those that tend to be pri-
groups of people fight, argue, or are extremely un- marily associated with humans received lower in-
pleasant to each other’. dexes, e.g. heart (2.63), heartburn (2.57),
In the case of answer and agreement, their near- scar (2.53), nausea (2.5), headache (1.61), dis-
est distributional semantic neighbors in the model tress (1.5), pain (1.21), queasiness (1.12), etc.
are fairly abstract concepts: explanation, answer, Thus, comparison of the KonKretiKa computa-
reply, solution, unanswerable, query, TV-talkback tional indexes with the psycholinguistic data of
answer, question, and yes (for answer), and ac- CONcreTEXT allowed us to detect a potential
cord, pact, deal, treaty, initial, negotiation, mem- shortcoming in our approach to the design of the
orandum, compromise, and negotiate (for agree- concrete paradigm. As was noted in previous
ment). Meanwhile, human subjects rank answer study (Badryzlova, 2020), the class of concrete
and agreement rather high in concreteness; pre- words seems to be more semantically heterogene-
sumably, this is a consequence of conflating the ous than of abstract words; therefore, it may rea-
mental representations of the action of answer- sonable in future experiments to diversify the con-
ing / reaching an agreement with their two modes crete paradigm and expand it in size by including
– the spoken and the written, i.e. with the physical words that denote human beings.
actions of speaking and writing. This conflation is
not reflected in discourse – it largely exists in the 4 Conclusions
mental representations of answer and agreement
We presented KonKretiKa system for computing
and, therefore, is not very distinguishable on the
concreteness indexes of English words in context;
level of linguistic representation.
the system was submitted to the English track of
Of interest are the cases of heart and pain,
the CONcreTEXT shared task. The best modifica-
which have much lower concreteness in
tion of KonKretiKa ranked third in the task, with
KonKretiKa than in the gold standard sentences
rs = .6634 and r = .6685 against the gold standard.
where these words are used in their physical, con-
We treat concreteness as a bimodal problem and
crete meanings. The nearest distributional seman-
use paradigm lists of concrete and abstract words
tic neighbors of heart are heart-related, coronary
to compute two indexes for each word, that of
artery, kidney, liver, lung, arrhythmia, cardiac,
concreteness and of abstractness. The single ag-
angina, and aneurism. The nearest neighbors of
gregate index indicative of both the word’s con-
pain are discomfort, ache, agony, tingling sensa-
creteness and abstractness is computed as the
tion, numbness, soreness, menstrual cramp, light-
function of the two respective indexes. The set of
headedness, stiffness, nausea, and arthritis. It
raw aggregate indexes is transformed using sig-
would be quite expected for such semantic neigh-
moid transformation to increase the variance and
borhood to entitle heart and pain to higher con-
to attain greater similarity to the psycholinguistic
creteness values than what they receive in
KonKretiKa. A more in-depth analysis into this
data. To dynamically adjust the concreteness in- Coltheart, M., 1981. The MRC psycholinguistic da-
dexes to the context, we apply an adjustment co- tabase. The Quarterly Journal of Experi-
efficient. Post hoc analysis of the adjustment co- mental Psychology Section A 33, 497–505.
efficient indicates that lower coefficients lead to Fellbaum, C., 1998. WordNet: An electronic data-
better performance. We hypothesize that the con- base. MIT Press, Cambridge, MA.
tribution of the adjustment coefficient could be in- Gregori, L., Montefinese, M., Radicioni, D.P., Rav-
creased by expanding the scope of the context, for elli, A.A., Varvara, R., 2020. CONcreTEXT
example, by considering one or more sentences @ Evalita2020: the Concreteness in Context
from the left and the right contexts of the target Task, in: Basile, V., Croce, D., Di Maro, M.,
sentence. According to our analysis, the main Passaro, L.C. (Eds.), Proceedings of the 7th
source of divergence between the computational Evaluation Campaign of Natural Language
and the psycholinguistic indexes lies in the differ- Processing and Speech Tools for Italian
ent representation, or salience, of word meanings (EVALITA 2020). CEUR.org, Online.
in distributional semantic models and in psycho- Kulagin, D., 2019. Opy`t sozdaniya mashinno-
linguistic reality. Besides, analysis of divergences proveryaemoj semanticheskoj razmetki russ-
between the computational and the psycholinguis- kix sushhestvitel`ny`x [Developing computa-
tic rankings prompted us a potential direction for tionally verifiable semantic annotation of
Russian nouns]. Presented at the Annual In-
reducing the bias in composition of the concrete-
ternational Conference “Dialogue,” Moscow.
ness paradigm, which can be overcome by diver-
sifying the paradigm. Kutuzov, A., Fares, M., Oepen, S., Velldal, E.,
2017. Word vectors, reuse, and replicability:
Towards a community repository of large-text
References resources, in: Proceedings of the 58th Confer-
Badryzlova, Y., 2020. Exploring Semantic Con- ence on Simulation and Modelling. Linköping
creteness and Abstractness for Metaphor University Electronic Press, pp. 271–276.
Identification and Beyond. Computational Lakoff, G., Johnson, M., 1980. Metaphors we Live
Linguistics and Intellectual Technologies 33– by, 2nd ed. The University of Chicago Press,
47. Chicago-London.
Badryzlova, Y., 2019. Automated metaphor identi- Macmillan Dictionary, Free English Dictionary and
fication in Russian texts. National Research Thesaurus [WWW Document], n.d. URL
University Higher School of Economics, https://www.macmillandictionary.com/ (ac-
Moscow. cessed 11.7.20).
Badryzlova, Y., Panicheva, P., 2018. A Multi-fea- Parker, R., Graff, D., Kong, J., Chen, K., Maeda,
ture Classifier for Verbal Metaphor Identifi- K., 2011. English Gigaword Fifth Edition
cation in Russian Texts, in: Conference on Ar- LDC2011T07 (Tech. Rep.). Technical Re-
tificial Intelligence and Natural Language. port. Linguistic Data Consortium, Philadel-
Springer, pp. 23–34. phia.
Barsalou, L.W., 2008. Grounded cognition. Annu. Pedregosa, F., Varoquaux, G., Gramfort, A.,
Rev. Psychol. 59, 617–645. Michel, V., Thirion, B., Grisel, O., Blondel,
Basile, V., Croce, D., Di Maro, M., Passaro, L.C., M., Prettenhofer, P., Weiss, R., Dubourg, V.,
2020. EVALITA 2020: Overview of the 7th 2011. Scikit-learn: Machine Learning in Py-
Evaluation Campaign of Natural Language thon Journal of Machine Learning Research.
Processing and Speech Tools for Italian, in: Tsvetkov, Y., Mukomel, E., Gershman, A., 2013.
Basile, V., Croce, D., Di Maro, M., Passaro, Cross-lingual metaphor detection using com-
L.C. (Eds.), Proceedings of Seventh Evalua- mon semantic features, in: Proceedings of the
tion Campaign of Natural Language Pro- First Workshop on Metaphor in NLP. pp. 45–
cessing and Speech Tools for Italian. Final 51.
Workshop (EVALITA 2020). CEUR.org,
Online. Turney, P.D., Neuman, Y., Assaf, D., Cohen, Y.,
2011. Literal and metaphorical sense identifi-
Birke, J., Sarkar, A., 2006. A Clustering Approach cation through concrete and abstract context,
for Nearly Unsupervised Recognition of Non- in: Proceedings of the Conference on Empiri-
literal Language., in: EACL. cal Methods in Natural Language Processing.
Brysbaert, M., Warriner, A.B., Kuperman, V., Association for Computational Linguistics,
2014. Concreteness ratings for 40 thousand pp. 680–690.
generally known English word lemmas. Be-
havior research methods 46, 904–911.