In God we trust. All others must bring data. — W. Edwards Deming
               Using word embeddings to recognize idioms
                          Jing Peng, Anna Feldman
                        Department of Computer Science
                           Department of Linguistics
                           Montclair State University
                                     USA
         pengj@mail.montclair.edu, feldmana@mail.montclair.edu

                     Abstract                                  Idioms are conventiolized expressions whose
                                                            figurative meanings cannot be derived from literal
    Expressions, such as add fuel to the fire,              meaning of the phrase. There is no single agreed-
    can be interpreted literally or idiomatically           upon definition of idioms that covers all members
    depending on the context they occur in.                 of this class (Glucksberg, 1993; Cacciari, 1993;
    Many Natural Language Processing appli-                 Nunberg et al., 1994; Sag et al., 2002; Villavicen-
    cations could improve their performance if              cio et al., 2004; Fellbaum et al., 2006). At the
    idiom recognition were improved. Our ap-                same time, idioms do not form a homogeneous
    proach is based on the idea that idioms vi-             class that can be easily defined. Some examples
    olate cohesive ties in local contexts, while            of idioms are I’ll eat my hat (I’m confident), Cut
    literal expressions do not. We propose                  it out (Stop talking/doing something), a blessing
    two approaches: 1) Compute inner prod-                  in disguise (some bad luck or misfortune results
    uct of context word vectors with the vec-               in something positive), kick the bucket (die), ring
    tor representing a target expression. Since             a bell (sound familiar), keep your chin up (remain
    literal vectors predict well local contexts,            cheerful), piece of cake (easy task), miss the boat
    their inner product with contexts should be             (miss out on something), (to be) on the ball (be at-
    larger than idiomatic ones, thereby telling             tentive/competent), put one’s foot in one’s mouth
    apart literals from idioms; and (2) Com-                (say something one regrets), rake someone over
    pute literal and idiomatic scatter (covari-             the coals (to reprimand someone severely), under
    ance) matrices from local contexts in word              the weather (sick), a hot potato (controversial is-
    vector space. Since the scatter matri-                  sue), an arm and a leg (expensive), at the drop of a
    ces represent context distributions, we can             hat (without any hesitation), barking up the wrong
    then measure the difference between the                 tree (looking in the wrong place), beat around the
    distributions using the Frobenius norm.                 bush (avoiding main topic).
    For comparison, we implement Fazly et al.                  It turns out that expressions are often ambigu-
    (2009)’s, Sporleder and Li (2009)’s, and                ous between an idiomatic and a literal interpreta-
    Li and Sporleder (2010b)’s methods and                  tion, as one can see in the examples below 1 :
    apply them to our data. We provide ex-                     (A) After the last page was sent to the printer,
    perimental results validating the proposed              an editor would ring a bell, walk toward the door,
    techniques.                                             and holler ” Good night! ” (Literal) (B) His
                                                            name never fails to ring a bell among local voters.
1   Introduction                                            Nearly 40 years ago, Carthan was elected mayor
Natural language is filled with emotion and im-             of Tchula. . . (Idiomatic)
plied intent, which are often not trivial to detect.           (C) . . . that caused the reactor to literally blow
One specific challenge are idioms. Figurative lan-          its top. About 50 tons of nuclear fuel evapo-
guage draws off of prior references and is unique           rated in the explosion. . . (Literal) (D) . . . He didn’t
to each culture and sometimes what we don’t say             pound the table, he didn’t blow his top. He always
is even more important than what we do. This,               kept his composure. (Idiomatic)
naturally, presents a significant problem for many             1
                                                                These examples are extracted from the Corpus of Con-
Natural Language Processing (NLP) applications              temporary American English (COCA) (http://corpus.
as well as for big data analytics.                          byu.edu/coca/




                                                       96
   (E) . . . coming out of the fourth turn, slid down          cur are more affective. Still, they treat idioms as
the track, hit the inside wall and then hit the atten-         semantic outliers.
uator at the start of pit road. (Literal) (F) . . . job
training, research and more have hit a Republican              3     Our Approach
wall. (Idiomatic)
   Fazly et al. (2009)’s analysis of 60 idioms from            We hypothesize that words in a given text seg-
the British National Corpus (BNC) has shown                    ment that are representatives of the local context
that close to half of these also have a clear lit-             are likely to associate strongly with a literal ex-
eral meaning; and of those with a literal mean-                pression in the segment, in terms of projection (or
ing, on average around 40% of their usages are                 inner product) of word vectors onto the vector rep-
literal. Therefore, idioms present great challenges            resenting the literal expression. We also hypoth-
for many Natural Language Processing (NLP) ap-                 esize that the context word distribution for a lit-
plications. Most current translation systems rely              eral expression in word vector space will be dif-
on large repositories of idioms. Unfortunately,                ferent from the distribution for an idiomatic one.
more frequently than not, MT systems are not able              This hypothesis also underlies the distributional
to translate idiomatic expressions correctly.                  approach to meaning (Firth, 1957; Katz and Gies-
   In this paper we describe an algorithm for auto-            brecht, 2006).
matic classification of idiomatic and literal expres-
sions. Similarly to Peng et al. (2014), we treat id-           3.1    Projection Based On Local Context
ioms as semantic outliers. Our assumption is that                     Representation
the context word distribution for a literal expres-
                                                               The local context of a literal target verb-noun con-
sion will be different from the distribution for an
                                                               struction (VNC) must be different from that of an
idiomatic one. We capture the distribution in terms
                                                               idiomatic one. We propose to exploit recent ad-
of covariance matrix in vector space.
                                                               vances in vector space representation to capture
                                                               the difference between local contexts (Mikolov et
2   Previous Work
                                                               al., 2013a; Mikolov et al., 2013b).
Previous approaches to idiom detection can be                     A word can be represented by a vector of fixed
classified into two groups: 1) type-based extrac-              dimensionality q that best predicts its surrounding
tion, i.e., detecting idioms at the type level; 2)             words in a sentence or a document (Mikolov et al.,
token-based detection, i.e., detecting idioms in               2013a; Mikolov et al., 2013b). Given such a vector
context. Type-based extraction is based on the                 representation, our first proposal is the following.
idea that idiomatic expressions exhibit certain lin-           Let v and n be the vectors corresponding to the
guistic properties such as non-compositionality                verb and noun in a target verb-noun construction,
that can distinguish them from literal expressions             as in blow whistle, where v 2 <q represents blow
(Sag et al., 2002; Fazly et al., 2009). While                  and n 2 <q represents whistle. Let vn = v + n 2
many idioms do have these properties, many id-                 <q . Thus, vn is the word vector that represents
ioms fall on the continuum from being composi-                 the composition of verb v and noun n, and in our
tional to being partly unanalyzable to completely              example, the composition of blow and whistle. As
non-compositional (Cook et al., 2007). Katz and                indicated in Mikolov et al. (2013b), word vectors
Giesbrecht (2006), Birke and Sarkar (2006), Fa-                obtained from deep learning neural net models ex-
zly et al. (2009), Li and Sporleder (2009), Li and             hibit linguistic regularities, such as additive com-
Sporleder (2010a), Sporleder and Li (2009), and                positionality. Therefore, vn is justified to pre-
Li and Sporleder (2010b), among others, notice                 dict surrounding words of the composition of, say,
that type-based approaches do not work on expres-              blow and whistle. Our hypothesis is that on av-
sions that can be interpreted idiomatically or lit-            erage, inner product blowwhistle · v, where vs are
erally depending on the context and thus, an ap-               context words in a literal usage, should be greater
proach that considers tokens in context is more                than blowwhistle · v, where vs are context words in
appropriate for idiom recognition.To address these             an idiomatic usage.
problems, Peng et al. (2014) investigate the bag of               For a given vocabulary of m words, represented
words topic representation and incorporate an ad-              by matrix V = [v1 , v2 , · · · , vm ] 2 <q⇥m , we cal-
ditional hypothesis–contexts in which idioms oc-               culate the projection of each word vi in the vocab-



                                                          97
ulary onto    vn                                                  represented by a vectors (Mikolov et al., 2013a;
                                                                  Mikolov et al., 2013b). Assuming wi s have been
                      P = V t vn                      (1)         centered, we compute the scatter matrix
where P 2 <m , and t represents transpose. Here                                         ⌃ = dt d,                   (4)
we assume that vn is normalized to have unit
                                                                  where ⌃ represents the local context distribution
length. Thus, Pi = vit vn indicates how strongly
                                                                  for a given target VNC.
word vector vi is associated with vn . This pro-
                                                                     Given two distributions represented by two scat-
jection, or inner product, forms the basis for our
                                                                  ter matrices ⌃1 and ⌃2 , a number of measures
proposed technique.
                                                                  can be used to compute the distance between ⌃1
   Let D = {d1 , d2 , · · · , dl } be a set of l text seg-
                                                                  and ⌃2 , such as Choernoff and Bhattacharyya dis-
ments (local contexts), each containing a target
                                                                  tances (Fukunaga, 1990). Both measures require
VNC (i.e., vn ). Instead of generating a term by
                                                                  the knowledge of matrix determinant. In our case,
document matrix, where each term is tf-idf (prod-
                                                                  this can be problematic, because ⌃ (4) is most
uct of term frequency and inverse document fre-
                                                                  likely to be singular, which would result in a de-
quency), we compute a term by document matrix
                                                                  terminant to be zero.
MD 2 <m⇥l , where each term in the matrix is
                                                                     We propose to measure the difference between
                         p · idf,                     (2)         ⌃1 and ⌃2 using matrix norms. We have experi-
                                                                  mented with the Frobenius norm and the spectral
the product of the projection of a word onto a tar-               norm. The Frobenius norm evaluates the differ-
get VNC and inverse document frequency. That                      ence between ⌃1 and ⌃2 when they act on a stan-
is, the term frequency (tf) of a word is replaced                 dard basis. The spectral norm, on the other hand,
by the projection (inner product) of the word onto                evaluates the difference when they act on the di-
  vn (1). Note that if segment dj does not contain                rection of maximal variance over the whole space.
word vi , MD (i, j) = 0, which is similar to tf-idf
estimation. The motivation is that topical words                  4     Experiments
are more likely to be well predicted by a literal                 We have carried out an empirical study evaluating
VNC than by an idiomatic one. The assumption is                   the performance of the proposed techniques. The
that a word vector is learned in such a way that it               goal is to predict the idiomatic usage of VNCs.
best predicts its surrounding words in a sentence
or a document (Mikolov et al., 2013a; Mikolov                     4.1    Methods
et al., 2013b). As a result, the words associated                 For comparison, the following methods are evalu-
with a literal target will have larger projection onto            ated.
a target vn . On the other hand, the projections
                                                                      1. tf · idf : compute term by document matrix
of words associated with an idiomatic target VNC
                                                                         from training data with tf · idf weighting.
onto vn should have a smaller value.
   We also propose a variant of p · idf representa-                   2. p · idf : compute term by document ma-
tion. In this representation, each term is a product                     trix from training data with proposed p · idf
of p and typical tf-idf. That is,                                        weighting (2).

                      p · tf · idf.                   (3)             3. p · tf · idf : compute term by document ma-
                                                                         trix from training data with proposed p*tf-idf
3.2 Local Context Distributions                                          weighting (3).
Our second hypothesis states that words in a local
                                                                      4. CoVARFro : proposed technique (4) de-
context of a literal expression will have a differ-
                                                                         scribed in Section 3.2, the distance between
ent distribution from those in the context of an id-
                                                                         two matrices is computed using Frobenius
iomatic one. We propose to capture local context
                                                                         norm.
distributions in terms of scatter matrices in a space
spanned by word vectors (Mikolov et al., 2013a;                       5. CoVARSp : proposed technique similar to
Mikolov et al., 2013b).                                                  CoVARFro . However, the distance between
   Let d = (w1 , w2 · · · , wk ) 2 <q⇥k be a seg-                        two matrices is determined using the spectral
ment (document) of k words, where wi 2 <q are                            norm.



                                                             98
  6. Context+ (CTX+): supervised version of the              and labeled as L (Literal), I (Idioms), or Q (Un-
     CONTEXT technique described in Fazly et                 known). The list contains only those VNCs whose
     al. (2009) (see below).                                 frequency was greater than 20 and that occurred
                                                             at least in one of two idiom dictionaries (Cowie
   For methods from 1 to 3, we compute a latent
                                                             et al., 1983; Seaton and Macaulay, 2002). The
space from a term by document matrix obtained
                                                             dataset consists of 2,984 VNC tokens. For our ex-
from the training data that captures 80% variance.
                                                             periments we only use VNCs that are annotated as
To classify a test example, we compute cosine
                                                             I or L. We only experimented with idioms that can
similarity between the test example and the train-
                                                             have both literal and idiomatic interpretations. We
ing data in the latent space to make a decision.
                                                             should mention that our approach can be applied
   For methods 4 and 5, we compute literal and id-
                                                             to any syntactic construction. We decided to use
iomatic scatter matrices from training data (4). For
                                                             VNCs only because this dataset was available and
a test example, compute a scatter matrix according
                                                             for fair comparison – most work on idiom recog-
to (4), and calculate the distance between the test
                                                             nition relies on this dataset.
scatter matrix and training scatter matrices using
                                                                We use the original SGML annotation to extract
the Frobenius norm for method 4, and the spectral
                                                             paragraphs from BNC. Each document contains
norm for method 5.
                                                             three paragraphs: a paragraph with a target VNC,
   Method 6 corresponds to a supervised version
                                                             the preceding paragraph and following one.
of CONTEXT described in Fazly et al. (2009).
                                                                Since BNC did not contain enough examples,
CONTEXT is unsupervised because it does not
                                                             we extracted additional ones from COCA, COHA
rely on manually annotated training data, rather
                                                             and GloWbE (http://corpus.byu.edu/). Two hu-
it uses knowledge about automatically acquired
                                                             man annotators labeled this new dataset for idioms
canonical forms (C-forms). C-forms are fixed
                                                             and literals. The inter-annotator agreement was
forms corresponding to the syntactic patterns in
                                                             relatively low (Cohen’s kappa = .58); therefore,
which the idiom normally occurs. Thus, the gold-
                                                             we merged the results keeping only those entries
standard is “noisy” in CONTEXT. Here we pro-
                                                             on which the two annotators agreed.
vide manually annotated training data. That is, the
gold-standard is “clean.” Therefore, CONTEXT+                4.3   Word Vectors
is a supervised version of CONTEXT. We imple-
                                                             For our experiments reported here, we obtained
mented this approach from scratch since we had
                                                             word vectors using the word2vec tool (Mikolov
no access to the code and the tools used in the orig-
                                                             et al., 2013a; Mikolov et al., 2013b) and the
inal article and applied this method to our dataset
                                                             text8 corpus. The text8 corpus has more than
and the performance results are reported in Table
                                                             17 million words, which can be obtained from
2.
                                                             mattmahoney.net/dc/text8.zip. The
                                                             resulting vocabulary has 71,290 words, each of
   Table 1: Datasets: Is = idioms; Ls = literals             which is represented by a q = 200 dimension vec-
      Expression     Train          Test                     tor. Thus, this 200 dimensional vector space pro-
      BlowWhistle    20 Is, 20 Ls   7 Is, 31 Ls
      LoseHead       15 Is, 15 Ls   6 Is, 4 Ls               vides a basis for our experiments.
      MakeScene      15 Is, 15 Ls   15 Is, 5 Ls
      TakeHeart      15 Is, 15 Ls   46 Is, 5 Ls              4.4   Datasets
      BlowTop        20 Is, 20 Ls   8 Is, 13 Ls
      BlowTrumpet    50 Is, 50 Ls   61 Is, 186 Ls            Table 1 describes the datasets we used to evaluate
      GiveSack       20 Is, 20 Ls   26 Is, 36 Ls             the performance of the proposed technique. All
      HaveWord       30 Is, 30 Ls   37 Is, 40 Ls             these verb-noun constructions are ambiguous be-
      HitRoof        50 Is, 50 Ls   42 is, 68 Ls
      HitWall        90 Is, 90 Ls   87 is, 154 Ls            tween literal and idiomatic interpretations. The
      HoldFire       20 Is, 20 Ls   98 Is, 6 Ls              examples below (from the corpora we used) show
      HoldHorse      80 Is, 80 Ls   162 Is, 79 Ls            how these expressions can be used literally.
                                                             BlowWhistle: we can immediately turn towards a
                                                             high-pitched sound such as whistle being blown.
4.2 Data Preprocessing                                       The ability to accurately locate a noise · · · Lose-
We use BNC (Burnard, 2000) and a list of verb-               Head: This looks as eye-like to the predator as
noun constructions (VNCs) extracted from BNC                 the real eye and gives the prey a fifty-fifty chance
by Fazly et al. (2009) and Cook et al. (2008)                of losing its head. That was a very nice bull I shot,



                                                        99
            Table 2: Average precision, recall, and accuracy by each method on 12 datasets.
        Method           BlowWhistle          LoseHead            MakeScene             TakeHeart
                       Pre Rec Acc         Pre Rec Acc         Pre Rec Acc           Pre Rec Acc
        tf · idf       0.23 0.75 0.42      0.27 0.21 0.49      0.41 0.13 0.33        0.65 0.02 0.11
        p · idf        0.29 0.82 0.60      0.49 0.27 0.48      0.82 0.48 0.53        0.90 0.43 0.44
        p · tf · idf   0.23 0.99 0.37      0.31 0.30 0.49      0.40 0.11 0.33        0.78 0.11 0.18
        CoVARFro       0.65 0.71 0.87      0.60 0.78 0.58      0.84 0.83 0.75        0.95 0.61 0.62
        CoVARsp        0.44 0.77 0.77      0.62 0.81 0.61      0.80 0.82 0.72        0.94 0.55 0.56
        CT X+          0.17 0.56 0.40      0.55 0.52 0.46      0.78 0.0.37 0.45      0.92 0.66 0.64
                           BlowTop          BlowTrumpet            GiveSack             HaveWord
                       Pre Rec Acc         Pre Rec Acc         Pre Rec Acc           Pre Rec Acc
        tf · idf       0.55 0.93 0.65      0.26 0.85 0.36      0.61 0.63 0.55        0.52 0.33 0.52
        p · idf        0.59 0.58 0.68      0.44 0.85 0.69      0.55 0.47 0.62        0.52 0.53 0.54
        p · tf · idf   0.54 0.53 0.65      0.33 0.93 0.51      0.54 0.64 0.55        0.53 0.53 0.53
        CoVARFro       0.81 0.87 0.86      0.45 0.94 0.70      0.63 0.88 0.72        0.58 0.49 0.58
        CoVARsp        0.71 0.79 0.79      0.39 0.89 0.62      0.66 0.75 0.73        0.56 0.53 0.58
        CT X+          0.66 0.70 0.75      0.59 0.81 0.81      0.67 0.83 0.76        0.53 0.85 0.57
                           HitRoof             HitWall             HoldFire             HoldHorse
                       Pre Rec Acc         Pre Rec Acc         Pre Rec Acc           Pre Rec Acc
        tf · idf       0.42 0.70 0.52      0.37 0.99 0.39      0.91 0.57 0.57        0.79 0.98 0.80
        p · idf        0.54 0.84 0.66      0.55 0.92 0.70      0.97 0.83 0.81        0.86 0.81 0.78
        p · tf · idf   0.41 0.98 0.45      0.39 0.97 0.43      0.95 0.89 0.85        0.84 0.97 0.86
        CoVARFro       0.61 0.88 0.74      0.59 0.94 0.74      0.97 0.86 0.84        0.86 0.97 0.87
        CoVARsp        0.54 0.85 0.66      0.50 0.95 0.64      0.96 0.87 0.84        0.77 0.85 0.73
        CT X+          0.55 0.82 0.67      0.92 0.57 0.71      0.97 0.64 0.64        0.93 0.89 0.88


but I lost his head. MakeScene: · · · in which the         two matrices act on the maximal variance direc-
many episodes of life were originally isolated and         tion, while the Frobenius norm evaluates on a stan-
there was no relationship between the parts, but           dard basis. That is, Frobenius measures the differ-
at last we must make a unified scene of our whole          ence along all basis vectors. On the other hand,
life. TakeHeart: · · · cutting off one of the forelegs     the spectral norm evaluates changes in a particular
at the shoulder so the heart can be taken out still        direction. When the difference is a result of all ba-
pumping and offered to the god on a plate. Blow-           sis directions, the Frobenius norm potentially pro-
Top: Yellowstone has no large sources of water to          vides a better measurement. The projection meth-
create the amount of steam to blow its top as in           ods (p · idf and p · tf · idf ) outperform tf · idf
previous eruptions.                                        overall but not as pronounced as CoVAR.
                                                              CT X+ demonstrates a very competitive per-
5   Results                                                formance. Since CT X+ is a supervised version
                                                           of CONTEXT, we expect our proposed algorithms
Table 2 shows the average precision, recall and ac-
                                                           to outperform Fazly’s CONTEXT method.
curacy of the competing methods on 12 datasets
over 20 runs. The best performance is in bold face.
                                                           6   Conclusions
The best model is identified by considering preci-
sion, recall, and accuracy together for each model.        In this paper we described an original algorithm
We calculate accuracy by adding true positives (id-        for automatic classification of idiomatic and lit-
ioms) and true negatives (literals) and normalizing        eral expressions. We also compared our algo-
the sum by the number of examples.                         rithm against several competing idiom detection
   Interestingly, the Frobenius norm outperforms           algorithms discussed in the literature. The per-
the spectral norm. One possible explanation is that        formance results show that our algorithm gener-
the spectral norm evaluates the difference when            ally outperforms Fazly et al. (2009)’s model. Note



                                                     100
that CT X+ is a supervised version of Fazly et al.         Afsaneh Fazly, Paul Cook, and Suzanne Stevenson.
(2009)’s, in that the training data here is the true         2009. Unsupervised Type and Token Identification
                                                             of Idiomatic Expressions. Computational Linguis-
“gold-standard,” while in (Fazly et al., 2009) is
                                                             tics, 35(1):61–103.
noisy. A research direction is to incorporate af-
fect into our model. Idioms are typically used to          Christiane Fellbaum, Alexander Geyken, Axel Herold,
imply a certain evaluation or affective stance to-           Fabian Koerner, and Gerald Neumann.         2006.
                                                             Corpus-based Studies of German Idioms and Light
ward the things they denote (Nunberg et al., 1994;           Verbs.     International Journal of Lexicography,
Sag et al., 2002). We usually do not use idioms to           19(4):349–360.
describe neutral situations, such as buying tickets
or reading a book. Similarly to Peng et al. (2014)         John R Firth. 1957. {A synopsis of linguistic theory,
                                                             1930-1955}.
we are exploring ways to incorporate affect into
our idiom detection algorithm. Even though our             K. Fukunaga. 1990. Introduction to statistical pattern
method was tested on verb-noun constructions, it             recognition. Academic Press.
is independent of syntactic structure and can be
                                                           Sam Glucksberg. 1993. Idiom Meanings and Allu-
applied to any idiom type. Unlike Fazly et al.               sional Content. In Cristina Cacciari and Patrizia Ta-
(2009)’s approach, for example, our algorithm is             bossi, editors, Idioms: Processing, Structure, and
language-independent and does not rely on POS                Interpretation, pages 3–26. Lawrence Erlbaum As-
taggers and syntactic parsers, which are often un-           sociates.
available for resource-poor languages. Our next            Graham Katz and Eugenie Giesbrecht. 2006. Au-
step is to expand this method and use it for idiom           tomatic Identification of Non-compositional Multi-
discovery. The move will imply an extra step –               word Expressions using Latent Semantic Analysis.
extracting multiword expressions first and then de-          In Proceedings of the ACL/COLING-06 Workshop
                                                             on Multiword Expressions: Identifying and Exploit-
termining their status as literal or idiomatic.              ing Underlying Properties, pages 12–19.

                                                           Linlin Li and Caroline Sporleder. 2009. A cohesion
References                                                   graph based approach for unsupervised recognition
                                                             of literal and non-literal use of multiword express-
Julia Birke and Anoop Sarkar. 2006. A clustering             sions. In Proceedings of the 2009 Workshop on
   approach to the nearly unsupervised recognition of        Graph-based Methods for Natural Language Pro-
   nonliteral language. In Proceedings of the 11th           cessing (ACL-IJCNLP), pages 75–83, Singapore.
   Conference of the European Chapter of the Asso-
   ciation for Computational Linguistics (EACL’06),        Linlin Li and Caroline Sporleder. 2010a. Linguistic
   pages 329–226, Trento, Italy.                             cues for distinguishing literal and non-literal usages.
                                                             In COLING (Posters), pages 683–691.
Lou Burnard, 2000. The British National Corpus Users
  Reference Guide. Oxford University Computing             Linlin Li and Caroline Sporleder. 2010b. Using gaus-
  Services.                                                  sian mixture models to detect figurative language in
Cristina Cacciari. 1993. The Place of Idioms in a            context. In Proceedings of NAACL/HLT 2010.
  Literal and Metaphorical World. In Cristina Cac-
  ciari and Patrizia Tabossi, editors, Idioms: Pro-        Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey
  cessing, Structure, and Interpretation, pages 27–53.       Dean. 2013a. Efficient estimation of word represen-
  Lawrence Erlbaum Associates.                               tations in vector space. In Proceedings of Workshop
                                                             at ICLR.
Paul Cook, Afsaneh Fazly, and Suzanne Stevenson.
  2007. Pulling their weight: Exploiting syntactic         Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Cor-
  forms for the automatic identification of idiomatic        rado, and Jeffrey Dean. 2013b. Distributed repre-
  expressions in context. In Proceedings of the ACL          sentations of words and phrases and their composi-
  07 Workshop on A Broader Perspective on Multi-             tionality. In Proceedings of NIPS.
  word Expressions, pages 41–48.
                                                           Geoffrey Nunberg, Ivan A. Sag, and Thomas Wasow.
Paul Cook, Afsaneh Fazly, and Suzanne Stevenson.             1994. Idioms. Language, 70(3):491–538.
  2008. The VNC-Tokens Dataset. In Proceedings
  of the LREC Workshop: Towards a Shared Task for          Jing Peng, Anna Feldman, and Ekaterina Vylomova.
  Multiword Expressions (MWE 2008), Marrakech,                2014. Classifying idiomatic and literal expres-
  Morocco, June.                                              sions using topic models and intensity of emotions.
                                                              In Proceedings of the 2014 Conference on Em-
Anthony P. Cowie, Ronald Mackin, and Isabel R. Mc-            pirical Methods in Natural Language Processing
  Caig. 1983. Oxford Dictionary of Current Idiomatic          (EMNLP), pages 2019–2027, Doha, Qatar, October.
  English, volume 2. Oxford University Press.                 Association for Computational Linguistics.




                                                     101
Ivan A. Sag, Timothy Baldwin, Francis Bond, Ann
   Copestake, and Dan Flickinger. 2002. Multiword
   expressions: A Pain in the Neck for NLP. In Pro-
   ceedings of the 3rd International Conference on In-
   telligence Text Processing and Computational Lin-
   guistics (CICLing 2002), pages 1–15, Mexico City,
   Mexico.
Maggie Seaton and Alison Macaulay, editors. 2002.
 Collins COBUILD Idioms Dictionary. Harper-
 Collins Publishers, second edition.
Caroline Sporleder and Linlin Li. 2009. Unsuper-
  vised Recognition of Literal and Non-literal Use
  of Idiomatic Expressions. In EACL ’09: Proceed-
  ings of the 12th Conference of the European Chap-
  ter of the Association for Computational Linguistics,
  pages 754–762, Morristown, NJ, USA. Association
  for Computational Linguistics.
Aline Villavicencio, Ann Copestake, Benjamin Wal-
  dron, and Fabre Lambeau. 2004. Lexical Encoding
  of MWEs. In Proceedings of the Second ACL Work-
  shop on Multiword Expressions: Integrating Pro-
  cessing, pages 80–87, Barcelona, Spain.




                                                      102