Metaphor identification in text is an open problem in natural language processing. In this paper, we present a new, supervised learning approach called MIL (Metaphor Identification by Learning), for identifying three major types of metaphoric expressions without using any knowledge resources or handcrafted rules. We derive a set of statistical features from a corpus representing a given domain (e.g., news articles published by Reuters). We also use an annotated set of sentences, which contain candidate expressions labelled as 'metaphoric' or 'literal' by native English speakers. Then we induce a metaphor identification model for each expression type by applying a classification algorithm to the set of annotated expressions. The proposed approach is evaluated on a set of annotated sentences extracted from a corpus of Reuters articles. We show a significant improvement vs. a state-of-the-art learning-based algorithm and comparable results to a recently presented rule-based approach.
E-mail:
A metaphor is defined in previous works
Metaphoric expressions have a variety of syntactic structures. In this paper, we focus
on three major types of syntactic structures discussed in the work of
This paper is organized as follows. Section 2 covers the state-of-the-art approaches to automated metaphor detection. Section 3 presents MIL (Metaphor Identification by Learning), a novel algorithm for metaphor detection. The proposed algorithm is empirically evaluated in Section 4. In Section 5, we conclude the paper with some insights and directions for future research. 2
Related Work
The work
The authors of
The work of
The authors of
MIL (Metaphor Identification by Learning)
Overview The first step in our proposed methodology is choosing a domain corpus, which represents a particular domain (area) of text documents (e.g., Reuter’s news articles). The next step is the extraction of candidate expressions that satisfy one of the three syntactic structures discussed above (types 1, 2, and 3). This can be done using existing natural language processing tools. We also use the domain corpus to construct a word-context matrix to be used in the feature extraction phase. Then we manually annotate a selected subset of candidate expressions, since in a large corpus, it is not feasible to annotate them all. Each selected expression is labeled as ‘literal’ or metaphoric’ by native language speakers.
Then we perform feature extraction, the largest and most important task in our work. The goal of feature extraction is to compute statistical features that may differentiate between metaphor and literal expressions. After the feature extraction is complete, a feature selection process must be carried out for choosing the features that are most relevant to the classification task.
The next step is inducing a classification model by running a supervised learning algorithm (e.g., C4.5). The model is built separately for each syntactic type. The resulting classification model can be used for classification of new expressions as literal or metaphoric. The model performance can be evaluated by such measures as precision and recall using cross-validation. 3.2
The basic actions on a domain corpus are parsing the corpus into sentences,
tokenization, stemming the tokens by the Porter Stemmer algorithm (Porter, 1980), removing
stopwords, and then calculating the frequency of each token. Then we build a
co-occurrence matrix for words with frequency of 100 and higher
The X matrix is smoothed with truncated Singular Value Decomposition (SVD)
SVD calculation has two parameters. The first one is k, which represents the number
of latent factors. We manually calibrated k starting from the value of 1000 as used in
the work of
mantic similarity of two words as a cosine similarity of two matrix rows
Finally, we represent each of the corpus documents by calculating the average vector of the frequent words the document contains. The idea behind that is that in our view, each document represents some concept of its own and it can contribute to the identification of the concept transition. 3.3
The proposed feature set contains four types of features: features, which apply to every single word in a candidate expression (two features for type 1 and type 3 expressions, three features for type 2), features, which apply to every word pair in a candidate expression (one feature for type 1 and 3, three features for type 2), features, which apply to the sentence containing the candidate expression, and features, which apply to the candidate expression itself.
The first set of statistical-based features builds upon the idea of concrete-abstract
mapping
Abstract Scale difference. The absolute difference between the abstractness levels of every two words in an expression.
Abstract Scale Average / Variance. The average / variance of the abstractness levels of all frequent words in the sentence that contains the candidate expression. We also define a set of statistical-based features that can indicate a conceptual mapping between different word categories. These features include word-level features, document-level features, and domain-level features. Word-level features are features which are associated with the semantic meaning of a given word. Document-level features are features which are associated with the entire document’s meaning (document which contains the expression) and domain-level features are features, which are associated with the entire domain corpus. The statistical-based features are defined below: Word-level features
Semantic Relation. The semantic relation value between every two words in a given expression. This value is taken from the associated entry of this words pair in the pre-processed matA (the large NxN matrix). Low values are an indication for low semantic relation between two words and thus imply metaphoric behavior (the conceptual mapping definition) and vice versa.
Semantic Relation Average / Variance. The average / variance of the semantic relation values between every two words in the sentence that contains the candidate expression (including the expression itself). Cosine-similarity. The cosine similarity between every two words in a given expression. The cosine similarity is between the two vectors associated with the given words pair, taken from the pre-processed matB (the low dimensionality matrix). Low values indicate low conceptual relation between two words and thus imply metaphoric behavior.
Cosine-similarity Average / Variance. The average / variance of the cosine similarity between every two words in a given expression. Low values indicate low conceptual relation between two words and thus imply metaphoric behavior.
First k Words. For each two words in a given expression, we first find the k most similar words to the noun (if both words are nouns then the subject noun is selected). We then calculate the average cosine similarity of the second word to those k words and use it as a feature. A similarity between two words is computed as the cosine similarity of their associated vectors in matB. The value k = 30 was chosen based on the quality of classification results.
First k Documents. Same as First k Words, based on document vectors rather than word vectors. Considering a document as a topic/s oriented, its representation by its word vectors average is actually the semantic representation of its topic/s. Here we also set k to 30 after manual calibration considering the results quality (F-measure). Documents Jaccard Similarity. It is calculated between semantically related neighborhoods of each two words in a given expression. The word neighborhood is defined as a window of ± 5 words surrounding a given word in the same sentence. In the previous features, we detect a conceptual mapping between two words by enriching one of the words with its semantically related neighborhood. In this feature, we take this idea one-step further and enrich both words with their semantically related neighborhoods. The feature is the Jaccard similarity of these two neighborhoods, calculated as follows: . (2) Where SDT(x) is a group of documents that are related to word x according to a predefined threshold T. In our experiments, we manually set T to 0.35 after trying different values.
Domain Corpus Frequency. The normalized frequency of a word in the domain corpus. This feature is extracted for every word in the candidate expression. A low word’s frequency in a given corpus can indicate metaphoric behavior since the word is not strongly related to the given domain and thus is used there as a metaphor. Domain Corpus Frequency Difference. The absolute value of the difference between the frequencies of every two words in a given expression.
Positive Point-wise Mutual Information. This feature
We used the Wrapper method of
All algorithms were run on Weka
Experimental Results
Our results in this paper are based on the Reuters Corpus, which was previously used
as a domain corpus in
We used the same annotated corpus as in
Literal is the most direct or specific meaning of a word or expression. Metaphorical is the meaning suggested by the word that goes beyond its literal sense.
The annotators were also given examples of literal and metaphorical expressions of
types 1, 2, and 3. Inter-annotator agreement, measured in terms of Cronbach's alpha,
was 0.78, 0.80, and 0.82 for type I, II, and III, respectively
Finally, an expression was labeled as metaphoric if at least three judges out of four
considered it as such; otherwise, it was labelled as literal. Table 1 shows the
distribution of expressions by their type and label (Literal / Metaphorical) in the set of
annotated sentences.
In this section, we present the comparative results of MIL vs. two state-of-the-art
algorithms - Concrete-Abstract (Conc-Abs) of
The MIL algorithm has clearly outperformed the Concrete-Abstract algorithm in terms of F-measure with an advantage of 12.7%, 5.2%, and 42.9% for expression types Precision Recall
F-measure
Precision
Recall
F-measure
Precision
Recall
F-measure
1, 2, and 3, respectively. Moreover, all type 3 expressions were identified by the
Concrete-Abstract algorithm as literal leading to the F-measure of zero. The most likely
reason for that is that our dataset rarely contains expressions where the noun is an
abstract noun (e.g., the noun “thoughts” in the expression “dark thoughts” is an abstract
noun) and most metaphoric expressions contain concrete nouns (e.g., the noun “heart”
in the expression “broken heart” is a concrete noun). Since Conc-Abs relies on a single
feature, which is the noun’s abstractness level, it cannot detect a metaphor in these
cases. However, CCO outperformed MIL, especially in type 2 and type 3 expressions.
Unlike MIL, which is a supervised learning approach, CCO is a rule-based method,
requiring for each new language and domain a significant amount of manual expert
labor along with multiple high-quality knowledge resources, which are unavailable for
most human languages. The F-measure results reported by
Table 5 shows the list of features and the classifier selected by the Wrapper method
of
Future research on using statistical features for metaphor detection may include experimentation with additional predictive features, domains, and languages. Transfer learning across different domains may also be explored.