Puns, by exploiting ambiguities, are commonly used in literature to achieve a humorous or rhetorical effect. Previous approaches mainly focus on machine learning models or rule-based methods, however, they have not addressed how and why a pun is detected or located. Focusing on this, we propose a system for recognizing and locating English puns. Regarding the fact of limited training data and the aim of measuring how relevant a predictor and its direction of the association is, we compile a dataset and explore different feature sets as input for logistic regression, and measure their influence in terms of the assigned weights. To our best knowledge, our system achieves better results than state-of-the-art systems on three subtasks for different types of puns respectively.
Puns are a type of wordplay that deliberately exploits two or more different meanings of the same or similar words in a sentence. Puns utilizing the same word with ambiguous senses is known as homographic. I used to be a banker but I lost interest; in this sentence, the trigger word “interest” could mean “curiosity” and “a fixed charge for borrowing money”1. Whereas, puns using different words with similar soundings are called heterographic. Are evil wildebeests bad gnus?; here, “gnus” and “news” (/nu:z/)2 have the same pronunciation.
With such ambiguities, they can usually achieve
a humorous or rhetorical effect. Puns can be seen
not only as jokes, but also are widely used in
literature and can be traced back as early as the Roman
playwright Plautus
Puns can be a challenge to appreciate, even for
humans. It requires not only sufficient
associations but also rich English and background
knowledge. A well-functioning system in machine
translation may help non-English users better
understanding literary criticism and analysis. Besides,
it may also enhance the experience of
humancomputer interaction
This paper focuses on the detection and
location of homographic and heterographic puns.
The state-of-the-art systems mostly deployed
rulebased strategies and a few purposed complex
machine learning models. Their experimental results
reached 83 % to 90 % F1 in pun detection and
80 % in the pun location identificati
Most previous work focus on pun generation and modelling. Starting in 2004, Taylor and Mazlack (2004) used N-grams to recognize and locate wordplay. In 2015, Miller and Gurevych (2015) adapted knowledge-based Word Sense Disambiguation (WSD) to “disambiguate” different meanings of puns. By processing sentences, Kao et al. (2016) built an information-theory-based computational model for interpreted puns with “ambiguity” and “distinctiveness”.
In the pun detecti
Two works by Mao et al. (2020) and Zhou et al. (2020) were reviewed and published concurrently. Mao et al. (2020) captured long-distance and short-distance semantic relations between words; Zhou et al. (2020) combined contextualized word embedding and pronunciation embedding with a self-attentive encoder, reaching 2 %, 2 %, 13 % and 7 % increase in F-score on the four tasks respectively.
Previous studies focused on machine learning
models or rule-based methods
The influence of each feature can be traced based on the results. These terms explore the statistic characteristics of a pun as well as its semantic properties. In general, they can be categorized into the following 4 types.
statistics of the dataset, nouns, verbs, adjectives and proper nouns take up about 98 % of all pun words. Besides, a verb-type pun word is almost certain to appear at the end.
trained doc2vec
Sentence separation: Many researchers
believe that the pun word often locates in the latter
half of a sentence. H
They from
hid gunman the a
in sauna
sweat where they could it out
Pun detection is a binary classification problem with a sentence as the input, and the decision of whether it is punning as the output.
The published dataset
After removing duplicates, we found 843 puns (disregarding pun types) with a significantly larger standard deviation of sentence length compared to the given corpus.
We fitted the dataset by limiting the range of word counts and ended up with 707 puns and a variety of negative samples made of non-punning jokes, famous sayings and other short collections. The compiled dataset can be made available upon request.
Setting: In this subtask, we did experiments on two different settings: 5-fold cross-validation and purely with collected training data. For the first setting, we cross-validated with the official dataset, since it is not split by the provider. To make it comparable to the previous research, we tested all the folds independently and calculated the macro score in the end, thus the final result covers all benchmark data with 5 sub-experiments; for the second one, we trained on self-collected data and evaluated on the official dataset. Both experiment metrics use standard precision, recall, accuracy and F-score.
Features: Table 3 lists the features; among them, f4, f5, f6, f8, f9, f10 and f12 give 3Pun of the Day: http://www.punoftheday.com/ 4Both teams used 10-fold cross-validation.
5Trained on part of the dataset, evaluated on 675 of the
2250 homographic contexts according to task organizer.
Diao et al. (2019)6
Sevgili et al. (2017)
D
Description The number of words regarding its POS tags.
The distance of last appeared POS tags respectively normalized to sentence length.
Individual sums of all founded PMI values according to POS tags. doc2vec sentence representation. doc2vec dot product for separated parts. doc2vec for both parts of the sentence. doc2vec cosine similarity of word pair from separated parts of the sentence in descending order.
The first 10 values are taken.
It has three elements: if the sentence contains a similar idiomatic representation; if it differs exactly one word; how much they are in common.
Word similarity based on the shortest path in WordNet. We evaluated with path similarity7.
For each sentence, all word pairs from two subsentences are evaluated, and the 4 largest results are chosen.
The number of associated words in the first part of the sentence. For each word ! from the second part that exists in Free Association corpus8, we count how many content words from the first part are listed as associative words of word ! according to Free Association corpus.
The number of words that are predicted differently if one word ! is masked, using BERT.
Sentence representation using BERT. a positive influence on the final result, and vice versa. To unify, we choose the same feature sets for both homographic and heterographic sets.
Results: Table 1 and 2 provide the experimental results for homographic and heterographic pun detection, respectively. The 5-fold CV utilizes all data provided in the task; the latter one does not use any data from the task for training.
From both results, our system with the first setting (5-fold cross-validation) leads to the best scores, compared with all the teams which used 65-fold cross-validation on the original dataset and our compiled corpus from Pun of the Day.
7path similarity from nltk returns a score denoting how
similar two senses are, based on the shortest path that
connects the senses in the is-a (hypernym/hyponym)
taxonomy. http://www.nltk.org/howto/wordnet.html
8Free Association is a collection of word pairs that
people tend to think first when given the other word:
http://w3.usf.edu/FreeAssociation/AppendixC/
part of official data for training (all teams lis
Besides, there is a 5 % performance drop from training in the 5-fold CV to the self-collected corpus respectively. This may result from multiple reasons. For instance, the self-collected corpus does not categorize the punning type; the non-punning samples may consist of some hidden puns; the two corpora vary in terms of their properties, etc.
Ablation test: In the ablation test, we found that the major factors are sentence representation (f4, f12), the relation between both parts (f5, f6, f7 and f10) and word meaning (f8, f9). While sentence representation offers the basis, the relation between both parts also helps in general or individual word pairs.
Furthermore, since all features have different dimensions, f12 occupies more than 2/3 of the feature space, and most of the top 15 % important components are from it. The third parameter in f8 calculates the maximum ratio of overlapping words to the word length of the found idiomatic representation and always has the largest influence, then comes f4, and sometimes f6. 4.2
Pun location is to find out which word in the content is punning, given a pun-containing sentence.
Data: The dataset provided by the organizer includes 1271 homographic puns and 1780 heterographic ones.
Setting: In this subtask, we used 5-fold
crossvalidation to test. Like in Subtask 1, the official
dataset was randomly split into 5 folds. The
predictions from each fold were then accumulated
System
Zhou et al. (2020)
Mao et al. (2020)
Zou and Lu (2019)
Vecht
Features: Table 6 lists all the features used in this subtask. All of them are word-based and each assigns a vector or value to words. f2 and f4 are only for heterographic since they contain homonymic information. We then concatenate all vectors from chosen features. After logistic regression, words with the highest score presume to be pun location.
Results: Our system achieves competitive results compared to teams from the competition. For location on homographic puns, our model’s performance is lower than Zou and Lu (2019) and Cai et al. (2018), which use LSTM (see Table 4). Added pronunciation features (f2 and f4), our system (using all features except f7) exceeds the state-of-the-art results by around 5 % for heterographic puns (see Table 5).
Ablation test: Both f6 and f7 lead to a significant increment in the result. They give words, especially content words, with latter order more weight. f3 uses doc2vec to extract relations between separated parts, while f9 is to find out the “surprise” within a pun using MaskLM. Together, they contribute to around 2 % improvement. These two features answer to our hypothesis, and also tend to focus on the differences between double meanings of the trigger words and the two parts of sentences. f10 concatenates the GloVe vector to represent the word itself. It results in an approximately 7 % boost. Homonymic information (f2) helps, but still left much to explore. First, the data is heterographic instead of homophonic (e.g., “orifice” and “office”). Besides, variances of the word are not considered (e.g., “knowingly” and “no”). Third, puns may exploit names or compounds (e.g., “Clarence” and “clearance”).
This problem is remedied by adding word fre
Description Assign value 1 to the last content word in the sentence. Namely, if the feature is used, the last content word in sentences will be concatenated with vector [1], while a vector [0] for the other words.
If there is the same pronunciation of the word in CMU Pronouncing Dictionary9.
Maximum doc2vec cosine similarity of word pair from separated parts of the sentence.
The number of context words that have lower doc2vec cosine similarity with word ! than with any of !’s homonyms.
Assign value 0, 1 or 2 for word ! according to its frequency in Brown corpus10. The rarer the word is, the higher the value it is assigned.
The position of word ! in the sentence, assign 1 if it is in the second half, plus additional 1 if it also lays in the last quarter.
Mark last N, V, Adj, Propn in form of a vector at their place (from subtask 1). For example, if word ! is the last verb in the sentence, its vector for this feature should be [0,1,0,0]. doc2vec of the whole sentence (from subtask 1).
The number of context words that are predicted differently using BERT if word ! is masked.
GloVe vector of the word. quency feature (f5) instead of given special attention to particular names or structure patterns. Although this feature works significantly in heterographic, it barely influences the homographic ones. Unlike heterographic puns, one needs a word with two senses that are widely known to people in the homographic case. So a rare word can hardly be used in that situation. 5
We provide a dataset for pun detection and built a model that achieves state-of-the-art on three subtasks for different types of puns. We found that three things affect a pun: general interpretation of the content, relation for both parts and word meaning. In case we know it is punning as a prior, we can utilize e.g., word position or “surprise” according to their type, to locate the punning word.
We deployed homophones to heterographic tasks; this could be an interesting topic for future work as well as a test of higher-order associations between word pairs. Nevertheless, with the improved results of pun detection and interpretation, our system provides a step for further understand9Carnegie Mellon University (CMU) Pronouncing Dictionary is an open-source pronunciation dictionary: http://www.speech.cs.cmu.edu/cgi-bin/cmudict 10It is a general English-language corpus with a total of roughly one million words: https://archive.org/details/BrownCorpus ing and interpretation, and may be assembled into machine translation in the future.
We thank the anonymous reviewers for suggestions on the submission; the paper was partially supported by the German Academic Exchange Service and partially supported by base.camp at Universita¨t Hamburg.