This CALAMITA challenge [1] addresses the task of disambiguating pejorative language to detect forms of misogyny that are masked within ambiguous and contextdependent expressions. Pejorative language refers to a word or phrase that has negative connotations and is intended to disparage or belittle. 1 An inoffensive word becoming pejorative is a form of semantic drift known as pejoration; thus, pejorativity is context-dependent: pejorative words have one primary neutral meaning, and another negatively connotated meaning. In this challenge, our objective is to evaluate large language models (LLM) in Italian by focusing on the disambiguation of pejorative epithets used online to express misogyny. In this work, misogyny is defined as a property of social environments where women perceived as violating patriarchal norms are "kept down" through hostile or benevolent reactions coming from men, other women, and social structures [2,3], in the form of sexual objectification, male privilege, gender discrimination, sexual harassment, belittling and violence [4].
An example of a pejorative epithet is balena (whale), whose standard meaning refers to the sea mammal, but it is used offensively to address an overweight woman. Encoder-based models struggle to correctly classify misogyny when sentences contain such terms: the occurrence of polysemic words with a pejorative conno-1 https://www.merriam-webster.com/dictionary/pejorative tation in the training set and a neutral connotation in the test set results in a great number of false positives [5]. This could be overcome by decoder-based LLMs, as they could rely on their implicit knowledge to grasp the meaning of such terms. By asking models to determine whether a term is being used in a pejorative or non-pejorative sense, we challenge the LLMs' ability to comprehend semantic shifts in Italian. Moreover, asking whether a sentence containing that term is misogynous or not, enables us to comprehend to what extent LLMs understand misogyny, even when it is conveyed through figurative language. We expect models to struggle with this challenge, particularly in sentences with non-standard or regional varieties of Italian, which occur in our corpus.
We introduce pejorative language disambiguation as a preliminary step to detect misogyny. Our goal is to assess whether the disambiguation of potentially pejorative epithets improves the detection of misogynistic language. Therefore, this challenge aims to address two tasks:
Task A Disambiguation of in-context polysemic words that can be used as pejorative epithets in misogynistic language;
Task B Misogyny detection at the sentence level.
Both tasks are conceived as binary classification tasks. Fig. 1 shows the pipeline for our tasks. Assume the sentence Quella balena coi jeans non si può guardare, translated as Can't look at that whale with jeans. Task A: First, the model is asked to identify whether the meaning of the target word (balena in our example) is pejorative or not. The model should rely on its internal knowledge accumulated during pre-training to understand whether the term balena (whale) refers to woman or cetaceus. Ideally, the model should exploit the context to perform the disambiguation, as the image of a whale with jeans is not plausible. That is why we encourage commonsense reasoning for this task.
Task B: In the second step, the model is first informed with the decision of Task A, whether the target word is pejorative or not, and then asked to classify the input sentence as misogynous or not.
The compilation of our corpus involves two steps: the creation of a lexicon of polysemic words that can function as pejorative epithets for women, and the retrieval of tweets containing such words.
Lexicon. We collect our lexicon by selecting words from three distinct sources.
(1) We ask ten Italian native speakers to provide a list of offensive words used online to address women. The speakers use social media on a daily basis and their age ranges between 27 and 39 years.
(2) We retrieve the keywords used in the two Italian corpora for the Automatic Misogyny Identification (AMI) shared task [6,7].
(3) We consult the 'List of Dirty Naughty Obscene Bad Words'. 2 We only keep polysemic words whose primary meaning is neutral and that are frequently used on Twitter with both pejorative and neutral connotations. To ensure the quality of our vocabulary, we qualitatively verify that such words are used with both connotations by manually searching them on Twitter. 3 Table 1 shows our lexicon of 24 words. For each word, we report the English translation of its literal and pejorative meaning, and their anchors in Italian. Anchor 2 https://github.com/LDNOOBW/List-of-Dirty-Naughty-Obsceneand-Otherwise-Bad-Words/tree/master, consulted on January 2023. 3 Due to their exclusive neutral or negative connotation on Twitter, the following words are discarded: barile, banco, botte, barbona, facile, gatta morta, passeggiatrice, porca, principessa, privilegiata, psicopatica, scrofa, somara, travestita.
words refer to the unambiguous words used to define polysemic words. We call these words anchors because their meaning is univocal and does not change according to the context. For instance, the word balena (whale) is used to refer to either a sea mammal or an overweight woman. In contrast, the anchor words cetaceo (cetacean) and grassa (fat) only refer to the animal in the first case and to being overweight in the second case, at least as far as their use in Twitter is concerned. 4Tweets. We use Twarc5 to retrieve tweets from December 2022 to February 2023 containing words in our lexicon. We select 50 tweets for each word in our lexicon, resulting in 1,200 tweets. We maintain a balance of pejorative and neutral use of lexicon words, although an equal distribution for each word could not be guaranteed. We choose tweets as source of data for three reasons. First, Twitter is a prominent platform for expressing opinions, where language is varied, conversational, and often informal, which makes it suitable to analyze misogyny conveyed through figurative language. Second, at the time of data collection, Twitter API was public and free, which facilitated our data collection process. Third, the character limit on tweets encourages condensed language, limiting the context of expression. Choosing tweets allows us to challenge LLMs in disambiguating pejorative language for misogyny detection within the constraints of limited or lack of context.
We recruit six annotators with a background in linguistics, gender studies, cognitive sciences, and NLP to label our corpus for pejorative word disambiguation (wordlevel) and misogyny detection (sentence-level).
We first devise a pilot annotation study to explore the complexity of the task. For this purpose, we follow a descriptive annotation paradigm [8], which encourages annotator subjectivity by not providing guidelines. We split the annotators into two groups and assign 50 tweets each for labeling. Each group is composed of two women and one man with ages ranging between 27 and 39 years Italian pejorative lexicon, their literal and pejorative translations in English, and their anchors.
old. We use Krippendorff's alpha [9] to measure the interannotator agreement (IAA). The IAA of the first group is moderate for both pejorativity (0.48) and misogyny (0.50), whereas the IAA of the second group is fair for pejorativity (0.33) and moderate for misogyny (0.50). We observe that, in terms of gender differences, men tend to consider sexual objectifying compliments as non-pejorative. More details about the annotation process, including the discussion of edge cases, can be found in Muti et al. [10].
After the pilot studies, we annotate our collected corpus of 1,200 tweets. Only one person carries out the whole annotation process. We select the annotator with the most interdisciplinary background, who is an expert in gender studies, linguistics and NLP, who has been a target of misogyny. This setting is considered among the best practices for the annotation of phenomena like misogyny [11].
Data are collected in an Excel file and published at https://github.com/arimuti/PejorativITy. Each row contains the ID of the tweet, the tweet, the target word, the annotation for pejorativity at word level and the annotation for misogyny at sentence level. Table 2 shows examples.
Table 3 shows the statistics of our corpus. The Pearson correlation between misogyny and pejorativity labels is 0.70, which is in line with our expectations. The tweets for which misogyny and pejorativity are not aligned are mainly reported speech or men-related offensive language. It is worth noting that some sentences are annotated as misogynous, although they do not express any form of hate towards women. However, they contain subtle sexist language, which we consider misogynous according to the definition provided in Section 1. For instance, the sentence "che bella bambola ciao tesoro" 6does not express hate, but perpetuates the objectification of women by addressing the target of the tweet as a doll, falling into the category of benevolent sexism [12].
We design two prompts to address the two task: pejorativity disambiguation at word-level and misogyny detection at sentence-level. We adopt a zero-shot approach, although participants are encouraged to experiment with different prompting techniques. Task A. We design the following zero-shot prompt that asks whether the target {word} in a given {sentence} is pejorative or not. We prompt the LLMs for each instance in our corpus. The prompt is given as follows:
La parola {word} è utilizzata in modo peggiorativo nel testo in input? Rispondi solo con 1 per sì, 0 per no. Testo: {tweet} Task B. We design two zero-shot prompts that ask whether a tweet is misogynous or not in order to evaluate two settings:
A We treat Task A and Task B as independent, i.e., without having the decision for Task A to inform the decision for Task B. B We enrich the prompt for Task B with the output from the previous task;
Prompt A:
Il testo in input è misogino? Rispondi solo con 1 per sì, 0 per no. Testo: {tweet}
Il testo in input è misogino? Rispondi solo con 1 per sì, 0 per no. Considera l'informazione sulla parola {word}: {response from Task A} Testo: {tweet}
In order to evaluate our tasks, we employ Macro F 1 score. We select two models as baselines: the fine-tuned Al-BERTo model [13] and llama-3-8b (both the standard and
Macro F1 AlBERTo 0.82 llama-3-8b 0.00 llama-3-8b-Instruct 0.64
Results on Task A. Scores are not comparable since the testing partitions are different.
the Instruct version). The results are not comparable though, since llama is evaluated on the whole corpus, while AlBERTo on the partition of the test set (see Table 3).
Task A. Table 4 shows the results for pejorative word disambiguation. The fine-tuned AlBERTo model reaches a macro F 1 -measure of 0.82 ± 0.03, as reported in [10]. When it comes to decoder-based models, llama3-8b-Instruct shows a lower score, with a difference of 0.18 points, showing room for improvement in the prompt design. However, those scores are not comparable as the testing partitions differ. Llama-3-8b fails to complete the task, since it only repeats the prompt without providing an answer. For this reason, we discard llama-3-8b in the next task. It should be noted that llama has undergone a safety tuning process, preventing the model from always providing an answer, responding I cannot provide a response that condones hate speech. We excluded such cases from the evaluation. Of the 174 excluded instances, 123 were pejorative and 51 were not pejorative according to the gold standard. Although the fine-tuned version of AlBERTo achieves a higher performance (in a smaller subset of instances), llama aids in explainability by deliberately adding explanations of why it considers the target word to be pejorative or not. We will explore the plausibility of such explanations in future work. Task B. Table 5 shows the performance regarding misogyny detection at sentence level.
In Setting A, where the model is not informed of the output for Task A, AlBERTo scores are much lower compared to Task A, achieving 0.68 ± 0.03. Llama performs better in Task B compared to Task A, overcoming Al-BERTo by just 0.01 point. However, the fact that all answers were provided in Task B (unlike in the previous task) plays a role and does not necessarily imply that misogyny detection is an easier task than pejorativity disambiguation for llama.
In Setting B, the model is informed of the decision on pejorativity of the target word. While for llama the information about pejorativity can be injected in the prompt, with AlBERTo we have adopted two approaches: i) we concatenate the information about the pejorativity of the target word at the end of the tweet or ii) we substitute the ambiguous word with its corresponding anchor word from our lexicon. Fig. 2 shows the pipeline. We observe a notable improvement over the baseline model for concatenation (+7 absolute points) and substitution (+9 absolute points) when using the predictions for Task A.
On the other hand, llama does not benefit from the injection of knowledge about pejorative words, with a drop of 0.09 points. This could be due to the noisy response from Task A, including the refusal to answer, and possible wrong explanations of why the target word is used pejoratively or not.
We have presented a new challenge for CALAMITA: pejorative word disambiguation as a preliminary step for misogyny detection. We have designed two tasks as binary classification problems: A) pejorative language disambiguation at word level and B) misogyny detection at sentence level. Our preliminary experiments show that a Transformer-based fine-tuned model performs better than llama-3-8b-Instruct in detecting pejorative words, while llama-3-8b-Instruct performs slightly better than the Transformer-based model in misogyny detection. In the future, we plan to explore how the unrequested explanations provided by llama-3-8b-Instruct about the pejorativity of a target word impact the classification of misogynous sentences.
Although our lexicon covers a wide variety of words that can serve as pejorative epithets for women, it is not an exhaustive list, as we have discarded all the terms that are not polysemic and that are used only with one connotation (either positively or negatively) on Twitter. Moreover, only 100 tweets are annotated by six annotators, while the remaining 1,100 are labelled by only one annotator. Although we select an expert with an interdisciplinary background in linguistics, gender studies and NLP to carry out all the annotations, their personal biases, opinions, or interpretations can lead to skewed or one-sided data.
Finally, our corpus is characterized by the presence of sarcasm, abbreviations, and non-standard varieties of Italian, which might make the semantics of our instances hard to be captured by current language models.
Another limitation of our study concerns the substitution approach. First of all, some words have more than one neutral anchor words. This is the case of balena, which has two neutral anchors: balenare (to flash) and cetaceo (sea mammal). In neutral examples, we substitute balena with both anchors. This process may alter the semantic meaning of the tweet since only one anchor is suitable for substitution. Moreover, in some cases, we replace a lexicon word with anchors that do not have the same meaning. For instance, the neutral anchor of acida is aspra (sour). However, expressions like sour beer or sour cream do not have a valid anchor replacement. Therefore, replacing aspra with acida is not an appropriate substitution.
Our data collection adheres to Twitter's terms of service and privacy policies. As this research involves the analysis of publicly available tweets, we do not seek explicit consent from individual users. Nevertheless, we make every effort to protect the anonymity of all individuals mentioned. However, the exposure to misogynistic content still poses a mental health risk for researchers and annotators.
Thanks to Federico Ruggeri, Cagri Toraman, Alberto Barrón-Cedeño, Lorenzo Musetti, Samuel Algherini, Silvia Ronchi, Gianmarco Saretto, and Caterina Zapparoli for helping with the brainstorming for the task design.