Introduction

Sexism in online content presents a significant challenge, considering the ease of sharing on social media and the various forms in which abusive messages can be represented (text, image, video, meme, ...). Moreover, the subjectivity of the tasks, typical for hate-related tasks, requires considering different interpretations and perspectives of the conveyed message that might be influenced by cultural differences and beliefs [1,2]. Traditional sexism detection systems usually rely on predefined labels derived from a fixed definition of sexism that only represents a single perspective and, therefore, are not able to capture the complexity and the subjectivity of the task. Although detecting sexism is crucial, it becomes even more challenging due to its subjectivity and the various forms in which such messages can be expressed, such as memes. In memes, in fact, a hateful message might be represented by the image, the text, or via their combination. Moreover, due to the presence of irony, the conveyed message might appear harmless at first glance [3,4,5,6].

An important contribution in this field that focuses on the problem of sexism identification under the paradigm of learning with disagreements in memes is represented by Task 4 at EXIST 2024: sEXism Identification in Social neTworks [7,8]. In this paper, we address the task by proposing a unified, multi-modal Transformer-based architecture capable of dealing with multiple languages (English and Spanish). The proposed approach exploits well-established, state-of-the-art multilingual models (i.e., mBERT and CLIP) and ensemble strategies.

The paper is organized as follows. An overview of the state of the art is provided in Section 2 focusing on Sexism Detection. The proposed method is described in Section 3, including both details about the shared task and dataset, and the description of the architecture of the proposed model. In Section 4, the results achieved by the proposed approaches are reported. Finally, conclusions and future research directions are summarized in Section 5.

Related Work

Social media platforms provide a fertile environment for users to share their opinions and ideologies through various forms of expression, including text, memes, and videos. Often encouraged by anonymity, elements that convey hateful messages are just as easily represented and disseminated. As a consequence, hateful messages, also linked to historical aversions (e.g., hatred towards women), have found new ways of expression, for example, in memes [9]. Likewise, the interest of researchers have as a consequence, researchers' interest has expanded to consider such representative forms.

Hateful content identification.

The research field dedicated to the identification of hateful content towards women is articulated into the identification of misogynistic or sexist content, also considering the different ways in which this type of hate can be expressed (e.g., by means of stereotypes, objectification, etc.). In particular, the majority of work in sexism/misogyny detection focuses on text (mostly Tweets) [10,11,12,13], and only in recent years it has expanded to include multimodal content, for instance, memes [14,15,16]. A first insight to counter sexist memes was proposed in [14], in which both unimodal and multimodal approaches are investigated to evaluate the contribution of textual and visual modality in sexism identification. Similarly, in [15], the authors also evaluate the information content introduced by both modalities, identifying the visual component as more informative.

As a consequence of the growing attention of researchers in the sector, challenges and benchmark datasets dedicated to this research area have been proposed. An initial dataset, proposed in [17], is composed of 800 memes gathered from the most popular social media platforms, labeled both by domain experts and by annotators from a crowdsourcing platform. A similar benchmark has been realized for the MAMI shared task at SemEval 2022 [18]. This benchmark is composed of 11k memes divided into train and test, allowing the investigation of two different tasks: (i) the identification of misogynistic memes, and (ii) the recognition of the misogynistic type among Shaming, Stereotype, Objectification, and Violence. The majority of the works in this context exploited pre-trained models-based approaches [19,20,21,22,16] and/or investigated ensemble strategies [23,24,25,26].

Learning with Disagreements.

For what concerns the learning with disagreements paradigm, up to our knowledge, EXIST 2024 [7,8] represents the first insight for multimodal sexism detection in memes, including perspectivism. Previous works under the learning with disagreement paradigm consider only textual expression. In this area, the main contribution is represented by [27], and by the previous edition of EXIST [28]. In particular, in the Learning With Disagreements (LeWiDi) challenge [27], four different datasets with different characteristics in terms of types, languages, goals, and annotation methods are proposed, including, for each instance, both hard labels (an aggregated hateful/non-hateful label) and soft labels (representing agreement among annotators). Similarly, the previous edition of EXIST [28] aimed at capturing sexism in all its forms while considering the perspective of the learning with disagreements paradigm. The challenge, articulated in different tasks, addressed sexism identification at different granularities and perspectives. The approach proposed by the participants of such challenges mostly relied on fine-tuned pre-trained models and/or ensemble methods [29,30,28].

Method

Task Description

The sEXism Identification in Social neTworks task at CLEF 2024 [7,8] aims at addressing the problem of sexism identification at a broad spectrum of sexism manifestation in social networks. Unlike previous editions of the same challenge [28], which focused solely on detecting and classifying sexist textual messages, the current edition also incorporates new tasks that address the same problems in a different form of representation: memes. As for the previous edition, the challenge embraces the Learning with Disagreements paradigm. For both tweets and memes, three different tasks are proposed that address the problem of sexism identification at different granularities, addressing (i) the identification of sexist messages, (ii) the author's intentions, and (iii) sexism categorization. This paper only focuses on Task 4, a binary classification task that consists of deciding whether or not a given meme is sexist.

Dataset

The meme dataset proposed for EXIST 2024 is composed of more than 3000 memes per language (considering English and Spanish). Memes that compose the dataset were collected via search query on Google Images exploiting 250 terms with varying degrees of use in both sexist and non-sexist contexts, all centered around women. Examples of memes are reported in Figure 1. The challenge also approaches the sexism identification task from the perspective of the Learning with Disagreements paradigm. For each meme, it provides labels and personal meta-data information (e.g. age, gender, etc.) gathered from six annotators, introducing two different evaluation settings:

• Hard Evaluation: Systems performances are evaluated considering a hard label derived from the majority class among the different annotators' labels. • Soft Evaluation: Systems performances are evaluated through a soft-soft evaluation that considers the probability distribution derived from the set of human annotators.

Unlike tasks based on Tweets, automatic sexism detection in memes, such as Task 4 addresses, did not come with a predefined validation dataset. Therefore, to conduct our hyperparameter optimization experiments, we partitioned the official training split to create a validation set, providing a basis for our initial research steps. The official training dataset comprises 4044 samples, evenly balanced in terms of sexism and non-sexism instances across both languages. From this dataset, we selected 20% of the samples for our validation set, using the remaining data for training our models. Once we identified the best-performing settings, we trained our models using the entire official training set for submission. The results discussed in Section 4 are based on the performance of these models on the official test set of the challenge.

Model Architecture

This section describes the modules that compose the cross-lingual, multi-modal, Transformer-based model proposed in this paper, whose overall architecture is depicted in Figure 2. Input Data. As described above, the dataset provides both the raw image of the meme and its corresponding text. However, this text does not have to be directly and clearly related to the content represented in the image; in many cases, this relationship depends on subtle and complex social and contextual details. Therefore, to further inform the model, we also incorporated a caption of the image content as an additional input. We used the state-of-the-art BLIP1 [31] model to automatically generate these captions. Consequently, our model processes three types of input data: the raw image, the OCR-based text, and the image caption.

Feature Extraction Frontend. We considered pre-trained, state-of-the-art models for feature extraction, namely the multilingual mBERT2 [32] and mCLIP3 [33]. These or similar models have been widely studied in the context of sexism detection [19,20,21,22,16] due to the multi-modality nature of memes, where the image and text are closely related or complementary, thus supporting our proposed approach. mBERT was used to extract feature embeddings for the OCR-based text and image-content captions separately, while all three types of input data, including the raw image, were processed by mCLIP to obtain their corresponding latent feature representations. As a result, we extracted five different input modalities. Note that mBERT and mCLIP remained frozen during the training process, so they were not adapted to the task. The rationale behind this decision was not only to lower the computational costs of our proposed approach, but also to investigate how robust this general, task-agnostic, pre-trained representations are in the context of sexism detection without fine-tuning.

Random Modality Masking.

We also employed a modality masking strategy. By randomly masking one of the five input modalities, the model is forced to extract relevant information from all modalities, thus preventing complete reliance on any single modality. This approach enhances the model's robust-ness against scenarios where a modality might be absent. The effectiveness of this strategy is supported by our experimental results. Note that not always one of the input modalities has to be masked.

Transformer Encoder Backbone. The extracted feature representations are first processed with 1D batch normalization layer followed by a linear layer to project all the feature embeddings to a 256-dimensional space. Furthermore, these projected features are conditioned via learnable embeddings to enable the model to differentiate between the five input modalities and their corresponding languages. This conditioning strategy, inspired by numerous works in the field of natural language processing [32], allows the model to effectively handle multiple languages and modalities simultaneously into a unified framework. Specifically, all these input modalities are concatenated and then processed by an encoder backbone based on the Transformer architecture [34], where the model performs what can be considered as a soft cross-attention between modalities. Finally, the final classification is predicted through an average pooling of the encoder output sequence, followed by a linear layer projection.

Implementation Details

All our models were developed using the open-source PyTorch library, as well as the corresponding HuggingFace pre-trained models for feature extraction. Experiments were conducted on a GeForce RTX 2080 GPU with 8GM memory.

Model Architecture. Hyper-parameter optimization was conducted to determine the best-performing model architecture for both hard-and soft-label evaluation approaches. This process involved exploring a range of learning rates, numbers of encoder layers, attention heads, and feed-forward layer sizes. For the hard-label approach, the optimal architecture was defined as a 1-layer encoder with a self-attention module of 8 attention heads and a modality masking probability of 0.8. For the soft-label approach, we found that a 4-layer encoder with 4 attention heads and a modality masking probability of 0.7 yielded optimal results. In both cases, the latent dimension of the Transformer encoder was set to 256, while the inner hidden representation of its feed-forward modules was laid on a 2048-dimensional space.

Training Settings. In all our experiments, we used the AdamW optimizer [35] and a one-cycle linear scheduler with a batch size of 16 samples. Based on our prior experiments, we determined the optimum settings for each task. For hard-label evaluation, the model was trained for 5 epochs with a learning rate of 0.0002. For soft-label evaluation, the best results were achieved by training the model for 7 epochs with a learning rate of 0.0005. In both cases, the dropout rate was set to 0.1 and we employed the cross-entropy loss as the objective function.

Evaluation Metrics. All the reported metrics were computed using the official PyEvALL library 4 . Depending on the type of labels considered for the addressed task, different metrics were used to evaluate our models. According to the challenge instructions, we employed the F 1 -Score, ICM, and ICM-Norm metrics for hard-label evaluation, and the cross entropy, ICM soft , and ICM soft -Norm metrics for soft-label evaluation.

Soft-Label Evaluation Postprocessing. As described in Subsection 3.2, each sample of the dataset was annotated by six annotators. Consequently, the ground truth probabilities can take one of seven possible values. Therefore, in soft-label settings, the raw logits predicted by our models were rounded to match one of the actual possible output values.

Proposed Approaches

In this work, we proposed three different approaches to address the automatic detection of sexism in memes both for the hard-and soft-label evaluation scenarios. Similar to previous studies [23,24,25,26], we explored not only single-model approaches, but also adopted two model ensemble strategies to provide a more stable and robust solution:

• Single Model (SM). This approach relies on a single model to obtain the final predictions. For each evaluation setting, we consider the corresponding best-performing model architecture determined through the hyperparameter optimization process described above.

• Majority Voting Ensemble (MVE).

To provide a more stable and robust approach that does not rely solely on one specific training procedure, we adopted a model ensemble strategy. We trained five models using the best-performing architecture, each initialized with a different random seed. The final predictions were then aggregated using a majority voting ensemble. For soft-label evaluation settings, we converted the continuous soft-label predictions into discrete classes by selecting the class with the highest predicted value. • Average Probability Ensemble (APE). Similarly, this model ensemble strategy involves training five models with different seeds based on the best-performing architecture. However, instead of working with discrete classes, we consider the raw logits predicted by the models and aggregate them via a non-weighted average. After combining these probabilities, we obtain the discrete classes by selecting the class with the highest predicted value.

Results & Discussion

Preliminary hyperparameter optimization experiments were carried out using the validation split described in Subsection 3.2. Once we determined the best-performing settings for each one of our three proposed approaches, we used the entire training set for final model estimation and submission in both hard-and soft-label evaluation scenarios. Tables 1, 2, and 3 compare our proposed approaches to the challenge baselines on the official test set in the context of the EXIST2024 Task 4. These challenge baselines are described as follows:

• Gold. Given that the ICM measure is unbounded, this baseline provides the best possible reference by using an oracle that perfectly predicts the ground truth. • Majority Class. A non-informative baseline that classifies all instances according to the majority class based on the six annotators. • Minority Class. A non-informative baseline that classifies all instances according to the minority class based on the six annotators.

Note that the ICM-Norm, both for hard-and soft-label scenarios, is considered the official evaluation metric of the challenge.

Overall Results Analysis Table 1

Overall analysis of our proposed approaches for EXIST2024 Task 4 for both English and Spanish instances. Results reported for the test set, using variants of ICM (↑), F 1 -score (↑), and cross entropy (↓) as evaluation metrics. Best results among our approaches are highlighted in bold for each evaluation metric. Table 1 provides a comparison of the overall performance of our submitted approaches on the official test set, considering both English and Spanish instances. Our best-performing approaches achieved 10th place for soft-label evaluation settings and 20th place for hard-label evaluation settings. In general terms, the model ensemble strategy based on majority voting (MVE) stands as the most robust approach. However, the performance of the single model (SM) does not substantially differ, suggesting that this simpler approach might be more appealing for deployment as it does not require considering multiple model decisions and their associated time cost. Furthermore, the SM approach was the one providing the best results for hard-label settings in terms of ICM-Norm. This reflects the greater complexity of the soft-label prediction task and its reliance on model ensembles to offer a more accurate outcome.

Language-Specific Results Analysis

Tables 2 and 3 provide a comparison of the overall performance of our submitted approaches on the official test set for the English and Spanish instances, respectively. For Spanish, our best-performing approaches achieved 5th place for soft-label evaluation settings and 13th place for hard-label evaluation settings. For Spanish, we achieved 11th and 21st places for soft-and hard-label settings, respectively. One of the most notable findings is not only the performance gap between both languages, but also the fact that the best-performing approach varies depending on whether we are dealing with English or Spanish memes. While English mostly benefits from model ensembles in all cases, the Spanish language shows a substantial improvement when addressing the hard-label scenario with a single-model approach. For soft-label settings, both languages reflect the same trend toward reliance on ensembles.

Table 2

Overall analysis of our proposed approaches for EXIST2024 Task 4 for the English language. Results reported for the test set, using variants of ICM (↑), F 1 -score (↑), and cross entropy (↓) as evaluation metrics. Best results among our approaches are highlighted in bold for each evaluation metric. One possible explanation for the performance gap between English and Spanish could be attributed to the discrepancy in data availability used to estimate our pre-trained, state-of-the-art feature extractors. Despite these models were designed to be multi-lingual, they may have been better optimized with English data due to its abundance. Consequently, when applied to Spanish memes, these models may not generalize as effectively, resulting in lower performance. This discrepancy highlights the need for further research to address the challenges posed by multiple languages in automatic sexism detection.

Table 3

Overall analysis of our proposed approaches for EXIST2024 Task 4 for the Spanish language. Results reported for the test set, using variants of ICM (↑), F 1 -score (↑), and cross entropy (↓) as evaluation metrics. Best results among our approaches are highlighted in bold for each evaluation metric.

Conclusions

In this work, we proposed a cross-lingual and multi-modal Transformer-based approach for sexism identification under the paradigm of learning with disagreements. Although achieved results did not show a significant improvement in performances by the introduction of ensemble methods, more complex aggregation strategies might be investigated for future work to aggregate models with different input configurations. Additionally, more sophisticated ensemble strategies can be explored, such as Bayesian Model Averaging (BMA) [36]. Finally, the discrepancy in performance between different languages highlights the need for further research to effectively handle multiple languages in the context of automatic sexism detection.