This paper presents our participation in Task 1 of the EXIST 2025 shared task, which focuses on the identification and characterization of sexism in tweets across three subtasks: (1) binary sexism detection, (2) categorization of sexist content, and (3) identification of the source and target of the sexist message. We leverage semantic embeddings generated using pre-trained models from Google's Generative AI suite, evaluated in both frozen and ifne-tuned forms. Classification is carried out using traditional machine learning models such as Random Forest, SVM, and MLP. Experiments are conducted in both English and Spanish, with results evaluated using 10-fold cross-validation. Our findings demonstrate that fine-tuned Gemini embeddings consistently outperform generic representations. Challenges remain particularly in subtask 1.3 due to label ambiguity and sparsity.
The EXIST 2025 shared task [
Sexism remains a persistent and deeply rooted issue in contemporary society, primarily afecting
women. It is defined as discrimination based on sex [
Twitter (now rebranded as X ), as a major platform for public conversation, reflects and amplifies these dynamics. While it serves as a space for social advocacy and visibility, it also facilitates the dissemination of sexist narratives, often normalized under the guise of humor, opinion, or cultural references. Given the speed and scale at which content is generated on such platforms, automatic detection systems are essential to identify, categorize, and ultimately help mitigate the spread of gender-based discrimination online.
In recent years, research on automatic sexism detection has evolved from traditional machine
learning approaches to neural-based architectures. Early systems relied on TF-IDF or bag-of-words
representations with classifiers such as SVM or logistic regression. More recent systems have leveraged
contextual embeddings from pre-trained language models such as BERT, RoBERTa, and BERTweet
[
Several approaches to automatic sexism detection have emerged in recent editions of the EXIST
shared task. In 2024, top-performing systems leveraged transformer-based models such as RoBERTa,
XLM-RoBERTa, and BERTweet, often combined with data augmentation and disagreement-aware
strategies. For example, the NYCU-NLP team [
The 2025 edition of EXIST covers content from Twitter, TikTok, and memes, structured into three main
tasks: (1) identifying whether content is sexist, (2) classifying the type of sexism, and (3) identifying the
source and target of the sexist discourse. This work focuses exclusively on the Twitter data, involving
English and Spanish languages, and adopts the Learning with Disagreement annotation scheme that
preserves label diversity from multiple annotators [
In parallel, large-scale embedding models have gained traction as lightweight yet powerful
alternatives to end-to-end fine-tuned systems. Google’s Gemini embedding models have shown promising
performance in multilingual benchmarks such as the Massive Text Embedding Benchmark (MTEB) [14]
and are designed to support downstream tasks like classification and semantic similarity through frozen
or parameter-eficient fine-tuning strategies [
To address this challenge, we generated embeddings using two pre-trained models from Google’s
Generative AI suite: embedding-001, a general-purpose encoder, and gemini-embedding-exp-03-07,
a model optimized for classification tasks. These embeddings were either kept frozen or fine-tuned
through lightweight neural layers and later fed into traditional classifiers. Our pipeline treats English
and Spanish data independently, respecting linguistic diferences in sexist expression. This architecture
aims to balance efectiveness, interpretability, and computational eficiency [
The remainder of this paper is organized as follows. Section 2 introduces the proposed architecture and design considerations. Section 3 presents the dataset and task setup. Section 4 details the classification models and embedding strategies. Section 5 describes the experimental setup and evaluation protocol. Results and rankings are presented in Section 6, followed by a discussion in Section 7. Finally, Section 8 concludes the paper and outlines future work.
Our approach leverages semantic embeddings generated by large language models (LLMs) provided by Google’s Generative AI suite. Specifically, we utilize two embedding models: embedding-001, a general-purpose encoder, and gemini-embedding-exp-03-07, an experimental model optimized for classification tasks.
The pipeline consists of generating fixed-size vector embeddings (768-dimensional) for each tweet using these models via API calls. We explore two strategies with these embeddings: (1) using them as frozen features directly fed into classical classifiers, and (2) fine-tuning the embeddings with lightweight trainable layers in a parameter-eficient way.
Figure 1 illustrates the full pipeline, including embedding generation, usage strategy, and classification.
While Gemini embeddings serve as the core representation model in our system, the pipeline remains model-agnostic and could support embeddings derived from other neural encoders such as XLMRoBERTa or multilingual BERT. Gemini ofers high-quality multilingual semantic vectors, allowing for robust feature extraction without the need to train task-specific deep models from scratch.
To adapt embeddings to the downstream task, we apply parameter-eficient fine-tuning (PEFT) techniques. Specifically, we freeze the base embedding model and introduce a lightweight classification head (typically a linear or shallow MLP layer), which is trained using cross-entropy loss on each subtask. Fine-tuning is performed separately for each subtask (1.1, 1.2, and 1.3) and language (EN/ES), using only the EXIST 2025 training data for that subtask. This strategy ensures low computational cost while improving task alignment.
The embeddings—either frozen or fine-tuned—are then passed to classical classifiers, including Support Vector Machines (SVM) and Multi-Layer Perceptrons (MLP), trained independently for each configuration. The modularity of this architecture allows for eficient experimentation and rapid switching between embedding strategies and classifiers.
By combining large language model embeddings with lightweight fine-tuning and classical classifiers, our system captures complex semantic nuances across English and Spanish tweets while maintaining interpretability and eficiency.
We worked with the oficial EXIST 2025 dataset for Task 1, which includes tweet-level annotations in both English and Spanish. Each tweet was annotated by multiple annotators, resulting in a set of potentially divergent labels per instance. To resolve this disagreement for training purposes, we applied majority voting to the label lists provided for each subtask. Tweets for which a clear majority label could not be determined were discarded from the training set.
Task 1 is divided into three subtasks: • Subtask 1.1 – Sexism Identification: A binary classification task in which each tweet is labeled as either sexist or non-sexist. • Subtask 1.2 – Sexism Categorization: A multi-class classification task where each sexist tweet must be assigned to one of the following categories: ideological and inequality, stereotyping and dominance, objectification, misogyny and violence, or non-sexist. • Subtask 1.3 – Source and Target Identification: A multi-label classification task where tweets may be annotated with zero, one, or more labels indicating the source (e.g., individual, group, media) and target (e.g., individual woman, feminist movement) of the sexist message. This task introduces additional complexity due to sparse label distributions and overlapping roles.
For Subtask 1.1, we computed the most frequent label among the labels_task1_1 field. For Subtasks 1.2 and 1.3, which include multiple possible labels, we discarded tweets with empty or unresolved annotations. The labels were treated as single-label (1.2) or multi-label (1.3) targets, depending on task formulation.
We maintained the original lang field to process English and Spanish tweets separately, allowing for language-specific model training and evaluation. The label distributions across tasks and languages were notably imbalanced, particularly in Subtask 1.3, where many tweets lacked clear source or target annotations, increasing classification dificulty.
No aggressive preprocessing was applied to the tweet text. Specifically, we chose to retain original casing, punctuation, user mentions, and emojis. This decision was based on the fact that many linguistic and pragmatic cues in Twitter (now X)—such as emphasis, sarcasm, or gendered tone—are often conveyed through informal conventions, including emojis. Since our semantic embeddings were generated using large language models capable of interpreting such features, we considered their removal potentially detrimental to performance.
The final processed dataset was stored in .parquet format and included tweet IDs, raw text, resolved label(s), and language metadata. We applied a consistent processing pipeline across all three subtasks and used 10-fold stratified cross-validation for evaluation, prioritizing data eficiency and robustness over the predefined development/test splits. This preprocessed dataset served as the foundation for all subsequent embedding and classification experiments described in the following sections.
To establish a strong baseline and evaluate the benefits of using semantic embeddings, we employed several classical machine learning models widely used in text classification tasks.
Logistic Regression (LR) and XGBoost were chosen as baseline classifiers due to their robustness, interpretability, and proven efectiveness in various NLP tasks. Logistic Regression uses a linear decision boundary and probabilistic outputs, making it suitable for binary and multiclass tasks with relatively fast training times. XGBoost is a powerful gradient boosting framework that builds an ensemble of decision trees, enabling it to capture complex feature interactions and nonlinear relationships often missed by linear models. Both models were trained on vector representations derived from word n-grams (1,2) and character n-grams (3,5), capturing both lexical and sub-lexical features relevant in social media text.
Our main approach utilizes semantic embeddings generated from large language models (LLMs) provided by Google’s Generative AI suite, specifically the embedding-001 and gemini-embedding-exp-03-07 models. These embeddings convert raw tweet texts into dense, ifxed-length vectors encoding rich contextual and semantic information beyond traditional sparse representations.
We experimented with two embedding strategies: • Frozen embeddings: Embeddings are computed once and kept fixed during training. Classical classifiers such as SVM and MLP operate on these static representations, relying on the pretrained semantic knowledge encoded by the LLMs. This approach significantly reduces training time and computational requirements, as only the classifier parameters are optimized. • Fine-tuned embeddings: We applied parameter-eficient fine-tuning (PEFT) techniques, such as adapter modules or low-rank adaptations, to update only a small subset of the embedding model’s parameters. This allows the embeddings to adapt to the sexism detection task without requiring full backpropagation through the entire LLM, thus balancing eficiency and task-specific specialization.
In our case, PEFT was applied using the same labeled training data used for the downstream classiifcation tasks, and optimized using categorical cross-entropy as the training objective. This enabled task-specific adaptation of the embeddings while keeping the majority of the model parameters frozen, thus making training feasible under limited resources.
Unless otherwise stated, results reported in subsequent sections correspond to classifiers trained on ifne-tuned embeddings, as this configuration consistently yielded higher performance.
The embedding representations were fed into the following classifiers: • Support Vector Machine (SVM): SVM with a linear kernel is efective for high-dimensional feature spaces and is commonly used in text classification. It attempts to find the optimal hyperplane separating classes with maximum margin, providing good generalization performance. • Multi-Layer Perceptron (MLP): A feed-forward neural network with a single hidden layer and nonlinear activations (e.g., ReLU), capable of modeling complex patterns in the embedding space. MLPs can learn nonlinear decision boundaries, which may be beneficial for nuanced tasks like sexism detection. • Random Forest (RF): RF constructs an ensemble of decision trees trained on random subsets of features and data samples, providing robustness to overfitting and ease of use. However, despite its theoretical advantages, RF showed lower predictive performance and higher computational costs compared to LR and XGBoost in our experiments. Consequently, RF was excluded from the ifnal system runs and submissions.
The full pipeline begins with raw tweets being converted to embeddings through API calls. Depending on the experiment, these embeddings are either kept frozen or fine-tuned via PEFT. The resulting embeddings are then used to train the classifiers independently for each subtask and language, enabling models to adapt to task-specific label distributions and linguistic variations.
This approach allows us to leverage the rich semantic power of LLM embeddings while maintaining model eficiency and interpretability through classical classifiers. Overall, this modular and languagespecific architecture allowed flexible experimentation across subtasks and supported robust performance comparisons across diferent embedding and classifier configurations.
All experiments were conducted using Python 3.10 within Google Cloud’s Vertex AI environment, which provided scalable GPU resources suitable for eficient training and evaluation of classification models.
Semantic embeddings were generated via the Google Generative AI API using two models: embedding-001 and gemini-embedding-exp-03-07. Each tweet was encoded into a 768dimensional vector using the CLASSIFICATION mode, which is optimized for downstream classification tasks. To mitigate API rate limits and network instability, we implemented a checkpointing mechanism with periodic ‘.parquet‘ dumps and exponential backof retry logic to ensure robustness in large-scale embedding extraction.
For classification, we trained Support Vector Machine (SVM) classifiers with linear kernels and default regularization parameters, selected for their strong performance in high-dimensional vector spaces. Multi-Layer Perceptron (MLP) classifiers were configured with a single hidden layer of 256 neurons and ReLU activation, optimized using early stopping based on validation loss. All models were implemented using scikit-learn and TensorFlow.
Parameter-eficient fine-tuning (PEFT) was applied to adapt the embeddings to the sexism detection task, using the same training data and classification objectives described in Section 4. This setup enabled semantic adaptation without backpropagating through the full LLM backbone.
To handle the known class imbalance in the EXIST 2025 dataset—especially severe in Subtask 1.3—we used stratified 10-fold cross-validation for model selection and evaluation. This approach preserved label proportions across folds, ensuring balanced comparisons. Evaluation metrics included macro-averaged F1 (primary), micro F1, precision, recall, and accuracy.
Hyperparameter tuning was conducted via manual grid search, with macro F1 on validation folds as the selection criterion. The best-performing configuration per subtask and language was retained for final predictions. Final predictions were obtained directly from these models, with no ensemble methods or post-processing applied.
This setup allowed us to systematically explore the interaction between embedding strategies, model architectures, and multilingual label distributions under controlled and reproducible conditions.
We report macro-averaged F1-scores obtained via 10-fold cross-validation for each subtask, classifier, and embedding type. All the results presented in this section correspond to our best-performing configurations using fine-tuned embeddings , as they consistently outperformed their frozen counterparts across tasks and languages.
The results are shown for each subtask separately and are broken down by language and system version. These include: • Version 1: A monolithic setup using the same embedding-classifier combination for both English and Spanish. • Version 2: An alternative combination using a diferent embedding-classifier pair for both languages. • Version 3: A hybrid system that selects the best-performing configuration independently for each language.
This task is more complex due to class imbalance and semantic overlap between categories. Table 2 shows macro F1-scores for each version. The best configuration involved combining MLP for English and SVM for Spanish, using fine-tuned embeddings.
EN ES EN ES EN ES EN ES EN ES EN ES
Our experiments reveal that the selection of embeddings, classifiers, and fine-tuning strategies significantly influences performance across all three subtasks. In particular, models using ifne-tuned embeddings consistently outperformed those based on frozen representations, confirming the efectiveness of adapting semantic features to the task.
The binary sexism identification task (Subtask 1.1) shows the highest performance, particularly when using MLP with fine-tuned embedding-001 for Spanish tweets and SVM with fine-tuned gemini-embedding-exp-03-07 for English tweets. This suggests that adapting embeddings to specific languages and subtasks captures linguistic nuances efectively.
The multi-class and multi-label subtasks (1.2 and 1.3) present greater challenges due to increased complexity, class imbalance, and ambiguity in labels. Macro F1 scores are generally lower for these subtasks, reflecting the dificulty in fine-grained categorization and source-target identification. Nonetheless, parameter-eficient fine-tuning (PEFT) provides measurable improvements in generalization and task alignment, especially in low-resource or imbalanced settings.
Additionally, the strategy of selecting the best model configuration for each language and task independently leads to improved overall performance. This highlights the importance of flexible, task-specific pipelines in handling the diverse expressions of sexism in multilingual social media data.
Future directions include exploring more advanced metric learning techniques and label adaptation methods to better manage label scarcity and annotator disagreement, potentially enhancing robustness and accuracy across subtasks.
Overall, our findings underscore the necessity of embedding adaptation, careful classifier choice, and task-tailored tuning to efectively tackle sexism detection and characterization in real-world social media data.
In this work, we presented our approach to sexism identification and characterization in tweets for the EXIST 2025 shared task. By leveraging semantic embeddings from large language models in combination with classical classifiers and parameter-eficient fine-tuning techniques, we achieved competitive results across multiple subtasks and languages.
Our experiments confirmed that fine-tuning embeddings consistently improved performance over frozen counterparts, particularly in multilingual and nuanced classification settings. Moreover, tailoring model configurations per task and language proved beneficial in capturing linguistic diversity and reducing error propagation across subtasks.
Despite these promising outcomes, several challenges persist—most notably in the fine-grained categorization and source-target identification subtasks, where label sparsity and semantic ambiguity hinder classifier performance.
For future work, we plan to explore advanced aggregation architectures such as Deep Averaging Networks (DAN), as well as improved implementations of zero-shot and few-shot learning techniques. These approaches may enhance generalization in low-resource scenarios and better capture subtle linguistic cues.
Additionally, incorporating domain adaptation techniques and modeling annotator disagreement more explicitly may improve system robustness and fairness. Expanding our experiments to other modalities such as memes and TikTok videos, as featured in the broader EXIST 2025 setup, is also a compelling next step.
Overall, our findings contribute to the ongoing development of interpretable, adaptable, and resourceeficient systems for sexism detection in multilingual social media environments.
We would like to thank the organizers of the EXIST shared task and the CLEF community for providing the dataset and evaluation framework that made this work possible.
This work has been developed as part of the author’s Bachelor’s Thesis at Universitat Politècnica de València. The project benefited from institutional support and resources provided by the university.
We also thank the developers of Google’s Generative AI suite for enabling access to powerful embedding models that were fundamental for our experiments.
The authors used Generative AI tools (e.g., ChatGPT) in a limited capacity to assist with phrasing and text refinement. All content was written, reviewed, and edited primarily by the authors, who take full responsibility for the final version of the manuscript. [12] N. Muennighof, S. Magister, J. Reimers, et al. (2023). MTEB: Massive Text Embedding Benchmark.
arXiv preprint arXiv:2210.07316. [13] Muller, T., Perez-Torró, G., & Franco-Salvador, M. (2024). Few-Shot Learning with Siamese
Networks and Label Tuning. arXiv preprint arXiv:2203.14655. [14] Muennighof, N., Magister, S., Reimers, N., et al. (2023). MTEB: Massive Text Embedding Benchmark. arXiv preprint arXiv:2210.07316.