This work presents a novel approach for sexism identification in social media (EXIST 2025 Task 1) by reformulating the binary classification problem into a seven-class task. We implement ModernBERT-large - a state-of-the-art bidirectional transformer - with layered learning rate decay for hierarchical feature optimization. The model is enhanced with Supervised Contrastive Learning (SCL) to improve discrimination of nuanced sexism expressions through metric learning. Our architecture incorporates: (1) Task reformulation from binary to fine-grained seven-class prediction, (2) ModernBERT's memory-eficient attention mechanisms for long-context understanding, and (3) Hybrid CE+SCL loss( = 0.9) for robust representation learning. Experiments demonstrate significant performance gains over baseline methods in both hard and soft evaluation settings.
Social media platforms have become ubiquitous channels for communication, activism, and social
discourse. However, they also facilitate the proliferation of harmful content, including explicit and
implicit forms of sexism—prejudice or discrimination based on gender, predominantly targeting women
and marginalized groups. Sexist content ranges from overt misogyny to subtle linguistic cues, such as
stereotyping, objectification, and victim-blaming, which normalize gender-based violence and inequality.
Automated detection of such content is critical for creating safer online spaces, yet it remains challenging
due to the subjective nature of sexism interpretation, where annotator demographics (e.g., gender, age,
cultural background) significantly influence labeling decisions[
The EXIST (sEXism Identification in Social neTworks)[
To tackle Task 1, we propose a novel approach that diverges from conventional binary classification.
We reformulate the binary task into a seven-class problem, where each class represents a distinct
combination of annotator votes (e.g., 4 "YES" votes + 2 "NO" votes → Class 4). This granular transformation
explicitly models the spectrum of disagreement among annotators, allowing the model to capture
nuanced subjectivity inherent in sexism annotation. Our architecture leverages ModernBERT-large[
Further, we integrate Supervised Contrastive Learning (SCL) [
This work marks the first application of seven-class reformulation, ModernBERT-large[
Task 1 of the EXIST 2025 challenge focuses on sexism identification in social media posts, formulated as a binary classification problem. The primary objective is to determine whether a given social media post (tweet, meme, or video) contains sexist expressions or behaviors. This task addresses the critical need for automated detection of gender-based discrimination in online spaces, which ranges from overt misogyny to subtle linguistic cues such as stereotyping, objectification, and victim-blaming.
A distinctive feature of EXIST is its Learning with Disagreement (LeWiDi) learning paradigm. Recognizing the subjective nature of sexism interpretation, each instance is annotated by multiple annotators with diverse socio-demographic backgrounds (gender, age, ethnicity, etc.). This approach intentionally captures annotator subjectivity, rejecting the notion of a single "gold label" and instead training models to learn from diverse perspectives and disagreements.
The dataset comprises 10,034 annotated social media posts in English and Spanish, curated from mainstream platforms. Table 1 details the distribution across training, development, and test sets. Key characteristics include: • Multilingual content: Balanced representation of English (48.5%) and Spanish (51.5%) • Rich annotation metadata: Each post includes annotator demographics (gender, age, ethnicity) and per-annotator labels • Disagreement modeling: 6 independent annotations per instance, explicitly capturing labeling subjectivity
Data collection followed strict ethical guidelines, with annotations performed by diverse annotator pools recruited through Prolific. The inter-annotator disagreement rate averages 32.7%, reflecting the inherent subjectivity in sexism identification.
The dataset is provided in JSON format, with each instance (tweet) represented as a JSON object containing the following attributes: • "id_EXIST": Unique identifier for the tweet. • "lang": Language of the tweet text ("en" for English, "es" for Spanish). • "tweet": Text content of the tweet. • "number_annotators": Number of annotators who labeled the tweet. • "annotators": List of unique identifiers for each annotator. • "gender_annotators": List of genders of the annotators ("F" for female, "M" for male). • "age_annotators": List of age groups of the annotators ("18-22", "23-45", "46+"). • "ethnicity_annotators": List of self-reported ethnicities (e.g., "Black or African American", "Hispano or Latino"). • "study_level_annotators": List of educational levels (e.g., "High school degree or equivalent", "Bachelor’s degree"). • "country_annotators": List of countries where annotators reside. • "labels_task1": List of labels (one per annotator) indicating sexist content ("YES" or "NO"). • "labels_task2": List of labels (one per annotator) for source intention ("DIRECT", "REPORTED", "JUDGEMENTAL", "-", "UNKNOWN"). • "labels_task3": List of arrays (one per annotator) indicating sexism types (e.g., "IDEOLOGICAL-INEQUALITY "STEREOTYPING-DOMINANCE").
• "split": Subset ("TRAIN", "DEV", "TEST" + language sufix).
Instances are categorized into three dificulty levels based on annotator agreement: • Easy: Full consensus (6 identical annotations). • Medium: Partial consensus (4–5 identical annotations).
• Hard: High disagreement (≤ 3 identical annotations).
This stratification reflects the inherent subjectivity in sexism identification, where ambiguous cases (e.g., sarcasm, implicit stereotypes) yield lower agreement.
To explicitly model annotator disagreement, we reformulate the binary task into a 7-class problem, where each class corresponds to a unique combination of annotator votes (e.g., 4 “YES” + 2 “NO” → Class 4). The preprocessing pipeline includes: • Vote Aggregation: Counting “YES” votes (0–6) per instance. • Class Assignment: Mapping vote counts to discrete classes (0–6). • Soft Label Conversion: For evaluation, class values are converted to probabilistic “YES”/“NO” scores (e.g., Class 4 → “YES”= 4/6, “NO”= 2/6) to align with the soft evaluation protocol.
The evaluation of the proposed models was conducted using the Information Contrast Measure (ICM), a similarity function that generalizes Pointwise Mutual Information (PMI) and assesses the alignment between system outputs and ground truth categories in classification tasks. The oficial evaluation included two modes: Hard-Hard and Soft-Soft. In the Hard-Hard evaluation, the system’s single predicted label was compared to the majority-voted ground truth label, while the Soft-Soft evaluation compared the system’s probability distribution to the annotators’ label distribution. Additionally, Cross-Entropy was used to provide a comprehensive assessment of model performance. The evaluation metrics are summarized in Table 2.
Description Measures similarity between predicted probability distribution and ground truth distribution.
Measures similarity between predicted hard labels and majority-voted ground truth labels.
Normalized ICM-Soft score for comparative analysis.
Normalized ICM-Hard score for comparative analysis.
Evaluates the diference between predicted probabilities and true distributions.
We reformulate the binary sexism identification task (Task 1) as a seven-class classification problem, where each class represents a distinct combination of annotator votes (0–6 “YES” votes). Given an input tweet text , the model predicts a class ∈ {0, 1, ..., 6} corresponding to the aggregated annotator votes.
Our architecture combines ModernBERT-large with Supervised Contrastive Learning (SCL), as illustrated in Figure 1.
For input text , ModernBERT generates contextual embeddings:
H = ModernBERT() ∈ R× (1) where is sequence length and = 1024. We extract the [CLS] token representation h = H[0] and apply L2 normalization for downstream tasks: ‖h‖2 The normalized embedding z is used for both classification and contrastive learning. z =
h
LR = LRbase × −
We apply layer-specific learning rates:
where:
• = 0.8 (decay rate[
For a batch of samples, we compute SCL loss using the normalized embeddings z: 1 ∑︁ ℒ = − 1
∑︁ log =1 | ()| ∈ ()
exp(z · z/ )
∑︀=1 1[̸=] exp(z · z/ )
where:
• () = set of positives (samples sharing the same class label as )
• = 0.3 (temperature parameter[
The total loss combines Cross-Entropy (CE) and SCL:
ℒtotal = (1 − )ℒ + ℒ
with = 0.9 controlling the balance[
• Optimizer: AdamW with 1 = 0.9, 2 = 0.999 • Batch Size: 16 (gradient accumulation for efective 64) • Learning Rate: 1e-5 with linear warmup (10% steps) • Epochs: 6 with early stopping
We evaluate using: 1. Hard-Hard: Accuracy vs majority vote 2. Soft-Soft: ICM (Information Contrast Measure): ∈ where = predicted distribution, = annotator distribution. ICM(, ) = ∑︁ () log () () (2) (3) (4) (5) (6)
All experiments were conducted on a single NVIDIA A800 GPU with 80GB memory. The hyperparameters were carefully tuned to optimize model performance, following the configurations used in our baseline implementations.
Our architecture combines ModernBERT-large[
The training protocol employed the following hyperparameters, summarized in Table 3:
Our experimental results demonstrate the efectiveness of the proposed approach in both Soft-Soft and Hard-Hard evaluation settings. Table 4 presents the performance comparison under the Soft-Soft evaluation protocol, while Table 5 shows results for the Hard-Hard setting. The oficial evaluation metrics include Information Contrast Measure (ICM), normalized ICM (ICM Norm), and Cross-Entropy for Soft-Soft evaluation, with additional F1-score for the Hard-Hard setting.
Our system achieved competitive performance in both evaluation settings. Under the Soft-Soft protocol, fosu-students_2 ranked 9th out of 66 submissions with an ICM-Soft score of 0.6663 (ICM-Soft Norm: 0.6070), significantly outperforming both majority-class (-2.1991) and minority-class (-3.8158) baselines. The Cross-Entropy value of 1.5069 indicates reasonable alignment with the annotator distribution, though there remains room for improvement compared to the gold standard (0.5770).
In the Hard-Hard evaluation, fosu-students_3 secured 12th position with an ICM-Hard score of 0.5661 (ICM-Hard Norm: 0.7889) and F1-score of 0.7638 for the “YES” class. This represents a substantial improvement over the non-informative baselines, demonstrating our model’s ability to capture majority voting patterns while maintaining balanced performance across classes. The normalized ICM-Hard score of 0.7889 suggests our predictions align well with the consensus labels, achieving approximately 79% of the perfect score.
Note that fosu-students_2 and fosu-students_3 denote variations optimized for Soft-Soft and HardHard evaluations respectively, using identical architectures but diferent loss weightings in the hybrid loss
The performance gap between Soft-Soft and Hard-Hard results suggests our approach handles clear-cut cases (Hard-Hard) more efectively than ambiguous instances with annotator disagreement (Soft-Soft). This observation aligns with the challenge’s Learning with Disagreement paradigm, where modeling subjective interpretations remains an open research problem. Future work should focus on improving the probabilistic outputs to better capture annotator subjectivity in the Soft-Soft setting.
This work presents a novel approach for sexism identification in social media by fundamentally
reformulating the binary classification task into a fine-grained seven-class problem that explicitly models
annotator disagreement. Our seven-class framework—where each class represents a distinct
combination of annotator votes (0–6 YES)—proves highly efective in capturing the inherent subjectivity
of sexism annotation, addressing a core limitation of traditional binary models. The integration of
ModernBERT-large[
This work is supported by the National Natural Science Foundation of China (No.62276064). During the preparation of this work, the author(s) used DeepSeek in order to: Grammar and spelling check. After using these tool(s)/service(s), the author(s) reviewed and edited the content as needed and take(s) full responsibility for the publication’s content.