Hope speech supports the development of inclusive digital environments by mitigating hostility and fostering better mental health. We participated in the HOPE task as part of IberLEF 2025 to explore hope speech detection in English and Spanish social media content. The challenge consists of two components: hopeful versus not hopeful classification and multiclass categorization of hope speech forms. Our NLP-CIC team took part in both subtasks of the provided datasets, resulting in an overall placement of 8th place. We applied XLM-roberta-base and BERT-base-multilingual-cased models to perform binary and multiclass classification while adapting it for cross-language hope expression detection. The evaluation showed high scores in the binary classification, where the systems yielded macro F1 scores of 0.8638 for English and 0.8520 for Spanish, and in the multiclass task, the scores reached 0.7139 for English and 0.6901 for Spanish. The results showed that automatic detection of hope is feasible, illustrating its crucial role in fostering positive communication across diverse languages on social platforms. In the binary classification task, our team ranked 8 th out of 30 for English and 1st out of 11 for Spanish. For the multiclass classification task, the team achieved 14 th out of 28 for English and 4th out of 8 for Spanish
Modern social life is deeply interconnected with social networks because these platforms facilitate
the sharing of thoughts, opinions and experiences. The expedited growth of social platforms like
Twitter and Instagram can be attributed to main characteristics including afordable pricing, limited
user identiafibility, seamless accessibility and ease of adoption [
Traditional ML techniques such as logistic regression (LR), support vector machines (SVMs) and
naive Bayes (NB) depend on manually engineered features for model building. DL algorithms such as
Long Short-Term Memory (LSTM) networks, Recurrent Neural Networks (RNNs), and Convolutional
Neural Networks (CNNs) have the ability to discover intricate patterns within textual data. The NLP
ifeld experienced a significant transformation when Transformer-based models, especially Bidirectional
Encoder Representations from Transformers (BERT), embraced attention mechanisms alongside
contextual embeddings. The models deliver outstanding achievements when used for diferent computational
tasks [
BERT and its various versions lead other models through their exceptional performance and efective
cross-lingual capabilities [
Hope represents the optimistic belief that beneficial results will emerge in the future and functions
as a major determinant of mental responses, emotional states, and behavioral patterns [
The widespread adoption of HSD systems continues to face numerous hurdles, although their usefulness has been established [16]. Advanced preprocessing techniques and filtration methods remain essential for identifying real hope messages among the overwhelming number of social media entries that exist on large scales. The classification results fail to perform accurately because underrepresentation of categories reduces performance but overrepresentation leads to biased outcomes. The detection process for hope outside training boundaries proves dificult since models struggle to extend their capabilities outside their learned contexts. Detecting hope in multilingual settings proves challenging because speakers employ distinct hope language expressions between Spanish and English so specialized detection methods become necessary. Language-aware robust models will be the essential solution to detect hope because they need to efectively process subtle cultural developments.
The workshop organizers launched the HOPE shared task at IberLEF 2025 to develop systems that detect hopefulness across multiple languages. The shared task aims to help researchers develop detection systems that identify hopeful expressions in social media content written in English and Spanish for better interlanguage communication.
In recent years, hope detection (HD) has gained growing attention for its role in fostering embracing
and optimistic communication on social media platforms [17]. For HD, in [18] researchers provide
in-depth analysis of diferent methods through binary text classification into hope and not-hope classes
by utilizing SVM and KNN algorithms. Model accuracy improvement depends heavily on hyper
parameter optimization because cross-lingual hope detection faces challenges from natural linguistic
diferences between languages. The research into multilingual classification benefits immensely from
these discoveries indicating the necessity of specific algorithm matches for diferent language contexts.
The research field gains significant value from [
The authors in [19] developed a model involving CNN-based architecture during the HOPE 2023 shared task competition at IberLEF. The model analyzes brief texts like English YouTube comments and Spanish tweets through its platform which features a thorough preprocessing mechanism for tokenization with special character management to produce lexical feature outputs. The CNN architectural structure works to discover features which relate to hope. The study identified two main restrictions because mislabeling afected the dataset and conflictive linguistic signals in English text caused class distribution problems. The model proves that CNNs can extract important signals about hope even when working with brief social media messages despite their limitations.
The psycholinguistic characteristics of hope speech receive analysis through established NLP tools, including LIWC, NRC Emotion Lexicon, and VADER sentiment analysis. Diferent subtypes of hope are explained through linguistic analysis before being classified through LightGBM and CatBoost models. The gradient boosting frameworks deliver efective modeling capabilities for emotional intensity characteristics and hope speech variation through competitive results which surpass traditional classification tools and match deep learning performance [20].
In [21] researchers gives an overview about the IberLEF 2024 workshop, which added value to the NLP community by developing benchmarking tasks and language datasets. Multiple hope speech assessment tasks make use of extensive evaluation metrics for measuring model success. This workshop functions as a key research venue for enhancing language minority studies and encouraging inter-task NLP collaboration among experts. The research highlights how dataset quality combines with model architecture to shape detection of hope trials in terms of linguistic diversity. Research activity regarding the English language predominantly dominates the field even though the Spanish language and others with lower research emphasis need methodological advancements.
This research foundation serves as the basis for our work that builds culturally-sensitive hope detection methods in both English and Spanish languages. Our research utilizes various datasets together with annotation systems to resolve current eficiency constraints while improving hope detection capabilities for both languages.
We focused on developing a system that could sort short texts within two classification categories ("hope" or "not hope") for binary tasks alongside multiple emotional category identification for multiclass classification. The five categories for multiclass classification included generalized hope, realistic hope and unrealistic hope, as well as not hope with sarcasm. The research aimed to detect diferent categories of hope by establishing diferences between authentic hope and sarcastic expressions. The sentiment analysis task was performed using XLM-R and mBERT transformers, which are pre-trained on extensive multilingual datasets specific to binary classification operations. The model received specialized training to diferentiate between texts that expressed hope and those that did not indicate hope. By implementing a specific loss function alongside backpropagation processes, the model learned to adjust its binary classification outputs in alignment with the supplied labeling information. The transformer architecture maintained its design for multiclass classification work but underwent modification to predict between generalized hope, realistic hope, unrealistic hope, not hope, and sarcasm. The prediction of emotional category relied on using categorical cross-entropy as a loss function with softmax activation to determine the most probable outcome.
Both the binary and multiclass tasks utilized k-fold cross-validation to build robustness into the model. The research used five subsets of data divided by K = 5. The training included the use of K-1 subsets, while the final subset functioned as validation in each round. The model underwent multiple epochs of repetition for optimizing its performance outcome. The learning curves showed satisfactory iftting performance in complex datasets which confirmed the capability of our approach to diferentiate binary and multiclass emotional categories related to hope.
The first task focuses on classifying social media content through a two-part framework between hopeful statements and statements without hope across Spanish and English languages. Online text analysis for hope detection is fundamental to understanding diferent modes through which hope appears in such content. Identifying hope needs to include both straightforward statements and nuanced indications that might be hidden within complicated language forms and implied emotional content. Subtask 1.a involves English language binary text classification, while Subtask 1.b performs a similar Spanish language classification procedure.
The second task separates hope into diferent categories to identify numerous expressions through a multiclass detection approach. The analysis requires researchers to segregate generalized hope from realistic hope and unrealistic hope together with sarcasm and non-hope within English and Spanish language texts. The multiclass classification approach enables comprehensive exploration of various hope categories to gain full insights into seasonal variations of hope expressed through social media text. Subtask 2.a and 2.b apply multiclass classification techniques for English and Spanish language texts to increase analytical details in their respective investigations[22].
The dataset contains English and Spanish social media texts annotated for both binary and multiclass HSD tasks. The binary classification dataset is almost equally split between "Hope" and "Not Hope" for both languages. The English training set has 2,426 "Hope" and 2,807 "Not Hope" (5,233 total) units, and the development set has 899 "Hope" and 1,003 "Not Hope" (1,902 total) units. Similarly, the training set for Spanish comprises 5,316 "Hope" and 5,927 "Not Hope" samples (11,243 total), and the development set consists of 1,926 and 2,162 samples, respectively (4,088 total). Five categories are used in the multiclass setting: Generalised Hope, Realistic Hope, Unrealistic Hope, Not Hope, and Sarcasm. In this regard, label imbalance becomes more obvious. For English, "Not Hope" dominates the training set (2,245), followed by "Generalized Hope" (1,284), and a high number of "Sarcasm" entries (692). The words "Realistic Hope" and "Unrealistic Hope" are low in frequency (540 and 472, respectively). This trend continues in the development set. The distribution in the Spanish multiclass training data is also similar: The majority class is "Not Hope" (5,383), followed by "Generalized Hope" (2,754), "Realistic Hope", and "Unrealistic Hope" with 1,113 and 1,300 samples, respectively. The lowest class, "Sarcasm", has 693 samples. The pattern for the development set is the same. This disparity, particularly for "Realistic" and "Unrealistic Hope", is a challenge to classification models favouring the majority classes. The more refined "Sarcasm" label brings a new layer of complexity where models must diferentiate between the subtle tone and the context. As such, using class-aware training techniques such as weighted loss functions and data augmentation is essential for better model performance, especially in under-represented categories.
The definition of hope in this work aligns with the PolyHope V2 dataset [ 15], where hope is understood as an emotional state that combines a positive desire with an optimistic expectation of future outcomes. In the binary classification setting, the dataset categorizes tweets as either Hope or Not Hope. In the multiclass setting, it further distinguishes five fine-grained categories: Not Hope, Generalized Hope, Realistic Hope, Unrealistic Hope, and Sarcasm. These categories were applied consistently across both English and Spanish annotations. Table 1 illustrates the dataset distribution across languages.
We used the Hugging Face Transformers library xlm-roberta-base to implement both binary and multiclass classification models for English and Spanish. The preprocessing pipeline consisted of the following steps: (1) text normalization (removing extra spaces, emojis, URLs, and converting to lowercase), (2) label mapping to integer values for classification (e.g., “Generalized Hope” = 0, “Not Hope” = 4), and (3) tokenization using the XLM-RobertaTokenizer, with a maximum sequence length of 256. To address the class imbalance, we computed class weights using inverse frequency from the training data and applied them in a custom loss function during training via the WeightedTrainer class. Data was split using k-fold cross-validation, and each subset was tokenized, padded, and batched using the DataCollatorWithPadding. All code was implemented using PyTorch with mixed precision (FP16) for training eficiency. The ensemble approach combined predictions from XLM-R and mBERT by averaging their output logits before applying the softmax function. This method was chosen over majority voting to preserve the probability distribution across all classes. Each model was fine-tuned separately with identical hyperparameters, and no additional meta-learner was used (i.e., no stacking). The ensemble helped stabilize predictions and improve recall in multiclass classification, especially for rare classes. Early stopping was applied based on macro F1 on the validation set to avoid overfitting.
The training was carried out in five epochs on hardware with a GPU and mixed precision (FP16) for faster training and optimized memory usage. Gradient accumulation was applied to overcome the GPU storage limitations. Early stopping was helped by using the validation macro F1 score to avoid overfitting. For the multiclass setup, we used a batch size of 16 and a cosine scheduler with restarts. For the binary classification setup, the batch size was 8, and the warm-up ratio increased to 0.2. The cosine learning rate schedule was used. The binary setup also applied higher gradient accumulation steps (4) and a lower max gradient norm (0.5). A tokenizer max length of 256 was implemented to normalize the lengths of the sequences. The input sizes within batches were made uniform using the DataCollatorWithPadding to maximize memory and time eficiency. In addition to accuracy and macro F1, we evaluated model performance using ROC (Receiver Operating Characteristic) curves and precision-recall (PR) curves to better assess behavior under class imbalance. These diagnostics helped visualize the trade-of between true positives and false positives, especially for underrepresented categories like Sarcasm and Unrealistic Hope. Curves were generated from validation predictions using Scikit-learn’s metrics module. Results showed excellent performance for balanced classification behavior, supported by macro F1 scores. For additional performance improvement, model ensembling was applied, averaging logits from several models. This led to an improved F1 score and more stable predictions. The results, including metrics and visualizations, were saved for reproducibility and further analysis.
Binary and multiclass analysis on English and Spanish was carried out with multilingual transformers. XLM-RoBERTa-Base did well in the binary setup, with macro average F1 scores of 0.8638 (English) and 0.8520 (Spanish), implying high reliability in identifying hope speech in the binary setup. The model remained moderately efective for multiclass classification, showing the result of 0.7139 (English) and 0.6901 (Spanish). These results determine the model’s good generalization in binary classification, and multiclass models proved more dificult because of label overlaps. Overall, XLM-RoBERTa showed strong multilingual performance across both tasks. Our team in the binary classification task ranked 8th out of 30 for English and 1st out of 11 for Spanish. For the multiclass classification task, the model achieved 14th out of 28 for English and 4th out of 8 for Spanish.
On both the binary and multiclass classification tasks, XLM-RoBERTa-Base model demonstrated superior performance to mBERT and ensemble models for the English and Spanish tasks. The model produced the best scores on macro F1 in the test set, 0.8638 (binary) and 0.7139 (multiclass) in English and 0.8520 (binary) and 0.6901 (multiclass) in Spanish. These results are a testament to the capacity of XLM-R to capture subtle semantic diferences (particularly within fine-grained hope categories). Whether tested on mBERT or not, XLM-R proved to have better recall and class separation, especially in multiclass settings where label overlap is an issue. Class-weighted training improved the detection of underrepresented categories, which the model could generalize well across tasks and languages. The observed lower performance for Spanish, especially in multiclass classification, is likely due to cultural and linguistic nuances in expressing hope, combined with more pronounced class imbalance and fewer semantic cues in the training data. XLM-R benefits English more due to the language’s dominance in its pretraining corpus. Nonetheless, XLM-R consistently outperformed mBERT because of deeper architecture, richer multilingual representations, and robustness to class imbalance through class-weighted training strategies. When compared with the PolyHope V2 baseline results [15], our system shows competitive performance. For English binary classification, our XLM-R model achieves a macro F1 of 86.38%, slightly outperforming the baseline RoBERTa score of 86.46%. In the English multiclass task, our macro F1 is 71.39%, closely matching the baseline of 75.87%. In Spanish, our model reaches 85.20% macro F1 (binary) and 69.01% (multiclass), compared to the baseline’s 83.97% and 72.81% respectively. These results validate the efectiveness of our multilingual training approach and suggest that XLM-R can achieve near state-of-the-art performance, even without architecture-specific optimization or handcrafted prompt engineering. Tables 2 and 3 compare the models’ F1 Scores across all languages for the binary and multiclass settings.
Substantial misclassification, particularly between closely related semantic categories, was found in both the English and Spanish datasets, indicating the model’s dificulty in accurately classifying nuanced expressions of Hope.
In the English binary classification task, the model confused 170 instances of Not Hope as Hope and 115 Hope instances were misclassified as Not Hope. These errors suggest challenges in drawing a clear boundary between neutral or ambiguous language and genuine hopeful expressions.
The English multiclass task revealed further confusion: 99 instances of Not Hope were misclassified as Generalized Hope, and 88 as Unrealistic Hope. Additionally, Generalized Hope was often mistaken for Realistic Hope. These confusions highlight the challenge of distinguishing between vague optimism (Generalized Hope), feasible positive expectations (Realistic Hope), and overly idealistic or improbable expressions (Unrealistic Hope), particularly when contextual cues are limited.
In Spanish, binary classification errors were even more pronounced, with 505 Not Hope texts misclassiifed as Hope, and 149 in the opposite direction. This suggests a language-specific issue in capturing subtle sentiment cues that indicate hopelessness or lack of positivity.
The Spanish multiclass task showed significant overlap between Not Hope and Generalized Hope (247 misclassifications) and Not Hope and Unrealistic Hope (280). There were also 130 confusions between Generalized and Realistic Hope. These patterns reflect the dificulty of modeling culturally embedded expressions and tone, particularly when sarcasm, exaggeration, or indirect emotional references are present. To gain deeper insight into these errors, we performed a qualitative review of misclassified instances. These findings underline the limitations of the current system in handling subtle tone, semantic ambiguity, and context-dependent meaning.
(a) English Binary (b) English Multiclass
The model often struggles when expressions of hope are implied rather than explicit, or when multiple emotional cues coexist in the same text. Additionally, overlapping definitions between categories (e.g., Generalized vs. Realistic Hope, or Unrealistic Hope vs. Not Hope) increase the likelihood of confusion. Annotation inconsistencies or ambiguous labels in the dataset may further contribute to these errors. Addressing these challenges would benefit from improved semantic representations, more consistent and context-aware annotation guidelines, and the integration of specialized modules such as emotion or sarcasm detection to enhance classification performance in future work. Figure 1a,1b,2a and 2b provides confusion matrices that visualize specific patterns related to misclassification.
This study assessed multilingual hope speech detection with XLM-RoBERTa-Base, mBERT, and an ensemble of the two models for English and Spanish for both binary and multiclass settings. Experiments revealed that XLM-RoBERTa-Base stabilised to record optimal performance, especially in distinguishing between fine-grained groups of hope speech. It showed good generalization and robustness across languages and outperformed baseline models in precision and recall. Though F1 scores from binary classification were higher overall, multiclass classification was more complex and emphasized the need for more sophisticated contextual modeling. In the future, more work may be done on covering low-resource languages and adding the cultural idiosyncrasies to improve speech recognition and enhance multilingual speech recognition.
The work was done with partial support from the Mexican Government through the grant A1-S-47854 of CONACYT, Mexico, grants 20241816, 20241819, and 20240951 of the Secretaría de Investigación y Posgrado of the Instituto Politécnico Nacional, Mexico. The authors thank the CONACYT for the computing resources brought to them through the Plataforma de Aprendizaje Profundo para Tecnologías del Lenguaje of the Laboratorio de Supercómputo of the INAOE, Mexico and acknowledge the support of Microsoft through the Microsoft Latin America PhD Award.
The authors acknowledge limited assistance from generative AI for text editing purposes. All conceptualization, experimentation, and interpretation are original human work. Responsibility for the accuracy and integrity of the manuscript rests solely with the authors. [15] S. Butt, F. Balouchzahi, A. I. Amjad, M. Amjad, H. G. Ceballos, S. M. Jimenez-Zafra, Optimism, expectation, or sarcasm? multi-class hope speech detection in spanish and english, arXiv preprint arXiv:2504.17974 (2025). [16] B. R. Chakravarthi, Hopeedi: A multilingual hope speech detection dataset for equality, diversity, and inclusion, in: Proceedings of the Third Workshop on Computational Modeling of People’s Opinions, Personality, and Emotion’s in Social Media, 2020, pp. 41–53. [17] M. Krasitskii, O. Kolesnikova, L. C. Hernandez, G. Sidorov, A. Gelbukh, Hope2023@ iberlef: A cross-linguistic exploration of hope speech detection in social media, in: Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2023), co-located with the 39th Conference of the Spanish Society for Natural Language Processing (SEPLN 2023), CEURWS. org, 2023. [18] Z. Ahani, G. Sidorov, O. Kolesnikova, A. F. Gelbukh, Zavira at hope2023@ iberlef: Hope speech detection from text using tf-idf features and machine learning algorithms., in: IberLEF@ SEPLN, 2023. [19] M. Tash, J. Armenta-Segura, Z. Ahani, O. Kolesnikova, G. Sidorov, A. Gelbukh, Lidoma@dravidianlangtech: Convolutional neural networks for studying correlation between lexical features and sentiment polarity in tamil and tulu languages, in: Proceedings of the Third Workshop on Speech and Language Technologies for Dravidian Languages, 2023, pp. 180–185. [20] M. Arif, M. S. Tash, A. Jamshidi, I. Ameer, F. Ullah, J. Kalita, A. Gelbukh, F. Balouchzahi, Exploring multidimensional aspects of hope speech computationally: A psycholinguistic and emotional perspective (2024). [21] L. Chiruzzo, S. M. Jiménez-Zafra, F. Rangel, Overview of iberlef 2024: Natural language processing challenges for spanish and other iberian languages, in: Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), Co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEUR-WS.org, 2024. [22] J. Á. González-Barba, L. Chiruzzo, S. M. Jiménez-Zafra, Overview of IberLEF 2025: Natural Language Processing Challenges for Spanish and other Iberian Languages, in: Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2025), co-located with the 41st Conference of the Spanish Society for Natural Language Processing (SEPLN 2025), CEUR-WS. org, 2025.