This paper presents our solution notebook for the HOMO-LAT@IberLEF2025 challenge, which consists of two tasks that require models that take advantage of subtle diferences in language between Spanish-speaking countries to identify the polarity of LGBTQI+ discourse in Reddit posts coming from Latin American Spanish-speaking countries. Our approach uses our BILMALAT model, a novel region-aware Large Language Model based on RoBERTa that is designed to understand and process Spanish variants from several countries, making it uniquely suited for capturing the diverse linguistic and cultural nuances present in the HOMO-LAT dataset. We show the high performance of BILMALAT in this challenge, demonstrating its eficacy in solution achieved the third position in both HOMO-LAT Tasks 1 and 2, solely fine-tuning the model for the task without extra strategies to balance the data and tackling large messages beyond the capacity of our model. hate and hope speech identification, multi-dialect Spanish language models, text classification models ∗Corresponding author.
Sentiment analysis, also known as opinion mining, applied to the online discourse on the LGBTQI+
community in Latin America, provides an indispensable framework for comprehending societal
perceptions and attitudes. Social media platforms, e.g., Reddit, which facilitate discussions, represent
extensive repositories of public sentiment, encompassing a spectrum from supportive to explicitly
harmful. Examining this content is essential to identify the predominant biases, monitor the
dissemination of discriminatory language, and evaluate the efectiveness of inclusion initiatives in various
Spanish-speaking contexts. This paper outlines our participation in the HOMO-LAT 2025 [
To promote safer online environments, Bel-Enguix et al. [
CEUR Workshop
ISSN1613-0073
[
When trained on extensive text datasets, it is acknowledged that transformer-based language models
(LMs) [
In addition, when considering the study of specific language variants, academic investigations such as
those cited in [
The HOMO-LAT of IberLEF 2025 asks systems to accurately predict the polarity of a Reddit post, which involves determining whether the post is positive, negative, or neutral regarding the LGBTQI+ community. These posts are composed in Spanish by users from various Latin American nations. Solutions should consider the regional context to account for local references and regionalisms that might ofer diferent interpretations, making it crucial to identify the intent of these communications. The posts are categorized into three types: identification of hate speech, hopeful messages, and sentiment analysis. The task comprises two subtasks: Task 1 was assessed using a test dataset from the same countries as in the training phase. Task 2 used a test consisting of messages from Bolivia, Costa Rica, Cuba, the Dominican Republic, Ecuador, El Salvador, Guatemala, Honduras, Nicaragua, Panama, Paraguay, Peru, Puerto Rico, Uruguay, and Venezuela.
The training set contains 5,700 Reddit posts and the development set includes 1,400 messages. Each post has at least one keyword from the following: trans, lgbt, gay/gays, lesbiana/as, bisexual/es, asexual/es, transexual/les, travesti/is, queer/s, transgénero, pansexual/es, intersexual/es. The training has 3174 posts classified as neutral, 1960 as negative, and 633 as positive. For the development partition, 888 are neutral, 475 negative, and 80 positive.
This section introduces the underlying opinion mining problem behind HOMO-LAT@IberLEF2025 and presents a review of the related work. Section 2 briefly describes our BILMALAT model and describes how we use it to solve the HOMO-LAT tasks. Section 3 details our experimental methodology and lists our results. Finally, conclusions are given in Section 4. 1Models and comparisons can be accessed at https://huggingface.co/dccuchile/patana-chilean-spanish-bert and https:// huggingface.co/dccuchile/tulio-chilean-spanish-bert.
Cuba _ar _co _cl _mx _uy _pa _pe _py _gt _ve _hn _sv _do _bo _ec _ni _cr _cu
Our approach was to fine-tune an early bird version of our BILMALAT 2 model. BILMALAT uses a RoBERTa-based training strategy on 600 million tweets from Latin America countries, trained from scratch. The model was designed such that the where and when of a given text is captured, i.e. it learns about regions and periods of time. Our first real-world application is to use it in HOMO-LAT; the model can be accessed and tested with the Huggingface framework 3.
For BILMALAT to be region- and period-aware, we use additional tokens to indicate the region. The first token corresponds to the country, the exact list is shown in Table 1. The second and third tokens correspond to the year and month of publication. The tokens can be _2015, _2016, … , _2023 and _01, … , _12 for the year and month, respectively. The rest of the tokens constitute the actual post.
Here, we describe our solution. We first used the BETO 4 model as a baseline. We fine-tunned the model in the training set for 6 epochs with a learning rate of 10−5. We conducted various experiments to choose these hyperparameters.
We then fine-tunned the BILMALAT with the messages provided. Since the training data included the country of origin, we could include the first token, for the second and third we simply used a generic x and _01 to force the model to predict without temporal bias 5. We fine-tuned the model using diferent hyperparameters.
Our experiments were carried out on Google Colab6. We employed the Huggingface framework with a Pytorch back-end. Our BETO baseline obtained a best accuracy of 0.5385 and an F1-score of 0.4487. 2https://huggingface.co/guillermoruiz/bilmaLAT 3https://huggingface.co/ 4https://huggingface.co/dccuchile/bert-base-spanish-wwm-cased 5For instance, our internal tests show that our model is aware of diferent meanings for terms like infección in COVID times and other periods; we need much more research to take advantage of this extra-information. 6Google Colab site https://colab.research.google.com/.
Our model selection procedure was quite simple, focused on fine-tuning the HOMO-LAT base-model; using the given training and development sets, we varied the learning rate in 10−4, 10−5, and 10−6; and looking for the best model among the computed epochs. This process was not systematic and was performed manually due to the lack of time. Table 2 displays these results for Task 1. We found that models with 10−5 show a higher performance, notoriously a learning rate of 10−6 produced very low scores, which means that we should give more epochs, but we were unable to do it due to our late joining the challenge. We did not fine-tune for task 2 and use the same model.
Our BILMALAT achieved a third place of seven in the gold standard with an F1 score of 0.5137 vs. 0.5296 of the best system. Our system also achieved third place (of seven) with an F1 score of 0.4639, where an F1 score 0.5086 was the best achieved.
This challenge was a perfect scenario to test our BILMALAT and compare with the popular alternative BETO model. Our approach, a language model enriched with geographical and cultural context, demonstrated a clear advantage in navigating the subtle expressions of sentiment prevalent in the dataset due to the message’s origin. Our model supports up to 128 input tokens, which might be enough for tweets, but the Reddit posts used in this task were much longer, which gives them greater complexity.
This result underscores the need for fine-grained regional awareness in natural language processing, moving beyond generic linguistic models to include local information in the training data. The insights gained from this competition provide a valuable contribution to the field of sentiment analysis and open the way for more sophisticated and culturally sensitive AI applications.
Due to our delayed awareness of the challenge, we had limited time to explore advanced strategies for task management. Consequently, we could not test sophisticated techniques like data augmentation or other methods to address the dataset’s inherent imbalance and the uneven distribution between training and development data. Additionally, similar to other BERT-based approaches, our model is constrained by sentence size, meaning it cannot handle huge messages without additional strategies, i.e., BILMALAT requires additional strategies to process large messages. With further focus on addressing the unique features of the challenge and overcoming the limitations of BERT-based models, our results can be enhanced.
During the preparation of this work, we applied the Writefull’s model for grammar and spelling checks. After using these services, we reviewed and edited the content as needed and assume full responsibility for the content of the publication. ume 1: Long Papers), Association for Computational Linguistics, Mexico City, Mexico, 2024, pp. 3689–3711. URL: https://aclanthology.org/2024.naacl-long.204. doi:10.18653/v1/2024.naacl-long. 204.