This paper details the participation of HULAT-UC3M research group at ADoBo 2025: Automatic Detection of Borrowings at IberLEF 2025. The architecture proposed tested several encoder-based transformers to classify anglicisms, being xlm-roberta-large-ner-spanish model the most efective. A pipeline designed to refine predictions by addressing specific error patterns was integrated to improve classification. Experiments were conducted using the COALAS dataset, demonstrating the model's capabilities and the pipeline's impact on performance, achieving a F1-score of 0.9182. The paper also includes some challenges encountered, such as handling words registered by the Spanish Royal Academy (RAE), proper nouns and tokenization artifacts. Finally, potential avenues for future research are outlined, including synthetic data generation and exploration of newer NLP models to further enhance anglicism detection accuracy.
Language is constantly evolving: including, deleting, and modifying new words and expressions each day, that usually are not registered in a dictionary. In the contemporary globalized world, each country has been influenced by other cultures and languages, leading to a significant influx of borrowings among languages. In the case of anglicisms we are talking about English words or expressions used in another language. The identification and analysis of these borrowed words are crucial for various linguistic and computational applications, such as lexicography, language teaching, machine translation, sentiment analysis and simplification.
This work describes the participation of HULAT-UC3M in ADoBo task [
The following sections will detail the methodology employed, starting with the data preparation and the initial model training. Subsequently, the selection process for the optimal model will be discussed, followed by an in-depth description of the pipeline designed to enhance the model’s performance. Finally, the experimentation phase, including the evaluation of the system on diferent datasets and the iterative improvements made, will be presented, culminating in a discussion of the results and potential avenues for future research.
Figure 1 describes the modules of our solution.
• Dataset preparation: Using the datasets library, a DatasetDict is created with the
partitions train, validation, and test, each pointing to a GitHub URL with the raw files from the
COALAS corpus [
Huggingface for the Spanish language, a fine-tuned version of bert-base-spanish-wwm-cased 1https://huggingface.co/MMG/xlm-roberta-large-ner-spanish 2https://huggingface.co/PlanTL-GOB-ES/roberta-base-bne-capitel-ner-plus 3https://huggingface.co/mrm8488/bert-spanish-cased-finetuned-ner 4https://huggingface.co/syubraj/spanish_bert_based_ner
for NER tasks using the Meddocan dataset [
The models were trained using the same hyperparameters: a weight decay of 0.01, a batch size of 16, a learning rate of 2e-5, and 5 epochs.
As can be seen in the results, the xlm-roberta-large-ner-spanish model stands out as the most efective overall, achieving the best performance in three of the evaluated metrics, in addition to obtaining excellent results in the others.
• Best F1: xlm-roberta-large-ner-spanish with a value of 0.9951. • Best overall F1: xlm-roberta-large-ner-spanish with a value of 0.9950. • Best F1 for anglicisms: xlm-roberta-large-ner-spanish with a value of 0.9277.
Although the model is capable of correctly detecting a large number of anglicisms, there are cases that it does not cover correctly, so a pipeline has been developed to amend these errors.
The goal of postprocessing is updating tags based on whether a given word or phrase is already registered in the dictionary of the Spanish Royal Academy (RAE). This is because even if a word comes from a language other than Spanish, the moment it is registered in the RAE, the word is not considered an anglicism anymore. This can happen for reasons such as the word having been used in our language for decades, as is the case with whiskey or jazz. In this way, various false positives that may be obtained from the model are resolved.
As next step spaCy model [
After adapting the RoBERTa model and developing the postprocessing, a series of tests were carried out to analyze the results. For these tests, the COALAS test dataset was used, which had already been used to obtain the model’s metrics. Table 3 shows the results just using the xlm-roberta-large-ner-spanish model.
As can be observed, these are good results, with an F1 score above 90%. Even so, it can be improved. Among the false positives obtained, there are diferent types of errors: There are words that the model simply interprets as anglicisms when they turn out to be Spanish, because they are anglicisms that have been adapted to Spanish. Some examples are cripto, derived from the anglicism crypto, or whiskazo, a verb formed from the word whiskey. There are also cases of words that, although they technically come from a foreign language, are listed in the RAE, so they do not count as anglicisms, such as videoclips or sake.
Other noteworthy cases not labeled as anglicisms following annotation rules [
Finally, the case of the word sms detected as an anglicism, and the digits 3, 25, and 10 are also incorrectly highlighted as anglicisms, although these cases should be resolved after using the pipeline. In the case of false negatives, we have anglicisms that the model has not recognized, such as webcams, SPOILERS, or workaholic. Table 4 shows the results integrating the postprocessing.
Comparing these metrics with those obtained using only xlm-roberta-large-ner-spanish model, a clear decrease in both recall and F1 score is observed. This is mainly because, although the number of false positives has more than halved, the number of false negatives has almost quadrupled.
Regarding false positives, the pipeline has correctly eliminated some cases, such as the digits 3 and 10, or words that do not count as anglicisms because they are recognized by the RAE, such as videoclips or sake. However, new false negatives have appeared, which can be divided into two types: those originating from spaCy, and those originating from querying RAE dictionary. To better appreciate which errors belonged to each stage of the pipeline, two new tests were conducted in which the model and only querying RAE dictionary or the spaCy model were used.
After an analysis of the results obtained, on the one hand, it was observed that although spaCy has a category for proper nouns, this category includes both proper and common nouns. Because of this, numerous nouns that were initially recognized as anglicisms cease to be so after being processed by spaCy, such as ecommerce or street. To solve this problem, an efective method to distinguish between common and proper nouns must clearly be sought.
Regarding RAE dictionary there are numerous anglicisms that appear int the dictionary although they have a diferent meaning. Some cases that can be highlighted from the errors obtained are, for example, late is an anglicism, which translated means tarde, but can also be understood in Spanish as a conjugation of the verb latir.
A possible solution that was considered was to search based on the lemma instead of the word itself but some are not recognized in RAE dictionary. Additionally, spaCy model did not correctly recognize the lemma in the case of anglicisms. This problem was solved by changing to the spaCy model trained for English, which obtained the lemma correctly and solved the problem. Although some cases were resolved, the number of errors was still higher than using the model alone, as there were anglicisms whose lemma was equal to its Spanish counterpart, such as the word me, a pronoun in English and Spanish, so it is only detected based on context. In the end, it was decided to discard querying RAE dictionary.
Finally, we decided to test the pipeline using only spaCy’s digit and email analysis, which resulted in a reduction of false positives. Even so, it was decided to analyze the results obtained more deeply, but this time, obtaining the model errors based on the number of tokens that made up the anglicism. Table 5 shows the analysis. We performed an analysis of the distribution of anglicisms in the dataset: • There are 917 single-word anglicisms, so almost a quarter are not detected as anglicisms. It is also noteworthy that there is a higher number of false positives in single-word anglicisms compared to other cases. • In the case of two-word anglicisms, there are 225 in the dataset, so proportionally, the number of false negatives is approximately 10%, while in the case of false positives, there are more than half the errors compared to the number of false negatives. One of the reasons discovered to explain this result is that there are anglicisms composed of multiple tokens, which are only partially detected. For example, for the anglicism endless - runner, composed of three tokens, endless and are detected as parts of the anglicism, but runner is not, which causes not only a false positive but also a false negative, since an error has been detected for the same anglicism because it is incomplete, and another for not detecting the entire anglicism. Some of the false negatives are anglicisms that are partially detected and other false positives such as are Future - ready and look working girl. The first error is a company name in the sentence: La nueva colección Future ready de Bershka hace aún más apetecible el streetwear, so although the words that make up the name are not from our language, it does not count as an anglicism. The second case is a bit more complicated. Look working girl are two anglicisms, Look and working girl, as they come from the sentence Tamara Falcó apuesta por un look working girl con chaleco en El Hormiguero, so the error is that the model detects these three words as if they belong to the same anglicism, although the reality is that they are two diferent anglicisms, causing a false positive and two false negatives for the two undetected anglicisms. • Although there are 88 3-token anglicisms in the dataset, the number of errors is drastically reduced.
Although proportionally, there is a higher number of 3-token false negatives than 2-token false negatives, we only find two false positives. • Finally, in the case of 4-or-more-token anglicisms, there are only 3 false positives and no false negatives. Two of the false positives are because, although the errors are composed of foreign words, they are not anglicisms due to grammatical rules, and the other because two anglicisms together have been detected as a single anglicism.
After this exploration of false positives and false negatives, we could observe how the highest percentage of false positives and false negatives are found in single-token anglicisms, while the percentage of errors is reduced in the other cases. Also noteworthy are those errors due to the partial detection of some anglicisms, although for the moment, no possible solution is contemplated by modifying the postprocessing to solve this problem; rather, the model itself would have to be improved.
Once the experiments were finished with the COALAS datasets, the datasets given for the task in the development phase were used. Table 6 shows the results using the roBERTa model and postprocessing phase.
As can be observed, if we compare these results with those obtained using the COALAS test dataset, they are considerably worse. After analyzing the results, it was discovered that this was mainly due to two reasons: • There were numerous false positives and false negatives due to the diference in formats. The developed pipeline had been created to work with the CoNLL format, so when creating the CSV, the anglicisms were copied into the CSV separated by tokens, and not based on the entire anglicism. This problem was solved • Problems detecting anglicisms composed of multiple words, but for those of only one word, they had been evaluated correctly. Something that caught attention with these errors was that a large number of the false negatives obtained were in quotation marks. For example, in the sentence: "Somos un país en el que ‘youtubers’ y ‘gamers’ millonarios deciden irse al paraíso fiscal de Andorra porque tributan menos sin preocuparse del bienestar de sus vecinos y de quienes les han hecho ricos, ni acordarse de la Educación, Sanidad, infraestructuras de las que han disfrutado durante años gracias a la solidaridad de todos", the anglicisms youtubers and gamers were not detected, unless the quotation marks were removed from the sentence. Because of this, a preprocessing step was added to the pipeline which would remove all single and double quotation marks from the text before passing it to the model.
After all these changes, the pipeline was used again and Table 6 shows the new results. A great improvement can be observed compared to previous tests, although multiple false negatives which contained some punctuation marks were found. Some examples are poetry slam?, dance cover), and deepfakes:. As can be seen, although the anglicisms were technically detected correctly, a tokenizer error led to the mistakes. To solve this, a post-processing stage was added to the pipeline, in which, for each token detected as an anglicism, punctuation marks would be removed if present in the token.
In the rest of the errors, no common characteristics or mistakes were detected that could be solved by modifying the postprocessing phase. Table 6 shows the best results.
Table 6 shows the results obtained with the final test dataset provided by the task organizers.
These results are likely due to the format of the sentences in the dataset, as there are multiple repeated sentences in diferent forms, for example: • """Eyeliner"" a todo color para para un verano fantástico" • Eyeliner a todo color para para un verano fantástico • """eyeliner"" a todo color para para un verano fantástico" • """Eyeliner"" A Todo Color Para Para Un Verano Fantástico" • """EYELINER"" A TODO COLOR PARA PARA UN VERANO FANTÁSTICO" • """Eyeliner"" a todo color para para un verano fantástico" • """EYELINER"" a todo color para para un verano fantástico" • eyeliner a todo color para para un verano fantástico • Eyeliner A Todo Color Para Para Un Verano Fantástico • EYELINER A TODO COLOR PARA PARA UN VERANO FANTÁSTICO • Eyeliner a todo color para para un verano fantástico • EYELINER a todo color para para un verano fantástico
To solve this problem, it was decided to normalize the text before being used by the model, as the diferences in the sentences seemed to afect the results obtained. The normalization consisted of eliminating the single and double marks from the text, and converting uppercase letters to lowercase. Finally, the table 6 shows the best metrics obtained.
After multiple tests, a model with an F1 score greater than 90% for predicting anglicisms has finally been obtained. Although it is a good score, it does not mean that a better model cannot be obtained. Firstly, as we have tested, increasing the training data improves the model’s efectiveness. For this project, data sources provided by the 2021 and 2025 ADoBo editions tasks have been used, but other options were considered to increase the available data.
One option that was contemplated was to generate synthetic data. The idea was to use a generative
artificial intelligence which, through various prompts, would generate sentences containing an anglicism.
The sentences would not need to be true, only syntactically meaningful, which AI does excellently, so
we could pass it a series of anglicisms to construct new sentences. These anglicisms could be obtained
from various data sources. For this project, multiple alternative data sources were investigated in case
they were needed:
• Anglicor, a database of anglicisms in the Spanish language [
Another possible option would be to change the model used. In the ADoBo 2021 task, the best model obtained an F1 score of 84.8, while the model chosen for this project, xlm-roberta-large-ner-spanish, has surpassed it. This project was last modified in 2023, two years after the ADoBo 2021 task. New, larger, and better-optimized models appear every day, so it would not be surprising if, in the future, a model appears that surpasses the one used.
Finally, one could also continue iterating on the pipeline. Although several ideas for improving it have been discarded, there are surely other options that would allow the model to improve its results, such as the ability to discard anglicisms that are proper nouns, or to discard those anglicisms that have been adapted to Spanish, such as cripto or craftear.
In the end, there are multiple options to continue exploring, and with suficient research and work, it will surely be possible to continue obtaining better results in the task of anglicism detection.
This work has been supported by grant PID2023-148577OB-C21 (Human-Centered AI: User-Driven Adapted Language Models-HUMAN_AI) by MICIU/AEI/ 10.13039/501100011033 and by FEDER/UE.
During the preparation of this work, the author(s) used GPT-4 in order to: grammar and spelling check and to translate correctly some sentences in the text. After using these tool(s)/service(s), the author(s) reviewed and edited the content as needed and take(s) full responsibility for the publication’s content. [13] M. P. Castillo Bernal, Diccionario de anglicismos del español estadounidense, by francisco morenofernández, ed. instituto cervantes at the faculty of arts and sciences of harvard university, 2018. informes del observatorio / observatorio reports. 037-01/2018 sp, isbn: 978-0-692-04726-2 doi: 10.15427/or037-01/2018sp, Transletters. 1 (2018) 195–196. URL: https://www.uco.es/ucopress/ojs/ index.php/tl/article/view/11042. [14] D. S. Iglesias, Diccionario de anglicismos y otros extranjerismos, Defensa del Idioma Español, España, 2018. URL: https://defensadelidioma.com/wp-content/uploads/2018/09/ diccionario-de-anglicismos-y-otros-extranjerismos.pdf, accessed: 2025-05-27.