We present LBAD, an LLM-Based Anglicism Detector, our submission to the ADoBo 2025 shared task at IberLEF 2025. We show that, with careful prompting, commercially available LLMs achieve state-of-theart performance on the task of Anglicism detection. We systematically evaluate the impact of model selection and prompting, including levels of instruction, chain-of-thought reasoning, few-shot learning, and self-refinement. Our experiments reveal that prompt specificity and model selection are critical, with F1 validation scores varying by up to 75 percentage points depending on these two factors. These ifndings highlight the practical efectiveness of commercial LLMs for lexical borrowing detection and provide a generalizable framework for leveraging LLMs in similar natural language processing tasks.
CEUR ISSN1613-0073
Lexical borrowing, the process by which words from one language are incorporated into another,
is a pervasive linguistic phenomenon driven by cultural, social, and technological interactions
between linguistic communities [
The automatic detection of these borrowed terms, particularly unassimilated ones that retain
their original orthography and are not yet fully integrated into the recipient language’s lexicon,
presents a challenge for Natural Language Processing (NLP). Identifying such borrowings
is important for downstream applications, including lexicography, machine translation [
The ADoBo (Automatic Detection of Borrowings) shared task series aims to foster research
and development in this area. The second edition of the ADoBo shared task at IberLEF 2025 [
The challenge of automatically detecting unassimilated borrowings, especially Anglicisms, in
Spanish has been approached from various perspectives. Early computational methods often
relied on dictionary lookups and rule-based systems [
In the years since the first ADoBo challenge, Large Language Models (LLMs) have become
the de facto approach for a variety of research tasks across disciplines. Recent LLMs have
demonstrated human-level (or superhuman) performance on data coding [
While computational NLP tasks have typically been somewhat inaccessible to the general
public, LLMs provide a powerful new tool for non-technical practitioners. LLMs are much more
accessible than traditional machine learning methods. [
Built on commercial LLMs, our submission to the ADoBo 2025 shared task achieves state-ofthe-art results. We provide a detailed description of our prompt creation process, as well as experiments across prompt variations and model sizes. While our main focus is on detailing our approach to prompting LLMs for the task of Anglicism detection in Spanish, we hope to provide a generalizable example of using commercial LLMs for NLP tasks.
We perform experiments across models and prompt variations. Prompt variations include: • Baseline prompting • Prompt with detailed guidelines • Chain-of-thought prompting • Few Shot prompting (with a varying number of in-context examples) • Self-refined prompt
In order to measure the efect of model size, we test on a full sweep of OpenAI’s latest
generation of commercially available models, including reasoning models [
The ADoBo-25 challenge has no oficial train set, only a development set and a final test set. For testing purposes, we adapt the ADoBo-25 dev set into our own train, test, and validation sets, which we call LBAD-train, LBAD-test, and LBAD-valid.1 We do this by randomly shufling the ADoBo-25 development set and splitting it into thirds. We draw from LBAD-train to find example Anglicisms for our prompts. Because prompt creation is an iterative process, we test our prompts and iterate using the LBAD-test set. Finally, we report results on all experiments in this paper are on LBAD-valid. Because some training and testing occurs on parts of the ADoBo-25 dev set, we do not report results on the complete dev set. However, because LBADvalid is randomly drawn from the ADoBo-25 dev set, we assert that results on LBAD-valid are a reasonable proxy for results on the complete dev set.
All of the prompts in this experiment are built of of the following baseline prompt, adapted from the ADoBo 2025 task description:
Given the following Spanish sentence, identify all anglicisms.
An anglicism is a word or multi-word expression borrowed specifically from English that has recently been imported into the Spanish language and is used without orthographic adaptation.
Examples of anglicisms include 'running', 'smartwatch', 'influencer', 'country managers', 'marketing'.
This is the sentence to evaluate: {sentence} Output only the identified anglicisms, separated by semicolons. If no anglicisms are found, output 'None'.
Because modern LLMs are explicitly trained to follow natural language instructions, properly formalizing and explaining the task is necessary to ensure peak performance.
To create a set of comprehensive guidelines, we pass the entire lexical borrowing annotation
guidelines from the ADoBo 2021 challenge [
We perform experiments using Chain-of-thought prompting [
First, think step-by-step about which words/phrases might be anglicisms and why.
After thinking things through, on a new line, output only the identified
anglicisms, separated by semicolons. If no anglicisms are found, output
'None'.
2.1.4. Few-Shot
Language models are capable of in-context learning [
We test prompt variations with five, ten, and twenty-five ICL examples. ICL examples were selected from the LBAD-train dataset using R1 1776, which was instructed to reason through the entire LBAD-train dataset and select the five, ten, or twenty-five most representative examples of Anglicism detection. Examples were added to the baseline prompt with the following addition (right before the sentence to evaluate):
Here are some example sentences, with the correct output on the line following the sentences:
LLMs are capable of refining their own outputs, as well as those of other LLMs. Several
self-refinement frameworks allow LLMs to recursively refine their own prompts [
Inspired by these frameworks, we begin by testing GPT 4.1 using the prompt with detailed guidelines on the LBAD-test dataset. We then pass those answers and the ground truth to R1 1776, and instruct it to classify GPT 4.1’s failure modes and then write a series of helpful reminders which can be added to the prompt before validation. These reminders are added to the prompt after the detailed guidelines. The full text of the reminders can be found in the appendix.
LLM performance can vary wildly depending on the size of the model, with smaller models
tending to perform worse than models with more trainable parameters [
Reasoning models [
We report F1 scores on LBAD-valid for all experiments. Following the convention from the task leaderboard, we round all scores to the nearest whole percent. To facilitate comparison with other entries, we also report precision, accuracy, and F1 scores on the ADoBo 2025 test set for our best performing prompt/model combination. Tables of all scores, including precision, recall, and F1 can be found in the appendix.
An examination of the detailed score breakdown tells us something about commercial LLMs’ strengths. Across all models and prompts, recall scores are high. Most variation in F1 scores is due to poor precision. In other words, LBAD tends to find almost all anglicisms, but often misidentifies non-anglicism spans.
We see a gap in F1 scores between LBAD-valid and the ADoBo-2025 test set, with LBAD-valid scores being lower. This is attributable to the fact that roughly half of LBAD-valid contains no anglicisms, but the ADoBo-2025 test set is unbalanced in favor of sentences with Anglicisms. As a result, the ADoBo-2025 test set is more in line with LBAD’s strengths.
Qualitative analysis of the best performance on ADoBo-2025 reveals where even the topperforming model struggles. The o3 model with Guidelines and Reminders fails 25 times. These failures comprise two main failure modes.
Several of the failures result from the model misidentifying a multi-word span as two single word spans, or identifying two single-word spans as one multi-word span. For example in the sentence "Casual Looks" con bufanda y guantes para triunfar esta temporada the model is supposed to identify Casual Looks as the Anglicism, but the model identifies Casual and Looks separately. Seven of the 25 errors (28%) have to do with incorrectly parsing span boundaries.
Many words in this set are orthographically identical in English and Spanish. (e.g. normal, total, error) In these cases, the model will misidentify one or more Anglicisms as Spanish words. As an example, given the sentence Durante su carrera profesional, también ha sido socio y director general de Ikea, global director de Apple y director de Comunicación de Basket Market. the model incorrectly guesses no Anglicisms are present, though the ground truth is global director. Because global and director are orthographically the same in Spanish and English, the model fails to detect this Anglicism although the syntax (global before director is English syntax) provides a clue that this is an Anglicism. Seventeen of the 25 errors (68%) are partial or complete misidentifications of words that share cross-lingual orthography.
These two failure modes explain almost all of LBAD’s errors on the ADoBo-25 test set. It is likely that additional rounds of prompt refinement could help the model avoid these failure modes in the future.
Anglicism detection is a challenging task with many particulars and precise rules. The baseline prompt relies heavily on models’ inherent knowledge of Anglicisms, since it provides only an outline of the task. We see evidence of this in the fact that baseline performance tends to increase with larger models. Larger models have more parameters in which to store information about the world. Consequently, they have more world knowledge across many topics. This world knowledge likely includes information about NLP and Anglicisms.
We find that an in-depth task description, provided in the form of detailed guidelines, is the best way to teach the task to all LLMs tested. Across models, detailed guidelines tended to cause an increase in performance between 10 and 45 percentage points over the baseline prompt. This increase was larger for larger models, which are better able to think through the Anglicism identification process. By far, the most important prompt variation for improving performance was chain-of-thought reasoning. Whether prompt-based or (in the case of reasoning models) explicitly trained, models utilizing chain-of-thought reasoning performed best. This approach worked best in conjunction with detailed guidelines, as the detailed guidelines provided a prototype for model reasoning, and the chain-of-thought helped the models think through the process. 3.2.3. Few-Shot Few-shot learning was less efective than chain-of-thought reasoning, regardless of the number of ICL examples. More examples were not necessarily better. Because Anglicism detection involves many subtleties, it is dificult to learn all of the rules from examples. We note that ICL examples do not appear to provide additional information when combined with detailed guidelines. To the contrary, ICL examples when combined with the detailed guidelines often caused a drop in performance as opposed to the baseline prompt with guidelines, probably by “confusing” the LLM.
Self-Refinement through the form of helpful reminders often increases performance across models, though the gains are modest. Three of the models improve with reminders, one achieves the same score, and one decreases by one percentage point. This may be due to a ceiling efect, as models with worse initial performance gain more from the reminders. These modest results should not discourage practitioners from trying iterative prompt refinement with LLMs. It is probable that additional human-based refinement of the reminders would result in greater performance gains, but that is beyond the scope of our experiments.
LBAD was the best performing system on the ADoBo-2025 challenge, and was built entirely of of commercial LLMs. Language models were used not only as the method of evaluation, but also to summarize the task guidelines, to analyze failure modes, and to select in-context examples. While LLMs are demonstrably capable of achieving state-of-the-art results, we emphasize that this capability depends heavily on model and prompt selection. In our experiments, F1 scores on LBAD-valid ranged from 12 to 90 simply by varying prompt and model choice. This 78 percentage point spread highlights the importance of testing prompts and iterating when using LLMs for research. The fact that a carefully prompted LLM can achieve SOTA performance on this task should encourage researchers to consider using LLMs for NLP.
The topic of this work is Generative AI (LLMs) and explores LLM capabilities in the context of Anglicism Detection. The only use of generative AI on this project was the LLM-based pipeline described in the paper. AI tools were not used in the drafting or editing of this manuscript.
You are an expert evaluator.
The following rules provide a robust, step-by-step protocol for identifying emergent, unassimilated English lexical borrowings (anglicisms) in Spanish text. 1. Initial Identification - Determine if the word or multiword expression is of English origin or mimics English word formation.
- If the word is not of English origin, do not consider it an anglicism. - If it mimics English formation but does not exist in English (pseudoanglicism, e.g., balconing), consider it an anglicism. 2. Exclusion of Proper Names and Code-Mixed Inclusions - If the word is a proper noun (person, organization, product, location, event, etc.) or a direct code-mixed quotation, do not consider it an anglicism. - Borrowings embedded within proper nouns or named entities do not count, unless the proper noun is part of a multiword borrowing used grammatically as in English (e.g., Google cooking is annotated). 3. Graphophonological Compliance - If the word's spelling and pronunciation conform to Spanish graphophonological rules (e.g., bar, club), proceed to dictionary checks. - If not (e.g., show, look), generally consider it an anglicism unless it is a long-registered realia word (see Rule 5). 4. Adaptation and Assimilation Status - If the word has been morphologically or orthographically adapted to Spanish (e.g., fútbol, tuit, líder), do not consider it an anglicism. - If the word remains unadapted, continue to dictionary checks. 5. Dictionary Verification - If the unadapted word is registered in the Diccionario de la Lengua Española (DLE): - If it appears in italics, consider it an anglicism.
- If it appears without italics and with the relevant meaning, do not consider it an anglicism (it is considered assimilated).
- If it is not registered or not with the relevant meaning, consider it an anglicism. - For multi-sense words (e.g., top), only consider them an anglicism when used with unregistered meanings. 6. Realia and Long-Registered Borrowings - If the word is a long-registered realia borrowing (cultural terms like jazz, pizza, whisky, club), do not consider it an anglicism, even if unadapted. - If the word is a recent or emergent realia borrowing not yet registered, consider it an anglicism. 7. Multiword Borrowings - Do consider multiword expressions borrowed as a unit from English (e.g., reality show, best seller). - For adjacent borrowings not forming a fixed English phrase (e.g., look sporty), select each word separately. 8. Exclusions and Special Cases - Do not consider an anglicism:
- Latinisms, scientific units, species names, acronyms (unless part of a multiword borrowing), or digits in isolation.
- Metalinguistic usages, literal quotations, or code-switched expressions not integrated into the sentence.
- Names of peoples or languages, and words derived transparently from proper nouns (e.g., un iPhone, un whatsapp). - Do consider an anglicism:
- Pseudoanglicisms (Spanish-coined words mimicking English, e.g., footing, balconing).
- Unadapted names of fictitious creatures (e.g., hobbit, troll). - Borrowings embedded in compounds or prefixed forms, if the borrowed element retains independence (e.g., ex influencer, nano influencers).
Because these rules refer to the DLE and dictionary checks, we add the following line to prevent model confusion:
The rules refer to dictionary checks. You don't have access to a dictionary, so do the best you can.
Examples of words that are NOT anglicisms include: - Words fully adapted to Spanish orthography (e.g., 'fútbol', 'líder', 'tuit') - Long-established borrowings (e.g., 'bar', 'club', 'jazz', 'whisky') - Proper names of people, places, or companies (e.g., 'Twitter' as a company name) - Scientific terminology with Latin or Greek roots Special rules for digital media and technology: 1. Names of online platforms (e.g., 'Facebook', 'Twitter') are NOT anglicisms when used as proper nouns to refer to the specific company or service. 2. However, these names ARE anglicisms when used generically (e.g., "Hizo un twitter" to mean "He made a tweet"). 3. Generic terms related to digital culture ARE anglicisms when they maintain English form (e.g., 'post', 'blog', 'meme', 'podcast', 'streaming'). 4. Terms for social media actions in their English form ARE anglicisms (e.g., 'like', 'share', 'tweet').
For multi-word expressions: 1. Analyze whether the entire expression functions as a unified borrowing from English. 2. Examples of multi-word anglicisms: 'fast food', 'big data', 'home office', 'fake news'. 3. Do NOT count individual English words that appear adjacent to each other but do not form a standard expression in English. 4. When in doubt about whether multiple words form a single anglicism, consider them separately.
For business, finance, and technical terminology: 1. Recent business terms that maintain English spelling ARE anglicisms (e.g., 'CEO', 'manager', 'marketing', 'branding', 'broker'). 2. Technical computing terms that maintain English spelling ARE anglicisms (e.g., 'hardware', 'software', 'online', 'web'). 3. Industry-specific English jargon ARE anglicisms (e.g., 'blockchain', 'big data', 'know-how'). 4. International brand names and trademarks are NOT anglicisms unless used generically.
Follow this step-by-step process for identifying anglicisms: 1. First, identify all words and phrases that appear to have English origin. 2. For each candidate term, apply the exclusion criteria (proper names, adapted words, etc.). 3. For remaining terms, verify they maintain English spelling/pronunciation patterns. 4. Group related words that form a single expression in English. 5. Verify each term against the specific rules for digital media, business terms, etc.
All models tested are April checkpoints of OpenAI models. • gpt-4.1-nano-2025-04-14 • gpt-4.1-mini-2025-04-14 • gpt-4.1-2025-04-14 • o4-mini-2025-04-16 • o3-2025-04-16
Scores reported in this section are rounded to the nearest whole number. 37 57 51 39 68 67 38 46 42 55 39 47 Baseline Baseline + Guidelines Baseline + Guidelines + Reminders