This paper explores various approaches to the task of text detoxication, including the use of lexical resources (such as toxic word dictionaries), methods based on deep learning algorithms (T5, VAE, etc.) and modern large language models (LLMs). The study is conducted on data from PAN: Multilingual Text Detoxication (TextDetox) 2025 - with the aim of identifying the most eective strategies for handling toxic language depending on linguistic specics. It is shown that for low-resource languages, like Tatar, dictionary-based methods show the highest eectiveness. For widely spoken languages, such as English, deep learning methods show better quality. The results highlight the importance of considering linguistic features when selecting a detoxication method and open up possibilities for the further development of adaptive multilingual content ltering systems.
The rapid growth of online communication platforms has led to an increasing amount of user-generated content, which ofien includes toxic, aggressive, or otherwise harmful language. This phenomenon poses a serious challenge for maintaining safe and inclusive digital environments. One of the key solutions to this issue lies in the development of automated systems capable of detecting and neutralizing toxic language — a task commonly referred to as text detoxication.
Text detoxication involves transforming a given text in such a way that its toxic or oensive elements are removed or sofiened, while preserving the original meaning and stylistic coherence. The eld of NLP has seen rapid development in recent years, enabling diverse approaches to address the problem of text detoxication.
Each of these approaches comes with its own strengths and limitations. Lexical methods, such as
ltering based on lists of toxic words, oer simplicity and speed but ofien fail to account for context,
sarcasm, or subtle forms of toxicity. Deep learning models provide better contextual understanding
when trained on high-quality annotated datasets[
This paper presents a comparative analysis of dierent detoxication strategies across multiple
languages, emphasizing how linguistic characteristics inuence the eectiveness of each method. We
argue that no single approach is universally optimal and that the choice of method should be guided
by the specic properties of the target language. This study utilizes data from PAN: Multilingual
Text Detoxication (TextDetox) 2025[
In modern linguistics, toxic words are commonly referred to as invective vocabulary. The invective lexicon of a language is enriched through several sources: reinterpretation of existing literary units, borrowings from foreign languages, and slang. Importantly, invective vocabulary diers from slang in terms of stability: while slang undergoes frequent changes and updates, the core set of invectives tends to remain relatively stable over long periods.
As an example, consider American English, where a signicant source of lexical borrowings is
AfricanAmerican vernacular, including such expressions as "motherf*cker" or "of*y". However, invectives should
be distinguished from colloquial speech, which may appear similar but lacks the same emotionally
charged connotation. Colloquial words such as "sh*t" or "a*s", although considered vulgar in formal
contexts, can function as neutral terms in informal speech — especially within certain social groups[
Thus, the meaning and communicative function of the same word depend on the speaker’s social status and context, leading to an overlap between invective and colloquial vocabulary. This process facilitates lexical exchange between these two linguistic layers.
The formation of invective vocabulary can be classied into several key directions. First, many invectives originate from colloquial, slang, or marginal vocabulary ofien used to express social dierences and conicts. Second, a major contribution to the development of invectives comes from pejoration — the semantic re-evaluation of literary and conversational words, during which originally neutral or positive meanings transform into negative ones.
Linguist I.I. Kremikh identies several types of pejoration, among which metaphorical pejoration is considered the most productive. It is based on the transfer of negative meaning to an object through similarity or association. Examples include the Spanish words "globo" ("balloon"), used metaphorically to refer to a fat person, and "buzo´n de correos" ("mailbox"), used to describe someone with a large mouth.
Of particular importance in invective vocabulary are zoomorphisms — metaphors in which people are compared to animals based on stereotypical associations. For instance, the Spanish words "cerdo" and "cochino" (both meaning "pig") or the English "pig" and "dirty dog" are used to describe messy or rude individuals. Similarly, repurposed names of fruits and vegetables, such as "melo´n" ("melon") and "carrot" (used derogatorily for red-haired people), illustrate how stable phonetic forms can undergo substantial semantic transformation and shifi in communicative function.
An interesting phenomenon is the positive use of invectives. In some cases, these expressions are
used to convey admiration or praise, reecting their deep integration into everyday speech and the
wide range of emotions they can convey. For instance, in the Tatar language, the word "эт" ("dog"),
which functions as an invective, appears in the phrase "Ах, эт, эшне ничек оста хәл иткэн!" ("Wow,
what a masterful way to solve the problem!"), where it serves as a compliment to one’s skills. Such
expressive usage of invectives is also widely employed in literature to reect natural, vivid speech[
Ethnolinguistic factors play a signicant role in the formation and functioning of invective vocabulary, especially evident when comparing related or contact languages. The Tatar language, with its rich historical and cultural layer, contains numerous invectives whose semantics are closely tied to collective memory and religious identity. A comparative analysis of Tatar invectives with corresponding units in Bashkir, Chuvash, Udmurt, Mari, and Turkish — a fellow Turkic language — reveals consistent patterns in the development of oensive vocabulary and its national-cultural characteristics.
For example, in Tatar, the lexemes "таре" ("christian cross", "damned") and "чукынган" ("christened", and in an invective sense — "damned") emerged under the inuence of the Christianization of Volga Tatars in the 16th–18th centuries. Originally neutral or positively connotated, these words transformed into strong invectives with clearly negative semantics within the Muslim community. The invective "чукынган" demonstrates high derivational productivity, generating forms such as "чукынчык" ("scoundrel"), "чукынып кит" ("disappear!", "go to hell!"), "чукынды" ("all is lost"), and "чукынгыры" ("may he perish"), which can serve as insults, curses, or exclamatory interjections.
It is important to note that the strength of the invective tone of these words depends on the religious and cultural identity of the addressee: for Muslims, they carry a clearly negative nuance, while for Christians, they retain a neutral or even positive meaning. Similar semantic shifis are observed in Russian as well (e.g., "нехристь" ("heathen") and "креста на тебе нет" ("no cross, no crown"), originally used to refer to people of dierent faiths).
Thus, invective vocabulary serves not only as a means of expressing insult but also as a marker of deep cultural-historical layers and boundaries of linguistic consciousness, reecting complex processes of identity interaction within society.
One of the simplest and fastest ways to reduce the level of toxicity in text is the dictionary-based approach, which involves replacing potentially toxic words with neutral equivalents or completely removing oensive terms. The replacement can be either full (replacing the entire word) or partial (applied when a toxic word appears as part of a longer lexical unit).
We use a Multilingual Toxic Lexicon [
Copy base method consisted in the fact that the toxic was not changed in any way. In other words, we looked at the metrics of unmodied sentences. Replace method involved removing the toxic component from a word. For example, if the dictionary includes the word "f*ck", but "f*cker" not in the toxic dictionary, only the sux "er" would remain afier processing.
Delete method consisted of removing an entire word if it contained any toxic substring. For instance, if the toxic dictionary included the word "f*ck", but "f*cker" was not included, then full word "f*cker" would be deleted.
Results of lexical methods presented in Table 1. The values in the table are given for the Joint metric.
However, in some languages, such as Ukrainian and Japanese, the replace method demonstrated signicantly better metrics. This eect may be attributed to specic contextual features that inadvertently amplify the negative tone of the text during word replacement. Delete method showed signicantly better results for the Arabic language. Therefore, when applying automated detoxication methods, it is essential to take into account the linguistic structure and contextual usage of words.
The highest level of toxicity was observed when using the copy base method, where the input text remained unchanged. Although this approach preserves the original meaning of the message, it does not address the task of reducing toxicity and should only be used as a baseline for comparison.
The purely dictionary-based approach has several limitations, including an inability to account for context, the need for regular dictionary updates, and the risk of false positives. Hence, a promising direction for future work appears to be the combination of lexical ltering with text rewriting models, which can not only eliminate toxic elements but also restore textual structure and preserve semantic meaning.
A more recent solution in the eld of text detoxication involves approaches based on deep learning
models, such as T5 (Text-To-Text Transfer Transformer) [
Such models demonstrate strong performance in neutralizing oensive language without signicantly altering the meaning of the original message. Their eectiveness is especially evident when they are ne-tuned on high-quality detoxication datasets. A more detailed discussion of these experiments will follow in the section on large language models (LLMs).
For the current set of experiments, we used the HRQ-VAE [
The experiments were conducted on the English language using both the pretrained version of the
model and several versions ne-tuned on the Paradetox [
Results of dierent HRQ-VAE models for English presented in Table 2.The values in the table are given for the Joint metric.
Our ndings show that the model’s performance depends heavily on the quality of the training data and the strategy used to form paraphrase clusters. In some congurations, the model was able to signicantly reduce toxicity levels, while in others, it failed to fully eliminate harmful content or even preserved negative connotations.
In summary, the HRQ-VAE model demonstrates potential for use in text detoxication, particularly when ne-tuned on carefully curated parallel datasets. However, like other methods, it requires careful tuning and contextual awareness to avoid unintended preservation or amplication of toxic tone.
A modern stage in the development of text models is represented by large language models (LLMs) such as ChatGPT [16], Llama3 [17] and others. These models possess signicant capacity for understanding and generating natural language, making them promising candidates for solving complex tasks, including text detoxication. Unlike traditional approaches based on rigid rules or simple neural architectures, LLMs are capable not only of removing or replacing potentially toxic words, but also of rewriting phrases while preserving the original meaning, style, and logical coherence.
One of the key advantages of LLMs is their ability to interpret user instructions through prompts, which enables exible control over the generation process. For example, prompts such as "Make this more neutral" or "Paraphrase this sentence" allow the model to understand that the tone of the text needs to be adjusted without sacricing its informational content. This makes LLMs particularly suitable for use in real-world content ltering systems, where both the reduction of toxicity and the maintenance of communication quality are essential.
In the course of this study, several versions of the Flan-T5 [18] model (small, large) were tested. These models were initially trained on the Paradetox dataset and further ne-tuned on a mixed dataset that included Paradetox, Parallel Detoxication Dataset small [19], Paranmt-for-detox [20], and Filtered_paranmt [21]. The training was conducted using dierent prompts, such as "Make more neutral" and "Detoxify", which allowed us to assess the impact of prompt formulation on the quality of text rewriting. mT0 [22] model from the TextDetox 2024 Multilingual Text Detoxication task.
Results of dierent prompts presented in Table 3. The values in the table are given for the Joint metric.
The results demonstrated that the Flan-T5-large variant outperformed smaller versions in terms of detoxication performance. Specically, afier 4696 training steps, the joint metric reached 0.70497 when using the "Make more neutral" prompt, compared to 0.68628 for Flan-T5-small. This indicates that increasing the number of model parameters has a positive eect on its ability to reduce textual toxicity.
It is also worth noting that T5-based models can be adapted to specic languages and domains. For instance, the spivavtor [23] model, designed for the Ukrainian language, showed improved results when prompted with "Перефразуйте" ("Paraphrase"). It achieved a joint metric of 0.33832, signicantly lower than that of dictionary-based approaches. This supports the conclusion that T5-based models have strong potential for application in multilingual environments.
Results for spivavtor models on Ukrainian language presented in Table 4. The values in the table are given for the Joint metric.
Additionally, we evaluated the Llama3[24] model with varying temperature settings, dierent prompts, and multiple LoRA adapters [25], which allowed us to explore the impact of both generation parameters and architectural modications on detoxication quality. In our experiments, pre-trained versions of Llama3 were enhanced with adapters trained on parallel datasets of toxic and non-toxic text.
Prompts were carefully selected to explicitly instruct the model to reduce toxicity while preserving the original meaning. Generation was performed using various temperature values (ranging from 0.01 to 0.95), allowing us to evaluate the balance between deterministic and diverse outputs.
The inuence of temperature on detoxication quality proved to be inversely proportional. Lower temperature values yielded better results, as the model generated outputs that closely adhered to the original sentence structure and meaning. In contrast, higher temperature values increased output variability — although the overall meaning was preserved, the rewritten sentences ofien used completely dierent wording, sometimes reintroducing potentially harmful expressions.
We also tested several LoRA adapters that diered in size and training data. Interestingly, an adapter trained on only 50 examples performed slightly better than another trained on 200 examples. However, the dierence between these congurations was minimal, and both still produced outputs that were relatively close to the base Llama3. Results of llama3-70B model presented in Table 5. The values in the table are given for the Joint metric.
Our team, The Toxinators 2000, participated in the Multilingual Text Detoxication task at PAN 2025. The developed system combined dictionary-based ltering with generative rewriting using a large language model (LLMs) mT0. According to the evaluation on the platform, we ranked 5th out of 32 teams, achieving an average score of 0.675 across all languages. Additionally, on the LLM-as-Judge Language English Spanish Deutsch Chinese Arabic Hindi Ukrainian Russian Amharic Italian Japanese Hebrew French Tatar Hindi leaderboard, we ranked 13th out of 32 teams, achieving an average score of 0.648 across all languages. The best results were obtained for the following languages on leaderboard:
In the course of this study, various approaches to the task of text detoxication were examined, including the use of lexical resources (dictionaries), deep learning models, and modern large language models (LLMs). The main objective was to compare the eectiveness of these methods across several languages — English, Russian, Ukrainian, and others — and to identify optimal strategies for processing toxic content depending on linguistic characteristics and data availability.
The mT0 model , when combined with preliminary removal of toxic words and word parts, demonstrated better performance than the baseline system. For example, in Tatar, the score increased from 0.580 to 0.617 and in Japanese — from 0.582 to 0.644. At the same time, the dictionary-based method achieved the best results for Hindi, increasing the score from 0.351 to 0.449.
These ndings conrm that the eectiveness of text detoxication methods is directly inuenced by linguistic specicity. No single approach proves universally optimal; instead, the choice of method should be guided by the morphological complexity, cultural context, and data availability of the target language.
Thus, future research should focus on developing language-adaptive detoxication systems, which combine the strengths of dictionary ltering, deep learning, and prompt-based LLM rewriting to ensure both high-quality output and meaningful reduction of toxic content.
N. Cheng, O. Chernoguz, O. Hart, O. Salpekar, O. Kalinli, P. Kent, P. Parekh, P. Saab, P. Balaji, P. Rittner, P. Bontrager, P. Roux, P. Dollar, P. Zvyagina, P. Ratanchandani, P. Yuvraj, Q. Liang, R. Alao, R. Rodriguez, R. Ayub, R. Murthy, R. Nayani, R. Mitra, R. Parthasarathy, R. Li, R. Hogan, R. Battey, R. Wang, R. Howes, R. Rinott, S. Mehta, S. Siby, S. J. Bondu, S. Datta, S. Chugh, S. Hunt, S. Dhillon, S. Sidorov, S. Pan, S. Mahajan, S. Verma, S. Yamamoto, S. Ramaswamy, S. Lindsay, S. Lindsay, S. Feng, S. Lin, S. C. Zha, S. Patil, S. Shankar, S. Zhang, S. Zhang, S. Wang, S. Agarwal, S. Sajuyigbe, S. Chintala, S. Max, S. Chen, S. Kehoe, S. Sattereld, S. Govindaprasad, S. Gupta, S. Deng, S. Cho, S. Virk, S. Subramanian, S. Choudhury, S. Goldman, T. Remez, T. Glaser, T. Best, T. Koehler, T. Robinson, T. Li, T. Zhang, T. Matthews, T. Chou, T. Shaked, V. Vontimitta, V. Ajayi, V. Montanez, V. Mohan, V. S. Kumar, V. Mangla, V. Ionescu, V. Poenaru, V. T. Mihailescu, V. Ivanov, W. Li, W. Wang, W. Jiang, W. Bouaziz, W. Constable, X. Tang, X. Wu, X. Wang, X. Wu, X. Gao, Y. Kleinman, Y. Chen, Y. Hu, Y. Jia, Y. Qi, Y. Li, Y. Zhang, Y. Zhang, Y. Adi, Y. Nam, Yu, Wang, Y. Zhao, Y. Hao, Y. Qian, Y. Li, Y. He, Z. Rait, Z. DeVito, Z. Rosnbrick, Z. Wen, Z. Yang, Z. Zhao, Z. Ma, The llama 3 herd of models, 2024. URL: https://arxiv.org/abs/2407.21783. arXiv:2407.21783. [18] Hugging Face, Flan-t5 models, https://huggingface.co/google/an-t5-small, https://huggingface.
co/google/an-t5-base, https://huggingface.co/google/an-t5-large, 2025. Accessed: 2025-07-06. [19] s-nlp, Parallel detoxication dataset (small), https://github.com/s-nlp/parallel_detoxication_data set/blob/main/parallel_detoxification_dataset_small.tsv, 2025. Accessed: 2025-07-06. [20] Hugging Face, Paranmt for detox, https://huggingface.co/datasets/s-nlp/paranmt_for_detox, 2025.
Accessed: 2025-07-06. [21] s-nlp, Detox releases, https://github.com/s-nlp/detox/releases/, 2025. Accessed: 2025-07-06. [22] Hugging Face, mt0-xl-detox-orpo, https://huggingf ace.co/s- nlp/mt0- xl- detox- orpo, 2025.
Accessed: 2025-07-06. [23] Hugging Face, Spivavtor models, https://huggingf ace.co/grammarly/spivavtor-xxl, https: //huggingface.co/grammarly/spivavtor-large, 2025. Accessed: 2025-07-06. [24] Hugging Face, Meta-llama-3-70b, https://huggingf ace.co/meta-llama/Meta-Llama-3-70B, 2025.
Accessed: 2025-07-06. [25] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, Lora: Low-rank adaptation of large language models, in: Proceedings of the 10th International Conference on Learning Representations (ICLR), 2022.