TurQUaz at CheckThat! 2024: Creating Adversarial Examples using Genetic Algorithm Basak Demirok1,*,† , Selin Mergen1,† , Bugra Oz1 and Mucahid Kutlu2,* 1 TOBB University of Economics and Technology, Ankara, Türkiye 2 Qatar University, Doha, Qatar Abstract As we increasingly integrate artificial intelligence into our daily tasks, it is crucial to ensure that these systems are reliable and robust against adversarial attacks. In this paper, we present our participation in Task 6 of CLEF CheckThat! 2024 lab. In our work, we explore several methods, which can be grouped into two categories. The first group focuses on using a genetic algorithm to detect words and changing them via several methods such as adding/deleting words and using homoglyphs. In the second group of methods, we use large language models to generate adversarial attacks. Based on our comprehensive experiments, we pick the genetic algorithm-based model which utilizes a combination of splitting words and homoglyphs as a text manipulation method, as our primary model. We are ranked third based on both BODEGA metric and manual evaluation. Keywords Adversarial Examples, Credibility Assessment, Natural Language Processing, Robustness 1. Introduction With the impressive developments in artificial intelligence (AI) technologies, we started to use AI tools in various daily tasks. Therefore, any failure of these models might negatively affect our lives. While it is already challenging to develop robust models that can work correctly in real-world, people might also intentionally attempt to deceive these models with adversarial examples [1]. Thus, it is crucial to develop robust models which are not vulnerable to such attacks. In this paper, we present our participation in Task 6 [2] of CLEF-2024 CheckThat! Lab [3]. We explore several approaches to create adversarial examples. Our proposed methods can be grouped into two groups: i) genetic algorithm-based methods and ii) large language model (LLM) based methods. In genetic algorithm [4, 5] based methods, we first identify the words that need to be manipulated. Next, we apply various text manipulation methods including adding/removing a letter, shuffling the order of the letters, using homoglyphs of letters, and splitting words by inserting space character. Regarding LLM-based approaches, we propose three different approaches: i) paraphrasing the text via LLAMA3, ii) utilizing LLMs to identify words to be manipulated, and iii) using LLMs to directly create adversarial examples. In our experiments, we observe that genetic algorithm-based methods outperform all LLM-based approaches. Among genetic algorithm-based methods, the one that uses a combination of splitting words and using homoglyphs outperforms others. Thus, we use this model as our primary model. In the official ranking, we are ranked third based on both BODEGA score and manual evaluation. CLEF 2024: Conference and Labs of the Evaluation Forum, September 09–12, 2024, Grenoble, France * Corresponding author. † These authors contributed equally. $ g.demirok@etu.edu.tr (B. Demirok); s.mergen@etu.edu.tr (S. Mergen); ioz@etu.edu.tr (B. Oz); mucahidkutlu@qu.edu.qa (M. Kutlu)  0000-0003-1620-2013 (B. Demirok); 0009-0002-3284-9490 (S. Mergen); 0009-0003-7145-6726 (B. Oz); 0000-0002-5660-4992 (M. Kutlu) © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings 2. Related Work Researchers have investigated various adversarial attacks and defense mechanisms for NLP tasks [6]. Chen et al. [7] examined backdoor attacks where manipulated training data causes models to fail with specific triggers but perform normally otherwise. Yang et al. [8] demonstrated that altering a single word embedding can fool sentiment analysis models without affecting clean data results. Kurita et al. [9] compared various backdoor attacks for different NLP tasks, finding that attack success varies across tasks. Dai et al. [10] showed that inserting trigger sentences into LSTM-based models is highly effective. In this study, we target already trained models. Researchers have also examined the vulnerabilities of NLP models in a black-box setting. The methods explored in prior work can be categorized into three types: 1) character-level changes, where words are modified with different spelling errors [11], 2) word-level changes, involving the replacement, removal, or addition of words [12], and 3) sentence-level changes, where new sentences or phrases are added, or existing ones are removed or paraphrased [13]. While our text manipulation techniques are similar to the ones in the literature, we use a genetic algorithm to decide the words to be changed and how to change them. In addition, we explore the utilization of LLMs for adversarial example generation. 3. Proposed Methods In our work, we propose two different methods including a genetic algorithm-based approach and LLM-based approach. Now we explain these methods. 3.1. Genetic Algorithm Based Approach Some words are more important than others in several NLP tasks. For instance, let us consider the following statement for the fact-checking task: The capital city of Turkiye is Ankara. If we change the word Ankara to any other city name, the statement will be false. Therefore, if we can make the word Ankara unreadable for the models, it is likely that the model will be confused in the prediction. Once an important word is selected, then the next question is how to modify it to fool the models. Therefore, our approach can be considered in two steps: i) detecting the words to be modified and ii) applying the modification method. Now we explain these two steps in detail. 3.1.1. Selecting Words to be Modified We develop a genetic algorithm for selecting the words to be modified. Algorithm 1 describes our genetic algorithm. As the first step, we tokenize the input text and create potential mutations to form an initial population [Line 1]. Based on our mutation strategy, we apply mutations to each word with a probability defined by the mutation_rate (0.1) Each candidate’s fitness is evaluated based on its ability to deceive the target model [Line 3]. If a candidate changes the label of the input, it receives a fitness score which is reflective of the modifications made. Otherwise, the candidate gets a 1000-point penalty additional to the modifications. Through the selection phase, the most promising candidates are retained, and through genetic operations like crossover and mutation, a new generation of text variants is created [Lines 4-5]. The crossover operation is executed by selecting a random, appropriate point in the token list of the chosen parents, ensuring that the structural integrity of words is maintained. Offspring are then produced by merging segments from each parent up to and beyond this point [Line 5]. As for the mutation, it alters each token based on the chosen mutation strategy/strategies, such as homoglyph replacement or various strategic word splits with a probability defined by the mutation_rate [Line 5]. We explain the mutation strategies in detail in Section 3.1.2. This iterative process continues until a successful adversarial text is generated or the maximum number of generations is reached [Lines 2-7]. If no successful manipulation is achieved, the original text is returned as a fallback [Line 8]. We set the maximum number of generations to 10 and the population size to 20 in all our experiments. Algorithm 1 : Genetic Algorithm Structure 1: Initialize Population: generate initial 𝑝𝑜𝑝𝑢𝑙𝑎𝑡𝑖𝑜𝑛_𝑠𝑖𝑧𝑒 many mutations. 2: for 𝑔𝑒𝑛𝑒𝑟𝑎𝑡𝑖𝑜𝑛 = 1 to 𝑚𝑎𝑥_𝑔𝑒𝑛𝑒𝑟𝑎𝑡𝑖𝑜𝑛𝑠 do 3: Evaluate Fitness: calculate fitness for each candidate. 4: Selection: select the top half of the population by fitness. 5: Crossover and Mutation: apply crossover and mutation on selected parents to create new population. 6: if any candidate change the label then 7: return adversarial text 8: return original_text as fallback 3.1.2. Mutation Methods In this section, we explain the word modification techniques used in our study. Figure2 shows an example modified sentence for each method. Homoglyph Replacement (HomoglyphRep). Some letters are visually similar, i.e., homoglyphs, but their encodings are different. So in this approach, we replace the characters with their visually similar ones. Figure 1 shows the letters we replaced and which letters were used for the replacement. In case there are multiple homoglyphs for a letter, we randomly select one of them. Figure 1: The letters with their homoglyphs used in our study. While there are other homoglyph character options available, we select only those that are indistinguishable to the human eye. Splitting Words Randomly (SplitR ). If a word is split into two, we can still easily understand the meaning of the corresponding sentence/phrase1 . However, models like BERT can be highly affected by this kind of typo because it will lead to incorrect tokenization and it might cause getting out-of- vocabulary representations for these words. Therefore, in this approach, we split words by adding a space character to a randomly selected index of the word. Splitting Words Meaningfully (SplitM ). In this method, instead of splitting the words from a random position, we focus on trying to create meaningful words after splitting, e.g., "langu age". By this approach, the model will get a completely irrelevant but valid word, causing huge changes in its representation. We use the NLTK word corpus to identify the valid words and split them accordingly. As an exceptional case, we avoid splitting the first and last characters unless the word starts or ends with ‘a’. Split Words Heuristically (SplitH ). In this method, we split based on the first and the last character of the targeted word. In particular, if the targeted word starts or ends with characters ‘a’ or ‘i’, we divide them from the beginning or the end accordingly. Otherwise, we choose a random index to split. CombineHomoglyphRep&Split : In this method, we combine HomoglyphRep and Split𝐻 methods. In particular, we randomly choose one of them and apply accordingly. 1 e.g., "nat ural language processing" CombineHADSSh . In this method, we randomly select from one of the following methods: i) Homo- glyphRep, ii) SplitR , iii) adding random characters into words, iv) deleting a randomly selected letter, and v) shuffling the order of the letters within the targeted word. Figure 2: Example outputs of genetic algorithm-based attacks. The texts written in red show the modified parts in original sentence: "Emma Watson. Emma Charlotte Duerre Watson ( born 15 April 1990 ) is a French-British actress , model , and activist . Ẽmma Watson is an Italian actress." 3.2. LLM Based Approach As LLMs have impressive performance in text generation and semantic analysis of text, we investigate how they can be utilized to create adversarial examples. We propose three different methods based on LLMs, LLAMA 32 and Mistral3 . The details of these methods are explained below. Figure3 shows an example modified sentence for each LLM based method. 3.2.1. Paraphrasing with LLMs (LLMParaphrase ) In this method, we explore the impact of paraphrasing using LLMs. We use LLAMA3 to paraphrase the texts with the following prompt: "Paraphrase the following sentence with similar length T: S" where S is the input sentence and T represents the token count of the given text. 3.2.2. Identifying Words to be Changed Using LLMs (LLMIdentify ) In this method, we use LLMs to identify the words that convey the most important information for the general meaning of the given text. We directly ask LLAMA 3 to identify two important words and then apply HomoglyphRep method for those methods. We use following prompt for this task: "You are an information extractor and your task is to extract and return the two most important words that convey the meaning of the sentence. You should output the extracted words in the ’word1, word2’ format. Sentence: {sentence}" 3.2.3. Creating Adversarial Examples (LLMAdversarial ) In this method, we utilize LLMs to create adversarial examples and pre-evaluate its validity by another LLM. Figure 4 shows the process flow of our method. In particular, firstly, we do not ask only to paraphrase a given text but ask LLAMA3 to create an adversarial example for a given text. Next, we 2 https://ollama.com/library/llama3:8b 3 https://ollama.com/library/mistral v02 Figure 3: Example outputs of LLM-based attacks. The red letter shows the modified letter by HomoglyphRep method. ask Mistral to check if the generated text is an adversarial attack for the corresponding task. If Mistral verifies it, we use that generated text. Otherwise, we generate another sample using LLAMA3. This generation and verification process continues at most three iterations. After three attempts, we use LLAMA3’s output although Mistral does not verify that. Figure 4: The process flow of our method LLMAdversarial . 4. Experiments In this section, we present the experimental setup and the results. 4.1. Experimental Setup Datasets. The dataset shared by the organizers of the lab covers five different binary classification tasks: Style-based news bias assessment (HN), propaganda detection (PR2), fact-checking (FC), rumor detection (RD), and COVID-19 misinformation detection (C19). Table 1 provides statistics about the datasets. Table 1 Dataset size for each task. Task Train Development Test Style-based news bias assessment 60,234 3,600 400 Propaganda Detection 11,546 3.186 407 Fact Checking 172,763 10,010 405 Rumor Detection 8,683 2,070 415 COVID-19 Misinformation Detection 1,130 - 595 Evaluation Metrics: This task uses the Bodega score [14] to evaluate the systems, which is basically the multiplication of confusion score (or success rate), semantic score, and character score. This score takes values between 0 and 1. A high score indicates that the model is deceived by preserving the meaning and appearance, while a low score indicates a weak deception by changing the meaning and/or appearance. 4.2. Experimental Results In this section, we present results for our proposed methods on the test set. Firstly, we present the result for the genetic algorithm-based approaches against victim models by taking the average of all problem domains (Section 4.2.1). Next, we report the results for both LLM-based and genetic algorithm-based approaches in the fact-checking task (Section 4.2.2). Lastly, we present our official results (Section 4.3). 4.2.1. Results for Genetic Algorithm Based Approach We report genetic algorithm-based results when the target model is BiLSTM, BERT, and Surprise. The results are shown in Table 2. Our observations based on the results are as follows. Firstly, among split models, SplitM has significantly lower success rate and BODEGA score, but slightly higher semantic and character scores. While SplitR and SplitH have highly similar scores, SplitR outperforms SplitR slightly in terms of BODEGA. Secondly, HomoglyphRep achieves the highest semantic score in all cases and the highest character score in two cases, but its BODEGA and success rate are lower than our combination-based methods. Thirdly, CombineHomoglyphRep&Split outperforms CombineHADSSh in all cases in terms of BODEGA, suggesting that we can focus on only a few text manipulation methods instead of covering all. Finally, among all models, CombineHomoglyphRep&Split and SplitR has the same average BODEGA score, but CombineHomoglyphRep&Split slightly outperforms SplitR in terms of success rate and semantic score. Table 2 Average performance of approaches on all problem domains on the test set. Evaluation measures include BODEGA score (B.), success rate (SR), Semantic Score (SemSc), character score (CharSc) and number of queries to the attacked model (Q.)[14]. The best performing score for each case is written in bold. Target Model Method BODEGA SR SemSc CharSc Q. SplitR 0.61 0.89 0.70 0.97 97.98 SplitH 0.61 0.89 0.70 0.97 103.30 SplitM 0.50 0.69 0.73 0.98 172.11 BiLSTM HomoglyphRep 0.59 0.83 0.74 0.95 149.41 CombineHomoglyphRep&Split 0.62 0.91 0.71 0.95 116.80 CombineHADSSh 0.59 0.94 0.66 0.95 118.86 SplitR 0.50 0.75 0.68 0.96 157.77 SplitH 0.50 0.75 0.68 0.97 162.17 SplitM 0.44 0.63 0.71 0.98 203.51 BERT HomoglyphRep 0.49 0.73 0.71 0.93 229.69 CombineHomoglyphRep&Split 0.50 0.77 0.68 0.94 213.93 CombineHADSSh 0.47 0.77 0.64 0.94 227.60 SplitR 0.37 0.56 0.68 0.96 236.94 SplitH 0.36 0.55 0.67 0.96 241.56 SplitM 0.31 0.44 0.70 0.98 276.73 Surprise HomoglyphRep 0.36 0.55 0.70 0.92 341.65 CombineHomoglyphRep&Split 0.36 0.55 0.68 0.94 331.73 CombineHADSSh 0.34 0.57 0.64 0.94 336.89 4.2.2. Results for LLM Based Approach As getting results for LLM based approaches require much more computation power and time, we could get results only for the fact checking task and for BERT and BiLSTM models. Table3 shows the results for LLM based approaches and also corresponding genetic algorithm based approaches for comparison. All LLM based approaches resulted in lower BODEGA scores compared to all genetic algorithm based methods. Furthermore, genetic algorithm based methods achieve very high success rates and character scores. CombineHomoglyphRep&Split achieves a perfect success rate in both cases but HomoglyphRep slightly outperforms CombineHomoglyphRep&Split in terms of average BODEGA score. Among LLM based approaches, our results are mixed. LLMIdentify achieves the lowest BODEGA score when the target model is BERT. However, it yields the highest BODEGA score when the target model is BiLSTM. This suggests that BiLSTM models are highly affected by homoglyph attacks. Interest- ingly, LLMParaphrase outperforms LLMAdversarial in both target models although the prompt we use in LLMParaphrase just asks to paraphrase the text without any intention of creating an adversarial example while we ask to create adversarial example in LLMAdversarial . Table 3 Results of LLM-used and GA-used methods on Fact-Checking Task’s attack dataset. Evaluation measures include BODEGA score, success rate (SR), semantic score (SemSc), character score (CharSC) and number of queries to the attacked model (Q.)[14]. Bold scores indicate the highest score of the corresponding target model. Target Model Method BODEGA SR SemSc CharSC Q. LLMParaphrase 0.066 0.380 0.429 0.397 2.380 LLMIdentify 0.027 0.032 0.86 0.96 2.02 LLMAdversarial 0.056 0.496 0.312 0.326 2.496 SplitR 0.73 0.98 0.76 0.97 69.20 BERT SplitH 0.72 0.97 0.77 0.97 75.41 SplitM 0.63 0.82 0.78 0.98 132.55 HomoglyphRep 0.75 0.97 0.82 0.94 79.85 CombineHomoglyphRep&Split 0.74 1.00 0.78 0.95 70.70 CombineHADSSh 0.69 0.98 0.74 0.95 99.42 LLMParaphrase 0.070 0.404 0.426 0.402 2.404 LLMIdentify 0.110 0.130 0.867 0.969 2.128 LLMAdversarial 0.048 0.420 0.312 0.319 2.420 SplitR 0.78 1.00 0.80 0.98 38.60 BiLSTM SplitH 0.77 1.00 0.79 0.98 42.55 SplitM 0.65 0.83 0.79 0.98 121.60 HomoglyphRep 0.79 0.98 0.84 0.95 54.54 CombineHomoglyphRep&Split 0.79 1.00 0.82 0.97 39.44 CombineHADSSh 0.75 1.00 0.77 0.97 45.67 4.3. Official Ranking We submitted the results of CombineHomoglyphRep&Split as our official run because of its superior perfor- mance on average. We achieved 0.4859 BODEGA score on average, ranking third among participants. Based on the results with manual annotations for preserving meaning, we achieved 0.62, ranking again third. 5. Conclusion In this paper, we present our participation in Task 6 of the CLEF 2024 CheckThat! Lab. In our study, we explore two different approaches to create adversarial examples. In the first approach, we use a genetic algorithm to detect the words to be changed and to identify text manipulation methods. We investigate various text manipulation methods, such as adding/deleting a letter, using homoglyphs, and shuffling the order of letters within a text. In the second approach, we utilize large language models to create adversarial examples. This involves three different methods: asking LLMs to paraphrase a given text, using LLMs to directly generate adversarial examples, and employing LLMs to identify the words that need to be changed to create adversarial examples. In our comprehensive set of experiments, which involve six different tasks, three different target models, and a total of nine methods, we have the following observations. Firstly, genetic algorithm-based methods outperform all LLM-based approaches. Secondly, among the genetic algorithm-based methods, using the combination of homoglyphs and splitting words as text manipulation outperforms the other methods. This suggests that we need to be more selective in text manipulation methods instead of using all possible methods. In the official ranking, our primary model is ranked third based on the BODEGA score and semantic preservation scores which are based on manual annotations. In the future, we plan to extend this work in two different directions. Firstly, although LLM models did not achieve high performance in this task, we believe that their effectiveness can be improved through several strategies, such as using different prompts and fine-tuning them specifically for this task. Secondly, regarding the genetic algorithm-based methods, we plan to explore other text manipulation methods to enhance this model further. References [1] W. E. Zhang, Q. Z. Sheng, A. Alhazmi, C. Li, Adversarial attacks on deep-learning models in natural language processing: A survey, ACM Transactions on Intelligent Systems and Technology (TIST) 11 (2020) 1–41. [2] P. Przybyła, B. Wu, A. Shvets, Y. Mu, K. C. Sheang, X. Song, H. Saggion, Overview of the CLEF- 2024 CheckThat! lab task 6 on robustness of credibility assessment with adversarial examples (incrediblae), in: Working Notes of CLEF 2024 - Conference and Labs of the Evaluation Forum, CLEF 2024, Grenoble, France, 2024. [3] A. Barrón-Cedeño, F. Alam, J. M. Struß, P. Nakov, T. Chakraborty, T. Elsayed, P. Przybyła, T. Caselli, G. Da San Martino, F. Haouari, C. Li, J. Piskorski, F. Ruggeri, X. Song, R. Suwaileh, Overview of the CLEF-2024 CheckThat! Lab: Check-worthiness, subjectivity, persuasion, roles, authorities and adversarial robustness, in: L. Goeuriot, P. Mulhem, G. Quénot, D. Schwab, L. Soulier, G. M. Di Nunzio, P. Galuščáková, A. García Seco de Herrera, G. Faggioli, N. Ferro (Eds.), Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings of the Fifteenth International Conference of the CLEF Association (CLEF 2024), 2024. [4] M. Alzantot, Y. Sharma, A. Elgohary, B.-J. Ho, M. Srivastava, K.-W. Chang, Generating natural language adversarial examples, in: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018, pp. 2890–2896. [5] N. Boucher, I. Shumailov, R. Anderson, N. Papernot, Bad characters: Imperceptible nlp attacks, in: 2022 IEEE Symposium on Security and Privacy (SP), IEEE, 2022, pp. 1987–2004. [6] K. Ren, T. Zheng, Z. Qin, X. Liu, Adversarial attacks and defenses in deep learning, Engineering 6 (2020) 346–360. [7] X. Chen, A. Salem, D. Chen, M. Backes, S. Ma, Q. Shen, Z. Wu, Y. Zhang, Badnl: Backdoor attacks against nlp models with semantic-preserving improvements, in: Proceedings of the 37th Annual Computer Security Applications Conference, 2021, pp. 554–569. [8] W. Yang, L. Li, Z. Zhang, X. Ren, X. Sun, B. He, Be careful about poisoned word embeddings: Exploring the vulnerability of the embedding layers in nlp models, in: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021, pp. 2048–2058. [9] K. Kurita, P. Michel, G. Neubig, Weight poisoning attacks on pretrained models, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 2793–2806. [10] J. Dai, C. Chen, Y. Li, A backdoor attack against lstm-based text classification systems, IEEE Access 7 (2019) 138872–138878. [11] Y. Belinkov, Y. Bisk, Synthetic and natural noise both break neural machine translation, in: International Conference on Learning Representations, 2018. [12] D. Jin, Z. Jin, J. T. Zhou, P. Szolovits, Is bert really robust? a strong baseline for natural language attack on text classification and entailment, in: Proceedings of the AAAI conference on artificial intelligence, volume 34, 2020, pp. 8018–8025. [13] D. Dogan, B. Altun, M. S. Zengin, M. Kutlu, T. Elsayed, Catch me if you can: Deceiving stance detection and geotagging models to protect privacy of individuals on twitter, in: Proceedings of the International AAAI Conference on Web and Social Media, volume 17, 2023, pp. 173–184. [14] P. Przybyła, A. Shvets, H. Saggion, Bodega: Benchmark for adversarial example generation in credibility assessment, arXiv preprint arXiv:2303.08032 (2023).