With the impressive developments in artificial intelligence (AI) technologies, we started to use AI tools in various daily tasks. Therefore, any failure of these models might negatively affect our lives. While it is already challenging to develop robust models that can work correctly in real-world, people might also intentionally attempt to deceive these models with adversarial examples [1]. Thus, it is crucial to develop robust models which are not vulnerable to such attacks.
In this paper, we present our participation in Task 6 [2] of CLEF-2024 CheckThat! Lab [3]. We explore several approaches to create adversarial examples. Our proposed methods can be grouped into two groups: i) genetic algorithm-based methods and ii) large language model (LLM) based methods. In genetic algorithm [4,5] based methods, we first identify the words that need to be manipulated. Next, we apply various text manipulation methods including adding/removing a letter, shuffling the order of the letters, using homoglyphs of letters, and splitting words by inserting space character. Regarding LLM-based approaches, we propose three different approaches: i) paraphrasing the text via LLAMA3, ii) utilizing LLMs to identify words to be manipulated, and iii) using LLMs to directly create adversarial examples.
In our experiments, we observe that genetic algorithm-based methods outperform all LLM-based approaches. Among genetic algorithm-based methods, the one that uses a combination of splitting words and using homoglyphs outperforms others. Thus, we use this model as our primary model. In the official ranking, we are ranked third based on both BODEGA score and manual evaluation.
Researchers have investigated various adversarial attacks and defense mechanisms for NLP tasks [6]. Chen et al. [7] examined backdoor attacks where manipulated training data causes models to fail with specific triggers but perform normally otherwise. Yang et al. [8] demonstrated that altering a single word embedding can fool sentiment analysis models without affecting clean data results. Kurita et al. [9] compared various backdoor attacks for different NLP tasks, finding that attack success varies across tasks. Dai et al. [10] showed that inserting trigger sentences into LSTM-based models is highly effective. In this study, we target already trained models.
Researchers have also examined the vulnerabilities of NLP models in a black-box setting. The methods explored in prior work can be categorized into three types: 1) character-level changes, where words are modified with different spelling errors [11], 2) word-level changes, involving the replacement, removal, or addition of words [12], and 3) sentence-level changes, where new sentences or phrases are added, or existing ones are removed or paraphrased [13]. While our text manipulation techniques are similar to the ones in the literature, we use a genetic algorithm to decide the words to be changed and how to change them. In addition, we explore the utilization of LLMs for adversarial example generation.
In our work, we propose two different methods including a genetic algorithm-based approach and LLM-based approach. Now we explain these methods.
Some words are more important than others in several NLP tasks. For instance, let us consider the following statement for the fact-checking task: The capital city of Turkiye is Ankara. If we change the word Ankara to any other city name, the statement will be false. Therefore, if we can make the word Ankara unreadable for the models, it is likely that the model will be confused in the prediction. Once an important word is selected, then the next question is how to modify it to fool the models. Therefore, our approach can be considered in two steps: i) detecting the words to be modified and ii) applying the modification method. Now we explain these two steps in detail.
We develop a genetic algorithm for selecting the words to be modified. Algorithm 1 describes our genetic algorithm. As the first step, we tokenize the input text and create potential mutations to form an initial population [Line 1]. Based on our mutation strategy, we apply mutations to each word with a probability defined by the mutation_rate (0.1) Each candidate's fitness is evaluated based on its ability to deceive the target model [Line 3]. If a candidate changes the label of the input, it receives a fitness score which is reflective of the modifications made. Otherwise, the candidate gets a 1000-point penalty additional to the modifications. Through the selection phase, the most promising candidates are retained, and through genetic operations like crossover and mutation, a new generation of text variants is created [Lines 4-5]. The crossover operation is executed by selecting a random, appropriate point in the token list of the chosen parents, ensuring that the structural integrity of words is maintained. Offspring are then produced by merging segments from each parent up to and beyond this point [Line 5]. As for the mutation, it alters each token based on the chosen mutation strategy/strategies, such as homoglyph replacement or various strategic word splits with a probability defined by the mutation_rate [Line 5]. We explain the mutation strategies in detail in Section 3.1.2. This iterative process continues until a successful adversarial text is generated or the maximum number of generations is reached [Lines 2-7]. If no successful manipulation is achieved, the original text is returned as a fallback [Line 8]. We set the maximum number of generations to 10 and the population size to 20 in all our experiments. Evaluate Fitness: calculate fitness for each candidate.
Selection: select the top half of the population by fitness.
Crossover and Mutation: apply crossover and mutation on selected parents to create new population.
if any candidate change the label then 7:
return adversarial text 8: return original_text as fallback
In this section, we explain the word modification techniques used in our study. Figure2 shows an example modified sentence for each method. Homoglyph Replacement (HomoglyphRep). Some letters are visually similar, i.e., homoglyphs, but their encodings are different. So in this approach, we replace the characters with their visually similar ones. Figure 1 shows the letters we replaced and which letters were used for the replacement. In case there are multiple homoglyphs for a letter, we randomly select one of them. Splitting Words Randomly (Split R ). If a word is split into two, we can still easily understand the meaning of the corresponding sentence/phrase1 . However, models like BERT can be highly affected by this kind of typo because it will lead to incorrect tokenization and it might cause getting out-ofvocabulary representations for these words. Therefore, in this approach, we split words by adding a space character to a randomly selected index of the word. Splitting Words Meaningfully (Split M ). In this method, instead of splitting the words from a random position, we focus on trying to create meaningful words after splitting, e.g., "langu age". By this approach, the model will get a completely irrelevant but valid word, causing huge changes in its representation. We use the NLTK word corpus to identify the valid words and split them accordingly. As an exceptional case, we avoid splitting the first and last characters unless the word starts or ends with 'a'. Split Words Heuristically (Split H ). In this method, we split based on the first and the last character of the targeted word. In particular, if the targeted word starts or ends with characters 'a' or 'i', we divide them from the beginning or the end accordingly. Otherwise, we choose a random index to split. Combine HomoglyphRep&Split : In this method, we combine HomoglyphRep and Split ๐ป methods. In particular, we randomly choose one of them and apply accordingly.
Combine HADSSh . In this method, we randomly select from one of the following methods: i) Homo-glyphRep, ii) Split R , iii) adding random characters into words, iv) deleting a randomly selected letter, and v) shuffling the order of the letters within the targeted word.
As LLMs have impressive performance in text generation and semantic analysis of text, we investigate how they can be utilized to create adversarial examples. We propose three different methods based on LLMs, LLAMA 32 and Mistral 3 . The details of these methods are explained below. Figure3 shows an example modified sentence for each LLM based method.
In this method, we explore the impact of paraphrasing using LLMs. We use LLAMA3 to paraphrase the texts with the following prompt: "Paraphrase the following sentence with similar length T: S" where S is the input sentence and T represents the token count of the given text.
In this method, we use LLMs to identify the words that convey the most important information for the general meaning of the given text. We directly ask LLAMA 3 to identify two important words and then apply HomoglyphRep method for those methods. We use following prompt for this task: "You are an information extractor and your task is to extract and return the two most important words that convey the meaning of the sentence. You should output the extracted words in the 'word1, word2' format. Sentence: {sentence}"
In this method, we utilize LLMs to create adversarial examples and pre-evaluate its validity by another LLM. Figure 4 shows the process flow of our method. In particular, firstly, we do not ask only to paraphrase a given text but ask LLAMA3 to create an adversarial example for a given text. Next, we ask Mistral to check if the generated text is an adversarial attack for the corresponding task. If Mistral verifies it, we use that generated text. Otherwise, we generate another sample using LLAMA3. This generation and verification process continues at most three iterations. After three attempts, we use LLAMA3's output although Mistral does not verify that.
In this section, we present the experimental setup and the results.
Datasets. The dataset shared by the organizers of the lab covers five different binary classification tasks: Style-based news bias assessment (HN), propaganda detection (PR2), fact-checking (FC), rumor detection (RD), and COVID-19 misinformation detection (C19). Table 1 provides statistics about the datasets. Evaluation Metrics: This task uses the Bodega score [14] to evaluate the systems, which is basically the multiplication of confusion score (or success rate), semantic score, and character score. This score takes values between 0 and 1. A high score indicates that the model is deceived by preserving the meaning and appearance, while a low score indicates a weak deception by changing the and/or appearance.
In this section, we present results for our proposed methods on the test set. Firstly, we present the result for the genetic algorithm-based approaches against victim models by taking the average of all problem domains (Section 4.2.1). Next, we report the results for both LLM-based and genetic algorithm-based approaches in the fact-checking task (Section 4.2.2). Lastly, we present our official results (Section 4.3).
We report genetic algorithm-based results when the target model is BiLSTM, BERT, and Surprise. The results are shown in
As getting results for LLM based approaches require much more computation power and time, we could get results only for the fact checking task and for BERT and BiLSTM models. Table3 shows the results
for LLM based approaches and also corresponding genetic algorithm based approaches for comparison. All LLM based approaches resulted in lower BODEGA scores compared to all genetic algorithm based methods. Furthermore, genetic algorithm based methods achieve very high success rates and character scores. Combine HomoglyphRep&Split achieves a perfect rate in both cases but HomoglyphRep slightly outperforms Combine HomoglyphRep&Split in terms of average BODEGA score.
Among LLM based approaches, our results are mixed. LLM Identify achieves the lowest BODEGA score when the target model is BERT. However, it yields the highest BODEGA score when the target model is BiLSTM. This suggests that BiLSTM models are highly affected by homoglyph attacks. Interestingly, LLM Paraphrase outperforms LLM Adversarial in both target models although the prompt we use in LLM Paraphrase just asks to paraphrase the text without any intention of creating an adversarial example while we ask to create adversarial example in LLM Adversarial .
We submitted the results of Combine HomoglyphRep&Split as our official run because of its superior performance on average. We achieved 0.4859 BODEGA score on average, ranking third among participants.
Based on the results with manual annotations for preserving meaning, we achieved 0.62, ranking again third.
In this paper, we present our participation in Task 6 of the CLEF 2024 CheckThat! Lab. In our study, we explore two different approaches to create adversarial examples. In the first approach, we use a genetic algorithm to detect the words to be changed and to identify text manipulation methods. We investigate various text manipulation methods, such as adding/deleting a letter, using homoglyphs, and shuffling the order of letters within a text. In the second approach, we utilize large language models to create adversarial examples. This involves three different methods: asking LLMs to paraphrase a given text, using LLMs to directly generate adversarial examples, and employing LLMs to identify the words that need to be changed to create adversarial examples.
In our comprehensive set of experiments, which involve six different tasks, three different target models, and a total of nine methods, we have the following observations. Firstly, genetic algorithm-based methods outperform all LLM-based approaches. Secondly, among the genetic algorithm-based methods, using the combination of homoglyphs and splitting as text manipulation outperforms the other methods. This suggests that we need to be more selective in text manipulation methods instead of using all possible methods. In the official ranking, our primary model is ranked third based on the BODEGA score and semantic preservation scores which are based on manual annotations.
In the future, we plan to extend this work in two different directions. Firstly, although LLM models did not achieve high performance in this task, we believe that their effectiveness can be improved through several strategies, such as using different prompts and fine-tuning them specifically for this task. Secondly, regarding the genetic algorithm-based methods, we plan to explore other text manipulation methods to enhance this model further.