This paper presents the experiments and results for the CheckThat! Lab at CLEF 2024 Task 6: Robustness of Credibility Assessment with Adversarial Examples (InCrediblAE). The primary objective of this task was to generate adversarial examples in five problem domains in order to evaluate the robustness of widely used text classification methods (fine-tuned BERT, BiLSTM, and RoBERTa) when applied to credibility assessment issues. This study explores the application of ensemble learning to enhance adversarial attacks on natural language processing (NLP) models. We systematically tested and refined several adversarial attack methods, including BERT-Attack, Genetic algorithms, TextFooler, and CLARE, on five datasets across various misinformation tasks. By developing modified versions of BERT-Attack and hybrid methods, we achieved significant improvements in attack efectiveness. Our results demonstrate the potential of modification and combining multiple methods to create more sophisticated and efective adversarial attack strategies, contributing to the development of more robust and secure systems.
In our subsequent experiments, we modified the BERT-Attack method to balance semantic preservation with attack success rate. By adjusting parameters and exploring the impact of increased substitute numbers, we developed a modified version of BERT-Attack, termed BAm. Additionally, we introduced a new method for selecting important words for replacement, leading to further enhancements in attack efectiveness.
To improve results, we integrated the Genetic algorithm for cases where initial modifications failed,
creating a hybrid approach. We also experimented with a synonym replacement method based on word
embeddings. Finally, we incorporated the CLARE method [
Through this study, we aim to demonstrate the potential of ensemble learning in enhancing adversarial attacks on various models. Our findings can provide valuable insights into the development of more sophisticated and efective adversarial attack strategies, ultimately contributing to the resilience and security of various systems.
The paper presents the experiments and results for the CheckThat! Lab at CLEF 2024 Task 6: Robustness of Credibility Assessment with Adversarial Examples (InCrediblAE) [18, 19]. The objective of this task was to evaluate the robustness of widely used text classification methods when applied to credibility assessment issues. There were three trained victim classifiers provided by task’s organizers: fine-tuned BERT, BiLSTM, and RoBERTa. The first two classifiers were made available as soon as the task was announced, so they were the foundation for our approaches and conducted experiments. The latter was made available only in the test phase and used to evaluate our methods. Moreover, five problem domains were defined, each being a binary classification task, and for each problem, a separate dataset was provided: • Style-based news bias assessment (HN dataset), • Propaganda detection (PR2 dataset), • Fact checking (FC dataset), • Rumour detection (RD dataset), • COVID-19 misinformation detection (C19 dataset).
Our task was to create adversarial examples for each dataset. The adversarial examples should include small modifications in each dataset’s record that, on the one hand, would preserve the meaning (semantics) of the original text and, on the other hand, would change the victim classifier’s decision.
To complete the task, we started with an in-depth analysis of the relevant literature on adversarial
attacks on NLP classification and the BODEGA framework, as described in section 3. In parallel, we
analyzed the datasets provided by the task’s organizers (see below). In the next step, we applied the
existing attack methods to the provided datasets and treated the received results as a baseline for our
approach. The baseline attack methods encompassed those used by [
The detailed description of datasets (apart from C19) is provided in [
The experiments were conducted on our own infrastructure, consisting of four NVIDIA GeForce RTX 2080 Ti GPU cards, each with 11 GB of VRAM memory and one NVIDIA GeForce RTX 4070 GPU card with 12 GB of VRAM memory.
Adversarial attacks are manipulations of input text designed to deceive machine learning models and
cause them to make errors. Adversarial attacks often involve making subtle, human-imperceptible
changes to the input, which can lead the model to produce incorrect or unexpected outputs. Those
changes might include one or many of the following, exemplary scenarios: character-level (adding typos
in the text, adding extra characters at the beginning/end of the sentence), word-level (replacing word
with its synonym, adding semantically neutral words), sentence-level (paraprasing) [
The goal of the presented research was to create adversarial examples targeted at victim models that classify text instances in five diferent domains (style-based news bias assessment, propaganda detection, fact checking, rumour detection, and COVID-19 misinformation detection) described further in section 2.
The methods presented in this paper have been evaluated within a framework provided in InCrediblAE,
called BODEGA (Benchmark fOr aDversarial Example Generation in credibility Assessment) [
The BODEGA is defined as follows:
BODEGA_score(, * ) = Con_score(, * ) × Sem_score(, * ) × Char_score(, * ),
(1)
where is original text instance and * is modified text instance. There are three components
of the BODEGA score. _ has 0 or 1 value (1 if the text instance alternation successfully
caused classifier to change its decision, and 0 otherwise). _ uses BLEURT [
Final BODEGA score is calculated an average over BODEGA scores for all instances in the attack set (i.e. all adversarial attack instances).
To generate adversarial attack examples, two notable Python libraries are TextAttack [
Based on the literature and our own preliminary test, we identified methods available in TexAttack
and/or OpenAttack that were selected to include in further experiments. Those were the following
methods:
• BAE [
In general our approach can be characterized as a special case of ensemble learning. We took the inspiration from this machine learning group of techniques. In case of ensemble learning it is assumed that employment of multiple learning algorithms will outperform any single algorithm used alone.
Although the idea of an ensemble is relatively simple it is more problematic on an implementational level. Especially it is hard to be deployed with respect to new application. In our case this new application is the adversarial attack domain. There are a number of dificulties to solve: • Firstly, there is a variety of approaches that can be taken when it comes to the realization of the ensemble learning. It has to be decided which specific approach will fit best to the projected task. • Secondly, there is an abundant set of methods already used for the execution for the adversarial attack task. The decision has to be made how to create a subset for methods that should be used as one of the ensemble algorithms.
In our approach we have started from a literature review and then systematically tested some of the
adversarial attack methods. One of the important works in this area [
Next we decided to experiment with diferent settings and provided own modifications to the BERTattacker implemented in OpenAttack library for better results. Due to the fact, that balance between semantic preservation and attack success rate can be regulated throughout a threshold of semantic similarity score of substituted word, we modified ”threshold_pred_score” parameter from 0.3 to 0.2. Also we decided to check how twice-increased number of substitutes (from 36 to 72) will afect results. Those modificated version of the BERT-Attack we called BAm. The table 2 presents a comparison of BERT-Attack with default parameters in OpenAttack library (’BA’) and modified version (’BAm’) on ifve datasets and three victims in BODEGA, success, semantic, character scores and queries number per example.
As we can see, BAm improves the BODEGA score in many cases due to a higher success score. However, the semantic score has been slightly reduced because this model additionally selects substitute words that are less related with the meaning of the original. Also we can see that BAm model performs more queries.1
In order to improve the results, we decided to provide diferent method of selecting important words for replacement. Comparison from initial version to modified version is shown below: • In initial approach for each word in the sentence (text), the word is replaced with a ”[UNK]” token (unknown token), creating multiple masked versions of the sentence. So, each masked version has a diferent word replaced by ”[UNK]”. The victim model is used to compute the probabilities for each of these masked sentences. For each word, an importance score is calculated based on the change in probabilities to an opposite class (label) caused by masking that word. The words are ranked based on their importance scores in descending order. This ranking indicates the significance of each word in contributing to the classification decision of the victim model. 1The model performs more queries, but not twice as many as would be expected after increasing the number of substitutes from 36 to 72. This is due to the fact that the algorithm has a stop function if it finds a significant substitute that changes the victim’s decision and does not has to check (all) other substitutes in the queue. By contrast, with more substitutes, this extended model has a better chance of changing the victim’s mind when the basic version of the model exhausts all the words without a success. • In our approach for each masked word in sentence (text) we try to use each of substitute (36 candidates at the first iteration) and find which of them maximizes the gap (diference between the original and new prediction probabilities). Next we created the list of those initially masked words sorted by the ability to change a decision of the victim model based on the maximum gap score.
As we could expect, the new importance ranking of the words for replacement afects the results. Now model can choose other word(s) for replacement based on the potential that can be done based on particular candidates for substitution. The figure 1 presents the most important words selected by our approach on the example of the 8th sample from PR2 dataset. We can observe diferences between Default Important Rank (DIR) of words in sentence (text) given by initial approach and New Importance Rank (NIR) given by our approach. Please note, that the numbering of word positions (’Pos.’ in the ifgure 1) in the considered sentence starts from 0.
After selecting the most important word in sentence (text) we replace it with substitute, which
gives the biggest positive (for us at this iteration) change in the probabilities of the victim model. This
iteration is called ”0th iteration”. It is important to note, here we used also filter for substitutes (which
excludes some words from the list of candidates) how it was proposed in initial approach (for example,
stop words as it was proposed in [
If the 0th and 1st iteration did not succeed, then our approach go through additional iterations from 2nd to 5th. Each of them increased word to be replaced by 1 in the sentence (text) selected from the most important ones. So in the 2nd, 3rd, 4th and 5th iteration it will replace 3, 4, 5, 6 and 7 words in the sentence in total respectively. Additionally, at the last ”5th iteration” we additionally added common punctuation and digits to the list of potential substitutes to increase the chances of success.
The original paper describing the BERT-Attack showed that using diferent number of substitutes
(candidates) afects the attack success rate [
Above described iterations are performed one by one and if a certain iteration succeed the attack
it will also stop the process. Those (additional to BAm) modifications of BERT-Attack are named as
BAm2. Additionally in cases when after all iterations the victim’s decision cannot be changed, we ran
another attacker - Genetic [
Those additional (second) modifications of BERT-Attack algorithm with Genetic algorithm for failed cases we called - BAm2&Genetic. After mentioned before additional modification we found that the results were improved. For example, on the FC task we observed the following improvements comparing to the previous modified version of BERT-Attack (’BAm’, see the table 2): • BERT victim. BODEGA score: 0.66 (instead of 0.54), Success score: 0.97 (instead of 0.78) • BiLSTM victim. BODEGA score: 0.71 (instead of 0.62), Success score: 1.0 (instead of 0.89) • RoBERTa victim. BODEGA score: 0.70 (instead of 0.59), Success score: 1.0 (instead of 0.84) Next we decided to try another approach which transforms a given text by replacing its words with synonyms in the word embedding space. It takes prebuilt counter-fitted GLOVE words embedding proposed by [21]. After checking the impact of each substitution of the selected word the approach selects those changes that have positive efect on the result, and after choosing the most significant one it greedily updating the set with the highest scoring transformations. Additionally it has constraints which disallow the modification of words which have already been modified and also which disallow the modification of stopwords. Let’s name this method as GSWSE (Greedy Search with Word Swap by Embedding).
The GSWSE algorithm also was unable to change the model decision in some cases. So we decided to
ran TextFooler method for such texts. This GSWSE&TextFooler (or GSWSE&TF) method gave
additional improvements of BODEGA score. However, it decreased the success score. For example, on the FC
task we observed the following changes in scores comparing BAm2&Genetic and GSWSE&TextFooler:
• BERT victim. BODEGA score: 0.74 (instead of 0.66), Success score: 0.94 (instead of 0.97)
• BiLSTM victim. BODEGA score: 0.75 (instead of 0.71), Success score: 0.93 (instead of 1.0)
• RoBERTa victim. BODEGA score: 0.71 (instead of 0.70), Success score: 0.93 (instead of 1.0)
In our last stage of experiments we decided to add the CLARE method [
The main purpose of the experiment was to test the BODEGA rate with the datasets and to check
how the research methods compare with the baseline of a semantic evaluation of the robustness. We
compared the results with the baseline presented in [
Table 3 includes results of the attack on BERT and BiLSTM classifiers.
In table 3 the method CLARE is presented only for the tasks PR2, FC and C19. The remaining tasks were skipped for the method due to the lack of time. The process of running this method was costly in time and resources. Additionally, we experienced dificulties with setting the number of queries to the attacked model for CLARE and GSWSE&TF; as a result, this parameter was skipped.
The hyperpartisan news detection task (HN) remained the easiest one; for the victim BERT, GSWSE&TF achieved 0.91 BODEGA score, which is the best value in the attacks against BERT. This high value was possible due to the confusion score on level 1.0, which is the maximum level. The other parameters – the semantic score and the character score – remained the highest values for attacking the BERT victim. The attack against the BiLSTM victim exhibits almost the same properties. GSWSE&TF has the highest scores apart from character score and confusion score, which is on the level of 0.99 and 0.98; only the method DeepWordBug yielded a better result in character score and BERT-ATTACK in confusion score.
The propaganda recognition task (PR2) significantly difers from the HN task in terms of the text length. The PR2 dataset contains rows with more sentences than HN. This is because the former consists of full articles, while the latter contains instances of isolated sentences. As a result, the PR2 task is less likely to achieve high semantics score than the HN task. The highest BODEGA score is on the level of 0.68 for BERT and 0.65 for BiLSTM when the attack is carried out using the CLARE method. These are one of the lowest BODEGA scores in all the tasks excluding the RD task in the BERT victim. As mentioned earlier, the cause of the lower BODEGA score values is mostly the low semantic score in both victims.
The BODEGA score of the fact-checking task (FC) reached the level of 0.8 for both victim models: BERT and BiLSTM. Both of these values were achieved due to the high values of the confusion score and the character score of that task. While the confusion score hit the maximum level of 1.0 for the BERT victim and achieved a relativly high level for BiLSTM (0.98), the character score achieved the level of 0.97. Both of the researched methods – GSWSE&TF and CLARE – achieved significantly better results than the baseline methods.
The rumour detection task (RD) was considered by [
Finally, the covid-19 task (C19) was solved on the level of 0.73 BODEGA score for both victims – BERT and BiLSTM. Achieving high BODEGA score attacking the BERT victim was done by CLARE, which was the best method in the confusion score (0.91) and the semantic score (0.83) and was only slightly worse than the best method in the character score category – DeepWordBug – which accomplished 0.99, whereas CLARE achived 0.96. On the other hand, the highest scores were achieved by diferent methods while attacking BiLMST: the confusion score – Genetic with the value on the level of 0.92; the semantic score – GSWSE&TF (0.85); the character score – DeepWordBug (0.99). However, it is worth to note that in all these cases CLARE was marginally worse, so it amounted to best BODEGA score.
Table 4 inculdes results of the attack on RoBERTa classifier. RoBERTa classifier was made available later than BERT and BiLSTM, which was mentioned ealier. The baseline wasn’t established and we found dificulties to establish them by ourself because of lack of time. Testing the RoBERTa classifier were the most time and resources consuming testing from all classifiers. RoBERTa was a surprise classifier and it was used to evaluate our methods.
In table 4 are presented results of two methods – GSWSE&TF and CLARE. Both of this methods ere the fundation of are research. Because lack of the time and resources we skipped the CLARE methods on the HN and RD tasks. As mention before we struggled with setting the number of queries to the • We found high semantic score the most dificult to achieve. • The larger the text, the more dificult it is to achieve high BODEGA score.
• We found RoBERTa classifier as the most dificult to achieve high the BODEGA score. 5.1. Qualitative Analysis of Manual Evaluation Results The CheckThat! Lab Task 6 is founded on the assumption that adversarial examples can alter classification results without changing the message’s underlying meaning. To support automated verification, a manual annotation procedure was implemented. Adversarial examples were manually classified into three categories: • (a) Preserve the Semantic Meaning, • (b) Change the Semantic Meaning, • (c) No Sense.
A significant discrepancy was observed between automatically generated semantic scores and manual verification results. Automatic scores ranged from 0.68 to 0.86, with CLARE consistently yielding higher results. In contrast, manual scoring indicated that only 0.11 of adversarial examples preserved the semantic meaning. We believe this discrepancy arises from two main sources.
Firstly, while BLEURT aligns well with human judgment, its sensitivity may be lower compared to human annotators, despite its ability to handle multi-word modifications and contradictions.
Secondly, BLEURT processes long text fragments by splitting them into sentences and averaging the semantic similarities between sentence pairs. This approach difers significantly from manual scoring, which evaluates adversarial examples as a whole, resulting in much lower scores.
Upon receiving guidelines for manual annotation, we conducted a qualitative analysis of a subset of adversarial examples generated by our methods. We found that approximately half of these examples introduced incorrect information not present in the original text (e.g., ’Teck was a professional athlete.’ changed to ’Teck was a professional maid.’). About a quarter of the changes were negations of the original text (e.g., ’Grease did not have a soundtrack.’ changed to ’Grease did include a soundtrack.’). Another 10% of examples failed to preserve the semantic meaning for various reasons, such as loss or addition of information, occasional French words substituting English, or pure nonsense. Only about 15% of successful perturbations fully preserved the semantic meaning (e.g. ’Stay safe when you return to work.’ changed to ’staying safe when you return to work.’). This analysis pertains to single perturbations. The datasets contained 2,231 examples, comprising over 13,000 sentences with an average of 6 sentences per example. This implies that most of the semantic meaning of the complete text was preserved, even when the semantic meaning of a single sentence in a multi-sentence text was altered.
In this study, we explored the application of ensemble learning to the domain of adversarial attacks on text classifiers. Our approach was motivated by the principle that leveraging multiple adversarial attack methods can yield better performance compared to any single method. Through systematic experimentation and modification of existing attack algorithms, we aimed to improve the efectiveness and eficiency of adversarial attacks against various victim models.
We began with a literature review and replicated some of the prominent adversarial attack methods, using implementations from the OpenAttack library. Initial experiments indicated that BERT-Attack and Genetic Algorithm were generally efective, though other methods like TextFooler, DeepWordBug, and SCPN also showed promise in specific scenarios. This led us to test these methods on our five datasets: C19, FC, HN, PR2, and RD, initially targeting two victim models: BERT and BiLSTM.
Our first set of modifications to BERT-Attack (resulting in the BAm variant) involved adjusting parameters to enhance the balance between semantic preservation and attack success rate. These adjustments improved performance across several metrics, as shown in our comparative analysis.
Further, we introduced a novel method for ranking word importance, which led to the BAm2 variant. This method evaluates the impact of potential word substitutions on model predictions, thereby allowing more informed and efective attacks. Additionally, for cases where BAm2 failed, we integrated the Genetic Algorithm to bolster the attack success rate. This combination, BAm2&Genetic, yielded significant improvements, particularly in the BODEGA and success scores.
We then explored an alternative approach, GSWSE, which employs word embeddings for synonym replacement. This method, coupled with TextFooler for the challenging cases, ofered further enhancements in BODEGA score, albeit with a slight reduction in the success rate.
In the final stage of our experiments, we used the CLARE method. CLARE’s innovative mask-theninfill procedure, leveraging a pre-trained masked language model, demonstrated superior performance across all metrics. It provided a high attack success rate while maintaining textual similarity and fluency, making it the most efective method in our ensemble.
Finally, the developed solutions were tested on the third victim model: RoBERTa. The evaluation confirmed the efectiveness of our approach in generating adversarial text in five problem domains.
Our findings underscore the potential of ensemble learning in the adversarial attack domain. By combining multiple attack strategies and continuously refining our methods, we achieved notable improvements in attack efectiveness. Future work could explore additional ensemble configurations and further optimizations, aiming to develop even more robust adversarial attack frameworks.
In future works we plan to extend the methods by using various data sources and large language models (LLMs). For example, as a comprehensive text corpus, Wikipedia can be used to train or fine-tune language models for better contextual understanding and generation. Using Wikipedia articles, the quality of adversarial examples can be further enhanced by providing more natural and contextually appropriate substitutions, particularly when we can consider quality diferences between Wikipedia articles in various language versions [22]. Especially, we plan to use publicly available services with measures related to information quality and reliability of sources, such as BestRef [23], WikiRank [24] and various revscoring models [25]. Future research can leverage the analysis of Wikipedia references to enhance adversarial attacks by incorporating high-quality scientific sources into training datasets, ensuring more credible and contextually relevant adversarial examples [26].
Such structured knowledge bases as Wikidata and DBpedia can provide rich semantic context and relationships between entities. Integrating those and other data sources [27] could help in refine word importance ranking and improve the selection of substitutes by ensuring semantic consistency and relevance. For example, entity linking and relationship extraction from these knowledge bases can ensure that replacements preserve factual accuracy and context.
The advancements in adversarial attack strategies can be further developed by integrating insights from the analysis of large language models (LLMs) like ChatGPT on the fake news phenomenon [28]. Utilizing state-of-the-art and novel LLMs, such as Claude, Gemini, GPT-4, LLama, Mistral, can improve the quality of generated adversarial examples. Fine-tuning these models on specific datasets or incorporating them into the ensemble can lead to more sophisticated and efective attacks. LLMs can also be used to simulate human-like understanding and generation of text, making adversarial examples more natural and harder to detect. Moreover, combining traditional adversarial attack methods with the capabilities of LLMs can create hybrid approaches that leverage the strengths of both. For example, initial perturbations can be generated using classical methods, followed by refinement and enhancement using LLMs. in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, pp. 6174–6181. URL: http://arxiv.org/abs/2004.01970. doi:10.18653/v1/2020. emnlp-main.498, arXiv:2004.01970 [cs]. [15] S. Ren, Y. Deng, K. He, W. Che, Generating Natural Language Adversarial Examples through Probability Weighted Word Saliency, in: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Florence, Italy, 2019, pp. 1085–1097. URL: https://www.aclweb.org/anthology/P19-1103. doi:10.18653/v1/P19-1103. [16] Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Qun Liu, Maosong Sun, Word-level Textual Adversarial Attacking as Combinatorial Optimization, 2020, pp. 6066–6080.
URL: https://api.semanticscholar.org/CorpusID:261432085. [17] D. Jin, Z. Jin, J. T. Zhou, P. Szolovits, Is bert really robust? a strong baseline for natural language attack on text classification and entailment, 2020. arXiv:1907.11932. [18] P. Przybyła, B. Wu, A. Shvets, Y. Mu, K. C. Sheang, X. Song, H. Saggion, Overview of the clef2024 checkthat! lab task 6 on robustness of credibility assessment with adversarial examples (incrediblae), in: Working Notes of CLEF 2024 - Conference and Labs of the Evaluation Forum, CLEF 2024, Grenoble, France, 2024. [19] A. B.-C. no, F. Alam, T. Chakraborty, T. Elsayed, P. Nakov, P. Przybyła, J. M. Struß, F. Haouari, M. Hasanain, F. Rugger, X. Song, R. Suwaileh, The clef-2024 checkthat! lab: Check-worthiness, subjectivity, persuasion, roles, authorities, and adversarial robustness, in: Advances in Information Retrieval, Springer Nature Switzerland, Cham, 2024, p. 449–458. [20] OpenAttack, An open-source package for textual adversarial attack, 2021. URL: https://openattack.
readthedocs.io/en/latest/. [21] N. Mrkšić, D. O. Séaghdha, B. Thomson, M. Gašić, L. Rojas-Barahona, P.-H. Su, D. Vandyke, T.-H. Wen, S. Young, Counter-fitting word vectors to linguistic constraints, arXiv preprint arXiv:1603.00892 (2016). [22] W. Lewoniewski, K. Węcel, W. Abramowicz, Companies in multilingual wikipedia: Articles quality and important sources of information, in: Special sessions in the advances in information systems and technologies track of the conference on computer science and intelligence systems, Springer, 2023, pp. 48–67. [23] BestRef, The best information sources of Wikipedia - assessment by BestRef, https://bestref.net, 2024. [24] WikiRank, Quality and Popularity Assessment of Wikipedia Articles, https://wikirank.net/, 2024. [25] Wikitech, Machine Learning/LiftWing, https://wikitech.wikimedia.org/wiki/Machine_Learning/
LiftWing, 2024. [26] W. Lewoniewski, K. Węcel, W. Abramowicz, Understanding the use of scientific references in multilingual wikipedia across various topics, Procedia Computer Science 225 (2023) 3977–3986. [27] S. Hellmann, J. Frey, M. Hofer, M. Dojchinovski, K. Węcel, W. Lewoniewski, Towards a systematic approach to sync factual data across wikipedia, wikidata and external data sources, in: CEUR Workshop Proceedings, volume 2836, 2021. [28] K. Węcel, M. Sawiński, M. Stróżyna, W. Lewoniewski, E. Księżniak, P. Stolarski, W. Abramowicz, Artificial intelligence—friend or foe in fake news campaigns, Economics and Business Review 9 (2023). URL: https://journals.ue.poznan.pl/ebr/article/view/736. doi:10.18559/ebr.2023.2. 736.