The rapid advancements in Artificial Intelligence, especially in the development of Large Language Models, have introduced a domain abundant with opportunities while presenting considerable challenges. Although LLMs are demonstrating impressive abilities in understanding and generating human-like text, they have also contributed to a rise in disinformation. An outstanding example of this influence is COVID-19 pandemic, when fake news and conspiracy theories impacted individuals and society. This paper aims to determine whether LLMs can efectively rephrase human-like text and, utilize their knowledge base in detecting rephrased disinformation, a subtle and increasingly prevalent form of manipulated content. Furthermore, this research investigates the criteria and reasoning behind how LLMs classify text as real or fake by LLMs, aiming to assess whether LLMs can be used to counter disinformation while examining the challenges associated with their use for this purpose.
LLMs are evolving rapidly, bringing along transformation in a variety of fields. They have inspired a
range of applications that not only help in understanding human behavior but also provide insights into
LLMs themselves by drawing upon human societal contexts [
This paper aims to explore two primary research questions: Disinformation, Misinformation and Learning in the Age of Generative AI: Joint Proceedings of the 1st International Workshop on Disinformation and Misinformation in the Age of Generative AI (DISMISS-FAKE’25) and the 4th International Workshop on Investigating Learning during Web Search (IWILDS’25) co-located with 18th International ACM WSDM Conference on Web Search and Data Mining (WSDM 2025) * Corresponding author. focuses on the results related to RQ1, providing insights into the ability of LLMs to generate human-like text. Section 5 discusses the findings for RQ2, highlighting the efectiveness of LLMs in detecting fake text and common types of errors observed. Finally, Section 6 ofers a conclusion and suggests directions for future research.
LLMs, being trained on vast and diverse data, possess advanced linguistic capabilities, making it dificult
to distinguish between human-generated and LLM-generated content. A comprehensive survey on the
challenges of diferentiating between these two types of texts was presented in a 2020 paper, following
the introduction of GPT-2 [
In the domain of disinformation detection, fine-tuned models based on BERT have shown success
in identifying LLM-generated text [
In light of the research questions posed in this study, our work aims to build on these findings by: (1) examining the extent to which LLMs can generate human-like text and its impact on detection, and (2) evaluating the efectiveness of these models in detecting disinformation, particularly in the context of a COVID-19-related dataset.
Figure 1 illustrates our approach, which begins with a human-written dataset. Using prompting techniques, we generated a rephrased LLM-generated database. Rephrasing was selected as the method for generating LLM text to reflect common strategies used by adversarial actors, who often tries to modify and make existing misinformation to appear more factual to evade detection. This method ensures that the original disinformation persists within the newly generated content.
During the linguistic and syntactic analysis phase, we assess any potential similarities between human-written and LLM-generated data (RQ1). Finally, we employ LLMs as a detection method, using prompts to evaluate their ability to identify disinformation (RQ2).
We used two diferent LLMs, Gemini-1.5-flash and LLaMA-3 8b-8192, to leverage their unique
strengths and ensure an unbiased evaluation of disinformation detection capabilities. This approach
also allows us cross-validation of findings across diferent architectures, while including both commercial
and opens-source solutions. Gemini-1.5-flash uses self-attention mechanism which enables it to process
nuanced language and identify subtle indicators of disinformation by considering the full context of
a passage. Its architecture is particularly suited for tasks requiring detailed contextual analysis [
Our dataset comprises manually annotated dataset of 10,700 social media posts and articles related
to real (5,600) and fake (5,100) COVID-19 news [
For LLM dataset generation, we employed Gemini-1.5-flash and LLaMA-3 8b-8192. To create diverse
outputs, we used two rephrasing prompting strategies: paraphrase generation and rewriting
generation [
Prompt 1: Para- You are a journalist. Given a “passage”, please paraphrase it. Do not generate any phrase Generation other word. The “passage” is <text>.
Original Text The main mode of transmission of #Novel #Coronavirus is through droplets and it is possible that infected smokers may blow droplets carrying the virus when they exhale. Regardless of #COVID19 you should steer clear of second-hand smoke as it may cause various health problems. gemini-1.5-flash The novel coronavirus spreads primarily through droplets, which infected smokers may exhale. Secondhand smoke is harmful and should be avoided. llama3-8b-8192 The primary way the new coronavirus spreads is through tiny droplets, and it’s conceivable that smokers who are infected may release virus-laden droplets when exhaling. It’s not limited to COVID-19; second-hand smoke can lead to various health issues, regardless of the virus.
To assess the similarity between human-written text and the text generated by LLMs, we conducted a
linguistic analysis. This analysis aimed to determine whether our prompting methods can modify
humanwritten content in a meaningful way and whether they may inadvertently introduce disinformation.
We employed the Linguistic Inquiry and Word Count (LIWC) [
The “passage” is: <text> .
Original Text The main mode of transmission of #Novel #Coronavirus is through droplets and it is possible that infected smokers may blow droplets carrying the virus when they exhale. Regardless of #COVID19 you should steer clear of second-hand smoke as it may cause various health problems. gemini-1.5-flash The primary route of transmission for the novel coronavirus is through respiratory droplets. Individuals infected with COVID-19 may release these droplets, potentially containing the virus, when they exhale, including when smoking. While the potential for transmission through secondhand smoke remains under investigation, it is important to note that secondhand smoke is known to cause a variety of health issues. As a precautionary measure, it is strongly advised to avoid exposure to secondhand smoke in all situations. llama3-8b-8192 The primary means of transmission for the novel coronavirus is via airborne droplets.
Furthermore, research suggests that individuals infected with the virus may
inadvertently spread it through the exhalation of droplets when they breathe. It is
essential to note that this phenomenon is not exclusive to the current
COVID19 pandemic. Exposure to second-hand smoke, a common phenomenon in
various settings, has been linked to a range of serious health issues. As a result,
it is crucial to maintain a safe distance from second-hand smoke, regardless of
the prevailing health situation, to minimize the risk of developing respiratory
problems and other illnesses.
between human-written and LLM generated text. The LIWC2015 Dictionary contains approximately
6,400 words, word stems, and select emoticons, categorized under linguistic or psychological
subdictionaries. Words may belong to multiple categories. For example, the word ’cried’ falls into five
categories: sadness, negative emotion, overall afect, verbs, and past focus. If this word appears in a
text, it increases the count for each of these categories accordingly [
(a) Human-Written Text (b) Text Rephrased by Llama3 (c) Text Rephrased by Gemini
We analyzed the LIWC distributions across human-written and LLM-generated text. While LIWC features for the text generated by both LLMs with Prompt 1 are distributed very similar to the humanwritten ones, for Prompt 2 distributions defer significantly. Figures 2 and 3 illustrate these diference for both "Real" and "Fake" tweets respectively. The box plots depict the distribution of LIWC features in the original human-written texts and their corresponding rephrased versions generated by the LLMs.
Prompt 2, which rewrites the text to make it more persuasive with a more serious and informative tone, a number of LIWC categories demonstrate significant changes with notable shifts highlighted in red. Llama-generated texts generally difer more from human-generated ones. While occurrences of categories such as function words, prepositions and articles are easy to understand, others such as ’cognitive processes’ (cogproc) and ’social processes’ (social) require deeper analysis. It is worth noting that although Gemini stick more closely to human-written text, when it comes to fake tweets, the ’cognitive processes’ category becomes more frequent.
To further quantify the similarity between human-written and LLM-generated text, the cosine similarity of their LIWC feature values was calculated. Among 8,880 tweets analyzed, 5,854 tweets (4,168 labeled "Real" and 1,686 "Fake"), had cosine similarity scores exceeding 0.8 across both LLMs and prompts. A high cosine similarity score suggests that the LLM-generated texts maintained similar feature distributions to the human-written texts, despite potential diferences in the absolute frequencies of individual features. In essence, while the LLMs may have altered the intensity or frequency of certain linguistic elements, the overall pattern and structure of the feature distributions remained consistent with the human-written content. Despite high cosine similarity, manual verification of some LLMgenerated text revealed factual inaccuracies. This indicates that while LLMs can produce text with linguistic patterns that closely mimic human writing, their rephrased content may still introduce factual errors, raising concerns about the risk of disinformation.
In this section, we assess how efectively the models distinguish between real and fake information and analyze the misclassifications to identify potential patterns. Initially, the LLM-generated outputs for both prompts were manually reviewed, and a balanced subset of 800 tweets (400 real and 400 fake) was selected. This manual review ensured that no factual inaccuracies were introduced during paraphrasing process in sampled data.
To further utilize LLMs for disinformation detection, we conducted experiments with the same models using identical parameters to those used during the text generation phase. The following prompt was used for detection tasks:
Prompt: Given a passage, determine whether it is a piece of disinformation. Only output ’YES’ or ’NO’. The passage is: <text>
Detection performance of bot models is shown in table 3 Gemini demonstrated better performance than Llama3 in identifying LLM-generated disinformation across both prompt types. As shown in Appendix A, Gemini initially exhibited balanced performance, achieving a True Positive Rate (TPR) of 65.5%and a True Negative Rate (TNR) of 90.6% for human-written texts. After rephrasing, Gemini improved in detecting real tweets, particularly with prompt GP1, where it reached a TPR of 81.8%, though its ability in detecting fake tweets slightly declined.
For Llama3, rephrasing improved real tweets detection, particularly with prompt GP2, achieving 88.4% TPR. However, its ability to detect fake tweets declined significantly, highlighting the model’s sensitivity to rephrasing.
To identify patterns in the model’s misclassifications, we conducted a detailed error analysis.
Rephrasing Impact: Llama3 model exhibited a slightly higher number of changes when rephrasing was applied compared to the Gemini model (Table 14). This suggests that Llama3 may be more sensitive or responsive to rephrased text. Notably, the Llama3 was more successful at correcting errors through rephrasing than the Gemini model, as indicated by a higher number of corrected misclassifications. However, Llama3 also displayed a greater tendency to amplify errors under certain prompts compared to Gemini. This variation in model performance highlights the importance of prompt engineering in determining how rephrasing influences both error correction and exacerbation.
LIWC Analysis: We also explored whether linguistic features contribute to misclassification (refer Figure 4). High counts in categories such as "Article," "Pronoun," and "Drives" in TP suggest these are characteristic of genuine tweets correctly classified. In contrast, categories like "Relativ," "Prep," and "Verb" appeared less frequently in FPs than in TPs and TNs, suggesting that while present, these features are less prominent in genuine tweets misclassified as fake. Similarly, "Article," "Drives," and "Cogproc" (Cognitive Processes) were significant in FNs, similar to their presence in TPs. This points to the challenge of distinguishing fake content that closely mimics the structure of real tweets.
LLM Analysis: We further refined our analysis by focusing on tweets where both Gemini and Llama3, regardless of the prompt used, changed their classification from the gold label. This filtering resulted in a subset of 29 tweets. To understand the reasoning behind these classification changes, we employed the chain-of-thought prompting technique, instructing Gemini and Llama to explain their decision-making processes. The models were prompted to consider factors such as tone, context, wording, or implied meaning that influenced their updated classifications.
Prompt: You are analyzing tweets to classify them based on their content. Below are the details of an original tweet and its rephrased version. Please read carefully and answer the following: Original Tweet: <text> Original Classification by you: <text> Now, consider the rephrased version: Rephrased Tweet: <text> Rephrased Classification: <text>
Question: Explain why you changed your classification for the rephrased tweet compared to the original tweet. Discuss any diferences in tone, context, wording, or implied meaning that led to the new classification.
Manual review of the models’ responses, revealed that certain aspects of the rephrased tweets, such as inclusion of words like ’joke’, ’disturbing’ or mentions of news agencies, prompted the model to alter their classification. Similarly, provocative or emotionally charged language (e.g., exaggerated claims) often led to a change in labels. In contrast, rephrased tweets with neutral, factual tone or including credible source also triggered change in classification. For detailed overview, refer to Figure 5 in Appendix B.
In conclusion, LLMs such as Llama3 and Gemini can generate human-like text with high linguistic similarity to human-written content. While these models show potential for detecting disinformation, their susceptibility to rephrased content indicate the need for further development. Our experiments confirm LLMs’ ability to create human-like text, though their performance in detecting disinformation is more complex. While Gemini showed greater accuracy in identifying real content, but both models displayed weaknesses, particularly when dealing with rephrased text. Factors such as prompt selection, text length, and model type can impact their efectiveness. The results from linguistic analysis using LIWC suggests that while LLMs can closely replicate the linguistic patterns of human writing, but they remain prone to errors, especially when subtle text modifications are introduced. This raises concerns about their reliability as standalone detectors, as adversarial actors could exploit these vulnerabilities by employing techniques like rephrasing.
During the preparation of this work, the author(s) used ChatGPT, DeepL Write in order to: Grammar and spelling check, Paraphrase and reword, Improve writing style. After using this tool/service, the author(s) reviewed and edited the content as needed and take(s) full responsibility for the publication’s content.
This section presents the confusion matrix for the LLMs used as detectors. We provided the original human-written text to both Gemini and Llama, followed by the output generated from Prompt 1 by both models, and then the output from Prompt 2 by both models, efectively creating a cross-evaluation. The tables below illustrate the confusion matrix for each scenario.
The table below illustrates the impact of text rephrasing on classification by the LLMs.
Figure 5: Error Analysis: Change in classification