The ability to accurately detect machine-generated text is becoming increasingly important in various fields, including academia, journalism, and online security. In this study, we propose a novel method for detecting machine-generated text, predicated on the hypothesis that the probability of reasoning from human-generated text to machine-generated text is inherently higher. Our approach is inspired by the principles of Natural Language Inference (NLI), leveraging the diferences in logical consistency and contextual coherence between human and machine-generated texts. However, our experimental results indicate that this method may not be as efective as anticipated. Despite the theoretical foundation, the practical application of our method revealed significant limitations, suggesting that it might not be a reliable solution for detecting machine-generated text. Further research and refinement are necessary to enhance the eficacy of detection techniques.
The rapid advancement of artificial intelligence has led to the widespread use of machine-generated
text in various domains [
Detecting machine-generated text is crucial for maintaining the integrity of information. In academia, it helps prevent plagiarism and ensures the originality of scholarly work. In journalism, it safeguards against the dissemination of fake news and misinformation. In online platforms, it enhances security by identifying automated accounts and reducing the spread of malicious content. Despite the growing need for efective detection methods, current techniques often fall short. Traditional approaches typically focus on stylistic and linguistic features, which can be easily manipulated by advanced language models. As a result, there is a pressing need for more robust and reliable detection methods.
In this study, we propose a novel approach inspired by Natural Language Inference (NLI). Our method is based on the hypothesis that we are able to judge which text is generated by human by comparing the probability of reasoning (See Section 4). By leveraging the logical consistency and contextual coherence diferences between human and machine-generated texts, we aim to develop a more accurate detection model.
However, our experimental results suggest that this method may not be as efective as initially anticipated. Despite its theoretical promise, practical application revealed significant limitations, highlighting the complexity of the detection problem. This paper presents our findings and discusses the implications for future research in this area.
This work was developed for the PAN task——Generative AI Authorship Verification [
Several studies have approached the detection of AI-generated text as a binary classification problem
using neural network-based detectors [
However, relying on neural networks for detection can expose these methods to adversarial and
poisoning attacks [
NLI is an NLP task involving determining the relationship between two sentences: whether one sentence (the hypothesis) can be inferred from another sentence (the premise). It has been proven that NLI can be used for inconsistency detection in summarization where the source document acts as the premise, and the generated summary acts as the hypothesis [16]. The NLI model evaluates whether the information in the summary can logically be inferred from the source document.
Inspired by the usage of NLI in the summarization task, we detect the machine-generated text by ways of detecting the logical relationship between the premise and hypothesis.
The model checks for three possible relationships between the premise and hypothesis: Entailment: The hypothesis (summary) logically follows from the premise (source text). Contradiction: The hypothesis contradicts the premise.
Neutral: There is no clear logical relationship, meaning the hypothesis might add information not present in the premise or omit critical information.
Given two texts, text_a and text_b, one authored by a human and the other generated by a machine, we calculated the probability of reasoning for each text pair independently. Assume that the probability of reasoning from text_a to text_b is (text_a → text_b) while the probability of reasoning from text_b to text_a is (text_b → text_a). If (text_a → text_b) is larger than (text_b → text_a), we could assume that text_a was written by human. It is worth noting that we did not conduct any pre-processing (i.e. segmentation) in order to provide suficient contexts for ratiocination by our model. Our hypothesis is as follows.
As known, the premise provides the basis or groundwork for a conclusion while the hypothesis, in a logical structure, is a statement whose validity is supported by the premise. On the one hand, the machine-generated text in our task was generated based on the human-written text, which means that the human-generated text provides the foundation thus the human-written text should be the premise. On the other hand, the text generated by AI may not match human authors in terms of semantic coherence and logical depth [17]. Accordingly, it is impossible to derive the human-generated text on the basis of the machine-written one.
Beside, we define that if the diference between (text_a → text_b) and (text_a → text_b) is lower than 0.05, their relation is neutral, meaning there is no clear logical relationship between these two texts.
The language model for NLI was DeBERTa-v3-large-mnli-fever-anli-ling-wanli which is a fine-tuned model specifically for NLI tasks [ 18] for the reason that this model was fine-tuned on distinct datasets including FEVER (Fact Extraction and VERification), ANLI (Adversarial NLI), and WANLI (Weaklysupervised ANLI).
We compared our model, detector, with the baseline models. The performance metrics indicates that the detector model significantly underperforms compared to all baseline approaches. Specifically, the detector (our model) achieved a ROC-AUC of 0.548, which is the lowest among all models, indicating poor discriminative ability. Its Brier score is 0.622, suggesting less accurate probabilistic predictions, while its C@1 score of 0.489 is the lowest, reflecting suboptimal performance. The detector’s F1 score of 0.442 and F0.5u score of 0.461 are also the lowest, indicating poor balance and precision-focused performance, respectively. In contrast, Baseline Binoculars exhibits the highest performance across most metrics, with a ROC-AUC of 0.972, a Brier score of 0.957, and C@1, F1, and F0.5u scores all around 0.965. The overall mean score of Baseline Binoculars is 0.965, compared to the detector’s mean of 0.512. The Fast-DetectGPT (Mistral) baseline also performed well, with a ROC-AUC of 0.876 and a mean score of 0.866. Quantile-based evaluations show the 95-th quantile achieving the highest scores, with a ROC-AUC of 0.994 and a mean score of 0.990, underscoring the best performance of the top 5 percentages of models.
Table 2 also shows the results, initially pre-filled with the oficial baselines provided by the PAN organizers and summary statistics of all submissions to the task (i.e., the maximum, median, minimum, and 95-th, 75-th, and 25-th percentiles over all submissions to the task).
We analyzed the reason why our model has such bad performance.
Firstly, our method relies on a single feature—logical inference—which might be insuficient for a comprehensive detection mechanism. Successful detection methods typically incorporate multiple features, including linguistic, syntactic, and semantic analysis, to capture the multifaceted nature of human versus machine-generated text. It suggests that we should establish more comprehensive classification features.
Besides, modern AI models like GPT-3 and GPT-4 are designed to generate text that closely mimics human writing, including coherence and detail. Consequently, the distinction between detailed AIgenerated text and detailed human text becomes blurred. Human writers can also produce highly detailed and coherent text, especially in structured or formal contexts. This overlap reduces the efectiveness of using coherence and detail as discriminative features.
Human-generated text can also exhibit inferential relationships, especially in informative or explanatory writing. For instance, when humans explain concepts or provide detailed descriptions, their sentences can logically infer one another. As mentioned, the dataset involves multiple genres. Our method might frequently misclassify detailed and coherent human text (such as news articles) as AI-generated, leading to a high rate of false positives.
From the NLI model’s perspective, the method we use is zero-shot which means that our model has not been specifically trained or fine-tuned on a dataset of human vs. AI-generated texts. Also, DeBERTa’s strength in recognizing logical relationships might lead it to frequently detect coherent inferences in both human and AI texts, making it dificult to distinguish between them based solely on coherence. This means it may not be optimized to distinguish the subtle diferences between the two types of text. 6. Further Direction To enhance the performance of AI-generated text detection method, it is crucial to fine-tune the DeBERTa model specifically on a dataset tailored for distinguishing human and AI-generated text. This specialized training will help the model learn the unique patterns and nuances of the task. Additionally, incorporating a broader feature set, including stylistic markers, syntactic complexity, and lexical diversity, can provide a more robust classification framework. Employing ensemble methods that combine zero-shot NLI models with supervised models trained on the detection task can further improve performance by leveraging the strengths of diferent approaches. Regular evaluation and refinement using diverse and updated datasets will ensure the model adapts to new patterns in text generation. Lastly, utilizing contextual embedding techniques can capture richer text representations, enabling deeper contextual analysis beyond simple logical inference.
In this study, we explored the potential of using Natural Language Inference (NLI) to detect machinegenerated text by examining the logical relationship between premises and hypotheses. Our hypothesis was that machine-generated text, being more detailed and coherent due to probabilistic generation, would difer significantly from human text in inferential relationships. However, our experimental results revealed significant limitations in this approach. Specifically, our zero-shot method using the "DeBERTa-v3-large-mnli-fever-anli-ling-wanli" model underperformed across various metrics, including ROC-AUC, Brier score, C@1, F1, and F0.5u scores, when compared to baseline models. The primary reasons for this underperformance include the overlap in coherence and detail between human and AI-generated texts, the limitations of a single-feature approach based solely on logical inference, and the model’s lack of fine-tuning on a task-specific dataset. Our findings suggest that successful detection of AI-generated text requires a multifaceted approach, incorporating diverse linguistic features and specialized training. Future work should focus on fine-tuning models on relevant datasets and integrating additional classification features to improve the robustness and accuracy of detection methods.
We appreciate the help from Simon Clematide and Andrianos Michail who provided their suggestions to improve our work. We would also extend our sincere gratitude to the anonymous reviewer whose insightful comments and suggestions significantly contributed to the improvement of this manuscript. preprint arXiv:1412.6572 (2014). [13] V. S. Sadasivan, M. Soltanolkotabi, S. Feizi, Cuda: Convolution-based unlearnable datasets, arXiv preprint arXiv:2303.04278 (2023). [14] J. Kirchenbauer, J. Geiping, Y. Wen, M. Shu, K. Saifullah, K. Kong, K. Fernando, A. Saha, M. Goldblum, T. Goldstein, On the reliability of watermarks for large language models, arXiv preprint arXiv:2303.04278 (2023). [15] X. Zhao, Y.-X. Wang, L. Li, Protecting language generation models via invisible watermarking, arXiv preprint arXiv:2302.03162 (2023). [16] P. Laban, T. Schnabel, P. N. Bennett, M. A. Hearst, Summac: Re-visiting nli-based models for inconsistency detection in summarization, Transactions of the Association for Computational Linguistics 10 (2022) 163–177. [17] O. Marchenko, O. Radyvonenko, T. Ignatova, P. Titarchuk, D. Zhelezniakov, Improving text generation through introducing coherence metrics, Cybernetics and Systems Analysis 56 (2020) 13–21. [18] P. He, J. Gao, W. Chen, Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing, arXiv preprint arXiv:2111.09543 (2021).