The advent of LLMs has revolutionized the way we produce text, greatly increasing our productivity. However, dual uses such as misinformation generation or human impersonation have also become a concern, making it desirable to be able to automatically distinguish between human and AI-generated text. In this paper, we present an ensemble method to detect AI-generated based on how suprising the text is for a group of open LLMs. The rise of Large Language Models (LLMs) is a new technological revolution, ofering the ability to generate human-like text at an unprecedented scale. While this promises great increases in productivity, it also raises concerns about the potential for misuse. In particular, LLMs can be used to generate fake news, impersonate people, or spread misinformation. As a result, there is a growing need for tools that can detect AI-generated text. In this paper, we propose an approach to detect AI-generated text based on the distribution of token probabilities of multiple open LLMs. This approach is submitted to the Voight-Kampf Generative AI Authorship Verification 2024 task [1] at PAN [2].
It is well-known that LLMs generate text one token at a time, predicting the next token based on the previous ones. To do this, the model outputs a distribution of probabilities over the vocabulary and the next token is sampled from this distribution using a fixed strategy.
Applying an LLM to a full text results in a sequence of probability distributions, one for each token.
If the text is generated by that LLM, we expect the following token to have a high probability in the
conditional distribution generated by all the previous ones. On the other hand, human text is expected
to contain more statistical outliers, and be more “surprising” to the model on average. This idea has
been used in the past, in works such as Ippolito et al. [
This method presents a series of problems.
1. First, we do not know or do not have access to the exact model that generated the text. Although
we expect most models to be pretrained on a similar large corpus of text, the choice of vocabulary
or fine-tuning data and procedure might afect the probabilities.
2. Second, we do not have access to the prompt that was used, which strongly conditions the
generation of the text. Without the prompt, the perplexity of the text will naturally be higher, as
the model does not see the context that conditioned it. This is especially the case if the prompt
forces the model to generate text about an unusual topic or in an unusual style. This problem
was already pointed out by Hans et al. [
Using the rank of the target token in the probability distribution already hinted at the third problem, although very unconditioned tokens might still yield high ranks.
Hans et al. [
If we consider using the same LLM 1 = 2, then the entropy values measure how much randomness there is in the distribution that generated the token. Normalizing by this value reduces the importance of having a low probability in a high entropy distribution, helping with the second and third problem. In practice, the authors used very similar LLMs, such as the foundation and instruct versions of the same model—perhaps because the tokenizer needed to be identical—. We expect distributions to be close and this interpretation to be valid.
Another similar idea is that of Bao et al. [
Our method consists of three main steps. First, we compute a metric derived from the log-probabilities output by an LLM, trying to solve the second and third problems discussed in the background. For each LLM, we get a distribution of this metric across the full sequence. Second, we extract a set of features from these distributions, as well as some classic NLP features from the text. Finally, we use these features to train a classifier that predicts whether the text was generated by an LLM or not.
can compute a normalized log-probability We use a metric that takes into account both the probability of the next token and how randomly the probability is distributed. Consider a sequence of tokens 1, . . . , . After passing through an LLM, we get a matrix of logits ∈ R× , where is the vocabulary size. Then, for all from 1 to − 1, we NLP = ︂(
max ∈{1,..., }
︂) , − ,+1 .
This metric compares the likelihood of the token +1 with the likelihood of the most likely token. If the token is the most likely one, then the value will be zero. If the token is very unlikely, then it will only be penalized if there are tokens that are much more likely.
The previous procedure yields, for a given text and LLM, a sequence of normalized log-probability values NLP1, . . . , NLP− 1. We propose to model it as a set of observations from a distribution. We can then extract features such as the mean, the variance, or the quantiles of this distribution. We also extract some features from seeing it as an ordered sequence. After getting these features, we concatenate them for several LLMs. Further, we add some classic NLP features such as the ratio of stop words, the ratio of punctuation, the ratio of each syntactic category...
Finally, we use an XGBoost classifier to predict whether the text was generated by an LLM or not based on the features.
We use the following models, given their relevance in the state-of-the-art literature and our
computational resources:
• google/gemma-2b-it [
For each model and its respective values NLP1, . . . , NLP− 1, we extract the following features: the mean, the standard deviation, the minimum value, the ratio of values that are close to zero, and the ratio of values where the sequence is increasing.
We also extract the following classic NLP features: • Ratio of verbs, nouns, adjectives, adverbs, personal pronouns, prepositions or subordinating conjunctions, coordinating conjunctions, determiners, interjections, particles, and cardinal numbers. • Ratio of words of other types. • Number of words.
• Mean sentence length.
We tune the XGBoost hyperparameters using cross-validation, and train using only the given dataset. As the dataset is unbalanced, we use oversampling for training and balanced accuracy as validation metric.
Finally, we train for raw classification, but TIRA [ 13] submissions require distinguishing, for a given pair of texts, which text is AI-generated. Thus, we need to transform our predictions. If we get probabilities of being human 1 and 2 from the first and second texts, respectively, we output 2− 1+1 . 2
We present the results of our model across cross-validation sets in the binary classification task of
detecting if a given text is AI-generated or not. We compare our model against two baselines: Binoculars [
Table 2 shows the results, initially pre-filled with the oficial baselines provided by the PAN organizers and summary statistics of all submissions to the task (i.e., the maximum, median, minimum, and 95-th, 75-th, and 25-th percentiles over all submissions to the task). In this case, the task is to distinguish which of two given texts is generated by AI. The results show that our model outperforms the baselines, although with a small margin. Our model appears to be outperformed by other solutions, except in the ROC-AUC metric. Further discussion could be provided with more knowledge of the test sets and other solutions.
Table 3 shows the summarized results averaged (arithmetic mean) over 10 variants of the test dataset. Each dataset variant applies one potential technique to measure the robustness of authorship verification approaches, e.g., switching the text encoding, translating the text, switchign the domain, manual obfuscation by humans, etc. 95-th quantile 75-th quantile Median 25-th quantile Min
ROC-AUC Brier C@1
F0.5 Mean
We present a method to detect AI-generated text based on a small improvement to the log-probability metric. We ensemble the predictions from several LLMs, as well as classic NLP features. Our method achieves promising results, but further study on the generalization of the method is needed, as well as a proper ablation study.
Minimum 25-th Quantile Median 75-th Quantile Max (a) LLM feature importances by model.
(b) LLM feature importances by statistic.
(c) Non-LLM feature importances.
Future work is needed to study the best way to normalize the log-probabilities to account for the problems we have discussed. Further, we should also take into account that human text is more likely to take the LLM into unchartered territory, statistically speaking. This problem and the previously discussed ones conflict, and balancing them might be needed.
Finally, one can also study the interaction between the predictions of diferent LLMs. We believe that having many LLMs with a diversity of training data, training procedures and vocabularies is helpful. Studying which LLMs are more likely to agree could help selecting a minimal ensemble of models, reducing the computational cost of the method.
This work has been funded by the project PCI2022-134990-2 (MARTINI) of the CHISTERA IV Cofund 2021 program, funded by MCIN/AEI/10.13039/ 501100011033 and by the “European Union NextGenerationEU/PRTR”; by the Spanish Ministry of Science and Innovation under FightDIS (PID2020-117263GBI00); by MCIN/AEI/10.13039/501100011033/ and European Union NextGenerationEU/PRTR for XAIDisinfodemics (PLEC2021-007681) grant. This publication has been also written in the context of Iberifier Plus project, co-funded by the European Commission under the Call DIGITAL-2023-DEPLOY-04, European Digital Media observatory (EDMO) – National and multinational hubs; reference: IBERIFIER Plus 101158511.