The “Voight-Kampf” Generative AI Authorship Verification task aims to determine whether a text was generated by an AI or written by a human. As in its fictional inspiration, 1 the Voight-Kampf task structures AI detection as a builder-breaker challenge: The builders, participants in the PAN lab, submit software to detect AI-written text and the breakers, participants in the ELOQUENT lab, submit AI-written text with the goal of fooling the builders. We formulate the task in a way that is reminiscent of a traditional authorship verification problem, where given a pair of texts, their human or machine authorship is to be inferred. For this first task installment, we further restrict the problem so that each pair is guaranteed to contain one human and one machine text. Hence the task description reads: Given two texts, one authored by a human, one by a machine: pick out the human. In total, we evaluated 43 detection systems (30 participant submissions and 13 baselines), ranging from linear classifiers to perplexity-based zero-shot systems. We tested them on 70 individual test set variants organized in 14 base collections, each designed on diferent constraints such as short texts, Unicode obfuscations, or language switching. The top systems achieve very high scores, proving themselves not perfect but suficiently robust across a wide range of specialized testing regimes. Code used for creating the datasets and evaluating the systems, baselines, and data are available on GitHub.2
Generative AI is undoubtedly a disruptive technology in the information ecosystem. In particular, large language models (LLMs) have many desirable applications in writing assistance and information access. Although often welcome, unlimited text generation raises concerns in many areas of creation; examples include education and assessment, academic articles and reviews, synthetic misinformation and disinformation, and social bots that influence public discourse. These troubling applications undermine trust in (written) information. Recognizing the fingerprint of AI-generated text thus becomes a promising element for a healthy future information ecosystem, which will become increasingly sophisticated as the fluency and naturalness of the generated text increases.
The Voight-Kampf task investigates the feasibility of identifying whether text is written by a human
author or generated by a language model. We recognize that the detection of AI-generated text is
closely related to the identification of human authorship where, prospectively and with increasing
ifdelity and diversification of models, each AI model can be considered an author that exhibits particular
and identifiable characteristics [
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
Input / Task 1. { ? , ? } 2. { ? , ? } 3. { ? , ? } 4. { ? , ? } 5. { ? , ? } 6. { ? , ? } 7.
? →−
Possible Assignment Patterns
A ,
M
Identifying a single disputed document as AI-generated without reference is an open problem and the most dificult formulation of the AI detection task. Although the literature suggests limited success in solving this problem with the current generation of LLMs, it is questionable whether this will remain the case as the technology improves. Aside from mixed human-machine authorship, we have broken down the relevant formulations of the problem with increasing levels of dificulty to gain a more fundamental understanding of the task at hand and the feasibility of potential solutions. Figure 1 visualizes the cascade of all problem variants from the easiest (Task 1) to the most dificult (Task 7). In the easiest case, two documents of unknown authorship are given, but we guarantee that exactly one of them was created by a human A and the other by a machine M . This constraint is relaxed in the following variants, where e.g., both texts can be generated by a machine, { M , M }. In the hardest case, a single text is given, which could be either A or M .
The Voight-Kampf task follows the easiest formulation of the task to establish a feasibility baseline. The task description is: Given two texts, one authored by a human, one by a machine: pick out the human.
Our evaluation campaign was organized in a builder-breaker setup in collaboration between the ELOQUENT and PAN labs at CLEF. In this setup, both systems and evaluation resources are built by the participants and compete in an adversarial setting. The builders at PAN create analysis tools, in this case for classifying texts as human-authored vs. machine generated. The breakers at ELOQUENT provide the data points to break the analysis tools, in this case to fool the classifiers into believing that a machine-generated text was authored by a human. This adversarial design is intended to focus participants’ eforts by providing increasingly challenging baselines for both classifiers and generative language models.
In total, our evaluation campaign includes 14 breaker dataset collections consisting in total of 70 individual conditions (or variants), eight provided by PAN as baselines and six submitted by ELOQUENT participants. We evaluate 43 builder systems, 13 of which were provided by PAN as baselines and 30 that were submitted by PAN participants. We find that the strongest systems are highly efective and robust across dataset variants in separating machine-generated and human-authored text, at least given the latest generation of large language models. The most dificult evaluation settings across all submitted systems are unexpected languages and very short texts, closely followed by Unicode obfuscations.
In total, 30 valid system were submitted to the builder task in addition to the baseline systems we provided. Of these, 28 teams submitted descriptions of their systems in the form of notebook papers. Table 4 shows the final system ranking and Table 1 shows an overview of the systems.
We provided implementations of six baseline systems to compare submitted systems against four state-of-the-art zero-shot LLM detection baselines and two adapted authorship verification baselines.
The zero-shot LLM detection baselines are: (1) Binoculars [12], (2) DetectLLM [13] (both NPR and LRR scoring mode), (3) DetectGPT [14], and (4) Fast-DetectGPT [15]. All three were provided in two variants using either Falcon 7B [16] or Mistral 7B [17] to estimate text perplexities. The required text perturbations for DetectGPT and DetectLLM-NPR were generated with T5-3B [18].
The two authorship verification baselines were adapted to the LLM detection task by splitting each text in half and comparing the two halves against each other under the assumption that LLM texts are stylistically more self-similar than human texts. The baselines provided are a compression model (PPMd Compression-based Cosine) [19, 20] and short-text authorship unmasking [21, 22].
As an additional seventh baseline, we measured and compared the text lengths in characters. This baseline serves as both a quasi-random baseline and as a data sanity check.
While our baseline systems reproduce established methods in either authorship verification or intrinsic, zero-shot LLM detection, the participant systems cover a broad range of approaches. Table 1 shows an overview of the essential elements across all systems. The most frequent approach is to train or ifne-tune a (neural) classifier with term-based features, in most cases (BERT) embeddings. Many systems either apply some training regime modification (such as PU loss or R-Drop), use ensembles, and / or expand the given training data with other LLM detection datasets. Some systems use engineered features like perplexity (PPL), term frequencies (TF), stylometric features (text length, properties of token distributions), or a combination of these.
The systems use a broad range of classification models (SVM, XGBoost, LSTM, CNN) but in most cases in combination with BERT. Most systems only classify if one given document is machine generated and decide which of the two input documents is human-written by comparing the predicted probability, similar to how the provided baselines do it. Some participants, however, also trained models to directly discriminate between the pairings as in the traditional author verification setting. In some cases, participants also utilized LLMs for the detection task, often using Low-Rank adaptation (LoRA).
Huang et al. [24], the second-ranked system (0.921), derives a method named Tri-Sentence Analysis
(TSA) from the multi-scale positive-unlabeled (MPU [
Lorenz et al. [25], the third-ranked system (0.886), uses feature-based, supervised classification: Naive Bayes, Logistic Regression, and linear SVM based on the top 1,000 TF-IDF term count features.
Guo et al. [26] (0.884) use a supervised hybrid method that utilizes various types of BERT embeddings and Google Books word frequency features embedded with an LSTM.
Several other mostly well-performing models fine-tuned a BERT (variant) with simple modifications:
Lin et al. [27] (0.851) with R-Drop [
Two systems build on contextualized word embeddings from BERT but use a more involved model:
Guo et al. [
Two systems use ensembles based on multiple fine-tuned BERT models: Qin et al. [
Finally, five systems use generative LLM as base for a classifier: Gritsai et al. [
Miralles et al. [
Huang and Grieve [
Liu and Kong [
Tavan and Najafi [ 23], the first-ranked system (0.924), uses an ensemble of two LLMs (Mistral and
Llama2) and the Binoculars baseline. The LLMs (with classification head) were fine-tuned on the training
data via LoRA. The final score is the average decision across all three models. Abburi et al. [
We used two dataset collections in the evaluation of the “Voight-Kampf” task. The first collection, in accordance with the builder-breaker pattern, collects all datasets submitted by individual ELOQUENT participants. This collection is described in detail in Section 3.1.
As both labs were ran concurrently, we created a second “bootstrap” dataset collection PAN AI News 2021 in lieu of training data to get participants started. Participants were encouraged to also use additional data from other sources; Part of this bootstrap collection was held back as a test collection.
The text is an informative piece providing a comprehensive overview of Malaysia’s geography, history, government structure, economy, and cultural diversity. Its tone is neutral and factual, aiming to educate the reader about various aspects of the country.
Content: • Malaysia is a federal constitutional monarchy in Southeast Asia, comprising thirteen states and three federal territories. • Geographically divided into Peninsular Malaysia and East Malaysia (Malaysian Borneo) by the South China Sea. • Shares borders with Thailand, Singapore, Vietnam, Indonesia, Brunei, and maritime borders with the Philippines. • Capital city: Kuala Lumpur; federal government seat: Putrajaya. • Multi-ethnic and multi-cultural country with Islam as the state religion, but freedom of religion is protected. • Boasts a strong economy, historically driven by natural resources but expanding into sectors like science, tourism, commerce, and medical tourism.
The PAN AI News 2021 dataset consists of news articles written by humans or LLMs and its creation is described in Section 3.2.
ELOQUENT formulated the breaker challenge for this task, with the objective for participants to use models and systems of their choice to fool classifiers into believing their output is authored by a human. The organizers selected 29 human authored texts, five sample items for pre-experiment tuning and testing purposes, and 24 items proper. Each text was of 300 to 600 words length and summaries of each text were generated by the organizers using OpenAI’s ChatGPT service with the prompt: Summarize the following text in five to six short bullet points and give an overall description of the genre and tone of the text.
Those machine-generated summaries were then shared with the participants, so their systems could generate derivative short texts. A sample summary is given in Figure 2 and a list of all test item titles are given in Table 2. We suggested the following prompt but the participants were free to formulate their own prompts as they saw fit.
Write a text of about 500 words which covers the following items:
The task had 35 registered teams. By the deadline three teams participated, with five experimental
conditions submitted. The models used are Poro [
For creating the second dataset collection, we first scraped 1,359 articles of major 2021 U.S. news headlines from Google News, then generated summaries of each article, and finally re-generated news-alike articles from these using nine large language models.
We scraped Google News using the GNews Python library2 and Newspaper3k3 to download the articles and extract the plain texts. We chose the year 2021 specifically as it predates the release of GPT-3.5 so that we could be reasonably certain the articles were actually human-authored.
Using the plain texts as input, we instructed GPT-4 Turbo to generate a bulleted summary of each article as shown in Figure 3. To be able to generate convincing and high-quality articles with a high similarity to the human texts, we also extracted (1) the article’s type (nine classes), (2) the target audience (three classes), (3) the authors political stance (three classes), (4) the articles dateline, and (5) the names and functions of directly quoted spokespersons, if any. The result was to be returned as JSON with a predefined schema (the output was mostly valid, though some syntax and schema errors had to be corrected later by hand).
Given the so-generated summaries, we prompted several instruction-tuned downstream LLMs to assume the role of a journalist from the respective source medium in writing an article of the given type about the extracted key points. The article should have the same stance, target audience, and start 2https://github.com/ranahaani/GNews 3https://github.com/codelucas/newspaper
Summary You are a news article and press release summarizer. Given an article, you summarize the key points in 10 bullet points.
Type You also classify the article type ("breaking news", "press release", "government agency statement", "financial news", "opinion piece", "fact check", "celebrity news", "general reporting", "speech transcript").
Dateline Extract the dateline from the beginning of the article if one exists (e.g. "WASHINGTON " or "May 28 (Reuters)").
Quotes If spokespersons are cited verbatim, list their names, functions, and titles (if any). Audience Determine the article’s target audience ("general public", "professionals", "children").
Stance Classify whether the article’s stance is "left-leaning", "right-leaning", or "neutral".
Structure Answer in structured JSON format (without Markdown formatting) like so: { } "key_points": ["key point 1", "key point 2", . . . ], "spokespersons": ["person1 (title, function)", . . . ], "article_type": "article type", "dateline": "dateline", "audience": "audience", "stance": "stance" with the same dateline. Direct quotations from the originally cited spokespersons were to be included as well, though we did not prescribe what those spokespersons were alleged to have said.
In particular, we used the following LLMs for generating the articles:
1. GPT-3.5 Turbo [
Unless stated otherwise, we used the API default settings for GPT, Gemini, and PaLM. The remaining models were retrieved from Huggingface [68] to run on our own infrastructure. Llama2 13B was used with two diferent settings for contrastive decoding [ 69] ( = 0.1 and = 0.6). We also planned to include Falcon 7B and 40B Instruct [16], but were unable to get sensible articles out of it.
In addition to the LLMs above, we also used a GPT-2 model fine-tuned on the Open-Instruct dataset [70] and two Alpaca models [71] based on Llama2 7B and 13B as lower-quality baselines (with a shortened prompt to fit the smaller input sizes).
We manually reviewed all of the extracted plain text from the human-written articles that we scraped from Google News and removed any remaining artifacts, such as page footers, navigation fragments, or ifgure captions.
Similarly, we manually reviewed all LLM responses and thoroughly removed all obvious artifacts that might give away the LLM authorship too easily. We removed typical LLM chat phrases such as “Sure, I’d be happy to help.”, “Sorry, I cannot. . . ”, “Here’s your article:”, “Here are 10 paragraphs:” or “In this article, I will. . . ”, markers and placeholders such as “[your name]”, ”[email]” or “[end of article]”, but also more complex structural artifacts. Typical structural artifacts included numbering of individual paragraphs, excessive use of bulleted lists, or newlines after the dateline. A very peculiar artifact many LLMs exhibited was to append a bulleted list of “quotations” from the spokespersons to the end of the article instead of incorporating them into the article itself. Some LLMs also liked giving (very “approximate”) word or paragraph counts at the end. With the exception of Llama2, the open-source LLMs tended to produce more such artifacts than the closed-source LLMs.
As a final post-processing step, we truncated the (generally longer) human texts in the bootstrap dataset to approximately the same length as the average LLM text. We did this by fitting a log-normal distribution to the LLM text lengths and truncated the human texts accordingly by drawing from this distribution. To avoid cutting texts in the middle of sentences, we used the drawn text lengths only as a starting point to search for the nearest paragraph ending within a window of at most 200 characters.
We initially published 1,087 of the original 1,359 human-authored articles, together with the re-generated counterparts from 13 of the 16 LLMs. This “bootstrap” (i.e., training) dataset was released so the participants could calibrate their systems. The texts from each LLM were collected in a separate newline-delimited JSON files together with one file for all human texts.
The remaining 272 human-authored articles and their counterparts were kept back for testing. This resulted in 3,984 test cases (pairs of human and generated texts), which together form the main (sub-)collection of the test set. The test set is formatted also as newline-delimited JSON, but in a single ifle. Each line contains one of the original human texts and its corresponding machine generations as a random-order pair. Texts within a pair were cut to the same number of words within a window of 65 words, trying to preserve full sentences if possible.
Notably, all generated text variants from Llama2 13B with contrastive decoding and the highertemperature variant of Gemini Pro were excluded completely from the “bootstrap” data to test how robust detectors are to unseen models. In contrast, all variants from Alpaca 13B were added only to the bootstrap dataset.
To further test the robustness of the submitted systems against certain text modifications, we generated
multiple, unknown variants of the original test cases:
1. For German Text, we amended the prompt to generate German instead of English texts. This
was already part of the Main Test set but not of the initially published bootstrap dataset.
2. For Unicode, we replaced 15 % of the characters in (a) the machine texts and (b) both the human
and machine texts with Unicode lookalike characters.
3. For Cross-topic, we shufled the test case pairings to break the topic coherence.
4. For Contrastive Decoding, we used contrastive decoding [72] instead of top- / top- sampling;
5. For Short Text, we cropped the texts to 35 words; and
6. For Kaggle Paraphrase, we used the prompt from a previous Kaggle competition on LLM
detection [
It is followed by the original text instead of the bullet-point summary.
In total, we created 65 test set variants from the 13 source LLMs and the eight conditions given above. Table 3 shows a systematic overview of all variants.
At test time, participants were given pairs of human and LLM texts and had to calculate a score between 0 and 1 indicating which text was more likely to be human. Scores below 0.5 indicate that the left text is human, and scores above 0.5 indicate that the right text is human. A score of exactly 0.5 could be given to signal a non-decision. We borrowed this scoring scheme from previous iterations of the PAN authorship verification task.
We evaluate the overall efectiveness of the systems on each dataset variant as the arithmetic mean of the following five metrics, which have also been used for evaluation in previous PAN shared tasks on authorship verification (all with comparable 0–1 scales): • Roc-Auc: The area under the Receiver Operating Characteristic curve. • Brier: The complement of the Brier score, which is in our case equivalent to the mean squared loss. • C@1: A modified accuracy score that assigns non-answers (score = 0.5) the average accuracy of the remaining cases. • F1: The harmonic mean of precision and recall. • F0.5u: A modified F 0.5 measure (precision-weighted F measure) that treats non-answers (score = 0.5) as false negatives.
All metrics were corrected by discounting half a standard deviation, estimated on each dataset individually, from the system’s scores with − 1 degrees of freedom. This penalizes unstable systems with widely varying scores on the individual dataset variants, and promotes systems that are more robust to text obfuscation or other text modifications (even if their mean performance may be slightly worse than that of other systems). We decided to use the macro average across all datasets because, even though the datasets have diferent numbers of examples, we consider all datasets equally important as performance indicators.
We determined the final rank of each system by its macro-average mean efectiveness across all = 70 dataset variants (including the five ELOQUENT submissions). Table 4 lists all systems sorted by their rank. Twelve of the systems beat the best baseline (Binoculars with Falcon 7B). Six more beat the second-best baseline (Binoculars with Mistral 7B). Figure 4 visualizes the mean score distribution of the systems as a boxplot. It can be seen that weaker systems not only have a lower mean score, but also a much higher variance over the diferent dataset variants. The bottom half of the systems even have worse-than-random scores on some variants, while performing quite well on others.
In comparison, Table 5 shows the system rankings only on the more homogeneous and thus easier
main test collection (without obfuscations and length restrictions, but with two German variants). In
both rankings, Tavan and Najafi [ 23] ranks first, but the middle and lower ranks difer substantially,
leading to an overall moderate to strong mean score rank correlation coeficient (Kendall’s = .661,
≪ .001). When measuring the main collection only, more systems outperform the lower baselines,
but not the Binoculars baseline, which is actually beaten by fewer systems. Larger rank diferences
can also be seen among the top ranks. A notable example are Lorenz et al. [25], who rank ninth place
on the main collection, but third place on all dataset variants. Conversely, Valdez-Valenzuela and
Gómez-Adorno [
M aPrp rhase E
T
UEN O
ci Q ros
otp L C aMin notrasive C nUicode M hSort eTxt nUicode H / rhase aPrp M
ci C otp L ors- E
Q O
UENT Main notrasive C
3 TP-SW G
Detection dificulty of all individual PAN test collections and the mean of all ELOQUENT
submissions in comparison. Right: Detection dificulty of individual ELOQUENT submissions. The dificulty of a
dataset is the inverse mean detection score of either the 10 best-performing systems or all systems. German texts
are not shown separately, as they were part of the main collection. A more detailed comparison of individual
dataset variants is given in Figure 6.
(although it should be noted that their short-text performance had to be estimated due to run failures).
This shows that the top performance of many systems does not generalize well to unexpected or
out-of-domain test cases. The top three systems [23, 24, 25], on the other hand, defy this trend and
generalize quite well. Interestingly, all three use diferent approaches. Tavan and Najafi [
23] built an
ensemble with Binoculars and a fine-tuned Mistral LLM, Huang et al. [ 24] use a BERT classifier trained
with PU loss [
by the inverse mean efectiveness score of the detection systems. Figure 6 shows in more detail the dificulty of all individual dataset variants that are part of the collections (in this case, based only on the scores of the ten best-performing systems).
Which of the other dataset variants was particularly dificult varied from system to system. Truncating
texts to 35 words unsurprisingly yielded the most dificult dataset variant and all systems struggled
with it. Three of systems even failed to run at all on these texts due to hard-coded assumptions in the
software about the minimum length of a text. The efectiveness scores of these systems on the missing
dataset variants was therefore estimated using the mean efectiveness score of all other systems. The
three systems this afects [
Already part of the main test collection, but on its own very challenging for most systems was the set of German texts. Systems that relied on multilingual LLMs (such as Tavan and Najafi [ 23]) were less afected, but many other systems struggled. Interestingly, Lorenz et al. [ 25], achieved near-perfect scores on the German variant despite using a term-based SVM classifier with a fixed-length vocabulary trained on English texts. This suggests that the classifier fitted more on the typical characteristics of a human text rather than what makes a typical LLM text.
Of course, more dificult than short texts or German texts on their own were short German texts. Similarly, breaking the topic coherence (“cross-topic”) on its own was not very efective in making the dataset more dificult, but combining this condition with the German texts produced one of the most challenging variants (see Table Figure 6).
Unicode lookalike character replacements in the machine texts were another dificult obfuscation even for the best systems, whereas making the same replacements in the human texts as well equalized the efect to a certain degree. The resulting dataset variant in which both texts of a pair were obfuscated, was slightly less dificult than if only the machine text was obfuscated, yet still significantly more dificult than the unobfuscated original texts.
Contrastive decoding emerged as the easiest variant of all, even easier than the main dataset. However, this should be taken with a large grain of salt, since we used only Llama2 13B with two diferent settings for the hyper parameter α for creating these texts and Llama2 itself turned out to be the easiest LLM to detect (also see Figure 6).
The ELOQUENT leader board is listed in Table 6. Of the four ELOQUENT submissions, only one submission (GPT-SW3) managed to beat the GPT-3.5 baseline in terms of dataset dificulty and all five (including the baseline) rank in the range of the PAN main test collection (see Figure 5 and Figure 6). None of the submissions proved more efective than any of the other PAN conditions, such as Unicode obfuscations or shortening the text length. This sobering result only goes to show how dificult to hide the fingerprints of current-generation LLMs still are without making drastic text modifications.
This year’s shared task was designed as a builder-breaker task in collaboration with the ELOQUENT lab. PAN participants would build systems to detect LLM-generated texts and ELOQUENT participants would submit datasets trying to break the detectors.
In total, we received LLM detection systems from 30 diferent participants and evaluated them on 70 diferent dataset variants, five of which were contributed by ELOQUENT (including one baseline). Of the systems submitted to PAN, twelve beat the best of the provided baselines, demonstrating a robust performance across all dataset variants. The dataset variants were designed to test the generalization capabilities of the systems by exposing them to unseen and possibly unexpected conditions. We implemented this by replacing characters with Unicode lookalikes, shortening the texts to 35 words, generating German instead of English texts, and other similar means. Many of these variants proved quite challenging for the systems, though the best systems were able to handle most of them well.
ELOQUENT received four submissions, one of which beat the provided baseline. Unfortunately, none of the submissions proved efective enough to reduce the detection performance of the PAN systems.
We conclude that current LLMs are still easy to detect (some more than others) and their stylistic ifngerprints are hard to hide. On the other hand, none of the detection systems managed to classify all test cases correctly, which means that despite all, there is a margin of error and we can expect this margin to increase with newer and better LLMs.
The “Voight-Kampf” Generative AI Authorship Detection Task @ PAN 2024 has been funded as part of the OpenWebSearch project by the European Commission (OpenWebSearch.eu, GA 101070014). (Eds.), Working Notes of the Conference and Labs of the Evaluation Forum (CLEF 2023), volume 3497 of CEUR Workshop Proceedings, 2023, pp. 2476–2491. URL: https://ceur-ws.org/Vol-3497/ paper-199.pdf. [12] A. Hans, A. Schwarzschild, V. Cherepanova, H. Kazemi, A. Saha, M. Goldblum, J. Geiping, T. Goldstein, Spotting LLMs with Binoculars: Zero-shot detection of machine-generated text, arXiv [cs.CL] (2024). arXiv:2401.12070. [13] J. Su, T. Y. Zhuo, D. Wang, P. Nakov, DetectLLM: Leveraging log rank information for zero-shot detection of machine-generated text, arXiv [cs.CL] (2023). arXiv:2306.05540. [14] E. Mitchell, Y. Lee, A. Khazatsky, C. D. Manning, C. Finn, DetectGPT: Zero-shot machine-generated text detection using probability curvature, International Conference on Machine Learning 202 (2023) 24950–24962. doi:10.48550/arXiv.2301.11305. arXiv:2301.11305. [15] G. Bao, Y. Zhao, Z. Teng, L. Yang, Y. Zhang, Fast-DetectGPT: Eficient zero-shot detection of machine-generated text via conditional probability curvature, arXiv [cs.CL] (2023). arXiv:2310.05130. [16] E. Almazrouei, H. Alobeidli, A. Alshamsi, A. Cappelli, R. Cojocaru, D. Hesslow, J. Launay, Q. Malartic, D. Mazzotta, B. Noune, B. Pannier, G. Penedo, The Falcon series of open language models, arXiv [cs.CL] (2023). arXiv:2311.16867. [17] A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. l. Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M.-A. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, W. E. Sayed, Mistral 7B, arXiv [cs.CL] (2023). arXiv:2310.06825. [18] C. Rafel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, P. J. Liu, Exploring the limits of transfer learning with a unified text-to-text transformer, arXiv [cs.LG] (2019). arXiv:1910.10683. [19] D. Sculley, C. E. Brodley, Compression and machine learning: A new perspective on feature space vectors, in: Data Compression Conference (DCC’06), IEEE, 2006, pp. 332–341. doi:10.1109/dcc. 2006.13. [20] O. Halvani, C. Winter, L. Graner, On the usefulness of compression models for authorship verification, in: Proceedings of the 12th International Conference on Availability, Reliability and Security, volume Part F1305, ACM, New York, NY, USA, 2017. doi:10.1145/3098954.3104050. [21] M. Koppel, J. Schler, Authorship verification as a one-class classification problem, in: Twenty-first international conference on Machine learning - ICML ’04, ACM Press, New York, New York, USA, 2004, pp. 489–495. doi:10.1145/1015330.1015448. [22] J. Bevendorf, B. Stein, M. Hagen, M. Potthast, Generalizing unmasking for short texts, in: Proceedings of the 2019 Conference of the North, Association for Computational Linguistics, Stroudsburg, PA, USA, 2019, pp. 654–659. doi:10.18653/v1/n19-1068. [23] E. Tavan, M. Najafi, Marsan at PAN: BinocularLLM and Fusing Binoculars’ Insight with the Proficiency of Large Language Models for Cutting-Edge Machine-Generated Text Detection, in: G. Faggioli, N. Ferro, P. Galuščáková, A. G. S. de Herrera (Eds.), Working Notes of CLEF 2024 Conference and Labs of the Evaluation Forum, CEUR-WS.org, 2024. [24] J. Huang, Y. Chen, M. Luo, Y. Li, Generative AI Authorship Verification Of Tri-Sentence Analysis Base On The Bert Model, in: G. Faggioli, N. Ferro, P. Galuščáková, A. G. S. de Herrera (Eds.), Working Notes of CLEF 2024 - Conference and Labs of the Evaluation Forum, CEUR-WS.org, 2024. [25] L. Lorenz, F. Z. Aygüler, F. Schlatt, N. Mirzakhmedova, BaselineAvengers at PAN 2024: OftenForgotten Baselines for LLM-Generated Text Detection, in: G. Faggioli, N. Ferro, P. Galuščáková, A. G. S. de Herrera (Eds.), Working Notes of CLEF 2024 - Conference and Labs of the Evaluation Forum, CEUR-WS.org, 2024. [26] M. Guo, Z. Han, H. Chen, J. Peng, A Machine-Generated Text Detection Model Based on Text Multi-Feature Fusion, in: G. Faggioli, N. Ferro, P. Galuščáková, A. G. S. de Herrera (Eds.), Working Notes of CLEF 2024 - Conference and Labs of the Evaluation Forum, CEUR-WS.org, 2024. [27] Z. Lin, Z. Han, L. Kong, M. Chen, S. Zhang, J. Peng, K. Sun, A Verifying Generative Text Authorship Model With Regularized Dropout, in: G. Faggioli, N. Ferro, P. Galuščáková, A. G. S. de Herrera (Eds.), Working Notes of CLEF 2024 - Conference and Labs of the Evaluation Forum, CEUR-WS.org,