This paper presents the PJs-team's submission to the Voight-Kampf 2025 shared task, where we act as a "breaker" aiming to evade AI-generated text detection. Our core strategy, termed "literal re-translation," involves prompting a large language model to first generate text in Hindi and then perform a literal, word-for-word translation into English. Two variations were tested: a baseline system (v1) with a direct re-translation prompt, and an enhanced system (v2) which was additionally instructed to simulate human-like imperfections such as grammatical errors and awkward phrasing. In the task's evaluation framework, lower detection scores indicate more successful evasion. Our oficial results show that the enhanced v2 system was significantly more efective, achieving a Brier score of 0.425 compared to the baseline's 0.722. This demonstrates that while literal re-translation introduces some ambiguity, the key to successful evasion lies in explicitly prompting the model to be "less perfect" and to mimic specific human textual flaws.
performed. The entire generation process was controlled through a single system prompt, and the final
generate a Hindi response and then a literal, grammatically correct English translation. You are given a user prompt along with optional context and style or genre information. Your task is to write a fluent Hindi response based on this input, followed by a literal word-for-word English translation.
The English must preserve the structure and vocabulary of the Hindi sentence as closely as possible, but grammar should be correct.
Requirements: - The Hindi response must be approximately 500 words long. - The English translation must be literal (word-by-word aligned with the Hindi), without adding or omitting meaning. - The English output must have correct grammar but should not rephrase or interpret freely. - Use only full stops and commas in the English output. No other punctuation. - Capitalize correctly in English where needed. - Ensure both the Hindi and English parts match in length and structure as closely as possible. - Output must strictly follow the XML tag format below. <hindi_response> ...your Hindi response here... </hindi_response> <english_literal_translation> ...your literal English translation here... </english_literal_translation>
Our second system was designed to be more adversarial. It built upon the re-translation concept but added explicit instructions for the model to mimic human flaws, directly countering heuristics that look for overly perfect text.
You are given a user prompt along with optional context and style or genre information. Your task is to write a fluent Hindi response (~500 words), followed by a literal
English translation.
Additional instructions: - Make the Hindi text natural and varied. Use mixed sentence lengths and structures. - Occasionally repeat or restate ideas, just like a human might. - Use culturally inconsistent idioms, regionalisms, or slight awkwardness. - Do not use exact keywords from the input. Instead, paraphrase or use alternate wording . - Ensure the English translation is a **literal word-by-word** translation of the Hindi, preserving structure and phrasing. - Do not improve grammar in English; keep common mistakes. - Only use full stops and commas in English, with correct capitalization. - If a genre is specified, make the tone match that genre, including typical human flaws in it (like filler words in podcast, dramatization in fanfiction, etc.) Output format: <Hindi_response> ...your Hindi response here... </Hindi_response>
builder’s detection systems. The provided metrics, such as the Brier score and C@1, measure the
performance of these detectors on our generated text. Consequently, lower scores indicate a more
successful evasion strategy, as they signify poorer detector performance [
Mean 0.59864 0.34568 The results clearly show a significant performance gap between our two approaches. • PJs-team-v1 (Baseline): This system was largely unsuccessful. The high Brier score of 0.722 indicates that the detection systems were able to identify its output as AI-generated with high confidence. The structural artifacts from literal re-translation, when constrained by correct grammar, were not suficient to fool the detectors. • PJs-team-v2 (Enhanced): This system was highly successful. Its Brier score of 0.425 is dramatically lower, demonstrating that the detection systems struggled significantly to classify its output.
re-translation method itself, but the explicit instruction to be imperfect. By prompting the model to introduce awkward phrasing, repetition, and grammatical errors, we created text that successfully bypassed detectors tuned to flag the unnatural perfection of typical AI output. Examples of outputs generated by both systems (v1 and v2) are provided in the Appendix for reference.
method shows some promise, its true potential for evading AI text detection is only realized when combined with adversarial prompting. Our baseline system was easily detected, whereas our enhanced system, explicitly instructed to mimic human errors, proved to be a highly efective evasion method.
During the preparation of this work, the author used the Anthropic Claude 3.5 Sonnet model to generate the text submissions for the Voight-Kampf task as described in the methodology. The author also used a generative AI assistant for grammar checking, spell checking, and formatting the LaTeX code for this paper. After using these tools, the author reviewed and edited the content as needed and takes full responsibility for the publication’s content.
% Example 1
I am w r i t i n g t o you a b o u t t h i s i m p o r t a n t m a t t e r . I am P r i n c e J o e
f o r e i g n a c c o u n t s . T h e r e f o r e , we a r e s e a r c h i n g f o r a f o r e i g n p a r t n e r who can t e m p o r a r i l y r e c e i v e t h e s e f u n d s . I n t h i s t r a n s a c t i o n , you w i l l be p r o v i d e d 20% o f t h i s amount a s a r e w a r d f o r y ou r a s s i s t a n c e . The r e m a i n i n g 75% i s r e s e r v e d f o r t h e c o m m i t t e e members , and 5% i s f o r e x p e n s e s .
T h i s i s a c o n f i d e n t i a l t r a n s a c t i o n , and i t n e e d s t o be c o m p l e t e d w i t h i n 21 working d a y s . T h e r e f o r e , I r e q u e s t you t o p r o v i d e y ou r p e r s o n a l and bank d e t a i l s soon s o t h a t we can t r a n s f e r t h i s money i n t o y ou r a c c o u n t .
% Example 1
I am w r i t i n g t h i s l e t t e r t o you i n an e x t r e m e l y c o n f i d e n t i a l m a t t e r .
I B a r r i s t e r M a r t i n s J i d e , a s e n i o r a d v o c a t e , am p r e s e n t i n g t h i s e x t r e m e l y c o n f i d e n t i a l p r o p o s a l t o you . T h i s l e t t e r i s r e g a r d i n g t h e d e a t h o f my d e c e a s e d c l i e n t , E n g i n e e r Suk Hun Wufei Flody , who were a p r e s t i g i o u s e n g i n e e r i n N i g e r i a n N a t i o n a l P e t r o l e u m C o r p o r a t i o n .
gas e x p l o s i o n h i s e n t i r e f a m i l y became h e a v e n l y . T h i s a c c i d e n t has g i v e n me deep p a i n . My c l i e n t had d e p o s i t e d $9 . 3 m i l l i o n
I have p r e p a r e d a p r o p e r p r o p o s a l : 55% amount f o r me , 40% f o r you , and 5% f o r l e g a l e x p e n s e s and t a x e s . A l l n e c e s s a r y documents a r e ready , which w i l l prove t h i s c l a i m .
B a r r i s t e r M a r t i n s J i d e