This paper details the participation of the pjmathematician team in all three tasks of the JOKER 2025 track: Humour-aware Information Retrieval, Pun Translation, and Onomastic Wordplay Translation. Our approach uniformly leverages the Qwen family of large language models (LLMs), applying distinct strategies tailored to each task's unique challenges. For Task 1, we implemented a two-stage process involving an LLM-based joke iflter and explainer followed by a dense retriever, achieving a MAP@1000 of 0.3501 for English and 0.4221 for Portuguese on the test set. For Task 2, we fine-tuned various Qwen models to translate puns from English to French, with our best model (Qwen2.5-14B) attaining a BLEU score of 0.379. For Task 3, we explored zero-shot prompting for the complex task of translating onomastic wordplay, where our Qwen3-32B model achieved an exact match accuracy of 0.22. These results demonstrate the versatility of modern LLMs in handling nuanced and creative language tasks, highlighting the efectiveness of filtering for humour-aware retrieval, fine-tuning for pun translation, and the challenges in zero-shot onomastic translation.
The automatic analysis of wordplay and humour presents significant challenges for natural language
processing, requiring a deep understanding of semantics, cultural context, and linguistic creativity [
Our team, pjmathematician, participated in all three tasks:
The work presented in this paper intersects with several key areas of computational linguistics and
information retrieval: computational humor, machine translation of creative language, and the
application of Large Language Models (LLMs) to these nuanced tasks. Our research is situated within the
context of the JOKER shared task at CLEF, which specifically encourages interdisciplinary work on the
automatic analysis of humor and wordplay [
Task 1 extends a line of research focused on retrieving documents that are not only topically relevant
but also possess a specific stylistic quality—in this case, humor. The JOKER track has been instrumental
in creating reusable datasets to foster research in this niche. Previous approaches in humor detection
and retrieval have ranged from methods using classic machine learning with engineered features to
deep learning models. More recently, the advent of LLMs has introduced new possibilities, with research
exploring their use for humor detection and explanation [
The translation of humor, particularly puns, is a classic and formidable challenge in both human and
machine translation. The dificulty lies in the fact that puns often rely on language-specific phonological
or semantic ambiguities that rarely have direct equivalents in a target language [
Task 3, the translation of meaningful names (onomastic wordplay), is a specialized subset of literary
translation [
Underpinning our entire approach is the transformative capability of modern LLMs. While early
research focused on humor detection with smaller models, today’s large-scale models have
demonstrated remarkable, if imperfect, abilities in generation and nuanced understanding [
We utilized various models from the Qwen series, applying a specific methodology for each task. All experiments were conducted using the oficial JOKER 2025 datasets.
Our approach for this task was a two-step pipeline: (1) content analysis and filtering, and (2) dense retrieval.
First, we processed the entire document corpus using Qwen3-14B and Qwen3-32B models to identify potential jokes and generate explanations. This step aimed to create a richer representation of each document by explicitly flagging humorous content. We used the prompt shown in Listing 1. This generated a ‘isJoke‘ flag and a textual ‘explanation‘ for each document, which were used in subsequent steps.
We used Qwen3-4B and Qwen3-8B embedding models as our retrievers [
For this task, we adopted a fine-tuning approach to teach the models the creative art of pun translation. We used LoRA to fine-tune several Qwen models on the provided English-French parallel corpus. The core of our approach was the instructional prompt shown in Listing 2, which guided the model to focus on the ‘pun->pun‘ translation strategy. We experimented with diferent model sizes and LoRA parameters (rank ‘r‘ and alpha ‘a‘).
Given the highly creative and context-dependent nature of onomastic wordplay, we opted for a zero-shot prompting approach with the powerful Qwen3-14B and Qwen3-32B models. The prompt (Listing 3) was carefully designed to be a comprehensive guide, instructing the model on various translation principles, including when to keep names identical, perform direct translations, or pursue creative adaptations based on the provided context (‘description‘).
This section presents the results for each task. Our runs were submitted with the team name pjmathematician.
We evaluated our retrieval configurations on the English training data to identify the best setup. As shown in Table 1, using a ‘jokequery‘ prompt and filtering the corpus with a joke detector consistently outperformed other methods. The combination of the Qwen3-8B retriever with the Qwen3-14B filter and 32B explanation yielded the highest MAP@1000 score of 0.48.
For our final submission on the test set (Table 2), we selected the best-performing configurations. For English, augmenting documents with explanations generated by a 14B model and filtering with a 32B model performed best, achieving a MAP of 0.3501. For Portuguese, where filtering was not applied, the approach was still efective, achieving a MAP of 0.4221. This indicates that augmenting documents with semantic explanations of humor is a viable strategy even without a pre-filtering step.
English English English Portuguese Portuguese
Qwen3-8B Qwen3-8B Qwen3-8B Qwen3-4B Qwen3-4B
32B filter 14B filter No filter No filter No filter
14B explanation 14B explanation 32B explanation 32B explanation 14B explanation
The results of our fine-tuning experiments are shown in Table 3. The larger Qwen2.5-14B model, ifne-tuned with a LoRA rank of 128, achieved the highest BLEU score of 0.379 on the test set. This demonstrates a clear correlation between model size and performance on this creative translation task. While BLEU is a limited metric for wordplay, these scores indicate that the fine-tuned models learned to generate syntactically valid and semantically relevant translations.
The zero-shot results for onomastic translation are presented in Table 4. This proved to be the most challenging task. The larger Qwen3-32B model achieved a higher exact match accuracy of 0.22 compared to the 14B model. While these scores may seem low, the strictness of the exact match metric for such a creative task must be considered. Many generated translations were semantically plausible and creative, even if they did not perfectly match the reference.
In our participation in the JOKER 2025 track, we systematically evaluated the Qwen family of models on three distinct humour-related tasks. Our findings are threefold. First, for humour-aware information retrieval, a two-stage approach of filtering and explaining content with a powerful LLM before passing it to a dense retriever is highly efective. Second, for creative translation tasks like puns, fine-tuning moderately sized LLMs with targeted instructional prompts yields strong results. Third, the zero-shot translation of highly idiosyncratic onomastic wordplay remains a significant challenge, though larger models show incremental progress.
Future work could explore more advanced RAG (Retrieval-Augmented Generation) frameworks for Task 1, combining retrieval with a generative re-ranking step. For Tasks 2 and 3, exploring few-shot incontext learning or more sophisticated fine-tuning methods could further improve creative generation capabilities.
During the preparation of this work, the author(s) used the Qwen family of models (Qwen3-32B,
Qwen314B, Qwen2.5-14B, Qwen3-8B, Qwen3-4B) in order to: Generate content, Analyze data. Specifically, the
models were used to generate joke explanations for Task 1, produce French translations for Tasks 2
and 3, and to perform the initial humor classification in Task 1. After using these tool(s)/service(s), the
author(s) reviewed and edited the content as needed and take(s) full responsibility for the publication’s
content. The design of the prompts and the experimental framework represents the core contribution
of the author(s).
You are a humor analysis assistant with expertise in jokes and wordplay. When presented
with text, evaluate it according to the following criteria and output your analysis
in JSON format enclosed within <json></json> tags.
## Evaluation Criteria:
1. Determine if the text is a joke (True/False)
2. If it is a joke, identify the type (pun, one-liner, knock-knock, observational, dad
joke, anti-joke, etc.)
3. Provide a 1-2 sentence explanation of the joke mechanics or wordplay
4. Rate the joke on a scale of 1-5 (1=barely amusing, 5=extremely funny)
If the text is not a joke, only provide the explanation of why it’s not considered a
joke.
## Rating Guidelines:
1 = Barely amusing, might elicit a slight acknowledgment
2 = Mildly funny, might cause a small smile
3 = Moderately funny, could provoke a chuckle
4 = Very funny, likely to cause laughter
5 = Extremely funny, potentially causing uncontrolled laughter
Input: "Why don’t scientists trust atoms? Because they make up everything."
"isJoke": True,
"type": "[joke type]",
"explanation": "[1-2 sentence explanation]",
"rating": [
Listing 1: Prompt for Joke Analysis (Task 1) You are an expert bilingual translator specializing in wordplay and humor across English and French. Your task is to translate English puns and jokes into French while preserving the humor mechanism of the original. ## Translation Guidelines: 1. Identify the wordplay mechanism in the English pun (homophony, polysemy, etc.) 2. Translate using Delabastita’s p u n pun strategy - preserve both form and meaning of the original wordplay 3. The French translation must contain an equivalent pun that works in French 4. Prioritize maintaining the same semantic field as the original when possible 5. If a direct translation doesn’t preserve the pun, find French words with similar ambiguity Output directly and only the translated french pun.
Listing 2: Fine-tuning Prompt for Pun Translation (Task 2) You are an expert translator specializing in onomastic wordplay translation from English to French. Your task is to translate names, phrases, and terms containing wordplay while preserving the humor, references, and creative essence of the original. ## Task Description You will receive: - **en**: The English name/phrase containing wordplay - **description**: Context explaining the name, its meaning, and usage You must provide: - Only the French translation in a specific JSON format ## Translation Principles 1. **Understand the Wordplay First** - Identify the pun, allusion, or linguistic creativity in the original - Consider the character/object’s traits, function, or role as described 2. **Translation Approaches (in order of preference):** a) **Keep Identical** when: - The name is Latin-based and works in both languages (e.g., "Vulnera Sanentur," "
Felix Felicis") - The reference is universal or already established b) **Direct Translation** when: - The wordplay relies on descriptive meaning (e.g., "Vipertooth"
") - Compound words where components can be directly translated c) **Creative Adaptation** when: "Dent-de-vip re - Cultural adaptation is needed for the pun to work - Suffix patterns matter (e.g., "-ix" in Asterix characters) - Sound patterns or pronunciation-based humor is involved - The original contains idioms or cultural references d) **Functional Equivalence** for: - Spells (focus on magical effect: "obliviate" "oubliettes") - Acronyms (maintain humor while changing letters: "SPEW" "S.A.L.E") - Location names (prioritize French geographic naming conventions) ## Output Format Your response must follow this exact JSON format: Provide only this JSON object with no additional text, explanations, or commentary.
Listing 3: Zero-shot Prompt for Onomastic Translation (Task 3)