=Paper=
{{Paper
|id=Vol-3180/paper-131
|storemode=property
|title=Poetic or Humorous Text Generation: Jam event at PFIA2022
|pdfUrl=https://ceur-ws.org/Vol-3180/paper-131.pdf
|volume=Vol-3180
|authors=Anne-Gwenn Bosser,Liana Ermakova,Florence Dupin de Saint-Cyr,Pierre De Loor,Victor Charpennay,Nicolas Pépin-Hermann,Benoit Alcaraz,Jean-Victor Autran,Alexandre Devillers,Juliette Grosset,Aymeric Hénard,Florian Marchal-Bornert
|dblpUrl=https://dblp.org/rec/conf/clef/BosserESLCPAADG22
}}
==Poetic or Humorous Text Generation: Jam event at PFIA2022==
https://ceur-ws.org/Vol-3180/paper-131.pdf
Poetic or Humorous Text Generation: Jam Event at
PFIA2022
Anne-Gwenn Bosser1,2 , Liana Ermakova3,4 , Florence Dupin de Saint-Cyr5 , Pierre De
Loor1,2 , Victor Charpennay6 , Nicolas Pepin-Hermann, Benoît Alcaraz,
Jean-Victor Autran8 , Alexandre Devillers, Juliette Grosset, Aymeric Hénard10,2 and
Florian Marchal-Bornert11
1
ENIB, Brest
2
Lab-STIC CNRS UMR6285
3
Université de Bretagne Occidentale, HCTI, Brest, France
4
Maison des sciences de l’homme en Bretagne, Rennes, France
5
IRIT CNRS UMR5505
6
EMSE
8
IRL CROSSING, ArianeGroup
9
ECAM Rennes
10
Université de Bretagne Occidentale, Brest, France
11
LORIA
Abstract
In this paper, we describe the organization and the results of a Jam event on computational creativity at
a major French national conference in AI PFIA2022. During this Jam event, participants were challenged
by the task of humor or poetry generation using a number of resources and corpus.
Keywords
Computational creativity, Computational Humour, Humour generation, Wordplay, Artificial Intelligence,
Scientific mediation, Large pre-trained models, Few-shot learning
1. Introduction
The first edition of the Poetic and/or Humorous Text-generation Jam organized by the French
Association for Artificial Intelligence (AFIA1 ) took place in Saint-Étienne, France, from June 27
to July 1, 2022 during the platform of AI conferences PFIA2022 2 . This annual French-speaking
event in AI gathers nine conferences, five workshops, five thematic days for working groups,
and ten tutorials (450 registered participants in total).
Inspired by the Game Jam subculture, at this Generation Jam, we proposed to participants
to play together to generate texts by experimenting with tools related to the practice of AI. A
jam is a playful event promoting creativity. It is an opportunity for experimentation, where
skill-sharing and learning are encouraged.
CLEF 2022: Conference and Labs of the Evaluation Forum, September 5–8, 2022, Bologna, Italy
© 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings CEUR Workshop Proceedings (CEUR-WS.org)
http://ceur-ws.org
ISSN 1613-0073
1
https://afia.asso.fr/
2
https://ci.mines-stetienne.fr/pfia2022/
In this paper, we describe the organization of this event, which made extensive use of the
resources provided by the CLEF workshop JOKER [1] as well as other resources. We also discuss
the outputs and the feedback on the participants’ experience. We believe that such resources are
a key for easily reproducible scientific mediation events, providing a relatively uncontroversial
context for discussing the capabilities and limitations of AI with a general audience.
2. Event preparation
Before the event, a sample of resources including research papers with code were gathered by
the organizers to provide inspiration with early or recent work.
• A system by He, Peng, and Liang [2] for generating puns using a language model and
semantic distance to create a humorous effect Pun Generation with Surprise 3 ;
• A system by Weller, Fulda, and Seppi [3] for creating humorous headlines from non-
humorous headlines: Humorous Headline Generation via Style Transfer (a.k.a. Humor
Translation) 4 ;
• Tim Van de Cruys’ poetry generation system [4], Automatic Poetry Generation from Prosaic
Text 5 ;
• How to generate puns and make people laugh with taboo, from a corpus of SMS, by
Valitutti, Toivonen, Doucet, and Toivanen [5]."Let Everything Turn Well in Your Wife":
Generation of Adult Humor Using Lexical Constraints 6 ;
• The Oulipo constraints 7 ;
• The page of Darius Kazemi 8 , artist in procedural generation;
• La Machine à Ecrire 9 from Jean Baudot (1964).
We also provided tools and data suggestions:
• Lexicon 3 10 , and the openlexicon project (Boris New and Christophe Pallier). This very
rich lexicon for the French language allows for example to obtain the phonetic form, the
frequency of use, and many other morphological and linguistic information to process
words. As many NLP libraries for the French language are much more limited when
compared to their English language counterparts, this lexicon proved to be an essential
building block for most of the participants’ creation. Besides, we provided a custom-made
lexicon of paraphones for French nouns and adjectives. We considered as candidate
paraphones words whose phonetic transcription were Levenshtein-distant by less than 2.
3
https://github.com/hhexiy/pungen
4
https://github.com/orionw/humorTranslate
5
https://github.com/timvdc/poetry
6
https://aclanthology.org/P13-2044/
7
https://www.oulipo.net/fr/contraintes
8
https://tinysubversions.com/
9
https://archive.org/details/xfoml0001/page/n13/mode/2up
10
http://www.lexique.org/
• Semantic networks for helping to compute the semantic distance of words: for example
ConceptNet 11 which uses word embeddings 12 , or Fasttext [6]13 .
• the corpus of puns collected during the JOKER workshop at CLEF 2022 14 [7, 1].
• A GPT-3 playground 15 : after registration, many prompt tools are available to experiment
with this powerful generator.
These resources were previously tested for reproducibility, and tried by several students
interning for small-scaled projects. These projects allowed us to verify that the Jam was correctly
dimensioned for the audience and the timespan of the conference (level of challenge, time to
complete a workable solution). For instance, these 2 projects tried out Large Language Models:
• Interactive poetic generation in French with GPT-2 16
• Generation of wordplay in French with the Jurassic language model and Fastttext 17 .
Specific challenges were proposed by the event coordinators or scientific organizations. These
took the form of themes or constraints on the generated text. They were unveiled during the
launching session:
1. Theme : Haikuweather. Constraints: the texts must be about the weather.
2. Theme : Together. Constraints: the texts must emphasize the human-machine comple-
mentarity in creation.
3. Theme: Guybrush Threepwood. Constraints: the generated texts must be in the style
of a famous video game character. (challenge proposed by the working group AI and
Games of the CNRS Research Group Formal Aspects of AI.
4. Theme: Source code. Constraints : the texts are generated from source code as input to
the system (challenge proposed by the IRILL 1819 )
3. Participation
The jam was intended to occupy the quiet times of the conference and to provide a physical
space for sociability.
Conference participants could join the jam at any time. Registration gave access to a Discord
group where resources were shared. All that was needed was a laptop or even a phone (one
of the participants was able to download word2vec on his smartphone), equipped with the
tools of their choice. The theme of the event was chosen for the range of AI techniques that
could be used, as well as its potential for inclusiveness. Everyone could of course come with
the objective of illustrating the creative power of their AI specialty, armed with their model
11
https://conceptnet.io/
12
https://github.com/commonsense/conceptnet-numberbatch
13
https://fasttext.cc/
14
https://www.joker-project.com/clef-2022/EN/project.html
15
https://beta.openai.com/playground
16
https://git.enib.fr/deloor/poesygeneration/-/tree/pytorch
17
https://gitlab.com/loicgle/computational-humor-pun-generation
18
https://www.irill.org/
19
Initiative for Research and Innovation on Free Software
previously trained for the occasion. But participants could also come with the intention of
simply experimenting with previously unknown techniques or libraries.
After the first full meeting where the challenges were unveiled, daily gathering allowed to
put together the texts generated to discuss the most promising approaches. These sometimes
involved guests specialized in computational creativity. A selection of the texts generated were
selected and displayed daily for the benefit of the conference attendees. The closing session
took place during the last coffee break of the conference and allowed to reflect about the event
and gather ideas for future developments.
4. Jam Progress
The first session of the first day presented the jam principle, challenges and resources available.
It ended with a round table discussion where everyone introduced themselves, and started to
sketch out ideas to develop. Most of the participants had a computer science or theoretical
computer science background, except for a doctoral student in art history member of the
CulturIA project20 .
Several ideas were progressively sketched-out:
• combining two poems with very contrasting styles such as in the mash-up Pride and
Prejudice and Zombies [8]. In the same vein, transforming an existing text using a very
different style (e.g. from classic French Molière vs rap music);
• generating portmanteaux-words by joining two words from Lexique 3 with an important
semantic distance (in order to generate a maximum contrast). This evolved towards the
generation of riddles based on the template "Mr and Mme X have a child, what is its
name?";
• prompting the large language model "Jurassic" with several examples of puns from the
JOKER-2022 Corpus, all following a strong template, to obtain new ones. This later
evolved towards using GPT-3.
• Various applications of Oulipo constraints (eg. Haikuisation)
During one of the sessions, the participants benefited from Alessandro Valitutti’s presentation
of his recent research on computational humor [9, 10, 11] accompanied by a demonstration of
the protocol described in [12].
Although we did not evaluate the results formally, the audience of the conference provided
feedback indicating that generated text improved during the course of the conference. Here are
some of the outputs that were produced and displayed in the conference hall (in French):
Day 1 : Le ballon a rencontré son petit-ami au bal. (obtained with Jurassic prompts)
Day 2 : Portmanteaux word: Narguiléviathan; Pun: Les vieux jardiniers ne meurent jamais, ils
sèment juste leurs dernières années.
Day 3 : "Gouache wesh ! Maintenant chevalet faire un peu de peinture" dit Tom d’un air pincé.
20
CulturIA project examines the cultural history of artificial intelligence through the combined perspectives of
the history of science, the history of ideas, the sociology of science and technology, and fieldwork.
Day 4 : Movie Summary generated from the previoulsy generated portmanteaux word: Film
title =Narguiléviathan, Summary=La petite fille fleur bleue et son narguilé en forme de
leviatan souffrent d’une même maladie : l’insouciance.
Day 5 : Movie summaries gained silliness for real movie titles and made-up ones.
– Antman: Un gars petit comme une fourmi devient gros comme un éléphant et ça le
sauve d’une vie de merde.
– Coco veut un gâteau: Un petit garçon mexicain veut un gâteau pour son anniversaire,
les araignées s’en chargent.
– Une jambe, un œil et une main, la révolte des faux: Une jambe, un œil et une main se
révoltent contre leur propriétaire pour être traités comme des êtres humains.
Genetic algorithms transformed the style of sentences, e.g. from "Les oiseaux ça vole et ça
fait des cercles dans le ciel." to "les oiseaux qui volent, stupides, qui semblent démanteler
craintivement le ciel." A Haiku was also generated using GPT-3 prompted with weather
forecast:
Contre vents et marées
Il y a des gens qui restent,
Pas de risque de pluie !
5. AI methods used by the participants
A number of AI techniques were either used or considered by the participants.
The generation of portmanteaux-words was done by combinatorial combination of words
from a lexicon including information for syllable decomposition and by using pre-trained neural
networks for word embeddings.
For the generation of poems, an approach was developed based on genetic programming,
starting from an input sentence and performing mutations on the sentences.
Pun generation with Large Language Models relied on prompts made up of labeled puns that
were part of the CLEF JOKER-2022 data. Other prompts were handcrafted (such as the title and
spoof summaries of movies).
Finally, although this was not finalized during the jam, we considered using Multi-valued
Decision Diagrams such as in [13] which were presented at the same conference.
One of the insights from this experiment was that the curation of the generated results is very
important. For the portmanteaux-words, it was initially done by humans, but later automated
based on score assignment depending on the semantic distances. We assumed that the semantic
distance is related to the level of incongruity.
Although we did not conduct any experiment to confirm this, we couldn’t help but notice that
participants laughed more when they were more involved (in terms of efforts) in the process of
joke creation. Jokes and poetry generated by GPT-3 with simple prompts, even when successful,
seemed to leave people dubious and suspicious, checking if they were not just already present
on the internet for instance before recognizing the creativity of the latest Large Language
Models (see Figure 1). This is similar to the "IKEA effect" that has been noticed in the context
of music generation [14].
Figure 1: Prompting the model with a code input and an instruction asking for a joke, it generates two
of them.
6. Conclusion
The event attracted about forty participants. Although the involvement of the participants
varied, we consider it a success. The Jam raised several questions that we plan to investigate
further:
• How much would an isolated person enjoy an artificially created joke? In other words,
without the jam collective, would people have laughed as much?
• How to evaluate the creativity of the Large Language Models? The difficulty to appreciate
jokes which are produced with minimum human involvement may come either from an
"Ikea effect" or a lack of trust towards the actual creativity of the AI that we were after.
For more information, you can visit the AFIA web page dedicated to this competition: www.
cecilia-afia.fr/.
Due to the interest of the public and the participants in the jam and its themes, we plan to
organize similar events in the future, including for a general audience.
Acknowledgements
The entire organizing committee would like to warmly thank the local organizing committee of
PFIA’2022 and the volunteer staff. We also thank the participants for investing time and energy,
even when they did not end up sharing a generated text.
A number of people helped to set up this event: Rym Bousrih, Erwan Claquin, Sarah Clauzade–
Robert, Kevin Cornut, Loic Glémarec, Oumaima Oumaadi, and Maxime Yonnet.
References
[1] L. Ermakova, T. Miller, F. Regattin, A.-G. Bosser, E. Mathurin, G. L. Corre, S. Araújo,
J. Boccou, A. Digue, A. Damoy, B. Jeanjean, Overview of JOKER@CLEF 2022: Automatic
Wordplay and Humour Translation workshop, Experimental IR Meets Multilinguality,
Multimodality, and Interaction. Proceedings of the Thirteenth International Conference of
the CLEF Association (CLEF 2022) 13390 (2022).
[2] H. He, N. Peng, P. Liang, Pun Generation with Surprise, in: Proceedings of the 2019
Conference of the North American Chapter of the Association for Computational Lin-
guistics: Human Language Technologies, Volume 1 (Long and Short Papers), Associa-
tion for Computational Linguistics, Minneapolis, Minnesota, 2019, pp. 1734–1744. URL:
https://aclanthology.org/N19-1172. doi:10.18653/v1/N19-1172.
[3] O. Weller, N. Fulda, K. Seppi, Can humor prediction datasets be used for humor generation?
humorous headline generation via style transfer, in: Proceedings of the Second Workshop
on Figurative Language Processing, Association for Computational Linguistics, Online,
2020, pp. 186–191. URL: https://aclanthology.org/2020.figlang-1.25. doi:10.18653/v1/
2020.figlang-1.25.
[4] T. Van de Cruys, Automatic poetry generation from prosaic text, in: Proceedings of the
58th Annual Meeting of the Association for Computational Linguistics, Association for
Computational Linguistics, Online, 2020, pp. 2471–2480. URL: https://aclanthology.org/
2020.acl-main.223. doi:10.18653/v1/2020.acl-main.223.
[5] A. Valitutti, H. Toivonen, A. Doucet, J. M. Toivanen, “Let Everything Turn Well in Your
Wife”: Generation of Adult Humor Using Lexical Constraints, in: Proceedings of the
51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short
Papers), Association for Computational Linguistics, Sofia, Bulgaria, 2013, pp. 243–248.
URL: https://aclanthology.org/P13-2044.
[6] P. Bojanowski, E. Grave, A. Joulin, T. Mikolov, Enriching word vectors with subword
information, Transactions of the Association for Computational Linguistics 5 (2016).
doi:10.1162/tacl_a_00051.
[7] L. Ermakova, T. Miller, O. Puchalski, F. Regattin, E. Mathurin, S. Araújo, A.-G. Bosser,
C. Borg, M. Bokiniec, G. L. Corre, B. Jeanjean, R. Hannachi, G. Mallia, G. Matas, M. Saki,
CLEF Workshop JOKER: Automatic Wordplay and Humour Translation, in: M. Hagen,
S. Verberne, C. Macdonald, C. Seifert, K. Balog, K. Nørvåg, V. Setty (Eds.), Advances
in Information Retrieval, Lecture Notes in Computer Science, Springer International
Publishing, Cham, 2022, pp. 355–363. doi:10.1007/978-3-030-99739-7_45.
[8] J. Austen, S. Grahame-Smith, Pride and Prejudice and Zombies: The Deluxe Heirloom
Edition, Chronicle Books, 2009.
[9] A. Valitutti, Humor facilitation of polarized events, in: International Conference on
Distributed, Ambient, and Pervasive Interactions, Springer, 2018, pp. 337–347.
[10] A. Valitutti, Making fun of failures computationally, in: International Conference on
Distributed, Ambient, and Pervasive Interactions, Springer, 2017, pp. 684–695.
[11] A. Valitutti, N. Novielli, Polarity contrast for the detection of verbal irony., in: NL4AI@
AI* IA, 2017, pp. 51–56.
[12] A. Valitutti, T. Veale, Infusing humor in unexpected events, in: International Conference
on Distributed, Ambient, and Pervasive Interactions, Springer, 2016, pp. 370–379.
[13] A. Bonlarron, A. Calabrèse, P. Kornprobst, J.-C. Régin, Génération de texte sous con-
traintes pour mesurer des performances de lecture: Une nouvelle approche basée sur les
diagrammes de décisions multivalués, in: Journées Francophones de Programmation par
Contraintes (JFPC), 2022.
[14] F. Pachet, P. Roy, B. Carr’e, Assisted music creation with flow machines: towards new
categories of new, in: Handbook of Artificial Intelligence for Music, Springer, 2021, pp.
485–520.