We present the first version of a system for interactive generation of theatre play scripts. The system is based on a vanilla GPT-2 model with several adjustments, targeting specific issues we encountered in practice. We also list other issues we encountered but plan to only solve in a future version of the system. The presented system was used to generate a theatre play script premiered in February 2021.
the occasion of the 100th anniversary of Karel Cˇ apek’s play R.U.R. [Cˇ ap20], in which the word “robot” first appeared.
for individual theatre play scenes. The tool is based on the GPT-2 XL [RWC+19] generative language model, using the model without any fine-tuning, as we found that with a prompt formatted as a part of a theatre play script, the model usually generates continuations that fit the format well. However, we encountered numerous problems when generating the script in this way. We managed to tackle some of the problems with various adjustments, but some of them remain to be solved in a future version.
Our tool was used to generate the script for a new play, AI: Kdyˇz robot p´ıˇse hru (AI: When a robot writes a play ), which was premiered online on 26th February 2021.2 Although there were various forms of human intervention when generating the script, we estimate that over 90% of the text comes from the automated tool; moreover, most of the interventions were similar to those a dramaturge and director would do in case of a humanwritten script (making cuts, rearranging lines, reassigning characters, minor edits of the lines, etc.).3 We present a preliminary analysis of the interventions in Section 2.1. The GPT-2 model was found unfit for generating long and complex texts such as a full play script; we therefore generated several individual scenes and then a dramaturge joined them into a full play.
• Video: https://youtu.be/ksrZouM7Wyg • Sample outputs: http://bit.ly/theaitre-samples • Source codes: http://hdl.handle.net/11234/1-3507 [RDK+21] 2
The process of generating a theatre play scene script starts by the user (a theatre dramaturge in our case) defining the start of the scene, typically a setting and several initial lines of dialogue, which defines the theme of the scene, introduces the characters, and encourages the GPT-2 language model to start generating a dialogue. For the first play, we defined a set of inputs revolving around a common topic to ensure some basic coherence of the whole play.4 The THEaiTRobot tool then uses the vanilla GPT-2 XL model to generate continuing lines, which then get translated from English to Czech by a Machine Translation service. The user has the option to discard any generated line (together with all subsequent lines), prompting the tool to generate a di↵erent continuation. 5
itself is implemented as a web application with a server backend, using the Huggingface Transformers library [WDS+20]. 2.1
Preliminary Analysis of Human Interventions
which we intend to publish once finished. So far, we have completed the analysis of the first scene, Death.
while the remaining 15 lines were slightly modified. We detail the modifications in Table 1; note that in some lines there was more than one intervention. Also, 11 lines from the generated script were deleted.
one of them also became part of the final scenario. In total, 91% of the words in the script of the first scene are used as they were generated, while 9% of the words were added, changed or reordered.
We have only performed the analysis on the English side of the script. On the Czech side of the script, there are additional edits which fix some errors of the automated machine translation, as explained in Section 2.3.4.7 2https://www.svandovodivadlo.cz/inscenace/673/ai-kdyz-robot-pise-hru/3445 3In fact, the dramaturge and the director reported that they made fewer and smaller edits to the script than they typically do with a human-written script.
4The play tells a story of a robot trying to find his place in the human society. Each scene revolves around a typically human theme, such as death, love, sex, or work, and the robot learns about this theme through the interaction with a human character. 5This option was used for approximately 5% of the lines in the script of the first play.
6This option was used very rarely. Apart from the input prompts, only approximately 1% of the lines were hand-written and manually entered into the script.
7On the Czech side, approximately 23% of words were modified, but most of the additional modifications were fixes of incorrect T-V distinction or gender.
Resolved Issues
Set of Characters
have an end!
enjoy my ending!
Master: I‘m afraid of what I‘ve been doing here.
characters too often.8 We resolve this by modifying the next token probability distribution within the GPT-2 model, so that at the start of a new line, only tokens corresponding to character names present in the input prompt are allowed. We also boost probabilities of characters that have not spoken for some time.9 2.2.2
Repetitiveness
2.2.3
Limited Context
input prompt and the generated output must fit. The typical solution is to crop the input at the beginning so that it fits into the window with sucient space for generating the output. However, this means forgetting potentially important information from the input prompt and the previously generated text, which can lead to an unwanted continual topic drift and also to generating contradictory text; the text is still locally consistent, but as a whole it may be inconsistent.
To handle this issue, we introduce automated extractive summarization into the process, hoping that the summarization algorithm will identify the most important pieces of information to remember. Whenever the input for GPT-2 (the input prompt + the so far generated script) exceeds a preset limit of M = 924 tokens,10 8If there are only two characters in the scene, the model often keeps to them or only introduces 1-2 additional characters. If the number of characters is higher than 3, the model usually forgets some of them after some time. Also, some character names push the model in unintended directions. We have seen the model generalize over character names originally involving only “Robot 1” and “Robot 2”, with the model continually introducing “Robot 3”, “Robot 4”, “Robot 5”, etc. We have also observed the model to immediately change a character called “Vladimir” to “Vladimir Putin”.
9Specifically, we multiply each character probability by 2c where c is the number of lines for which the character has not spoken. 10Most script lines in our setting fit within 100 tokens, so ensuring there is space for generating at least 100 tokens means that usually the model will generate a complete line, ending with a newline symbol; in case the generated line is too long, it is simply cut o↵ once the limit of 1024 tokens is depleted. we summarize the input using TextRank11 [MT04] before feeding the input into the GPT-2 model: • We keep all lines within the last R = 250 tokens from the input12 to ensure local consistency. • We summarize all the preceding lines into N = 5 lines (while keeping their original order) to ensure global consistency. • We concatenate the summary and the kept lines.
• If the resulting text is still longer than M tokens, we crop it at the beginning to M tokens. 2.2.4
Machine Translation
translate the generated script using the CUBBITT [PTT+20] neural translation model. As the translation tends to discard character names from the lines, we add them by identifying them in the input and translating them independently. 2.3 2.3.1
Unresolved Issues and Future Plans
Generating a Whole Play
we intend to generate the script hierarchically, first generating a synopsis for the whole play, then expanding it into synopses for individual scenes, and finally generating each scene individually based on its synopsis. This approach is inspired by the work of Fan et al. [FLD18, FLD19], who take a similar coarse-to-fine approach to story generation. Our situation is, however, more complex, as we plan to use one more step of the hierarchy. 2.3.2
Character Personalities The characters in the play do not seem to have independent personalities in the generated script; the model seems to simply ensure consistency with already generated text, not taking the character names into account much. The character personalities thus appear to switch and merge. We intend to resolve this by learning theatre character embeddings and using them to condition the language model. We plan to resolve this by clustering our data into several basic character personality types [AKDM19], then train separate character-aware language models, either by finetuning the GPT-2 model, or by using adapter models [MIL+20, WTD+20]. 2.3.3
Dramatic Situations The text is generated word by word and line by line, whereas human authors of theatre plays typically operate on a more abstract level, such as dramatic situations [Pol21].13 While there is some work on identifying dramatic turning points [PKL19, PKFL20], it is too coarse-grained for our application. We are thus currently annotating a corpus of theatre play scripts with a modified set of dramatic situations, and plan to enhance the tool with this abstraction, either by adding one more layer in the hierarchical setup, or by using special tokens or embeddings to mark dramatic situations in the generated text. 2.3.4
Machine Translation
This leads to various issues, including errors in morphological gender (which should pertain to the character), variance in the honorific T–V distinction (which may vary but should be consistent for each pair of characters), and erroneous sentence splitting. We intend to tackle these issues by using a document-level translation system which takes larger context into account, fine-tuning the model on a corpus of theatre play scripts, and adding various heuristic modifications where necessary.
11We use the pytextrank library with minor modifications to reflect the specific structure of our inputs, so that the algorithm returns N most important (potentially multi-sentence) full lines from the script instead of just N most important sentences. We set limit phrases=100.
12We find the first newline symbol in the last R tokens and keep all the lines after it.
13https://en.wikipedia.org/wiki/The_Thirty-Six_Dramatic_Situations We have not yet devised any automated or semi-automated evaluation setup to measure the quality of the generated scripts. Our design decisions so far have thus been based solely on manual analyses of small numbers of outputs, performed by theatre experts. While such analyses are very trustworthy, they are not easy to perform at an adequate scale. On the other hand, we are not aware of any meaningful automated measures of theatre script quality. We are currently exploring automated chatbot quality measures, which might or might not provide some useful indications of the script quality. We are also working on making the manual evaluation more ecient by designing a set of evaluation prompts and a standardized binary evaluation procedure.
which, in case of the presented play, have been rather positive.14 3
Acknowledgements
support of Technological Agency of the Czech Republic within the E´ TA 3 Programme. [Cˇ ap20]