Recent advancements in large language models (LLMs) had a significant resonance in the artistic sector. As a result, numerous dialogue interfaces show potential applications for creative practices, of which creative writing is the focus of this paper. Although some studies have identified roleplaying with LLMs as a strategy to support artistic inspiration, there are still many open questions. For example, studies on how writers could employ LLMs based on roleplay with fictional characters require further investigation. To address this gap, we present a case study we are designing for the involvement of creative writers in roleplay interaction with LLMs. This study aims to provide training on how to use Faraday. dev, (a platform designed to create LLM-based characters). Subsequently, we will invite the writers to roleplay with their creations and complete a creative writing task. Collecting the prompt used to edit the characters, the chat logs between the writers and the characters, the final writing excerpts, and conducting follow-up interviews, we aim to gain insights on how LLM-based characters impact creative writing. Ultimately, this study seeks to inform the design of roleplay-based systems and enhance support for creative practice in the HCI domain. Joi is one of the central characters presented in the dystopic sci-fi universe of Blade Runner 2049. Despite aesthetically appearing as a young woman, in the narrative, she is just an AI hologram, a device designed to be a customisable romantic partner. Throughout the story, we witness Agent K, the main character, experience the relationship with Joi with a certain intensity. He, as the spectator, is fully aware that his partner is a commercial product. However, this does not prevent him from attributing thoughts, desires, and emotions to Joi. This does not make Agent K a fool. He knows the truth, but perhaps because of her ideal characteristics as a partner or his lack of sincere human contact with others, she suspends his disbelief. However, neither Joj nor Agent K exists. They are fictional characters created by talented scriptwriters to arouse intense emotions in the reader. Indeed, while reading a captivating novel, watching a movie, or a play, humans tend to empathise with characters unfolding within the stories. The suspension of critical judgement promotes the appreciation of fiction as the audience finds themselves emotionally absorbed in the narrative.
2024 Copyright for this paper by its authors. CEUR
ceur-ws.org
To document how this use could potentially impact creative writing, we are designing a case study involving amateur and professional writers. We pose the following research question: How could LLM-based characters contribute to creative writing?
This paper is organised as follows. In section 2, we introduce the related work presenting the concept of the paradox of fiction in interactive systems and connect it to the suspension of disbelief. Section 3 describes the methodology we are developing to answer the research questions. In this section, we focus on participants’ selection criteria, present the LLM character training experience we are organising, and report methods of data collection and analysis. Finally, in section 4, we present expected results and critical topics of investigation.
There are meaningful philosophical and psychological discussions about the ontology of fictional experiences, how they compare to real ones and their intensity [9, 16, 21]. Of particular relevance for this study are the reflections on the paradox of fiction [21]. This concept was initially applied to writing [14] and progressively to technological interactive media [13, 23]. According to Konrad and colleagues [10], this term captures the contradictory nature of feeling moved by fictional entities, arguing that the following three statements cannot be jointly true at the same time: 1.
Whether the subject of our emotions is Anna Karenina or Joi, the same questions arise: Are the emotions we feel for fictitious characters real? How do these emotions differ from those we feel for real entities? To better understand this paradox, suspension of disbelief is a key element to consider, as both are closely related concepts that address the relationship between fiction and our emotional responses to it. Originally conceived in the 19th century as an act of “poetic faith” promoted by the authors’ ability to imbue their work with “semblance of truth” [19], suspension of disbelief is a fundamental principle in contemporary narrative. Despite the inherent complexity in the definition, there is a general agreement that suspension of disbelief is a necessary state for the audience to invest in the characters and events unfolding before them emotionally [6]. Accordingly, there is no deception, but an implicit agreement between the audience, the author, and the performer, who together conjure the experience. The connection between these two concepts lies in the fact that suspension of disbelief is necessary for the paradox of fiction to occur. For audiences to experience genuine emotions while engaging with fiction, they must temporarily suspend their disbelief and invest emotionally in the characters and events of the narrative, despite knowing they are not real.
Moreover, as for the paradox of fiction, suspension of disbelief is often elicited by technology,
such as in movies [6], or interactive media such as video games [13], or social robots [5]. Unlike
non-interactive media, the active role of the people engaging with these systems allows them to
make choices and manipulate, within some limits, the fictional situation. At the same time, the
virtual environment emotionally affects the users, impacting the choices they make and their
actions. Right now, many AI-based commercial products are available on the market, and their
anthropomorphic design, is a key component of the interaction with the user [15]. There is
nothing new about the fascination that talking machines exert on humans [
To better understand how LLM-based characters could impact creative writing practice, we propose to conduct a case study and analyse the results through qualitative methods. The study will require the voluntary participation of 5-10 creative writers selected with the following criteria: they must have documented amateur or professional experience as authors in writing novels, poetry, screenplays, short stories, or prose. Participants will be invited to a two-session training experience.
In the first session, we will present Faraday. dev, an application created to dialogue with AIpowered characters. This system, supported by Cloud - Mythomax 13B (Llama 2), was fine-tuned to support roleplaying and storytelling. The interface allows the creation of customisable characters. To do so, the user defines basic instructions for the model in natural language, describing the character persona, aesthetic, behaviour, and other relevant information (Figure 1).
Moreover, the system allows defining a specific scenario where the interaction occurs and provides a description (real or fictional) of the user. Modifying other models’ parameters (e.g., temperature, min-P, repeat penalty) is also possible. Subsequently, they will test the characters they created by roleplaying with them through a chat interface. The purpose of session one is to explain how to use Faraday, support the participants in creating interactive characters and make them experience the outcome in a roleplay dynamic.
Participants will be asked to complete a creative task, which consists of writing a dialogue/a short story excerpt of 1800 words inspired by the interaction with the LLM-based character they created. Finally, during a final collective discussion, every participant will present the character created, explaining their choices and the ratio behind them. Moreover, they will be asked to read their final work and comment on how the character was employed and how the roleplay impacted the creation.
1. The prompts created by the writers and used to edit the characters. We will ask participants to provide a description of a fictional character. During the creation phase they will try to adapt this description to make the model interpret the character. We will then compare the first description with the final one. This data will be divided according to the interface sections (character persona, user persona, scenario, etc.). The descriptions will be analysed through inductive content analysis, allowing us to identify similarities and differences. 2. We will collect the chat logs between the writers and the characters they created. We expect to collect different sets of conversations with the characters and perform a conversation analysis to document how roleplay interaction unfolds, especially focusing on repair mechanisms and strategies used to resolve conversational difficulties. 3. The final excerpts inspired by the roleplay will also be analysed through inductive content analysis. This data could be the key to understanding the leap between roleplay and artistic reworking. 4. We will conduct follow-up interviews with the participants. Through this data we aim to give voice to the writers commenting the experience, highlighting limitations, opportunities and expressing how they created their final scripts.
Combining the analysis of these data we expect to perform a comprehensive thematic analysis according to the general inductive approach [18]. Given that roleplaying with LLMs can be considered a frontier practice in human-computer interaction, we believe that the inductive method may be better suited to uncover the overall value of the experience.
Conducting this case study, we aim to understand how roleplaying with LLM-based characters can impact creative writing practice. Although previous documented experiences have highlighted both limitations and opportunities of using these systems for writing [2, 7, 8, 12], specific modalities of use, such as roleplay, still need to be investigated. Adopting an ad hoc model, we want writers to play with the characters they created, expressing considerations on the entire process, starting from creation through roleplay interaction and concluding with writing fiction.
As in previous studies, we expect LLMs may show limitations in directly fulfilling creative writing tasks[8]. However, we argue that suspension of disbelief could contribute to ignore those features, granting the writers to gain inspiration through roleplaying with the character they created. We speculate that emotional engagement with the characters, fostered by suspension of disbelief, could be useful to support creative inspiration, and the data we aim to collect could contribute to understanding how. The data could also contribute to understanding if and how the paradox of fiction affects the interaction between creatives and AI-systems acting and behaving as defined characters. Moreover, this research could highlight exciting insights about the role of anthropomorphism of technological entities.
This study aims to identify which nuances support the process specifically, and to develop ideas to amplify the creative utility of LLM. We expect this study to provide suggestions on how to improve the design of new roleplay-based systems and open trajectories for supporting creative practice in the HCI domain.
Given the rapid developments in LLMs, it is possible that the model used for research will soon be obsolete. However, subsequent studies could investigate different models and specialisations. Furthermore, as we noted in our previous study, the digital skills of the participants could have an impact on performance. It is therefore necessary to support participants in learning and using the system.
This work was supported by a Ph.D. grant awarded by the European Commission under the NextGeneration EU program (Mission 4, Component 2 - Investment 3.3 - call for tender No. 351 of 09/04/2022 of the Italian Ministry of University and Research).