Suspense is known to play a crucial role in storytelling phenomena in general, and computational storytelling systems in particular. While several story generation algorithms have addressed suspense, they have usually done so either by focusing on the narratology-related aspects, or by providing functional approximations of the potential response. In this paper, following the overall objective of devising a cognitive model of suspenseful stories, we have adapted a story generation system so that it includes elements represented by a ective terms extracted from experiments. In this way, we put the focus on the cognitive aspects. The obtained results suggest that the generation model works and that evaluators perceive levels of suspense comparable to the ones obtained in the previous contributions.
The long-standing goal of achieving creative behavior in an automatic story generation system requires, at least, mimicking aspects of narrative that humans commonly use and thus expect when they experience storytelling. Among these aspects, suspense is one of the most prominent and e ective ones. Actually, it is a key factor in a wide range of narrative media: together with coherence and thematic complexity, suspense explains 54% of the variance in interest of a narrative, making the single greatest contribution explaining roughly 34% [30, p. 436, 444].
Being such an important aspect, several computational systems producing
suspenseful stories have been created. Many of them have focused on partial
perspectives. They mostly tend to focus on cognitive-related aspects by analyzing
the escape strategy of the protagonists, as can be seen in the work of Gerrig &
Bernardo (1994) [
With this objective in mind, we have previously proposed an architecture that
tries to address the cognitive aspects of suspense as a whole [
In this paper we describe the process and the results of adding a cognitive
model of the in uence of decorative elements versus other scene a ective elements
{threat features and resources{ in the perception of suspense to an automatic
storytelling system. To achieve this, the architecture has been built as integrated
part of the generator Stella [
In order to test the results, we have developed a stripped-down version of Stella. This version is able to produce stories that include several elements and actions that trigger speci c cognitive responses in the audience. With these generated stories, an experiment with N = 37 human subjects was conducted. The subjects rated their perceived suspense from an small core of suspenseful stories based on a classical threat-discover and escape sequence.
From the set of stories generated by the story generator, four representative scenes were selected by hand. The selection was carried out according to the affective values of a set of features: threat profession, threat physical aspect, threat weapon and decorative elements. The results suggest that including explicit, validated cognitive model of the reaction to suspenseful improves generation results and, more importantly, provides a new set of parameters for story generation.
The paper is structured as follows: Section 2 describes the related previous work on suspense through emotions evoked by story concepts, and how suspense is currently treated by automatic storytellings. Section 3 explains the inclusion of the architecture model in Stella. Section 4 describes the experiment, whose results are detailed in Section 5. Finally, Section 6 and Section 7, respectively, discuss and summarize these results. 2
This section tries to provide a general description of the state of the art in the related elds: emotions and their relation with decorative elements (sec. 2.1), automatic suspense generation (sec. 2.2) and the story generation system that was adapted for this research (sec. 2.3). 2.1
The impact in suspense of the elements in the scene seems to be strongly
related to their semantics and the emotional characteristics that, on average, they
trigger on the audience [13, p. 28]. This hypothesis was demonstrated in [
The American ANEW was created as the result of an experiment of Bradley
and Lang (1999). Participants were asked for the emotional a ection of 1034
words. To evaluate the set of words, participants selected each dimension
painting in a 9-point rating scale represented by the Self-Assessment Manikin (SAM)
[
Emotional valence describes the extent to which something cause a positive or a negative emotion [8, p. 79]. In terms of the story, an element has a negative valence when it pushes towards a negative outcome. It has been extensively investigated the paradox in that texts with negative valence are perceived as more amusing than texts with neutral or positive valence.
The second dimension is the arousal [
Finally, the third dimension, called dominance, control or power, re ects the degree of control an individual feels over a speci c stimulus and extends from out of control to in control [21, p. 888].
In Delatorre et al. (2017), results show that these a ective dimensions make it possible to suspense in a scene through the modi cation of the decorative elements present in it. Thus, decorative elements, even when not in uencing the narrative plot, impact the perception of suspense. This impact is related to the emotional features of these objects [11, p. 297]. Concretely, the experiments support that valence and dominance are moderately correlated with suspense ( val = 0:579, p < 0:01; dom = 0:423, p < 0:01) for textual stories. 2.2
This section has summarized the treatment of suspense in the main computational narrative systems. Given the scope of the proposed model of suspense, the review focuses on generative systems.
MEXICA [
MEXICA is an exception in the use of positive emotions to implement the narrative tension. The system works with two prede ned types of emotion: brotherly love and amorous love, both ranging from -3 (negative emotion) to 3 (positive emotion). Additionally, ten types of tension are de ned (actor dead, love competition, health normal. . . ), which are generated based on the type and emotional value of each character. The stories search degradation-improvement curves through actions that transform the extent of the tensions.
MINSTREL [
Another initiative is Suspenser [
Also based in Gerrig & Bernardo's work, Dramatis proposes an
implementation of a system to evaluate suspense in stories that utilizes a memory model
and a goal selection process [26, p. 5], assuming that the reader, when faced
with a narrative, evaluates the set of possible future states in order to nd the
best option for the protagonist. With a similar target, Dramatis generates
escape plans attempting to \break" the causal links that would reach non-desired
3 \Readers feel suspense when led to believe that the quantity or quality of path
through the hero's problem space has become diminished". [
Finally, we review IDtension [
Suspense is treated by IDtension as a reaction to the obstacles (con icts), and is correlated to the risk of facing every expected obstacle (high or low risk, without intermediate values). The narrative e ects of the tension are calculated by six criteria: ethical consistency, motivational consistency, relevance, cognitive load (in uence in the story), characterization and con ict. Also, the condition is managed by a series of actions as accepting, refusing, congratulate, etc., available for use on / among the characters.
With respect to our goals, the review of the above systems has exposed the aforementioned comparative limitation: we can observe that none of them takes into account a general cognitive theories as explicit part of the model, neither physical aspect, resources nor environmental issues. 2.3
Stella is a story generation system based on a hybrid model in which exhaustive,
non-deterministic simulation is controlled by a narrative layer [
Stella models stories as time-ordered sequences of states. Each state contains a detailed representation of each of the entities that populate it: physical information, emotions, intentions, knowledge about the world, and others. The simulation is carried out non-deterministically. On each generation step, the current state scurrent is expanded and all its potential next states fs1next; s2next; : : : ; snnextg are generated. This means that, for each non-deterministic option for each next value of each attribute for each entity, a new path is created. This produces a vast generative space of stories.
The narrative layer receives this huge space and uses narrative information to identify the best alternatives. In order to do this, Stella uses several techniques. The system allows to set conditions for the output stories. For instance, it is possible to lter out those stories which do not include a murder, or search for stories in which there is a love scene. Stella can also control the generative process by setting narrative aspects like length, amount of interaction between characters and other general aspects of narrative.
Additionally, Stella can drive the generation by controlling the evolution
of user-de ned dimensions, which are represented as curves [
As previously introduced, the main objective of this research is to include a partial cognitive model of suspense in a story generation system. This section explains how the Stella story generation system has been adapted to generate short suspenseful stories by including elements and actions that convey a ective response related to suspense. 3.1
As introduced in section 2.3, the story generation system Stella addresses many characteristics of storytelling. This implies a complex execution model and an elaborated data model. These two aspects make it di cult to isolate and control a speci c subset of the generation parameters, which is crucial if the output is to be tested against human evaluators.
In order to run a more controlled environment, a new, stripped-down version of Stella has been created. In this version, only the core generation engine and the knowledge model have been kept. The curves generation and matching engine have been removed, and the generation constraints and objectives have been made simpler.
The simulation engine has also been simpli ed. The physics subsystem in Stella is based on Newtonian equations, and Stellite only performs simple computations for movement in a discrete world. Instead of having a full physics engine, in Stellite characters move only to interesting locations (i.e. hard-coded landmarks). Besides, manipulation in the full version of Stella lets characters grab and store things, and do multiple things at the same time if they are possible. In Stellite, characters can only grab or release one item. Instead of developed behavioural models, they have simple traits (like \being a murderer") and straightforward objectives (\escaping" or \killing") which are now speci c symbols with semantics intended to guide the generation.
With this added information, Stellite performs story generation. This process creates stories as ordered sequences of states. The generation in carried out nondeterministically, in such a way that for each current state scurrent all possible next states fs1next; s2next; : : : ; snnextg are explored, in the same way that original Stella does. The expansion of the nodes uses the non-deterministic simulation engine to produce all candidate new states. On each new state, all characters are updated: they receive the world information, reason, update their objectives, and act accordingly.
Compared to Stella, the set of output stories is noticeable smaller. This
reduces the chances to nd a highly original story, and makes it impossible to
generate at a very ne detail, but the generation is faster and the output stories
are all coherent.
As introduced in Section 2.1, we have previously found a correlation between
the a ective aspects of terms and the suspenseful feeling they trigger [
ANEW includes 1034 words. For implementing Stellite and testing the e ect
of terms, we have selected a subset justi ed by the category of the terms:
professions (20) {from terrorist (valence of 1.51) to writer (6.47){, physical adjectives
(30) {from sick (valence of 1.61) to elegant (7.23){, handheld weapons (9) {from
pistol (valence of 1.83) to bottle (5.10){ and decorative elements (25) {{from
corpse (valence of 1.41) to ower (7.34); this one coming from the set already
collected in [
In order to model professions in Stellite, we have selected names that represent tasks which may be performed by the corresponding characters. Physical adjectives are considered as visually perceptible features. The set of chosen handheld weapons includes resources that can be used by a character to attack another character. Likewise, decorative elements are de ned as those entities present in a scene or story which do not play a role in the main plot, and could therefore be interchanged by others or removed without any relevant change in the narrative structure of a story. Accordingly, no entity in Stellite interacts with them. 4
This section describes the experiment and the methodology that was applied to extract the information about the existing di erences between audiences of interactive and non-interactive stories. Section 5 describes the results. 4.1
The experiment took place in the Computer Science Faculty of University of Cadiz (Spain). The experiment was publicly announced to all students, and those wanting to take part in it voluntary enrolled. The objective and the process were explained to them and they all accepted the conditions. Thirty seven undergraduate students (N = 37), 32 males (86.49%) and 5 females (13.51%) participated. All the participants were students of Computer Science degree. Their ages ranged from 21 to 33 years (mean = 23:92, stdev = 3:15). All participants were Spanish native speakers. There was no compensation for participating in the experiment.
The low proportion of volunteer female subjects made the experiment unbalanced in terms of participants' gender. This issue is analyzed in Section 5 and discussed in Section 6. 4.2
Stellite was parameterized to generate stories about one character trying to escape and one \murderer" trying to kill him. The world was formed by a simple map including a bedroom, a corridor and a room with an exit door. The generator was con gured to generate stories with 6 events.
By combining character decisions, a ective terms and di erent endings, Stellite generated 10275 stories. Several stories did not include any suspenseful event, and some others did nish without a clear outcome. In order for the evaluation to be doable and useful, we decided to choose one instance in which there was some suspense. It is obvious that this selection a ects the overall results, and this is discussed in Section 6.
The text was generated from the structured representation. The content was ltered and rendered in Spanish with simple text templates. The chosen story is shown next4:
John walked down the corridor, trying not to make noises. Suddenly, he heard something behind. When he turned back, he saw the terrorist. The terrorist was sick and carried a pistol. John tried to escape through the door. On the oor, there was a ower. John opened the door. The terrorist chased John.
In order to provide variation, we selected four versions of the previous short story in Stellite. For selecting these versions, we combined decorative elements and profession-aspect-weapon (threat), and crossed maximum and minimum values of valence. The next list details the selection: 1. The threat is a sick terrorist carrying a pistol. There is a corpse on the oor: minimum valence for scene, minimum valence for decoration. 2. The threat (the murderer) is a sick terrorist carrying a pistol. There is a ower on the oor: minimum valence for scene, maximum valence for decoration. 3. The threat is an elegant writer carrying a hammer. There is a ower on the oor: maximum valence for scene, minimum valence for decoration. 4. The threat is an elegant writer carrying a hammer. There is a corpse on the oor: maximum valence for scene, maximum valence for decoration. The experiment was run as a paper-and-pencil session in one single classroom. Participants were randomly placed, keeping an empty table between each pair. 4 In the original Spanish: \Juan caminaba por el pasillo tratando de no hacer ruido.
De repente, oyo algo a su espalda. Cuando se volvio, vio al terrorista. El terrorista estaba enfermo y llevaba una pistola. Juan intento escapar por una puerta. Junto a la puerta, en el suelo, hab a una or. Juan abrio la puerta. El terrorista persiguio a Juan." A single demographic survey was lled by each participant. The experiment was explained. Subjects were warned that the scene that they were about to read could have been written either by a human or by a computer. After this point, the content was presented, handing the evaluator over a sheet with one version of the story. Versions were equally distributed.
The test invited the participant to imagine the scene and rate the level of suspense using a 1 to 9 range (according to the SAM model, which is used by ANEW). Additionally, subjects were asked to answer the question: \Do you think this story has been written by a human?". 5
This section details the results obtained from the experiments described in previous Section 4, comparing the suspense of the di erent versions of the scene. For all measures, the criteria for statistical signi cance was set at = 0:05.
Globally, results show a moderate-high downhill correlation between reported suspense and valence of the scene ( = 0:749, p < 0:000). There are three different signi cant groups identi ed. There were no signi cant di erences between version 2 and 3. Di erences by story elements are showed5 in Fig. 1.
When comparing the e ect of the threat with decorative elements, results show that both in uence reported suspense. However, the e ect of decoration (Z = 2:394, p < 0:02) is weaker than the e ect of the threat features (Z = 4:945, p < 0:000). Illustratively, Table 2 shows the mean and standard deviation for each dimension, where a sightly higher standard deviation is observed when the decorative element is a ower.
This e ect of decorative elements may be contrasted by analyzing the responses on whether the question had been written by a human or not. Actually, while signi cant in uence of the threat features was not found (Z = 0:176, p = 0:860), the decorative elements used in the story clearly in uence it (Z = 2:129, p < 0:05): 72.22% of the subjects reported that the creator was a machine when the decorator was the ower; in contrast, only 36.84% of the subject reported that opinion when the decorator was the corpse. This may be in uenced by the coherence of the element in the story, as will be discussed in Section 6. Tak5 Terrorist includes sick and pistol. Likewise, writer includes elegant and hammer. It has been reduced due to the limited space.
e s n e p su 6 s d e tr o p re 4 a 7.9 (0.99) dimension mean std threat feature sick terrorist carrying a pistol 7.15 1.80 elegant writer carrying a hammer 4.11 1.93 decorative element corpse 6.63 1.95
ower 4.67 2.47 ing into account all the answers, 54.05% of subjects thought that the story was generated by a machine, 35.13% when the ower was in scene.
Finally, no signi cant in uence of gender in general reported suspense has been found (Z = 0:135, p = 0:893). The same result has been obtained when analyzing the impact of the a ective elements by itself. 6
Despite of the promising results, some issues need to be addressed.
In contrast to the idea that audience gender in uences suspense [
The seed story used for the experiment, while generated by the story generation system, was chosen by hand. Automatically identifying which story would provide the best outcome (or a list of valid set of answers in terms of suspense) would require a computational system at least as complex as the story generation system. This would have made it impossible to run the proposed texts. However, further research considers addressing this issue and provide a model of identifying promising stories.
It is perceptible that most subjects consider the story as created by a computer when the scene includes the ower. We understand that this e ect is due to a sort of learned-inconsistence: while corpse is a self-explained typical decorator in a suspenseful chase scene, focusing a ower may be decontextualized.
In fact, there are combinations of elements that have not been tested and could evoke a similar perception {for example, what would happen if the terrorist is carrying the hammer{. However, checking the consistence of a ective elements working together, despite it is an essential mechanism of an automatic generator of suspense, it is outside the scope of this paper. 7
The paper has described an original system that produces short stories by including suspenseful decorative elements in them. The decorative elements and their expected impact in humans were previously validated, and we have used this information to compare the performance of the story generation system against the expected value with human evaluators.
Additionally, other evaluable a ective elements, which were not tested previously {threat features and resources{, have been included in Stellite. The in uence found in the reported suspense by the experimental subjects supports the evidence that the e ect of the emotional a ectivity is not limited to the decorative elements. It leads to analyze the particularities of this e ect and quantify it, in comparison with the decorative elements, as immediate future work. This will be addressed as part of the future work.
This work has been funded by the Andalusian Government under the University of Cadiz programme for Researching and Innovation in Education 2015/2016 (SOL-201500054211-TRA), and by the IDiLyCo project (TIN2015-66655-R) funded by the Spanish Ministry of Economy, Industry and Competitiveness.
The research presented in this paper has been carried out as part of the rst author's work in the PhD Programme in Computer Science of Universidad Complutense de Madrid (RD 99/2011) and is part of his PhD dissertation.