What’s the Story? A Proposed Approach for the Evaluation of Experimental Interactive Narrative. Charlie Hargood Ben Artis Corey Stevens Creative Technology Creative Technology Creative Technology Bournemouth University Bournemouth University Bournemouth University chargood@bournemouth.ac.uk i7673319@bournemouth.ac.uk i7673602@bournemouth.ac.uk ABSTRACT 1 INTRODUCTION Evaluation of experimental digital narrative often focuses Digital narrative is, by its nature, experimental. How authors on the overall user experience. While this is important, we tell stories through technology has been changing rapidly recognise the need for more granular forms of evaluation to following continued exploration of the possibilities and the measure the efficacy of individual digital narrative delivery poetics of a range of digital mediums. Interactive fiction, techniques. These continue to grow in variety as authors ex- hypertext narrative, RPGs, and computer games have all ex- plore different approaches to telling stories using interactive perimented with different delivery mechanisms for stories media. In this paper we propose a multi-layered evaluation that play with both the interactive nature of digital nar- methodology based on the principle of deconstructing an rative as well as its dynamic qualities to create nonlinear interactive narratives internal fabula and story. This is fol- narratives. These delivery techniques include hypertextual lowed by three separate stages of interview for collecting patterns of narrative structure [3], hypertextual presentation evidence of the efficacy of different techniques used within paradigms such as spatial hypertext [4], and game narrative story payloads to deliver content. This proposed methodol- delivery techniques such as environmental storytelling [10] ogy shows early promise, and potentially provides a means and mechanics as metaphor [12]. For the purpose of this work to identify the individual efficacy of techniques within a we might define narrative ’delivery techniques’ as a story wider digital narrative. structure, medium, or other delivery format design decision within the narrative made for the presentation of a part of CCS CONCEPTS the story, whether a section of plot content or something • Human-centered computing → Hypertext / hyper- more subtextual such as a theme. media; While there has been a range of work exploring these different possibilities there is somewhat less work in the KEYWORDS area of bespoke evaluation methodologies for the narrative efficacy of these different techniques. Existing evaluations Digital Narrative, Evaluation, Game Narrative of these approaches normally follow the path of traditional user experience studies - often seeking to deconstruct the Reference format: usage of a system and its impact on the user [8]. However Charlie Hargood, Ben Artis, and Corey Stevens. 2017. What’s the Story? A Proposed Approach for the Evaluation of Experimental while these tell us much the ability of the system as a whole Interactive Narrative. In Proceedings of Narrative and Hypertext to deliver on ’immersion’ and ’flow’ [8] they tell us little Workshop @ ACM Hypertext 2017, Prague, Czech Republic, July 2017 about how much of the narrative was successfully delivered (NARRATIVE AND HYPERTEXT’17), 5 pages, CEUR-WS.org. and understood by the audience or the efficacy of individual techniques within the technology. In this paper we propose a multi-layered approach to the evaluation of the efficacy of digital narrative delivery tech- niques to deliver story. We present two games we have cre- ated as a group, and how we have deconstructed their nar- rative design in order to form an evaluation methodology Copyright held by the author(s). that goes beyond user experience and seeks to explore the efficacy of individual techniques to deliver story elements. NARRATIVE AND HYPERTEXT’17, July 2017, Prague, Czech Republic Charlie Hargood, Ben Artis, and Corey Stevens 2 BACKGROUND its story through a sequence of micro narratives embedded This work draws on both techniques for digital narrative into the world itself [5]. This includes the idea of environ- delivery (in surveying the different forms that might be eval- mental storytelling - that the story is not only delivered uated) and interactive entertainment evaluation (existing through the core plot surrounding its protagonist but also methodologies that might be applied). through the world and environment. In his work on envi- ronmental storytelling Jenkins [10] explores four different Hypertext Narrative techniques through which it has been delivered: evocative Hypertext fiction such as Joyce’s ’afternoon, a story’ and spaces (such as richly defined level environments that con- Moulthrop’s ’Victory garden’, has explored a range of nar- tribute thematically and depict the world), enacted stories rative delivery methods, often through different hypertext (micro narratives encountered as part of the idea of story as structures such as the calligraphic patterns originally ex- journey [1]), embedded stories (fragments of narrative de- plored by Bernstein [3]. This form of structural storytelling livered through discoverable game artifacts), and emerging may be used to deliver its own poetic effect whether it be narratives (actions and events in front of the player that lead the emphasis or change of context from rereading given to them to infer their own narratives). us by a cycle or the feeling of disorientation from a tangle. Interactive Entertainment Evaluation Similarly, sculptural and location-aware hypertext presents its own patterns [9] where an author may choose to deliver Existing game evaluation, as with most application evalua- narrative through simultaneously available parallel threads tion, is user experienced focussed. In her summary Denisova or an array of concurrent nodes. The variety of hypertext et al. [8] explore a range of game evaluation methodolo- forms and structures extends further still, and includes the gies exploring the player experience including the IEQ [11], juxtaposed montages of spatial hypertext [4]. These design GEQ [6], and PENS [16] methods. These approaches use ques- choices are important to the poetic value of the story and tionnaires to explore Flow, Presence, Control, and a range of they may radically change how a story is interpreted. They other variables connected to the UX of game play. While this are vehicles for delivery of narrative content to the reader is undoubtedly valuable it only provides limited insight into which are themselves part of the message. Where one struc- the narrative impact of a technique used. These approaches ture might be effective for a given story and might be less might tell us something about the players general enjoyment effective or distort the delivery in ways both desirable and or the suitability of the modes of interaction but not whether undesirable. a particular technique communicates the story we want it to. Game Narrative 3 DEVELOPMENT CONTEXT While some modern game narratives leverage the techniques common to more traditional mediums, such as film or text, A pair of narrative centric games under recent development there are experiments in new delivery mechanisms to deliver (which employ some of the delivery techniques discussed) as narrative in less traditional ways that leverage the interactive part of an exploration into the efficacy of these techniques nature of the form. The idea of ’Mechanics as Metaphor’ [12] was the primary motivator for this work. To give context we builds upon the more theory of ’the medium is the mes- offer the following descriptions of these games Paramnesia sage’ [15] from more traditional media and is used to describe and Blitz (as shown in figure 1’s screenshots). games who use their mechanics (the actions and interactions the player makes) as a means for conveying narrative itself. Paramnesia This may be rules of the game that reflect aspects of the The premise of Paramnesia is to use subconscious psycho- characters, changes to the dynamics to communicate affect logical techniques to control player actions as well as convey or emphasis in a scene, or metaphorical interactions within a strong narrative; using these subconscious techniques the gameplay. This is strongly connected to the idea of ’Diegetic’ theory is to better convey narrative and object points to choice within games [14] where choices made within the players, resulting in a closer cohesion of scripted narrative narrative of the game are made within the game’s own me- and the player’s interpretation of story. These subliminal chanics and dynamics, rather than abstracting the agency of techniques are similar in premise to conventional forms of the game away from the game play itself (so called ’Extra environmental storytelling in that the narrative emerges Diegetic’ choice). through the delivery of small sections often through the Other game narrative designers prefer to avoid more tra- space the player explores - though they are more heavily ditional grand narratives and instead use different narrative couched in related psychological theory. Paramnesia is a delivery techniques that focus on the game world delivering first person atmospheric game set in an apartment block and What’s the Story? NARRATIVE AND HYPERTEXT’17, July 2017, Prague, Czech Republic told the numbers. Suggestion [13] allows control over the player’s intentions through merely introducing the idea in a specific way. For example, throughout the game ’the voice’ uses the phrase ’We’d greatly appreciate it if you could...’. This is purposely used, and can be broken down to: ’We’d’ - Which suggests more than one person is watching and added pressure, ’greatly appreciate it’ - a nicety, which creates a bond between communicator and receiver quickly, ’if you could’ - which creates pressure just to accept while the re- ceiver is still thinking about the two previous steps. This culminates in a stronger bond and higher levels of compli- ance between ’the voice’ and the player. Blitz Blitz is a first person interactive story set in 1940s London. You play as a young boy who has been abandoned by his par- ents and has woken up in a version of his childhood house that has been separated from society. Exploring the house, the player discovers what happened to their parents. Blitz makes use of both Environmental Storytelling, and the no- Figure 1: Paramnesia (top) and Blitz (bottom) tion of ’Mechanics as Metaphor’. Environmental storytelling being the technique that is more heavily used. Throughout the first area there are clues and hints to the subway. The player is convinced to find, set and detonate a location, time and events that relate to the house you are in. bomb in an empty subway, under the intention of it ending a Graffiti on the walls as well as a range of propaganda posters communist uprising currently taking place in America. After reveal somewhat happened to the father and gives an idea of the detonation of the bomb, it becomes apparent that the the date that the game is set. There are other subtler hints, player was manipulated to perform this terrorist attack un- such as the photographs on the wall; in one of the images, der false pretenses. There were 4 psychological techniques a young boy is wearing glasses, and in the next area, there employed within the level: Conditioning, Compliance, Mere are a pair of glasses on a crate, that indicate you are the boy Exposure and Suggestion. in that photograph.The use of ’Mechanics as Metaphor’ is Conditioning [7] allows control over player direction, con- somewhat lighter. Player movement speed is slowed, and the tinuation of player motivation and player satisfaction. For camera height of the player lowered and increased in FOV so example, the use of red light was common throughout the that the player felt surrounded by everything within vision level which was paired with the idea of communism, this to highlight the overwhelming notion of events beyond the therefore meant a red light could be used to trigger the idea of protagonist. communism for the player allowing to keep the motivation 4 PROPOSED EVALUATION METHODOLOGY for their actions going. Compliance [17] allows the designer to control the player’s reaction to requests. This was used In order to explore the efficacy of the techniques within for control over the player’s direction as well as the player’s the games, we required a methodology that was granular interpretation of the narrative. For example, the voice (which enough to focus in on the impact of the delivery of individual is used to convey some objective and narrative) is created as pieces of story. We require a method that can consider the an authoritative figure to create compliance from the player coherency of delivery (did the technique successful deliver towards the instructions given. Mere Exposure [18] allows content in a way the audience understands), and to what for numbers, patterns and objects to be in the mind of the degree was its efficacy different to other techniques. player, purely because it has been show to them multiple times. For example, the code for the security door is never Multi-Layered Deconstruction explicitly told to the players, however, the player sees the 4 To begin to understand the answer to these questions we digit code up to 10 times on the way down to the subway in need to first identify what techniques are used where and graffiti, posters, price tags etc. This results in the numbers to deliver what. This process is effectively identifying what being in the player’s mind without them specifically being Narratologists such as Bal [2] would call the ’Fabula’. Here NARRATIVE AND HYPERTEXT’17, July 2017, Prague, Czech Republic Charlie Hargood, Ben Artis, and Corey Stevens the ’Fabula’ describes the collection of elements that com- if the payload successfully delivered its content and how prise the story’s content: its characters, events, places, and this was received by the user. This way we can evaluate the facts. We can then connect these fabula elements to their techniques based on how successful they were at content delivery in ’story payloads’ that are presented to the player delivery in the context of the payloads in which they were through particular techniques in a given scene. Each story used. It is not, however, suitable to simply ask a player ’did payload recognises the combination of 1 to n fabula elements you notice element ?’ as the question itself informs the and 1 to n delivery techniques as depicted in figure 2. player before they give their answer. As such we propose a three layer evaluation methodology for subsequent player interviews: (1) User-driven discussion - The user is encouraged to lead the discussion by discussing the story and its setting general terms without guidance to particular parts. This collects evidence for all story payloads without the player being lead to the significance of particular story elements through questioning. The player may mention particular elements and conse- quently demonstrate some effect for the payloads Figure 2: Each story payload is made up of 1-n fabula ele- connected to those elements. ments and 1-n delivery techniques (2) Interviewer-driven discussion - Fabula - The inter- viewer now delivers pre-constructed questions for As an example the code to the security door in Paramne- each fabula element to explore whether this content sia is a fabula element, it is delivered to the player in ten was successfully delivered to the player and how different payloads through out the game. These payloads all it was interpreted. This stage may lead the player make use of the delivery technique of ’Mere Exposure’ as to elements but not their delivery techniques, and part of environmental storytelling. It is sometimes packaged ensures what content was or wasn’t received by the into a payload with other fabula elements, for example the player. posters also communicate the anti-communist themes of the (3) Interviewer-driven discussion - Story - The interviewer piece. Consequently we now have a mapping from ’fabula’ now delivers pre-constructed questions for each story elements, to delivery payloads as part of the story which payload. This is the final layer of questioning and uses a technique. This example is depicted in figure 3. directs the players attention to specific parts of the story delivery. This may heavily lead the answer, but won’t affect previous sections and captures specific efficacy of each delivery technique. For each stage of this method the interviewer does not need to ask specific questions that have already been covered by evidence in a previous stage. For example, if evidence is gathered to support the player understood a given fabula element in stage 1, then the question for that element does not need to be asked in stage 2. As the interview progresses the evidence for player engagement becomes weaker due to their being potentially lead by the lines of inquiry, but it also becomes more targeted. In analysis, a researcher might conclude that a technique that was noted as effective for de- Figure 3: Example of two story payloads within Paramnesia livering part of the story unprompted in stage 1 has stronger evidence than one that has been discussed in stage 3 af- ter prompting. As a methodology we would also propose Multi-Layered Interview Methodology. Having identified that alongside the suggested interview approach an obser- exactly what is delivered and how within our narratives, we vational study is conducted where by the researcher studies can now propose a methodology for detecting the efficacy the player playing the game noting interaction with and of these techniques. The premise here is to interview users comment on different story payloads. This will then provide with questions connected to each story payload to detect a three layered data set for analysis from the interview along What’s the Story? NARRATIVE AND HYPERTEXT’17, July 2017, Prague, Czech Republic Figure 4: After deconstruction, three stages of interview capture evidence from the player on the efficacy of payloads. Starting with general discussion that will not lead the player (stage 1) and then progressing to fabula specific (stage 2) and story specific (stage 3) questions. with context from the observational study. This method is [5] Jim Bizzocchi. 2007. Games and narrative: An analytical framework. described diagrammatically in figure 4. Loading-The Journal of the Canadian Games Studies Association 1, 1 (2007), 5–10. [6] Jeanne H Brockmyer, Christine M Fox, Kathleen A Curtiss, Evan McB- 5 CONCLUSIONS AND FUTURE WORK room, Kimberly M Burkhart, and Jacquelyn N Pidruzny. 2009. The In this paper we have presented a multi-layered method- development of the Game Engagement Questionnaire: A measure of engagement in video game-playing. Journal of Experimental Social ology for the evaluation of interactive digital storytelling Psychology 45, 4 (2009), 624–634. technique efficacy. We have given context for our work in [7] Dennis Coon and John O Mitterer. 2012. Introduction to psychology: the form of the games Paramnesia and Blitz, and detailed a Gateways to mind and behavior with concept maps and reviews. Cengage multi-layered approach to deconstruction of an interactive Learning. narrative and then an evaluation methodology that this de- [8] Alena Denisova, A Imran Nordin, and Paul Cairns. 2016. The Conver- gence of Player Experience Questionnaires. In Proceedings of the 2016 construction enables. This methodology potentially enables Annual Symposium on Computer-Human Interaction in Play. ACM, game and interactive fiction analysis to go beyond criticism 33–37. and user experience and explore the efficacy of different tech- [9] Charlie Hargood, Verity Hunt, Mark Weal, and David E. Millard. 2016. niques which are increasingly varied in this fundamentally Patterns of Sculptural Hypertext in Location Based Narratives. In experimental medium. Proceedings of the 27th ACM Conference on Hypertext and Social Media. ACM, New York, NY, USA. While this position paper provides a proposition for the [10] Henry Jenkins. 2004. Game design as narrative. Computer 44 (2004), technique, future work needs to validate its efficacy as a 53. form of evaluation. Initial studies on the two games detailed [11] Charlene Jennett, Anna L Cox, Paul Cairns, Samira Dhoparee, Andrew are underway and initial researcher feedback suggests that Epps, Tim Tijs, and Alison Walton. 2008. Measuring and defining the this methodology is effective. One researcher stated ’the experience of immersion in games. International journal of human- computer studies 66, 9 (2008), 641–661. multi-layered approach allows me to get a more in-depth [12] Matthew Wilhelm Kapell. 2015. The Play Versus Story Divide in Game understanding of what technique was more influential, what Studies: Critical Essays. McFarland. design implementation communicated the technique etc’. [13] Irving Kirsch and Wayne Braffman. 2001. Imaginative suggestibility However the subtle nature of some of these techniques con- and hypnotizability. Current directions in psychological science 10, 2 tinues to make them particularly challenging to evaluate. (2001), 57–61. [14] Stacey Mason. 2013. On Games and Links: Extending the Vocabulary of Agency and Immersion in Interactive Narratives. Springer. REFERENCES [15] Marshall McLuhan. 1994. Understanding media: The extensions of man. [1] Ernest Adams. 2014. Fundamentals of Game Design (3rd ed.). New MIT press. Riders Publishing, Thousand Oaks, CA, USA. [16] Richard M Ryan, C Scott Rigby, and Andrew Przybylski. 2006. The mo- [2] Mieke Bal. 2009. Narratology: Introduction to the theory of narrative. tivational pull of video games: A self-determination theory approach. University of Toronto Press. Motivation and emotion 30, 4 (2006), 344–360. [3] Mark Bernstein. 1998. Patterns of Hypertext. In Proceedings of the [17] Chun Siong Soon, Marcel Brass, Hans-Jochen Heinze, and John-Dylan Ninth ACM Conference on Hypertext and Hypermedia. ACM, New York, Haynes. 2008. Unconscious determinants of free decisions in the NY, USA, 21–29. human brain. Nature neuroscience 11, 5 (2008), 543–545. [4] Mark Bernstein. 2011. Can we talk about spatial hypertext. In Proceed- [18] Robert B Zajonc. 2001. Mere exposure: A gateway to the subliminal. ings of the 22nd ACM conference on Hypertext and hypermedia. ACM, Current directions in psychological science 10, 6 (2001), 224–228. 103–112.