=Paper=
{{Paper
|id=None
|storemode=property
|title=Measuring the User Experience in Narrative-Rich Games: Towards a concept-based Assessment for Interactive Stories
|pdfUrl=https://ceur-ws.org/Vol-634/Entertainment-Interfaces-Proceedings06.pdf
|volume=Vol-634
}}
==Measuring the User Experience in Narrative-Rich Games: Towards a concept-based Assessment for Interactive Stories==
https://ceur-ws.org/Vol-634/Entertainment-Interfaces-Proceedings06.pdf
Measuring the User Experience in Narrative-Rich Games:
Towards a concept-based Assessment for Interactive
Stories
Christian Roth, Peter Vorderer, Christoph Klimmt & Ivar Vermeulen
Center for Advanced Media Research Amsterdam
VU University Amsterdam
De Boelelaan 1081
1081 HV Amsterdam
The Netherlands
roth@spieleforschung.de
vorderer@gmail.com
Abstract: Narrative-rich game genres such as role playing games and adventure
games aim at specific dimensions of user enjoyment. Technology development in
“Interactive Storytelling” will enrich such narrative-based player experiences in
the near future, yet the conceptualization and empirical assessment of how players
respond to interactive narrative and their own role within the game story is
underdeveloped. The present paper introduces a set of self-report measures tailored
particularly for games with interactive story elements. It is based on entertainment
research in media psychology and has so far produced promising empirical
performance results.
1 Introduction
The integration of ‘story’ with the interactivity of video game play is both tradition-rich
and a current hot topic in game design and the evocation of player experiences.
Adventure games with limited interaction options but strong characters and complex
narratives belonged to the first commercially successful game genres (e.g., the “Sierra”
© adventures such as “Space Quest” ©). Today, new game concepts such as “Heavy
Rain” © explore enhanced possibilities to synthesize a pre-authored story with user
interactivity, by means of allowing dynamic shifts in plot development and user
influence on (parts of) the story at runtime. Connections between story and user
interactivity are likely to grow further in the near future, as technology development in
“Interactive Storytelling” is making progress in delivering truly dynamic, individualized
narratives to users (e.g., Cavazza, Lugrin, Pizzi, and Charles, 2002 [Ca02]).
From a user experience perspectives, such narrative-rich game genres represent a major
challenge in terms of theory and empirical measurement. First, with both strong narrative
and broad interactivity, a great variety of experiential dimensions is likely to play a role
in user responses to interactive stories. This diversity requires solid conceptual reflection
of existing approaches in game enjoyment and media entertainment in order to gain a
viable understanding of players’ experiences in narrative-rich environments. Second, the
fusion of story and interactive participation within one user experience calls for concept-
based empirical measures. Both tasks are related to improved game design serving
enhanced user experiences. Given the rapid dynamics in Interactive Storytelling
technology development, they are also important to help system designers in making
decisions which types of (preferable) user experience to go for and which according
technology options to select.
The present paper introduces a multidimensional self-report measure for the user
experience in interactive stories. Based on theory work in entertainment research as well
as empirical pilot studies, it is designed to both support scientific examinations of the
user experience in narrative-rich games and to enable comparative testing of new game
prototypes and game technology demonstrators. The measurement set is being developed
in collaboration with design experts from the “Interactive Storytelling” community (e.g.,
[Ca02].
2 Conceptual Groundwork
From a theoretical perspective, the integration of story and user interactivity in
generating player experiences requires the review of those experiential processes that
have been examined in conventional media entertainment (e.g., in film viewers or
readers of a fiction novel). At the same time, interactive story games display great
affinity with other computer game genres that focus less on narrative and more on
interactive task resolution (e.g., shooter games). Therefore, findings from research on
video game enjoyment need also to be considered if a valid conceptual model of the user
experience is to be developed.
To establish the theoretical ground for the intended measurement tool for user
experiences in interactive stories, the present research therefore combined literature
work in media psychology (both conventional entertainment media and video games)
with an expert interview methodology that addressed system creators’ experiences and
assumptions about how to describe user reactions to story-rich (game) interaction [Kl10].
By matching conceptual issues (such as player efficacy or video game identification:
[KHF07], [KHV09]) with experts’ assumptions (e.g., salience of player impact on story),
as set of experiential dimensions was distilled that covers ‘typical’ user responses to
interactive stories.
These dimensions were organized in a working model that assigns system usability and
character believability as system-bound preconditions for meaningful user experiences.
The model then assumes affective reactions towards game characters and perceived
interactive impact on the story and the system (“effectance”: [KHF07]) as fundamental
user experiences on which more complex responses can build, such as suspense or flow
(see figure 1).
Figure 1: Conceptual working model of the user experience in story-rich games [Kl10]
3 Empirical Implementation
Based on the conceptual groundwork, a set of brief self-report scales was developed. For
those experiential dimensions that have already been addressed in past research on video
games or conventional entertainment, existing measures were reviewed and adopted
(partially also adapted to the specific media context); for some dimensions, all-new
scales were created. The application logics of the measurement set is that users are
invited to fill in the user experience questionnaire immediately after their exposure to a
narrative-rich game is over. All items are scaled in a Likert way with 5 scale points that
range from 1 (“do not agree at all”) to 5 (“fully agree”) [Ve10].
A pilot experiment with players of the successful adventure video game “Fahrenheit”
suggests that all scales work from a statistical point of view (internal consistencies are
satisfying to excellent) and that they produce predictable or interpretable group
differences following manipulation of interactivity (on/off) [Ve10].
4 Discussion
The current set of user experience measures is evolving further based on more empirical
studies (currently, a pilot experiment with a recent technology demonstrator of
interactive storytelling is being conducted) and discussions of the overall approach in the
games and system design communities. So far, the conceptual approach and the derived
measures seem to work nicely in producing interpretable results of the player experience.
More testing will be required to see whether the scales deliver useful results across
different kinds of narrative-rich games and systems (e.g., RPGs, adventure games,
dialogue-based interactive stories).
A conceptually important question is whether the assessment toolkit covers all those
experiential dimensions that should be deemed characteristic or typical for story-rich
game experiences. For instance, the working model does not include the aspect of
performance-based user affect (e.g., pride, perceived challenge), which is known to play
a role in the enjoyment of other game types (e.g., shooters [JT07]). However,
performance seems to play a role in adventure game appreciation also, as a qualitative
pilot study has revealed recently [KVN10]. Adding a conceptual element (and an
according subscale) on perceived challenge and/or mastery experiences may thus
improve the overall assessment approach. The workshop presentation of the research line
is explicitly intended to collect other scholars’ recommendations on how to extend the
set of addressed user experiences to accommodate all important phenomena in story-rich
game experiences.
Finally, from a methodological point of view, linking the self-report methodology of the
current assessment instrument to other information sources, such as game event analytics
or physiological data from the player, will be useful in further studies to validate the
measures and to generate further insight into how the enjoyment of interactive narratives
and story-rich games works.
References
[Ca02] Cavazza, M.; Lugrin, J.L.; Pizzi, D.; Charles, F.: Madame Bovary on the Holodeck:
Immersive Interactive Storytelling. ACM Multimedia 2007; pp. 651-660
[JT07] Jansz, J.; Tanis, M.: Appeal of playing online first person shooter games.
CyberPsychology and Behavior, 10 (1), 2007; pp. 133-136
[KHF07] Klimmt, C.; Hartmann, T.; Frey, A.: Effectance and control as determinants of video
game enjoyment. CyberPsychology & Behavior, 10 (6), 2007; pp. 845-847
[KHV09] Klimmt, C.; Hefner, D.; Vorderer, P.: The video game experience as ‘true’
identification: A theory of enjoyable alterations of players' self-perception.
Communication Theory, 19 (4), 2009; pp. 351-373
[Kl10] Klimmt, C.; Roth, C.; Vermeulen, I.; Vorderer, P.; Roth, F. S.: Forecasting the
experience of future entertainment technology: “Interactive Storytelling” and media
enjoyment. Full paper presentation (poster) at the Annual Conference of the
International Communication Association (ICA), Communication & Technology
Division, June 22-26, Singapore, 2010
[KVN10] Klimmt, C.; Vorderer, P; Nuss, S.: Interactivity versus narrative: Using think-aloud
data to understand the enjoyment of playing adventure video games. Presentation at the
Annual Conference of the International Communication Association (ICA), Game
Studies Interest Group, June 22-26, Singapore 2010
[Ve10] Vermeulen, I.; Roth, C.; Vorderer, P.; Klimmt, C.: Measuring user responses to
interactive stories: Towards a standardized assessment tool. Full-Paper presentation,
submitted to the International Conference on Interactive and Digital Storytelling
(ICIDS 2010), November 1-3, Edinburgh 2010