User interfaces are substantial parts of the gamer's experience, but they are not only showing useful information to the player. Indeed, they can also be used to transmit narrative elements, even if sometimes only under the form of the “mood” of the game. Since interfaces can be used to enhance storytelling, their design should be aware of issues of narrative theory. The suspension of disbelief is probably the main cognitive mechanism to be considered, that is the process of both knowing and forgetting that what is happening in a game is fictional. The equilibrium between those two sides is precarious. This is why all the narrative elements have to be created to keep the balance: user interfaces are not excluded. In this paper, the focus of the analysis is the possible contribution of interfaces to the storytelling, and the contribution of narrative theory in game design, particularly regarding the suspension of disbelief.
What I intend here by “multi-sensory interface” is a connecting system that enables a person to exchange multimedia messages with a software application (the game, in this case) constituted by simultaneous, intercurrent or convergent use of different media, tools and languages (text, graphics, animations, sound, static and moving images, vibrations). It is clear why the exchanged messages are GHItaly18: 2nd Workshop on Games-Human Interaction, May 29th, 2018, Castiglione della Pescaia, Grosseto (Italy) Copyright © 2018 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors. “multi-media”, and many are the attempts to establish a twoway multimedia approach (like in many Nintendo DS games). Talking about the visual appearance of a game, we can distinguish two layers: the “lower” layer is the one of the gaming world, characterized by the purpose to create a fictional environment in which the player can act through his/her in-game alter-ego, i.e. the Player Character (PC); on the other hand, the “upper” layer is constituted by the User Interface (UI), with the purpose of showing important information to the player, without which is practically impossible to play, but which sometimes are not part of the fictional world itself. It is a well-established custom to have a UI to display all needed information to the player, so much that, contrarily, it is strange to not find one at all. Of course, this distinction is arbitrary, as there might be borderline and halfway cases, as we will see.
In this paper, the focus of the analysis is on how and to what extent the visualization of information in overlaying interfaces is concurring to the storytelling, and on the issues and possibilities of UIs in the sensemaking process, to enhance the narrative aspects of a game.
RELATED WORKS AND METHODOLOGICAL NOTES
Many studies concerning the relation between different game
elements and narrative have been done. Probably the most
felt one is the relationship between narrative and gameplay
(see e.g. [
Contrarily, there have been some writings claiming that
video games are not generally telling stories, and video game
development has not to deal with a storytelling process as
commonly intended [
I disagree with the idea of dividing storytelling and game
design in the development of a game (not alone: it will be
enough to compare the positions of [
My analysis will be conducted firstly by applying semiotic
categories to video games (particularly interesting will be the
Halo series by Microsoft) to define different types of UIs that
is possible to distinguish in video games, to better see how
they are related to the game world. By analysing two
different game series, namely the Assassin’s Creed series
(Ubisoft) and the Grand Theft Auto series (Rockstar), I will
point out an apparent trend of the design of UIs, tending to
prefer in the last years less visually impactful interfaces.
Secondly, on Juul’s footsteps [
The behaviour of these parts of the UI is sometimes very different even in the same game, but they are nonetheless capable of transmitting elements of the story, also only by giving a general idea about the game narrative. It is sufficient to see what happens in The Elder Scrolls V: Skyrim (TESV: Skyrim; Bethesda Game Studios, 2011): the design of the main menu with the dragon emblem, the fog/smoke and the choral music brings the user into an epic fantasy world with dragons and an unknown language before the game is even started. To have a clearer view on the impact of those elements on the narration, a distinction has to be made first. TYPES OF HEADS-UP DISPLAYS For a matter of simplicity of explanation, we will talk first about HUD and I will later try to extend the same distinction to menus and UIs in general. A distinction that is useful to make is that between external-HUD and internal-HUD or, with a more literature-oriented definition, I would say between heterodiegetic HUD and homodiegetic HUD. This distinction is based on the relation between the game world and the interface itself, i.e. how the latter is being created in relation with the former and integrated in it. As we will see in a moment, this distinction entails a totally different view over HUDs and their integration in the narrative of the world. On the one hand, what I call external-HUD is the most frequent custom, in which the interface is totally detached from the fictional world and its characters. In this first case, what we have is a simple layer inserted in the screen between the world being played and the real-world player, a layer that displays information that are as useful as valueless (from the point of view of the in-game characters). An example for this kind of interfaces might be the Grand Theft Auto (GTA) San Andreas (Rockstar North, 2004) interface, in which all the information like active weapon, ammunitions, health bar, etc. are placed in a layer stuck to the player’s camera lens. The more recent Grand Theft Auto V (Rockstar North, 2013), represents a slight change of perspective, and underlines a trend that is gaining consensus.
On the other hand, internal-HUDs are a less travelled road, but, I believe, they have the huge benefit of improving immersion, by integrating the UI inside the game itself.
What we have here, differently from external-HUD is that all the information showed are not placed in a layer stuck between game world and players, but rather in a layer between the character’s eyes and the world s/he sees. The player, in turn, is seeing the world through the eyes of his/her character, and all the information s/he sees are shared with his/her in-game alter-ego. In the Halo series, for example, we have a HUD that is visible also to the player’s character, as it is a projection on the screen of the helmet he wore during the game, created by the computer installed in his high-tech armour. When the helmet is offline, no UI is visible to both the player and the character (like in the very beginning of the game Halo 4 (343 Industries, 2012)). Of course, this is more problematic to realize when the camera is not in a firstperson view, and indeed, even in Halo, there is an exception of the mimetic internal-HUD way of displaying the information: when the point of view is switched to thirdperson (e.g. when riding vehicles), the interface remains visible, violating the general iconic intent. “Icons”, on Pierce’s footsteps, are the product of a primary modeling system, and their main characteristic is their “topological similarity” with the object they are standing for. Of course, this switch from an internal-HUD to an external one is not noticed by the player. S/he is deceived by the fact that the interface looks always exactly the same, even if there are actually two HUDs that are completely different in the very substance. This might be considered a borderline case, but the hope is that it is enough clear to show what is intended with internal-HUDs. Also, it suggests that there could be cases of interaction of the two different types of HUDs (and UIs in general), in which, in a single view, we may have internal and external element (e.g. the Metal Gear Solid (Konami Computer Entertainment Japan, 1998) Codec, in which iconic elements are mixed with non-iconic ones, like text).
GENERALIZING THE DISTINCTION As mentioned, the distinction between external and internal HUDs is generalizable and can be applied to the entire system of UIs, but probably the mimetic intent is not applicable everywhere in the same way. As for menus, it seems to be quite difficult to apply, and find example of iconic intent in menus is not so easy. Many options listed in menus does not deal with the game world but with the multisensory interface, i.e. they are affecting not the game world itself but how the player relates with the game as a software application, and vice-versa. For this reason, a totally iconic menu is probably not possible, but there has been for sure some attempts in this way: in the game Crash Bandicoot 3: Warped (Naughty Dog, 1998), the Hub Level is a playable recreation of a menu in which the user can move freely instead of simply choosing a level from a list, and in the main “square” there is a screen with which it is possible to save and load the game. This can be intended as a meta-menu, in which an in-game menu is applying effects on the game itself. Anyway, also in this game an external menu is used for the general options like music and video settings, and to show the achievements of the player.
There is still a large part of the UIs that has not been treated, i.e. the variable number of interfaces where users come across during the game and that are created for a well-defined in-game function. They respond, normally, to the need of a more precise interaction with an in-game element, in comparison with the normal game mechanism, and they constitute of an element that is presented in a “zoomed” way to the player, like the interface used for the lockpicking minigame in the already cited TESV: Skyrim. For this kind of UIs, the mimetic intent is the most frequent custom. This makes no surprise since our kind of UIs is generally by its own nature the direct reproduction of an in-game element that is recreated similarly to the “real” (fictional) element, but in a way that facilitate its usage, whatever function it absolves (i.e. both an active function, like the Skyrim locks to lockpick, and a passive one, like the map on the wall in Pokémon games).
Lastly, when the UI is used to highlight a part of the game world inside the world itself, again the behaviour can be of both types: in TESV: Skyrim, for example, the pointer is an external one, since the PC does not see it, while again in Halo it is internal: showed on the screen of the helmet. I already mentioned to not agree with the distinction (made in [29]) based on four different types of UIs because it does not consider the essence of two of them. In the article, the writer distinguishes between non-diegetic, spatial, meta and diegetic UIs. Notwithstanding the interesting way of operating the distinction, it does not take in consideration that what it is called there “spatial” and “meta” UIs consist basically of different ways of showing the interface to the player, not different essence of the interfaces themselves. Also, naming them “diegetic” and “non-diegetic” means do not treat them, in the case of non-diegetic ones, as elements constituting the game. In addition, they also name the 2D HUD as always external, but we have already seen that this is not true.
Those two ways seem to present no complications in the analysis of UIs and its impact on narrative, but the sensemaking process used to represent them is slightly different, and this difference is in turn impacting on the storytelling: as mentioned, internal-UIs are formed by their own nature with a more iconic intent, that is, they tend to represent in a realistic way the fictional world in all its aspects. Indeed, they represent directly the reality of the player’s character. On the other hand, this is not true for external-UIs, where the interface is made of signs stuck on the “window” through which the player sees the game, and thus not representing anything as a complex set in a realistic way. However, this does not mean that no icons, intended as signs with a mimetic intent, are involved in the creation of external-UIs. The difference is particularly important to enhance the narrative aspects of a game because internal-UIs are making thinner the distance between characters and players, and they are a preferable road to follow, since they help to establish the “suspension of disbelief”. In the case of non-iconic interfaces, the suspension of disbelief has to be considered even more: if for iconic interfaces is almost automatic, as we will see in a moment, for non-iconic ones there should be a precise intent to preserve it with special precautions.
THE SUSPENSION OF DISBELIEF
At this point, it is necessary to introduce in the analysis the
concept of suspension of disbelief. Jurij Lotman defined
what Coleridge in 1817 called «suspension of disbelief» [4]
in his article The place of art among other modeling systems
[
The player must simultaneously remember that he is participating in a conventional (not real) situation (a child knows that the tiger in front of him is a toy and is not afraid of it), and not remember it (when playing, the child considers the toy tiger to be a real one).
When the suspension of disbelief is established, the user
finds himself in the “Magic Circle”: «The Magic Circle» is
«a closed circle, the space it circumscribes is enclosed and
separate from the real world [...] In a very basic sense, the
Magic Circle is where the game takes place» [
This general cognitive mechanism must deal indeed with the problem of having a playable world that is composed of a set of signs that should have a uniform appearance and a meaning that is internally coherent. There are many influences to take into account, since many semiotic resources are concurring in the production of the overall content. If we increase the complexity of the content to be transmitted, more and more sign systems should carry a coherent set of information for the user to understand it in a correct way. This set of information, i.e. the set of signs constituting a video game, has to be coherent both internally and externally. For internal coherence I mean the consistency within the sign set of each sign with the others, while external coherence means the agreement of the set with the general structure and game world.
As for UIs, when we are considering an iconic (internal) UI, the problem should not arise and if it does, then the issue is probably to a higher level, i.e. in the overall construction of the environment of the game (for a more precise analysis, see e.g. [9]). Thus, if in a game set in prehistory there is a futuristic internal-HUD without a motivation, the problem is of the game as a whole. On the contrary, if we are talking about external-UIs and the elements of the interface do not agree with the general atmosphere of the game, then the problem remains at the level of the UI itself and can cause the disruption of the suspension of disbelief, if not properly managed. But this is not always perfectly true, as we are just going to see.
SUSPENSION OF DISBELIEF AND USER INTERFACES A contradiction between theory and practice seems to arise from the case of UIs. The contradiction comes from the fact that UI is always remembering, during the game, the fictionality of the world in which the player is immersed. This memento in theory should break the equilibrium on which is based the suspension of disbelief, thus breaking the magic circle, but actually it does not. This is probably due to the fact that there is a sort of pact between creators and players, for which if you play a game, you have the UI overlying the world, in the form of HUDs, menus, etc. However, this does not mean that a better solution is not to be found, if possible. Indeed, in many video game series it seems there is a general trend that is leading towards a simplification of the HUD, to make its impact lighter on the screen and to better keep the player in the magic circle. If we compare the HUD in Assassin’s Creed (Ubisoft Montreal, 2007; the first game of the series) with the one in Assassin’s Creed: Origins (Ubisoft Montreal, 2017; the last one since now, 10 years later than the former), we notice that the UI is clearly less impacting on the visual interface. The same considerations could be done comparing the already cited GTA San Andreas and the more recent GTA V (developed, again, almost 10 years later): from a clear indication of the mini map, active weapon, health bar, etc. to a minimal mini map with small bars for health, bulletproof vest and skill meter. Thus, not all information is displayed in the always visible HUD, but some of them of minor importance, are placed in a mimetic UI shaped as a screen of the phone of the characters. Also, weapons are showed in another interface, which appears only when needed.
Those two examples, together with many others, are showing
us that the issue of the disruption of the suspension of
disbelief caused by UIs is heard as a problem. However,
there is a certain level of acceptable incoherence in a fictional
product, but it must be remembered the effect of Woloda’s
behaviour, «that took away all the fun of the game» [
USER INTERFACES AND STORYTELLING It has already been introduced the possibility given by UIs to transmit narrative elements of games, but it would be interesting to have a closer look over the phenomenon. What should be noted is that by “narrative aspects” I do not mean only the mere story of games, i.e. not only the fabula, but rather all the elements concurring in the creation of an atmosphere surrounding a text (of whatever form of text we are talking about). The atmosphere of a book may be given by the cover, by images and even by the font chosen, before even reading it. This is what is called here the “atmosphere”: a sort of mood in which is carried the public (user, reader, player, etc.) through the text s/he is accessing. This mood has not to be considered a peripheral part of the storytelling, but rather it is sometimes a crucial aspect for the narrative itself, as proven by probably the most famous English tale of modern times: in The Raven by Edgard Allan Poe, it is clear the importance of the mood in which the character finds himself during the events, but it is not less clear the importance of the feelings of the reader in understanding the story. Forgetting them would not lead to a proper interpretation of the tale. Refusing the importance of the mood of the public when talking about a fictional environment, would mean claiming that a horror game would be exactly the same with no dark, with cute kittens or with happy music in the background.
Since the UIs are part of the visual interfaces, and I believe visual interfaces constitutes without any doubt a crucial part in the creation of some feelings in the mind of the players, then it is clear that also UIs can be very useful in the creation of the mood.
Keeping this in mind, the possibilities given by UIs to enhance the narrative aspects of a video game are countless, and, as every other part of video game development, highly depend on the project they are inserted in. An emblematic example of this has been already mentioned in regard of the title screen of the game The Elder Scrolls V: Skyrim and can be generalized to many title screens of games, in particular for more recent games (again, a consequence of the trend going towards a more mimetic intent of UIs, already treated). For HUDs, the possibilities have already been exemplified, but for other kind of UIs, the given chances may vary a lot from one case to another and from one genre to another. In a point-and-click game, for example, many part of the game are UIs, in the sense that they are used for a well-defined ingame function and direct reproduction of an in-game element, presented in a “zoomed” way. Apart from pointand-click games, examples could be in-game books or documents to read in a separated interface, events seen through a screen of a CCTV camera, etc.
FINAL CONSIDERATIONS AND FUTURE STUDIES Concluding the analysis, it is possible to point out some notes for the creation of a UI for a game (that is a game with a PC, expressed or implicit): after having defined if the interface is an internal or an external one, it is necessary to specify what has been called the “mood” of the game in which the UI is inserted; thirdly, during the process of designation and creation of the UI, the elements constituting it should be shaped in a way that coherently fit with the atmosphere decided for the game.
In general, internal-UIs are to be preferred since they can enhance immersion, they are used to better keep the player in the magic circle, and they give more chances to the storyteller than external ones. However, when an internal interface is not filling in the set or atmosphere of the game, or simply are not suitable for the function it should accomplish, external ones can be used, if designed in a way that does not break the suspension of disbelief. The advantages of enhancing the immersion are proven empirically by a general trend particularly visible in famous video games series, in which the latest games have a lighter presence of UIs compared to the previous titles.
In this brief overview of the impact of UIs in game immersion and storytelling, I opted out an extensive portion of games, that are of particular interest regarding casual gamers: puzzle games and games in which players do not control a player character. In further analysis, it would be interesting to see how UIs behaves in those games, regarding immersion and the suspension of disbelief. Furthermore, notwithstanding the capability of internal-UIs to actively drive the story, it would be interesting to analyse also this possibility regarding external ones.
What has been told since here tells us the impact of UIs on the storytelling of many video games, in respect also to the different kind of UIs is possible to find in those digital products. Apart from this and on the contrary, the storytelling can be very useful when developing the appearance of UIs to give a strong motivation for them, and in particular when talking about internal ones. This has not to be seen as a claiming that storytelling can justify everything, but more that in the development process, the story in its broad sense must be carefully considered, or the player would end up remembering to not be an Orestes, but a Hamlet.
If we were always to judge from reality, games
would be nonsense; but if games were nonsense,
what else would there be left to do? [