=Paper=
{{Paper
|id=Vol-2166/afcai18-paper2
|storemode=property
|title=Affective VR Serious Game For Firefighter Training
|pdfUrl=https://ceur-ws.org/Vol-2166/afcai18-paper2.pdf
|volume=Vol-2166
|authors=Jan K. Argasiński,Paweł Węgrzyn,Paweł Strojny
|dblpUrl=https://dblp.org/rec/conf/afcai/ArgasinskiWS18
}}
==Affective VR Serious Game For Firefighter Training==
https://ceur-ws.org/Vol-2166/afcai18-paper2.pdf
Affective VR serious game for firefighter training
Jan K. Argasiński1 , Paweł Węgrzyn1 , and Paweł Strojny2,3
1
Department of Games Technology,
Faculty of Physics, Astronomy and Applied Computer Science,
Jagiellonian University in Krakow, Poland
2
Institute of Applied Psychology,
Faculty of Management and Social Communication,
Jagiellonian University in Krakow, Poland
3
Nano Games sp. z o.o.
Abstract. The article discuss the use of design patterns for serious
games with affective feedback in order to uphold immersion into Virtual
Reality, and to influence motivation and effort put into training. The ex-
ample of training simulation in Virtual Reality addressed to firefighters
is described. We consider two different conceptual and theoretical frame-
works, perhaps even two contradictory concepts, namely Motivational
Intensity Theory and Embodied Cognitive Science. Virtual Reality game
prototype created with the participation of firefighters by Nano Games
sp. z o.o. is presented.
Keywords: Affective Computing, Virtual Reality, Serious Games, Pat-
terns in Game Design
1 Motivation
1.1 Affective feedback in firefighter VR training
Firefighters represent specific occupational group of highly motivated specialists,
whose daily duty consists in protecting human property and life. What follows,
it would be difficult to imagine any kind of an artificial training that would per-
manently increase their motivation, there may be simply no chance for such kind
of influence. There is a hidden trap in the craft of screen-writing and designing
for VR firefighter simulators. Firefighters must be ready for various situations,
including hard to predict extremely rare events (black swans). This is the reason
why their job consists of numerous hours of skill training. And here comes the
trap, they may be highly motivated but their engagement in training may be
significantly reduced because they usually struggle with more serious challenges
on a daily basis.
1.2 The research problem
Serious games and simulations intended for firefighters training appear to focus
predominantly on either target-based practices with some professional equipment
or on the logistics of rescue operations. Many training systems offer the possi-
bility of including modules that could allow to simulate high stress situations.
Our literature review [1,2,3,4,5,6,7,8,9] shows that there are no ready solutions
for developing in a systematic way complex affective scenarios and designing
simulation games based on controlling emotional feedback.
The proposal here is to include the paradigm of affective computing in the
area of simulations and training games. The idea is to adapt design patterns
in game design [12,19]. The main research problem is to formulate appropriate
theoretical framework allowing to capture the problem of engagement, motiva-
tion and impact of affects in a manner applicable in VR simulations and serious
games through patterns.
2 Materials & Methods
The Design Patterns (here: patterns in game design) are reusable solutions that
should evade low-level details of large software applications and allow us to
express their architecture and complexity and discuss relations, processes and
other issues at a higher level of abstraction. Our aim is to attain right design
patterns to make the firefighter VR training highly immersive. It refers to the
motivation of the trainee and his effort put into training. We are aware of two
different theoretical frameworks of theories and concepts which we can apply to
the problem. In this paper, we are submitting to Motivational Intensity Theory
(MIT) and Embodied Cognitive Science (ECS). In both of these cases, our task
is to define design patterns that make the conceptual framework and theoretical
background apparent in our context.
2.1 Motivational Intensity Theory
Effort may be defined as an investment of resources enabling execution of be-
havior. Terms “motivation” and “effort” should not be confused. Psychologists
define motivation as a complex cognitive structure. According to one of the most
comprehensive theory of motivation, Motivation Intensity Theory [15], motiva-
tion should be understood in terms of individual cognitions regarding need and
possibility of achieving a goal. In other words, one’s motivation (i.e. potential
motivation) depends on three antecedents: need, incentive value and probability
that desired outcomes will be attained once instrumental behavior will be car-
ried out [16]. Motivation may be extremely high but remain hidden in terms of
effort manifesting itself in observable behavior. It is not a surprise that maxi-
mum effort is limited by (potential) motivation. What should be stressed is the
fact that no motivation but the task’s requirements directly determine the effort
level actually invested in the task – there is simply no need of investing resources
exceeding the level required by a task. The research problem concerns the an-
swer to the question how to design a training to encourage trainees to high effort
without involving means absent in natural environment (e.g. exam stress).
There are three major categories of effort indicators used in psychological re-
search: self-report based effort indices (e.g. NASA Task Load Index), behavioral
indicators (e.g. perseveration) and physiological correlates (e.g. Pre-Ejection Pe-
riod, PEP). All of them may be used separately but since each of those indicators
has flaws, combining more than one may provide deeper and more reliable re-
sults. It is the reason why the proposed affective system is going to use two
relatively independent physiological indicators of effort [16]: Heart Rate Vari-
ability (HRV) and Electrodermal Activity (EDA), and additionally self-report
data that are going to be used as a source of supplementary data in the initial
stages.
These two are considered as indicators of auto-
nomic nervous system (ANS) activation. ANS func-
tions mostly without the consciousness of the per-
son and regulates the most important activities of
the body, such as heart rate, blood supply and di-
gestion [17]. Its role is to regulate the functions im-
portant for the survival of the organism, depend-
ing on circumstances one of its two branches pre-
vail (sympathetic and parasympathetic, some aca-
demics indicate the third branch – enteric). Both
branches innervate the same organs but (except spe-
cific situations such as sexual arousal) act in the
opposite way – sympathetic activation is related to
quick mobilization of the system while parasympa-
thetic activation is related to dampening which is
also less dynamic. Electrodermal Activity (EDA) is
considered as one of reliable indicators of sympa- Fig. 1. Biosensors & VR
thetic activation while high frequency Heart-Rate simulation by Nano Games
Variability (nHF-HRV) indicates parasympathetic
branch activation. Combining these two indicators will make continuous assess-
ment of subject’s affective state plausible which leads to possibility of modulating
this state (e.g. effort) with affective computing techniques (test setup on Fig.1).
Motivation intensity theory [15,16], described briefly above, may be utilized
as a conceptual framework and theoretical background for the tool. Taking this
point of view, we assume that a firefighter during real rescue action acts with high
level of motivation which leads to potentially extreme effort moderated only by
task requirements. In other words one would expect that the firefighter would do
his or her best if task is extremely difficult because his or her motivation is high
due to the importance of the task. On the other hand, proximal consequences
of traditional training tasks are far less profound, it may result in significantly
lower motivation which may lead to faster giving up in the face of difficulties
which, in turn, may result in poor preparation for the most difficult tasks. We
can believe that by combining highly immersive and realistic environments with
adaptive difficulty level based on affective computing the training system that
can encourage increased effort could be developed.
2.2 Embodied Cognitive Science
Embodied Cognitive Science is an alternative theory to cognition in which it
denies that the human mind or brain is an information input/output processing
system and that thinking is a form of computing. The ECS theory considers the
mind and body as a single entity, and an entire organism’s body determines how
and what a human thinks. Following this point of view, emotions are also embod-
ied. According to Prinz [11], emotions are perceptions (conscious or unconscious)
of patterned changes in the body.
The ECS framework is the most popular within the field of Affective Com-
puting. In her paradigm establishing book Affective Computing, R. Picard states
that emotions are both physical and cognitive [10]. This means that if we want to
collect and process data on affects, we can rely on information obtained from the
body of the surveyed subjects. We do not have to rely solely on the psychological
aspects. According to old James and Lange theories, emotions are primarily (if
not exclusively) reactions to the changes in physiology under the influence of in-
ternal or external stimuli [11]. They can be either conscious or not. In the case of
embodied emotions, the direction of the process proceeds from the physiological
reaction to a possible psychological response. The only problem is the choice of
biophysical information acquisition channels (sensors) and the question of the
minimum necessary accuracy of the measurement.
The term "affect" usually takes on a more complex meaning. Damasio [18] de-
fines it as the collection of processes that includes drives, motivations, emotions,
and feelings. Feelings (that are conscious and valenced here) are motivators,
monitors and negotiators of the cultural process. Affects are powered by homeo-
statis, that is a regulator of human organism processes such that human life can
benefits. Therefore, feelings can be also interpreted as the mental expression of
ongoing homeostatic states in a human organism. Emotive responses to sensory
stimuli (emotional feelings) are some action programs superposed on ongoing
physiological regulations provoked by external conditions. Feelings are natural
reports on the momentary state of life within the human organism. They open
a way to learn about all processes within the organism.
Effective acquisition and processing of information on physiological condi-
tions (implying emotional states) allows for their use in order to create a spe-
cific kind of feedback. Detection of emotions (or feelings) can allow for dynamic
strengthening or weakening of further affects. In the experience of affective loop,
as Höök claims: "(i) emotions are seen as processes, constructed in the inter-
action, starting from everyday bodily, cognitive or social experiences; (ii) the
system responds in ways that pull the user into the interaction, touching upon
end users’ physical experiences; and (iii) throughout the interaction the user is
an active, meaning-making individual choosing how to express themselves — the
interpretation responsibility does not lie with the system" [14].
Both computer games and simulations based on mechanics (in sense of [13])
seem to be on the one hand suitable systems allowing for increasing motivation
(and thus affecting the effort), on the other hand - it is very easy to implement
the mechanisms of affective feedback in them. One of the approaches to the
problem of designing games that allow for stable and reproducible results in the
field of gameplay is the use of a special type of design patterns for creating games
(see: [12]). In our approach [19], we suggest using the modified proposal of Björk
& Holopainen, enriched, among other things, with emotional correlates.
3 Current state of research and development
3.1 Game prototype
The Nano Games sp. z o.o. company has created a
prototype of the game intended for training fire-
fighters. The game allows for simulation various
variants of traffic accidents with different types of
victims. Thanks to Virtual Reality technology fire-
fighters wearing protective suits and equipped with
real gear (see Fig. 2) can practice procedures associ-
Fig. 2. VR simulator by ated with providing appropriate help at the accident
Nano Games in testing site. Initial research is to check the real usefulness
of VR simulations in the training and evaluation of
firefighters. Different ways of collecting physiological data for the development of
the affective feedback mechanisms and patterns are being tested and evaluated.
The game is created in close cooperation with local fire training centers and
is tested with the participation of real firefighting units.
3.2 Affective Patterns in Serious Games
Based on the above premises and preliminary results, the new gameplay archi-
tecture is under development for further testing and trial implementation. It is
based on the concept of design patterns in game design.
One of the basic elements of the game are mechanics - it is through them,
beside the narrative and aesthetic design, that the creator has a direct impact
on the player’s experience. They are also the most special element of designing
game software. The rules of a game are translated into mechanics. They can be
expressed using verbs that describe what an agent can do and what impacts it is
subjected to. For instance, the player can "collect" the object, "open" the door,
"solve" the problem. Mechanics are repetitive and their character is strongly
associated with game genre special features. In their work, Björk and Holopainen
[12] proposed a semi-formal description methods for patterns in game design. Our
proposal is to identify some patterns as affective (e.g., "time pressure" as a stress
inducing factor) and predict their impact.
3.3 Measurement of body signals
For preliminary tests BIOPAC Bionomadix system of wearable wireless devices
has been used with AcqKnowledge software. It includes ECG, EDA and EMG
sensors.
Due to the fact that users must be burdened with devices as little as possible
- for further works on the prototype of the affective simulation game in VR it is
planned to use the Bitalino platform (equipped with the ECG, EDA and EMG
sensors). In this case, main advantages are: extremely compact size, modularity,
numerous APIs and low price.
4 Future work
In the near future, the following tasks are scheduled:
– confront two conceptual and theoretical frameworks MIT and ECS in the
light of suitability for the needs of designing VR firefighter simulators,
– prepare a design of architecture for appropriate solutions (rule systems and
context aware systems for MIT and affective design patterns for feedback
loops in ECS),
– create hardware and software to measure and visualize biosignals based on
Bitalino,
– prepare advanced affective game design patterns that allow to control en-
gagement/effort and some selected emotional states,
– build an affective plug-in that will automatically and in real time respond
to changes in biophysiological states (affective feedback loop),
– build a tool for level designers and firefighter trainers to create new environ-
ments and scenarios,
– conduct numerous tests with real firefighters units.
References
1. L. Jaikyung, Ch. Moohyun, Ch. Byungil, K. Taesung. A Team-based Firefighter
Training Platform Using the Virtual Environment. in: Proceedings - VRCAI, 2010
2. D.L. Tate, L. Sibert, T. King. Virtual Environments for Shipboard Firefighting
Training. in: Virtual Reality Annual International Symposium, 1997
3. P. Backlund, H. Engstrom, C. Hammar, M. Johannesson, M. Lebram. Sidh – a
Game Based Firefighter Training Simulation. in: Information Visualization, 2007
4. J. Bliss, P.D. Tidwell, M.A. Guest. The Effectiveness of Virtual Reality for Ad-
ministering Spatial NAvigation Training to Firefighters. in: Human Factors and
Ergonomics Society Annual Meeting Proceedings 6(1):73-86, 1997
5. Ch. Moohyun, H. Soonhung, L. Jaikyung, Ch. Byungil. A virtual reality based fire
training simulator integrated with fire dynamics data. in: Fire Safety Journal, Vol.
50, 2012
6. J.M. Houtkamp, A. Toet, , F.A. Bos. Task-Relevant Sound and User Experience in
Computer-Mediated Firefighter Training. in: Simulation & Gaming, Vol 43, Issue
6, 2012
7. N. Keren, Sh.T. Bayouth, W.D. Franke, K.M. Godby. Examining the effect of level
of stress on firefighters’ time-to-decision in virtual reality. in: Proceedings of the
Human Factors and Ergonomics Society Annual Meeting, 2013
8. D.L. Tate, L. Sibert, T. King. Using Virtual Environments to Train Firefighters.
in: IEEE Computer Graphics and Applications, 1997
9. Z. Xu, Z. Lu, H. Guan, C. Chen, A.Z. Ren. A virtual reality based fire training
simulator with smoke hazard assessment capacity. in: Advances in Engineering
Software, Vol. 68, 2014
10. R. Picard. Affective Computing. MIT Press, 1997
11. J. Prinz. Gut Reactions. A Perceptual Theory of Emotion. Oxford University
Press, 2006
12. S. Björk, J. Holopainen. Patterns in Game Design. Charles River Media, 2005
13. M. Sicart. Defining Game Mechanics in: Game Studies 8, 2008
14. K. Höök. Affective loop experiences: designing for interactional embodiment. in:
Philos. Trans. R. Soc. Lond. B. Biol. Sci. 364(1535): 3585-3595, 2009
15. J.W. Brehm, E.A. Self. The Intensity of Motivation. in: Ann. Rev. Psychol, 40:109–
31, 1989.
16. R.A. Wright, L.D. Kirby. Effort determination of cardiovascular response: An
integrative analysis with applications in social psychology. in: Adv. in Exp. Soc.
Psych., 33:255-307, 2001.
17. R.F. Schmidt, G. Thews (eds.). Human Physiology. Springer, 1989.
18. A. Damasio. The Strange Order of Things: Life, Feeling, and the Making of Cul-
tures. Pantheon, 2018.
19. J.K. Argasiński, P. Węgrzyn. Affective Patterns in Serious Games. to be published,
2018.
Project "Widespread Disaster Simulator - research and preparation for im-
plementation", realized by Nano Games is co-financed under the Smart Growth
Operational Programme, sub-measure 1.1.1. Industrial research and develop-
ment work implemented by enterprises.