=Paper=
{{Paper
|id=Vol-2092/paper17
|storemode=property
|title=Interactive Gamified Virtual Reality Training for Affine Transformations
|pdfUrl=https://ceur-ws.org/Vol-2092/paper17.pdf
|volume=Vol-2092
|authors=Sebastian Oberdörfer,David Heidrich,Marc Erich Latoschik
|dblpUrl=https://dblp.org/rec/conf/delfi/OberdorferHL17
}}
==Interactive Gamified Virtual Reality Training for Affine Transformations==
https://ceur-ws.org/Vol-2092/paper17.pdf
Carsten Ullrich, Martin Wessner (Eds.): Proceedings of DeLFI and GMW Workshops 2017
Chemnitz, Germany, September 5, 2017
Interactive Gamified Virtual Reality Training of Affine
Transformations
Sebastian Oberdörfer,1 David Heidrich,1 Marc Erich Latoschik1
Abstract: Affine transformations which are used in many engineering areas often escape an intuitive
approach due to their high level of complexity and abstractness. Learners not only need to understand
the basic rules of matrix algebra but are also challenged to understand how the theoretically grounded
aspects result in object transformations. Therefore, we developed the Gamified Training Environment
for Affine Transformation that directly encodes this abstract learning content in its game mechanics.
By intuitively presenting and demanding the application of affine transformations in a virtual gamified
training environment, learners train the application of the knowledge due to repetition while receiving
immediate and highly immersive visual feedback about the outcomes of their inputs. Also, by providing
a flow-inducing gameplay, users are highly motivated to practice their knowledge thus experiencing a
higher learning quality. As the immersion, presence and spatial knowledge presentation can have a
positive effect on the training outcome, GEtiT explores the effectivity of different visual immersion
levels by providing a desktop and a VR version. This article presents our approach of directly encoding
the abstract learning content in game mechanics, describes the conceptual design as well as technical
implementation and discusses the design differences between the two GEtiT versions.
Keywords: Gamification; Virtual Reality; Education; Knowledge Training
1 Introduction
In-depth understanding of affine transformation (AT) is critical for many engineering areas
including robotics, 3D computer graphics, or Virtual and Augmented Reality (AR, VR).
However, due to the complexity of the learning content, e.g., ATs for operations in R3 are
commonly expressed as 4 × 4 matrices, developing an in-depth understanding often escapes
an intuitive approach as students are challenged to learn how the theoretically grounded
mathematical aspects achieve a transformation of an object thus resulting in a high degree
of frustration. Furthermore, ATs are order dependent and hence different sequences of the
same transformation operations can result in different outcomes. Finally, students have to
understand the basic rules of matrix algebra as mappings between affine spaces are executed
via matrix multiplications.
1 University of Würzburg, Human-Computer Interaction, Am Hubland, 97074 Würzburg, Germany,
sebastian.oberdoerfer@uni-wuerzburg.de, david.heidrich@stud-mail.uni-wuerzburg.de,
marc.latoschik@uni-wuerzburg.de
cbe
Sebastian Oberdörfer, David Heidrich, Marc Erich Latoschik
Fig. 1: GEtiT intuitively demonstrates the effects of the AT and challenges learners to apply their
knowledge to solve puzzle exercises.
Therefore, we developed a virtual gamified training environment–the Gamified Training En-
vironment for Affine Transformation (GEtiT)2–to intuitively train and master the application
of ATs, i.e., application of transformations allowing for object translation, rotation, scaling,
reflection, and shearing [OL16]. In order to do so, we developed a model to directly encode
the AT knowledge in game mechanics that periodically demand the knowledge’s application,
provide visual feedback about the correctness and hence lead to a knowledge training
due to repetition (see Figure 1). Also, as a higher visual immersion can lead to a higher
presence and performance [Sl96] in the case of a virtual training simulation [SK15], we
developed a specific VR version to potentially increase GEtiT’s training effects by achieving
a higher presence as well as a higher and more intuitive spatial knowledge presentation. For
this purpose, GEtiT-VR implements the same core game mechanics but utilizes a higher
immersive visual presentation to allow for a direct experience of an AT operation’s effects.
Ultimately, GEtiT is intended to demonstrate the effectivity of our knowledge encoding
model which we currently prepare for publication.
This paper describes the conceptual design and technical implementation of both GEtiT
versions which are based on our direct gamified knowledge encoding model, discusses their
differences and presents our expectations towards their individual training effects. The paper
begins with a brief description of our model we developed in order to directly encode the
AT knowledge in GEtiT using game mechanics. Subsequently, we present the concept as
well as technical implementation of the two GEtiT versions. Finally, this paper discusses
2 http://www.hci.uni-wuerzburg.de/projects/getit.html
Interactive Gamifed Virtual Reality Training of Afne Transformations
the similarities and differences between GEtiT’s desktop as well as the VR version and
concludes with our expectations towards the individual training effects.
2 Direct Knowledge Encoding
Game mechanics are the rules of a computer game that define what is possible, create the
virtual game world [AD12] and allow players to interact [Si08] with the game. Furthermore,
utilizing game mechanics demands and hence trains a specific set of human skills [OL13].
By periodically executing game mechanics, players train the encoded knowledge due to
repetition [Ge07]. Learning due to repetition, or practicing, is a very important aspect of
learning new knowledge as it helps learners to achieve an automatization or deepening of
the learning content which facilitates a knowledge transfer to a different domain or unknown
problem [Br00, Me04]. Hence, game mechanics have the potential to directly encode even
abstract knowledge as their rules thus creating intuitive training environments for complex
learning contents. That way, the game mechanics create a gamification metaphor for the
learning content that acts as a learning affordance [DL10, KN12]. Learning affordances
achieve a periodic knowledge training by demanding the application of the encoded
knowledge and informing about the underlying principles.
Moreover, GEtiT moderates the knowledge’s level of abstractness to facilitate the training
process by intuitively presenting and demanding the learning content. For this purpose,
the game mechanics were designed to scale in the complexity and level of abstractness
thus resulting in a gradual increase in the learning content’s as well as game’s difficulty.
Also, the gradual difficulty increase in combination with an immediate feedback as well as
a constant stream of new challenges creates the potential for game flow [Cs10, Mc11] that
keeps learners motivated and engaged.
3 Conceptual Design
Aside from encoding the AT knowledge rules in its game mechanics, GEtiT, in order to
ensure an effective AT training, needs to fulfill three additional requirements: (1) The
moderation of the level of abstractness requires a tailored gamification metaphor that scales
the learning content’s complexity. (2) A clear, intuitive and immediate feedback has to
be provided to allow learners to analyze and visualize the effects of an AT operation and
to learn from their potential mistakes. (3) Finally, a well-defined game goal is needed to
challenge and motivate the learners to apply their AT knowledge.
As, in case of 3D computer graphics and VR/AR applications, the AT is used to transform
and display objects, we adopted a frequently used manipulable object game mechanic in
order to achieve similarities between the gamified training environment and a potential
real world application of the learning content to facilitate a knowledge transfer process
Sebastian Oberdörfer, David Heidrich, Marc Erich Latoschik
Fig. 2: The symbols displayed on the AT cards indicate the transformation type.
[DL10, OP13]. However, instead of allowing for a direct manipulation which is commonly
used in many computer games, GEtiT requires the application of an AT operation to
manipulate, or transform, the object. That way, as the object immediately gets transformed
based on the learners’ inputs (see Figure 1), the game mechanic provides them with a visual
feedback about the effects and correctness of their chosen approaches. For the purpose
of enhancing the feedback, the object also casts a trail each time it gets transformed thus
visualizing the effects of an individual AT and helping the learners to intuitively develop a
spatial understanding for the AT knowledge. Furthermore, as the object internally stores
its status, the object game mechanic also is used to provide the players with a clear goal.
Each training exercise challenges them to transform the object in such a way that it matches
specific victory conditions which are displayed in form of a half-transparent object ultimately
representing the players’ goal.
Fig. 3: On hard difficulty, learners see the matrix representation for the first time, but are only
required to enter the matrix elements relevant for the chosen transformation type using the direct value
configuration screen.
Interactive Gamifed Virtual Reality Training of Afne Transformations
For the purpose of using AT operations as game inputs and achieving a moderation of
the level of abstractness, we developed a special UI that provides users with access to
AT operations and simultaneously informs them about the object’s, target’s and player’s
position. The AT operations are represented by AT cards of which each represents an
individual mathematical operation. The AT cards display a symbol indicating the AT type
(see Figure 2) and a symbolized vector or matrix representation showing predefined and
undefined elements. Hence, the AT cards scale the complexity of the learning content thus
achieving a moderation of the level of abstractness. GEtiT features four difficulty levels
that gradually increase the learning content’s abstractness. On easy difficulty, the gamified
training environment only provides predefined vector AT cards that, upon activation,
automatically perform the displayed transformation, and, as a result of this, learners merely
need to select the correct cards to solve a level. The remaining three difficulty levels feature
undefined AT cards that open a direct value configuration screen (see Figure 3) on activation
allowing for the use of self-obtained computational results as inputs to the game. On medium
difficulty, GEtiT still utilizes the vector representation but challenges the users to enter the
vector elements. Once students move on to hard difficulty, GEtiT starts to use the 4 × 4 matrix
representation but only requires the learners to configure those of the matrix elements that
are relevant for the AT type displayed on the selected card. Finally, on expert difficulty, the
moderation of the level of abstractness is scaled back completely as learners are challenged
with a full transformation matrix demanding them to enter every element. At this point,
the expert difficulty simulates the AT knowledge as it implements the complete set of AT
knowledge rules that are directly encoded in the gamification metaphor.
Fig. 4: After having matched the victory conditions with the object, a portal gets activated and allows
players to exit the level.
The gamified training exercises are created by the level design, a selection of available AT
Sebastian Oberdörfer, David Heidrich, Marc Erich Latoschik
cards, and the level-specific victory conditions. The level design determines the object’s
initial position, the origin’s position and the position of potential obstacles that can block
the object thus adding another challenge to the gameplay as players are required to translate
the object around them. Also, in order to give the puzzle exercises an important meaning
[Mc11], they were embedded in an escape scenario being inspired by the gameplay of
Portal3 which puts players in sealed rooms and challenges them to open the levels’ exits by
solving spatial puzzles. Similar to Portal, each of GEtiT’s levels represents a sealed room
players have to escape from by opening the level’s exit–a portal (see Figure 4)–and walking
through it. This, however, can only be done by solving spatial puzzles, i.e. transforming the
object in such a way that it matches a level’s victory conditions which subsequently opens
the exit thus allowing the player to proceed to the next level (see Table 1). In addition, some
levels challenge the learners to use the object as a stepping stone in order to reach the top of
an obstacle or to cross a bottomless gap. As a result of this, not only the gameplay but also
the AT knowledge itself becomes meaningful to the players as GEtiT turns it into a tool that
allows them to exhaust the challenges.
Tab. 1: Overview of GEtiT’s gameplay
Step Task Game mechanics
1 Enter a level -
2 Analyze the level’s spatial puzzle Level design
Object start position
Victory conditions
3 Transform the object to match victory AT cards
conditions using the available AT cards Manipulable object
Victory conditions
4 Leave the level Portal
Finally, GEtiT implements additional game mechanics to keep the learners engaged and
to avoid breaking the immersion. On the one hand, the gamified training environment
challenges players with the indication of the minimum of cards that are needed to solve a
particular level. Solving a level with the minimum or a small deviation from the minimum
rewards players with points that represent their progression towards the completion of the
game. That way, learners simultaneously receive feedback about their efficiency applying
their AT knowledge and are challenged to retry a level when they exceeded the minimum.
Moreover, users can unlock achievements for efficiently solving a level, completing the
game or finding a special easter-egg hidden in one of the levels. On the other hand, GEtiT
also provides a small built-in wiki that summarizes the AT knowledge for the purpose of
keeping players immersed as they can look up the theoretically grounded aspects directly
inside of the game.
3 http://www.thinkwithportals.com
Interactive Gamifed Virtual Reality Training of Afne Transformations
4 Technical Implementation
GEtiT was developed in Unity 3D4 for PC and Mac to make the game available for most
systems used by students as well as in classrooms without requiring additional powerful
hardware. In addition, using Unity 3D facilitates the implementation of further game
mechanics and other improvements of the game. This decision also made it easy to develop
a VR-version as Unity 3D provides a good support for current VR devices of which we
chose the HTC Vive5 as it offers room-scale VR and hence a potentially higher presence.
However, in order to play GEtiT-VR, a more powerful computer setup as well as the VR
device are needed.
Fig. 5: Playing GEtiT-VR in the lab.
5 Similarities and Differences
GEtiT-VR provides a similar gameplay to GEtiT’s desktop version as it utilizes the same
gamification metaphor, additional game mechanics as well as training exercises, but, instead
of being played on a desktop computer setup, it is played using the HTC Vive to achieve a
higher level of visual immersion, presence and spatial knowledge presentation. However,
despite implementing the same game mechanics, the VR port required some UI as well as
interaction adjustments to ensure a good usability as well as a believable and immersive
environment.
4 https://unity3d.com
5 https://www.vive.com
Sebastian Oberdörfer, David Heidrich, Marc Erich Latoschik
5.1 Game Controls
The interaction adjustments [un17a] were required to implement the HTC Vive controllers
as the input devices used to interact with the GEtiT-VR and to successfully utilize the HTC
Vive’s room-scale function. For this purpose, all movement controls were mapped to the
system’s tracking function that tracks the position of the HTC Vive Head-Mounted Display
(HMD) thus allowing users to look and walk around (see Figure 5) as long as they stay
within the boundaries of the tracking area. However, as the levels are larger than the tracking
area, the gamified training environment also provides the option to teleport within a level by
pressing the trackpad on one of the controllers and subsequently selecting a new location by
pointing at it with a target selection marker (see Figure 6). On release of the trackpad, the
player is teleported to the selected location inside of the level thus providing the option to
move over larger distances, to get on top of the object and to enter the portal to escape a
room.
Fig. 6: Players can teleport in GEtiT-VR to cross larger distances, to get on top of the object and to
enter the portal.
In addition to the teleport feature, the controllers are used to allow a user to select one
of the AT cards, to configure a card’s elements and, finally, to activate a card to apply a
transformation to the object inside of GEtiT-VR. In order to select and grab a card, a player
merely has to touch the desired card–they are placed on a floating console (see Figure 7)
to avoid static UI elements–with one of the controllers. Afterwards, the selected card is
attached to the player’s controller and can be played by pulling the controller’s trigger button,
configured by using the controller’s trackpad or placed again on the console by touching it
Interactive Gamifed Virtual Reality Training of Afne Transformations
Fig. 7: All available AT cards are placed on a floating console in GEtiT-VR.
with the controller that holds the card. In contrast to the desktop version, GEtiT-VR provides
no direct value configuration screen in order to allow for a change of a card’s transformation
values. Instead, by using the trackpad, a user can select the element to be changed and
subsequently use an input matrix that is shown on the opposite controller (see Figure 8)
to enter the desired value. For this purpose, the opposite controller itself is used for the
selection and the confirmation of the values. That way, GEtiT-VR can intuitively be played
independent of the player’s handedness.
5.2 User Interface
As a static UI often breaks the immersion of a VR application, GEtiT’s UI got adapted to
fulfill the technical requirements for a good VR interface [un17b]. Instead of using a fixed
bar in the UI displaying the available AT cards, GEtiT-VR follows the idea of a spatial UI
and implements a floating console to provide access to them. The console can be grabbed
and moved around using one of the controllers to allow players to place the console at a
spot from where they can simultaneously see the available cards as well as the object thus
facilitating the process of selecting the correct card. The cards itself also received a physical
property and hence can be carried around. This decision was mainly made to make the
virtual environment more believable and immersive.
Sebastian Oberdörfer, David Heidrich, Marc Erich Latoschik
Fig. 8: GEtiT-VR allows for a direct value input via a special input matrix UI.
Additionally, instead of tying the indication of the object’s and target’s position to the
player’s view, both game mechanics, following again the idea of a spatial UI, received a label
that displays the position information. The labels, despite being attached to their relevant
object, have no fixed position or orientation. Instead they always face to the player, and, in
case of one of the labels is relative to the player behind one of the level’s obstacles, the label
starts to shine through the obstacle thus ensuring a good visibility from any position inside
of a particular level.
5.3 Walk-In Game Menu
The final challenge of the VR port was to avoid breaking the immersion when a player
accesses one of the game menus, such as the level selection screen, the game options, and
the wiki. This challenge was solved by turning the individual menus of the desktop version
into control consoles that are placed inside of the player’s futuristic playing room that
provides a connection between GEtiT-VR’s training exercises and the real world. Aside
from the control consoles and relevant displays for the various menus, the playing room also
features a fictive game console that loads and previews the available training levels–they are
provided in form of cubes in a shelf–by placing one of the level cubes on top of it. In order
to play a loaded level, a player has to wear a virtual in-game HMD that is connected to the
game console (see Figure 9) by grabbing it with one of the controllers and putting it on with
Interactive Gamifed Virtual Reality Training of Afne Transformations
Fig. 9: GEtiT-VR implements the functionality of VR Head-Mounted Display devices to allow for a
transition between menus and the normal gameplay.
a similar gesture one would perform to wear normal glasses. Similarly, a user can return to
the playing room from one of the training exercise levels by simply taking off the virtual
HMD. That way, GEtiT-VR implements the VR technology itself as a method to transition
between the game and the menus in a believable and immersive way.
6 Conclusion
We described the conceptual design and technical implementation of GEtiT which is used as
a demonstrator for our model describing a direct knowledge encoding using game mechanics.
The virtual gamified training environment achieves an intuitive presentation and demand
of the abstract AT knowledge by moderating the learning content’s level of abstractness.
GEtiT comes in two versions which are distinguished by their level of visual immersion
and is, to our knowledge, the first gamified training environment that is based on a general
knowledge encoding model.
GEtiT’s desktop version was already used in three AT training modules associated with an
interactive computer graphics lecture and achieved a training outcome equal to a traditional
paper-based training method. In addition, GEtiT achieved a significantly higher intuitive
training and a higher enjoyment of use in general. The results of the completed studies are
currently in preparation for publication. Therefore, we expect GEtiT-VR, due to a higher
Sebastian Oberdörfer, David Heidrich, Marc Erich Latoschik
visual immersion, presence and spatial knowledge presentation, to yield a similar or even
better training outcome in a forthcoming evaluation that compares the efficiency of the two
different versions.
References
[AD12] Adams, Ernest; Dormans, Joris: Game Mechanics. Advanced Game Design. New Riders,
Berkeley, 2012.
[Br00] Brophy, Jere: Teaching, volume 1 of Educational Practices Series. International Academy of
Education & International Bureau of Education, Brussels, 2000.
[Cs10] Csikszentmihalyi, Mihaly: Flow : Das Geheimnis des Glücks. Klett-Cotta, Stuttgart, 15
edition, 2010.
[DL10] Dalgarno, Barney; Lee, Mark J. W.: What are the learning affordances of 3-D virtual
environments? British Journal of Educational Technology, 41(1):10–32, 2010.
[Ge07] Gee, James Paul: What video games have to teach us about learning and literacy. Palgrave
Macmillan, New York, 1 edition, 2007.
[KN12] Kaptelinin, Victor; Nardi, Bonnie A: Affordances in HCI: Toward a Mediated Action
Perspective. CHI 2012, pp. 967–976, 2012.
[Mc11] McGonigal, Jane: Reality is broken : why games make us better and how they can change
the world. Penguin Press, New York, 1 edition, 2011.
[Me04] Meyer, Hilbert: Was ist guter Unterricht? Cornelsen, 2004.
[OL13] Oberdörfer, Sebastian; Latoschik, Marc Erich: Develop your strengths by gaming. Informatik
2013 - Proceedings of 43rd annual German conference on informatics, P-220:2346–2357,
2013.
[OL16] Oberdörfer, Sebastian; Latoschik, Marc Erich: Interactive gamified 3D-training of affine
transformations. In: Proceedings of the 22nd ACM Conference on Virtual Reality Software
and Technology (VRST ’16). ACM Press, Garching, Germany, pp. 343–344, 2016.
[OP13] Oei, Adam C; Patterson, Michael D: Enhancing Cognition with Video Games: A Multiple
Game Training Study. PLoS ONE, 8(3):e58546, February 2013.
[Si08] Sicart, Miguel: Defining game mechanics. Game Studies, 8(2), 2008.
[SK15] Stevens, Jonathan A.; Kincaid, J. Peter: The Relationship between Presence and Performance
in Virtual Simulation Training. Open Journal of Modelling and Simulation, 3:41–48, 2015.
[Sl96] Slater, Mel; Linakis, Vasilis; Usoh, Martin; Kooper, Rob: Immersion, Presence, and Per-
formance in Virtual Environments: An Experiment with Tri-Dimensional Chess. In: ACM
Virtual Reality Software and Technology (VRST). pp. 163–172, 1996.
[un17a] unity3d.com: , Interaction in VR. https://unity3d.com/learn/tutorials/topics/virtual-
reality/interaction-vr, last checked 31.05.2017.
[un17b] unity3d.com: , User Interfaces for VR. https://unity3d.com/learn/tutorials/topics/virtual-
reality/user-interfaces-vr, last checked 31.05.2017.