=Paper=
{{Paper
|id=Vol-2359/paper20
|storemode=property
|title=Badges or a leaderboard? How to gamify an augmented reality warehouse setting
|pdfUrl=https://ceur-ws.org/Vol-2359/paper20.pdf
|volume=Vol-2359
|authors=Paula Bräuer,Athanasios Mazarakis
|dblpUrl=https://dblp.org/rec/conf/gamifin/BrauerM19
}}
==Badges or a leaderboard? How to gamify an augmented reality warehouse setting==
https://ceur-ws.org/Vol-2359/paper20.pdf
Badges or a leaderboard?
How to gamify an augmented reality warehouse setting
Paula Bräuer1 and Athanasios Mazarakis1
1
Kiel University, Kiel 24118, Germany
p.braeuer@zbw.eu, a.mazarakis@zbw.eu
Abstract. In gamification research, most game design elements are currently
being tested simultaneously. To gain more detailed knowledge about the effect
of game design elements in non-game contexts, the impact of individual ele-
ments should be considered more closely. Also, the interactive experience of
using augmented reality in real-world environment is being explored increas-
ingly in a range of commercial environments. This article is about an experi-
mental study (N=80) which investigates the effects of two game design ele-
ments (badges and leaderboards) in an augmented reality supported warehouse
setting, on user’s motivation and task completion time. The implementation of
the leaderboard significantly increased performance, but both game design ele-
ments generate mixed results about the influence on motivation. The results
show that gamification can help to make working in warehouses more interest-
ing and possibly more efficient. Nevertheless, it should be taken into account
that the use of game design elements can have negative side effects.
Keywords: Gamification, Game Design Element, Augmented Reality, Leader-
board, Badges, Motivation, Order Picking, Logistics, Psychological Need Satis-
faction, Self-Determination Theory
1 Introduction
Motivating individuals is a challenging task, especially when it comes to activities
that are little or no fun. Especially work in a warehouse, in particular order picking,
can be tiresome and boring. But this is a very important part because the most labor
and costs in a warehouse emerge in the area of order picking [1]. Goetschalckx and
Ashayeri [2] define order picking as “the activity by which a small number of goods
is extracted from a warehousing system to satisfy a number of independent customer
orders”. Costs for order picking arise not only from wages but also from errors in the
processing of orders, such as picking the wrong articles or the wrong number of arti-
cles. Activities such as order picking are very monotonous and repetitive and thus
very demotivating in the long run. This leads to very high staff turnover rates and thus
to rising costs for hiring new employees [3].
One way to create motivation in such situations is gamification. Deterding et al. [4]
define gamification as “the use of game design elements in non-game contexts”. Us-
ing gamification to increase performance in production and logistics, e.g. a warehouse
GamiFIN Conference 2019, Levi, Finland, April 8-10, 2019 229
setting, has recently become a rising topic in research as well as in practice [5]. How-
ever, work can also motivate on its own, especially if a flow condition is achieved [6].
One way to create or promote a flow condition is gamification [7]. But in contrast to
flow theory, which requires full concentration to get into a flow condition [6], gamifi-
cation works with both conscious and unconscious perceived game design elements.
Also supporting workers in order picking by using augmented reality is another
way to fight the decrease of motivation. A very common definition for augmented
reality defines it as a “form of virtual reality where the participant's head-mounted
display is transparent, allowing a clear view of the real world” [8]. Such a head-
mounted display is e.g. the HoloLens1 from Microsoft, which was used in our study.
Augmented reality has already been shown to reduce picking time and improve pick-
ing accuracy [9].
The aim of this study is to test two game design elements, leaderboards and badg-
es, individually using augmented reality to determine possible differences in their
effect on motivation and performance of warehouse workers and eventually getting
positive results.
The rest of our article is structured as follows: In section two we give a brief over-
view of the related work and we conclude this section with our hypotheses. The next
section describes the study in detail, giving information about the procedure of our
experiment, measurements and the subjects which participated. The results follow in
the fourth section. In the last section, the results are discussed and pointed to limita-
tions and possible future research.
2 Related Work
Previous studies have already shown that badges and leaderboards have a motivating
effect [8–10]. Both game design elements, are among the most frequently used game
design elements in the field of gamification [11, 12]. In gamification research, game
design elements are usually tested together, but no conclusions can be drawn about
the effect of the individual elements. The effect of individual game elements can vary
greatly, however, this has been investigated in just a few studies so far [8, 9, 13].
Mekler et al. investigated the effect of points, levels and leaderboards individually in
an online experiment. They were able to capture an effect of the elements on the
number of tasks processed in a picture tagging process, but no effect on the intrinsic
motivation of the participants [10]. Hamari also deals with the effect of an individual-
ly applied game design element (badge) and was able to prove a motivating effect for
this in an online experiment [11]. Mazarakis and Bräuer started to research and com-
pare systematically individual game design elements, showing partly considerable
large differences, but have not yet done this so far for leaderboards [15].
In order to understand the effect of individual game design elements correctly, they
should also be tested in different contexts. Also none of the previous studies deals
with the individual application of badges or leaderboards in logistics. The present
1
https://en.wikipedia.org/wiki/Microsoft_HoloLens - retrieved 20 March 2019
GamiFIN Conference 2019, Levi, Finland, April 8-10, 2019 230
work is intended to provide new insights into this research field combining it with
augmented reality.
A study investigating the use of gamification in order picking was conducted by
Sailer et al. [16], using a gamified version of an order picking setting. In their study,
the authors combine the game design elements points, team leaderboards, perfor-
mance graphs, badges, avatar, narrative and tutorial levels. To measure the motivation
of the subjects, the authors designed a questionnaire based on the concept of fulfilling
basic psychological needs [17]. Sailer et al. could show effects on all basic psycholog-
ical needs. Also the performance of the subjects showed a positive effect in the gami-
fied group, as they worked faster and made fewer mistakes than the control group.
In a second study Sailer et al. [18] examined the effects of different combinations
of game design elements to show the satisfaction of psychological needs using an
online simulated order picking-task. In a video game setting the subjects processed
various picking orders on the computer from a bird's eye view. Three groups were
compared: a group with badges, leaderboard and performance graphs, a second group
with avatar, story and teammates, and a control group without game design elements.
The results show effects on the basic psychological needs competence need satisfac-
tion and the autonomy need satisfaction regarding task meaningfulness through the
combination of badge, leaderboard and performance graph.
Passalacqua et al. [19] also deal with the application of gamification in the field of
order picking. In their research proposal the authors present an initial approach for an
experiment in a warehouse. The aim of their proposal is to investigate possible differ-
ences regarding the engagement and performance of users through self-set or external
assigned goals both in combination with feedback.
However, it is not possible to derive the effect of the individual elements in the
context of warehouse work from one of these studies. The present article aims to ad-
dress the aforementioned research gap by investigating leaderboards and badges indi-
vidually in the context of order picking.
In line with previous findings [10, 11, 15, 16, 18], we express the following hy-
potheses separating between the game design elements and motivation or perfor-
mance:
H1a: Badges increase user’s motivation during the order picking task, compared to
the control condition.
H1b: Leaderboards increase user’s motivation during the order picking task, com-
pared to the control condition.
H2a: Badges increase performance in the order picking task, compared to the con-
trol condition.
H2b: Leaderboards increase performance in the order picking task, compared to
the control condition.
3 Method
To test our hypotheses, we conducted a between-subject experiment. The independent
variable is the use of the different game design elements: badges and leaderboard
GamiFIN Conference 2019, Levi, Finland, April 8-10, 2019 231
(badges vs. control condition and leaderboard vs. control condition). The dependent
variables were user motivation (measured through a basic psychological needs ques-
tionnaire) and performance (time needed to complete the task).
3.1 Procedure
We performed the experiment with an order picking application for Microsoft Ho-
loLens, which we developed for performing this research, followed by a questionnaire
asking different questions about the satisfaction of basic psychological needs, experi-
mental manipulation check and demographic data like age and sex. HoloLens are
augmented reality glasses that allow users to see and interact with virtual information
directly in their field of view. This will enable e.g. workers in a warehouse to pick
orders without holding a device or piece of paper in their hands, which makes the
work much easier. We created three different versions of the order picking applica-
tion. Two versions display game design elements, one badges and another one a lead-
erboard. In the control condition, no game design elements were presented.
In each version of our experiment users have to complete a short tutorial explain-
ing the task and teaching how to use the HoloLens. Upon clicking on a button to start
the experiment, subjects are randomly assigned to one of the three experimental con-
ditions. In addition, the subjects are asked to enter their name or a pseudonym. The
name or pseudonym is required in the condition with the leaderboard, where it can be
displayed. The subjects are explained the name/pseudonym would be essential for
linking the data from the experiment with those of the subsequent survey.
Fig. 1. Green path (left) and red shelf marking (right)
During the experiment, all subjects have to pick ten fixed orders. These are identical
for all subjects. The goal is to complete all orders as fast as possible without neglect-
ing accuracy. Each order contains three to five predefined products, which are located
in shelfs on a test course. The course consists of several shelves in which different
products are placed. In addition, there is a fixed starting point at which the subjects
get new orders and a new box for each order in which the picked products are stored.
Finished orders have to be delivered back to starting point. When a new order is exe-
cuted, the subjects takes a box and follow a green path (see Figure 1 left), which is
displayed via the HoloLens, guiding to the first product in the order. The shelf, in
which the next product is located, is marked with a red frame (see Figure 1 right). If
the subject reaches this shelf and clicks on the red frame, a text will appear to inform
GamiFIN Conference 2019, Levi, Finland, April 8-10, 2019 232
the subject to activate the QR code scanner in the HoloLens. After removing the
product from the shelf, the subject scans the product, confirming the correctness of
the removal. After the product has been scanned successfully and put to the carrying
box, the subject is shown the path to the next shelf. Once the subject has collected all
relevant products, the green path will guide the subject back to the starting point.
There the subject confirms the delivery of the order and receives a new order and a
new empty box. If the subject has fulfilled all ten orders, the message “Thank you for
your participation!” appears and the subject can take off the HoloLens.
Fig. 2. Blue board with badges (left) or leaderboard (right), both in German language
In the leaderboard condition, subjects can see a big blue virtual screen next to the
starting point (see Figure 2 right). On the screen they can compare their current score
to four fictitious subjects in a leaderboard. The ranking is sorted by the time (in sec-
onds) taken to complete the last task. The leaderboard is updated after each completed
order. If the subject reaches a place on the leaderboard, the name/pseudonym entered
during the tutorial will be displayed in green color surrounded by two arrows on the
leaderboard. By using self-chosen names or pseudonyms, we expect the subjects to be
more emotionally involved. To avoid a confounding effect of the positioning on the
leaderboard on subjects’ motivation, all subjects are shown the same ranking. Only
their position on the leaderboard can change according to their performance. The
remaining slots always remain identical. The scores, e.g. time, on the leaderboard
were determined on the basis of a brief pre-study.
Similar to the leaderboard condition, subjects in the badge group will also see a
blue board at the starting point (see Figure 2 left). The board shows the badges col-
lected during the experiment. In addition to the visual representation on the board, the
badges also appear for three seconds in the middle of the field of view of the Ho-
loLens as soon as they are unlocked and an acoustic signal in the form of a short jin-
gle can be heard. The badges were placed directly in the center of the field of vision,
as it was determined during a test phase that they were otherwise undetectable. This is
partly due to the very small field of view of the HoloLens.
A total of 10 different badges were designed for the experiment. Four of the badges
are awarded for picking certain products. Two other badges are tied to a time condi-
tion. Three of the badges were built on top of each other and are awarded for walking
a predefined distance. The last badge is awarded when a subject has processed all
orders correctly. The badges were designed in such a way that they are distributed as
evenly as possible over the entire course of the experiment. In order not to neglect the
GamiFIN Conference 2019, Levi, Finland, April 8-10, 2019 233
actual task in favor to obtain as many badges quickly as possible, the subjects are not
informed in advance how the badges are awarded. Only when a badge has been un-
locked, then it can be clicked on the board and a text with the information how the
badge was awarded will be displayed.
After the HoloLens experiment the subjects are asked to complete a survey. In or-
der to minimize any influence on the subjects by the experimenter, the questionnaire
is provided on a separate computer and answered by the subjects alone without the
presence of the experimenter. The purpose of the survey is to determine the motiva-
tion of the subjects. It also asks about demographic data of the subjects such as age,
height, employment and gender. At the end of the questionnaire, the subjects can
finally provide general feedback and comments on the experiment in a free-text field.
3.2 Measurements
Performance was measured by tracking the time each subject needed to complete the
orders. This was done automatically by the HoloLens application. Additionally a sur-
vey was used to measure the motivation of the users during the task. We adapted the
survey items from the study by Sailer et al. [16]. The questionnaire developed by
Sailer is based on various standardized questionnaires for measuring intrinsic motiva-
tion and determines it on the basis of the three basic psychological needs autonomy,
competency and relatedness. According to Deci and Ryan, satisfying these three
needs can create intrinsic motivation [17]. The questionnaire comprises a total of 13
items. All items were measured along a 7-point Likert scale (from strongly disagree
to strongly agree). The need for autonomy is divided into decision freedom and task
meaningfulness in this scale, as autonomy refers to psychological freedom and the
volition to fulfill a certain task [18].
3.3 Subjects
A total of 80 subjects took part in the experiment and completed the survey. As a
reward for their participation, the subjects received sweets. However, this remunera-
tion was not explicitly promoted and not stated in the beginning of the experiment.
Seven subjects had to be excluded from the analysis because the data was not com-
pletely stored in the log file due to technical issues and therefore not available.
Since the motivation of the subjects could only be measured if they were not seri-
ously distracted from their experimental condition, we conducted a manipulation
check for each game design element. After that ten subjects had to be excluded be-
cause they did not meet this criterion, e.g. they could not name the correct experi-
mental condition, after asking them if they observed a leaderboard, badges or nothing.
Four subjects were removed from the leaderboard condition, five from the badge con-
dition and one from the control group.
Thus, the sample for statistical analysis consists of N = 63 subjects. Of these,
35 are students (55.5 %), 27 employees (42.9 %) and 1 self-employed individual
(1.6 %). 32 of the subjects were men (50.8 %) and 31 women (49.2 %). The mean age
GamiFIN Conference 2019, Levi, Finland, April 8-10, 2019 234
is 30.92 years (SD 10.84). The youngest individual is 19 years old and the oldest is 60
years old. In total, 9 of the subjects (14.3 %) already had experience in order picking.
4 Findings
The distribution of subjects to the three experimental conditions as well as the mean,
median and standard deviation of the time in seconds that subjects needed to complete
the experiment are given in Table 1. The fastest subject only needed 8.35 minutes to
finish the experiment, the slowest subject needed twice as long with 19.47 minutes.
Table 1. Number of subjects, mean, median and standard deviation in seconds per condition
Condition N Mean Median SD
Control 24 774 766 158
Leaderboard 14 689 681 117
Badges 25 763 716 151
Total 63 751 725 149
Figure 3 shows the mean and standard deviation for the four basic needs variables
in the three experimental conditions. All three conditions have low values for the
experience of decision freedom and the feeling of relatedness questions. On the other
hand, the experience of task meaningfulness and experience of competence are rated
with higher values.
7 5,88 5,32 5,91
4,83 5,17
6
3,8 2,88
5 2,64
Mean
2,24
4 2,31 1,95 1,93
3
2
1
decision freedom task competence relatedness
meaningfulness
CG Leaderboard Badges
Fig. 3. Means and standard deviation of psychological needs for all conditions
4.1 Motivational Outcomes
According to the Shapiro-Wilk test [20], all four variables that are used to measure
the basic psychological needs yield a statistically significant result concerning the
normal distribution: experience of decision freedom (autonomy) p = .000, experience
GamiFIN Conference 2019, Levi, Finland, April 8-10, 2019 235
of task meaningfulness (autonomy) p = .000, experience of competence p = .007, feel-
ing of relatedness p = .000. As a result, all of the variables are not normally distribut-
ed (p < .001). Mann-Whitney U test is therefore used as a non-parametric statistical
method for the comparison of the mean values between the groups [20]. The follow-
ing sections describe the results in terms of basic psychological needs in more detail:
Feeling of Autonomy
The comparison between the control group (CG) (Mdn = 2.00) and the group with
the leaderboard (Mdn = 2.17) shows for experience of decision freedom a statistically
non-significant result: U = 129.00, z = -1.19, p = .117 (one sided), r = -.19. Also the
comparison between the CG and the group with badges (Mdn = 2.00) does not be-
come statistically significant for this variable: U = 250.00, z = -1.02, p = .155 (one
sided), r = -.14. Regarding the experience of task meaningfulness, there is no statisti-
cally significant result for comparing the means between CG (Mdn = 6.50) and the
group with the leaderboard (Mdn = 5.83): U = 115.00, z = -1.62, p = .053 (one sided),
r = -.26. The comparison between the CG and the group with badges (Mdn = 6.00) is
also not statistically significant: U = 290.00, z = -.20, p = .419 (one sided), r = -.03. In
summary, it can be stated that no influence of the two examined game design ele-
ments on the feeling of autonomy of the subjects could be determined.
Experience of Competence
The comparison of the mean values for the experience of competence becomes sta-
tistically significant between the CG (Mdn = 5.00) and the group with the leaderboard
(Mdn = 3.88): U = 103.00, z = -1.97 p = .025 (one sided), r = -.32. However, no sta-
tistically significant result is obtained for the comparison between CG and the group
with badges (Mdn = 5.50): U = 271.00, z = -.58, p = .281 (one sided), r = -.08. Inter-
estingly, the results are contrary to the established hypothesis. As it can be seen in
Figure 3, the mean of the CG is higher than the value of the leaderboard group. The
leaderboard thus had a negative impact on the experience of competence.
Feeling of Relatedness
Regarding the feeling of relatedness, the means between the CG (Mdn = 2.17) and
the group with the leaderboard (Mdn = 1.33) differ statistically significant:
U = 106.50, z = -1.72, p = .043 (one sided), r = -.28. The comparison of the means
between the CG and the group with badges (Mdn = 1.00) also becomes statistically
significant for this variable: U = 170.50, z = -2.29, p = .011 (one sided), r = -.33. Nev-
ertheless, one can also see here, considering the descriptive results, that the mean of
the CG is highest. Subjects who have received the leaderboard or badges, felt less
socially related than subjects who could see no game design elements.
Since an overarching increase in the four motivation variables was not found in ei-
ther the leaderboard or badge group compared to the CG, both hypotheses H1a and
H1b are discarded.
GamiFIN Conference 2019, Levi, Finland, April 8-10, 2019 236
4.2 Behavioral Outcomes
With regard to the task completion time, the Shapiro-Wilk test yields a statistically
significant result with p = .007. Thus, there is again no normal distribution of the data
and the Mann-Whitney U test is used again as method for statistical analysis.
The comparison of the means between the CG and the group with the leaderboard
yields one-sidedly tested a statistically significant result U = 113.00, z = -1.66,
p = .048, r = -.27. It can be assumed that subjects who receive a leaderboard accom-
plished the experiment faster than subjects who haven’t seen a game design element.
This supports hypothesis H2b.
A comparison of the means of the CG with those of the group with badges does not
yield a statistically significant result U = 278.00, z = -.44, p = .330 (one-sided),
r = -.06. Consequently, H2a cannot be supported because it is assumed that subjects
of the group with badges did not accomplish the experiment faster than subjects who
did not get a game design element displayed.
Influence of Age and Gender
Recent studies show that the effect of gamification can be influenced by both age
and gender [19]. For this reason, it was checked for both parameters whether there
was a correlation with subjects’ task completion time.
In the leaderboard group a strong positive correlation was found between task
completion time and age, r = .698, p = .006. The results indicate that younger subjects
accomplished the tasks faster than older subjects. For the group with badges no corre-
lation between the age of the subjects and the task completion time could be deter-
mined, r = .257, p = .215. In the control group also no correlation between age and
task completion time could be observed, r = .129, p = .547. Since a correlation only
occurs in the group with the leaderboard, the assumption can be made that leader-
boards may have a stronger effect on younger individuals.
For both groups a strong correlation could be determined between the gender of the
subjects and the task completion time: leaderboard r = .573, p = .032; badges r = .527,
p = .007. Since men were coded with zero and women with one, it could be deduced
from the results that men accomplished the tasks faster than women. However, the
influence of gender can be traced back to the height of the subjects. A partial correla-
tion between sex and task completion time, taking into account the height of subjects,
revealed no correlation between the two factors: leaderboard r = .227, p = .360; badg-
es r = .179, p = .404. The mean value for the height is 183 cm (SD 6 cm) for men and
167 cm (SD 8 cm) for women. Since men are taller on average, they have a bigger
step length and can therefore accomplish the tasks faster.
5 Discussion
Our motivation for the present study was to assess how leaderboards and badges af-
fect subjects’ performance (completion time) and motivation in an augmented reality
order picking setting. Contrary to the results of previous studies [8, 9, 14], no positive
GamiFIN Conference 2019, Levi, Finland, April 8-10, 2019 237
effect on the motivation of the subjects could be achieved by using individual game
design elements. We interpret these results briefly in this section.
The findings of the present study suggest that leaderboards are an effective way to
influence user behavior. However, the positive effect on the task completion time
achieved by the leaderboard could not be replicated for the badge element. One rea-
son that the badges did not work as expected could be that they distracted the sub-
jects. This is supported by observing the subjects, who stopped between the orders to
have a look at the board with the badges they had achieved. Possible positive effects
on the task completion time of the subjects could thus have been lost.
It was not possible to determine the expected positive effect on the motivation of
the subjects either in the group with the leaderboard or in the group with badges. Con-
trary to expectations, even negative effects on psychological needs could be recorded.
Both the leaderboard and the badges had a negative effect on the feeling of related-
ness. The leaderboard also had a negative influence on the subjects' perceived experi-
ence of competence. We interpret these results as such that the design of the leader-
board as a competitive leaderboard (in contrast to e.g. a cooperative team leader-
board) may have led to these effects.
5.1 Limitations and Future Research
The present study has limitations, which offer starting points for future research. First,
we only examined the short-term effects of leaderboards and badges. Since the aug-
mented reality technology used is still very novel, most of the subjects were very
fascinated by this. This could blur the positive effect of gamification. Therefore, it is
necessary to repeat the experiment with a significantly longer duration.
Also the difference between the two conditions should be noted. It is possible to
use goal-setting behavior in the leaderboards condition but not in the badges condi-
tion. The badges were implemented in such a way that subjects could not see the cri-
teria for unlocking in advance. In other words, each awarded badge would be a sur-
prise to the subject. This means that the badge system, unlike the leaderboard system
which displayed task completion times, did not allow subjects to set goals. We do not
see this issue as a thread to the validity of our results but we acknowledge that the
comparison between the two conditions is not straightforward. For future studies we
suggest to provide the badges not as a surprise. But this might lead to even longer task
completion times, because the subjects could spend at the beginning too much time to
find out, how to unlock the different badges. Future studies must clarify these issues.
Another possibility would be to carry out a similar study under real conditions in
practice. On the one hand, it could also be determined under real conditions whether
an influence on the error rate is detected, which is also an important cost factor in
order picking. On the other hand, it would be possible to study individuals who are
familiar with the job and are therefore likely to be less motivated by the novelty of the
task, as may have been the case with the subjects in this study.
Since the results concerning the influence of age on the effect of gamification was
only a byproduct of this study but offered interesting and surprising results, the effect
of this factor should be considered in a future study. Additionally the attitude towards
GamiFIN Conference 2019, Levi, Finland, April 8-10, 2019 238
new technologies might be an interesting confounding variable, worth to conduct
analysis using technology acceptance questionnaires.
Finally what should be kept in mind is that subjects have repeatedly stated that Ho-
loLens is very uncomfortable to wear. This circumstance could have had a negative
effect on the motivation and satisfaction of the subjects. For subsequent studies, the
use of more comfortable data glasses of the upcoming generation is recommended.
5.2 Conclusion
The results show that the use of game design elements has to be done with caution in
order to avoid unwanted or negative effects. Leaderboards showed significant effects
for performance but negative for motivation. Badges instead were superior for the
feeling of competence but fell short in terms of performance. These results were only
possible through the experimental design and comparison of individual game design
elements as well as taking into account social and context-related influences. This
shows the importance of comparing individual game design elements.
Though leaderboards showed positive effects on performance, motivation might be
endangered and could have in the long run undesirable effects, e.g. leaderboards can
harm the feeling of relatedness if competition is the main focus of the game design
element. But also applying a very common game design element like badges can be
no panacea, if context is not considered completely, e.g. distraction by novelty. Gami-
fication can therefore be no solution without considering context.
References
1. Grosse, E.H., Glock, C.H., Neumann, W.P.: Human factors in order picking: a content
analysis of the literature. International Journal of Production Research. 55, 1260–1276
(2017).
2. Goetschalckx, M., Ashayeri, J.: Classification and Design of Order Picking. Logistics
World. 2, 99–106 (1989).
3. Hense, J., Klevers, M., Sailer, M., Horenburg, T., Mandl, H., Günthner, W.: Using Gami-
fication to Enhance Staff Motivation in Logistics. In: Meijer, S.A. and Smeds, R. (eds.)
Frontiers in Gaming Simulation. pp. 206–213. Springer International Publishing, Cham
(2014).
4. Deterding, S., Dixon, D., Khaled, R., Nacke, L.: From Game Design Elements to Game-
fulness: Defining “Gamification.” In: Proceedings of the 15th international academic
MindTrek conference: Envisioning future media environments. pp. 9–15. ACM Press
(2011).
5. Warmelink, H., Koivisto, J., Mayer, I., Vesa, M., Hamari, J.: Gamification of the work
floor: A literature review of gamifying production and logistics operations. In: Proceed-
ings of the 51st Hawaii International Conference on System Sciences. pp. 1108–1117
(2018).
6. Csikszentmihalyi, M.: Flow: The Psychology of Optimal Experience. Harper [and] Row,
New York, NY, USA (2009).
GamiFIN Conference 2019, Levi, Finland, April 8-10, 2019 239
7. Hamari, J., Koivisto, J.: Measuring Flow in Gamification: Dispositional Flow Scale-2.
Computers in Human Behavior. 40, 133–143 (2014).
8. Milgram, P., Takemura, H., Utsumi, A., Kishino, F.: Augmented reality: a class of dis-
plays on the reality-virtuality continuum. In: Das, H. (ed.) Telemanipulator and
telepresence technologies. pp. 282–293. International Society for Optics and Photonics
(1995).
9. Hanson, R., Falkenström, W., Miettinen, M.: Augmented reality as a means of conveying
picking information in kit preparation for mixed-model assembly. Computers & Industrial
Engineering. 113, 570–575 (2017).
10. Mekler, E.D., Brühlmann, F., Opwis, K., Tuch, A.N.: Do points, levels and leaderboards
harm intrinsic motivation?: an empirical analysis of common gamification elements. In:
Proceedings of the First International Conference on gameful design, research, and appli-
cations. pp. 66–73. ACM Press (2013).
11. Hamari, J.: Do Badges Increase User Activity? A Field Experiment on the Effects of
Gamification. Computers in Human Behavior. 71, 469–478 (2015).
12. Mazarakis, A.: Using Gamification for Technology Enhanced Learning: The Case of
Feedback Mechanisms. Bulletin of the IEEE Technical Committee on Learning Technol-
ogy. 4, 6–9 (2015).
13. Werbach, K., Hunter, D.: For the win: how game thinking can revolutionize your busi-
ness. Wharton Digital Press, Philadelphia, PA, USA (2012).
14. Dicheva, D., Dichev, C., Agre, G., Angelova, G.: Gamification in education: A systematic
mapping study. Educational Technology & Society. 18, 75–88 (2015).
15. Mazarakis, A., Bräuer, P.: Gamification is Working, but Which One Exactly?: Results
from an Experiment with Four Game Design Elements. In: Proceedings of the Technolo-
gy, Mind, and Society. p. 22:1. ACM, New York, NY, USA (2018).
16. Sailer, M., Hense, J., Mandl, H., Klevers, M.: Fostering Development of Work Competen-
cies and Motivation via Gamification. In: Mulder, M. (ed.) Competence-based Vocational
and Professional Education. pp. 795–818. Springer International Publishing, Cham
(2017).
17. Ryan, R.M., Deci, E.L.: Intrinsic and Extrinsic Motivations: Classic Definitions and New
Directions. Contemporary Educational Psychology. 25, 54–67 (2000).
18. Sailer, M., Hense, J.U., Mayr, S.K., Mandl, H.: How Gamification Motivates: An Exper-
imental Study of the Effects of Specific Game Design Elements on Psychological Need
Satisfaction. Computers in Human Behavior. 69, 371–380 (2017).
19. Passalacqua, M., Léger, P.-M., Sénécal, S., Fredette, M., Nacke, L.E., Lin, X., Grande, K.,
Robitaille, N., Ziemer, L., Caprioli, T.: The Impact of Using a Gamified Interface on En-
gagement in a Warehousing Management Task: A NeuroIS Research Proposal. In: Davis,
F.D., Riedl, R., vom Brocke, J., Léger, P.-M., and Randolph, A.B. (eds.) Information Sys-
tems and Neuroscience. pp. 187–194. Springer International Publishing, Cham (2019).
20. Field, A.P.: Discovering Statistics Using Spss: And Sex, Drugs and Rock “N” Roll. SAGE
Publications, Los Angeles, CA, USA (2009).
21. Jent, S., Janneck, M.: Using Gamification to Enhance User Motivation: The Influence of
Gender and Age. In: Freund, L.E. and Cellary, W. (eds.) Advances in The Human Side of
Service Engineering. pp. 3–10. Springer International Publishing, Cham (2018).
GamiFIN Conference 2019, Levi, Finland, April 8-10, 2019 240