=Paper=
{{Paper
|id=Vol-2186/paper3
|storemode=property
|title=Exploring characteristics of students’ emotions, flow and motivation in a math game competition
|pdfUrl=https://ceur-ws.org/Vol-2186/paper3.pdf
|volume=Vol-2186
|authors=Kristian Kiili,Antero Lindstedt,Manuel Ninaus
|dblpUrl=https://dblp.org/rec/conf/gamifin/KiiliLN18
}}
==Exploring characteristics of students’ emotions, flow and motivation in a math game competition==
Exploring characteristics of students’ emotions, flow and motivation
in a math game competition
Kristian Kiili
Tampere University of Technology, Finland
kristian.kiili@tut.fi
Antero Lindstedt
Tampere University of Technology, Finland
antero.lindstedt@tut.fi
Manuel Ninaus
Leibniz-Institut für Wissensmedien, Germany
m.ninaus@iwm-tuebingen.de
Abstract: The overall objective of the present study was to investigate associations between emotions, flow
experience, intrinsic motivation, and playing performance in a math game competition. Finnish 3rd – 6th
graders (n = 251) participated in a math game competition relying on intra-classroom cooperation and inter-
classroom competition. During a three-week competition period, students were allowed to freely play a
digital rational number game founded on number line estimation task mechanics. An online questionnaire
was used to collect students’ experiences after the competition. The results indicated that students who
experienced higher positive emotions in the competition also experienced higher levels of flow. Findings
also showed that intrinsically motivated students experienced more positive emotions and higher flow as
well as indicated a higher willingness to participate in math game competitions again as compared to low
intrinsically motivated students. Moreover, current results provided some evidence that game based
competitions might engage also lower performing students.
1. Introduction
Devlin (2013) has argued that video games can provide new interfaces to learn mathematics that
are far easier and more natural to use than symbolic expressions that we have used to employ in
conventional education. Game-based interfaces have the potential to provide means to develop
effective ways of training mathematics that may also motivate persons who are anxious about
mathematics. In fact, some recent studies have shown that the use of games in mathematics
education can be beneficial for both cognitive (e.g. Kiili, Moeller & Ninaus, 2018; ter Vrugte, et
al., 2017) as well as affective outcomes (e.g. Kiili & Ketamo, 2017; Ke, 2008). On a more general
level, a recent meta-analysis of the cognitive and motivational effects of serious games
demonstrated that serious games were more effective in terms of learning and retention than
conventional instructional methods (Wouters et al., 2013). However, surprisingly, the analysis
revealed that serious games were not more motivating than conventional instructional methods.
The authors argued that motivation in serious games might be undermined by a limited sense of
control of using such a game. That is, contrary to entertainment games, which are played on users
own will, serious games are usually used within an instructional context and thus, the decision on
GamiFIN Conference 2018, Pori, Finland, May 21-23, 2018 20
the type of game and when to play the game is made by educators and not the players. In this
context, the self-determination theory (Ryan & Deci, 2000; Deci & Ryan) argues that conditions
which limit the sense of freedom of action or control may undermine intrinsic motivation. This is
particularly interesting, because intrinsic motivation seems to be associated with positive emotional
experiences (e.g., Koestner & Losier, 2002) and improved learning performance (e.g., Ninaus,
Moeller, McMullen, & Kiili, 2017). Therefore, the current study examines the affective and
cognitive outcomes of a math game competition in which users themselves decided if and when
they wanted to participate in such an instructional intervention with a game.
Individuals partaking in a competitive game may experience an array of different emotions.
Importantly, emotions experienced in competitive educational settings influence outcomes of an
educational intervention. For instance, Plass and colleagues (2013) found out that a competitive
math game mode enhanced game performance compared to individual play. Their study also
indicated that both competitive and collaborative game modes increased enjoyment compared to
individual play. In line with these results, a study by Cagiltay, Ozcelik, and Ozcelik (2015) showed
that competitive aspects in a serious game significantly improved motivation and learning
outcomes. However, very little is known about emotions that students experience in competitive
game-based learning settings and how domain specific intrinsic motivation is related to
experienced emotions in a competitive game. Importantly, though, previous studies have identified
(math) self-efficacy and (math) interest as important predecessors and determinants of intrinsic
motivation (Campbell & Hackett, 1986; Pajares & Miller, 1994; Ryan & Deci, 2000; Skaalvik et
al., 2015).
Positive emotional experience in gameplay or even everyday life is often related to the experience
of flow. For instance, flow and enjoyment are usually highly correlated (Landhäußer & Keller,
2012) and thus, flow might even be described as a special form of enjoyment (Baumann, Lürig, &
Engeser, 2016). But of course, enjoyable experiences are not always related to flow. Landhäußer
and Keller (2012) have argued that it is important to note that flow is not the same as having fun,
but a combination of concentration, a merging of action and awareness, reduced self-
consciousness, a sense of control, a transformation of time, and a intrinsically rewarding activity
makes experiencing flow possible. For instance, enjoying the sunset at the beach is most likely not
related to flow experience. Thus, it is necessary to measure emotions and flow independently
(Baumann et al., 2016).
2. Present study and hypotheses
In the current study, we added a competitive aspect to a non-competitive math game by organizing
a competition around the game with online scoring boards based on players’ highscores achieved
in the game. The overall objective of the present study was to investigate the associations between
affect, flow, performance, and intrinsic motivation during a math game competition. Therefore, we
formulated the following three main hypotheses.
Hypothesis 1: According to Abuhamdeh (2012) flow theory is relevant for predicting enjoyment
(see also Baumann, Lürig, & Engeser, 2016; Landhäußer & Keller, 2012). Therefore, we expect
that there is significant positive relation between flow experience and pleasant emotions
experienced during the game competition.
Hypothesis 2: On the one hand, self-efficacy and interest – predecessors and determinants of
intrinsic motivation (Campbell & Hackett, 1986; Pajares & Miller, 1994; Ryan & Deci, 2000;
GamiFIN Conference 2018, Pori, Finland, May 21-23, 2018 21
Skaalvik et al., 2015) – are usually related to more positive emotional experiences (e.g., Koestner
& Losier, 2002). On the other hand, flow itself is considered to be positive emotional state
(Baumann et al., 2016; Landhäußer & Keller, 2012). Based on this, we expect that students with
high intrinsic math motivation (self-efficacy and interest) like the competition more, are more
willing to participate in game competitions again, as well as experience higher flow and more
positive feelings during the competition than students with low intrinsic math motivation.
3. Method
1.1. Participants
The participants of the current game competition were Finnish students. Altogether approximately
1500 students participated in the competition from which 258 filled in the digital questionnaire and
managed to complete the competition level at least once. According to Meade and Craig (2011)
careless, partially random, or otherwise inattentive survey responses should be identified and
removed from the data set. Thus, we utilized following strategies to exclude inattentive and careless
survey responses from the analyses. First, participation to the survey was voluntary and responses
were anonymoys. Second, the survey instructions emphasized that only the participants who
carefully responded to all questions were included in the movie tickets lottery after the competition.
Third, we removed four participants whose overal response time was too fast indicating that these
participants did not read the questions of the survey at all. Fourth, we removed three participants
who used a clear response pattern (all likert scale responses were on the beginning or on the end of
the scale). Finally, we carried out an outlier analysis with the most important sum variables used
in the analyses (flow experience, positive emotions, intrinsic math motivation, participation
willingness, and liking of the competition). Based on this analysis we did not find any extreme
outliers. Consequently, 251 participants were included in the analyses of this paper. Four of the
participants were 3rd graders, 69 were 4th graders, 24 were 5th graders, and 154 were 6th graders. Of
these participants, 104 were females, and 147 were males. Participants were 9-13 years old with a
mean (SD) of 11.63 years (1.02 years). Following the Finnish classification scheme, in which 10
reflects the best and 4 the lowest grade, the mean math grade (SD) of the participants was 8.58
(1.10). 84.5% of the participants reported that they play digital games at least once in a week.
2.1. Description of the competition
The competition was open for all Finnish 3-6 graders. In the competition, students had the
opportunity to play a number line based rational number game during a three-week period.
The competition was organized around one game level that included fourteen randomly selected
number line estimation tasks. The goal of the competition was to motivate students to play the level
several times in order to optimize their playing strategies, increase understanding of rational
numbers, and share effective playing strategies with their peers. In order to support collaboration
between peers each participating class formed a team that competed against other teams.
Furthermore, municipalities competed among each other. The web-page of the competition
included leaderboards for both teams and municipalities. Two best teams, best municipality, most
hard-working student, and a randomly selected player who answered to the research questionnaire
at the end of the competition were awarded with movie tickets.
We used our rational number game engine, Semideus, to develop a digital game for the competition
(see examples of studies using the same engine: Ninaus, Kiili, McMullen, & Moeller, 2017; Kiili,
Moeller, & Ninaus, 2018). In the game, the player had to estimate either fraction or decimal number
GamiFIN Conference 2018, Pori, Finland, May 21-23, 2018 22
magnitudes on a number line ranging from zero to one (Figure 1). The tasks of the game could
include also traps and enemies, time limits, mysteryboxes, and health kits. To assist the player,
some of the tasks contained visual landmarks, such as sequentially placed torches that divided the
number line according to the denominator of the fraction to be estimated or revealed the midpoint
of the number line.
Figure 1. An example of a number line estimation task in which the player should dig up a coin
cache at the location reflecting point 4/7. In this task the player should avoid the trap located at
point 0.7 and the purple enemy walking around. The player has activated the partition skill and as
a consequence the birds have divided the number line into seven parts.
The game also included specific skills that players could activate by using in-game currency, i.e.
diamonds. The player began each game with 10 diamonds with the possibility to acquire maximally
15 extra diamonds by smashing enemies, avoiding traps, and collecting mystery boxes. Some of
these skills made the solving of mathematical tasks easier and some just helped the player to survive
in the game. Altogether, five different skills were available:
• The player could ask help fom the tutor. When the tutor skill was activated, a goat showed
the approximate location of the coin cache to the player. The tutor skill was always available
(cost: 8 diamonds).
• The partition skill visually divides the number line to as many parts as the denominator of
the task. Specifically, if the skill is activated, birds fly onto the number line in a sequence
of unit fractions. That is, players could easily get an accurate answer as long as they
understood the partitioning of the number line based on these visual markers (see Figure
1). The partition skill was available if the denominator of the fraction to be estimated was
below 11 (cost: 6 diamonds).
• The player could use the bomb skill to destroy all the enemies from the task (cost: 6
diamonds).
GamiFIN Conference 2018, Pori, Finland, May 21-23, 2018 23
• The player could locate traps of the task with the trap finder skill (cost: 4 diamonds).
• Some tasks featured a time limit. In these tasks the player could remove the limit with the
time manipulation skill (cost: 2 diamonds).
Appendix A provides details of the tasks that were included in the competition. It also shows the
logic of the task randomization that was used. Moreover, the video of the game competition tutorial
clarifies the gameplay of the used game (https://youtu.be/mNCQ5dgOaMg).
3.1. Measures and analyses
Students playing performance (e.g. highscore) was logged in order to assess students’ rational
number estimation accuracy. At the end of the competition students had to fill in a questionnaire
to determine flow experience, intrinsic math motivation, positive and negative emotions, how much
students liked the game, and students’ willingness to participate in future game competitions.
Flow experience was measured using the 9-item short flow scale (adopted from Martin & Jackson,
2008). We translated the short flow scale to Finnish and contextualized it to address game playing.
A 7-point Likert-type response format (1 = strongly disagree to 7 = strongly agree) was used.
Intrinsic math motivation was measured with self-reported measures of math interest and math
self-efficacy. The three items used to measure math interest and the three items used to measure
math self-efficacy were adopted from (Berger & Karabenick, 2011). A 5-point Likert-type response
format (1 = strongly disagree to 5 = strongly agree) was used.
The measurement of positive and negative emotions was derived from items of the Sport Emotion
Questionnaire (Jones, et al., 2005). For positive emotions (scale 0-4) happiness (pleased + happy)
and Excitement (excited + energetic), and for negative emotions (scale 0-4) anxiety (uneasy +
anxious), anger (irritated + angry), and dejection (sad + dejected) were used.
In addition, the questionnaire included one item to explore how much students liked the game
competition, and one item to explore students’ willingness to participate in math game
competitions in the future. A 5-point Likert-type response format (1 = strongly disagree to 5 =
strongly agree) was used. Moreover, students had to report their digital game playing frequency on
a 4-point scale (Rarely - Once a week – Couple times a week – Almost daily).
The average scores of all flow items, emotion items, and intrinsic math motivation items were used
in the analyses. Statistical analysis was performed in IBM SPSS Statistics, version 25.
4. Results
Descriptive statistics (means, medians, and standard deviations) of variables used to investigate
students’ experiences about the competition and math motivation are presented in Table 1. The
flow construct (α = .84), pleasant emotion construct (α = .87), unpleasant emotion construct (α =
.83), and the intrinsic math construct (α = .92) had a high level of internal consistency, as
determined by Cronbach's alpha.
GamiFIN Conference 2018, Pori, Finland, May 21-23, 2018 24
Table 1. Descriptive statistics about students’ experiences and math motivation
Mean Median SD Scale
Flow experience 4.83 5 1.15 1-7 (α = .84)
Positive emotions 1.89 2 1.11 0-4 (α = .87)
Negative emotions 0.40 0.17 0.65 0-4 (α = .83)
Intrinsic math motivation 3.66 3.83 1.04 1-5 (α = .92)
Liked the game competition 3.54 4 1.16 1-5
Willingness to participate again 3.24 3 1.36 1-5
Highscore (performance) 55.13 56 23.18 0-100
Hypothesis 1: As expected, the correlation analysis yielded a strong positive relation between flow
experience and positive emotions supporting Hypothesis 1, r = .62, p < .001. In a linear regression
flow (unstandardized β = .599, standardized β = .623, p < .001) accounted for 39% in variance of
experienced positive emotions [F(1,249)= 157.70, p < .001, R2= .39].
Hypothesis 2: T-test was run to determine whether there were differences in the experienced
positive emotions, experienced flow level, and willingness to participate in game competitions
again between students whose intrinsic math motivation was high and students whose intrinsic
math motivation was low, using median split. There were statistically significant differences in the
mean scores of experienced positive emotions (M.high = 2.28, M.low = 1.47, t = 6.224, p < .001),
experienced flow (M.high = 5.33, M.low = 4.31, t = 7.842, p < .001), liking of the competition (M.high
= 4.02, M.low = 3.04, t = 7.319, p < .001), and willingness to participate in game competitions again
(M.high = 3.74, M.low = 2.70, t = 6.473, p < .001) between high and low motivation groups,
supporting hypothesis 2.
Table 2. Correlations between students’ experiences, motivation, and highscore
1 2 3 4 5 6 7
1 Flow experience 1
2 Positive emotions .62** 1
3 Negative emotions -.22** -.18** 1
4 Liked the game competition .66** .62** -.27** 1
5 Willingness to participate again .67** .59** -.23** .67** 1
6 Intrinsic math motivation .54** .42** -.12 .48** .44** 1
7 Highscore .38** .27** .02 .36** .27** .16* 1
Note. ** p < .001; * p < .05
5. Discussion and conclusion
The current study investigated the relationship between intrinsic motivation, flow, performance,
and emotions in a math game competition. Students experiencing higher positive emotions also
experienced higher levels of flow. Moreover, students categorized as highly intrinsically
motivated, i.e. high math interest and high math self-efficacy, experienced more positive emotions
GamiFIN Conference 2018, Pori, Finland, May 21-23, 2018 25
and flow and indicated a higher willingness to participate in math game competitions in the future
as compared to low intrinsically motivated students.
Current results underline the tight coupling of intrinsic motivation, positive emotion, flow, and
performance. Therefore, we were able to show similar results with a math game competition as in
non-competitive and non-game based learning environments. The independent measurement of
flow and emotional experience supports general findings of flow being considered as a positive
emotional state (e.g., Landhäußer & Keller, 2012) and its relation to intrinsic motivation (e.g.,
Koestner & Losier, 2002). Importantly, almost 40% in variance of experienced positive emotions
was predicted by flow, which clearly underlines the tight link between flow and positive emotions
or more generally the interdependence of cognition and emotion.
In the context of math game competition, students’ willingness to participate again was highly
related to flow, positive emotions, and the overall liking of the game competition (see Table 2).
Importantly, though, the correlation between actual performance and students’ willingness to
participate again in a game competition was rather small (see Table 2). This might indicate that the
game-based competitions might not only be interesting for high performing students but low
performing students as well. However, this implication needs to be treated carefully and examined
in future studies, which might also use more advanced performance measures than mere highscore.
To conclude, the results provide some evidence regarding the educational potential of game-based
math competitions. However, in order to better understand the usefulness of game competitions as
an instructional method, more research focusing also on learning gains in game competitions and
the meaning of different kind of game mechanics and user interfaces is needed.
Overall, the present study yielded promising results regarding the use of a math game competition
that relies on intra-classroom cooperation and inter-classroom competition. However, there are
some limitations that should be considered in future studies. First, better and more detailed metrics
about students performance might improve our understanding of underlying processes in emotion
and motivation when employing (math) game competitions in educational settings. Second,
students could choose on their own when they answered the employed questionnaire as we were
hoping to achive a higher response rate. Thus, some students might have answered immediately
after their last playing session while others waited longer, which may have led to inaccurate recall.
Third, in spite of flexible response schedule, response rate were rather low. Nevertheless,
responders’ ranks in the competition varied a lot while the median rank was 286. This indicates
that the results presented in this paper are not based only on the experiences of the top players of
the competition, but reflect students’ experiences about the competition more generally.
References
Baumann, N., Lürig, C., & Engeser, S. (2016). Flow and enjoyment beyond skill-demand balance: The
role of game pacing curves and personality. Motivation and Emotion, 32, 67–77.
Cagiltay, N. E., Ozcelik, E., & Ozcelik, N. S. (2015). The effect of competition on learning in games.
Computers & Education, 87, 35-41.
Campbell, N. K., & Hackett, G. (1986). The effects of mathematics task performance on math self-
efficacy and task interest. Journal of Vocational Behavior, 28(2), 149–162.
Jones, M. V., Lane, A., Bray, S., Uphill, M. A., & Catlin, J.. (2005) Development and validation of the
Sport Emotion Questionnaire. Journal of Sport and Exercise Psychology, 27(4), 407-431.
GamiFIN Conference 2018, Pori, Finland, May 21-23, 2018 26
Ke, F. (2008). A case study of computer gaming for math: Engaged learning from gameplay?. Computers
& Education, 51(4), 1609-1620.
Kiili, K. & Ketamo, K. (2017). Evaluating Cognitive and Affective Outcomes of a Digital Game-Based
Math Test. IEEE Transactions on Learning Technologies.
Kiili, K., Ninaus, M., & Moeller, K. (accepted in January 16, 2018). Evaluating the effectiveness of a
game-based rational number training - In-game metrics as learning indicators. Computers & Education.
Koestner, R., & Losier, G. F. (2002). Distinguishing three ways of being highly motivated: A closer look
at introjection, identification, and intrinsic motivation. In Handbook of self-determination research (pp.
101–121). Rochester, NY, US: University of Rochester Press.
Landhäußer, A., & Keller, J. (2012). Flow and Its Affective, Cognitive, and Performance-Related
Consequences. In S. Engeser (Ed.), Advances in Flow Research (pp. 65–85). New York, NY: Springer
New York.
Meade, A. W., & Craig, S. B. (2011). Identifying careless responses in survey data. Paper presented at the
26th Annual Meeting of the Society for Industrial and Organizational Psychology, Chicago, IL.
Ninaus, M., Moeller, K., McMullen, J., & Kiili, K. (2017). Acceptance of Game-Based Learning and
Intrinsic Motivation as Predictors for Learning Success and Flow Experience. International Journal of
Serious Games, 4(3), 15–30.
Ninaus, M., Kiili, K., McMullen, J., & Moeller, K. (2017). Assessing fraction knowledge by a digital
game. Computers in Human Behavior, 70, 197-206.
Pajares, F., & Miller, M. D. (1994). Role of self-efficacy and self-concept beliefs in mathematical problem
solving: A path analysis. Journal of Educational Psychology, 86(2), 193–203.
Plass, J. L., O’keefe, P. A., Homer, B. D., Case, J., Hayward, E. O., Stein, M., & Perlin, K. (2013). The
impact of individual, competitive, and collaborative mathematics game play on learning, performance, and
motivation. Journal of educational psychology, 105(4), 1050-1066.
Ryan, R. M., & Deci, E. L. (2000). Intrinsic and Extrinsic Motivations: Classic Definitions and New
Directions. Contemporary Educational Psychology, 25(1), 54–67.
Skaalvik, E. M., Federici, R. A., & Klassen, R. M. (2015). Mathematics achievement and self-efficacy:
Relations with motivation for mathematics. International Journal of Educational Research, 72, 129–136.
ter Vrugte, J., de Jong, T., Vandercruysse, S., Wouters, P., van Oostendorp, H., & Elen, J. (2017).
Computer game-based mathematics education: Embedded faded worked examples facilitate knowledge
acquisition. Learning and Instruction, 50, 44-53.
Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D. (2013). A meta-analysis of
the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105(2), 249.
GamiFIN Conference 2018, Pori, Finland, May 21-23, 2018 27
Appendix A. Details about the tasks used in the competition level
Target Trap Enemies Time limit Landmarks Collectible Start Probability
1/3 - - - Denominator - 0 14%
2/3 - - - Denominator - 0 14%
3/4 - - - Denominator - 0 14%
1/4 - - - Denominator - 0 14%
0.25 - - - Denominator - 0 14%
0.8 - - - Denominator - 0 14%
5/9 - - - Denominator - 0 14%
2/5 0.2 - - - - 0 14%
3/5 0.4 - - - - 0 14%
4/10 0.2 - - - - 0 14%
6/10 0.4 - - - - 0 14%
0.600 2/5 - - - - 0 14%
0.099 1/4 - - - - 0.5 14%
0.400 3/5 - - - - 0 14%
0.75 - 1 - - - 0 50%
0.21 - 1 - - - 0 50%
0.6250 - 1 - - - 0 50%
0.3750 - 1 - - - 0 50%
3/9 (dots) - - - Denominator - 0 25%
6/9 (dots) - - - Denominator - 0 25%
4/6 (dots) - - - Denominator - 0 25%
2/6 (dots) - - - Denominator - 0 25%
3/8 - 1 - Midpoint - 0 33%
5/8 0.75 1 - Midpoint - 0.3 33%
4/7 0.7 1 - Midpoint - 0.3 33%
4/9 - 1 - Midpoint - 0 33%
5/9 0.7 1 - Midpoint - 0.3 33%
3/7 - 1 - Midpoint - 0 33%
1/6 - - - - Health 0.5 33%
1/5 - 1 - - Health 0.5 33%
1/4 - 1 - - Health 0.5 33%
23/45 - - 10s - - 0 25%
19/37 - - 10s - - 0 25%
27/50 - - 10s - - 0 25%
15/45 - - 10s - - 0 25%
3/10 + 1/5 0.3 1 - - - 0 25%
1/4 + 2/4 0.25 - - - - 0 25%
2/3 – 2/6 0.66 - - - - 1 25%
6/8 – 2/8 0.75 - - - - 1 25%
0.65 - 3 - - - 0 50%
3/4 - 3 - - - 0 50%
8/9 - - - - Health/Diam. 0.5 20%
5/6 - - - - Health/Diam. 0.5 20%
7/8 - - - - Health/Diam. 0.5 20%
GamiFIN Conference 2018, Pori, Finland, May 21-23, 2018 28
0.089 - - - - Health/Diam. 0 20%
7/9 - - - - Health/Diam. 0.5 20%
8/9 3/9 2 - - Health 0.5 50%
7/9 0.4 2 - - Health 0.5 50%
0.9511 0.6 1 - - - 0 33%
0.6 0.25 1 - - - 0 33%
0.45 0.55 1 - - - 0.5 33%
Column explanations
Target: Value to estimate
Trap: Value to avoid
Enemies: Number of enemies
Time limit: Player lost energy, if he or she did not manage to estimate the target value within the time
limit.
Landmarks: Visual markers that divide the number line into sections. Denominator: For example, if the
target number is 1/3 the markers divide the number line into three sections. Midpoint: The marker was in
the middle of the number line
Collectible: A mystery box that the player was able to pick up to gain a reward. Either Health or
Diamonds.
Start: Player’s starting position on the number line
Propability: How likely this particular task is to feature in one run of the game. The horizontal lines on the
table separate task groups. For example from the first group, one task was selected from seven possible,
resulting in about 14% appearance chance for a single task. From some groups, two tasks were selected
instead of one.
GamiFIN Conference 2018, Pori, Finland, May 21-23, 2018 29