This paper describes the design of a 3D running game with educational content. The goal of the game is to teach children capitalization in German. Sentences are presented to the children in increasing order of difficulty, determined by the syntactic structure of the sentences. Words within sentence were presented decaptialized and the children had to select the words that should have been capitalized. The game design uses an emotional avatar along with speed factor in order to motivate the children to play longer and improve their performance. 20 children played the game with the avatar and 16 children played the game without the avatar. A qualitative feedback was collected via an online survey. In addition, their performance profile was logged and analyzed. Based on the data, we report some trends that indicate increased engagement and performance. These indicators can be used for improving user-dependent, adaptive design for the player.
INTRODUCTION
When designing games, it is relevant to look at design
frameworks that allow us to guide and judge design, motivation, and
learning. A research and practice model [
Socio-cultural
The final pillar of engagement leads to an often forgotten
component of the world surrounding the game itself. In the
design we chose, even the runner in the game and the avatar
are already friends that the player is helping out. Future work
would have to spend more analysis on how the customization
of the avatar and runner could help engagement though social
interaction with avatar and runner or other players.
The goal of the presented work is to show the following:
1. Using an emotional avatar should engage players for longer
2. Using an emotional avatar should result in returning players
3. Increased playtime should result in higher skill level
THE GAME
One of the most significant problems German school children
face is the concept of capitalization that does not exist in many
other languages. In German, nouns are capitalized. In addition,
even a verb or an adjective can become nominalized. Wrong
capitalization is one of four most frequent spelling errors that
persists even for adults [
The player uses a chicken "Henry" to move in the game and
pick answers in an endless run. When the correct word is
picked, it is capitalized in the sentence shown. For example
a sentence like "Auch jede möwe und jede biene ist da." is
shown and the player has to pick "möwe" and "biene" as they
have to be capitalized (see Figure 1). The game contains 123
sentences sorted by difficulty. The difficulty reflects sentence
complexity. The goal of this game is to practice the concept
and automate it through speed and increasing difficulty of
sentences. In order to compute difficulty measures of a sentence
a dependency parser is needed. For this purpose, spaCy, an
open-source natural language processing (NLP) library written
in Python, was chosen. Given a sentence, it can return
CoNLLU formatted encoding of a sentence that includes tokenization,
Part-of-Speech (PoS) tagging and dependency parsing [
Emotions The current design of the game has a single environment which is a forest scenery. The sentence is placed at the top of the scene and originally the question was placed at the bottom. The on-going score is added to the top left of the scene. The avatar "Sam" is added to the scene (see Figure 1. Sam has five emotional expressions that are designed to provide feedback to the player, namely: Idle, happiness, sadness, "fun death", and stop. The default emotion is in idle position. On expressing happiness, a sound effect of kids cheering "yaay" is played. While on expressing sadness and stop, a sound effect of a sudden buzz is played to indicate something wrong happened. "Fun death", includes a sound effect of hitting the ground due to Sam’s falling.
When Henry makes a mistake, Sam expresses sadness. He expresses it using both a facial expression and a gesture. He raises his eyebrows while his head moves downwards and his hand is moving towards his mouth. This indicates to the player that something was wrong and the player might be encouraged to fix the situation or check the sentence for the given correction in order to avoid the same mistake later on. When Henry selects the correct answer, Sam jumps with happiness and has a wide smile to transfer this feeling to the player (See also Figure 2).The additional emotion of "fun death" is an exaggerated emotion of the reaction "falls over backwards", where Sam faints with exasperation. It occurs after repeated mistakes committed by the player. The intend of fun death is humor to increase engagement and focus while providing the feedback that there is room for improvement in the player’s performance. This emotion places two ’X’s in Sam’s eyes and a tongue to indicate fainting as he falls to the ground. After a short period he returns to upright position. Figure 3 shows Sam in his execution of "fun death" manoeuvre. Finally, when Sam realizes (by number of mistakes made) that the player doesn’t know how to play, he stops Henry and shows the player a panel of grammar instructions as depicted in Figure 4. After spending as much time as the players chooses with the instructions, they can restart the game from the beginning. The panel contains the informative feedback, that explains the rules explicitly to the player. In this case, the explanation has grammatical rules for capitalization.The emotion chosen for Sam as he stops the player, is an exaggerated version of "stop, what in the world are you doing? Let me explain it to you once again before you keep messing up even more" without so many words. The face has an angry expression and the hand is raised to stop the player from running around like a crazy person with no idea of what they are doing. It is intended to be humorous.
Visual and Informative Feedback Both correct and incorrect answers are displayed in the sentence at the top as the player runs past the words in the game: Correct choices for words that should have been capitalized are changed to capital and displayed in green. In contrast, any missed or wrongly picked word turn red and their spelling is not changed (see also Figure 5). A performance bar was added to indicate how good the player’s performance is. It starts with 50% at the start. It increases and decreases with correct and faulty answers respectively. As it falls below 30% it turns red. The game speed depends on this performance bar. As a result, the difficulty level is increased for high performers to encourage skill automation. It also prevents boredom for good players and faster movement towards more difficult sentences. As such, the speed is individual and adaptive to the player. Last but not least, a panel showing the final score and a button to play again is shown when players stop the game.
UI ADJUSTMENTS Initial informal evaluations of the UI focused on understanding children’s first reaction to the game and verifying their interest and understanding of the humor. Feedback was collected regarding the game and design. Their improvement and ideas were collected and dislikes noted. Furthermore, it was important to test their understanding of the game interface for playing since there are no instructions at the beginning. The feedback was collected through observations as they played and through questions that were answered directly while interviewing all of the participants together to encourage each other to respond thoroughly. Changes The following changes were made due to the feedback.
A female avatar (Sami) was added as suggested by most participants in all evaluations. The player chooses the avatar he/she wants (Figure 6).
To personalize the avatar, more color and texture options were provided for the T-shirt selection. The player chooses between pink, blue, white and polka dots.
The progress-bar constituted a source of misunderstanding since it does not indicate progress but performance. Since the performance coincides with speed, the caption was changed to "speed X m/sec", where X denotes the current value. The bar now visualized the speed and is easier to interpret.
Since the majority of the participants had requested a change in the instructions panel, the original long text instruction was replace with two example buttons that the players can choose to listen to as often as they want. On clicking the examples buttons, an audio is played reading this example. Some participants had difficulty to read both the sentence and its question. The question was moved directly below the sentence at the top of the scene. The light green color previously employed was replaced with a much darker version to increase visibility, taking into account color blind issues.
Additions In addition to the above changes, new ideas were added.
Getting the players to feel the game and to know the story behind, it is very important to increase motivation and enhance perception. A skip-able introduction supports the story after the player chooses his avatar. Here, an exhausted Sam(i) is shown with Henry. Sam notices that the player has arrived on the scene and asks the player why it took so much time to arrive. Afterwards, the avatar introduces himself and that he is helping Henry the chicken to learn German but he is really tired from all the running and Henry’s mistakes. So he asks the player to help make Henry learn and win. Notice the indirection built into the story of whose performance is on the line. The players themselves are here to help Henry the chicken rather than learn themselves. The intend is to lower the pressure on the player. This approach has the potential to create a relationship with the avatar or Henry during a joint effort to be accomplished to increase the motivation and engagement for the game (see also Figure 7).
Furthermore, as suggested, we have added the "Chicken Dance" as it is popular worldwide to most of the kids. This animation occurs when the player gets the correct answer several times. We have also added a part of the song to it. This animation plays fast so that makes it funnier too. The fast speed is due to short timing between player’s choices in game. This animation is also accompanied by a facial expression of a wide smile to express great happiness to motivate the player (see Figure 8).
The avatar’s idle status was substituted with a running backwards animation that indicates that the avatar is running with Henry. This animation increases the fun factor because Sam running backwards looks hilarious, especially as Henry speeds up. DATA ACQUISITION Two versions of the game were deployed as browser games with and without the avatar. Each version of the game is followed by a questionnaire. The research question is whether the avatar will lead to longer playing times and an increased return rate to play more. The presence of the avatar would have a positive impact on learning, assuming longer play-time improves performance.
Data The data acquired for analysis were from the two surveys and from game logs. Each sentence and word that the player encountered was logged along with correctness. The participants had different German background knowledge. The participants who had more than average German knowledge were 50% in the without Sam evaluation and 62.5% in the with Sam evaluation. The number of log files are less than
Cognition
It has been shown in various science fields that learning
through games boost the educational benefits in term of
enhancing the learning progress [
I make less mistakes in writing when I get good in this game.
I talk to others about the questions in this game.
Flow State
Flow is initiated when an individual reaches a state of effortless
concentration and enjoyment and is exceedingly productive
while feeling happy. It happens when someone’s skills are
completely focused on winning a manageable challenge [
The questions are designed to capture flow with the following survey questions by asking about the four states of ability and difficulty, in other words, boredom, stress, ease and difficulty and normalizing it against their feeling of fun. number of participants because some participants encountered internet connection problems and some closed their browsers quickly that the logs were not posted. The score and time was computed from the log files.
ENGAGEMENT EVALUATION
In order to understand engagement, a survey was designed
according to theoretical derivation of questionnaire sections
as described further in [
Basic Needs
Basic needs are the essential fundamentals of the human
beings for survival and well-being and an important prerequisite
for learning [
Self-Efficacy
A person’s self-image is a result of combination of some or all
of the following four factors. Self perceptions also depends
on how other perceive us and our own interpretation of other’s
opinion [
Were you good with upper and lower case before? Affect of Game Liking the game details is important so that players choose to play this game. So we asked about the agreement of the statements in Table 2. And at the end of the surveys, we added an open paragraph question to let the participants add their comments and suggestions about the game.
I wonder what Sam(i) wants to do next.
I was able to help Henry.
Sam(i)was happy in the end.
Sam(i) is funny.
Did you have the same feelings as Sam(i)? Sam bothers me while I play.
Sam helped me. Engagement Results For both games, the results are depicted in Figure 10. Games motivate children more than regular class time. The only answer where both groups differed significantly (p=.0382), is the question whether it’s ok to make a mistake in the game. While the group with Sam was indifferent, the group playing without Sam said that its ok to make mistakes with an average of 2 on the Likert scale. In contrast, making mistakes in the classroom was not significantly different from either group or when compared to the game-play within each group. Answers to the Affect questions lie between 2.3 (I was able to help Henry) and 3.0 (do you want Henry to win?) The set of questions regarding the flow show similarities between the two groups. But the results also show that both games seem to produce a sort of flow. The game was neither too easy nor too hard. It was not stressful and it was not boring. The players feel they improved while having fun and would play again. After analyzing the results of both surveys, it is clear that the participants like both versions of the game. There seems to be more worry regarding making mistakes in the version with Sam. On interviewing children, why they were worried more in the game with Sam, it seems that they cared when Sam got sad and wanted to avoid his sadness.
PERFORMANCE EVALUATION Computing Performance Computing performance can be tricky in a game. In this version, the student has to actively pick words that should be capitalized. Doing nothing should not be rewarded. Table 3 summarizes how speed and performance is computed. While speed depends on the correct and wrong active picks the student makes, the correct performance is computed based on the correct identification of nouns only (ignoring correctly not picked items).
noun not noun noun (picked) speed up +20 Word turns green Word is capitalized performance = picked/nouns[30] slow down 20 Word turns red
achievable in idle word remains white Performance Results Figure 11 shows difficulty vs speed vs correct pick percentage. The red line represents the speed, the green line represents the sentence difficulty (as defined earlier) and the blue line represents the % correct pick of nouns. The graph indicates what we call peaks of performance. We note the following: increasing speed produces dip in performance that can be recovered increasing difficulty produces dip in performance and speed that is recovered but more slowly
Duration of Play Based on the log files, the play time can be computed as summarized in Table 4 it can be seen that players with Sam have a tendency to play longer. Comparing the differences of the two distributions a p-value of 0.0941 (> .05) is not quite significant for a 95% confidence interval. Clearly one of the problems is the small number of trials but there seems to be a tendency. According to the data, all players got faster in speed while playing. High speed results in difficulty of handling the mechanics of the game and thinking about the cognitive task simultaneously. This leads to one of three possibilities: The instruction panel pops up in case of continuous decreasing performance that leads to zero valued speed and then player restarts the run (A).
The player continues, gets better and faster again (C).
Sam 3.4 2.6 11.3 EFFECTS OF REPEATED PLAY It is important to see how repeated play affects performance. Additional logs of multiple play for three players are observed. Players 1-3 are girls from different international schools in Cairo, Egypt in grades 2, 3 and 4 at ages 8, 9 and 10 years old, respectively.
Data Description Each time the performance peaks (as observed and explained in Section 7.3), it is computed for the given difficulty level. Figure 12 depicts the amount of peaks by level for each round of play. Figure 13 shows the duration of play with each new round of play. It can be seen that each round of play leads to a longer session. It can also be seen that with each trial, the child has been able to reach a higher level of difficulty. The players were able to pass the point that they had been “stuck" on in a previous trial and continue on to more difficult sentences with new performance peaks. In Table 5, the normalized numbers of occurrences for each category are computed by dividing the number of occurrences of each by total number of words seen. Given this data, both versions show instruction panel the same number of times. More often participants gave up in the game version without Sam indicating a higher engagement and motivation with Sam. Moreover, the comparison of the numbers of occurrences of players recovering from the setbacks (C) supports their endurance with Sam at a much larger rate. Learning Curve As an example of a learning curve, Figure 14 plots performance vs. words seen for Player 2. The blue line represents the first game run, the red line represents the second game run and the green line represents the third game run. The graph is plotting the correct pick performance vs words for the three game runs. The learning curves for the other two players have similar behaviour. From the learning curve exemplified in Figure 14, it can be seen that repeated playing results in improved performance and time of play increases. CONCLUSION AND FUTURE WORK Results are preliminary studies on a first prototype of the game that serves as a data exploration effort to gain clarity on log file collection and interpretation. A large number of players is needed to show the impact of the learning content on skill acquisition and improvement that transfers into writing skills. We plan to deploy the game with undergraduate students and school children learning German for this large-scale analysis. ACKNOWLEDGMENTS We would like to thank all the children who have participated in the preliminary evaluation of this game and Sam(i) the avatar.