This paper describes a short-term study of employing learning games in a German school setting. Games focusing on long and short vowel identification are chosen from the suite of iRead Project games and presented to groups of first and second graders. The children are presented with a game of identifying long vowels seeing around 25 words in 1-2 minutes once a week for four weeks. Their writing skills are recorded before and after the intervention. Recordings made during game play are used to analyze their performance during game play with respect to correctness and speed. Reading skills practiced in the game are not always transferred to the children's writing skills. In general, the impact was greater for first grade children. Our findings will support future study designs for how games can be most supportive in classrooms.
INTRODUCTION The origins for this paper lie in the question that poses itself naturally when taking the first steps of digitalizing the classroom. Digitalization has many meanings. In our context, we wanted to look at how learning games can be used in the classroom. In this publication we would like to share the learning path that we took but more importantly share our findings that will help in understanding where the gain of games in schools can be and how they can be employed in the classroom. The data and their evaluation or analysis can be considered a preliminary study with strong indications of which research directions should be pursued to see if the findings can be supported. As a result, we can strengthen the effectiveness of the games in the classroom and gain a fuller iRead is part of a project that has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No 731724. The results presented reflect only the authors’ view and the Agency is not responsible for any use that may be made of the information it contains.
Copyright © 2019 for this paper by its authors. Use permitted under
Creative Commons License Attribution 4.0 International (CC BY 4.0).
In: J. Arnedo-Moreno, C.S. González, A. Mora (eds.): Proceedings of
the 3rd International Symposium on Gamification and Games for Learning
(GamiLearn’19), Barcelona, Spain, 22-10-2019, published at
http://ceurws.org
understanding of why and where they should best be employed.
At the moment the German government is pushing into the
direction that schools should start using computers and mobile
devices in schools. For German language games applicable
for school use in elementary school, not too much is known on
how to choose educational games and how teachers effectively
“orchestrate” their use within the classroom, a persisting issue
in the field of serious games [
IREAD PROJECT DESCRIPTION The iRead games are developed for L1 and L2 learners of English. In addition the games are ported to German, Spanish and Greek.
Game Content There are a variety of different game mechanics provided by the iRead game collection, such as sorting (combining letters to words) or decomposing (cutting of prefixes), choosing the correct answer among 3-5 options (see for example Figure 1) or true/false (see for example Figure 2). Some of the games have time-limits. Each game mechanic has several graphical realisations. For example, multiple choice is visualized as choosing a path across water, picking the correct rail-road track or moving the correct article to construct a path. The content is automatically supplied to the game using a rule-based word selection mechanism. The difficulty level is informed through a domain model that sequences a range of language prerequisites according to the school curricula. Words are presented in a sequenced manner of difficulty at the learner’s level of phonology (the study of grapheme-phoneme correspondence), morphology (the study of prefix and suffix recognition and usage), and syntax (the study of endings within sentences, such as conjugation, and grammatical knowledge that requires syntax analysis).
Users The game is built to support reading acquisition in L1 and L2 setting for young school children within the classroom setting. For the L1 setting, the applicability of the games to dyslexic children is also studied. The games are developed for Greek, Spanish, English and German. The iRead applications are deployed in the classroom setting to support teachers with their work. The teacher makes the final decision on how to incorporate the games and eReader. For example, the teacher can give learners the option of playing this software program or read on the eReader as two of several “learning stations” during a free study period.
Points and Feedback Within a mini-game, points and penalties are given for correct and incorrect input. Feedback is therefore immediate when performing each particular task. If the player makes mistakes, specific hints can be given to help rethink the problem in a second attempt. The hints are task specific. For example, the learner sees the words ‘Schule’, ‘sehen’, ‘spaten’, ‘stehen’, and ‘sagen’and the learner has to click on the words that contain the sound /sch/. When the user selects an incorrect word, the system gives the hint “the letter <s> can make the sound /sch/ when it comes before <t>”. Then the user is given an additional time to select the correct words.
Subset of Games chosen For the purpose of this study, two representative game mechanics were chosen because they provided the maximum content within the shortest amount of time. Their mechanics are (a) 20 23
pick the correct answer out of an average of 5 choices without time-limit and (b) true/false with time limitation. Each game provides about 25 words in one playing session of about 1-2 minutes. The automatically generated content for the games was chosen according to the explanation in the next section. DESCRIPTION OF STUDY The set up of the study included four visits to a classroom. 10 classrooms participated in the study but for the purpose of this paper, only two were transcribed and evaluated. The numbers are listed in Table 1.
A Tiny Introduction to German Orthography
A “Trochee" is a metrical foot consisting of a stressed syllable
followed by an unstressed one. This word pattern forms the
basis of the core German word and its orthography [
The words are chosen from the childlex [
The words are represented by a picture and a sentence with the missing word to support correct word choice.
The spellings are analyzed for the two features that were trained with the game (see above).
Meta-data for each child is included in the data elicitation. However, this information is not relevant for the purpose of this paper.
Visit 1 The first visit starts with a small 1-1 introduction of an orthographic concept for each participant for the game play. The graphemes <ie> vs. <i> and their function within a word is explained. Examples of minimal pairs are given, which the children are asked to read: “Riese" (giant) vs. “Risse" (crack). In the beginning, with few exceptions, all children will read both of those words as giant. They mostly agree though that these have two different meanings and that it would be useful to be able to distinguish these while writing. “Wiese" (meadow) vs. “Wissen" (knowledge) is used as a second example. The second of the double consonants is named the magic letter (as it also cannot be heard - in analogy to the silent-e in English). No mention of the other vowels is made during this introduction. We do ask the kids to sound out what a short /I/ would sound like. This is a new task to them, as they are only taught the long sound of the vowel. This introduction takes only five minutes, after which the children are asked to play their first game where they are asked to identify the long “ie” in one of the presented words. The game takes an average of only two minutes. The interface is shown in Figure 1 and is the same for Visit 2 and 3. Distractors are short vowels with double consonants or <tz>. As the kids play, they receive feedback from the game: Correct words are turned green on the chosen stone. Incorrect words turn red and the avatar jumps back to the previous stone. After a second wrong answer, the avatar automatically moves to the correct stone. An oral feedback is given by the system, saying that the short vowels can be recognized because of the double consonants.
Visit 2 and 3 The next two visits, the children are playing the game with different content. Whereas previously, they had to only find the long “ie", these games now include all the long vowels. Because the oral feedback after a mistake is barely audible with the music, the system feedback, that a double consonant denotes short vowels, is repeated by the researcher sitting with the child. We also remind them, that spotting an <ie> is the 2Analyzing the performance differences between these two groups is another study and out of scope for this publication easiest way to recognize a long vowel. The child is reminded that the computer could be making a mistake also. Often kids will be excited to recognize these mistakes. A screen recorder records all sessions again.
Visit 4 During the final visit the children will repeat the game from sessions 2 and 3. During this game they are asked to tell the researcher how they know whether they did something correct or wrong during the play. Next, they are allowed to play a fast game if they wish, which they all did. In this speed game, limited time is available for the player to choose words with long vowels. A screenshot of this game is provided in Figure 2.3 Post-Visit A post-test is distributed to the classroom. Different words following the same patterns of word selection are presented to the children.The spelling errors with respect to the features practiced in the game are then analyzed: Correct spelling for short and long vowels using the “magic” consonant and the correct usage of <ie>. 3Analyzing results from the fast game is beyond the scope of this paper. 100% 88% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 63% 100% 100% 100% 100% 97% 9% 100 100 0 100% 63% 100% 86% 88% 100% 100% 78% 100% 86% 100% 100% 60% 73% 100% 100% 78% 100% 100% 86% 100% 100% 100% 100% 100% 100% 71% 100% 100% 100% 43% 57% 86% 100% 100% 100% 90% 90% 17% 14%
Grade 2 78 55 74 62 23 19 86% 88% 100% 100% 100% 100% 70% 100% 70% 89% 83% 93% 100% 33% 100% 83% 91% 87% 17% 90% 100% 100% 100% 100% 89% 100% 80% 93% 40% 100% 100% 100% 100% 100% 100% 93% 15%
RESULTS - GAMEPLAY Each of the play sessions is recorded with a screen recorder.4 The recording is then transcribed according to the example given in Table 3.
For each trial/visit and each step in the game, the list of words the child has to choose from is transcribed. Additionally, we record which word was chosen and whether this was correct. In addition each step time is recorded. As a result, we are able to compute the % correct, the number of words a child sees in the game, and how long each decision takes. Since interviews interrupted the timing in the fourth session, these times are excluded.
Results for each of the children are recorded in Table regarding their performance in terms of %correct choices taken, no clear trend is visible as can be seen in the example listing for Grade 1 in Table 2. What is remarkable however, is that while they often answer correctly, the main difference between the players seems to be their increase in speed. Figure 4 shows example timelines for a child from first and second grade each across all four sessions.
While the time between moves does not vary significantly across the two grades in the study, there were other differences. Separating children in each grade according to their performance on the written test, some significant differences in 4AZ Screen Recorder for Android speed could be found. N Children are selected who fit best into one of each of these three groups: children with large changes in pre- and post-test (N = 3), those who remain weak (N = 2), and those who were strong from the start (N = 2). Since the first game ensures that the game mechanics are known for the second trial and the fourth trial was interrupted with interview questions, we compare time until next move for second and third trials only. The resulting distributions of speed for each group are shown in Figure 5.
By conventional criteria, the difference in time-play for the group with largest performance change in their written tests is considered to be very statistically significant, where N=21 moves (2nd trial), and 17 moves (3rd trial) with a p-value of 0.0015. There is no change in distribution of response time for the low performing children with no improvement in the final test. Even for those children who were already good in their orthographic abilities, the change in speed is significant. The two-tailed P value equals 0.0451, considered to be statistically significant, where N=12 moves (2nd trial), and 13 moves (3rd trial). For Grade 2, only the change in speed of the top performers is statistically significant (p=0.0231). Speed relates to automaticity, which in turn should facilitate actively using the skill in other tasks. This relationship might be visible in the data.
KNOWLEDGE TRANSFER
Theory On Writing
The analysis of lexical patterns in language acquisition are
considered to be a complex procedure [
Correct reading ability is the precursor to reading practice,
which in turn creates better readers (through self-teaching)
and probably better writers (to be studied). Therefore,
measuring progress in reading ability and training misconceptions
is absolutely essential in the early phases of reading
acquisition [
It remains to be seen how skills acquired during game play might transfer to reading and writing abilities. This is the goal of the next sections.The present study makes no claims on cause and effect but reports the data that was collected. Given the findings however, it seems that there is a clear need to study this in more detail as there are indeed very strong indications that practicing with the games has an impact on reading ability of long and short vowels.
MEASURING WRITING ABILITY The pre- and post-test were transcribed from the tests according to what the child wanted to write vs. what they actually wrote. Most of the time, the word corresponded to the target that we had aimed for with the sentence or the picture. When this was not the case, the target consonant duplication and the <ie> target number of occurrences were changed accordingly. Next, the errors were counted and normalized by the amount of potential occurrences in their resulting list of words. Based on each child’s own performance gain, three groups can be distinguished. Those that have improved, those that remain low performing and those that were already high performing in the pre-test. (These are the groups that we used to look at the speed change while playing.) Table 3 reports the overall improvement in the results for both error categories. It can be seen that the largest impact was on <ie>, which was explicitly targeted on our first visit.5 There was also a barely significant improvement in the double consonants for second graders.The teacher of this classroom started to introduce the concept to the children after our posttest was finished.
Rather than looking at the overall average of the classroom, Table 4 counts how many of the kids had significant changes of larger than 20% in each of the areas. In second grade almost no-one can improve on <ie> since the topic was mastered by the time of the pre-test. Regarding the double consonant, it can be said that about half the classroom has improved their score by at least 20%.
MEASURING READING ABILITY When first starting to use the eye-tracker to find out whether there were differences between children in reading ability, we went through several rounds before focusing on a couple of features that seem to pin-point their ability very simply. The 5To put these results into perspective: In our study another first grade went from 8% to 75% while the control-group in the same classroom went up to %50, another classroom went from %12 to %26. key to our break-through in measuring reading ability was firstly, to ask the children to read out loud so we could observe whether they were reading the words or skimming. At the same time, this slowed them down, which was good for the eye-tracker.
Our hypothesis was experience driven. Children learn that German orthography is shallow, which means they are only taught the long pronunciation of a vowel and they will sound out the letters as they are written: /mUtEK/ (Mutter=Muterno meaning) and /ROsE/ (Rosse=Rose-different semantic) and /wIsEn/ (Wissen=Wiesen-different semantic). Semantically mapping the word “Rosse" (horses) to “Rose" (rose) or “Wissen” to “Wiese” will cause problems for the reader and cause the reader to backtrack multiple times and finally guess which word would make sense and look similar. A sentence like “Wir Wiesen nicht wo die Rose lang laufen” (“We don’t meadows where the rose are running” will take some time to decode. Since long and short vowels are quite frequent, struggling on every sentence for meaning, will not create readers. We expect readers with no training on double consonants to be faster with L1 words than with L2 words since they have not understood the concept of the magic consonant.6 However, if they understand double consonant concept (or have memorized the words), then L2, since it has a short vowel should be read faster. (This only works for child readers who are still decoding as they read. There would be no difference for experienced readers here.) In the first data collections, we found that it was hard to find any trends separating strong from weaker students. The breakthrough came, when we separated high-frequency (HF) words from low-frequency (LF) words. Low frequency words are unlikely to have been memorized by students. Therefore, they lend themselves well for separating the frequent readers (memorization) from those that understood the concept (generalize to new words). This concept resulted in four categories of words, namely L1-HF, L2-HF, L1-LF, and L2-LF. We found that the reading speed is comparable within these four word-groups for a given reader. In other words, picking out a specific word in a text could represent the skill for that group. Readers that were fast with the HF words, are expected to be frequent readers. If they were slow with L2-LF then they have not understood the pattern but did well with memorization on the L2-HF. The best readers would therefore show shorter reading time even for L2-LF because it shows they are able to generalize to unseen words, which improves reading (comprehension).
Figure 6 shows a prototypical distribution of the speed for
all four categories for an example child from first grade in
each of the three categories, top performer, low performer
and most improved (according to the orthographic exams).
It can be seen that those children from the “most improved”
group are in the process of learning. The HF words are not
6According to the Phontasia method, the easiest words to to read and
write as they have bidirectionally the highest correspondence between
phonemes and graphemes are 2-syllable words with long vowel [
Unfortunately, we do not have a pre-test for either group before the game play. We were only able to extract words from three of the four categories from Grade 2.7 Figure 7 plots the distribution over all children for each of the four words chosen to represent the four categories. L1-HF: “leben” (to live), L1-LF: “jagen” (to hunt), L2-HF: “können” (to be able to), L2-LF: “Welle” (wave), and L1-LF: “Feder” (feather) was used instead of “jagen” from the other text read by Grade 2.8 Both eye-tracker visits took place after the fourth visit of the study. In the diagram, we can see some overall tendencies looking at the total fixation duration. The first graders are comparable to the second graders. We are missing the L2-LF unfortunately, but in general, HF is faster than LF. The first graders performance for L1 on HF and LF are almost the same but in L2, they still have more difficulties parsing the LF words fast.
In future, it will be necessary to test the reading skill beforehand. However, seeing that the first graders are as strong as the 7We had to change the text to ensure that words from each of the four categories appeared. The original text was meant to be easy so it specifically did not contain rare words. 8Multiple trials with word choices for these categories have shown that they are pretty much easily exchangeable within category and exhibit the same reading behavior. second graders seems unusual and may or may not be the effect of the game training and should be verified across schools and classrooms. This data was compared to a collection from another event (EFFEKTE) where we tracked 5 children from first grade. For the Effekte group L1-HF (leben) was faster than L2-LF (Welle), indicating the struggle with the double consonant on unknown words. In contrast the Grade 1 from our study was slightly faster on L2-LF (Welle) than on L1-HF (leben), which is what we want to see. These are tendencies and not statistically significant and can be coincidental. Further studies are needed with more data. While these were explorative studies, the results give us a clear direction on how to set up a good study next year.
CONCLUSION AND FUTURE WORK This paper reports on some results for a study that was performed when going into schools with games that are designed to practice and not teach concepts regarding the German vowel duration. We were able to look at aspects of game play such as accuracy and speed. The transfer of children’s performance on orthographic and reading skills was studied. The learnings and the results are very useful for re-designing parts of the study for next year. Our impression from working across 10 further classrooms with a similar approach including Grades 1-3, the combination of teaching and practice with these games can be very powerful and the choice of grade when to apply this particular content seems to have a huge impact on the speed of learning. Due to the immediate feedback of misconceptions in the games as well as the large amount of automatically selected unique words shown to the children, the games reenforce pattern building over memorization. This seems to help some children to generalize to low frequency words both in spelling and reading. While we only spent 6 minutes across four weeks with the children, we could already show tendencies of positive impact and we can only imagine what might happen when regular use of the game enters daily classroom routine. This will be our task to show over the final year of the iRead project.