Designing a comprehensive evaluation method for learning games - a general approach with specific application to iRead

Abstract

The iRead project is developed by a Europe-wide consortium. It contains games that support reading acquisition for elementary school children in both L1 and L2 contexts. It is of interest to understand child motivation during the use of the games. In order to plan for the evaluation, the games are analyzed with respect to frameworks that are presented in the literature. To concentrate on the motivational aspects, needs are derived from a spectrum of psychological and learning theories. In addition, a model of motivation in work environments can be used as a foundation to relate game design elements to a larger psychological framework. The resulting analysis should provide a general guideline on how to evaluate game design decisions with respect to motivation. It will serve the project as a first step towards evaluating iRead games when they are deployed in the classroom. 3

Keywords

• Gamification
• Motivation
• Psychological Need

Introduction

Games for learning should concern educators because motivation to learn particular school-based material can be enhanced by wrapping the content into a context that is otherwise not possible in a traditional classroom setting. Digitization supports individualized learning progress and it constitutes an easier way to provide immediate formative feedback. In addition, providing an aesthetic learning environment is something that game designers are specialized in while schools settings are generally not designed for the same.

When employing games to transmit academic content, the question usually posed is whether the acquisition of that content was successful, or more specifically, more

iRead is part of a project that has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No 731724. The results presented reflect only the authors’ view and the Agency is not responsible for any use that may be made of the information it contains. successful than traditional teaching of class material. Recent work though has documented that an evaluation is much more complex in that it should involve among other dimensions, player types, motivational settings in addition to learning achievements. Maximizing learning achievement is not necessarily the primary goal of a game. Motivating a student to re-engage with academic material may be a more valuable outcome. As such, the game prepares the student to learn a material later or outside the game. Such a positive outcome is neglected by the pure measure of academic achievement within the game. It is therefore important to study the appeal of the games not only from the academic point of view but also from several other aspects. In order to do this properly, recent theoretical advances in research have provided us with several frameworks to use as the basis for analyzing an educational game. Based on a detailed analysis of a game within that framework and a deeper understanding of motivation and performance, a more intricate evaluation plan for a game can be designed.

Foundations of Game-Based Learning by Plass, Homer and Kinzer [ 26 ] provides a model with a comprehensive view encompassing affect, motivation, cognition and socio-cultural aspects of the game as it is embedded into its application environment. Locke [ 19 ] provides an integrated model of work motivation that can be used to understand further why someone would engage with the ”magic circle” of challenge-responsefeedback that the game uses to engage a player. Assuming that a child in an educational setting thinks of the game as ”work”, this model may be appropriate for the purpose of evaluating educational games and, specifically, iRead, which is explained further below. The goal of this paper is therefore not to justify why games are a useful or good environment for learning, as this is already sufficiently documented in other research.Rather, the paper explores a series of frameworks that allow us to evaluate the game as part of an environment including the viewpoint of the player/learner along a continuum of learning and motivational theories. iRead is a 4-year (2017-2020) project that is aimed at developing personalized learning technologies that support reading skills. It will be evaluated using the chosen frameworks.4

The rest of this paper will discuss each of the two chosen theoretical frameworks for our purpose in Sections 2 and 3. Section 4 describes the iRead applications in the context of these two frameworks. Based on the analysis of the game and the frameworks, an evaluation methodology is proposed in Section 5. Finally, Section 6 points out some of the conclusions from the analysis and previews some of the findings we expect to see during the evaluation study. 2

Game Design Elements - Games Framework

Plass et al [ 26 ] have developed an integrated design framework in order to study games in learning environments. This model takes into account an encompassing set of perspectives in order to gain a full understanding of the number of variables that are involved in the process of playing and learning in the interaction with a game. The stated goal of Plass et al. [ 26 ] for their model is to support research to ”provide game design patterns. 4 iRead is a 4-year EU Funded project (January 2017 to December 2020). This project has received funding from the European Union’s Horizon 2020 innovation program under grant agreement No 731724. The model takes into account the cycle of challenge as input to the player, their output or reaction and the feedback that is followed by the next challenge/input to the player.The goal of this section is to present a brief review and explanation of their framework in order to apply it to the iRead game environment. 2.1

Game Design Elements

Game design elements include generally speaking the following items [ 26 ]: Mechanics, visual aesthetics, narrative, incentives, musical score and content (learning objectives). Most of these items are self-explanatory. Two items need further explanation. Incentives: Incentives depend in many ways on the motivation of the learner/player. Incentives interact with the disposition of the players and their surroundings. They are influenced by the player’s ability to choose goals and are related to their self-efficacy. Incentives cover the continuum of internal to external motivations that influence the players. The incentive transcends the game and becomes part of the players surrounding to gain meaningfulness and relatedness [ 30,8 ]. Section 3 will expand on this notion with a more complex model that will be useful for our purpose of evaluating iRead. Learning Objectives: The section of content and learning objective deserves additional thought. The usual way of evaluating a game with academic content is a comparison of academic achievement between a group using the game and a control group that is taught in the traditional way. However, the model introduced by Plass et al. [ 26 ] and Brom et al. [ 3 ] show that it is important to refine this component by looking at specific goals or usage of each teaching game.

Plass et al. [ 26 ] distinguish several learning outcomes, namely: – Preparation of future learning – Introduction of new knowledge – Practicing of already introduced knowledge – Development of 21st-century skills such as teamwork, collaboration, problem solving, creativity, communication, etc [ 27,17 ]

Depending on these intended or unintended goals, the evaluation of the game under consideration has to include measurements for each of these dimensions. A simple measurement of improvement in the learning outcomes will not cover this entire spectrum of possible effects and therefore miss essential effects. 2.2

Pillars of Engagement

Why do players engage in the magic circle of challenge, result, feedback that games with particular game design elements offer. Plass et al. [ 26 ] offer four pillars of engagement in their model that influence and are influenced by the game. These five areas are briefly reviewed for the purpose of developing the evaluation study in the next section. Affect: Care has to be taken that the engagement level of the game does not result in cognitive overload that may hinder the learning objectives [ 4 ]. Emotion can however, influence learning in a positive way. Visual design of learning materials can impact learner’s emotional state and can enhance engagement [ 25,14 ]. Music and story-lines can add to affect. Emotion and Flow [ 7 ] are different effects, which brings up the motivational pillar.

Motivation: Motivation takes place along a continuum of intrinsic and extrinsic factors. Incentives offered (in games or classrooms) reflect either the behaviorist models of the player/learner or include a more constructivist and cognitivist model in the reward systems offered. Eccles et al. [ 11 ] as described by Plass et al. defines three questions that cover the complexity of motivational drive of the player: Can I do this? Do I want to do this, and why? What do I need to do in order to succeed? The various theories at the confluence of motivation are listed in Plass et al.5 Important points include the relatedness of the task to the players own goals and self-concept that are reflected in those questions and create the connection to intrinsic, meaningful goals. Section 3 will go into more detail of the complexity of motivation. The aspect of flow (see Section 5) plays an important part in improving learning outcome [ 3 ] in this construct relates to adaptivity, a cognitive aspect that is summarized next.

Cognition: Several aspects of cognition are relevant to the motivation of the player. – context of skills application – skills meaningful outside of game and transferable – scaffolding through personalization – formative and immediate feedback – content representation – mechanics aligned with learning goals – mapping gesture to features of content

Given the concept of cognitive overload by Mayer et al. [ 20 ], design of games should be careful not to interfere with the learning, and so extras that do not relate to the learning context should be avoided. Instead, ideally, game mechanics could be equivalent to learning mechanics. Examples given are typically physics games where the angle and the impact of angle calculation is a direct mechanic inside the game as it is in Angry Birds. A mathematical equation solved incorrectly will result in an airplane that crashes as it runs out of fuel. Such overlap may be more difficult in language learning. Here, affinity spaces become more important, where the goal of winning is pursued outside of the game [ 32 ]. In contrast Multi-user text-based Dungeons include ”speech acts” within text in order to act on and interact with the world and receive immediate feedback about language correctness within the game itself [ 31 ]. An example of transcending the boundary of the game into reality is a discussion between children about an orthographic concept in order to finish the level successfully without making a mistake that would catapult the team back to the beginning of a level.To capture this part of the game that takes place outside the platform, socio-cultural aspects are covered by the described model. 5 https://www.researchgate.net/publication/293328392_Foundations_of_

Game-Based_Learning/figures Socio-cultural: The final pillar of engagement leads to an often forgotten component of the world surrounding the game itself. While the social aspect of games is obvious within multi-player games, it exists for all genres of games in the interaction of two players sitting in front of a tablet game and interacting with it. It also includes the affinity spaces, where games are discussed with other people, including friends. Leaderboards that indicate the presence of others can motivate or demotivate players depending on what has motivated them to play [ 22,29,13,24 ]. It was shown that even watching a game can help the spectator learn [ 9 ]. The environment of the game most likely has an influence on the players’ preparation for future learning and transfer according to Plass et al. [ 28,2 ]. 3

Motivation - Work Framework

Motivation is the reason why a player engages with the magic circle offered by the game. The aspect of motivation deserves further detail because it is influenced by environment, self and an envisioned ideal self [ 10,23,18,30 ]. In a school setting, children may engage with a learning game because the teacher requires it, because they enjoy the game mechanics. They might also play in order to become better students. Locke et al. [ 19 ] explain the motivational concepts in a unified framework within a work environment. The framework is comprehensive in taking into account a person’s environment and their changing attitudes.6 Even though there are significant differences between work and school setting [ 21 ], there are also similarities [ 15,5 ]. The model’s components translate directly to a classroom setting as described next. In this section, we outline some of the aspects that seem important to an evaluation study of the iRead applications and that should inform the choice of parameters to be measured. The diagram can roughly be separated into three major parts, the player themselves and their background with which they come to school, the magic circle providing the interaction with the learning material or game, and finally the environment in which the magic circle is located, in our case the school.

Players: Various theories combine to explain some of the relationships but not all between the various aspects. Rather than explaining each of the theories, we are interested in looking at each of the aspects (needs, values, personality, ...) and how they are relevant for the games deployment and evaluation with the viewpoint of school as a work environment. Needs are the ”objective requirements of the organisms survival and well-being” and are specific to the learning environment [ 6 ]. Experience has shown that learning is difficult in an environment where failure has terrible consequences [ 1 ]. Basic needs for a learning environment can therefore be interpreted not as the need for food and water but basic safety: failure is not a public embarrassment, does not have devastating consequences and is accepted as a normal pathway to learning. One of the strengths of games is that they allow for ”graceful failure” or ”fun death”. Most importantly, the player is allowed to repeat a level until they are happy with their own performance. Even partial failure then can be overcome. In contrast, traditional classrooms 6 for space reasons: https://www.researchgate.net/publication/270345814_ What_Should_We_Do_About_Motivation_Theory_Six_Recommendations_for_the_ Twenty-First_Century/figures often provide only one-time summative evaluations without recourse to improvement. Values and personality reflect the student’s internal attitude towards the material to be learned, including also their family values that are transferred into the classroom [ 12 ]. Engaging with the magic circle: Elementary students are able to set goals for themselves. In the example above, a student can choose to work on improving a particular skill. However, this presupposes that the student has a clear understanding of the deficiency and how to fix it. This relates to the three questions in Section 2 addressed by Eccles et al. [ 11 ]: ”What do I need to do in order to succeed?” Evaluating the correct deployment of a learning game in the classroom should therefore take into account whether there is a connection visible for the child between the goal of improving their performance in class by using the game.

Incentives then relate to the second question: ”Do I want to do this, and why?” The question is then which incentives are provided by the environment, the parents, the teacher, fellow students or through the game itself or the desire to reach a self-proclaimed goal (self-regulation).

The question ”Can I do this?” relates to self-efficacy and whether the students perceive themselves as able to reach the goal or incentive. Collected data should help answer questions like: Does the game support the feeling of self-efficacy? Does the feeling transfer to regular class work? A higher feeling of self-efficacy is related to higher accomplishment in the skill to be achieved [34].

A classic situation for false attribution by students in learning environments is for example that high achievers perceive that effort was the determinant of positive outcome in achievement, while low effort would result in failure. Students with low achievement perceive that outcome is not as much influenced by hard work but their lack of ability [33]. The use of avatars in games are known to project a positive image of the ideal self on the player that can provide incentive by displaying a positive image of efforts even before accomplishments are reached [ 16 ]. An interesting question would then be to see if the game can promote a positive ideal self for the players. To do this, games provide goal moderators such as feedback, progress bar. Game analytics can provide information on efficacy of the players. This in turn relates directly to performance and outcomes that can be measured within the game as well as outside in a transfer setting provided by regular classroom interaction.

Classroom Environment: How job satisfaction and work characteristics translate into school settings, is an interesting question. However, satisfaction and well-being in the classroom are most likely also determined by performance outcomes as depicted in the model. This satisfaction or dissatisfaction can in turn influence the students’ involvement and engagement in the learning process. If the game contributes to a positive satisfaction of the student then a player may be better prepared for future learning outside of the game. Data should therefore capture the change in attitude of a student in the regular classroom procedures. The intent of this section is to describe the iRead application, the environment (sociocultural aspects), the content (cognition), and its motivational and affective factors.7 Together with the above framework, this will inform the design for the game evaluation to be discussed in the next section.

The games take place in an Egyptian setting of pyramids, oases and desert. There are some 14 different game settings that can be fed with different content specific to the players learning profile. By playing successfully, the content adapts in difficulty. Vocabulary is drawn from the readers and difficulty is informed through a domain model of the underlying language that covers a wide range of language phenomena and corresponding prerequisites in knowledge required to process these. Materials in the e-reader are sequenced on the basis of the student progress in the games while still offering choices of texts to read. Texts are annotated with specific learning points that can be trained with a particular text. When reading the texts, students can maintain a list of difficult words that they have assembled themselves in their readings. The e-reader indicates reading progress for each book. Read books can be seen on a bookshelf, showing how many books have already been finished. Books can be marked as favorites.

The players are young school children learning to read and write. The environment, such as affinity spaces, of a game plays a very important role in order to study the effects of the game appropriately. The iRead applications are deployed in the classroom setting to support teachers with their work. The teacher makes the final decision on how to incorporate the application. Classroom visits and subsequent recommendations are given by the researchers. The projects are also accompanied by researchers and training for iRead is provided to teachers. Social interactions pertain to the interactions between teacher, students and between students. These interactions can take place as students play games together or talk about the games outside the immediate game play. Classroom observations should take these into account in their observations. Teachers have already noted that they would encourage game play in teams.8

Content of the games is based on spelling and grammar. Words are presented in sequence of difficulty from orthography, over morphology to syntax.The content is sequences from easy (phonology) using easier vocabulary to more complex structures (prefixes for example). The player proceeds through the games as the content changes dynamically given the player performance. The content sequencing is managed by an adaptivity engine that runs in the background to help select the correct materials from the database for the specific games. There is a large variety of mini-games within the ”game landscape” to practice different kinds of puzzles from analysis (decomposition) to synthesis (construction), from regular practice to automation through speed, seeing the same content in different settings until reaching a defined level of achievement in order to move on to the next problem/content. Different games are played on the same feature, sometimes for accuracy, others for word-building and others for automation.

Motivational incentives are given through points and penalties for correct and

incorrect choices, showing student progress. Feedback is immediate when performing 7 Due to page limitation, refer to https://vimeo.com/235546593 8 ...though the games are adaptive to the player that is currently logged into the system. each particular task. The player can gain new clothing and dress up their avatar. The avatar does not reflect the skill or performance level in any way. There is an overall storyline that takes the player through the landscape of the game with the goal of solving puzzles in order to save the grandmother. The storyline is not directly related to the skills that are being learned. But the storyline functions as a sort of progress report as the player makes their way through the provided skill sets that are practiced within the mini-games that lead to the solution of the mystery they are solving. The graphics are based on a fun, bright colored adventurer that the children are able to configure themselves. The figure is followed by a muse that buzzes around their head and leads the figure through the adventure to find the grandmother. The graphics all relate to the Egyptian theme with bright sunshine, colors of yellow, blue and green dominating the scenes. A simple music score is chosen that fits the graphics but does not distract from the common theme. 5

Evaluation Study Design

Given the discussions so far on various theoretical model, data should be collected to answer specific research questions. These may be limited by the school setting and the existing format of the game which cannot be modified. The two research questions driving our study are: RQ1: Does the game affect students’ performance and attitude towards the content? RQ2: Are sources of problems with students achieving goals identified?

The used instruments should tap into the constructs of dispositions, environment, motivation, affective engagement, environment and game mechanics.By gathering information about each dimension we can study which variable (learner, environment, game) may explain differences in performance and attitude. The approach combines quantitative surveys, observations, and open questionnaires and triangulated in order to tackle the constructs under investigation. The quantitative survey uses a 6-point Likert scale. Observations and Interviews will focus on the same construct covered by the survey.

Meta data is collected at the beginning of the study: Age, gender, grade, knowledge of language under study and known learning disability. The following questions are aimed at documenting the student’s dispositions such as self-efficacy, attribution, incentives and goals of the student. The answers are gathered in 1-1 interviews with a researcher at the beginning and end of the intervention to capture any changes.9 – Do you enjoy reading? – Are you a good reader? – Do your parents think that you are a good reader? – Do you think it is important to have good grades in LANGUAGE? – Do your parents think it is important to obtain good grades in LANGUAGE? – Why do you want to play this game? (I like games, a change from regular class, I want to be a better student.)

The following questions should determine whether the students basic needs are met within their environment. Furthermore, it is of interest to observe the environment to 9 An example of such a survey can be seen here: https://form.jotformeu.com/ 90796101867364 understand the level of transfer that takes place between game and classroom. Answers are elicited in 1-1 interviews with the researcher.

– Is it ok to make mistakes in LANGUAGE class? – Is it ok to make mistakes in the game? – Do you like to participate in the class during LANGUAGE class? – Does your teacher think you are a good reader? – Does the game help you become a better student? – Does the game help you to participate more in class? – Do you like to discuss the game with your friends in class?

To capture socio-cultural interactions, the researchers will attend classes in which the game is deployed and make observations with respect to discussions between students while playing together or while discussing the game without playing at the same time. The researchers should make a note when teacher-student or student-student interaction contains references to the game in order to explain a concept in the classroom. These observations will document and inform about transfer from the game to the classroom and support for the game from interaction with others similar to affinity spaces. Conversations and mentions should be transcribed when possible by the researcher along with location, time and actors in the conversation.

It was shown that performance is influenced by flow. In order to test for motivation through enjoyment, flow and generalized positive affect, we learn from [ 3 ]. In this paper, flow is used in Csikszentmihalyi’s (1988: 36) sense: “the holistic sensation that people feel when they act with total involvement.” Flow must be measured in the middle of the game, and the student must be disrupted for this measurement to answer questions. The following are additional questions that provide an indication regarding the three motivational questions by [ 11 ].

– Do you like playing this game? – Are you good at playing? – Are you learning CONTENT? – Does the game help you to get better at CONTENT? – Do you find the game too difficult? – Do you find the game too easy?

The following set of questions should inform the researchers about the affective engagement of the player in the game. The players answer these questions by themselves on a Likert scale 2-3 times during the intervention.

– Do you like the story in the game? – Are you making progress with finding grandma? – What do you think about the music in the game? – Will you change the look of your avatar?

At the end of the questionnaire, open ended interview questions include the following: What have you learned so far? How did the game help you with learning? What are you working on currently? What is your reward when you learned something new?

Performance and learning outcomes (retention and transfer) are based on standardized exams of the material that was studied in the content of the games, namely, vocabulary, reading comprehension, and word recognition. Pre- and posttests will be administered.

With the above data, we should be able to answer the following research questions and study the dependencies to any variables like: meta data, dispositions, motivation (including flow), affective engagement, environment and game mechanics. Did the student’s skills10 improve within the game? Did the student’s skills improve on a standardized test? Did the student’s attitude towards learning the skill improve such that it is visible in the classroom to provide for future learning? Has students’ attitude towards reading changed during the course of the intervention? Did the adaptivity work to provide flow for the player? 6

Conclusion and Future Work

This paper presents theoretical background about the context of learning through games by looking at an overarching framework that covers not only the games but also their context. In order to take a deeper look into the components of motivation another metaframework is discussed in the context of school. Finally the iRead game is presented using variables from both models, resulting in a proposal for an evaluation plan to measure the impact of the game on children. The evaluation in turn should then inform future changes to the game and the teaching environment in which the children are learning. 10 With skills, we are referring solely to the academic skills that the games profess to teach. This excludes for the moment so-called 21 century skills, such as team work or creativity and problem solving.

References

1. Berkling , K. , Thomas , C. : Gamification of a software engineering course and a detailed analysis of the factors that lead to it's failure . In: Interactive Collaborative Learning (ICL) , 2013 International Conference on. pp. 525 - 530 . IEEE ( 2013 )

2. Bransford , J.D. , Schwartz , D.L. : Chapter 3: Rethinking transfer: A simple proposal with multiple implications . Review of research in education 24(1) , 61 - 100 ( 1999 )

3. Brom , C. , Děchtěrenko , F. , Šisler , V. , Hlávka , Z. , Lukavskỳ , J.: Does motivation enhance knowledge acquisition in digital game-based and multimedia learning? a review of studies from one lab . In: Joint International Conference on Serious Games . pp. 120 - 132 . Springer ( 2018 )

4. Chandler , P. , Sweller , J.: Cognitive load theory and the format of instruction . Cognition and instruction 8 ( 4 ), 293 - 332 ( 1991 )

5. Cheng, Y.C.: Classroom environment and student affective performance: An effective profile . The Journal of Experimental Education 62 ( 3 ), 221 - 239 ( 1994 )

6. Chew , E. , Jones , N. , Turner , D. : Critical review of the blended learning models based on maslow's and vygotsky's educational theory . In: International Conference on Hybrid Learning and Education . pp. 40 - 53 . Springer ( 2008 )

7. Csikszentmihalyi , M. : Finding flow: The psychology of engagement with everyday life . Basic Books ( 1997 )

8. Deci , E.L. , Ryan , R.M. : Human autonomy . In: Efficacy, agency, and self-esteem, pp. 31 - 49 . Springer ( 1995 )

9. DeHaan , J., Reed , W.M. , Kuwanda , K. : The effect of interactivity with a music video game on second language vocabulary recall ( 2010 )

10. Dweck , C.S. : Self-theories: Their role in motivation, personality, and development . Psychology press ( 2013 )

11. Eccles , J.S. , Wigfield , A. , Schiefele , U. : Motivation to succeed. ( 1998 )

12. Englund , M.M. , Luckner , A.E. , Whaley , G.J. , Egeland , B. : Children's achievement in early elementary school: Longitudinal effects of parental involvement, expectations, and quality of assistance . Journal of educational psychology 96(4) , 723 ( 2004 )

13. Groh , F. : Gamification: State of the art definition and utilization . Institute of Media Informatics Ulm University 39 , 31 ( 2012 )

14. Häusel , H.G.: Brain view: warum Kunden kaufen , vol. 143 . Haufe-Lexware ( 2016 )

15. Holley , L.C. , Steiner , S. : Safe space: Student perspectives on classroom environment . Journal of Social Work Education 41 ( 1 ), 49 - 64 ( 2005 )

16. Jin , S.A.A. : Avatars mirroring the actual self versus projecting the ideal self: The effects of self-priming on interactivity and immersion in an exergame, wii fit . CyberPsychology & Behavior 12 ( 6 ), 761 - 765 ( 2009 )

17. Lemke , C. : engauge 21st century skills: Digital literacies for a digital age (01 2002 )

18. Locke , E.A. , Latham , G.P.: A theory of goal setting & task performance . Prentice-Hall, Inc ( 1990 )

19. Locke , E.A. , Latham , G.P. : What should we do about motivation theory? six recommendations for the twenty-first century . Academy of management review 29(3) , 388 - 403 ( 2004 )

20. Mayer , R.E. , Moreno , R.: Nine ways to reduce cognitive load in multimedia learning . Educational psychologist 38(1) , 43 - 52 ( 2003 )

21. Moore , D. : Engaged learning in the academy: Challenges and possibilities . Springer ( 2013 )

22. Nicholson , S.: A recipe for meaningful gamification . In: Gamification in education and business , pp. 1 - 20 . Springer ( 2015 )

23. Oettingen , G. , Pak , H.j., Schnetter , K. : Self-regulation of goal-setting: Turning free fantasies about the future into binding goals . Journal of personality and social psychology 80(5) , 736 ( 2001 )

24. Pink , D.H. : Drive: The surprising truth about what motivates us . Penguin ( 2011 )

25. Plass , J.L. , Heidig , S. , Hayward , E.O. , Homer , B.D. , Um , E.: Emotional design in multimedia learning: Effects of shape and color on affect and learning . Learning and Instruction 29 , 128 - 140 ( 2014 )

26. Plass , J.L. , Homer , B.D. , Kinzer , C.K. : Foundations of game-based learning . Educational Psychologist 50 ( 4 ), 258 - 283 ( 2015 )

27. Qian , M. , Clark , K.R. : Game-based learning and 21st century skills: A review of recent research . Computers in Human Behavior 63 , 50 - 58 ( 2016 )

28. Reese , D.D.: First steps and beyond: Serious games as preparation for future learning . Journal of Educational Multimedia and Hypermedia 16 ( 3 ), 283 - 300 ( 2007 )

29. Richards , C. , Thompson , C.W. , Graham , N.: Beyond designing for motivation: the importance of context in gamification . In: Proceedings of the first ACM SIGCHI annual symposium on Computer-human interaction in play . pp. 217 - 226 . ACM ( 2014 )

30. Ryan , R.M. , Sheldon , K.M. , Kasser , T. , Deci , E.L. : All goals are not created equal: An organismic perspective on the nature of goals and their regulation . ( 1996 )

31. Sharma , N. , Berkling , K. : Training reading and writing through text-based games; Step 1: Adapting to gamer type . In: 2019 IEEE Global Engineering Education Conference, EDUCON 2019, Dubai, April 10-12 , 2019 . IEEE ( 2019 )

32. Thorne , S.L. , Fischer , I. , Lu , X. : The semiotic ecology and linguistic complexity of an online game world . ReCALL 24 ( 3 ), 279 - 301 ( 2012 )