Programming Teaching Experience for Boys and Girls following a Gamified Approach Iago Cruz-G arcía G Juan Antonio Martín-García Diana Pérez-Marín Computer Science and Statistics Dept. Computer Science and Statistics Dept. Computer Science and Statistics Dept. Universidad Rey Juan Carlos Universidad Rey Juan Carlos Universidad Rey Juan Carlos Móstoles, Madrid, Spain Móstoles, Madrid, Spain Móstoles, Madrid, Spain i.cruz.2016@alumnos.urjc.es ja.martin.2016@alumnos.urjc.es diana.perez@urjc.es Celeste Pizarro Applied Mathematics Dept. Universidad Rey Juan Carlos Móstoles, Madrid, Spain celeste.pizarro@urjc.es Abstract—Teaching programming in Primary Education is successfully develop high level abilities in students such as a worldwide research interest topic. Currently, there are several solving problems, teamwork, spatial vision development as approaches that are being analyzed with more or less success in well as to reduce stress and develops motor skills [3, 6]. terms of learning gains, levels of motivation, and satisfaction. In this paper, the proposal is to use a gamified approach. However, to maintain gamification it is important to Hypothesis 1 (H1) is that the gamified approach will increase highlight the need of stablish concrete and achievable goals students' learning gains while also increasing their level of for the player [7] and challenges with incremental levels of satisfaction and motivation. Hypothesis 2 (H2) is that there will difficulty. be no significant differences between boys and girls from 10 to 12 years old. During the 2019/2020 academic year, an Hypothesis 1 (H1) is that the gamification approach for experiment was carried out with 100 students from 10 to 12 teaching programming in Primary Education will improve years old. All students have started their programming learning students’ learning scores as well as their satisfaction and through a gamified approach, varying the resource used motivation levels. Hypothesis 2 (H2) is that there will not be (teacher, Blockly, Blockly + video game). The results gathered significant differences between boys and girls from 10 to 12 support H1 and H2 and yield interesting results to continue years old when learning to program with gamification. analyzing the differences between the resources used. To verify these hypotheses during 2019/2020 school year, Keywords — Teaching programming, videogame, an experiment has been done with 100 students from 10 to 12 gamification, Primary Education years old taking 4th, 5th and 6th school years of Primary Education, from October 2019 to February 2020 before the I. INTRODUCTION lockdown stopped from continuing in-person data gathering. Teaching programming in early ages has received a lot of During the experiment, all students have followed a attention worldwide in the latest decades [1]. Some authors gamification approach to understand basic programming assert that children could improve their understanding of the concepts: input/output, conditionals and loops, varying the digital world where they live and even develop their gamified resources: a teacher suggesting goals and challenges computational thinking [2] to be able to solve life problems in class playing with metaphors to teaching programming, with computer resources. Blockly for introducing basic programming concepts, and a videogame developed as final degree work from first author This paper focuses on teaching programming to improve during 2019/2020 school year. primary education children’s programming abilities, while they enjoy the learning process and keep their motivation. To Research questions are the following ones: achieve this, the proposal focuses on teaching programming in Primary Education by using a gamification approach.  Q1. Which differences exist between results of different groups? Gamification can be defined as using game design elements in non-gaming environments to increase motivation  Q2. Which differences exist between results from and satisfaction. Previous research show that gamified boys and girls? environments usually get better results in education than non- Results gathered support H1 and H2. This paper is gamified ones, as long as the design is done correctly. This structured in four sections: Section 2 presents the proposal, consists of introducing rewards, game mechanisms and global Section 3 describes the experiment, and Section 4 ends the design, not only isolated elements [3]. paper with main conclusions and future lines of work. The proposal focuses on using resources as metaphors [4] as if programming was cooking, Blockly [5] and videogames [6]. Using videogames for educational purposes has proven to Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). II. PROPOSAL A. Gamified use of metaphors with teachers To teach programming teachers need a guide. In our previous work, it was proposed the use of scripts step by step based on metaphors [4]. Underlying concept is that children between 10 and 12 years old usually enjoy cooking, and programming could follow the cooking metaphor. In both cases there is a goal, make food or achieving program’s goal, and to get to that point instructions must be followed step by step. In case of food, instructions would be the recipe steps. Order is important because if, for example, we wanted to cook a potato omelette and oil is not hot, potatoes will not be fried correctly and omelette will not taste good. Likewise, if we want to make a program achieve its goal and we do not follow order of output instructions, the output Fig. 1. Blockly screenshot (source: [5]) will not be the expected one. Blockly has been chosen over Scratch because, in the Table I shows a summary of main metaphors used [4]. opinion of the authors and our proposal, because students have TABLE I. SUMMARY OF THE METAPHORS USED a console they could use as if it were a professional programming environment. It is also likely that students from # Subject Concept Metaphor 1 Program, Program- recipe 10 to 12 years old they have already used Scratch in previous programming, sequence years (in example, between 8 and 10 years old), therefore sequence, memory Memory, Pantry, box continuing with Blockly allows them to go deeper into variables knowledge about input/output instructions, conditionals and 2 Intput / output Output Computer Screen loops. instructions Gamification integrates itself in Blockly by elements Input Keyboard, computer previously described: concrete goals [7] and increasing screen result difficulty challenges [8]. Goals are given by the teacher 3 Conditionals Child taking Computer taking choices choices because Blockly is a blank canvas for the student to program, 4 Loops Preparing table Computer repeating but it does not have a guide to show them what to do. Same for X persons instructions X times goals as case II.A are proposed, related to input/output, conditionals and loops concepts. The playful component as was stated in the introduction In particular, exercises’ statements requested to children has to be planned with concrete goals [7] and challenges with as programming challenges in Blockly are: increasing difficulty levels [8]. For this reason, it is proposed  Write a program to show your name in screen that the teacher not only uses metaphors, but that they propose (program concept, sequence, I/O). challenge to children where they have to use those metaphors to solve a challenge, such as making a banquet and using  Write a program to ask how old a classmate is, and pseudocode programming on paper for that purpose, as show it in screen (program concept, sequence, I/O, explained in [9]. variable). Once children get certainty and rewards like being  Write a program to check if a number is odd or even congratulated by the teacher, acknowledgement from their (program concept, sequence, conditional, I/O, equals in class and some stickers in a physical medal board, variable). then challenge difficulty can be increased. This way, if they could achieve to make the program that allowed them to make  Write a program to add, subtract, multiply and divide the banquet, the following step could be making a program two numbers (program concept, sequence, that allows them to organize weekly menus of a fantasy conditional, I/O, variable). kingdom that could be created by teacher and students. As in II.A case when boys and girls successfully completed exercises, they got a congratulation and acknowledgment from the teacher. As students were B. Use of Blockly completing exercises, they were ordered in a way that Blockly [5] is an open source visual programming challenges had an increasing difficulty. language in which programs are built using instructions as if they were puzzle pieces. It is similar to Scratch because it also focuses on using blocks, as it shown in Figure 1. C. Educational videogame use Develop Learning is the proposed videogame designed and developed to satisfy the needs of students who want to learn programming. Figure 2 shows an example of a screenshot of the videogame. Fig. 3. Input/output level (source: own elaboration) For each minigame, the interface is the same: the console Fig. 2. Develop Learning screenshot (source: own elaboration) with the code is in the upper right corner and, if there is an explanation, it would be located as a text box at the bottom. In When students were asked if they wanted to have help in boss levels, the text box with explanations and dialogs is in the the system, most answers told that they wanted to have an lower left corner to leave room to the text area for writing animal pet. This is why in the videogame the user represented code. Every time a minigame is completed, stars get coloured by the character (4) comes along with a dog as a pet named as part of the gamified approach to rewards. Binary (2) which guides them through challenges that has to overcome through pseudocode programming. In the first level minigames are focused on input/output instructions. In description minigames, the player will have to The pet (2) shows texts (1) to students so they do not have read closely descriptions in the console in the upper right a blank canvas as it happens in Scratch or Blockly, which both corner and introduce in the well the corresponding word. of them, despite having tutorials, they do not have accurate instructions about what to do. In easier levels of the If they succeed, the block will stay in the well. The player videogame, the own videogame shows the code (3), while in will have to do this task five times in total. If they chose a harder levels the difficulty increases and students have to write wrong block, this and every block inside the well will be the code to overcome the challenges that are introduced thrown (as in a parabola to avoid coming back to the hole) and through the different worlds where they are moving. they will have to start again. When application boots, the student will find a main menu This is based on that, when starting a learning process, consisting of: attention should be paid to concepts that are being studied. One way to achieve this is to describe those words that have  Play menu, where they can access the tutorial and different meaning in the current context to understand further different worlds and levels. explanations and teachings.  Settings menu, where they can change interface In the choosing minigame, the player must discern the colours in minigames. program output, falling through a hole between three possible  Help menu, where they can check concepts in a ones. If they fail, they will come back to the beginning of that comprehensive way with examples. This menu can floor. If they succeed, they will go to next floor, until they also be accessed during minigames. reach the end of the level. The intention is to improve abstract thinking, to understand possible environments and step ahead  Exit button to shut down the application. of solutions to take apart knowledge from context, which means, not to memorize exercises. The videogame interface was made in a way that it could be used without mouse and keyboard at the same time. In other In the reordering minigame, the player will have to place words, when the user interacts with menus, they do not need blocks on stands which match with the position shown in the to use the keyboard and when they are in a minigame, they do console. The order of the console and stands match with not need to use the mouse. They game also does not require occidental reading order: the console goes from top to bottom internet connection. These choices were made because not all and stands go from left to right. Likewise, each block has the schools have a good internet connection, and also, laptops are same colour as the corresponding space to arrange in the usually used, so using simultaneously mouse and keyboard console. could get tedious. The user should look at the console on the upper right For each main concept to teach, there is a world: I/O, corner and at the colour of the text that they should order, conditionals and loops. All worlds shared the same structure: because they match with the colour blocks and stands on the four minigames (descriptions, choosing, reorder and fill in the scene. From left to right, they should place the block in their gaps), a review level and a boss as a final challenge as it shown equivalent position on the console, from top to bottom. If they in Figure 3 and as it was requested by school students as the are sure about the result, they should press ‘F’ key near the users following a User Centered Design [10]. button at the back of the level to check it. If they fail, blocks will disarrange. An important skill to solve problems is the ability to B. Procedure organize elements that compose them. As in mathematics the During the research, students were allocated in three first thing that is tasked in an exercise is to group known data groups: students using Blockly + videogame, students using or in morphosyntactic analysis to locate subject and predicate Blockly and students taught by the teacher. first, in programming it should be logically structured with variables at the start to use them in later instructions. All groups followed a gamified approach. Since the videogame was in development during these classes were In fill in the gaps minigame, the player should enter the being taught, first sessions were covered always by Blockly blocks in the stablished order. If they are right, they will light (except for the group taught by the teacher, which only used up in green, otherwise in red. This tries to emulate usual blackboard, chalk, paper and pen) and it was since January exercises, made to learn a language, filling empty spaces with when videogame could also be used for the Blockly + a word from an array so the sentence makes sense. This videogame group. minigame seeks to complement the previous ones, because at this point, the user should understand the structure and Every group made a programming pre-test (October 2019) instructions purposes. to check initial concepts that could have been obtained externally, and also to check how old students were and what In the review minigame, the player will go through the devices they usually use. For resolving this test, it was four previous levels in order and in a reduced way. In other required to have knowledge about the following concepts: words, they should complete description, choosing, reordering and fill in the gaps minigames to succeed.  Implementation of an output instruction. In this level, every previous minigame will appear in  Implementation of an input instruction and an output reduced versions, to build on what has been learnt and to get instruction. ready for the final level against the boss. It is done to be similar as the previous preparation for an exam, to remember what  Implementation of output instructions specified by they have learned and to face the test with fresh knowledge. conditional blocks. In the boss minigame, the player should pay attention to  Implementation of an infinite loop, with output the helper dog Binary indications, and the boss, to manually instructions. write code to advance. If the player makes a mistake, a basic This test was presented again to students after four hint will be offered, a guide of what is required. If they make sessions (December 2019) to check their progress until that another mistake, an explicit hint will be offered. Another date, reviewing differences between groups. mistake, the player should start again the level. These sessions, in the three groups, consisted in lessons In these levels, the player should write, in the designated about the following basic programming concepts: variable, area, instructions to complete the needed program, when they memory, instruction, sequence and program. Groups also are instructed to until the end of the level. Throughout this worked in exercises, with previously mentioned resources, level, if the player fails a hint will be offered to know what requesting to introduce the input and output flow of a program, they should write. If they fail again, another explicit hint will with exampled based on natural language that allowed be offered. At third fail, they will restart the level. students to associate familiar concepts with the ones that were This minigame was inspired by exams and final being presented to them. knowledge tests to put the student in a light pressure to address Class dynamic was structured always in the same way: it the situation in a playful way. This tries to fulfil the need of was started with a recap of previously explained teachings, concrete objectives and challenges to get good results with after that, a new definition was introduced, and then students videogames [7,8]. had to do a new practical exercise to solidify acquired concepts, solving any doubts that could be asked. The same test was presented again to students in February III. EXPERIMENT 2020, just before the lockdown due to COVID-19, which A. Sample disallowed to continue with the investigation, since every on- 100 Primary Education students from 10 to 12 years old site class in Spain was cancelled. The goal was to check the has participated in the experiment. 46% of them are boys and evolution in grades of students from each group to evaluate 54% are girls, from two public schools from Madrid. the gamified approach and if there were any differences between boys and girls. 94% of students answered the user profile survey, used as reference for designing and developing the gamified system. In this user profile survey, there were questions about how C. Results familiar they were with technology and videogames, their In the first place, a descriptive study of data is made, which personal preferences, how they handled challenges and their shows grades obtained in the test made in different dates: willingness to programming. October 2019, December 2019 and February 2020. The goal of these questions was to establish how to Figure 4 shows box plots regarding the punctuation approach the design of help systems, menu navigation, how variable, separate by the different proposals. Table II sums up long where game sessions and what goals had to be the numerical results. This table does not show the median, accomplished. Another goal for these questions was to since its value its always 0.00. As can be seen, in October 2019 establish the structure and dynamic for the programming students did not know how to correctly answer any question. classes since these concepts are new for these students. Subsequently, in December 2019, student groups that used October and February, something that was already guessed in Blockly and Blockly with Develop Learning showed a similar the previous descriptive study. behaviour. Worst average results were obtained from control group. TABLA III. ANOVA OF TWO FACTORS WITH INTERACTION Sum of gl F Sig. squares Month 5.94 2 26.22 0.00 Group 0.08 2 0.38 0.68 Interaction 1.66 4 3.67 0.06 Fig. 4. Boxplots of the scores per methodology TABLE II. AVERAGE AND STANDARD DEVIATION OBTAINED FROM STUDENTS’ SCORES IN THE TESTS n=100 December February x̄ sd x̄ sd Control 0.18 0.26 0.51 0.46 Fig. 5. Marginal averages of the scores obtained Blockly 0.35 0.46 0.19 0.27 Blockly + 0.32 0.43 0.30  0.45 Video- Furthermore, this significative interaction shows that game depending on the month, the efficiency of each group has behaved differently. Thus, considering only December data, there is not a meaningful difference between groups, but a It should be mentioned that the three distributions are difference in February can be noticed, with control group strongly asymmetrical to the right, which means that there is being the best positioned, followed by the one with Blockly a greater clustering of students with low grades in every case. and Develop Learning and in the last place the one with The median in the three cases is 0, but the average is higher Blockly. Figure 5 shows this fact. due to this asymmetry. When the study is made splitting between boys and girls, However, in February 2020, the control group’s data it doesn’t show meaningful differences between groups. In distribution becomes symmetrical to the left, which means October 2019 test case, both boys and girls got an average that there is a high clustering of high grades over low grades. score of 0 because they didn’t have any knowledge about Therefore, even if there are a lot of atypical data, median programming. In December 2019, both girls and boy average shows that this control group is the one that got better results. score was 0,28 and in February 2020, average score for boys There is a greater scatter in data from Blockly and videogame was 0,31 and for girls 0,27. groups. IV. CONCLUSIONS Related to meaningful differences that could be between different groups, and on the basis of a scenario where there are Gamified approach for programming teaching in Primary not any previous learning from students as can be seen in their Education has significantly improved the learning profit both grades, a process to check if there are differences between from students taught by the teacher (unplugged approach) and groups is started. from the ones with Blockly and the videogame. This validates the importance of teaching using playful resources with For that purpose, a two factors with interaction Anova has increasing difficulty challenges. been performed, where one factor is the month when the test was performed and the other one is the group (teacher, Answering the research question about what differences Blockly, Blockly + videogame), and the obtained grade as the exist between results from different groups, against expected, answer variable. Table III shows the obtained results. better results are obtained by students from the group with teacher, without Blockly and the videogame. It can be seen that there are no isolated meaningful evidence between groups, but there are in months. A more Answering the research question about what differences detailed analysis of this result is found in a post-hoc analysis. exist between results from boys and girls, there are not This analysis shows that the meaningful difference between meaningful differences between groups. This means that both grades is found related to October, in other words, there are no girls and boys can learn programming using gamified meaningful differences between December and February approach. This result is very important, especially for girls grades, but between October and December and between from 10 to 12 years old who could think, based on social stereotypes, that they couldn’t enjoy videogames or that they http://csta.acm.org/Curriculum/sub/CurrFiles/CS_K8_Building_a_Fo couldn’t be able to reach the same performance as their male undation.pdf. classmates. [2] J.M. Wing. “Computational thinking”. Communications of the ACM, 49(3), pp. 33–35, 2006. As future work, it is planned to continue the study during [3] C. Groening, C. Binnewies. “Achievement unlocked! - The Impact of more time when in-site classes can be resumed if COVID-19 digital achievements as a gamification element on motivation and allows to it. In addition, it is planned to add the age component performance”. Computers in Human Behavior, Volume 97, pp. 151- 166, 2019. to the study, comparing obtained results in a larger age range, [4] D. Pérez-Marín, R. Hijón-Neira, M. Martín-Lope. “A Methodology from 10 to 15 years old, to identify if there are meaningful Proposal based on Metaphors to teach Programming to children”. differences and interactions between gamified resources, age Revista: IEEE Revista Iberoamericana de Tecnologías del Aprendizaje, and gender components. 13(1), pp. 46-53, 2018, https://doi.org/10.1109/RITA.2018.2809944 [5] Blockly, https://blockly.games/?lang=es (fecha última visita: 06/2020) [6] C. González-González, F. Blanco-Izquierdo. “Designing social videogames for educational uses”. Computers & Education, Volume ACKNOWLEDGMENTS 58, Issue 1, pp 250-262, 2012. Research funded by the TIN 2015-66731-C2-1-R project [7] K. Øygardslia, P. Aarsand. “Move over, I will find Jerusalem”: and the Madrid Regional Government, through the e-Madrid- Artifacts in game design in classrooms”. Learning, Culture and Social CM project (P2018/TCS-4307). The e-Madrid-CM project is Interaction,Volume 19, pp. 61-73, 2018. also co-financed by the Structural Funds (FSE and FEDER). [8] M.J. Dondlinger. “Educational video game design: A review of the We would also like to thank the school, teachers and students literature”. Journal of applied educational technology, 4(1), pp. 21-31, 2007. for their collaboration. [9] A. Bacelo-Polo, R. Hijón-Neira, D. Pérez-Marín. Piensa y programa con Scratch…en casa y en clase. Anaya. 2018. REFERENCES [10] J. Lorés (Ed.). La Interacción Persona-Ordenador. Disponible on-line en https://aipo.es/libro/pdf/00Portad.pdf (fecha última visita: 06/2020) [1] Computer Science Teachers Association. Computer Science K–8: Building a Strong Foundation. 2012. Available from: