Gamification Experience with Scratch in Teaching Programming in a Vocational Training Classroom Antonio García-Gutierrez Raquel Hijón-Neira Computer Science Department Computer Science Department Universidad Rey Juan Carlos Universidad Rey Juan Carlos Móstoles, Madrid, España Móstoles, Madrid, España a.garciagu.2018@alumnos.urjc.es https://orcid.org/0000-0003-3833-4228 Abstract— This article analyzes the impact that the use of an increasing their commitment and increasing their application with gamified environments can have on the participation. learning of theoretical programming concepts and on the development of students' computational thinking. The It is reasonable to think that an activity that is surrounded application offers theoretical and practical learning by a similar technological environment and that uses the same environments based on the Scratch programming software. For components and dynamics as these games can motivate and this analysis, a pilot experience has been carried out in which change the student's perception of academic activities, the application is tested with students. The aim is to analyze the resulting in more attractive and motivating performance. progress of both the student's learning of theoretical concepts Three lines of work stand out that seek to gamify education.. and the development of computational thinking. For this, two tests have been carried out, analyzing the results, collecting data The first seeks to use games in a controlled way, the by performing a pre-test and post-test. teacher chooses the game and the moment, so that the student acquires the skills and abilities that are supposed to appear in Keywords— Programming, Computational Thinking, Scratch, them. The second way seeks to use the characteristic elements Gamification of the games (the levels, the points, the medals, the interface ...) to take advantage of the students' predisposition to play to I. INTRODUCTION increase the motivation for learning. The third way is to Currently, video games are one of the most direct ways redesign a learning process as if it were a game. The teacher that children and young people have to interact with must design the subject in such a way that the student has to computers. Their ability to encourage participation can be of play and acquire knowledge, skills and competences [1]. great help when it comes to involving the student and fostering It is necessary to define what conditions a process must a competition that encourages learning by awarding badges. meet to be considered a game or object to be gamified. The By promoting a context of participation, the student breaks requirements or conditions are reduced to the fact that the with the feeling of frustration that the appearance of obstacles activity carried out by the game or process can be learned by in their training can cause and is useful to motivate students to the user, that the feedback can be delivered in a pertinent way carry out tasks that are complicated. to the user and that the actions carried out by the player An experience is presented in the use of an application for throughout the development of the game. process or activity the teaching of programming and computational thinking of can be measured [2]. gamification in professional training. A pilot experience was carried out that consisted of carrying out a pre-test of theoretical knowledge and computational thinking. Once the tests prior to the interventions had been carried out, theoretical concepts were explained from the Scratch-based application, making examples of the different concepts. Once the interventions had been completed and all the concepts were explained, a post-test was carried out to evaluate the students' progress, comparing the results obtained both in the pre-test and in the post-test and determining if there had been significant progress. II. STATE OF THE ART A. Status of Gamification Figure 1. Hierarchy of gamification elements [3] Nowadays, society is faced with numerous technological and social changes. Technology changes the way society The components suppose the badges, points, rankings and interacts and facilitates access to new knowledge. other elements that manage to implement the mechanical and Gamification responds to this need for motivation, dynamic elements. The mechanical elements are made up of transporting game scenarios to formal educational contexts, in all those elements that make the game user act in the game. It order to involve students and open the door to the acquisition can be the obtaining of a reward, a challenge or the same of learning. To do this, it is necessary to surround students in competition between users within the same class or center. playful environments, facing challenges and missions, Finally, the dynamic part is made up of the concept of the game. Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). To achieve the objective, the game must integrate all these Another interesting example for the introduction of elements, in addition to achieving the student's motivation, theoretical concepts and computational thinking at an early which may be intrinsic, when the student seeks to carry out age is RoDy. It consists of two parts: on the one hand, an activities for their own interest, or extrinsic, when the student application that allows teachers to generate their own performs the tasks for a reward you can get. activities according to the syllabus and, on the other hand, a robot that allows students to interact with the application. This The teaching of programming is closely tied to robot is represented in the game by a virtual agent in the shape computational thinking. Solving the problems using a series of a bear, which will interact with users through dialogues [7]. of instructions structured and sequenced in such a way that they allow to reach the resolution of the problem. As a future project for the integration of gamification in the classroom, it is worth highlighting the proposal of the Therefore, it is necessary that, when learning GameMo module for Moodle, which will allow integrating the programming, students structure the resolution of a problem most common elements of gamified courses within the analytically; analyzing the problem and establishing and Moodle platform, thus facilitating the teaching work for the ordering a series of steps to follow to solve the problem. implementation of gamified courses. Once an initial prototype A simple tool when working on computational thinking is has been developed, it will be tested in real environments by Scratch, since it allows the student to interact with a simple means of a quantitative usability assessment, leaving it open programming language, in which the programming language to future improvements after tests in real environments [6]. is sequenced by blocks, eliminating much of the complexity B. Hypothesis of the programming languages. programming and allowing focus on the logic and structure of problem solving, thus Due to the need to study and analyze the impact that the working computational thinking. application can achieve in the learning and development of the theoretical concepts of programming and computational It is worth highlighting the role that the Dr. Scratch thinking. Therefore, the hypotheses to be determined are the application can play in this part of learning. This application following: allows you to correct, analyze and catalog the degree of • H1: With the methodology / intervention proposal, learning of a project by assigning a score distributed among learning in programming concepts can be improved. the different computational concepts. • H2: With the methodology / intervention proposal, It is important to highlight the analysis of the Dr. Scratch one can improve the learning of Computational Thinking. application that was carried out in a program made up of 109 elementary and high school students in which it was intended III. METHODOLOGY to analyze the progress of computational thinking using the Scratch programming tool. To do this, some work sessions The methodology to be followed in the interventions with Scratch were carried out, and the Dr. Scratch tool was consists of developing the theoretical concepts outlined in the also presented to the students so that students could practice, application guide. The development of the concepts consists develop and correct their own codes through the tool. of a first theoretical part in which the teacher explains and defines the concept that will be worked on in that topic, and To measure the progress of these students, a pre-test and later a practical part in which the student interacts with post-test were carried out before and after practicing with the different programs developed in the software-based tool. The results were analyzed by means of a t-test, application Scratch programming. establishing a level of significance alpha = 0.05, returning a result of p <0.05, so it could be stated that the learning was For the development of the interventions a local web significant. application is used that uses the Scratch block programming software. Another significant experience was the development of a study carried out in sixth grade students whose objective was At the beginning of the application there is a main page to verify whether the Scratch programming tool allows the where the different topics to be developed by the students are development of computational thinking. The methodology of shown. The application consists of two parts, presentations work with these subjects consists of 4 phases. (Figure 2) and practical (Figure 3). A first phase in which a Scratch guide is developed as a learning resource. The guide consists of 5 units in which different activities are developed that increase the difficulty. The guide is aimed at subjects who do not have any knowledge of programming and use of Scratch. A second phase where the guide is developed with the students and proposed exercises are worked on. This second phase occupies 14 sessions of 45 minutes where the subjects work with the teacher the concepts and activities proposed in the guide. In the third phase, the evaluation of computational thinking is carried out by means of a test. In the last phase, the analysis of the data obtained is carried out. Due to the positive results obtained, it can be concluded that the Scratch programming software is a good tool to Figure 2. Screen of the interactive presentation of develop computational thinking at an early age [5]. the theoretical concepts on Computational Thinking % ALUMNOS CON CONOCIMIENTOS EN SCRATCH 33% NADA POCO 67% Figure 4 Percentage of students with prior computer knowledge Figure 3 Scratch Practice Screen. In the second intervention, the activity was resumed from the point where the previous session left off. The session The presentation part is used to support the teacher in the lasted 50 minutes and the rest of the concepts (loops, events, presentation and development of theoretical questions. It parallelism and computational thinking) were explained. In offers an environment in the form of a blackboard in which addition, the two post-tests were carried out, thus ending the theoretical questions are exposed, defining the concepts interventions. explained clearly and schematically and giving simple examples from day to day, thus facilitating the understanding B. Data collection of the theoretical concept. Two tests were used to collect samples. The first of them The practical part offers an interactive environment in has been used to analyze the progress of knowledge of which the student can put into practice the concepts previously theoretical concepts, and the second has been used to measure progress in computational thinking experienced by the student explained by the teacher. For this, the application has different thanks to the application. The research carried out is carried exercises developed with the Scratch programming software. out within the framework of quantitative research, describing The topics developed in the application are the following: the observed reality and establishing a cause-effect relationship, explaining the results obtained. A quasi-  Topic 1: Sequences. experimental method will be followed, since due to the  Topic 2: Variables and data. number of students it has not been possible to separate the sample into a test group and a control group. The two tests  Topic 3: Operators. (pre-tests) will be applied before starting the sessions, and the two tests at the end (post-tests).  Topic 4: Conditionals. To carry out both questionnaires, the Google Docs forms  Topic 5: Loops. tool has been used due to the ease of use and accessibility of  Topic 6: Events. the application.  Topic 7: Parallelism and synchronization. The theoretical knowledge test consists of 16 questions, both multiple choice questions and open questions. The  Topic 8: Computational thinking. objective of the test is to evaluate how the application has influenced the learning of theoretical concepts. An example of IV. EXPERIMENTATION the questions that make up the theoretical knowledge test is A. Participants and sequencing shown in Figure 5. The participants are 1st grade administrative students, a group made up of 9 students aged between 16 and 17 years. The students belong to an institute in the province of Toledo. This group is chosen for accessibility and availability of the sample. Figure 4 shows the level of programming knowledge of the participating students. Most of them state that they do not have any prior knowledge about programming and, furthermore, they see theoretical concepts as abstract and difficult to understand. In the first intervention, the activity was explained to the students. Before starting to work with the application, the theoretical concepts and computational thinking pre-tests were carried out. The session took place with the regular teacher, where the activity was explained to the students. The Figure 5 Theoretical concepts test duration of the session was 50 minutes and the concepts The computational thinking test consists of 28 questions, sequences, variables, operators and conditionals were with closed questions in which the student must choose explained with examples in Scratch. between 4 options, only one of them being the correct answer (Figure 6). The computational thinking test is based on the To verify that learning has been significant in the work carried out by Román González, establishing the criteria computational thinking test, the T_test is performed again, for the elaboration of the test [4]. obtaining in this case a p-value = 0.316 that is greater than the alpha value = 0.05 (see Table 2 and Figure 8). Table 2 Results T-test theoretical concepts Typ. Var. Obs. Min. Max. Media Devia . PRE- TEST GRADE 9 1,071 7,143 5,198 1,899 POST- TEST GRADE 9 2,500 8,214 5,675 2,146 Figure 6 Computational thinking test C. Analysis of the results To evaluate the degree of significance of the learning of the theoretical concepts, the t_test for dependent samples has been performed, obtaining the following results (see Table 1 and Figure 7): Table 1 Results T-test theoretical concepts Typ. Var. Obs. Min. Max. Media Devia . Figure 8 Boxplot of the t_test for the Computational Thinking samples. PRE- TEST GRADE 9 1,538 4,038 2,393 0,764 Although in most cases a slight evolution is observed (see POST- Figure 6), this is not significant as has been observed in the TEST learning of theoretical concepts. In addition, we find the GRADE 9 2,692 6,154 4,509 1,397 average obtained both in the pre-test and post-test (marked with quite a few crosses) quite close to each other. This lower evolution is attributed to the fact that computational thinking The results observed in Table 1 and in Figure 7, on which requires a systematic change in the way of thinking, the t_test has been carried out, show a value p = 0.002, well analyzing and approaching problems much more profound below the alpha value = 0.05, so it can be stated that learning than that obtained during the development of the sessions. the concepts theoretical has been significant. Analyzing separately the results obtained in the different theoretical concepts, it is observed that the concepts that have presented the most difficulty among the students have been the concepts of events, conditionals and variables. Instead, the concepts where the greatest progress can be seen have been the concepts of parallelism, events, and operators. V. CONCLUSIONS When comparing the results obtained with the results obtained in the experience described in section II. State of the art, in which the Dr. Scratch tool was used for its analysis, it is observed that the development of computational thinking is not significant, unlike the results analyzed with the use of the Dr. Scratch application that are described in that study. Although in the present study, the learning of theoretical Figure 7 Boxplot of the t_test for samples of theoretical concepts has been significant, the application for the teaching concepts of programming and computational thinking used has not had enough impact to achieve the development of computational thinking with a sufficiently significant degree. The main encouraged to participate. actively. The concepts that the difference with respect to the previous study is that the students previously perceived as abstract and difficult to members of the study of this tool worked for weeks using the understand have been worked on in an easy way interacting Scratch application, achieving a knowledge base that would with the different examples designed in Scratch, which allow help them progress more quickly when they started working the assimilation of theoretical concepts. with the Dr. Scratch. At the beginning, the students had a certain feeling of rejection of the intervention approach, when they conceived ACKNOWLEDGMENT the programming as something difficult and distant. Thanks to This work has been funded by the research projects the intervention, most of the students abandoned this feeling iPROG: New generation of tools for learning Programming that causes ignorance of concepts, they learned that with emerging interactive technologies from MINECO (ref. programming is based on thoughts and logic present in TIN2015-66731-C2-1-R) and e-Madrid: Research and everyday life. At the end of the sessions, although it is not directly related to the professional field of the higher grade, Development of Educational Technologies in the Community the activity was very well received by the students. of Madrid (ref. P2018 / TCS-4307). In view of the results, the duration of the interventions has REFERENCES been sufficient to achieve a significant learning of theoretical [1] Fidalgo, A. (2014, 26 de marzo). Qué es gamificación educativa. knowledge, but not to achieve enough impact to develop Innovación Educativa. Blog de Ángel Fidalgo para reflexionar sobre computational thinking, although it does improve it. How it innovación educativa. Recuperado de improves future interventions, a higher number of sessions is http://innovacioneducativa.wordpress.com/2014/03/26/que-es- gamificacion-educativa proposed, or failing that, an increase in their duration that [2] Cook, W. (2013). Training Today: 5 Gamification Pitfalls. Training allows students, apart from the theoretical explanation and Magazine. Recuperado de: interaction with the application programs, to develop your http://www.trainingmag.com/content/training-today-5-gamification- own codes with teacher guidance. pitfalls. [3] Hunter, D., & Werbach, K. (2012). Gamificación. Revoluciona tu The results obtained reflect a better understanding of the negocio con las técnicas de los juegos. theoretical concepts after the sessions. In the case of the [4] Román-González, M., Pérez-González, J.C., Jiménez-Fernández, C. theoretical concepts test, once the sessions are over, an (2017). Which cognitive abilities underlie computational thinking? evolution is observed in all the students, being more Criterion validity of the Computational Thinking Test. Computers in noticeable in some students, and to a lesser extent in the rest Human Behavior, 72, 678-691. doi: of the students. The results of the t-test were positive, https://doi.org/10.1016/j.chb.2016.08.047. observing a significant degree of learning. The time of the [5] Álvarez Rodríguez, M. (2017). Desarrollo del pensamiento computacional en educación primaria: una experiencia educativa con interventions and the use of the application was correct to Scratch.. Universitas Tarraconensis. Revista de Ciències de l'Educació, develop, in students who had no prior programming 1(2), 45-64. doi:https://doi.org/10.17345/ute.2017.2.1820. knowledge, the theoretical concepts that make up [6] García-Iruela, M., & Hijón-Neira, R. (2018). Propuesta de interfaz de computational thinking. gestión de entornos gamificados en Moodle. It is concluded, therefore, that the use of the application [7] Hidalgo Rueda, L., Pérez-Marín, D. “RoDy: peluche robótico para enseñar a compartir en Educación Infantil. 21st International has notably improved the disposition of students to learn and Symposium on Computers in Education (SIIE 2019).pp:101-106. understand theoretical concepts that they conceived as difficult. After the sessions, the students have significantly improved their knowledge of these concepts, thus confirming the first hypothesis raised. This has not been the case in computational thinking, in which more sessions may be needed to provoke a more profound change in the methodology and approach that these students have to address problems (second hypothesis). The results reflected in the t- test indicate that, although there is an improvement and a development in the learning of computational thinking, it does not reach a significant degree, rejecting the second hypothesis raised. One possible reason may be that the study subjects did not have previous programming knowledge, therefore, it is possible that the duration of the sessions and the application work, although they have had positive results, have been insufficient to achieve a significant development of the program. computational thinking. How aspects of improvement can be considered increasing the number and time of the sessions, being able to delve more deeply into the contents of the application. The work with the students has been fun and fluid and, in general, they have perceived the sessions as something positive, a way of learning in a fun way, creating a relaxed atmosphere in the class dynamics in which the students were