Prototyping a gamified system to persuade school-age children in developing countries: using Kahoot in online environments

Abstract

Socio-economically disadvantaged school-age children in developing countries, e.g., primary school students, often feel isolated and lack the opportunity to use modern technology for their educational growth. This may negatively impact their schooling through low grades and, in the worst-case, even dropping out of their classes. Gamification is the use of game design elements in non-gaming contexts. It is envisaged that using a gamified learning system can significantly impact different learners. Our aim is to conduct a case study with a Kahoot-based gamified learning system. The design of the system followed a design thinking iterative process. The test result informs us that those participants using a Kahoot based online platform for solving primary level math quizzes and puzzles preferred the reward-based tasks in the gamified system. Our future effort is to utilize Cognitive Social Learning Theory (CSLT) that supports design instruction and the gamification of learning to deploy and implement our Kahootbased gamified application, followed by an experimental design which will be carried out with students to measure behavioural change and learning.

Keywords

• Gamification
• CSLT
• design thinking

-

New digital technologies offer valuable resources for students' academic and social development [ 1 ], e.g., with digital gamified systems. Socio-economically disadvantaged students at a primary level mostly lack the opportunity to use modern technology for their educational growth; in fact, two-thirds of children around the world lack access to the internet [ 2 ]. This work aims to contribute to reducing inequalities and quality education, which are internationally acknowledged goals, for example, as a part of the United Nations’ Sustainable Development Goals [ 3 ]. There are efforts to facilitate and promote technology-enhanced solutions in developing countries, such as touch screen tablets [ 4 ] or smartphones. Deploying a low-cost digital gamified system in tutoring students is seeing a rapid expansion. A digital gamified system makes educational modules or topics simple, understandable, and interesting to students and thus aids in learning [ 5 ]. Due to continuous growth, it is expected that in the future, various gamified systems will be readily available to the users based on their topic of interest. In recent years, gamified-learning platforms such as Kahoot! have drawn significant attention to positively affect learning performance among students [ 6 ]. An assessment method and tool are vital to assess the performance and quality of the digital learning systems for disadvantaged students and indicate the appropriate gamified learning system. Moreover, digital gamified systems' quality in various contexts and the impact of gamification techniques, such as points-based rewarding in socio-economically disadvantaged students, are not available widely.

Researchers [ 7 ] pointed out the following as methodological limitations on a gamified learning system: sample sizes are small, proper psychometric measurements were not used while surveying user experience, some experiments only considered users evaluation, lack of consideration for individual motivational affordance, absence of multilevel measures, for instance, psychological outcome, and behavioural outcomes. Relevant research has been done without considering the sample size. To the best of our knowledge, little is known about utilizing content gamification learning pedagogy to examine the quality of the digital learning system. However, the current literature shows the following relevant research works: • [ 8 ] presented a model that measures the success of the learning system. The reliability of the efficient tool was determined through Cronbach’s Alpha Coefficient. • [ 9 ] used fuzzy ANP (Analytical Network Process) to evaluate the effectiveness of the learning system in uncertainty. Only literature reviews had been conducted for the proposed evaluation; longitudinal studies and interviews were not conducted.

The sampling size was not sufficient. • [ 10 ] considered traditional pedagogy to measure the impact of technology in learning. Student performance has been observed in this research. On the other hand, traditional pedagogy is not sufficient to measure the efficacy of learning systems.

Our present study focuses on introducing content gamification learning pedagogy in a gamified learning system that can significantly impact different learners. Content gamification uses game-play mechanics for non-game applications [ 11 ]. Gamification, particularly content gamification, can help students in multiple areas within an institution to improve learning performance [ 12 ]. Thus, the expected outcome in using the gamified learning system can also change significantly.

The study's primary research question is: What is the effect of the content gamified learning pedagogy in the digital learning system to influence economically disadvantaged students? Two other guiding research questions are included in this study from the main research question. ● RQ (1): What is the performance of the gamified learning system in students’ learning? ● RQ (2): What is the impact of rewarding techniques, i.e., point-based, in motivating students?

The previous study utilized a theoretical construct with the gamification technique to promote students' progression and social connection in their study program [ 13 ]. Therefore, to answer the research question, Cognitive Social Learning Theory (CSLT) [ 14 ] will be utilized in this study that supports design instruction and the gamification of learning. CSLT refers to learning that occurs in a social environment with a dynamic and reciprocal interaction of the people, atmosphere, and behaviour. This theory contributes to the instructional design of a gamified learning system and the student's education and learning experience. Its method involves problem-solving and decision making of the students. We have applied Kahoot as an interactive gamified learning platform among the students. This gamified learning system will allow the students to interact with artefacts within the game and with other students online. Problem-solving strategies such as quizzes and mathematics puzzles will be practised and refined within this learning context. This gamified learning system is guided by a Design Thinking approach such as user-centred design, i.e., students will be engaged to design the mathematics puzzle and quiz within the Kahoot-based system. 2

Design Thinking: A Sustainable Technique

We have utilised the design thinking iterative process to guide the design of the gamified learning system. Design Thinking is an iterative design process of an application in which users are involved in every design stage [ 15 ]. Design thinking comprises five steps: empathize, define, ideate, prototype, and test (Fig 1).

Empathizing signifies the sense of understanding to solve problems. Define suggests creating the sense of the information collected from the Empathize step and resulting in a problem statement called point-of-view (POV). Ideate directs on making ideas and concepts that help design prototypes fitted for users’ feedback and recommendations, such as mind mapping, body storming, sketching, and mind mapping. The next step, prototyping, alludes to an early stage of application delivery, for example, prototyping a gamified learning system. This prototype can be a low-fidelity, mid-fidelity, or highfidelity prototype, i.e., a paper object, soft copy or electronic object, or an interactive display in technology-based solutions. Testing confirms that the prototype is refined, exploring the users' requirements, and clarifying the POV. The design thinking process was utilised to design and develop gamified learning systems for students for their study progression and social interaction [ 16 ][ 17 ]. 3

Iterative design of prototype and evaluation

We conducted the first iteration of the design thinking process. Based on the above observation, our future research will move to the next define step to draw the conclusions from the empathize step and develop an actionable problem statement. The next step will be the ideate, by which multiple ideas shall be brought forwarding into prototyping and then testing the prototype of an app. 3.1

Empathise

Starting with the empathizing step (observe the users and their views about the gamified intervention). We tried to observe school-age children’s behaviour, engaged them through interviewing, and watched and listened to them thoroughly. Face-to-face empathize gatherings were conducted with seven (7) children (five girls and two boys, aged between 5-10) living in a slum area and studying in a government primary school (with almost free tuition fees) in Dhaka, Bangladesh, between October 15-20, 2021. The number of school-age children was relatively low due to the ongoing COVID-19 pandemic measurements, limiting public gathering. They were asked to describe their experience of solving their mathematics-related quizzes and puzzles, e.g., how they connect with others for group-based mathematical problem solving, things that might increase their tasks, and what type of technological tools might be helpful to them to solve the quizzes and puzzles. Six (6) school-age students (86%) reported a lack of connection with others for group-based problem-solving tasks. Overall, all seven (7) participants (100%) strongly recommended a personalized gamified learning system to connect with others for conducting problem-solving tasks such as mathematical puzzles and quizzes. They showed interest to use the game elements such as points and leaderboard. 3.2

Define

Based on the results of the interviews, the following problem statement has been drawn: “School-age children in developing countries sense a lack of opportunities to conduct group-based mathematical quizzes and puzzles. A technological tool using game elements can support them to motivate in performing their quizzes and puzzles in the group environment.”

The above POV has concluded how we might build an application with the help of game elements to which users can experience more fun in their mathematical quizzes and puzzles. 3.3

Ideate

The POV from the Define step has led us to design the prototype of the gamified learning system. The idea of developing the prototype was to observe users’ responses and how they responded using a gamified learning system. The prototype was designed by adding game elements (points and leaderboards). 3.4

Prototype

The objective of constructing the prototype was to verify whether students can run a gamified learning system and how do they react in using the system. An example of an app's gamified learning system prototype is presented below (Fig 2). Students can enter the system using a given PIN in the system (screen 1). Students can input their personalized name and tutor monitors once everyone enters the system (screen 2). A math quiz question will then be visible on the screen with an allocated time, such as 30 seconds so that students can solve the puzzle by choosing the correct answer (screen 3). Once they choose the answer, the Kahoot system automatically calculates the correct answer and those who answered at the earliest time are placed on the top of the ranking leaderboard with points earned so far (screen 4). We then tested both the low-fidelity paper and mid-fidelity interactive prototypes with thirteen (13) participants for a short period of 10 minutes (Fig 2, Fig 3). Participants had been asked to pretend to use the prototypes as if they were solving the tasks in real life (Fig 4). The test result has informed us that the participants preferred a Kahoot based online platform for solving primary level math quizzes and puzzles and preferred the reward-based tasks in the gamified system. Eleven (11) participants (85%) reported that the Kahoot-based gamified prototype is user-friendly, answering the RQ1. In terms of the RQ2, all participants reported points-based tasks into the Kahoot was fun as they felt so happy to receive those points in doing the puzzles and quizzes. In addition to, these 6 participants (46%) also wanted to share their points with those who did not have enough points. However, the students' overall experience of motivation in learning was positive. The number of participants in the study is comparatively low due to the COVID-19 pandemic. Most students study from home, and they can be difficult to reach. Talking about gamification techniques such as points badges might negatively affect the gamified learning system's participants [ 18 ]. Earning points or earned badges may cause the negative effects of gamification [ 19 ]. While gaining points may motivate those doing well, it is important to consider the possible demotivational impact of ranking low in comparison to peers. The design of the gamified learning system is still at an early stage and needs to be further evolved by involving the students with more iterative design thinking steps. Participants only pretended to use the prototypes; they did not use the real gamified system, which might bring different value to them. Gamification positively impacts some users over a short period [ 20 ]. In this study, participants were involved in using a learning system installed in smartphones or tablets. Literature review, suggestions and comments from the participants will be considered to introduce the real working gamified learning system.

This is a work-in-progress paper. Future studies will focus on a brief description of how the CSLT theory could be incorporated into the system and an additional iteration of the Design Thinking iterative process. Further ethical reflection in relation to our research design should be followed, especially as the participants are school-age children or students from socio-economically disadvantaged backgrounds. Research design can, in a way, be seen as taking advantage of an already disadvantaged group. Relevant organizations such as UNICEF have several resources regarding research ethics on school-age children, which we could utilize in our study.

Once the real working system is developed, an experimental design (in-between subject technique) will be conducted with 60 economically disadvantaged students aged between 5-12 years from Bangladesh for four weeks. Participants who are living in the slums and low-level cosmopolitan areas will be contacted and invited to take part in the study via invitation letter and consent form as well as ethical clearance approval. Upon participants' consent, they will be appointed randomly to 1 of the 2 groups (30 students of group A will be in the experimental group, and thirty students of group B will be in the control group). The experimental group will use the gamified learning system in solving primary level math quizzes and puzzles, and the control group will have their usual daily routine in solving primary level math quizzes and puzzles without using any technological system for four weeks.

Systems usability scale (SUS) [ 21 ] evaluation, a usability scale questionnaire method, will be applied to assess the quality and performance of the digital learning system. The impact of the point-based rewarding system can be notified by accessing the online database of each learner group within the Kahoot-based online gamified system and how they are scoring daily points. After using a gamified learning system and usual daily routines for a week, two parallel surveys shall be conducted for the experimental and control groups. While participants do their daily routines, i.e., solve their mathematics-related quizzes and puzzles without using any learning system, a postquestionnaire form will be provided by asking specific questions regarding their daily activities without using any learning intervention. Once participants complete the study, a post-questionnaire form will be provided to the participants by asking specific questions regarding the digital gamified learning intervention. Selected participants (those who are critical of the gamified system to better understand their perspective and pinpoint the issues with the design) shall be face-to-face interviewed and audio recorded. Hence, semi-structured 10-minute interviews [ 22 ] will be conducted and audio recorded with all participants who complete the study. Thus, the analyses of the collected data will demonstrate empirically that the gamified learning system positively impacts socio-economically disadvantaged students’ satisfaction and completion of their learning tasks. It is hoped that the gamified system will positively impact students' competence to increase their learning performance, and completion of this study is expected at the end of 2022.

References

1. Anderson , J. ICT Transforming Education: Regional Guide . Bangkok: UNESCO ( 2010 ).

2. UNICEF Report . Two-thirds of the world school-age children have no internet access home [Online] Available from: https://www.unicef.org/press-releases/ two-thirds-worlds-schoolage-children-have-no-internet-access-home-new-unicef-itu [ Accessed on 10.02 . 2022 ] ( 2020 ).

3. Hubber , P. J. , Outhwaite , L. A. , Chigeda , A. , McGrath , S. , Hodgen , J. , and Pitchford , N. J. Should Touch Screen Tablets Be Used to Improve Educational Outcomes in Primary School Children in Developing Countries? Frontiers in psychology, 7 , 839 . ( 2016 ).

4. UN . Goal 3: Ensure healthy lives and promote well-being for all at all ages . Retrieved from https://sdgs.un. org/goals [Accessed 17 . 02 . 2022 ] ( 2019 )

5. Devedzic , V. ( 2006 ). Semantic Web and Education . Integrated Series in Information Systems ( 2006 ).

6. Wang , A. I. and Tahir , R. The effect of using Kahoot! for learning - A literature review , Computers & Education , 149 . ( 2020 ).

7. Hamari , J. , Koivisto , J. and Sarsa , H. Does Gamification Work ? - A literature review of Empirical Studies on Gamification, 47th Hawaii International Conference on System Science ( 2014 ).

8. Hassanzadeh , A. , Kanaani , F. , and Elahi , S. A model for measuring e-learning systems success in universities, Expert Systems with applications: An International Journal , 39 ( 12 ), pp. 10959 - 10966 . ( 2012 ).

9. Tseng , M. , Lin , and A. R. Chen , H. Evaluating the effectiveness of e‐learning system in uncertainty , Industrial Management & Data Systems 111 ( 6 ). ( 2011 ).

10. 10. Jenkinson , J. Measuring the Effectiveness of Educational Technology: What are we attempting to Measure? Electronic Journal of e-Learning , 7 ( 3 ). pp. 283 - 280 ( 2009 ).

11. Deterding , S. , Khaled , R. , Nacke , L. and Dixon , D. Gamification: Toward a Definition, CHI 2011 , May 7- 12 , 2011 , Vancouver, BC, Canada ( 2011 ).

12. Geelan , B. , Salas , K. , D. , Lewis , I. , King , C. , Edwards , D. , and O 'Mara , A. Improving Learning Experiences Through Gamification: A Case Study , Australian Educational Computing , 30 ( 1 ) ( 2015 ).

13. Haque , M.S. , O 'Broin , D. and Kehoe , J. Designing a Theory-Driven System Model for Postgraduate Research Students to Encouraging Progression and Social Connectedness , CEURWS The 1st gamiFIN Conference , May 2017 , 1857 , pp. 22 - 31 . Pori, Finland ( 2017 ).

14. Bandura , A. Social foundations of thought and action . Upper Saddle River, NJ: Prentice Hall ( 1986 ).

15. Smaradottir , B. , Gerdes , M. , Martinez , S. , and Fensli , R. The EU-project United4Health: User-centered design of an information system for a Norwegian telemedicine service , Journal of Telemedicine and Telecare , 22 ( 7 ), 422 - 429 ( 2016 ).

16. Haque , M.S. , O 'Broin , D. and Kehoe , J. To Gamify or not to Gamify? Analysing the Effect of Game Elements to Foster Progression and Social Connectedness . Proceedings of GAMEON: 18th Annual European GAME-ON'2017 Conference , Carlow, Ireland ( 2017 ).

17. Haque , M. S. , Konopelko , M. , O 'Broin , D. and Kehoe , J. Prototyping a gamified app to investigate the effect of an individual game element in encouraging progression and social connectedness: An empirical approach . Proceedings of European Conference on Gamesbased Learning (ECGBL 2018 ), Nice, France ( 2018 ).

18. Thom , J. , Millen , D. and DiMicco , J. Removing gamification from an enterprise SNS , In Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work, February 11-15 , 2012 , Seattle, Washington, USA, ACM, pp. 1067 - 1070 ( 2012 ).

19. Hamari , J. Perspectives from behavioral economics to analyzing game design patterns: loss aversion in social games , In the CHI 2011 Social Games Workshop , Vancouver, BC, Canada, ACM ( 2011 ).

20. Farzan , R. , DiMicco, J.M. , Millen , D.R. , Brownholtz , B. , Geyer , W. and Dugan , C. Results from deploying a participation incentive mechanism within the enterprise ( 2008 )

21. Brooke J. SUS: A "quick and dirty" usability scale . In T.B. A.W. Jordan P. W. and I.L . McClelland (Eds.), Usability Evaluation in Industry, Taylor Francis, London. 189 - 194 ( 1996 ).

22. Kallio , H. , Pietilä, AM., Johnson , M., and Kangasniemi , M. Systematic methodological review: developing a framework for a qualitative semi-structured interview guide ” J Adv Nurs , 72 ( 12 ): 2954 - 2965 ( 2016 ).