Coding skills have gained popularity beyond the technical programs of university education, leading to the emergence of many free online resources for informal learning. Among them, self-paced "learn to code" (SPL2C) apps are very popular in digital distribution stores. However, the academic literature agrees that acquiring coding skills is a demanding endeavour, with one of the highest dropout rates among all fields in Computer Science. For that reason, such apps are very sensitive to user retention, which they often address by including game-like design principles or elements that nicely mesh with their sandbox "study at your own pace" nature. The present paper studies this category of apps in order to identify the most relevant ones at this moment, while analyzing which are the most popular game elements used in them. This makes it possible to compare the gameful design approaches in this kind of apps with others in the wider educational context.
•Human-centered computing ! Human computer interaction (HCI); •Applied computing ! Computer-assisted instruction; •Social and professional topics ! Adult education;
The increased interest in acquiring coding skills has lead to
the appearance of a large number of compelling informal
resources. Directed at kids, there are LOGO-like games, such as
Coddy [
Focusing on this last group of mobile apps, their approach to
cope with users’ motivation and retention problem is to offer
a more flexible and engaging experience in comparison to
traditional courses. Usually, as their core approach, they offer
a sandbox environment, structuring knowledge into small bits
that can be consumed step-by-step, with a very clear learning
path that allows students to slowly advance at their own pace,
without the constraints of rigorous study terms. Often, this
scaffolded learning approach is complemented with the use
of game-like design principles or elements (without
becoming full fledged games themselves) [
The papers is structured as follows. First of all, a brief background on the study of gameful approaches in the context of learning is laid out in Section 2. Following, Section 3 presents the methods of research, how the data was compiled and the main body of the analysis of the selected apps. Section 4 provides reflection on on the results and how they relate to other works on gameful approaches and learning. Finally, Section 5 concludes this paper and outlines further work.
The applications of gameful approaches in the context of
education at large have been extensively studied in several
literature reviews or surveys, all of them published after the term
"gamification" had reached the peak of Gartner’s Hype Cycle
for Emerging Technologies [
Some of such reviews focus on aggregating large amounts
of demographic data, such as the ones by Caponetto [
Focusing on gameful approaches in the specific subject of
learning programming, a survey was published in 2017 [
The purpose of this study was to find out which kind of gameful approaches are the most popular in "learn to code" mobile apps. On that regard, it is worth noting that all of the previous literature reviews (see Section 2) studied published papers, but did not directly evaluate technological platforms per se.The data gathering procedure followed a systematic process devised to identify as many apps as possible that could then become candidates for a more in-depth personalized study of their characteristics.
There are some literature reviews that do categorize papers by
game elements. The one by Nah [
The research questions for this study were: RQ1 - How common is the use of gameful approaches among the most relevant SPL2C apps? terms used and the inclusion of other more generic keywords, C language apps were sufficiently included nevertheless. RQ2a - What are the most common gameful elements in SPL2C apps? RQ2b - Are the gameful approaches in this context different from the ones found in education at large? RQ3 - How are the gameful elements implemented and which are the idiosyncrasies of the "app" context?
Candidate apps were selected through a multi-step process that relied mostly on the Google Play Store. The Play Store was chosen given Android’s mobile market share (about 75%), and its simplicity in the app publication process, which makes it a popular default choice for developers. Its public API also allows the automation of some tasks, opening the possibility to create an initial very large pool of apps and process their meta-data, something that would be extremely time consuming using a fully manual approach.
Step 1) A very broad search was executed on Google’s Play
Store [
Using this approach, a total of 114 different queries were executed, obtaining a total of 2214 individual results. Once repetitions across separate queries were taken into account (612), the number of starting app candidates was set at 1602. A preliminary inspection showed that, given the broad search
Step 2) The list of candidates was pruned according to the following set of strict and basic criteria which would ensure some degree of popularity and relevance of the chosen apps. It should be taken into account that the uncurated nature of the Play Store allows anybody to publish an app, regardless of its quality or following. Many apps are hardly ever known by the public, or downloaded by very few people. The criteria used to prune the list of candidates from Step 1 were:
To provide some perspective, a very popular learning app such as Duolingo, ranked 100 in the Play Store at the time of this writing, has been rated 7.327.126 times at 4,7 stars, and downloaded more that 100 million times.
Again, this was a fully automated process, from the app metadata also obtained during Step 1. The chosen values were finetuned several times until an approachable number of results was obtained. Under these final criteria, 594 apps did not fulfill b), 763 did not fulfill c), 1552 did not fulfill d), and 1291 did not fulfill e), for a total of 1565 discarded apps. It must be noted that many apps did not fulfill several criteria at once, and all apps fulfilled criteria a). After the filtering process, the number of app candidates was reduced to only 37. Step 3) From this step on, the process moved from an automated system to a manual one. The resulting apps were installed on a Pixel Android phone. In order to identify whether gameful approaches were used for user engagement, for each app, its description was read, and then executed and manually inspected. The main reasons for discarding apps during this step follows, with the total number of discarded apps between parentheses at the end:
Directed at children or games. Not about professional programming languages (2).
Actually, instances of the same app, each for a different programming language (9).
Programming tools, but not really learning apps. E.g. An integrated Development Environment (IDE) or plain API references (10).
The final number of core candidate apps was thus reduced to 9.
Step 4) As an additional step to fine tune the selection process, the list of "similar apps" provided by the Play Store for each of the 9 apps was then taken into consideration, searching for additional suitable candidates, somehow missed during the automated processes. During this step, interesting apps were taken into consideration even when they did not strictly meet all criteria applied during the automated process of Step 2 (mostly, minimum number of times rated). Four additional apps were considered eligible, for a total of 13 final candidates.
The final list of apps considered for study is shown in Table 1.
A summarized name, that is used in some parts of the rest of
this paper, is written in column "Short". Some general stats
such as store rating (Stars), number of times rated (Ratings)
and approximate minimum number of Downloads are also
included, to provide some insights about their popularity. It
should be noted that the Play Store does not provide an exact
number of downloads, just a minimum-maximum range. In
addition, a footnote with the exact package name has been
included in the table for those apps with a very generic name,
broadly repeated in the store, to better identify them.
The analysis was based on interaction with each application
as a standard user for one week. During this time, different
gameful elements were identified and classified accordingly.
Some possible categorizations were presented in Section 2.
However, the chosen approach was based on "The Periodic
Table of Gamification Elements" proposed by Marczewski in
his Gamification Hexad User Types framework, a widely
accepted and popular approach [
Socializers (SOC): Motivated by relatedness. They want to interact with others and create social connections. Free Spirits (FRS): Motivated by autonomy and selfexpression. They want to create and explore.
Achievers (ACH): Motivated by mastery. They are looking to learn new things and improve themselves. They want challenges to overcome.
Philanthropists (PHI): Motivated by purpose and meaning. They are altruistic, enjoying when giving to other people and enriching the lives of others in some way with no expectation of reward.
Players (PLA): Motivated by rewards. They will do what is needed to collect rewards from a system.
Disruptors (DIS): Motivated by change. They want to disrupt the system, either directly or through other users, to force change that suits their interests.
It must be noted that, even though the list is divided by user type, in this study, the eight categories defined by Marczewski are only used to better organize game elements. Studying the relationship between the identified elements and user type preferences was out of the scope of the study.
The summary of the results provided by the in-depth analysis is shown in Table 3, located in Appendix I because of its size. Columns describe apps, listed using its summarized name (see Section 3.2) and rows enumerate gameful approaches, using Marczewski’s classification (see Section 3.3). Even though Marczewski’s list includes up to 52 gamification elements, only those that were actually identified in any app during the analysis are listed, for a total of 25. Elements included in a given app are marked as "Y", and those not included as "N" (i.e. Yes/No). The end column shows how many times a particular gameful element appeared across each of the different apps, whereas the end row shows the total sum of game elements included in each individual app.
From the information in the last row of the summary table, it is possible to provide an answer to research question RQ1, "How common is the use of gameful approaches among the most relevant "learn to code" apps?". The number of elements included range between 2 and 18 (mean=7.615, std=4.61). In 5 out of all 13 studied apps the use of gamification is very marginal. 2 elements only, in three of them: "Programming Hub", "Master Android" and "Java Programming". In the rest, the use of gameful approaches could be considered common and consistent with the app itself. A special mention is worth for "Programming Hero" and "SoloLearn", both at the top with 15 and 18 elements, respectively. Therefore, these two could be considered the most relevant ones in the context of gameful approaches in SPL2C apps.
Regarding differences among gameful approaches in apps and other educational contexts, the last column from Table 3 allows clearly answering research question RQ2a, "What are the most common gameful elements in SPL2C apps?". The top five most common gameful elements in this kind of apps are, by order: "Progress/Feedback", "Points", "Badges", "Fixed Rewards", and "Challenges". Its is noteworthy that in those applications with a very limited number of gameful elements, Points and Badges (both among the most common) are usually the ones of choice. Also, since there are many instances of different kinds of reward elements, Marczewski defines three different subtypes (fixed, time-dependent and random), they appear in 8 out of 13 different apps.
In order to answer RQ2b, "Are the chosen approaches in this context different from the ones found in education at large?", the top five elements from each review that did perform a categorization (see Section 2), are also listed in Table 2. For a better comparison, the first column shows the results from RQ2a, ordered by rank. On that regard, it is worth noting that there is a difference between the concept of "Level" as defined by Nah, and the rest of the literature (including this paper). In the former, its meaning is equivalent to "Stage", whereas in the rest of the publications it is used to define some kind of character growth or power. Therefore, it has been considered that Nah’s "Levels" are similar to Marczewski’s "Challenges" or Koivisto’s "Quests". Also, Marczewski considers "Progress/Feedback" as a single element, whereas other studies often differentiate between "Feedback" and "Progress" (usually just labelled as "Progress bars"). Therefore, for a fair comparison, both kind of elements are aggregated in the final results.
From this ranking, it can be seen that elements such as "Points", "Badges" And "Leaderboards" are very common and appear in almost all of them, one might say unsurprisingly. This triad of elements is very common in the literature, usually used in conjunction and labelled as PBL (referring to the initials of each element). However, it is an interesting finding that "Leaderboards" are not actually that common in SPL2C apps in particular (2 out of 13 apps use them). In addition, the top element in this kind of apps is "Progress/Feedback", which only tops the rankings, 5th position but very close to the 6th, in Koivisto’s ssurvey. Therefore, two key differences have been detected between approaches in education at large and SPL2C apps.
Finally, the answer to research question RQ3, "How are this gameful elements implemented and which are the idiosyncrasies of the app context?", requires going beyond the summarized data, and using a more qualitative approach, extracted from the interaction with the apps as users during the analysis. Answering this provides better insights on how game elements work.
As seen in the answer to RQ2b, some very common gameful elements in education at large are also popular in SPL2C apps, such as points, badges or level ups, and their implementation is not too different. However, the top element is "Progress/Feedback", usually implemented also in a very straightforward manner with progress bars, showing how far into a lesson or course the student has progressed. However, sometimes the preferred metaphor is a semicircular gauge and a percentage. There are also some more elaborate variations, such as an adventure map that an avatar representing the student traverses (Grasshopper, Programming Hero; see Figure 1a). On regards to the "feedback" part of this element, lesson completion is often assigns stars or medals, in a similar manner to levels in some videogames, depending on exercise accuracy (Codemurai, Py, Programming Hero; ; see Figure 1a and 1b). Only SoloLearn provides some extra aggregate feedback and global student stats by given different categories, quite similar to gamification player-types: learner, programmer, challenger, influencer and collaborator (see Figure 1c). The next kind of gameful element at the the top five in this kind of apps are Reward related ones, almost always implemented using a Fixed Schedule or Time Dependent strategy (using Marczewski’s nomenclature). The former type is very common, always taking the form of daily or weekly "streaks". The app registers, and clearly shows, the number of consecutive days a student has logged in and advanced in a lesson. After some number of consecutive days, usually a week, the student is rewarded, mostly, with some variation of points. A few apps also use a Time Dependent strategy (DataCamp, Codemurai, Mimo, SoloLearn), on top. In addition, points are also automatically awarded every day the user logs in, or there is a daily challenge (i.e. an exercise or test) that is only available for 24 hours, mimicking the behaviour of many Freeto-Play mobile games. In one case (Codemurai), this system also incorporates a Consequence element when exercises are failed, using a videogame-like "3 lives" mechanism, that goes beyond simple scoring.
It is also worth noting that, even though several different gameful elements related to socialization (Marczewski’s Philatropist or Socializer player types) have been identified, few apps take them into consideration. They are seldom integrated in the app itself, and mostly part of the dynamics of an external discussion forum. The only exception is the Sharing Knowledge element, based on the ability to publish coding personal projects and process comments/voting mechanisms from other users. In fact, SoloLearn is the only app that makes a relevant use of this kind of elements, of special note being the ability to duel with other students in a player-vs-player competition. On regards to the rest of gameful elements, even though the analysis shows a wide breadth of different ones, up to the 25 identified, their use is marginal (about half of them make 1-3 appearances at most). However, among them, the use of narrative or themes in two apps, an often underused element, deserve some attention. In Grasshopper, the student is guided through the lessons by this animal, as a kind of pet and companion during the coding exercises. In Programming Hero, the student becomes an astronaut and uses the metaphor of space travel through different planets (lessons) to advance through the content (see Figure 1a). Therefore, the use of a metaphor goes beyond the "Progress/Feedback" element, as previously mentioned, and permeates the whole user experience.
This paper has analyzed the most relevant SPL2C apps in order to assess their use of gameful approaches for user retention, and how they are implemented. The goal was to compare them to approaches in education at large, to try to find the particularities of the mobile app educational context, if any. Also, this study has shown that gameful approaches are also used in this kind of apps, and in some cases make an extensive use of them.
From the answers to the proposed research questions, it is shown that this kind of apps have their own idiosyncrasies. Even though some very popular gameful elements in education at large are included, such as points and badges, some others are not, as is the case of leaderboards. Furthermore, they especially emphasize elements related to progress/feedback, which play a minor role in most proposals in the literature related to gamification in education. Timed challenges are also found to be quite relevant, but not usual in education at large.
Expected future work is twofold. On one hand, a more usercentric analysis would be suitable, measuring up to which degree the chosen gameful elements are actually engaging, and taking into account player types (or any other approach to categorize the different kinds of users). On the other hand, and moving beyond the topic of gamification itself, the chosen apps will be further analyzed in order to classify their preferred educational resources and tools. This will allow to assess how common these resources are in the educational literature and their perceived usefulness in learning programming.
This work was partly funded by the eLearn Center (Universitat Oberta de Catalunya) through the Xtrem Project programme.
1. J. Boustedt, A. Eckerdal, and R. et al McCartney. 2011.
Students’ Perceptions of the Differences Between Formal and Informal Learning. In Proceedings of the Seventh International Workshop on Computing Education Research (ICER ’11). ACM, 61–68.
r P a N N N N N N N N N N Y N N Y N N N N N N N N N N N 2 J r P e N N N N Y N Y N N N Y N N Y Y N N N N Y N N Y N N 7 L y Y N Y N Y N N N N N N N N Y Y N Y N N N N N N N N 6 P c A 0 o Y Y N N Y N N Y N N Y N N Y N Y Y N N Y N N N Y N 1 C e L 8 o N Y Y N Y Y N Y Y Y Y Y N Y Y Y Y N Y Y Y Y N Y N 1 S u rH N N N N Y N N N N N N N N N N Y N N N N N N N N N 2 P e H 5 r N N N Y Y Y N Y N Y Y Y N Y Y Y Y Y N Y N N N Y Y 1 P s a r Y N Y Y Y N N Y N N Y N N Y N N Y N N N N N N Y N 9 G o m i N N Y N Y N N Y Y N Y N N N N Y Y N N N N N N N N 7 M i k n N Y Y N Y N N Y N N N N N N N N N Y N N N N N N N 5 E e d o N N N N Y Y N Y Y N Y N Y Y Y N N N N N N N N Y N 9 C a C a N N N N Y N N Y Y N N N N Y Y N Y N N N N N N Y N 7 D
s le rd ) ls u a ts rs P /SFSTEERAU t/ii--rraaougnodbnTO iit-spognngS l-oFw -eehTm /r-rsseaeecbdkgoFP -sseecequnnoC ttr-aegyS i-recaeedhddxSFRw it-eeeeepndnTRDwm -rsaaeoddnRRwm /i-saeeecegvhndBAm /-rrsaeeaaebddoddLL tilr-acuyonoEVm it/i-r(seceenpxnoPEX l/i-rrssseeegoonvPL titr-secaefiC ll-saeenghC li/-ssaeduTGm titi-()epvnooPPCm iltt-scaauoSS ilt-rcaekooSNw titilr-saevooyTC tiit-sazonouCm ilr-aeengdgonhSKw it/i-ecognoVV lllitseeeecndudnm P N N N N N N N H H H A A A A H H H C C C C S S I S ta P E E E E E E E C C C L L L L C C C O O O O R R H I o A G G G G G G G S S S P P P P A A A S S S S F F P D T