Profile-Based Algorithm for Personalized Gamification in Computer-Supported Collaborative Learning Environments

Abstract

In this paper we present an approach for personalizing gamification to the needs of each individual person. We designed the personalization for computer-supported collaborative learning environments by synthesizing three existing design frameworks: the lens of intrinsic skill atoms, gamification user type hexad and heuristics for effective design of gamification. The result of the design process is a context-aware and personalized gamification ruleset for collaborative environments. We also present a method for translating gamification rulesets to machine-readable classifier algorithm using the CN2 rule inducer and a framework for connecting the produced algorithm to collaborative software. Lastly, we present an example software for personalized gamification that was produced by applying the process presented in this paper.

Keywords

• Author Keywords gamification
• adaptive systems
• personalization
• computer-supported collaborative learning

-

INTRODUCTION Collaborative learning is a learning method where students have a symmetry of action, knowledge and status, and have a low division of labor [ 9 ]. Computer-supported collaborative learning facilitates the interaction with software tools and Permission to make digital or hard copies of all or part of this work for personal or academic purposes is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

Copyright © by the paper’s authors.

GHITALY17: 1st Workshop on Games-Human Interaction, September 18th, 2017, Cagliari, Italy. increases potential for creative activities and social interaction [ 26 ]. In recent studies, it has been shown that students can be guided towards educational goals like collaboration by using gamification [ 20 ], which is the application of game-like elements to non-game environments [ 8 ].

However, gamification is not a "one size fits all" solution [ 29 ]. Early research on the gamification of education concentrated on exploratory research and proof of concepts, or specifying the user interface elements by which gamification manifests in systems [ 24 ]. Recent literature reviews assessing the potential of gamification in education found several positive implications, like increased engagement and motivation [ 20, 24 ], although some studies also link gamification to negative consequences, like unproductive competition or reward saturation that leads to demotivation [ 10, 24 ]. Different authors have pointed to contextual and personal differences to explain these mixed results and have called upon future research to take these characteristics into account [ 2, 18, 29 ]. We propose that in order to make gamification more usercentric and customized to the individual user in computersupported collaborative learning (CSCL) environments, the systems should include profiling of users in its design principles and selectively choose gamification features presented to each user. In this study, we present the design process for creating an algorithm to choose challenge-type gamification tasks in CSCL systems, and a proof-of-concept algorithm. More specifically, our research goals are: 1. How can personalized gamification features be designed to address the preferences of different user types? 2. How could customized, profile-based gamification challenges be assigned to different users in CSCL environments? In the design process, we use the design heuristics for effective gamification in education [ 23 ] to create a gamification task ruleset personalized for each user type as defined in the gamification user type hexad [ 28 ]. We then use the CN2 rule induction algorithm [ 4 ] to create a classifier to identify different conditions that occur in a CSCL environment as discovered by Knutas [ 15 ] and to recommend gamification tasks for the main CSCL system.

GAMIFICATION IN EDUCATION Approaches that use some elements of gamification have been shown to increase student collaboration and motivation in educational settings [ 20 ]. However, effective gamification is about using the game elements to foster users’ three innate needs for intrinsic motivation1 [ 24 ], originally adapted from Deci and Ryan’s self-determination theory [ 5 ]. These principles are [ 5 ]: Relatedness, the universal need to interact and be connected with others; Competence, the universal need to be effective and master a problem in a given environment; Autonomy, the universal need to control one’s own life.

Studies in the field indicate that gamification methods are successful in fostering collaboration, especially when following the principles of self-determination theory [ 16, 27 ]. At the same time, individual elements of gamification have been studied, and recent research concludes that simply applying a single outward aspect of gamification, like badges or other repetitive rewards [ 13, 24 ], does not work, and instead gamification has to consider the motivation and goals of the course as a complete system.

PERSONALIZATION IN GAMIFICATION Different authors have pointed to various potential confounds while aiming to explain the mixed results found in literature. For example, the unexpected negative effects could be due to bad gamification design [ 10, 13 ] or to the particular interplay between the gamified system and the implementation context [ 8, 29 ]. Also, personal characteristics are hypothesized to impact gamification’s potential [ 2, 18 ], as such being a possible explanation for the otherwise-presumed negative consequences.

Research shows that different users interpret, functionalize and evaluate the same game elements in radically different ways [ 22 ]. In much the same way, Koster [ 17 ] reasons that it is impossible to design a universal "fun" game, as different predispositions and social structures bring a unique, personalized sense of fun for everyone. To exemplify this personalized meaning making of game elements, Antin and Churchill [ 1 ] theoretically distinguished five different functions a user can ascribe to a badge. Taken together, these results make an argument for using gamification that is specifically tailored to its users, in order for gamification to live up to its full potential [ 6, 23 ]. Furthermore, the success of such personalization techniques has already been proven in other digital contexts, like persuasive technologies and games (see for example [ 14, 25 ]). In this line of reasoning, we argue that player types can be a valuable tool to personalise gamification. This way, we build on Monterrat and colleagues’ work [ 19 ] in which they use predefined player types and a player adaptation model in order to improve the matching of gamification elements to the preferences of the user.

1Intrinsic motivation in gamification literature; autonomous motivation in self-determination theory literature Gamification Player Type Hexad We selected the gamification user type hexad by Tondello et al. [ 28 ] as a model for personalized design when creating gamification approaches. They used a survey with 133 participants and quantitative methods first to develop and then validate a response scale for assessing user preferences. This user model was selected over alternatives because it is evidence-based and gamification-specific.

The user types are summarized in Table 1. With each user type we also present intended gamification approach. The disruptor user type was defined as out of scope in this project. This user type tends to disrupt the system and is difficult to address within the context of the system. Instead, they will be addressed by other types’ autonomy and relatedness -related challenges and by being involved in the development of the system.

GAMIFICATION DESIGN PROCESS Technology designed for changing users’ attitudes or behavior in online systems is known as persuasive technology [ 11 ]. Oinas-Kukkonen and Harjumaa further define persuasive software as “computerized software or information systems designed to reinforce, change or shape attitudes or behaviors or both without using coercion or deception” [21, p. 486]. Adding gamification features to computer-supported collaborative learning can be considered persuasive software because the design intent is to change user behavior.

We used the three-element persuasion context framework defined by Oinas-Kukkonen and Harjumaa [ 21 ] to initially frame the design for the personalized gamification system. The intent (1) is on the part of the designers is to increase collaboration. The designers intend to use the principles of gamification first to affect behavior that leads to positive attitude changes. The event (2), or the use context, is user activity in the collaborative system. Their goal is to accomplish course-related tasks. Our strategy (3) for persuasion is to use gamification elements to give users personalized, constructive gamification tasks and motivating feedback through the system.

The overall design process followed Deterding’s framework [ 7 ] for creating gameful designs. The framework presents principles to create gameful designs for motivation and enjoyment, which can be applied to create gamified software. Using these principles, Deterding states that "in pursuing her needs, a user’s activity entails certain inherent, skill-based challenges. A gameful system supports the user’s needs by both (a) directly facilitating their attainment, removing all extraneous challenges, and (b) restructuring remaining inherent challenges into nested, interlinked feedback loops of goals, actions, objects, rules, and feedback that afford motivating experiences." [7, p. 315] Our design process followed the five steps presented in Deterding’s framework [ 7 ], detailed in the following paragraphs. 1. Strategy. Target outcome is increased collaboration between students and increased engagement in the CSCL platform. The flexibility of the system is constrained by automatically measured environmental variables and the functionality of the platform.

Player type

Description

Tasks that channel exploration into sharing resources, and tasks that acknowledge and reward the joy of discovery.

Motivated by competence. They seek to progress within a system by completing tasks, or prove themselves by tackling difficult challenges.

Easiest to address within the framework. Tasks that are competitive or gather around achieving the "next level", e.g. with points or badges.

Motivated by extrinsic rewards. They will do whatever to earn a reward within a system, independently of the type of the activity.

Similar to achiever’s, except the mix of tasks includes more tasks that encourage working with others and building a positive sense of community. 2. Research. The user activity was translated into behavior chains by analyzing current literature on CSCL and using principles of persuasive design to frame the event structure. User needs and motivations were adapted from current literature on motivation and Tondello’s evidence-based gamification user type hexad. 3. Synthesis. The principles of self-determination theory [ 5 ], collaborative learning [ 9 ] and heuristics for the design of gamification in education [ 23 ] were used to design challenges in the form of gamification tasks presented to the users. These were considered in the context of possible actions that can be taken in a CSCL system. 4. Ideation was performed in a series of workshops, where a panel of experts ideated rules with a note-taker translating the ideas to the skill atom framework and presenting the results for approval. The panel of experts consisted of three experts on game design, three experts on gamification and education, and two software engineers. The ideation process resulted in a total of 69 gamification tasks for five different player types. When duplicates were collated, it resulted in 42 individual tasks. 5. Iterative prototyping, the last step, was performed partly and left partly for future work. The ruleset and the algorithm were tested and evaluated. Combining the ruleset with a live CSCL system is part of future work.

Design Heuristics for Gamification The panel of experts that participated in design workshops were informed by principles of good collaborative learning [ 9, 12 ], gamification user type hexad [ 28 ], and the selfdetermination theory -based design heuristics for effective gamification of education [ 23 ] during the design process of the ruleset. Below, we present the design heuristics and how they guided the design process. #1 Avoid obligatory uses. The computer-supported collaborative learning environment and especially its gamification features should be voluntary to use. #2 Provide a moderate amount of meaningful options. The user is able to choose which gamification tasks to accomplish, if any. Furthermore, as the challenges are based on the user’s characteristics, these challenges are relevant to each person and as such present meaningful options to the user. #3 Set challenging but manageable goals. No task is meaningless or impossible to accomplish. Also, the difficulty level of the implemented challenges are tuned to the users’ capabilities, as such keeping the tasks manageable, while at the same time being challenging. #4 Provide positive, competence-related feedback. Just as tasks should be meaningful, the feedback is meaningful and positive. There should not be any feedback that can be perceived as a punishment. When presented in a CSCL system, the feedback should make the user feel capable. #5 Facilitate social interaction. There are several gamification tasks that show the positive impact the user’s actions can have on each other. CSCL systems are social by their nature and several tasks promote positive interaction. #6 When supporting a particular psychological need, be wary to not thwart the other needs. The gamification tasks should not concentrate on promoting only one aspect over others. For example, when promoting relatedness and prompting users to interact, users should not feel that they are forced to, and thus feel less autonomous. #7 Align gamification with the goal of the activity in question. Gamification tasks support both motivation and goal achievement. CSCL systems should not distract from accomplishing actual team and learning goals. #8 Create a need-supporting context. The system should be voluntary, open and supportive. When the algorithm is integrated to a CSCL environment, it should be presented as a supportive feature, not the main feature. #9 Make the system flexible. The gamification system is adaptive, providing personalized challenges to different user types. The adaptive approach is the main novel contribution of this project for CSCL systems.

Structuring Gamification Tasks Deterding’s framework provides a method to structure gamification design elements, called the lens of intrinsic skill atoms [ 7 ]. It uses two elements, skill atoms and design lenses, to identify challenges in a user’s goal pursuit and restructure them to afford gameplay-characteristic motivating, enjoyable experiences. Deterding names this design perspective the lens of intrinsic skill atoms. Design lenses combine a memorable name, a concise statement of a design principle and a set of focusing questions to evaluate game design from a specific perspective [ 7 ]. Skill atoms originate from an effort to develop a formal grammar for games, in which skill atoms are the smallest defined elements, of which the following are used in gamification: goals, actions, objects, rules, feedback, challenge, and motivation.

We used this lens of intrinsic skill atoms to structure our gamification system’s elements. The columns of Table 2 follow this structure. The table presents one sample gamification task for each player type.

Goal: An extra, quest-like challenge that the user needs to accomplish. Something that is presented to the user by the system based on the recommendation of the algorithm. Action: Set of actions that the user can take in the system to achieve the goal. Defined in columns Task 1 and 2. Object: What the user can act upon, or the system state. In this case the conditions of Prerequisite 1 to 3 define which goals and actions are presented to the user.

Rules: Specification of what actions the user can take and how they affect the system. In this system’s case they are inherent to the functioning of the CSCL environment and the variables monitored by the system.

Feedback: Sensory information that informs the user of system state changes. In this system’s case this is left open for the implementer of the CSCL environment. However, one minimal approach is presenting a notification and a badge when a goal has been achieved by a user’s actions.

Challenge: The difficulty of achieving the goal, caused by the difference in system state and user’s perceived current skill. The tasks should be meaningful and always make the user feel that he or she made a real contribution to the collaborative environment.

Motivation: The psychological needs energizing and directing the user to seek out and engage with the system. In this system’s case feelings of competence, relatedness, and autonomy. ALGORITHM FOR ADAPTIVE GAMIFICATION The algorithm is based on the ruleset presented in the previous section. It is designed to choose context-dependent, personalized gamification tasks for users of a specific variety of a computer-supported collaborative learning system. It is based on a classifier created with the CN2 rule induction algorithm [ 4 ], which condensed the ruleset into a set of if-else -conditions. When activated, it uses the environmental variables to decide which quest-type task should be presented to the user.

In this case, gamification task means tasks that correspond to a set of goals that need to be met, in a manner that is for example similar to a quest in a video game. The task assignment, accomplishment and feedback process follows the "new goal - rules - action - challenge - feedback - motivation" loop of the lens of intrinsic skill atoms [ 7 ], as presented in the design section.

The algorithm is designed to act as a stateless plugin for a specific type of computer-supported collaborative learning environment. It integrates to the CSCL system as presented in Figure 1. It depends on the system to give it snapshots of status variables, which it uses to recommend gamification tasks. The system is responsible for task accomplishment tracking, feedback, and other interaction features. However, the ruleset is also presented in a human readable format in the online appendix and contains some recommendations for task presentation. The algorithm depends on the CSCL system for system status as input, such as user gamification type, user skill, issue tracker task activity and discussion system activity. The full list is presented in the Online Appendix 2. 1. Interaction (2). User behavior parameters 4. Response and gamification tasks (3). Gamification task proposal, if conditions match The algorithm design makes the following assumptions on the system: 1) The users of the systems are students who are willing and allowed to help each other, 2) the students are engaged in collaborative teamwork and have series of tasks to do, 3) there is a system to track the tasks assigned, such as GitHub or a CSCL system presented by Knutas [ 15 ], 4) the system tracks when participants work on tasks and allows external help, and 5) there is a free-form synchronous discussion system associated with the CSCL environment. Machine-Format Rule Creation with CN2 The CN2 rule induction algorithm is a basic component of many machine learning systems. It creates a list of classification rules from examples using entropy as its search heuristics [ 4 ]. In this case, the examples are the list of prerequisites that can trigger the conditions for providing personalized gamification tasks and the classes are individual gamification tasks the algorithm should offer. The CN2 rule inducer was originally designed to function in a noisy environment and to find a minimal number of rules that cover a maximal number of cases. The list of cases was already pre-vetted by the panel of 2 https://doi.org/10.5281/zenodo.827225

Prerequisite 1 (player type)

experts, so the CN2 inducer parameters were deliberately set to cause overfitting in order to cover all of the cases. The rule induction process from 69 human-defined rules resulted to 59 machine format if-else rules. For example, the rules for the third task (Free spirit) in Table 2 was induced into a following rule: "IF Hexad = Free Spirit AND Chat Activity != Low AND Ownteam opentasks = high AND Ownteam task age = high AND Ownteamactivity != high THEN Challenge_class = 7 (Quality 0.125)". The CN2 rule inducer was used in unordered mode, which means all the rules are evaluated and the algorithm does not stop after the first match. When several rules match, the one with the highest quality is selected.

The full list of rules, training data, variables and the algorithm itself, stored as an Python-based Orange Data Mining classifier 3, are available in the Online Appendix 2. Orange was selected as the classifier implementation because it provides a Pythonbased library and enables programmers to load and use the classifier without in-depth knowledge of machine learning. The appendix contains a short, interactive program for testing the classifier.

DISCUSSION AND CONCLUSION All gamification approaches are not suited for everybody, which means that for gamification to have more of an impact, the gamification system should be personalized to respond to the needs of each individual user. In this paper we presented an approach to create personalized gamification rulesets using a framework [ 7 ] for creating playful designs and design heuristics [ 23 ] for effective gamification (research goal 1). The ruleset was induced into machine-format rules that can be used as a plugin algorithm for computer-supported collaborative learning environments in order to select personalized gamification tasks for specific user types and situations (research goal 2).

The presented algorithm can be used to improve collaborative learning systems that are looking to add or improve personalized gamification features. Moreover, it makes a distinction between the interaction environment and interaction rules. The decoupling between the environment and the ruleset allows gradual development and improvement of gamification without having to re-develop the logic of the entire system. It also allows sharing rulesets as plugins for others to use. 3https://orange.biolab.si Previous studies indicate that personal characteristics affect how people respond to game elements [ 22 ] and this can have an impact on the effectiveness of gamification [ 2, 18 ]. The approach to designing user type specific rules presented in this paper are one solution to increasing personalization in gamification. While models and designs have been published (e.g. [ 3, 19 ]) for personalization through adaptation in gamification, to our knowledge this is the first published realization of such designs in collaborative systems for education.

The approach presented in this paper builds on theoretical work from the field of gamification research, and existing concepts from other domains. As future work more testing with the algorithm will be conducted by implementing a CSCL system, which can be used to evaluate and validate the algorithm in a series of tests.

ACKNOWLEDGMENTS Research was partially funded by European Union Regional Development Fund grant number A70554, "Kyberturvallisuusosaamisen ja liiketoiminnan kehittäminen," administrated by the Council of Kymenlaakso.

References

1. Judd Antin and Elizabeth F. Churchill . 2011 . Badges in social media: A social psychological perspective . In CHI 2011 Gamification Workshop Proceedings . ACM New York, NY, 1 - 4 .

2. Gabriel Barata , Sandra Gama, Joaquim Jorge, and Daniel Gonçalves . 2015 . Gamification for smarter learning: tales from the trenches . Smart Learning Environments 2 , 1 (Dec. 2015 ).

3. Martin Böckle , Jasminko Novak, and Markus Bick . 2017 . TOWARDS ADAPTIVE GAMIFICATION: A SYNTHESIS OF CURRENT DEVELOPMENTS . In Proceedings of the 25th European Conference on Information Systems (ECIS) . Guimarães, Portugal.

4. Peter Clark and Robin Boswell . 1991 . Rule induction with CN2: Some recent improvements . In European Working Session on Learning . Springer, 151 - 163 .

5. Edward L. Deci and Richard M. Ryan . 2012 . Motivation, personality, and development within embedded social contexts: An overview of self-determination theory . The Oxford handbook of human motivation ( 2012 ), 85 - 107 .

6. Sebastian Deterding . 2014 . Eudaimonic Design, or: Six Invitations to Rethink Gamification . SSRN Scholarly Paper ID 2466374. Social Science Research Network , Rochester, NY.

7. Sebastian Deterding . 2015 . The Lens of Intrinsic Skill Atoms: A Method for Gameful Design . Human-Computer Interaction 30 , 3 - 4 (May 2015 ), 294 - 335 .

8. Sebastian Deterding , Dan Dixon, Rilla Khaled, and Lennart Nacke . 2011 . From Game Design Elements to Gamefulness: Defining "Gamification" . In Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments (MindTrek '11) . ACM, New York, NY, USA, 9 - 15 .

9. Pierre Dillenbourg . 1999 . What do you mean by collaborative learning? Collaborative-learning: Cognitive and computational approaches 1 ( 1999 ), 1 - 15 .

10. Adrián Domínguez , Joseba Saenz-de Navarrete , Luis de Marcos, Luis Fernández-Sanz, Carmen Pagés , and José-Javier Martínez-Herráiz. 2013 . Gamifying learning experiences: Practical implications and outcomes . Computers & Education 63 ( April 2013 ), 380 - 392 .

11. Brian J. Fogg . 2002 . Persuasive technology: using computers to change what we think and do . Ubiquity 2002 , December ( 2002 ), 5 .

12. David W. Johnson and Roger T. Johnson. 1999 . Making cooperative learning work . Theory Into Practice 38 , 2 ( 1999 ), 67 - 73 .

13. Karl M. Kapp . 2012 . The gamification of learning and instruction: game-based methods and strategies for training and education . John Wiley & Sons.

14. Maurits Kaptein , Panos Markopoulos, Boris de Ruyter, and Emile Aarts . 2015 . Personalizing persuasive technologies: Explicit and implicit personalization using persuasion profiles . International Journal of Human-Computer Studies 77 (May 2015 ), 38 - 51 .

15. Antti Knutas . 2016 . Increasing Beneficial Interactions in a Computer-Supported Collaborative Environment . Number 718 in Acta Universitatis Lappeenrantaensis . Lappeenranta University of Technology.

16. Antti Knutas , Jouni Ikonen, Uolevi Nikula, and Jari Porras . 2014 . Increasing Collaborative Communications in a Programming Course with Gamification: A Case Study . In Proceedings of the 15th International Conference on Computer Systems and Technologies. ACM , 370 - 377 .

17. Raph Koster . 2013 . Theory of fun for game design. " O'Reilly Media , Inc.".

18. Elisa D. Mekler , Florian Brühlmann, Alexandre N. Tuch , and Klaus Opwis . 2017 . Towards understanding the effects of individual gamification elements on intrinsic motivation and performance . Computers in Human Behavior 71 ( June 2017 ), 525 - 534 .

19. Baptiste Monterrat , Michel Desmarais, Élise Lavoué, and Sébastien George . 2015 . A Player Model for Adaptive Gamification in Learning Environments . In Artificial Intelligence in Education . Springer, Cham, 297 - 306 .

20. Fiona Fui-Hoon Nah , Qing Zeng, Venkata Rajasekhar Telaprolu, Abhishek Padmanabhuni Ayyappa, and Brenda Eschenbrenner . 2014 . Gamification of education: a review of literature . In International Conference on HCI in Business . Springer, 401 - 409 .

21. Harri Oinas-Kukkonen and Marja Harjumaa . 2009 . Persuasive systems design: Key issues, process model, and system features . Communications of the Association for Information Systems 24 , 1 ( 2009 ), 28 .

22. Rita Orji , Regan L. Mandryk, Julita Vassileva, and Kathrin M. Gerling . 2013 . Tailoring persuasive health games to gamer type . In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM , 2467 - 2476 .

23. Rob van Roy and Bieke Zaman . 2017 . Why Gamification Fails in Education and How to Make It Successful: Introducing Nine Gamification Heuristics Based on Self-Determination Theory . In Serious Games and Edutainment Applications , Minhua Ma and Andreas Oikonomou (Eds.). Springer International Publishing, 485 - 509 .

24. Katie Seaborn and Deborah I. Fels . 2015 . Gamification in theory and action: A survey . International Journal of Human-Computer Studies 74 (Feb . 2015 ), 14 - 31 .

25. Hayeon Song , Jihyun Kim, Kelly E. Tenzek , and Kwan Min Lee. 2013 . The effects of competition and competitiveness upon intrinsic motivation in exergames . Computers in Human Behavior 29 , 4 ( July 2013 ), 1702 - 1708 .

26. Gerry Stahl , Timothy Koschmann, and Dan Suthers . 2006 . Computer-supported collaborative learning: An historical perspective . Cambridge handbook of the learning sciences 2006 ( 2006 ), 409 - 426 .

27. C. Thomas and K. Berkling . 2013 . Redesign of a gamified Software Engineering course . In 2013 International Conference on Interactive Collaborative Learning (ICL) . 778 - 786 .

28. Gustavo F. Tondello , Rina R. Wehbe , Lisa Diamond, Marc Busch, Andrzej Marczewski, and Lennart E. Nacke . 2016 . The Gamification User Types Hexad Scale . In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '16) . ACM, New York, NY, USA, 229 - 243 .

29. Rob van Roy and Bieke Zaman . 2015 . The inclusion or exclusion of teaching staff in a gamified system: an example of the need to personalize . In CHI Play '15 Workshop 'Personalization in Serious and Persuasive Games and Gamified Interactions'.