Towards a Systematic Approach for Designing Gamification for RE Naomi Unkelos-Shpigel Information Systems Department, University of Haifa Carmel Mountain 31905, Haifa, Israel naomiu@is.haifa.ac.il Abstract. [Context and Motivation] RE presents several challenges and risks stemming from the required collaboration and knowledge transfer between many different stakeholders, including analysts, developers and customers. [Objectives] This research aims to address these challenges by developing a gamification framework for enhancing motivation among professional software developers and analysts, to perform RE. To provide a theoretical basis for the developed framework, we perform a systematic literature review, in order to understand motivation-related issues, and the use of gamification techniques, in the context of underperformed tasks in RE. [Preliminary Results]. This paper presents preliminary results based on a subset of this review, eliciting docu- mented associations between motivation issues and RE tasks, and between mo- tivation issues and gamification. [Contribution] The research will provide a theory-based framework for systematically designing RE environments embed- ding gamification techniques, so to motivate engagement, collaboration and knowledge sharing between the stakeholders involved in the RE process. Keywords: Requirement Engineering, Gamification, Motivation. 1 Introduction Stakeholders involved in requirements engineering (RE) processes include customers, managers, developers, and more. The RE process requires knowledge sharing and collaboration between different stakeholders, varying in their backgrounds and tech- nical knowledge. Engagement is described as reflecting a person’s active involvement in a task or activity and can be expressed in behavioral and in cognitive aspects1. Proposed solu- tions to facilitate high engagement of stakeholders and an ongoing communication between them in the RE process typically focus on user involvement (e.g., 10). Some solutions have recently employed the idea of gamification, which has been receiving increasing attention, considered of high potential in this context [7]. Literature reports on various attempts to gamify different processes of RE ([7] [12] [13]). However, additional research is required to substantiate and fulfill the potential of gamification. Moreover, while these efforts show promise for improvements, a Copyright 2018 for this paper by its authors. Copying permit- ted for private and academic purposes. systematic approach is yet to be provided for leveraging gamification for improving RE. 2 Background 2.1 Gamification and its implications in RE Gamification is defined as the use of game design elements in non-game contexts, in order to encourage individuals to participate in certain tasks and contribute to their success 5]. In recent years, various gamification elements have been embedded in different information systems and applications in general and, more recently, in appli- cations intended for the use of software engineers, and specifically in the context of RE. For example, gamification was used in virtual teams during requirements elicita- tion, and was found to assist the teams to locate experts and share their knowledge14. Dubois and Tamburrelli 6 identified three types of activities needed to be performed when engaging gamification into RE: analysis, integration, and evaluation, and found that students performing these activities had better results in RE. These research works have mostly focused on measuring the outcomes of embed- ding gamification techniques in early stages of RE processes 61114, but with no spe- cial attention to the individual influence of different gamification techniques, or of the combination thereof, on the results. Moreover, much of the research in this area has so far focused on creating gamified tasks, and analyzing their motivational contribu- tion post-implementation. Fulfilling the potential benefits of gamification requires more consideration of the various motivation theories when designing gamified envi- ronments 15[18]. 2.2 Motivation theories and gamification Gamification techniques are designed to enhance motivation among participants in the gamified environment. Therefore, it is to be expected that when designing such a mechanism, motivation theories would be considered. Recently, an understanding has emerged that these theories should be used for the analysis of the success or failure of gamification techniques 15. Motivation theories have been used as a basis for game mechanics for several dec- ades, but only few empirical attempts explicitly used motivation theories in order to understand and analyze game players’ behavior [12][15]. For example, Vorderer et al. [16] studied motivation among video games’ players. They explained players’ en- gagement, motivation and enthusiasm by the short-term goals and challenging tasks in the game, based on motivation theories. The conclusion from this research was that competition during the game helps to motivate participation in the game. While each of the empirical studies conducted so far provides some specific insights and contri- bution, a comprehensive framework which maps evidence-based associations between motivation and gamification is yet to be achieved. This research-in-progress aims at analyzing existing literature on motivation and gamification, and accordingly propose a research agenda toward bridging this gap in the field of RE. 3 Research objectives and questions This research aims to understand how gamification techniques can be used in RE for promoting productive behavior contributing to their successful completion. As a first step toward addressing this objective, and while acknowledging that gamification aims to invokes motivation and engagement 5, we set out to answer the following questions: RQ.1. What RE tasks may benefit from increasing the motivation of their executors? RQ.2. What gamification techniques affect motivational aspects? RQ.3. What gamification techniques affect observed behavior? The answers to the above questions will enable constructing a framework linking between RE tasks and examples of gamification techniques that may be effective for enhancing these tasks, using motivation theories as mediators. This framework will thus provide a structured guidance for a research agenda, toward its completion as a knowledge base for designing gamification solutions for RE. Thus far, a first round of literature review was conducted, in order to provide a pre- liminary answer to each of the above-listed questions. For RQ.1 we looked for research works that examined enhancement of various tasks in the RE process, building on motivation theories. For RQ.2 and RQ.3, we looked for research examining the use and effect of gamification in relevant contexts. 4 Research Agenda and Method The research will result in an integrated gamified collaborative framework, which will be built iteratively using design research method. During the research, data is gathered via SLR, complemented by interviews and questionnaires. The goal of the data collection is to understand the challenges of the current RE process, and how the process can be enhanced using gamification tech- niques. The theoretical framework will be developed according to qualitative data analysis. In order to ensure the validity of the research, triangulation among different knowledge sources is applied [2]. Following data collection and analysis, we plan to propose a conceptual framework for gamifying RE processes. We will construct and evaluate our solution according to the design research guidelines [8, p.10]:  We will build a technology-based artifact, the conceptual framework and the gami- fied environment that will be embedded in the RE process. The solution will assist in addressing current RE challenges (design as an artifact; problem relevance)  We will use rigorous qualitative and quantitative methods to evaluate this solution, while applying it in several case studies, in order to evaluate the actual contribution to the industry (research rigor design and contribution)  We will iteratively construct our solution, utilizing the solution in several steps, to ensure that it will be applicable to many organizations. We will pursue the publica- tion of our solution in both academic and practitioners’ venues, in order to receive feedback, and to enrich the existing knowledge in the field of RE from both theo- retical and practical points of view. (design as a search; communication)  We will use well-executed qualitative and quantitative methods to evaluate the final version of the proposed solution. The qualitative study will focus on under- standing how different gamification techniques affect stakeholders’ motivation and behavior. In the quantitative part of the evaluation, we will focus on measuring the quality of the requirements document, to assess improvement (design evaluation). 5 Research Progress So Far The literature review conducted thus far was not executed according to the principles of systematic literature review (SLR). Rather, a more exploratory, initial search was conducted in order to grasp a first understanding regarding the research questions and the kind of information relevant search could reveal. Since a full SLR research is highly demanding in time and effort, we found it important to first test the water and assess whether perusing this direction of research has merit. The following sub-sections describe the preliminary search conducted, the results obtained, and a first conceptualization based on these results. Based on the search attempts and the preliminary findings, we now have a better picture of what we can expect from conducting the full SLR research. 5.1 Motivational and behavioral factors in RE The main goal of this part of the literature review was to identify literature that ad- dresses motivation-related issues in the context of underperformed tasks in RE. The search was conducted in Google Scholar, which is acceptable as an exclusive source for reference and cited-by data, in the domain of software engineering [17]. Our criteria for paper inclusion were (a) reported empirical studies, and (b) exami- nation of motivational aspects in RE tasks. Following several attempts to accurately define the search so to identify relevant papers accordingly, the following query string was used as a first screening: “requirement engineering motivation empirical”. This search yielded about 288,000 results in Google Scholar. Reviewing the first 50 results obtained, the number of papers found eligible to be included in the literature review, according to the inclusion criteria listed above, was 12. Reviewing results 51-100 yielded no relevant papers, and we therefore did not continue, at this stage, to review results beyond the first 100 listed. Fig. 1 maps the topics investigated in these papers to behavioral and cognitive elements, and to the context in which they were investigated in terms of the RE stages (according to 16). Fig. 1 Cognitive and behavioral elements of motivation in RE 1 Six different cognitive elements related to motivation were examined with regard to RE tasks, most of them in the context of elicitation and specification, and one ele- ment (commitment) was examined in the context of feasibility study. Examining be- havioral elements was performed mostly in the context of elicitation, in which the work process is more easily monitored and controlled. Some of the papers addressed specification, and one examined communication in the context of validation. As evi- dent from Fig. 1, some research effort has been invested in examining different cogni- tive and behavioral elements of motivation in the context of various RE tasks. Howev- er, these research efforts seem to be performed in silos, with no commonly agreed upon theory or ontological framework to systematically guide these efforts and their outcomes. 5.2 Gamification, motivational and behavioral factors This part of the literature review aimed at reviewing research works, which examined the associations between gamification techniques and motivational affordances. The main goal of this step was therefore to examine which different gamification tech- niques are associated with motivational affordances, and which gamification tech- niques, if any, were examined separately to isolate their individual correlation with, or effect on motivation or behavior. Here, too, the search was conducted via Google Scholar, using the query string: “gamification motivation empirical” yielding about 127,000 results. Our criteria for paper inclusion were (a) reported empirical studies, and (b) exami- nation of associations between using gamification in any performed task, and motiva- tional factors. We decided to exclude papers reporting on gamification designed for education purposes, but rather include only gamification for tasks in the workplace, as motivational aspects in educational and work settings differ significantly. Thus far, over 80 papers were reviewed, 50 of which were found to be addressing both motivational affordances and gamification techniques. Some of the research works discussed the behavioral aspects of motivation, and some discussed its cogni- 1 Papers cited on arrows are listed in :https://sites.google.com/site/naomiushpigel/gamification tive aspects. Fig. 2 presents the mapping between investigated gamification tech- niques and their associations with motivational affordances. Most reviewed papers addressed the elements of commitment, collaboration or (directly referring to the term) motivation. Several papers discussed immersion, but only with regards to fun, an inconclusive and subjective term 13. Importantly, most gamification techniques were not discussed with regard to the synergy with other techniques. Fig. 2. Gamification and cognitive elements of motivation 6 Conclusions and Future Work We are now in the midst of conducting a large-scale SLR, based on relevant libraries (IEEE, Scopus, AIS, ACM), to be completed in a few months. The SLR is based on the previously described queries, and is widened according to the results of the full review, as well as identifications of relevant papers using additional search tech- niques, such as the snowball technique, eliciting additional relevant key terms. Based on the SLR results, and as a basis for a strategic approach for designing gamification for RE, we will construct a framework associating RE tasks with motiva- tional factors, and motivational factors with gamification techniques. This evidence- based framework will serve as association-based mapping between gamification and RE, mediated by motivational factors. It will enable to leverage on the progress of general gamification research to guide RE gamification efforts, directing empirical research on gamification in RE. As the framework evolves, it will provide guidance for designing gamification solutions for different RE tasks, according to motivational needs. Thus, allowing more systematic efforts of enhancing RE via gamification. Based on our framework, we will develop a tool called REVISE: Requirement Elicitation and Verification Social Environment. The tool will enable practitioners to use various gamification mechanisms each phase of the RE process, dynamically choosing examples of different gamification combinations for RE tasks. We intend to perform a user study among practitioners, and to what extent, these means promote desired behavior and improve RE performance. Acknowledgments. The advisor of this research is Dr. Irit Hadar, Department of Information Systems, University of Haifa, Haifa, Israel. References 1. Appleton, J. J., Christenson, S. L., Kim, D., and Reschly, A. L.: Measuring cognitive and psychological engagement: Validation of the student engagement Instrument. Journal of School Psychology, 44(5), 427-445. (2006). 2. Bjarnason, E., and Sharp, H.: The role of distances in requirements communica- tion: a case study. Req. Eng., 1-26,(2015). 3. Bogdan, R. C., and Biklen, S. K. Qualitative research in (validation) and qualitative (in- quiry) studies. (2006). 4. Damodaran, L. (1996). User involvement in the systems design process practical guide for users. Behaviour and Information Technology, 15(6), 363-377. 5. Deterding, S., Khaled, R., Nacke, L., and Dixon, D.: Gamification: Toward a definition. In CHI 2011 gamification Workshop Proceedings, 12-15. (2011). 6. Dubois, D. J., and Tamburrelli, G.: Understanding gamification mechanisms for software development. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, ACM, 659-662. (2013). 7. Fernandes, J., Duarte, D., Ribeiro, C., Farinha, C., Pereira, J. M., and da Silva, M. M.: iThink: A game-based approach towards improving collaboration and participation in re- quirement elicitation. Procedia Computer Science, 15, 66-77. (2012). 8. Hevner, A. R., and March, S. T.: The information systems research cycle. Comput- er, 36(11), 111-113. (2003). 9. Huotari, K., and Hamari, J.: Defining gamification: A service marketing perspective. In Proceeding of the 16th International Academic Mindtrek Conference, ACM, 17-22. (2012). 10. Kujala, S.: User Involvement: A review of the benefits and Challenges. Behaviour and In- formation Tech., 22, 1-16. (2003). 11. Law, F. L., Mohd Kasirun, Z., and Gan, C. K.: Gamification towards sustainable mobile application. In 5th Malaysian Conference on SE (MYSEC), 2011, IEEE, 349-353. (2011). 12. Lombriser, P., Dalpiaz, F., Lucassen, G., and Brinkkemper, S.: Gamified requirements en- gineering: model and experimentation. In Int. Working Conference on Req. Eng.: Founda- tion for Software Quality, 171-187, (2016). 13. Mandryk, R. L., Atkins, M. S., and Inkpen, K. M.: A continuous and objective evaluation of emotional experience with interactive play environments. In Proc. of the SIGCHI Conf.e on Human Factors in Computing Sys., ACM, 1027-1036. (2006). 14. Marshburn, D. G., and Henry, R. M.: Improving Knowledge Coordination in Early Stages of Software Development Using Gamification. In Proceedings of the Southern Ass. for In- fo. Sys. Conf. Savannah, Ga, USA. (2013). 15. Richter, G., Raban, D. R., and Rafaeli, S.: Studying gamification: The effect of rewards and incentives on motivation. In Gamification in Education and Business. Springer Inter- national Publishing, 21-46. (2015). 16. Sommerville, I.: Software Engineering. Intl. comp. sci. series. Addison Wesley. (2012). 17. Vorderer, P., Hartmann, T., and Klimmt, C.: Explaining the enjoyment of playing video games: the role of competition. In Proceedings of the second international conference on Entertainment computing (pp. 1-9). Carnegie Mellon University. (2003) 18. Unkelos-Shpigel, N., and Hadar,: Inviting everyone to play: Gamifying collaborative re- quirements engineering. In Empirical Requirements Engineering (EmpiRE), 2015 IEEE Fifth International Workshop on ,13-16, (2015). IEEE.