Data-Driven Gamification Design: An Enterprise Systems Perspective from the Front Line Marigo Raftopoulos Strategic Innovation Lab Marigo@strategicinnovationlab.com Abstract systems can be considered important for the design, development and implementation of robust The need for data-driven gamification in management information systems (IS) capability. enterprise systems is essential for the design, development and implementation of robust management information systems capability. Decision support systems (DSS) and in particular However, the gamification of enterprise business intelligence systems (BI) are critical for systems need to look beyond the user interface enterprise management decision making [1] [2] [3] and as a key driver of the effectiveness of system have been subject to extensive research on user design and implementation by taking a more acceptance and utilisation of this technology [4] [5] [6]. holistic approach. A survey was undertaken of Information systems have also recently been subject to 25 global organisations that have implemented how they can be gamified to improve motivational a gamification project which found that affordances [7] [8] [9]. Research has also focussed on a enterprises are reporting positive results from wide range of fields that include the use gamification gamification projects, but are also claiming mechanics and dynamics [10], experimentation of that there’s still room for improvement across using then taking away gamification elements as a test many operational areas. In particular, there are for its stickiness [11], an exploration of gamification effectiveness issues associated with effects on user constructs [12], and the development of technology and vendor maturity, and a need to a modelling language for information systems use [13]. improve the capabilities of organisations in BI software is a collection of decision support the design and implementation of gamification technologies for the enterprise aimed at enabling projects. Design was considered to be a knowledge workers such as executives, managers, and collaborative activity amongst stakeholders, analysts to make better and faster decisions [6] [14] and the process and inclusiveness of the [27]. Cognitive and behavioural sciences have design process was considered to be just as traditionally produced empirical information that has important as the specific design elements assisted in the design of decision support systems from employed. a human-computer interaction perspective [15] [17] and more recently, gamification has been used to 1 Introduction develop more engaging user interfaces to encourage Data-driven gamification design (DDGD) has been asset utilisation [16] [18] and user enjoyment [15] [10]. defined as the automation of the gamification design The contention of this paper is that gamification of process using data mining and algorithms to enterprise systems needs to look beyond the user personalise the user experience [30]. This domain has interface or motivational affordances as a key driver of arisen out of the need to overcome the perceived the effectiveness of system design and implementation. problem in gamification design in assigning game A gamified enterprise system has essentially two appropriate design elements to motivate users and to interdependent components; a front-end and a back-end maximise their expected contribution to the overall [19]. The front-end relates to the motivational system goal [31]. affordances and user interaction elements, which has The need for data-driven gamification in enterprise been the focus area of gamification researchers and practitioners. However, the back-end, relating to DSS Copyright © by Marigo Raftopoulos. Copying permitted for private and and BI systems design and implementation, has not academic purposes. received as much attention. As an emerging domain, In: M. Meder, A. Rapp, T. Plumbaum, and F. Hopfgartner (eds.): DDGD is focussed on the motivational affordances to Proceedings of the Data-Driven Gamification Design Workshop, match or personalise game design elements to user, Tampere, Finland, 20-September-2017, published at http://ceur-ws.org however, research in management information systems informs us that the engaging design of user interfaces is only one of many determinants of the success of an Europe, India and Australia, and the projects were a information system [20] [6] [4]. Therefore, the need for mix of internal facing (staff) and external facing a more holistic approach to DDGD is important for its (customers) gamification projects. ongoing growth and maturity. A total of 17 multiple choice questions were asked on a range of operational areas, and three sets of questions using 5-point Likert scales for responses to 20 sub- 2 Research Focus questions relating to organisational experiences with designing and implementing a gamification project To delve deeper into understanding how gamification over a range of strategic and operational areas. In can be designed and implemented as a holistic system, addition to these structured questions, three open-ended a research project was undertaken to investigate the questions were asked on the topics of key success direct experience of organisations that have factors, barriers to success, and recommendations on a experimented with gamification in their business design process. Due to space limitations, and relevance processes in order to identify the key enablers, barriers, to this call, only the results of the three open-ended and capabilities for successful implementations. This questions are presented in this paper. research was one of three inter-related research studies The procedure for analysing the data from the open- undertaken as a part of a doctoral research program that ended questions commenced with the documentation of also developed and published a gamification design themes using a code-book method which was then used process [32] and gamification taxonomy [28]. for the systematic evaluation of the text-based This research found that what was missing from the responses. Card sorting and affinity mapping methods current discourse in gamification research was a lack of were used to provide a broad visual display of all the first-hand perspectives from enterprise project leaders key words/phrases and this then enabled the grouping on the procurement, development and integration of of responses into themes and categories. This then gamification with enterprise systems and processes, enabled the quantitative analysis of the qualitative data. and on navigating the internal systemic, cultural and The key theme that had emerged out of the open- decision-making processes required for effective ended questions was that the factors raised by implementation. respondents tended to cluster around the three To address this gap, the focus of the research project categories of management, technology and design involved a confidential, in-depth online survey of 25 issues related to the gamification project. Further global organisations that have implemented an investigation showed that this three-part classification enterprise gamification project. This was a selective of technology, design and management is not sample based on organisations that have implemented a uncommon and is a widely used schema in information gamification project. A total of 40 organisations were systems research [33]. contacted and 25 had agreed to participate in the Each of these three categories are explored in detail survey. In all cases, the project leader who was within each of the open-ended questions in the responsible for the gamification project had completed following section of this paper. the survey. The combined projects in this sample equated to 11.4 million users (a combination of both internal staff and external customers or stakeholders) 3 Results and Discussion that have been affected by these gamified enterprise 3.1 Question 1: Key Success Factors applications. Most previous research in the enterprise gamification Respondents were asked: “Please name up to three domain has focused on an evaluation of peer-reviewed strategies that were key to the relative success of your studies or experiments undertaken in single project”, and a total of 42 responses were received. organisations. Thus, an opportunity was identified in Management factors received 43% of overall this study to survey a cross-section of global responses, design received 36% and technology 21%. organisations based on their direct experiences with In relation to management, the key success factors that enterprise gamification across a range of strategic and were raised included: operational factors, to ascertain their common views on  Project management. This included stakeholder enablers and barriers to successful enterprise engagement and management, communication, gamification implementation. The organisations were sponsorship, and building internal networks. large global companies with operations in the US, 2  Teamwork. This included interdepartmental co- gamification project: (a) the unique challenges of operation, teamwork with vendors and consultants gamification in terms of generating an appropriate and participation of stakeholders. gameful design and selecting appropriate gamification  Measurement. This included the setting of clear technologies, which are often new capabilities for an goals, targets and key performance indicators organisation; and (b) the adeptness in which a project (KPIs), as well as measuring and reporting on manager can navigate the gamification project through performance against KPIs. a business transformation process. The implications for DDGD is that the findings In relation to design, the key success factors that were support the notions of a need for more improved raised included: gamification design elements and methods, however it  Design aspects. This included setting design warns of the need for more considered attention to the objectives and design principles, possessing design technology that is employed, and the need for building skills and an understanding of motivational implementation capability and the development of psychology, prototyping and testing, and aligning appropriate metrics to make the project an overall game elements to business goals. success.  Target audience. This included understanding of the target audience, organisational culture, and 3.2 Question 2: Barriers to Success undertaking a deep analysis of the players. Respondents were asked: Please name three barriers to success that you experienced during the project; and a In relation to technology, the key success factors that total of 50 responses received. As a barrier to project were raised included: success technology received 38 per cent of all  Agile development. This included flexible and mentions, followed by management at 34 per cent, and iterative development, usability testing, internal design at 28%. support and freedom to select and develop the right Technology factors listed by respondents as a barrier technology, and learning from mistakes. to project success indicate critical shortcomings in core  Technology. This included two key themes – the technical IT and IS elements, including: vendor experience of the vendor, and the flexibility of the capability, technological limitations, gamification gamification platform to meet project requirements. platform restrictions, data integrity issues, limited reporting capabilities, vendors not knowing the target Respondent sentiment on what was critical to the market, on-time delivery, scalability issues, success of their gamification project is illustrated in the development team resources and user adoption of the sample responses listed below: platform.  “Interdepartmental cooperation was essential – IT, These technology barriers also suggest a significant HR, Marketing, Financial Planning.” limitation in the enterprise gamification domain that  “We started with a test and learn phase (beta) has not been previously identified in the industry, developed by a small, tight, focused team over a where gamification failure has generally been long gestation period (24 months).” attributed to poor design decisions [29]. The existence  “It was designed and built brick by brick, and we of this level of technological limitation presents a never lost focus of what we wanted to achieve.” potentially high barrier to the further growth and It appears as if most enterprises treat a gamification development of the domain. project as they would any other project management To gain a deeper perspective of respondents’ views exercise, with results indicating the key to successful on gamification technology barriers, below is a implementation centres around project integration and selection of their corresponding quotes: business transformation. This brings into question the  “Barriers were primarily with technology: we need to distinguish what parts of an enterprise waited a long time for vendors to mature, [and] gamification project are unique to gamification, and even then I do not believe vendor solutions are which parts are standard project management issues if mature enough yet to handle large-scale, complex they are to be managed effectively. enterprise use cases. We faced a lot of challenges A deeper investigation of these responses suggests with integration, especially with our data security that there may be two key considerations for requirements.” developing and implementing an enterprise 3  “Our IT infrastructure is not state-of-the-art. That content to the process or system under review, whilst meant that the vendor had to develop for an ‘old’ being constrained by limited stakeholder skills, situation. They could not re-use their new familiarity and acceptance of gamification. This technologies, neither their experience.” suggests a difficulty among project teams in  “Gamification platform restrictions are not yet understanding design principles and design capability, adapted to communities with serious content where including how gameful design elements can be reputation and quality are key. There are limited creatively integrated into ‘serious’ business reporting capabilities and data integrity issues.” applications. The key issues raised in relation to project To gain a deeper perspective of respondents’ views management as a barrier to successful gamification on design, below is a selection of corresponding implementation are as follows: decision-making, quotes: stakeholder management, management buy-in,  “There were times I felt that I was playing the inadequate envisioning, budget constraints, lack of a wrong game.” clear strategy, resourcing, time pressures, unrealistic  “Not everyone liked our design.” expectations and assumptions, and limited  “There was meaningless use of arbitrary game organisational priority and communication. mechanics.” It can be said however, that these factors are not  “There was an inability to identify useful business uncommon in the domain of business transformation or topics on which to apply gamification.” change management [21] [22] as well as innovation Specific issues relating to gamification design often management [23] [24] [25], yet appear to have received appear to stem from frustrations in understanding how limited attention in enterprise gamification research. To design can provide the ‘bridge’ between the business gain a deeper perspective of respondents’ views, here is problem, and the technology front- and back-ends of a selection of relevant quotes: the proposed solution. The survey results show that  “In a big organisation such as ours, getting approval enterprise interest in applying gameful design to for these kinds of projects is tough.” business problems is often paralleled with a lack of  “The path of decision-making in content finesse and balance in the design component of the development and implementation was and still is process. This suggests that the role of the designer is quite bureaucratic.” essential; yet design and design process expertise  “Decision-makers could not envision what users appears to be underdeveloped, which has often resulted will experience when playing the game. That in less effective design decisions. caused the inability to decide.” The most significant finding in regard to barriers to  “It was hard to measure success and set up KPIs.” success (and enablers) was not the responses that were  “We didn’t have a clear strategy when we started – voiced, but rather those that were not. When addressing we had to make it up as we went.” open-ended questions about barriers and enablers, Once again, these issues are common to the project respondents did not refer to the motivational management and change management domain, and affordances of the gamification elements or the these survey results indicate that better use of these effectiveness of the project to engage users. corresponding domains could help inform the ongoing Respondents mostly believed that indicators of success, development of data-driven gamification for enterprise or barriers, are predominantly based on how well a applications. project is managed, the robustness of the technology, Design factors was deemed to be both a major and its integration within the organisation’s systems success factor for gamification projects, as well as a and processes. It would appear that motivational notable barrier if it was not done well. Some of the key affordances in terms of the right balance of gameful issues that caused design to be perceived as a barrier design features, while of significant importance, rank include: staff not being familiar with gamification, user secondary to enterprise system and process integration. resistance to gamification, use of arbitrary game mechanics, lack of game design expertise, too much 3.3 Question 3: Creating an Optimal Design focus on game elements, and balancing the right game Process content. Respondents were asked: Knowing what you do now, Respondents’ concerns focussed on the challenge of how would you create a better gamification design balancing the right selection of gamefulness and process? and a total of 27 responses were received. 4 Respondents mostly echoed what was said in relation  “We would like to see an extended version of the to success enablers and barriers, in terms of the game to turn passion and intuitive gameplay into a importance of rigorous project management and robust deeper consideration of the issues.” technological platforms.  “I think that it is more important to be clear on your More revealing however was the high concentration goals and your audience. There was a disconnect of design factor responses, which mostly related to the between the prototypes and concepts being importance respondents placed on internal design discussed and the stated goal, the audience of the capability issues. This implies that project management game.” and technology tend to be standard core competencies  “Be focused on the target audience, define critical in the enterprise, while design is less so. This also success factors for the game at the game design indicates that design methodologies and capabilities are stage, be innovative with the game mechanics.” not yet at the level they should be for enterprise gamification. A close examination of such responses indicates that project owners are in effect talking about the need for Furthermore, these results show that the language sophisticated forms of experience design, game- used by respondents in the open-ended questions thinking, and creativity in their gamification designs. conveyed an operational and tactical focus in their However, it would seem that these factors are currently recommendations, rather than strategic or systemic. beyond the capabilities of the available technology and This indicates that project managers had perhaps the common designs that dominate the enterprise confined their gamification projects within an gamification domain. This is supported by the recent operational paradigm that was within the scope of their findings of the development of a gamification capabilities or job description. Alternatively, the taxonomy [28] that to date, gamification has not projects have so far been smaller and tactical in nature produced new or novel design patterns. due to gamification only recently being introduced. Very often such projects were reported to be trials, The implications for DDGD is that motivational experiments or prototypes, rather than a full-scale re- affordances are one element of many in determining think or re-design of an enterprise system or process. the success of a gamified system, particularly if that system is an enterprise DSS or BI. However, caution Most respondents indicated that they would like to needs to be made here as design was considered to be a develop a more rigorous design process (59%), collaborative activity and it is the process and followed by more considered project management inclusiveness of the design process that is just as practices (21%), as well as selection of the right important as the specific design elements employed. technology for the job (20%). The key factors raised by Therefore, for DDGD to be successful, given its focus respondents in relation to improving the gamification on data and algorithms, attention needs to be made on design process revolved around the use of more how this can be integrated in a human-centred, thoughtful use of design practices and the use of collaborative design process. gameful elements. The elements included: improved ideation and prototyping, facilitating learning opportunities, using more meaningful design features, 4 Conclusions developing innovative mechanics (narrative, A better understanding of the experiences of experience, reputation), using less traditional organisations gives researchers and practitioners deeper mechanics (rewards, points, leaderboards), reduction in insight in how to design, develop and implement data- technological limitations, and careful selection of more driven gamified enterprise systems. This is particularly capable vendors. pertinent as design knowledge is partly informed by To gain a deeper perspective of respondents’ views practice [26] [27]. Enterprises are reporting positive on the optimal design process, below is a selection of results from gamification projects, but are also relevant quotes: claiming that there’s still room for improvement across many operational areas. In particular, there are  “I would have spent more time at the beginning effectiveness issues associated with technology and looking at more into game-thinking elements and vendor maturity, and a need to improve the capabilities fewer game mechanics. I think we would have of organisations in the design and implementation of created a more engaging program.” gamification projects. The implications for DDGD is 5 that predictive models of personalising the user [10] Thiebes, S., Lins, S., and Basten, D. 2014. experience with game design elements are only one Gamifying information systems-a synthesis of aspect of what would be considered a successful gamification mechanics and dynamics, in implementation of a (gamified) enterprise system. For Proceedings of the 22nd European Conference on the ongoing development of the DDGD domain, Information Systems, Munster, Germany. attention needs to also be given to developing a holistic [11] Thom, J., Millen, D., and DiMicco, J. 2012. approach to system development and implementation. Removing gamification from an enterprise SNS. In proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work pp.1067- References 1070, New York: ACM Press. [1] DeLone, W. H., and McLean, E. R. 1992. [12] Herzig, P., Strahringer, S., and Ameling, M. 2012. Information systems success: The quest for the Gamification of ERP systems-Exploring dependent variable. Information Systems gamification effects on user acceptance constructs. Research, 3(1), pp.60-95. In Multikonferenz Wirtschaftsinformatik pp.793- [2] Infinedo, P., Rapp, B. Infinedo, A., and Sundberg, 804. Braunschweig: GITO. K. 2010. Relationships Among ERP Post- [13] Herzig, P., Jugel, K., Momm, C., Ameling, M., & Implementation Success Constructs: An analysis at Schill, A. 2013. GaML-A modeling language for the organizational level. Computers in Human gamification. In Proceedings of the 2013 Behavior, 26, pp.1136-1148. IEEE/ACM 6th International Conference on Utility [3] Sedera, W., Rosemann, M., and Gable, G. 2002. and Cloud Computing (pp. 494-499). IEEE Measuring Process Modelling Success. In S. Computer Society. Wrycza, (Ed.), Proceedings 10th European [14] Chaudhuri, S., Dayal, U., & Narasayya, V. 2011. Conference on Information Systems, pp.331-341, An overview of business intelligence Gdansk, Poland. technology. Communications of the ACM, 54(8), [4] Van der Heijden, H. 2004. User acceptance of 88-98. hedonic information systems, MIS Quarterly, [15] Power, D. J., & Sharda, R. 2009. Decision support pp.695-704. systems. In Springer handbook of automation (pp. [5] Venable, J.R., 2009. Identifying and Addressing 1539-1548). Springer Berlin Heidelberg. Stakeholder Interests in Design Science Research: [16] Marache-Francisco, C., and Brangier, E. 2013. An Analysis Using Critical Systems Heuristics, Perception of gamification: Between graphical IFIP WG 8.2 International Conference, design and persuasive design. In A. Marcus (Ed.), CreativeSME 2009, Guimarães, Portugal Lecture Notes in Computer Science 8013, Design, [6] Venkatesh, V., Morris, M. G., Davis, G. B., and User Experience, and Usability: Health, Learning, Davis, F. D. 2003. User acceptance of information Playing, Cultural, and Cross-Cultural User technology: Toward a unified view. MIS Quarterly, Experience: Second International Conference pp.425-478. (DUXU 2013), 21–26 November, Las Vegas: HCI International, pp. 558–67. [7] Hamari, J. 2013. Transforming homo economicus into homo ludens: A field experiment on [17] Kuka, D., and Oswald, D. 2012. Visual rhetoric of gamification in a utilitarian peer-to-peer trading self-optimization systems. In 10th Congress of the service, Electronic Commerce Research and International Association of Visual Semiotics (1-6), Applications, 12(4), pp.236–245. Buenos Aires, Argentina. [8] Hamari, J. 2015. Do badges increase user activity? [18] Lee, T. Y., Dugan, C., Geyer, W., Ratchford, T., A field experiment on the effects of Rasmussen, J., Shami, N. S. and Lupushor, S. 2013. gamification. Computers in Human Behavior, Experiments on Motivational Feedback for http://dx.doi.org/10.1016/j.chb.2015.03.036 Crowdsourced Workers. In Seventh International AAAI Conference on Weblogs and Social Media [9] Hamari, J., Koivisto, J., and Sarsa, H. 2014. Does 2013. gamification work? A literature review of empirical studies on gamification. In proceedings of the 47th [19] Raftopoulos, M. 2016. How organisations play: Hawaii International Conference on System creating stakeholder value with enterprise Sciences, Hawaii, USA. gamification, RMIT Research Bank https://researchbank.rmit.edu.au/view/rmit:161749 6 [20] DeLone, W. H., and McLean, E. R. 1992. [33] Hevner, A., S. March, J. Park, and S. Ram, S. 2004. Information systems success: The quest for the Design science in information systems dependent variable. Information Systems research. MIS Quarterly, 28(1), pp.75-105. Research, 3(1), pp.60-95. [21] Kotter, J. P. 1996. Leading Change. USA: Harvard Business School Press. ISBN 978-0-87584-747-4. [22] Kotter, J. P., and Cohen, D. S. 2012. The Heart of Change: Real-Life Stories of How People Change Their Organisations. USA: Harvard Business School Press. [23] Christensen, C. M. 1997. The Innovators Dilemma: When new technologies causes great firms to fail. Boston, Massachusetts, USA: Harvard Business School Press. [24] Chesbrough, H., and Rosenbloom, R. S. 2002. The role of the business model in capturing value from innovation: Evidence from Xerox Corporation's technology spin-off companies. Industrial and corporate change, 11(3), pp.529-555. [25] Von Hippel, E., 2005. Democratising Innovation, MIT Press, London [26] Friedman, K. 2003. Theory construction in design research: criteria: approaches, and methods. Design studies, 24(6), pp.507-522. [27] Hevner, A., S. March, J. Park, and S. Ram, S. 2004. Design science in information systems research. MIS Quarterly, 28(1), pp.75-105. [28] Raftopoulos, M., Waltz, S. and Greuter, S. 2015. How Enterprises Play: Towards a Taxonomy for Enterprise Gamification: Conference Paper: Diversity of Play: Games-Cultures-Identities. DiGRA. [29] Gartner, 2012, Why projects fail? Hint – It’s not technical skills. Retrieved from http://blogs.gartner.com/mike- rollings/2013/03/28/why-projects-fail-hint-its-not- technical-skills/. Accessed 1 September 2017. [30] Meder, M., and Plumbaum, T., 2017. A Primer on Data-Driven Gamification Design, Competence Center for Information Retrieval & Machine Learning, DAI-Labor, Technische Universität Berlin, Germany [31] Meder, M. and Jain, B-J., 2014. The Gamification Design Problem. CoRR abs/1407.0843 (2014). http://arxiv.org/abs/1407.0843 [32] Raftopoulos, M. 2014. Towards gamification transparency: A conceptual framework for the development of responsible gamified enterprise systems. Journal of Gaming and Virtual Worlds, 6(2). 7