Gami cation gradually gains more attention. However, gami cation and its successful application is still unclear. There is a lack of insights and theory on the relationships between game design elements, motivation, domain context and user behavior. We want to discover the potentials of data-driven optimization of gami cation design, e.g. by the application of machine learning techniques on user interaction data. Therefore, we propose data-driven gami cation design (DDGD) and conducted a questionnaire with 17 gami cation experts. Our results show that respondents regard DDGD as a promising method to improve gami cation design and lead to a general de nition for DDGD.
Gami cation has been a hot topic for some time now and gained attention of academics and practitioners alike. Previous work has focused mainly on models from psychology, user tests and personal experiences [Yee16, Yee07, HT14a, KH14, Dix11]. In a more and more data-driven world, new possibilities emerge to replace previously manually created models with machine-made ones. Recent research indicates that instead of using only prede ning player types to select or assign game design elements a data-driven gami cation design approach (DDGD) [HT14b, HHS14, Det15, JXKV16, SBSH16, OND17], which would allow us to learn the assignments on collected real user behavioral data, could improve gami cation design. One advantage would be that the selection and implementation of game design elements or motivational a ordances could be adapted in real-time. Finally, based on live interactions and goals, a data-driven gami cation system would automatically select the best gamication design approach. In this paper, we rst introduce the data-driven gami cation design (DDGD) approach. We then present a cunducuted questionnaire which asked leading experts in the eld of Gami cation about their opinion on DDGD, what impact they expect DDGD to have and what obstacles they see to successfully implement DDGD. As a result of the questionnaire we propose a general de nition for datadriven gami cation design. The main contributions of this paper are:
A questionnaire collecting opinions from gami cation experts on DDGD.
The rst comprehensive de nition of DDGD as a new emerging topic within the eld of Gami cation.
The paper is structured as follows. In Section 2, we summarize the currently existing literature on DDGD. We then outline the questionnaire and highlight the process of collecting answers in Section 3. In Section 4 we analyze and discuss the results of the questionnaire and deduce a de nition for DDGD. In Section 5 we summarize our ndings and give recommendations for future work.
To our knowledge, this paper is the rst work introducing DDGD as a new research eld. The presented outcomes lay the foundation for a future successful adaption of DDGD. 2
Several studies on user or user type speci c gami cation design [HT14b, HHS14, Det15, JXKV16, SBSH16, OND17] encourage experiments with empirical data encapsulating user interactions. The main reasons for this are lacking detailed knowledge about the complexity and interdependencies of user types, motivation types, game design elements, user interface elements and actual goals of gami cation. Exactly this knowledge is necessary to arrive at reliable and well-founded statements about successful gami cation design. The question \Does gami cation work? [...]" [HKS14] and how to make it work has remained unclear. One method to approach this is to consider well-known player typologies [HT14b] but this approach is challenging for questionnaires and interviews. The 3rd International Workshop on Gami cation for Information Retrieval (GamifIR 2016) [MHKK16] assisted by Sebastian Deterding's keynote speech \Desperately Seeking Theory: Gami cation, Theory, and the Promise of a Data/AI-Driven New Science of Design"1 [Det16] has concluded that we should take the opportunities which AI and data-driven techniques provide in order to gain deeper insights on successful gami cation design.
As a result, more and more researchers have proposed data-driven approaches to gami cation design. In 2013, Paharia [Pah13] suggested to use big data and gami cation for customer and employee gami cation. In 2014, Meder and Jain [MJ14] de ned the gami cation design problem and considered it as a \[...] special case of a recommendation problem for which matrix factorization constitutes a state-of-the-art solution". In 2016, Meder et al. [MPA16] suggested a two phase procedure of gami cation experiments to collect user interaction data largely avoiding negative in uences such as bad usability and bugs. They further planned to apply machine learning methods to detect and learn typical interaction patterns. In 2017, Tondello et al. [TON17] likewise to Meder and Jain [MJ14] also suggested recommender systems as a solution for more personalized gami cation. For all those studies an empirical evaluation of user speci c gami cation design is missing.
Heilbrunn et al. [HHS14, HHS17] \[...] de ne gamication analytics as the data-driven processes of moni1slideshare.net: https://goo.gl/YZg65N toring and adapting gami cation designs." They evaluated seven analytics tools towards their ability to support those gami cation analytics. Their ndings show that no analytics tool exists which ful lls their gamication analytics requirements. 3
For further insight and to learn how the research community thinks about data-driven gami cation design, we conducted a questionnaire with gami cation experts. As target respondents we picked authors of recently published research papers on gami cation plus well known gami cation experts. The survey was sent via e-mail to the selected gami cation experts (N=65), which contained a link to a website where, after a brief introduction, all questions were displayed at once. Since we assumed that the concept of Data-Driven Gami cation Design (DDGD) is rather unknown and not well-de ned so far, we explained our understanding of DDGD in the introduction of the questionnaire as follows: \Gami cation is a hot topic for some time now and gained attention of academics and practitioners. The work carried out so far relies primarily on models from psychology, user tests and personal experiences. We argue that rather relying only on psychology theories of motivation, machine learning approaches for a data-driven gami cation design approach should be used. Instead of prede ning player types and matching gamication elements, DDGD allows to learn this based on collected user behavioral data. The selection and implementation of game design elements or motivational a ordances can also be adapted in real-time. Based on live interactions and goals, a data-driven gami cation system can select the best approach automatically." The questionnaire contained a mandatory part along with an optional part. The mandatory part contained eight statements regarding which respondents had to specify their level of agreement or disagreement on a ve-level Likert item from \Strongly disagree" to \Strongly agree." The statements were as follows:
2. DDGD will allow for better gami cation design. 3. DDGD will allow replacement of the current gami cation approaches. 4. DDGD will allow for successful real-time adaption of applied game design elements. (S1)DDGDisaknownconceptforme. (S2)DDGDwillallowforbetter gamificationdesign. (S3)DDGDwillallowreplacementof thecurrentgamificationapproaches. (S4)DDGDwillallowforsuccessful real-timeadaptionofappliedgame designelements. (S5)DDGDrendersplayertypemodels redundantbylearningplayertypes throughbehavioraldata. (S6)ThereisplentyofdataforDDGD available. (S7)Traditionalgamificationdesign techniquesandtoolsaresufficientfor successfulapplicationofgamification. (S8)Traditionalgamificationandhow toapplyitsuccessfully,isstillunclear. 5. DDGD renders player type models redundant by learning player types through behavioral data. 6. There is plenty of data for DDGD available. 7. Traditional gami cation design techniques and tools are su cient for successful application of gami cation. 8. Traditional gami cation and how to apply it successfully, is still unclear.
In the optional second part, we asked for further information on how the respondents apply or plan to apply gami cation, if they know any corresponding datasets or if they have other comments or suggestions. 9. Techniques or tools I use, plan to use or promising, for the application of gami cation: nd
Game or Gami cation Design Frameworks (like MDA, Gami cation Model Canvas, Octalysis, Six Steps To Gami cation, etc.) Web/App Analytics User Behavior Statistics Key Performance Indicators Recommender Systems Machine Learning Arti cial Intelligence Other (please specify) 10. Do you know one or more gami cation (user interaction) datasets? Please provide an URL if possible. 11. What kind of data in your opinion would be useful for gami cation design? 12. Do you have any suggestions for data-driven gami cation design approaches? 13. Comments and other suggestions: 14. Your Name 15. Agreement for publication.
I hereby agree that my answers can be used for publication. 4
In the following, we will show how respondents answered the questionnaire. From a total of 65 invitations, to academic authors of recently published gami cation papers, we have received 17 replies. Ten of them provided their full name. The responses of the participants on the mandatory part of the questionnaire are depicted in Figure 1. The ratings for the rst statement (S1) show that slightly more than half (52:94%) of the respondents has an idea about datadriven gami cation design (DDGD) whereas the other half seems to be uncertain. At the same time our respondents are quite optimistic about the bene ts of
DDGD as merely one respondent disagrees on S2. Furthermore, respondents are rather optimistic regarding the opportunities (S2, S4, S5) of DDGD. Whereby the real-time adaption (S4) seems to be more viable to the respondents than rendering player types models redundant (S5). Thus we think, respondents are somehow skeptical about the possibility to compute something similar or even better than recent player type models from e.g. user interaction data. But the comments show that there is much hope that it will work (see below). While they are optimistic about the chances of DDGD, respondents disagree on a possible replacement of current gami cation approaches by DDGD (S3). Simultaneously, respondents do not think that traditional tools and techniques are su cient (S7) which is therefore consistent with the general optimism or even hope on DDGD (S2). In contrast, and thus a bit contradictory, the strongest agreement (64:7%)has been given to the last statement which claims that it is still unclear how to apply gami cation. The prospect of low availability of data (S7) may be one reason for this. This could have dampened the optimism for DDGD as a replacement of current gami cation approaches.
For the second and optional part of the questionnaire we have received 13 responses on the our questions on techniques (Q9). As depicted in Figure 2 the most common techniques for gami cation design seems to be player and motivation types and statistics on user behavior closely followed by game or gami cation design frameworks and machine learning. Unfortunately, as expected, no participant could provide a link to a dataset (Q10). Seven respondents submitted suggestion on the kind of data we need for DDGD (Q11). Almost all suggest user behavior data or data of interactions with \gami cation features." Also data on time spent (especially \time well-spent"), goals, user intentions and performances, all situational data as well as the surroundings and the context of the applied gamication have been mentioned. Further suggestions on DDGD (Q12) are like \Collect as much data as you can [...]" and that the users' interests need a strong focus. One respondent argued that \[...] it's important to agree on a success indicator that we can take as guidance and investigate how di erent data and behaviors relate to it. Without this e ective DDGD will be much more di cult." In Q13 general comments on DDGD were given. Instead of the replacement of traditional gami cation design could \[...] one approach can complement the weaknesses of the others." They further state that \[...] we might need a blend of a priori theoretical knowledge [...]" and with \[...] enough data [...]" we might be able \[...] to quickly categorize a new player [...]." Another respondent believes that psychology models are bene cial for gami cation design but it is hard to successfully integrate them. The respondent is also against player type models and hopes that DDGD will be an alternative: \So while I don't think DDGD will make Player Type models redundant | I hope it will." De ning Data-Driven Gami cation Design Taking into account previous works and the results of our questionnaire, we propose the following general de nition: Data-Driven Gami cation Design (DDGD) is the automation of the gami cation design process using data mining approaches to apply game design elements tailored to each individual that maximizes their expected contribution to achieve well-de ned objectives. 5
In this paper, we have studied the opportunities of data-driven gami cation design. Therefore, we have examined related work and conducted a questionnaire with 17 gami cation experts to collect their expectations and suggestion for DDGD. Our ndings show that, although they are slightly skeptical towards DDGD (S3, S5, S6), there is a strong demand for further solutions because it is still unclear how to apply gami cation successfully (S8). Beyond that, the respondents were very optimistic that DDGD allows better gami cation design (S2) but their optimism seems to be dampened by worries on available data. In the second part of the questionnaire, we collected suggestions and comments from which we derive the following recommendations for future work. We need user behavior data or data of interactions with game design elements collected in real world studies whereby additional data like goals, user interests, time spent and general context of the applied gami cation design should be considered. It would be even better if those data would be publicly available. Furthermore, researcher should nd an agreement on a \success indicator" to make ndings comparable and improvements measurable. Altogether, we deduced the following definition: Data-Driven Gami cation Design (DDGD) is the automation of the gami cation design process using data mining approaches to apply game design elements tailored to each individual that maximizes their expected contribution to achieve well-de ned objectives. [Det15]
sic skill atoms: A method for gameful design. Human-Computer Interaction, 30(34):294{335, 2015.
theory: Gami cation, design, data, machine learning, and the promise of double loop learning systems. In GamifIR 2016 workshop co-located with SIGIR 2016, page 1, 2016.
tion. In In Workshop on Gami cation at CHI2011, pages 12{15, 2011.
IEEE/ACM 7th International Conference on Utility and Cloud Computing, UCC 2014, pages 603{608, 2014.
Analytics|Methods and Tools for Monitoring and Adapting Gami cation Designs, pages 31{47. Springer International Publishing, Cham, 2017.
Sarsa. Does gami cation work?{a literature review of empirical studies on gamication. System Sciences (HICSS), 2014 47th Hawaii International Conference on, pages 3025{3034, 2014.
dro Bozzon, Robert-Jan Sips, and GeertJan Houben. Work and play: An experiment in enterprise gami cation. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & [TON17] [Yee07] [Yee16]