Intelligent Authoring of Gamified Intelligent Tutoring Systems Diego Dermeval Computing and Systems Department Federal University of Campina Grande diegodermeval@copin.ufcg.edu.br ABSTRACT Meanwhile, teachers are increasingly demanding to act as Intelligent Tutoring Systems can successfully complement active users of systems with such features. For instance, a and substitute other instructional models in many contexts. recent survey [14] with 41,805 K-12 teachers in USA reports However, it is very common to students to become bored that more than a half of them consider learning how to use or disengaged using ITS. The inclusion of gamification ca- educational technologies which distinguish instructions to pabilities (e.g., level, points and so on) in ITS design aims students (i.e., ITS) the most important item for their pro- to engage students and to drive desired learning behaviors. fessional development. Moreover, another survey [13] with Researchers have been noting that teachers are increasingly aspirants teachers in USA reports that they consider the ac- demanding to act as active users of systems with such fea- cess to educational technologies with support to customized tures. In this context, the main challenge of this project instructional plans as one of the main factors that will de- is contributing to the actively participation (i.e., design) of termine their future success as teachers. teachers in the use of gamified intelligent tutoring systems. In this context, the main challenge of this work is con- This challenge leads to the following research questions: (i) tributing to the actively participation of teachers in the use “how could we enable teachers to customize the construc- of intelligent tutoring systems that consider motivational as- tion of gamified ITS in a simple way and without requiring pects of the students using gamification. However, since technical capabilities from them?”; and (ii) “how could we teachers have different expectations and/or methodologies also provide good design principles in order to aid teachers as well as could use ITSs in several domains and different in the customization of gamified ITS?”. Thus, our aim is to educational levels, they should be able to customize them develop an intelligent authoring platform to enable teachers according to their preferences. Thus, by actively participa- for customizing gamified ITSs. In this way, we describe in tion we mean that teachers may be primary actors of gam- this text a set of specific objectives that must be completed ified ITSs, for example, by selecting which functionalities to achieve this general aim. they are interested to incorporate in ITSs, by defining which gamification behaviors they expect from their students, by choosing which pedagogical strategies they may consider or Keywords by creating and/or reusing content. Intelligent Tutoring Systems; Gamification; Intelligent Au- The design of ITSs is very complex. It should take into thoring account the four classic ITS models (domain, student, peda- gogical and interface models) [22] as well as should deal with several stakeholders, such as developers, authors, teachers, 1. INTRODUCTION students and so on. The inclusion of gamification features Empirical evidences suggest that Intelligent Tutoring Sys- in ITS design significantly increases the complexity of con- tems (ITSs) can successfully complement and substitute other structing these systems, since gamification elements may be instructional models in many situations [10]. They are con- combined to several ITS features. sistent with the most frequently implemented ITS features On the one hand, there is an increasing interest by teach- enabled by student modeling, namely high individualized ers to actively use (i.e., customize) systems with such fea- task selection, prompting and response feedback. tures, but on the other hand it is very complex to build In general, the traditional development of ITSs do not them. Thus, “how could we enable teachers to customize make efforts to engage and motivate students. On the other the construction of Gamified Intelligent Tutoring Systems hand, motivated, challenged and intrigued students tend to in a simple way and without requiring technical capabilities have better learning results [20]. In this way, relying on the- from them?”. However, only enabling teachers to customize ories and models of motivation and human behavior, many these systems without providing some kind of support for works have been using persuasive technologies (e.g., gamifi- their decision-making is not enough, because it is likely that cation) in connection with education [7]. Gamification can they would build ineffective tutors both from performance be defined as “the use of game elements and game design and motivational aspects. In this way, “how could we also techniques in non-game contexts” [21]. It has been used in provide good design principles in order to aid teachers in the the context of web-based education by adding game elements customization of gamified ITS?” (e.g., levels, points, badges, and so on) to learning contexts aiming to engage students and to drive desired learning be- haviors [8]. 2. MAIN CONTRIBUTIONS to other mechanisms for automatic analysis of features mod- Aiming to contribute to the challenges previously mentioned, els, description logic (DL) based methods (e.g., ontologies) we present in this section the objectives of this PhD project. promise to provide improved automated inconsistency de- However, before presenting our objectives, we will briefly tection, reasoning efficiency, scalability and expressivity [3]. discuss some theoretical concepts and important technolo- Figure 1 presents an overview of the intelligent authoring gies that are used in this work. process for building gamified ITS from teacher’s perspective. Researchers have been investigating the use of authoring In order to achieve our general objective, we intend to reach tools for building intelligent tutoring systems since the be- the following specific objectives: ginning of ITS research [12, 22, 17]. The aim of using such tools includes, for example: (i) reducing the effort for de- (a) Investigate and select proper ITS models from the lit- signing ITSs; (ii) decreasing the required level of skill to erature to be represented in ontologies; construct ITSs; (iii) aiding ITS authors/designers to orga- (b) Investigate and select proper models of motivation and nize domain or pedagogical knowledge of the system; (iv) human behaviors from the literature to be represented supporting good ITS design principles; (v) enabling quickly in a gamification ontology; prototyping for ITS design; and so on. However, the development of authoring tools to aid the (c) Define a set of gamification behavior good practices construction of ITS with game elements may be considered from empirical research papers in the context of ed- an open problem/challenge in computers and education re- ucation online to be incorporated in the gamification search. So far, to the best of our knowledge, we could not ontology; find any work in the literature that propose to use authoring tools for enabling teachers to build gamified ITS. Further- (d) Specify an integrated ontology relating gamification and more, as mentioned by Sottilare et al [17], the opportunity of ITS ontologies; integrating game elements and intelligent tutoring systems (e) Design and implement the authoring platform taking by the use of authoring tools may enable an expected in- into account the gamification and ITS knowledge rep- creasing in students motivation and engagement along with resented in ontologies. This platform must consider effective instructional techniques provided by ITSs. authoring of content as well as authoring for customiz- In this way, to address the questions raised in the end of ing the design of gamified ITS by teachers; Section 1, and taking into account the potential benefits of using authoring tools in the context of Gamified ITSs, our (f ) Define an ontology-based feature modeling approach to aim in this PhD project is to develop an intelligent authoring represent the configuration knowledge of the authoring platform to enable teachers for building Gamified ITSs. platform which can be used to instantiate a specific As part of the “intelligent” aspect of our authoring plat- gamified ITS. form, we represent the knowledge about gamification theo- ries and ITS models as well as good gamification behavior 3. RELATED WORK practices in a way that it can be used to aid the authoring process. For instance, some pre-defined gamification behav- The literature review about the use of authoring tools to iors (e.g., performance, participation and so on) with pre- build gamified ITS was conducted in three different ways: defined game elements choices may support a better and (i) analysis of the papers that propose authoring tools and more simple decision from the teacher. In order to represent that are included in a recent book [17] that reviews the use such knowledge in a way that could be processable by the of authoring tools for building ITS; (ii) searching in google authoring platform, we are relying on the concept of ontolo- scholar for papers in the topic; and (iii) conduction of a sys- gies. Ontologies can be logically reasoned and shared within tematic review of the literature in topic, which is currently a specific domain. Thus, ontologies are a standard form for in the writing stage. representing the concepts within a domain, as well as the After performing these three steps, we have found seven relationships between those concepts in a way that allows authoring tools that can be considered related to our plat- automated reasoning. form: ASSISTments [15], ASPIRE [11], CTAT [1], SimStu- Additionally, taking into account the high variability of dent [9], xPsT [5], GIFT [18] and Ataide’s tool [2]. Al- gamified ITSs (i.e., there are several technological, ITS and though, these works present important contributions for au- gamification features that could be combined in different thoring ITS, none of them address the challenge of author- tutors) as well as the need for representing the knowledge ing gamified ITS. Moreover, Gonzalez et. al [6] propose about configuration choices of a teacher, we also intend to a conceptual architecture for building gamified ITS. How- create a configuration model that could be automatically ever, this architecture does not allow intend to allow teach- reasoned by a gamification ITS platform to deliver a specific ers for customizing gamified ITSs. This PhD project intends system according to author’s choices. In this context, we are to develop an authoring platform in order to enable teach- relying on the concept of feature modeling to manage the ers without any technical ability (e.g., programming skills) variability of gamified ITS. to customize gamified ITSs. Our platform makes use of a Moreover, enabling the automatic analysis of feature mod- knowledge layer which includes gamification and ITS theo- els and hence providing reconfiguration of a gamified ITS is ries as well as good design principles to support the author- also required. Achieving these characteristics could allow ing process. for example, to monitor learner’s motivational levels at the time they are interacting with the ITS and to reconfigure the system with a different gamification behavior that could improve the engagement of students. Thus, in comparison Figure 1: Overview of the Intelligent Authoring for Building Gamified Intelligent Tutoring Systems 4. ONGOING WORK AND FURTHER RE- in the previous objective to represent these practices in the SEARCH ontology. To achieve the objective (d) we have connected gamifi- In this section, we explain the specific objectives that we cation concepts to ITS concepts in an integrated ontology have already performed and which activities we still need to (named Gamified Tutoring Ontology - GaTO) based on the execute. In addition, we will further explain how we intend gamification ontology and on the ITS ontologies specified in to validate our proposal. the execution of objectives (a), (b) and (c). To perform the objective (a) we have first studied several In order to achieve the objective (e) we have conducted the ITS theories and models (i.e., domain, student and peda- requirements engineering and architectural design phases for gogical) and then we have adapted ITS ontologies available the authoring platform. We are currently implementing the in the literature. specified architecture (75% of the implementation is already In order to complete the objective (b), mentioned in Sec- developed). It is noteworthy that some non-functional re- tion 2, we have first studied several theories related to gam- quirements are crucial to the design of the authoring plat- ification (e.g., Fogg’s behavior model, Self-Determination form, for instance, usability. As we are designing an author- Theory, Reinforcement theory, flow theory and so on) to ing platform for teachers, this requirement drives several understand the gamification domain. Then, we specified a decision-makings, since it is of utmost importance to the domain ontology according to the Self-Determination The- effectiveness of the authoring platform. ory (SDT) – since one of the main definitions [21] of gami- The output of the authoring platform is a configuration fication is supported by such theory – to represent the core model that represents which features of a gamified ITS should concepts about gamification domain. Several gamification be incorporated in the tutor authored by teachers. In this concepts along with their relationships are represented in way, to achieve objective (f) we defined and validated an the ontology, for instance, Game Design Element, Game ontology-based feature modeling (OntoSPL) approach [4, Components, Game Mechanics, Game Dynamics, Motiva- 19] that is used to constrain the design space for features tion, Player and so on. selection by authors. This ontology may be further used by Afterwards, in order to achieve objective (c), we have first a gamified ITS platform to deliver a configured tutor accord- searched for empirical studies that report positive effects ing to the configuration represented in the ontology. on the use of gamification in education from published sys- Finally, after concluding the implementation of the au- tematic literature reviews on the topic (e.g., [16] and [7]. thoring platform we will further validate our platform in Next, we grouped these effects by using a gamification de- three different ways. First, we intend to evaluate the us- sign framework (i.e., 6D framework [21]) according to com- ability (i.e, based on Nielsen’s heuristics) for customizing mon target behaviors for using gamification. Based on these gamified ITS from teachers perspective in academic settings. groups, we defined six behaviors, which we call good prac- Second, we intend to analyse the authoring platform inte- tices, i.e., performance, participation, enjoyment, competi- grated with a gamified ITS platform aiming to characterize tion, exploration and effectiveness. They are called good the effort for creating gamified ITSs with respect to the time practices because they establish a gamification design that of creation and ease of use from a teacher viewpoint in in- has presented positive empirical results in the literature. Fi- dustry settings (i.e., MeuTutor). Finally, we also intend to nally, we relied on the gamification domain ontology defined analyse gamified ITSs designed by teachers using our au- thoring platform to characterize them with respect to moti- [11] A. Mitrovic, B. Martin, P. Suraweera, K. Zakharov, vation and learning performance from a student viewpoint N. Milik, J. Holland, and N. McGuigan. Aspire: an in academic settings. authoring system and deployment environment for constraint-based tutors. International Journal of 5. ACKNOWLEDGMENTS Artificial Intelligence in Education, 19(2):155–188, This PhD student is supervised by Professor Ig Ibert Bitten- 2009. court (Federal University of Alagoas). This work has been [12] T. Murray, S. Blessing, and S. Ainsworth. Authoring supported by the Brazilian institutions: Conselho Nacional tools for advanced technology learning environments: de Desenvolvimento Cientı́fico e Tecnológico (CNPq), Coor- Toward cost-effective adaptive, interactive and denação de Aperfeiçoamento de Pessoal de Nı́vel Superior intelligent educational software. Springer Science & (CAPES). Business Media, 2003. [13] ProjectTomorrow. Learning in the 21st century: 6. REFERENCES Digital experiences and expectations of tomorrow’s teachers. Available at [1] V. Aleven, J. Sewall, O. Popescu, M. van Velsen, http://www.tomorrow.org/speakup/tomorrows S. Demi, and B. Leber. Reflecting on twelve years of its authoring tools research with ctat. Design teachers report2013.html, 2013. Access on March, Recommendations for Intelligent Tutoring Systems: 2016. Authoring Tools and Expert Modeling Techniques, [14] ProjectTomorrow. Speak up 2014 research project page 263, 2015. findings - the results of the authentic, unfiltered views of 41,805 k-12 teachers nationwide. Available at [2] W. A. Ataide, P. H. Brito, A. P. Silva, E. Costa, I. I. Bittencourt, and T. Tenório. A semantic tool to assist http://www.tomorrow.org/speakup/pdfs/SU2014 authors in the instantiation of software product lines TeacherTop10.pdf, 2014. Access on March, 2016. for intelligent tutoring systems context. IEEE [15] L. Razzaq, J. Patvarczki, S. F. Almeida, M. Vartak, Technology and Engineering Education (ITEE), M. Feng, N. T. Heffernan, and K. R. Koedinger. The 7(3):52–61, 2012. assistment builder: Supporting the life cycle of [3] D. Benavides, A. Felfernig, J. Galindo, and tutoring system content creation. Learning F. Reinfrank. Automated analysis in feature modelling Technologies, IEEE Transactions on, 2(2):157–166, and product configuration. In J. Favaro and 2009. M. Morisio, editors, Safe and Secure Software Reuse, [16] K. Seaborn and D. I. Fels. Gamification in theory and volume 7925 of Lecture Notes in Computer Science, action: A survey. International Journal of pages 160–175. Springer Berlin Heidelberg, 2013. Human-Computer Studies, 74:14–31, 2015. [4] D. Dermeval, T. Tenório, I. I. Bittencourt, A. Silva, [17] R. Sottilare, A. Graesser, X. Hu, and K. Brawner. S. Isotani, and M. Ribeiro. Ontology-based feature Design Recommendations for Intelligent Tutoring modeling: An empirical study in changing scenarios. Systems: Authoring Tools and Expert Modeling Expert Systems with Applications, 42(11):4950 – 4964, Techniques. Robert Sottilare, 2015. 2015. [18] R. A. Sottilare, K. W. Brawner, B. S. Goldberg, and [5] S. B. Gilbert, S. B. Blessing, and L. Blankenship. The H. K. Holden. The generalized intelligent framework accidental tutor: overlaying an intelligent tutor on an for tutoring (gift), 2012. existing user interface. In CHI’09 Extended Abstracts [19] T. Tenório, D. Dermeval, and I. I. Bittencourt. On the on Human Factors in Computing Systems, pages use of ontology for dynamic reconfiguring software 4603–4608. ACM, 2009. product line products. In IARIA, editor, Proceedings [6] C. González, A. Mora, and P. Toledo. Gamification in of the Ninth International Conference on Software intelligent tutoring systems. In Proceedings of the Engineering Advances, pages 545–550, 2014. Second International Conference on Technological [20] K. Vanlehn, W. Burleson, M. C. Echeagaray, Ecosystems for Enhancing Multiculturality, TEEM R. Chirstopherson, R, J. Sanchez, J. Hastings, Y. H. ’14, pages 221–225, New York, NY, USA, 2014. ACM. Pontet, and L. Zhang. The affective meta-tutoring [7] J. Hamari, J. Koivisto, and H. Sarsa. Does project: How to motivate students to use effective gamification work?–a literature review of empirical meta-cognitive strategies. In 19th International studies on gamification. In System Sciences (HICSS), Conference on Computers in Education, Chiang Mai, 2014 47th Hawaii International Conference on, pages Thailand, 2011. 3025–3034. IEEE, 2014. [21] K. Werbach and D. Hunter. For the win: How game [8] K. M. Kapp. The gamification of learning and thinking can revolutionize your business. Wharton instruction: game-based methods and strategies for Digital Press, 2012. training and education. John Wiley & Sons, 2012. [22] B. P. Woolf. Building intelligent interactive tutors: [9] N. Li, N. Matsuda, W. W. Cohen, and K. R. Student-centered strategies for revolutionizing Koedinger. Integrating representation learning and e-learning. Morgan Kaufmann, 2010. skill learning in a human-like intelligent agent. Artificial Intelligence, 219:67–91, 2015. [10] W. Ma, O. O. Adesope, J. C. Nesbit, and Q. Liu. Intelligent tutoring systems and learning outcomes: A meta-analysis. Journal of Educational Psychology, pages 901–918, 2014.