Using Gamification for Teaching UML in Information System Design Course Mantas Jurgelaitis Lina Čeponienė Vaidotas Drungilas Deparment of Information Systems, Deparment of Information Systems, Deparment of Information Systems, Kaunas University of Technology Kaunas University of Technology Kaunas University of Technology Informatics faculty Informatics faculty Informatics faculty Kaunas, Lithuania Kaunas, Lithuania Kaunas, Lithuania mantas.jurgelaitis@ktu.lt lina.ceponiene@ktu.lt vaidotas.drungilas@ktu.lt Abstract—Nowadays the practice of introducing gamification [10]. Some courses try to automate the process of teaching and into areas such as education became quite popular. In this paper, evaluating students, although for now this automation covers we are examining the effects of applying gamification into the just two out of fourteen UML diagrams [7]. Other courses use process of teaching the Information System design using Unified UML class and sequence diagrams as tools in improving student Modelling Language. In gamified UML course, we focused on comprehension of software code [6] [7]. introducing such game elements as points, levels, badges, leaderboard, and bonuses into the teaching process. Students’ In Kaunas University of Technology Information Systems activities during the course were logged and later analyzed. During Design course fourth year undergraduate students are analysis a positive influence on the student grades was observed. extensively taught of using UML in object-oriented Moreover, a positive effect was noticed on the student intrinsic development of Information Systems. The course encompasses motivation. a broad range of UML diagrams and their uses in requirements engineering, design and implementation of Information Keywords— gamification; education; UML; RUP; modelling. Systems. I. INTRODUCTION Unified Modelling Language provides only the notation for Gamification is described as a practice of using game describing visual models but does not define the process and elements in a context, which has no direct association with context of using these models. Many software engineering games [1]. Recently the practice of introducing gamification into processes exist, but the one used in this course is Rational areas such as education, sales, banking, customer loyalty Unified Process (RUP) [11]. RUP is use case driven, iterative programmes etc. became quite popular [2]. Various degrees of development framework, which helps to mitigate risk, defines success in applying gamification in education are observed as easily visible progress, provides early feedback and helps gamification provides common structure for motivating and managing software projects of varied complexity [12]. engaging students into the learning process [3]. Student’s motivation and engagement play a huge role in the In this paper we are analyzing the process of applying teaching process, many of the students tend to lose their gamification into the process of teaching the information system motivation and thus the quality of teaching diminishes. To design using Unified Modelling Language (UML) [4]. combat this problem and to increase student engagement into the learning process an idea of gamifying the Information System UML provides a standard for visualizing and specifying the Design course was proposed. When applied correctly, design of software systems and is commonly used during the gamification tends to increase motivation, helps to engage software engineering process [5] [6] [7]. UML notation is students [13]. Unfortunately, no gamified courses or tools for introduced and taught in various higher education institutions [8] teaching UML were found. [9] [10]. At the start of 2017 autumn semester student were invited to UML is a graphical notation which enables modelling participate in the gamified course. Students’ activities during the software engineering concepts represented in structure and course were logged and later analyzed. Results show that behavior diagrams. These diagrams are taught during the course gamification had a positive effect on student grades. In addition for undergraduate students in Kaunas University of Technology to the logged data, students were surveyed measuring their Informatics faculty’s curated study programmes. Students are intrinsic motivation. Surveys’ results indicate that students’ tasked in preparing specification and documentation for intrinsic motivation increased, provided they used the gamified software projects. system regularly. Other higher education institutions also often use UML in The rest of the paper is organized as follows. The second their software engineering courses’ curriculum [5] [6] [7] [9] section analyzes related work in the area of applying gamification in education. The third section presents the proposed methodology for gamifying Information System Copyright held by the author(s). Design course, course structure and its implementation in 88 Moodle learning management platform. The fourth section is gathering for raising levels [14] [15] [17]. Thus, environment is dedicated to analyzing the results of application of the gamified required, which gives instant feedback and tangible results for course in practice. The last section overviews the paper, outlines students’ activities. In various degree of success, leaderboards the major outcomes and provides a glimpse into future research were used [14] [15] [16] [17]. This helped to facilitate ideas and upcoming planned tasks. competition among students and enable comparing personal results that of their peers. Additionally some studies used virtual II. RELATED WORK currency for trading between users (one for forecasting changes Gamification in education helps to enhance courses in order in market and maximizing the profits of transaction [16], the to increase user engagement, productivity and motivation [1] [2] other for unlocking further tasks and activities [17]). [3]. Gamifying the educational material improves Authors of [14] [16] used questionnaires to measure the comprehension of difficult topics and helps to better understand effects of gamification as well as analyzed system data, for the area such as software engineering [14]. There exists a number of confirming the proposed hypotheses. The questionnaires were case studies on applying gamification in education that had conducted twice, before the start and after the completion of the direct ties with software engineering, but not with applying gamified courses [14] [16]. In all cases, the most common used UML. The case studies gamified course themes ranged from C method of data gathering was the platform itself [14] [16] [17]. programming [14], Service Oriented Architecture [15] to Other authors [15] compared the results of two different group national budget forecasting [16]. sets for determining the effects of gamification. The case study presented on Gamification for Engaging After reviewing the relevant case studies, it was determined Computer Science Students in Learning Activities [14] tried to that the most common and effective gamification elements are measure the effectiveness of gamified C programming language points, levels and badges and these elements should be applied teaching platform Q-Learning-G and student engagement into in the proposed gamified Information System Design Course. the process. The authors of the study analyzed what kind of The gamified course should also include a leaderboard. During learning activities are the most attractive to students. the gamified course, at least two surveys should be executed. An The research by Buckley and Doyle [16] focused on finding Intrinsic Motivation Inventory scale [18] was found for out whether the gamification has any positive effect on student measuring student intrinsic motivation and is suitable for motivation. They used the gamification for introducing the preparing the questionnaires. Additionally, students’ gamified national tax system. The results of this study show that Information System Design course usage should be logged, as it gamification has the bigger effect on students that are already can provide a different perspective for finding insights on the inherently motivated. gamified course. These insights could later help to improve the course. It is also important to outline a clear measurable goal for The research on Gamification in Higher Education [15] tried easier confirmation. to determine whether gamification has any positive results for student development. The authors tried to discover the most III. PROPOSED METHODOLOGY FOR GAMIFYING THE effective gamification methods or elements. The case study had INFORMATION SYSTEM DESIGN COURSE a sample size of 62 graduate students in four different groups. The goal of our research was to increase student motivation Two groups were taught in a traditional course, and other two and engagement into the learning process in Information System were taught using gamification elements like points, badges and Design course. A relevant way to attain the set goal is to gamify leaderboards. the course by implementing some game elements into the course A case study on The Gamification Model for E-Learning curriculum and teaching process. Participants Engagement [17] performed by Kaunas University In gamified UML course, we focused on introducing such of Technology Informatics faculty developed an online system game elements as points, levels and badges into the teaching for teaching programming. The online system was intended for process. In order to encourage the competition among students, secondary school students. The system was developed on the a leaderboard was introduced. However, we decided not to focus proposed gamification model and with a goal of confirming the on grade bonuses for rewarding students, and instead to use the model’s validity for usage in an educational context. complete example UML models for the upcoming laboratory Although no gamified courses for teaching UML were practical works as bonuses. found, it is clear that the principles and methods used in the During the analysis of the teaching material of existing aforementioned case studies could successfully be applied to the course, it became clear that some restructuring of the material gamification of Information System Design course. itself was required in order to facilitate the gamifying process. A In order to test the gamification effect on students new course structure was proposed to include levels and points, researchers tend to formulate hypotheses [14] [16] or outline which would help to guide students through the learning process. goals [14] [17]. Other authors formulate questions or problems We based the structure of the course on the Rational Unified [15]. Our research would also benefit from outlining a goal, Process disciplines and diagrams used in this process. Rational which would determine whether motivation and engagement of Unified Process disciplines like business modeling, students increases with the implementation of gamification. requirements, requirement analysis and design, implementation and deployment provide developers a clear platform on which to Reviewed case studies mostly consisted of applying such build their project [12]. game elements as badges [14] [15] [17], levels and point 89 Levels and points would not only provide structure, but also databases. Other disciplines such as implementation, test and engage students, by giving them an instantaneous feedback on deployment are only partly covered the curriculum scope by the result of the tasks. Levels and points serve two purposes – several diagrams, as these disciplines deal more with actual guiding student behavior and giving student feedback at any programming, building and realization of the application and is point in time, signaling the students’ progress. At any point not the focus of the course curriculum. when the student completes a task, he would be rewarded with points. Achieving the base level of completion additionally B. The proposed structure of the gamified course would reward a student as well. Based on the course curriculum, ten levels were introduced into a course. Five for teaching the basics of UML diagrams’ With levels and points, badges were also introduced into the syntax and five for teaching the semantics, and their usage in course. Students would receive a badge at any point when they RUP. The course progression was locked behind the levels and would level up in the course. structured in such a way that the student would not be Lastly, we decided that students should also be able to overwhelmed with vast amount of information from the get-go. receive some useful rewards, like example UML models. Such Student at the start only had access to a few resources and only items would be used for increasing student motivation as after achieving some levels the course would open up. external motivators. Syntax levels in the gamified course were divided by A. The contents of Information System Design course diagrams and for each diagram a task was designed. Use case and activity diagrams were described in the first syntax level, in During the base course curriculum, students were taught of the second syntax level class and state machine diagram were eight UML diagrams out of total fourteen, and their various introduced. Third syntax level consisted only of robustness application in software engineering process based on Rational diagram. Fourth syntax level introduced class diagram elements, Unified Process (RUP). previously not explored in second level and sequence diagram. The set of UML diagrams used in this course includes class And lastly fifth syntax level consisted of component and diagram, use case diagram, state machine diagram, activity deployment diagrams. A total of 97 questions were created to diagram, package diagram, robustness diagram (specific for test student knowledge on the UML diagram syntax. RUP), sequence diagram, component diagram and deployment Other five levels for teaching the semantics of UML diagram. diagrams were based on the Rational Unified Process Class diagram is used to describe domain entities, their engineering disciplines. Business modelling for the first structure and relations. State machine is used to represent entity semantics level, requirements for the second, analysis and lifecycle’s states and transitions between them. Use case design for the third, implementation for the fourth and lastly fifth diagram is used to represent system functional requirements, for deployment. The course did not include test discipline as it system users as actors and their relations with designed systems falls outside the course curriculum scope. use cases. Activity diagram is used to specify use case scenarios, Level order was chosen based on the curriculum material as by defining system and user interaction in the most abstract way well as introducing diagrams based on their role and usage in the possible. Package diagram presents the initial logical RUP lifecycle and its respective system model. Each level is architecture of the system under development. Robustness composed of at least one test and lecture material for the diagram is a stereotyped communication diagram used for corresponding topic. robustness analysis. It is used to fill the gap between system requirement analysis and design steps. After the class definition The very first level was business modeling which had one in robustness diagram, sequence diagram is used to specify the test, which introduced business modeling and RUP business interaction between system objects and the external actors. The profile stereotypes and their usage. After achieving, the passable basic logic of sequence diagram is supposed to correspond to the result students could access the example UML models. Second use case scenarios in the previously defined use case activity level presented tasks for the most common UML diagrams, class diagrams. Component diagram is used to define system (attributes, classes and relations) and use case diagram (actors, components that are later realized by previously defined class use cases and relations). The third level was composed of state objects. This diagram also represents component manifestation machine diagram and its elements states, pseudo states and by artifacts. And lastly deployment diagram is used to define the transitions, as well as activity diagram and its elements – actions, system physical architecture and its artifacts’ distribution inside objects, flows, nodes and partitions. The fourth level had tasks, various nodes, such as devices, execution environments and so which outlined the use of previously introduced diagrams and on. their adoption in RUP requirements discipline model. Completing this level unlocked access to the second example of The course curriculum is based not only on the UML UML model. The fifth level introduced robustness diagram for diagram notation, but also on the Rational Unified Process class syntax with specific stereotypes. The sixth level dealt with disciplines and their respective requirements. Business object-oriented analysis and design step in RUP discipline modelling is used to define goals. Business analysis provides process. The seventh level introduced sequence diagram and its opportunities to determine possible enterprise process elements – lifelines, fragments and messages, as well as class improvements. Requirements discipline provides framework for diagrams with previously not analyzed elements such as identifying and describing application functional requirements operations, interfaces and specific relations between them. The (such as use cases). Design discipline encapsulates all aspects of eight level introduced implementation discipline and its place in design, including but not limited to architecture, objects, classes, 90 the UML system model. The ninth level introduced component and deployment elements – components, artifacts, nodes and relations between them. Once the ninth level was completed, students gained access to the complete example UML models. Lastly, the tenth level described deployment discipline and diagrams used in this discipline and their semantics. C. Implementation of the proposed course structure Learning management platform Moodle [19] was chosen for the implementation of the proposed methodology because of the extensiveness and adaptability of the platform. A Moodle course (Fig. 1) was created where the designed model of levels was implemented, and 230 test questions were created to check student knowledge on the UML diagrams syntax or semantics. Fig. 2. The example of the test from the gamified Moodle course In addition to the nine tasks for syntax, five tasks were created for semantics levels. Each semantics task was worth 200 points. Like in the syntax levels, students were only awarded full points once, when meeting the passable requirements for the task. Additionally, students who were able to reach passable grade in semantics levels 1, 2 and 4 were rewarded with example UML models. As basic Moodle environment does not have the functionality for awarding points based on task results, and levels. A plugin [20] was used to implement levels and the handing out of points. The plugin also included a leaderboard for rating the students based on their level and earned points (Fig. 3). Fig. 1. Gamified Information System Design Course In syntax levels, nine tasks were created for each diagram type. Each diagram task consisted of test. Each time while attempting the task, student received a grade, which if passed awarded student 100 points only once. Under no circumstances, students’ failure was meant withholding the award. In case of failure, students were awarded only for attempting the task. The example of the Moodle test, used in the course is presented in Fig. 2. Fig. 3. Gamified Information System Course leaderboard after the course completion 91 A plugin Level UP! [20] was used to implement the required IV. RESULTS OF EXPERIMENT PERFORMED USING THE GAMIFIED changes for gamification of the course. During the course COURSE implementation, the plugin was adapted to work seamlessly with Lithuanian language, as the course curriculum material and A. Experiment environment setting language of instruction is Lithuanian. At the start of semester in autumn 2017 students were invited to partake in a gamified course for the duration of the whole The proposed course structure was implemented by locking semester. A total of 27 students were added to the Moodle course contents based on student level. The maximum attainable points on September 21st. of the levels tasks’ determined each level point requirements. Students were also asked to provide responses to the Overall, the syntax levels had a set of 129 questions. questionnaire based on the IMI scale [18]. In order to assess the Respectively each diagram task had the set of ten questions, base group level of intrinsic motivation a 17 question except for the robustness diagram as it had five. questionnaire was created. In total two questionnaires were Likewise, the semantics levels were composed of 97 prepared, one to measure the base level of motivation, the other questions over all five tasks. Business modeling task had ten to assess the level of student motivation during the gamified questions of the available 23. Requirement task had 16 questions course. out of the 34 available questions set. Requirement analysis task The first questionnaire was available from the September 28, had 5 questions out of 10 available. Implementation/ Design task a week after the student introduction into the course. 22 students had 8 questions out of 15 available. Lastly, the deployment task completed the first questionnaire. had 6 displayed question in a task, out of the 15 questions set. The second questionnaire became available after the final The implemented course consisted of 10 levels, in total of exam of the course, on the 15 of January, 2018. 18 students 2300 (Fig. 4) required points to achieve the maximum level, completed the second questionnaire. Respectively each question leaderboard, badges, structured curriculum content and in the second questionnaire directly corresponded to the question additional rewards like example UML models. from the first questionnaire (e.g. “I enjoy studying” in the first questionnaire and “I enjoyed doing activities in the gamified course” in the second questionnaire). Both questionnaires were anonymous, and the collected results were calculated based on the averages. Students participated in the gamified course activities mostly from October 2017 to January 2018, as of writing this paper the last visit by a student was made on 22nd of January. B. Analysis of experiment data For determining the experiment results, an exploratory data analysis was performed for the data compiled during the experiment. The data from Moodle platform was used, as it provides extensive reports for user participation in the courses. In addition to the exploratory data analysis, two questionaries’ results were compiled to measure the intrinsic motivation of the student group. The questionnaire was anonymous, and results are based on the averages. Each question is grouped by a type which the question measures. These types can be used to determine not only the intrinsic motivation but also aspects that have effect on motivation in general. IMI scale supplies seven question group types. Any survey can be tailored to meet the specific needs of the study. For the experiment four group types were selected – interest/enjoyment, perceived competence, effort/importance and value/usefulness. Fig. 4. Gamified Information System Design Course levels requirements An additional question was introduced into the questionnaire, where gamified course was evaluated. The The proposed changes to the course were successfully question asked to specify the frequency of students activity in implemented into a local Moodle platform used by Kaunas the gamified course. University of Technology Informatics faculty. A question bank of 230 questions was created, ten levels for structuring the By comparing the data, we can see that students who did not material, a leaderboard, example UML model rewards and 13 use the gamified course, had much lower intrinsic motivation tasks were effectively put into practice. results except for the interest/enjoyment group (Fig. 5). This could mean that students were interested in the idea of gamification, but were not attracted enough to participate in the course. 92 Lastly comparing the overall results (Fig. 8), we can see that the students’ general intrinsic motivation increases with the frequency of activity in the gamified UML course. Fig. 5. Intrinsic motivation of students which did not participate in the course For the students, who used the gamified course less than once a month, results indicate that student motivation increased in all levels comparing to the base student level, except for the third question type, which measures perceived competence (Fig. Fig. 8. Intrinsic motivation of students grouped by the frequency of usage in 6). This could mean that students feel that the questions were too the course difficult and should be simplified for the gamified course. Additionally, student results were compared to evaluate gamified UML course effect on students’ grades (Fig. 9). Two sets of data were compiled. The first set consisted of the students grades for the 2016 course, during which students did not use or had access to the gamified material. The second set encompassed students’ grades for 2017 course, where students were able to use gamified Information System Design course. The grades are represented as follows: for the test, which is the quiz evaluating student theory knowledge of UML; for the exam, which consists of the test and a practical task for creating an UML model with CASE tool; the suggested grade, which student earns during the semester; and the final grade, which is the final grade of the course. Fig. 6. Intrinsic motivation of students who participated in the course less than once a month In addition, the last response group, which say that they used the gamified UML course at least once per month or more frequently, have even more favorable results (Fig. 7). The responses of students, which used the gamified course regularly, indicate that the motivation increased across all measured aspects. Fig. 9. The grades of the students in the course, with and without gamification Fig. 7. Intrinsic motivation of students which participated in the course once Traditional and gamified courses were organized to take a month or more often place in parallel. Student participation in gamified course was voluntary and students were not offered any other incentives 93 except for rewards in the gamified course such as practical work [4] OMG, "UML specification," Object Management Group, [Online]. examples. The complied data includes all students regardless of Available: http://omg.org/spec/UML/Current. the fact whether they used gamified course or not. The averages [5] C. Larman, Applying UML and Patterns: An Introduction to Object- are being compared to previous year of 2016 students’ grade Oriented Analysis and Design and Iterative Development, 2004. averages, who did not have access to gamified course. [6] J. Yang, Y. Lee, D. Gandhi and S. G. Valli, "Synchronized UML Diagrams for Object-Oriented Program Comprehension," 2017 12th It is clear that students’ results in test portion increased International Conference on Computer Science and Education (ICCSE), drastically around 0.8 point on average, more than 10 percent. 2017. Other results do not show any dramatic change and therefore [7] J. Schramm, S. Strickroth, N.-T. Le and N. Pinkwart, "Teaching UML could not be attributed as the effect of gamified UML course. Skills to Novice Programmers Using a Sample Solution Based Intelligent Although other three group results exhibit an overall small Tutoring System.," Twenty-Fifth International FLAIRS Conference, 2012. positive change. [8] L. Kuzniarz and M. Staron, "Best Practices for Teaching UML Based V. CONCLUSIONS AND FUTURE WORK Software Development," International Conference on Model Driven Engineering Languages and Systems, vol. 3844, pp. 320-332, 2005. Most of the studies that we have analyzed, place their focus [9] P. Tsarmpou and E. Tambouris, "Using learning analytics to enhance mostly on gamifying the learning material to improve the UML use case diagrams assimilation in a distance education course," understanding and student engagement. The case study International Journal of Learning Technology , vol. 10, no. 4, pp. 274- described in this paper aims to engage students and attempts to 290, 2015. increase student motivation. We have proposed the structure of [10] J. Börstler, L. Kuzniarz, C. Alphonce, W. B. Sanders and M. Smialek, the gamified course, the required gamification elements and "Teaching software modeling in computing curricula," Proceedings of the final reports on Innovation and technology in computer science implemented the proposed course in Moodle platform. The education 2012 working groups, pp. 39-50 , 2012. experiment was carried out during 2017 autumn semester, which [11] "Rational Unified Process," IBM, [Online]. Available: had 27 participating students in the gamified course. While https://www.ibm.com/software/rational. analyzing the results gathered during the experiment, a positive [12] K. Scott, The Unified Process Explained, 2002. influence is observed on the student grades. A marginal difference is recognized in the student intrinsic motivation. [13] J. C. Muñoz, M. A. Cowling and J. Birt, "Using gamification and mixed reality visualization to improve conceptual understanding in ICT system Though we have to admit, that the results are inconclusive, analysis and design," Show Me The Learning. Proceedings ASCILITE because of the small sample of participants. For that reason, we 2016 Adelaide, pp. 455-460, 2016. plan on having a second round of the experiment. The new [14] M.-B. Ibanez, A. Di-Serio and C. Delgado-Kloos, "Gamification for Engaging Computer Science Students in Learning Activities: A Case experiment will have a larger sample size and would be using an Study," Ieee Transactions On Learning Technologies, vol. 7, no. 3, 2014. improved version of the same course. In order to better [15] M. Laskowski and M. Badurowicz, "Gamification In Higher Education: understand the gamification effects on the student motivation, A Case Study," Human Capital without Borders: Knowledge and more gamification elements will be introduced into the second Learning Quality of Life Management, Knowledge and Learning iteration of the course. According to the feedback from the first International Conference 2014, 2014. experiment, the reducing of requirements for passing the task [16] P. Buckley and E. Doyle, "Gamification and student motivation," and the larger sets of questions will be included as well. Interactive Learning Environments, vol. 6, no. 24, 2016. [17] D. Rutkauskiene, D. Gudoniene, R. Maskeliunas and T. Blazauskas, "The Gamification Model for E-Learning Participants Engagement," in Smart Education and e-Learning 2016, Smart Innovation, Systems and REFERENCES Technologies, Springer International Publishing Switzerland, 2016. [1] S. Deterding, D. Dixon, R. Khaled and L. Nacke, "From game design [18] "Intrinsic Motivation Inventory (IMI)," Self-Determination Theory, elements to gamefulness: defining "gamification"," MindTrek '11 [Online]. Available: http://selfdeterminationtheory.org/intrinsic- Proceedings of the 15th International Academic MindTrek Conference: motivation-inventory/. Envisioning Future Media Environments, pp. 9-15, 2011. [19] "Moodle," Moodle HQ, [Online]. Available: https://moodle.org/. [2] B. Burke, Gamify: How Gamification Motivates People to Do [20] F. Massart, "Moodle plugins directory: Level up!," Moodle, [Online]. Extraordinary Things, Bibliomotion, 2014. Available: https://moodle.org/plugins/block_xp. [3] M. Urh, G. Vukovic, E. Jereb and R. Pintar, "The model for introduction of gamification into e-learning in higher education," 7th World Conference on Educational Sciences, pp. 388-397, 2015. 94