=Paper=
{{Paper
|id=Vol-3692/sample5
|storemode=property
|title=GLiDE: Integrated Gamified Learning Dashboard Environment
|pdfUrl=https://ceur-ws.org/Vol-3692/paper5.pdf
|volume=Vol-3692
|authors=Carles Farré,Lidia López,Marc Oriol,Adrià Espinola,Albert Miñana,Xavier Franch
|dblpUrl=https://dblp.org/rec/conf/caise/FarreLOEMF24
}}
==GLiDE: Integrated Gamified Learning Dashboard Environment==
https://ceur-ws.org/Vol-3692/paper5.pdf
GLiDE: Integrated Gamified Learning Dashboard
Environment
Carles Farré*, Lidia López, Marc Oriol, Adrià Espinola, Albert Miñana and
Xavier Franch
Universitat Politècnica de Catalunya, Jordi Girona 1-3, 08034 Barcelona, Spain
Abstract
The Integrated Gamified Learning Dashboard Environment (GLiDE) project aims at fostering
student engagement, teamwork, and project performance in software engineering education. By
integrating gamification elements and enhanced user experience in an existing Learning
Dashboard, GLiDE offers a novel approach to learning in team-based software development
projects. This paper discusses the motivation behind GLiDE, its current status, and the open
research areas. This project offers an opportunity to gain valuable insights into the fields of
educational technology and software engineering education.
Keywords
Learning Dashboard, Information Systems Education, Gamification, User Experience
Project Information Funding Agency: Universitat Politècnica de Catalunya; Period: May 2023 - July 2024;
URL: https://gessi.upc.edu/en/projects/glide-gamified-learning-dashboard-environment.
1. Introduction
Team-based software development projects provide an excellent opportunity for students
to experience the challenges of teamwork and project management that they will face in the
industry [5]. When students are introduced and motivated correctly, they become
enthusiastic about working in teams [9]. However, the actual experience may sometimes
fail to meet students' expectations [5]. For this reason, we decided to investigate whether a
learning dashboard could assist student teams when working on development projects in
the context of software engineering courses [2]. Learning Dashboards are information
systems in the education domain designed to support students and instructors in their
learning/teaching activities [12].
RPE@CAiSE’24: Research Projects Exhibition at the International Conference on Advanced Information Systems
Engineering, June 3–7, 2024, Limassol, Cyprus
* Corresponding author.
carles.farre@upc.edu (C. Farré); lidia.lopez@upc.edu (L. López); marc.oriol@upc.edu (M.Oriol);
adria.espinola@estudiantat.upc.edu (A. Espinola); albert.minana@estudiantat.upc.edu (A. Miñana)
xavier.franch@upc.edu (X. Franch)
0000-0001-5814-3782 (C. Farré); 0000-0002-6901-9223 (L. López), 0000-0003-1928-7024 (M. Oriol) ;
0000-0001-9733-8830 (X. Franch)
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
34
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
This paper presents an ongoing research project, Integrated Gamified Learning
Dashboard Environment (GLiDE), which leverages our initial Learning Dashboard [2] to
design and implement an enhanced version aimed at improving student motivation and
engagement, team collaboration and cohesion, project performance, and quality of the
artifacts produced while working on Software Engineering projects.
2. Context
Software Engineering Project (SEP) and Web Applications and Services (WAS) are two
subjects of the Bachelor’s degree in Informatics Engineering of the Faculty of Informatics of
Barcelona (Universitat Politècnica de Catalunya, UPC). Both subjects apply project-based
learning by developing a software project, which the students carry out organized in teams.
In the 2021/2022 academic year, we launched the project "Implementation of a learning
dashboard to visualize and monitor the achievement of learning objectives in subjects based
on the team development of software projects," partially financed by the UPC [10]. With the
addition of the Learning Dashboard (LD) in the SEP and WAS subjects, we intended that
through its use, students and teachers of these subjects could receive accurate feedback on
the individual and team learning process and also make informed decisions about how to
prioritize and plan their actions during the development of software projects.
LD leverages the Q-Rapids Dashboard, initially aimed at monitoring and evaluating
software production in agile environments [7], and integrates data from various software
development tools used by student teams, such as Taiga, GitHub, and Google Sheets. From
these data sources, metrics, factors, and strategic indicators are calculated to provide a
global view and assessment of the project. An interactive graphic interface shows the
current state and also the historical evolution of these metrics, factors, and strategic
indicators. The values are classified and displayed according to predefined thresholds,
which the teacher can customize to evaluate the project's performance. Figure 1 shows two
screenshots of the LD.
Access to LD was given to roughly half of the teams to have a control group. Its use was
not promoted or encouraged, nor used for evaluation purposes to avoid discrimination
against teams without access to the tool. Students and teams using the tool answered
questionnaires addressing the understanding of metrics, perceived usefulness of metrics,
and perceived clarity of visualizations and complemented with personal interviews with a
member of each team [13, 2]. The obtained feedback can be summarized as follows:
• Students infer the purpose of the metrics but sometimes have difficulty
understanding the intricate details of their definitions and making sense of the
resultant values.
• Students do not perceive that metrics help them manage their projects or achieve
their objectives.
• While the LD offers a clear user experience, it lacks the visual appeal and
engagement necessary to truly captivate users.
35
• Most students cite time constraints and inadequate LD training as barriers to regular
use, feeling overwhelmed by the prospect of integrating “yet another tool” into their
workflow.
Figure 1: Screenshots of the Learning Dashboard. On the left, a snapshot of the values of
the metric "Fulfillment of Tasks". On the right, the evolution of these values over a period
of time.
3. Objectives
Our evaluation of the Learning Dashboard (LD) implementation has been generally positive,
although it has not fully realized its potential. We have identified three primary barriers
that limit the LD's effectiveness: 1) a lack of student motivation stemming from insufficient
incentives and ambiguous promotion; 2) students' limited understanding of how the LD can
enhance teamwork and goal achievement; 3) a functional yet unengaging user experience
that fails to maintain student interest. To address these issues, we aim to improve the LD by
developing an updated version, named GLiDE, with the following objectives:
• Increased Student Engagement and Motivation: We plan to redesign the LD’s
user experience to make it more user-friendly and attractive, adding gamification
elements such as points or badges to foster engagement.
• Enhanced Team Dynamics: GLiDE will incorporate new features to facilitate
better communication and coordination.
• Improved Team Performance: GLiDE will include tools that allow teams to
monitor and compare their progress against others through a gamified interface,
promoting a healthy competitive environment that encourages productivity.
• Higher Quality of Project Outputs: By implementing gamification strategies that
recognize and reward high-quality work, GLiDE will motivate students to produce
superior artifacts.
• Exploring Innovative Gamification Strategies: We aim to investigate novel
gamification techniques tailored for software engineering education and beyond.
This includes focusing on personalized learning experiences, seamless integration
36
with various development tools, and evaluating the potential for adaptability to
other team-based project settings.
These targeted improvements are designed to overcome the LD's current limitations and
maximize its impact as an educational tool.
4. Relevance to Advanced Information Systems Engineering
The GLiDE project aligns closely with several topics of interest for the CAiSE community:
• Human-centered techniques: GLiDE's focus on user experience and gamification
aligns with human-centered design principles, which are essential for creating
engaging and effective information systems. By prioritizing students' needs and
motivations, GLiDE aims to make the learning process more interactive and
responsive.
• Visualization techniques in IS: The project leverages advanced visualization
techniques to make complex data understandable and actionable for students.
These techniques are critical in helping them visualize their progress and the
dynamics of their team interactions within software development projects.
• Educational Systems and Learning Analytics: GLiDE is a prime example of
innovation in educational systems and learning analytics. It's designed to enhance
the learning experience in software engineering education through a gamified
dashboard that provides real-time feedback and analytics.
The funding of the GLiDE project by a university grant emphasizes our dedication to
improving educational tools through practical engineering. While the project incorporates
significant engineering activities, its research aspects are equally vital. GLiDE tests and
expands upon theories of gamification and user experience in real educational settings,
contributing new insights into their practical applications. This blend of engineering and
research ensures that GLiDE not only implements but also innovates, providing valuable
data and theoretical advancements to the field of Information Systems Engineering.
5. Current status
As a result of the activities carried out during the project, we have achieved the following
outcomes.
5.1. Analysis of the state of the art in gamification techniques
The GLiDE project started with a comprehensive review of the state of the art in
gamification techniques for software engineering activities and learning dashboards. The
studies analyzed from the literature confirm that gamification has been applied successfully
to several software engineering activities in industry and education, including software
requirements, development, testing, and project management [4]. This literature review
identifies different gamification strategies and techniques, such as organizing projects as a
37
set of challenges or incorporating gamified elements, such as points, badges, virtual coins,
levels, leaderboards, voting, and feedback [8]. Although the literature suggests that these
techniques can improve users’ engagement and motivation, we also identified some
negative effects that need to be considered during the tool’s design [1].
5.2. Creating an enhanced user experience for the Learning Dashboard
Questionnaires and interviews from past semesters revealed that students were reluctant
to use the LD regularly, citing time limitations and a lack of interest in incorporating “yet
another tool” (see section 2). As a first attempt to explore new avenues to encourage using
the LD, we developed a Chrome extension that integrates metric visualizations directly into
Taiga, one of the tools used by student teams. This solution aimed to provide immediate
access to essential metrics, eliminating the need for an additional platform and reducing the
number of tools students must manage, thus addressing the concerns raised in the feedback.
A screenshot of the Chrome extension integrating the visualization of the LD within Taiga is
depicted in Figure 2.
Figure 2: Screenshot of the Chrome extension of the Learning Dashboard.
5.3. Evaluation of the enhanced Learning Dashboard
In the Fall 2023 Semester, students from the subjects WAS and SEP started to use the Google
Chrome extension following the same protocol as the one followed in previous semesters
(see section 2): approximately half of the student teams were given access, and online
questionnaires were handled to assess students' perceptions, supplemented by personal
interviews.
These questionnaires incorporated questions to analyze the System Usability Scale
(SUS), widely recognized as the most commonly used tool for assessing usability [6]. The
SUS scores obtained in the WAS and SEP questionnaires have been 64.25 and 61.59,
respectively. Considering that scores over 68 (0–100) would be considered above average,
the results suggested that there was still room for improvement during the execution of the
GLiDE project.
In addition to the questionnaires, personal interviews were conducted, selecting one
team member as a representative. The feedback can be summarized as follows:
38
• Some students see a utility directed at them, especially to evaluate large projects
and group performance, while others see a utility directed at facilitating teacher
control.
• Generally, individual metrics (e.g., tasks completed by a developer) are more
appreciated than project metrics (e.g., unassigned project tasks). However, there is
no consensus among students on whether having a large number of metrics (some
of them being considered insignificant) may become a problem.
• The user interface (GUI), user experience (UX), and usability are generally well-
valued. Nevertheless, some problems were identified (e.g., the need to scroll down
to see all metrics, the difficulty of understanding the meaning of some metrics), and
some feature requests were gathered (e.g., the capability to visualize specific
metrics directly using more advanced filters).
• Students find gamification a well-considered idea to encourage LD use, especially in
each sprint. However, gamification must only apply to certain metrics. In general,
students regard a relationship between their gamification winnings and their
subject marks favorably.
• Inter-team gamification and knowing the status of the other groups can lead to
healthy competitiveness. However, intra-team gamification can lead to
counterproductive competitiveness inside the same team.
In addition to the previous results, it can be added to the conclusions that there has still
been little use of the LD, generally due to overwork. When employed, its usage tends to be
more for monitoring than supporting decision-making.
5.4. Gamification Framework
GLiDE will base its gamification proposal on the existing framework GOAL [3, 8]. GOAL
defines an ontology with three central concepts (behaviors, achievements, and game rules)
that will be used in GLiDE to articulate the gamification artifacts. Achievements are rewards
given for exhibiting specific behaviors in accordance with game rules. GLiDE will reward
students based on learning objectives and team practices. For example, medals and points
could be used to encourage collaboration, leadership, and teamwork.
5.5. GLiDE Tool
Figure 3 depicts the architecture for the GLiDE tool, which comprises three main modules:
Gamification Manager (blue components on the left of the figure), Integration Manager (red
components on the right of the figure), and a central database (purple component in the
middle).
The two user roles drive component differentiation: students and teachers will use the
basic modules Basic Gamification Manager (BGM) and Basic Interaction Manager (BIM), but
only teachers will have access to Analytical Gamification Manager (AGM) and Advanced
Interaction Manager (AIM). The current implementation of the LD backend will remain a
black box used by the new components through its REST API.
39
The implementation of the GLiDE tool is ongoing, with two MSc. students working full-
time following the Scrum methodology and four three-week sprints. The first two sprints
target basic components (BGM and BIM), while the third and fourth sprints target advanced
components (AGM, AIM).
Figure 3: GLiDE architecture.
6. Open lines of research
As the GLiDE project advances to completion, several research challenges must be
addressed, opening prospects for future work:
• User Engagement and Interaction Analytics: Comprehensive analysis of how
students interact with the GLiDE tool, focusing on engagement metrics, usage
patterns, and feedback to refine gamification strategies and dashboard
functionalities.
• Impact Assessment on Learning Outcomes: Evaluating the effectiveness of the
GLiDE tool in enhancing learning outcomes, particularly in following key software
engineering practices, teamwork, and project management skills.
• Dashboard Customization and Adaptivity: Research into adaptive features of the
dashboard that can tailor the learning experience to individual or team needs,
including personalized challenges, feedback mechanisms, and learning paths.
• Gamification Mechanics Optimization: Investigating the optimal mix of
gamification elements (e.g., points, badges, leaderboards) that maximizes
motivation and engagement without overwhelming or distracting students from
their learning objectives.
40
• Scalability and Broader Applicability: Assessing the scalability of the GLiDE
approach to accommodate larger class sizes and its potential applicability to other
team-based project settings like the onboarding of new employees.
• Seamless Integration with Developer Tools: Leveraging our experience with the
Taiga plugin, we plan to explore avenues for integrating or ensuring the
interoperability of the GLiDE tool with other developer tools such as Jira.
Acknowledgements
This project is funded by the Universitat Politècnica de Catalunya: Call for grants for
teaching improvement and innovation projects Galàxia Aprenentatge 2023 [11]. The project
was granted under the Catalan title "Introducció d' elements de ludificació a l'eina Learning
Dashboard per millorar l'aprenentatge en assignatures basades en el desenvolupament en
equip de projectes software" as it appears in the call's resolution.
References
[1] Almeida, C., Kalinowski, M., Uchôa, A., Feijó, B.: Negative effects of gamification in
education software: Systematic mapping and practitioner perceptions. Inf. Softw.
Technol. 156, 107142 (2023).
[2] Farré, C., Franch, X., Oriol, M., Volkova, A.: Supporting Students in Team-Based Software
Development Projects: An Exploratory Study. In: RCIS 2023. pp. 568–576 (2023).
[3] García, F., Pedreira, O., Piattini, M., Cerdeira-Pena, A., Penabad, M.: A framework for
gamification in software engineering. J. Syst. Softw. 132, 21-40 (2017).
[4] García-Mireles, G.A., Morales-Trujillo, M.E.: Gamification in Software Engineering: A
Tertiary Study. In: Mejia, J., Muñoz, M., Rocha, Á., Calvo-Manzano, J.A. (eds.) CIMPS 2019.
AISC, vol. 1071, pp. 116-128. Springer, Cham (2020).
[5] Iacob, C., Daily, S.: Exploring the Gap between the Student Expectations and the Reality
of Teamwork in Undergraduate Software Engineering Group Projects. J. Syst. Softw.
157, 110393 (2019).
[6] Lewis, J.R.: The System Usability Scale: Past, Present, and Future. Int. J. Hum.–Comput.
Interact. 34(7), 577-590 (2018).
[7] López, L., Martínez-Fernández, S., Gómez, C., Choraś, M., Kozik, R., Guzmán, L., Vollmer,
A.M., Franch, X., Jedlitschka, A.: Q-rapids Tool Prototype: Supporting Decision-Makers
in Managing Quality in Rapid Software Development. In: CAiSE-Forum 2018, pp. 200–
208 (2018).
[8] Pedreira, O., García, F., Piattini, M., Cortiñas, A., Cerdeira-Pena, A.: An architecture for
software engineering gamification. Tsinghua Sci. Technol. 25(6), 776-797 (2020).
[9] Raibulet, C., Fontana, F.: Collaborative and Teamwork Software Development in an
Undergraduate Software Engineering Course. J. Syst. Softw. 144, 409-422 (2018).
[10] Universitat Politècnica de Catalunya: Resolution of the UPC's 2021 call for teaching
innovation projects (2021). https://ice.upc.edu/ca/suport-a-la-
docencia/convocatoria-millora-docent-UPC/resolucio-de-la-
convocatoria_22jul2021.pdf
41
[11] Universitat Politècnica de Catalunya: Call for grants to carry out teaching improvement
and innovation projects Galàxia Aprenentatge (2023). https://ice.upc.edu/ca/suport-
a-la-docencia/ajuts-per-a-la-realitzacio-de-projectes-de-millora-i-innovacio-docent-
galaxia-aprenentatge-2023
[12] Verbert, K., Govaerts, S., Duval, E., Santos, J.L., Assche, F.v., Parra, G., Klerkx, J.: Learning
Dashboards: An Overview and Future Research Opportunities. Pers. Ubiquit. Comput.
18, 1499–1514 (2014).
[13] Volkova, A.: Specification and design of a dashboard for monitoring the learning
process in software projects developed by teams of students. MSc Thesis, Universitat
Politècnica de Catalunya (2022). https://upcommons.upc.edu/handle/2117/371383
42