=Paper=
{{Paper
|id=Vol-2985/paper5
|storemode=property
|title=Gamified Learning Analytics:
An Initial Outline of Design Concept Synergies from Two Fields
|pdfUrl=https://ceur-ws.org/Vol-2985/paper5.pdf
|volume=Vol-2985
|authors=Adam Palmquist,Fabian Gunnars,Morten Njå,Christina Gkini,Fredrik Breien,Izabella Jedel
}}
==Gamified Learning Analytics:
An Initial Outline of Design Concept Synergies from Two Fields==
https://ceur-ws.org/Vol-2985/paper5.pdf
Gamified learning analytics:
An initial outline of design concept synergies from two fields
Adam Palmquist 1, Fabian Gunnars 2, Morten Njå 3, Christina Gkini 4,5, Fredrik Breien 4,6 and
Izabella Jedel 7
1
University of Gothenburg, Dept. of Applied IT, Address, Sweden
2
Mid Sweden University, Dept. of Education, Sundsvall, Sweden
3
University of Stavanger, Knowledge Centre for Education, Norway
4
University of Bergen, Centre for the Science of Learning & Technology, Norway
5
University of Bergen, System Dynamics Group, Department of Geography, Norway
6
University of Bergen, Dept. of Information Science & Media Studies, Norway
7
Insert Coin, Sweden
Abstract
Technology advancement has dynamically improved the ability to conduct research on large
amounts of data and produce innovative ways to engage students. This work-in-progress paper
presents tentative research proposals devised from two related and emerging fields: learning
analytics and educational gamification. We highlight three shared concepts – processes and
elements of development (design), institutional actors and practitioners (stakeholders), and
perceptions about usability and adoption (acceptance). We explore the unique nature of each
field pertaining to these concepts. Further, we explore how these fields intersect and present
opportunities for implementing design that can be beneficial to researchers and practitioners
currently working at the interaction between the fields. We also want to bring awareness to the
potential synergies that the combination of these fields presents.
Keywords 1
Gamification, Learning Analytics, Synthesis, Conceptual, Narrative review, Design,
Stakeholder, User, Acceptance, Adoption
1. Introduction
Learning Analytics (LA) and Educational Gamification (EG) are two fields that have previously
been developed as two parallel constructs. Since the 2010s, EG and LA have rapidly evolved in
education research [1], [2]. However, until recently, the fields have had separate expansion paths.
Today, more attention has been directed towards their intersection, indicating a growing interest and
value in finding synergies between the fields.
In the Nordics, there is an ongoing tendency of academic-industrial cooperation in researching1,2 and
productizing EG and Serious Games (SG)3. Correspondingly, there is a rising educational technology
industry4 and game industry5 in the region. By outlining predominant conceptual denominators between
Nordic Learning Analytics (Summer) Institute 2021, KTH Royal Institute of Technology, Stockholm, 23 August 2021
EMAIL: adam.palmquist@ait.gu.com (A.Palmquist); fabian.gunnars@miun.se (F.Gunnars); morten.nja@uis.no (M.Njå);
christina.gkini@uib.no (C.Gkini); fredrik.breien@uib.no (F.Breien); izabellajedel@hotmail.com (I.Jedel)
ORCID: 0000-0003-0943-6022 (A.Palmquist); 0000-0002-9803-3358 (F.Gunnars); 0000-0002-3876-8598 (M.Njå);0000-0001-6344-5734 (C.Gkini); 0000-0003-3849-1709
(F. Breien);0000-0001-9212-3259 (I.Jedel)
© 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Wor
Pr
ks
hop
oceedi
ngs
ht
I
tp:
//
ceur
-
SSN1613-
ws
.or
0073
g
CEUR Workshop Proceedings (CEUR-WS.org)
1
An ongoing co-founded PhD project Bergen Science Center VilVite, University of Bergen and Turbo Tape Games exploring effects
from narratives in SG.
2
Finnish LudoCraft, Danish Serious Games Interactive and Norwegian Kahoot! is all outcomes of research and PhD project at
Scandinavian Universities.
3
HolonIQ (2020) HolonIQ Nordic-Baltic EdTech 50: HolonIQ’s annual list of the most innovative EdTech startups across the Nordic-
Baltic region.
4
Dataspelsbranschen. (2018). Spelutvecklarindex. Stockholm: ANGI.
EG and LA on the basis that the two fields are operating in the same context, this paper aims to benefit
from the Nordics in particular and worldwide in general, practitioners and scholars of both fields
respectably. Addressing these specific concepts, we intend to pave the way for upcoming
collaborations, prevent loss of resources while enabling the development and innovation of learning
technology towards a fusion of LA and EG, advancing both the academic and industry field.
LA is an emerging multidisciplinary research field [3]. The field has a strong community rooted in
the Society for Learning Analytics Research (SoLAR), and its emergence can be traced back to the
announcement of the First International Conference on Learning Analytics & Knowledge (LAK) in
2011 [4], [5]. Currently, one of the most cited papers defines LA as: “the measurement, collection,
analysis and reporting of data about learners and their contexts, for purposes of understanding and
optimising learning and the environments in which it occurs” [5 p.31]. LAs educational approach
collects, analyses, and visualises student data based on interactions and engagement with digital tools
in a Learning Environment (LE) [5]. LA provides a dynamic understanding of the learning process by
visualising student behaviours [6], [7].
The LA field’s underpinning application provides dynamic insights and improves students’ learning
outcomes based on the collection and analysis of data from their interaction with digital tools in a LE.
This premise has sparked a growing interest for LA in the educational sectors [8].
Initially, gamification was deployed as a marketing strategy but has emerged in various business [9]
and research fields [10]. Gamification has also been adapted for educational purposes, where learning
tools such as Duolingo, Khan Academy and Kahoot signal an emerging large gamified educational
technology industry. Today, education is the largest research field for gamification [10].
EG aims to enhance student engagement and performance in a LE by adding game design elements
such as collecting badges, setting goals, or collaborating in learning missions. The game design
elements are based on active user participation, staged exploration and instant feedback rather than
passive user determination and observation [11]. These traits are often conceptual qualities with
potential in a LE as educational tools [12]. The widespread implementation of gamification in a LE has
been explained in terms of its intuitive fit with education, as gamification shares similarities with
formative assessment principles of step-by-step progression and obtaining feedback for each step [10].
EG and LA share many similarities regarding their opportunities in a LE. Both fields empower
students and facilitate improved learning outcomes by providing multimodal feedback on digital
behaviours [13]. Moreover, they both focus on enhancing various stakeholders' outcomes in a LE [2],
[14], [15]. The similar aims of LA and EG indicate that integration would be beneficial, as displayed
by recent intersectional work that used LA methods to personalise game elements facilitating higher
student engagement [16] or successfully designing an entire learning management system (LMS) based
on EG and LA elements [13].
However, the fields are stipulated with complications in both business and academia: EG deals with
the novelty effect [17], [18], student exploitation and manipulation[19], and the current need for guiding
theories that enable longitudinal investigation [20]. Also, EG has been questioned regarding the
knowledge transfer [21] and the complexities surrounding digital game-based learning in a school LE
[22].
For LA, complications relate to privacy issues, consent and transparency, and the broader issues of
datafication of higher education [23], [24]; ‘black-box’ technology and algorithmic decision-making
[25].
By combining techniques and procedures from LA and EG, there is a potential to unfold an
additional application dimension in education practice and research. Merging LA and EG could also
entail severe ethical concerns for the practitioner and/or the researcher that needs further scrutiny. Both
fields are surrounded by overly optimistic broadcasting with ready-to-go-to-market products that may
not be sufficiently thought-through [26]–[28]. An initial theoretical integration of the two fields that
can support and guide further research on the application of LA and EG is needed. Therefore, this
conceptual paper outlines opportunities for researchers and practitioners to improve learning outcomes
by combining LA and EG.
2. Conceptual Foundation and Concept Development
The paper builds on a narrative review [29] rooted in a synthesis outlook that focuses on
summarisation, differentiation and integration [30]. This approach provides a framework for identifying
theoretical commonalities between LA and EG by translating concepts for each field towards a common
understanding [31].
Theoretically developing these areas may be helpful for both researchers and practitioners. The
concepts were identified by contrasting two omnibuses from each field. From the field of LA, the
Society for Learning Analytics Research (SoLAR) Handbook of Learning Analytics [32] while for EG,
a special section on Gamification: Gameful Design, Research, and Applications in Computers in Human
Behaviour [27] were used as a frame of reference and primary foundation for the conceptualisation and
theory synthesis.
The Handbook of Learning Analytics is devised to display the current LA research field blending
rigour, quality, interest and appeal [33]. The publication consists of several peer-reviewed papers
(chapters) thematically categorised under four prevalent LA research field: (1) Foundational concepts
– focusing on high-level concepts of LA; (2) Techniques and approaches – discussing critical
methodologies and their development in LA; (3) Applications – Reviewing the methodologies together
with several fields occurring in LA; and (4) Institutional Strategies & Systems Perspectives – addressing
practical challenges implementing LA.
Similarly, the three prevailing research fields in gamification as laid out in Gamification: Gameful
Design, Research, and Applications are: (1) Theory-driven empirical studies – outlining the current
knowledge structure of the field; (2) Design methods – focusing on challenges, tools, and processes
when designing gamification; and (3) Application areas – considering activities and a LE which lend
themselves to being gamified and activities in a LE that do not.
The two publications have a similar raison d'etre to capture the current state and themes of the field's
prevalent research and were published the same year, 2017, which is attractive because the scientific
investigation of EG and LA shares approximately the same duration. The primary justification for
choosing these two particular omnibuses as the foundation was their concurrence of focal research
phenomenons [32] concerning the concepts design, stakeholders and acceptance in EG and LA. These
concepts require further elaboration concerning scope, practice, unit of analysis, techniques, and
conclusion in the intersection of LA and EG.
The SALSA (Search, AppraisaL, Synthesis and Analysis) framework [34] was used to construct the
synthesis with four inclusion criteria: had to address design, stakeholders and acceptance, present a
defined empirical or theoretical case, undergone a review process and constitute a valid contribution to
the conceptual synthesis. If a manuscript had three of these four criteria without considerable overlap
between domains, it was included in the analysis.
A limitation of the study is inherently linked to the omnibuses. First, their credibility towards
encapsulating conceptual aspects of entire fields may be questioned. Second, LA and EG are two
relatively novel and fast emerging fields. Thus, they are both evolving through academic research, and
further, their development accelerates, which are both evolving through academic research and are
thrust forward by a continuously accelerating evolving tech industry development. Knowing this,
synthesising theoretical commonalities on agreed-upon prevailing concepts from 2017 is a shortcoming
that may fail to capture recent conceptual developments in both fields. Having an awareness of these
circumstances, the authors of this paper agreed that a narrative review that encompassed knowledge
leading up to and preceding 2017 was necessary to improve this paper's rigour.
The authors of this article have inductively identified different conceptualisations of the focal
research phenomenons in the omnibuses and exceeding research. Building on this knowledge, we argue
that the prominent aspect of interest in the focal research phenomenons is best addressed as summarised,
differentiated and integrated concepts for further investigation for practitioners and researchers alike.
3. Conceptual Foundation and Concept Development
The two representations of the research fields of EG and LA were translated to the research concepts
design, stakeholders and acceptance, which are the focus for the conducted narrative review aiming to
synthesize the research intersection of EG and LA. Below, we present each concept separately, along
with opportunities for implementing design. The design concept describes the process, the elements,
and the rationale of their usage for specific purposes in a LE. The stakeholder concept describes the
various institutional actors and practitioners that affect or are affected by the implementation of LA or
EG. Lastly, the concept acceptance describes both adoption and stakeholders’ perceptions regarding the
usability and usefulness of LA or EG.
3.1. Design Elements and Process
During the first half of the 2010s, LA practitioners were mainly from "hard" disciplines (e.g.,
computer science, mathematics) [23], which, to some extent, attuned the field towards design thinking
of LA instruments in the fields early days. Since then, LA has put significant effort into exploring the
opportunities of design: the purposeful shaping of the human processes required in delivering and
applying analytic tools, data, and reports as part of an educational environment [35]. Design in LA
relates to three main dimensions: interaction and visualisation design, learning design, and study design
[4].
Recently, the design of interactive visualisations, visual dashboards, employed to increase teachers'
awareness of student progress and workflow manipulation, have received considerable attention in LA
research [36] [6]. These dashboards serve as an example of LA with the potential for practitioner and
research opportunities [6], [37], [38]. To a greater extent, there are opinions that LA research should
synthesise with learning theory, e.g., Self-Regulated Learning (SRL) [39]. While perhaps not yet to an
optimal degree [40], LA tools increasingly use pedagogical approaches and learning theories in their
design, e.g., SRL, inquiry-based learning, and constructivist approaches such as collaborative learning
[40][41]. Learning theories are also used to evaluate LA tools’ design, where the primary focus is
“whether the dashboard brings any benefit to learners” [42 p.37].
Learning design, “the practice of devising effective learning experiences aimed at achieving defined
educational objectives in a given context” [43 p.221], has been long discussed in relation to LA [34].
This is not surprising, as LA can offer powerful tools and provide actionable measurements that can
inform practice in a LE [43].
With footing in several design-focused fields (e.g., game studies, human-computer interaction,
information systems), gamification research emphasises the design- process, context and practice [44].
Agreed upon skills required for designing EG are comprehensive and interdisciplinary, including game
design, behavioural science, human learning and, preferably, the topic studied, although a best practice
is yet to be discovered [44]–[46].
A characteristic of the gamification field is the frequent collaboration between academia and
industry, which has facilitated the field’s rapid development, provided valuable insights, produced
various artefacts, and contributed to a surplus of quantitative gamification effect studies.
In EG, various practitioners’ and scholars’ design frameworks occur, from generic to more specific,
composed differently, yet most focus on the LE and the students. The prime purpose of the design
frameworks is to foster motivation towards a particular goal by using game design elements. The game
design elements are accompanied by distinctive amplifying components of various characteristics,
significance, and positions in the design framework. The amplifying components mainly originate from
a broad spectrum of non-game disciplines and business fields, frequently motivational and social
psychology [47], likewise organisational behaviour or behavioural economics [48].
Even though the design is extensive in EG, the field is neither fully understood, defined, or
interpreted. One challenge is reproducing, e.g., an EG design to increase a course completion rate could
play out well the first time but fail in subsequent attempts [45], [46], [49]. To address these issues,
recently, gamification researchers have started to examine the gamification design elements
systematically and to chart the distinct stages in a gamification implementation: ideation, user, design,
context, project planning, implementation and evaluation. Distinguishing the stages of an EG project
has visualised knowledge gaps, showing that the stages ideation, user, design, and context had an
overabundance but less on project planning, implementation, and evaluation.
The recent design research in gamification has transformed the research community interest from
whether gamification works to how it works [27]. By changing the field’s fundamental research
questions from whether?, what? and why?, to how?, when?, as well as how and when not? sets the
stage for a more comprehensive, theory-driven, hypothesis-testing and multidisciplinary research
interest and approach towards gamification design.
3.2. User and Stakeholders
Implementing EG or LA in a LE involves several stakeholders such as students, teachers,
educational providers and management, which are beneficial for implementing projects.
LA categorised stakeholders as data clients and data subjects [50], with the former describing
beneficiaries of LA with authority to act on its products, and the latter the data suppliers, commonly the
students. While the LA community prioritizes students and teachers as the main beneficiaries of LA
tools [50], those are commonly the data subjects, while institutions emerge as the most common data
clients [51]. The benefits for institutions and the related data clients reported in LA literature include:
improving student retention, supporting informed decision making, increasing cost-effectiveness,
understanding students’ learning behaviours, and lastly, providing personalized assistance for students
[15], [52]. LA applied in a digital LE is oriented towards monitoring/analysis and
prediction/intervention [53], to the extent that even when learning theories such as SRL are utilized, the
focus might remain on measurement rather than students' support [54]. This might be further
problematic as the introduction to LA can possibly lead to changes in the behaviour of both educators
and students [8]. The field acknowledges these above issues with a panel of international experts
advocating for enhancing learning as the goal of interventions and prioritising the student as an audience
[37].
Student-centered LA [55] is an approach including students “as equal partners in the collection,
analysis and use of their data” [56 p.618]. Centring students is considered as an ethical way forward, as
it acts to decrease student vulnerability and increase their agency [57].
LA research has recently started to address design from a user perspective, engaging students [58],
[59]and teachers [60], [61] in co-design processes. Still, in a recent systematic review, a “lack of
involvement of stakeholders and adoption of participatory or co-design approaches” [62 p.15] was
observed, indicating the potential of such approaches for the future of the LA field.
In EG, and gamification in general, there is an emphasis on a user-centric approach both in
practitioner [63], [64] and researcher literature e.g., [44], [46], [49]. In previous work on gamification,
end-users’ needs and requirements in the design process have been acknowledged to be fundamental.
In a meta-study on gamification design frameworks, the most critical requirement identified was to
“understand the user needs, motivation and behaviour as well as the characteristics of the context.” [46
p.7]. The meta-study concluded that designing gamification without considering its compliance with its
users' needs and perceptions may hinder its implementation and cause adverse side effects, such as
discouragement or demotivation, even threatening the well-being of end-users, depending on the
gamified LE. However, in EG studies, stakeholders other than the end-user receive limited attention
[65], [66]. Transforming players to customer/user and vice versa has been addressed and problematised
regarding its effect on manners, actions, understandings and expectations [67]. LAs accepted and
adopted stakeholder-definition could provide a more robust definition than those practised and display
a comprehensive research scope.
3.3. Acceptance and Adoption
LA research has investigated barriers and drivers to adoption at multiple levels (e.g., institutional,
classroom, individual) and for various stakeholders. Adoption is influenced by factors relating to the
institutional context, the strategy of their deployment, the stakeholders, and challenges at the
intersection of the above [68]. Institutions are fundamental for LA adoption by providing supporting
structures, such as integrated and trustworthy infrastructures and incentives or support in the usage of
LA, as highlighted by faculty and advisors [69]. From a broad perspective, social challenges emerge as
the most significant for LA adoption [67] and the multifaceted nature of the adoption process can be
viewed as not merely a technical or pedagogical issue, but also as a “human challenge — cognitive,
social, organizational, political” [70 p.2.] Further, due to the multiplicity of stakeholders, research on
LA adoption has incorporated perceptions of institutional leaders and senior management [67], as well
as teachers and students, allowing for the identification of differences in perspectives [70]. Due to those
differences, multi-stakeholder communication has been advocated for improving LA acceptance and
adoption [71], as well as the inclusion of teachers and students in design processes [55], [60].
More recently, the focus has turned to the primary stakeholders (students and teachers) and studies,
especially on LA dashboards, have begun more specifically addressing the notion of acceptance. For
example, the technology acceptance model (TAM) [72] has been utilized to explore teachers’
perspectives, finding that, somewhat surprisingly, greater perceived usefulness might not be translated
to greater usage or acceptance of tools [73]. Under the same model, perceived ease of use emerges as
an important challenge [74][75]. This finding might relate to the importance of the institutional context,
available infrastructure, and teacher training [74].
Acceptance for LA also relates to its potential to provide quality information and feedback, with
different needs identified for managers, teachers, and students [76]. Perceptions on LA as promoting
student success also differ between stakeholders. Students view dashboards as useful at a broader level,
e.g., assisting with the transition to university and management skills that are not course-specific [77].
That being said, the ability of LA to facilitate learning is crucial [78]. Research on how LA can support
students’ SRL shows the benefits of tools promoting awareness and reflection [42]; however, those
goals do not necessarily imply that learning outcomes are achieved [40]. Allowing for some interactivity
and social comparison has positive effects on student motivation [2], although suggestions of cautious
usage of such comparisons with peers have not been absent [40, 42].
In EG, one current field of interest is the acceptance and adoption of gamification in a LE. Several
studies on acceptance and adoption have used theories, frameworks and lenses from information
technology, e.g., TAM, Unified theory of acceptance and use of technology (UTAUT) [79] and social
studies, e.g., diffusion of innovations [80] and models adjusted for the research design.
Previous findings regarding higher education students’ acceptance and adoption of EG indicate a
positive attitude [81]–[83]. Factors that influenced students’ acceptance and adoption were performance
expectancy and effort expectancy [82], knowledge improvement, engagement, and playfulness [83],
Learnability, reward and level of e-skills [81].
Overall, studies show a mixed impression of teachers’ views on gamified digital tools in a LE. The
teachers' acceptance and adoption of EG show discrepancies ranging from optimistic [84]–[86] to
neutral/negative depending on condition [66], [87], [88]. In the studies reporting an optimistic attitude
towards EG, few teachers had used EG in their LE. In the more neutral/negative reports, factors
affecting teachers’ acceptance and adoption appear to be related to LE and student dependent [66], [88].
Also, gamification’s compatibility with the instructional content facilitates teacher reception [66], [87],
[88]. Also, gamification’s compatibility with the instructional content facilitates teacher reception [66],
[89].
4. Implications, Future research, and Limitations
Even though gamification design research pioneered early in education, the extensive body of
knowledge encircles (1) single university campus courses as research circumstances and (2) higher
education students and teachers as research participants. This predicament is comparable to LA research
regarding participants and circumstances. Therefore, further research should extend LA and EG design
research beyond higher education and systematically study the moderating effects of different learner
demographics and LE. It is imperative to note that these fields need continual support from further
research and practitioners in industry fields.
Regarding design research in LA and EG, there is also a distinction between the theoretical
frameworks that underpin the fields. Whereas LA mainly focuses on the learning environment and on
promoting self-regulated learning, EG focuses on student engagement and promoting motivation. The
fields could benefit from each other by integrating motivational research from EG into LA as well as
integrating learning research from LA into EG. Further research should explore the application of self-
regulation in EG and the application of motivational theories in LA.
An important distinction when comparing the two fields is that whereas EG can be integrated into
LA (e.g., by adding game elements to the students’ or teachers’ dashboard), meaningful application of
EG will typically involve aspects of LA. Students’ behaviour in gamified LE relies on collecting data,
measuring progress and displaying the student progress, which is all central LA aspects. Due to the
iterative nature of design processes in EG, student data should be used to improve students' LE by e.g.
optimising individual game elements.
LA and EG stakeholders may have distinct design processes in terms of their focal points. In
previous years, LA has to some extent suffered from a lack of user-centric design approaches, which
bring attention to different stakeholders’ needs. On the other hand, EG has prioritised end-users and
thus addressing design primarily from a user perspective. The fields could benefit from each other’s
experiences by conducting research integrating student-centered design with user-centric approaches.
Despite having a more prevalent user-centric approach, EG has been challenged by a viewpoint that
regards end-users as customers, which can be especially problematic when adopting EG applications in
a LE. Stakeholder engagement in LA emphasises ethics, agency, and user empowerment (i.e., students
and teachers), which can assist in combating limitations that are currently a challenge for EG in a LE.
In terms of acceptance, the two fields have developed along somewhat distinct paths: LA research
has been concerned with investigating barriers and drivers to adoption at multiple levels (e.g.,
institutional, classroom, individual), whereas EG has focused on measuring acceptance at the end-user
level (i.e., students and teachers). The integration of a variety of aspects (e.g., emotional, behavioural,
physical, social and cultural) could ultimately be strongly beneficial towards designing a LE with
inclusive goals. As outlined in LA research, this design could be properly adopted in a more vital
educational infrastructure of personal and social relationships, organisational arrangements and
architecture of material space of settings. Both fields can develop links between these collective levels.
Clear opportunities lie in the mutual consensus in LA and EG on methods and theories measuring
acceptance, e.g., the technology acceptance model. Agreed upon frameworks are beneficial when
synthesising and translating LA and EG concepts. Also, such frameworks are exceedingly adaptable to
the context and perspective; researchers from each field may combine factors of interest as part of their
investigations. E.g., one initial investigation could examine how factors from EG (i.e. playfulness) and
LA (i.e. self-regulated learning) affect each other and contribute to acceptance and adoption.
5. References
[1] D. Maciuszek, M. Weicht, and A. Martens, “Composing Game-based Learning Scenarios by
Connecting Instructional Design Patterns,” in Game-Based Learning: Challenges and Opportunities,
["Felicia and Patrick"], Eds. Newcastle upon Tyne: Cambridge Scholars Publishing, 2014, pp. 29–54.
[2] R. Pérez-Álvarez, J. Maldonado-Mahauad, and M. Pérez-Sanagustín, “Tools to Support Self-
Regulated Learning in Online Environments: Literature Review,” Lect Notes Comput Sc, pp. 16–30,
2018, doi: https://doi.org/10.1007/978-3-319-98572-5_2.
[3] S. Dawson, D. Gǎsević, G. Siemens, and S. Joksimovic, “Current State and Fu-ture Trends: A
Citation Network Analysis of the Learning Analytics Field,” in Pro-ceedings of the Fourth International
Conference on Learning Analytics And Knowledge, 2014, pp. 231–240. doi:
10.1145/2567574.2567585.
[4] D. Gǎsević, V. Kovanovíc, and S. Joksimovi\’c, “Piecing the learning analytics puzzle: A
consolidated model of a field of research and practice,” Learn Res Pract, vol. 3, no. 1, pp. 63–78, 2017,
doi: 10.1080/23735082.2017.1286142.
[5] G. Siemens and P. Long, “Penetrating the Fog: Analytics in Learning and Educa-tion,” EDUCAUSE
Review, vol. 5, pp. 30–32, 2011, doi: 10.17471/2499-4324/195.
[6] M. Cukurova, M. Giannakos, and R. Martinez‐Maldonado, “The promise and challenges of
multimodal learning analytics,” 2020.
[7] D. Hernández‐Leo, R. Martinez‐Maldonado, A. Pardo, J. A. Muñoz‐Cristóbal, and M. J. Rodríguez‐
Triana, “Analytics for learning design: A layered framework and tools,” Brit J Educ Technol, vol. 50,
no. 1, pp. 139–152, 2019, doi: 10.1111/bjet.12645.
[8] S. Knight and S. B. Shum, “Theory and Learning Analytics,” in The Handbook of Learning
Analytics, ["Lang, Charles and Siemens, George and Wise, Alyssa Friend and Gaševic, and Dragan"],
Eds. Alberta: Society for Learning Analytics Research (SoLAR), 2017, pp. 17–22. [Online]. Available:
http://solaresearch.org/hla-17/hla17-chapter1
[9] T. W. Kim and K. Werbach, “More than just a game: ethical issues in gamifica-tion,” Ethics Inf
Technol, vol. 18, no. 2, pp. 157–173, 2016, doi: 10.1007/s10676-016-9401-5.
[10] J. Koivisto and J. Hamari, “The rise of motivational information systems: A review of gamification
research,” Int J Inform Manage, vol. 45, pp. 191–210, 2019, doi: 10.1016/j.ijinfomgt.2018.10.013.
[11] K. M. Kapp, The gamification of learning and instruction. Wiley San Francisco, 2012.
[12] R. N. Landers, “Developing a Theory of Gamified Learning: Linking Serious Games and
Gamification of Learning,” Simulat Gaming, vol. 45, no. 6, pp. 752–768, 2014, doi:
10.1177/1046878114563660.
[13] D. Dicheva, K. Irwin, and C. Dichev, “OneUp: Supporting Practical and Exper-imental
Gamification of Learning,” Int J Serious Games, vol. 5, no. 3, pp. 5–21, 2018, doi:
10.17083/ijsg.v5i3.236.
[14] W. Chen, “Knowledge-aware learning analytics for smart learning,” in Procedia Computer
Science, 2019, vol. 159, pp. 1957–1965. doi: 10.1016/j.procs.2019.09.368.
[15] B. Tak and M. Wong, “Learning analytics in higher education: an analysis of case studies,” Asian
Assoc Open Univ J, vol. 12, no. 1, pp. 21–40, 2017, doi: 10.1108/aaouj-01-2017-0009.
[16] S. Hallifax, A. Serna, J.-C. Marty, and É. Lavoué, “Transforming Learning with Meaningful
Technologies, 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Delft,
The Netherlands, September 16–19, 2019, Proceed-ings,” Lect Notes Comput Sc, pp. 294–307, 2019,
doi: 10.1007/978-3-030-29736-7_22.
[17] B. J. Morris, S. Croker, C. Zimmerman, D. Gill, and C. Romig, “Gaming sci-ence: The
‘Gamification’ of scientific thinking,” Front Psychol, vol. 4, no. SEP, p. 607, 2013, doi:
10.3389/fpsyg.2013.00607.
[18] K. Seaborn and D. I. Fels, “Gamification in theory and action: A survey,” Int J Hum-comput St,
vol. 74, pp. 14–31, 2015, doi: 10.1016/j.ijhcs.2014.09.006.
[19] A. Palmquist and J. Linderoth, “‘ Gamification does not belong at a university ,’” 2020.
[20] S. Bai, K. F. Hew, and B. Huang, “Does gamification improve student learning outcome? Evidence
from a meta-analysis and synthesis of qualitative data in educa-tional contexts,” Educ Res Rev-neth,
vol. 30, no. June 2019, p. 100322, 2020, doi: 10.1016/j.edurev.2020.100322.
[21] J. Linderoth, “Why gamers don’t learn more: An ecological approach to games as learning
environments,” Journal of gaming & virtual worlds, vol. 4, no. 1, pp. 45–62, 2012.
[22] B. B. Marklund, “Unpacking Digital Game-Based Learning : The complexities of developing and
using educational games,” 2015.
[23] N. Selwyn and D. Gašević, “The datafication of higher education: discussing the promises and
problems,” Teach High Educ, vol. 25, no. 4, pp. 527–540, 2020, doi: 10.1080/13562517.2019.1689388.
[24] B. Williamson, S. Bayne, and S. Shay, “The datafication of teaching in Higher Education: critical
issues and perspectives,” Teach High Educ, vol. 25, no. 4, pp. 351–365, 2020, doi:
10.1080/13562517.2020.1748811.
[25] P. Prinsloo, “Of ‘black boxes’ and algorithmic decision-making in (higher) education – A
commentary,” Big Data Soc, vol. 7, no. 1, p. 2053951720933994, 2020, doi:
10.1177/2053951720933994.
[26] R. Stoneham, “Failing Students need Big Data and Learning Analytics: Hype or Reality?,”
Compass J Learn Teach, vol. 7, no. 11, 2015, doi: 10.21100/compass.v7i11.221.
[27] L. E. Nacke and S. Deterding, “The maturing of gamification research,” Comput Hum Behav, vol.
71, no. Computers in Human Behavior712016, pp. 450–454, 2017, doi: 10.1016/j.chb.2016.11.062.
[28] M. Raftopoulos, “Has gamification failed, or failed to evolve? Lessons from the frontline in
information systems applications,” 2020.
[29] R. Ferrari, “Writing narrative style literature reviews,” Medical Writ, vol. 24, no. 4, pp. 230–235,
2015, doi: 10.1179/2047480615z.000000000329.
[30] S. E. Fawcett, M. A. Waller, J. W. Miller, M. A. Schwieterman, B. T. Hazen, and R. E. Overstreet,
“A Trail Guide to Publishing Success: Tips on Writing Influen-tial Conceptual, Qualitative, and Survey
Research,” J Bus Logist, vol. 35, no. 1, pp. 1–16, 2014, doi: 10.1111/jbl.12039.
[31] J. Webster and R. Watson, “Analyzing the Past to Prepare for the Future: Writ-ing a Literature
Review,” MIS Quarterly, 2002.
[32] E. Jaakkola, “Designing conceptual articles: four approaches,” Ams Rev, vol. 10, no. 1–2, pp. 18–
26, 2020, doi: 10.1007/s13162-020-00161-0.
[33] “Handbook of Learning Analytics,” 2017, doi: 10.18608/hla17.
[34] M. J. Grant and A. Booth, “A typology of reviews: an analysis of 14 review types and associated
methodologies,” Heal Information Libr J, vol. 26, no. 2, pp. 91–108, 2009, doi: 10.1111/j.1471-
1842.2009.00848.x.
[35] L. Lockyer and S. Dawson, “Where learning analytics meets learning design,” Proc 2nd Int Conf
Learn Anal Knowl - Lak ’12, pp. 14–15, 2012, doi: 10.1145/2330601.2330609.
[36] A. L. Liu and J. C. Nesbit, “Dashboards for Computer-Supported Collaborative Learning,” in
Machine Learning Paradigms: Advances in Learning Analytics, ["Virvou, Maria and Alepis, Efthimios
and Tsihrintzis, George A. and Jain, and Lakhmi C."], Eds. Cham: Springer International Publishing,
2019, pp. 157–182. doi: 10.1007/978-3-030-13743-4_9.
[37] R. Kitchin, “Big Data, new epistemologies and paradigm shifts,” Big Data Soc, vol. 1, no. 1, p.
2053951714528481, 2014, doi: 10.1177/2053951714528481.
[38] D. Ifenthaler, D. Gibson, D. Prasse, A. Shimada, and M. Yamada, “Putting learning back into
learning analytics: actions for policy makers, researchers, and practitioners,” Educ Technology Res
Dev, pp. 1–20, 2020, doi: 10.1007/s11423-020-09909-8.
[39] W. Matcha, N. A. Uzir, D. Gaevi, and A. Pardo, “A Systematic Review of Em-pirical Studies on
Learning Analytics Dashboards: A Self-Regulated Learning Per-spective,” Ieee T Learn Technol, vol.
13, no. 2, pp. 226–245, 2018, doi: 10.1109/tlt.2019.2916802.
[40] I. Jivet, M. Scheffel, H. Drachsler, and M. Specht, “Awareness is not enough: Pitfalls of learning
analytics dashboards in the educational practice,” in Lecture Notes in Computer Science (including
subseries Lecture Notes in Artificial Intelli-gence and Lecture Notes in Bioinformatics), 2017, vol.
10474 LNCS, pp. 82–96. doi: 10.1007/978-3-319-66610-5_7.
[41] K. Mangaroska and M. Giannakos, “Learning Analytics for Learning Design: A Systematic
Literature Review of Analytics-Driven Design to Enhance Learning,” Ieee T Learn Technol, vol. 12,
no. 4, pp. 516–534, 2019, doi: 10.1109/tlt.2018.2868673.
[42] I. Jivet, M. Scheffel, M. Specht, and H. Drachsler, “License to evaluate: Prepar-ing learning
analytics dashboards for educational practice,” Proc 8th Int Conf Learn Anal Knowl, pp. 31–40, 2018,
doi: 10.1145/3170358.3170421.
[43] Y. Mor, R. Ferguson, and B. Wasson, “Editorial: Learning design, teacher in-quiry into student
learning and learning analytics: A call for action,” Brit J Educ Technol, vol. 46, no. 2, pp. 221–229,
2015, doi: 10.1111/bjet.12273.
[44] S. Deterding, “The lens of intrinsic skill atoms: A method for gameful design,” Hum Comput
Interact, vol. 30, no. 3–4, pp. 294–335, 2015, doi: 10.1080/07370024.2014.993471.
[45] L. Hassan, “Means to Gameful Ends: How Should Gamification Be Designed?,” 2018.
[46] B. Morschheuser, J. Hamari, K. Werder, and J. Abe, “How to Gamify? A Meth-od For Designing
Gamification,” Proc 50th Hawaii Int Conf Syst Sci 2017, 2017, doi: 10.24251/hicss.2017.155.
[47] T. Reiners and L. C. Wood, Gamification in education and business. 2015. doi: 10.1007/978-3-
319-10208-5.
[48] D. Liu, R. Santhanam, and J. Webster, “Toward meaningful engagement: A framework for design
and research of gamified information systems,” Mis Quart, vol. 41, no. 4, pp. 1011–1034, 2017, doi:
10.25300/misq/2017/41.4.01.
[49] A. Mora, D. Riera, C. González, and J. Arnedo-Moreno, “Gamification: a sys-tematic review of
design frameworks,” J Comput High Educ, vol. 29, no. 3, pp. 516–548, 2017, doi: 10.1007/s12528-
017-9150-4.
[50] H. Drachsler and W. Greller, “The Pulse of Learning Analytics Understandings and Expectations
from the Stakeholders,” in Proceedings of the 2nd International Conference on Learning Analytics and
Knowledge, 2012, pp. 120–129. doi: 10.1145/2330601.2330634.
[51] K. Misiejuk and B. Wasson, “State of the Field Report on Learning Analytics,” Centre for the
Science of Learning & Technology (SLATE), University of Bergen, 2017.
[52] N. Elouazizi, “Critical Factors In Data Governance For Learning Analytics,” J Learn Anal, vol. 1,
no. 3, pp. 211–222, 2014, doi: 10.18608/jla.2014.13.25.
[53] S. N. Kew and Z. Tasir, “Learning Analytics in Online Learning Environment: A Systematic
Review on the Focuses and the Types of Student-Related Analytics Data,” Technology Knowl Learn,
pp. 1–23, 2021, doi: 10.1007/s10758-021-09541-2.
[54] O. Viberg, M. Khalil, and M. Baars, “Self-Regulated Learning and Learning Analytics in Online
Learning Environments: A Review of Empirical Research,” in Proceedings of the Tenth International
Conference on Learning Analytics & Knowledge, 2020, pp. 524–533. doi: 10.1145/3375462.3375483.
[55] X. Ochoa and A. F. Wise, “Supporting the shift to digital with student-centered learning analytics,”
Educ Technology Res Dev, vol. 69, no. 1, pp. 357–361, 2021, doi: 10.1007/s11423-020-09882-2.
[56] C. Broughan and P. Prinsloo, “(Re)centring students in learning analytics: in conversation with
Paulo Freire,” Assess Eval High Edu, vol. 45, no. 4, pp. 617–628, 2020, doi:
10.1080/02602938.2019.1679716.
[57] P. Prinsloo and S. Slade, “Student Vulnerability, Agency and Learning Analyt-ics: An
Exploration,” J Learn Anal, vol. 3, no. 1, 2016, doi: 10.18608/jla.2016.31.10.
[58] C. G. Prieto-Alvarez, R. Martinez-Maldonado, and T. D. Anderson, “Co-designing learning
analytics tools with learners,” Learning Analytics in the Class-room, no. December 2019, pp. 93–110,
2019, doi: 10.4324/9781351113038-7.
[59] M. A. Chatti et al., “How to Design Effective Learning Analytics Indicators? A Human-Centered
Design Approach,” in Addressing Global Challenges and Quality Education, pp. 303–317.
[60] M. Dollinger, D. Liu, N. Arthars, and J. M. Lodge, “Working together in learn-ing analytics
towards the co-creation of value,” J Learn Anal, vol. 6, no. 2, pp. 10–26, 2019, doi:
10.18608/jla.2019.62.2.
[61] R. Martinez-Maldonado, D. Elliott, C. Axisa, T. Power, V. Echeverria, and S. B. Shum, “Designing
translucent learning analytics with teachers: an elicitation pro-cess,” Interact Learn Envir, vol. 0, no. 0,
pp. 1–15, 2020, doi: 10.1080/10494820.2019.1710541.
[62] J. Samuelsen, W. Chen, and B. Wasson, “Integrating multiple data sources for learning analytics—
review of literature,” Res Pract Technology Enhanc Learn, vol. 14, no. 1, p. 11, 2019, doi:
10.1186/s41039-019-0105-4.
[63] B. Burke, Gamify: How gamification motivates people to do extraordinary things. routledge, 2016.
[64] J. McGonigal, Reality is broken: Why games make us better and how they can change the world.
Penguin, 2011.
[65] C. Dichev and D. Dicheva, “Gamifying education: what is known, what is be-lieved and what
remains uncertain: a critical review,” Int J Educ Technology High Educ, vol. 14, no. 1, p. 9, 2017, doi:
10.1186/s41239-017-0042-5.
[66] A. Palmquist, “‘Gamification was not the problem’ A case study exploring fac-tors affect teachers
approvement of gamification,” presented at the Academic Mind-trek 21, 2021. doi:
10.1145/3464327.3464347.
[67] S. P. Walz and S. Deterding, The gameful world: Approaches, issues, applica-tions. Mit Press.
[68] Y. S. Tsai, V. Kovanović, and D. Gǎsevíc, “Connecting the dots: An exploratory study on learning
analytics adoption factors, experience, and priorities,” Internet High Educ, vol. 50, no. February, p.
100794, 2021, doi: 10.1016/j.iheduc.2021.100794.
[69] C. Klein, J. Lester, H. Rangwala, and A. Johri, “Technological barriers and in-centives to learning
analytics adoption in higher education: insights from users,” J Comput High Educ, vol. 31, no. 3, pp.
604–625, 2019, doi: 10.1007/s12528-019-09210-5.
[70] S. B. Shum, R. Ferguson, and R. Martinez-Maldonado, “Human-centred learn-ing analytics,” J
Learn Anal, vol. 6, no. 2, pp. 1–9, 2019, doi: 10.18608/jla.2019.62.1.
[71] L. P. Prieto, M. J. Rodríguez-Triana, R. Mart\’inez-Maldonado, Y. Dimitriadis, and D. Gǎsević,
“Orchestrating learning analytics (OrLA): Supporting inter-stakeholder communication about adoption
of learning analytics at the classroom level,” Australas J Educ Tec, vol. 35, no. 4, pp. 14–33, 2019, doi:
10.14742/ajet.4314.
[72] F. D. Davis, “Perceived usefulness, perceived ease of use, and user acceptance of information
technology,” Mis Quart, vol. 13, no. 3, pp. 319–339, 1989, doi: 10.2307/249008.
[73] C. Herodotou, B. Rienties, A. Boroowa, Z. Zdrahal, and M. Hlosta, A large-scale implementation
of predictive learning analytics in higher education: the teachers’ role and perspective, vol. 67. Springer
US, 2019. doi: 10.1007/s11423-019-09685-0.
[74] R. Kaliisa, A. I. Mørch, and A. Kluge, “‘My Point of Departure for Analytics is Extreme
Skepticism’: Implications Derived from An Investigation of University Teachers’ Learning Analytics
Perspectives and Design Practices,” Technology Knowl Learn, no. 1, pp. 1–22, 2021, doi:
10.1007/s10758-020-09488-w.
[75] B. Rienties, C. Herodotou, T. Olney, M. Schencks, and A. Boroowa, “Making Sense of Learning
Analytics Dashboards: A Technology Acceptance Perspective of 95 Teachers. International Review of
Research in Open and Distributed Learning,” International Review of Research in Open and Distributed
Learning, vol. 19, no. 5, p. 19, 2018, [Online]. Available: https://doi.org/10.19173/irrodl.v19i5.3493
[76] I. Hilliger et al., “Identifying needs for learning analytics adoption in Latin American universities:
A mixed-methods approach,” Internet High Educ, vol. 45, no. April 2019, p. 100726, 2020, doi:
10.1016/j.iheduc.2020.100726.
[77] D. B. Knight, C. Brozina, and B. Novoselich, “An Investigation of First-Year Engineering Student
and Instructor Perspectives of Learning Analytics Approaches,” J Learn Anal, vol. 3, no. 3, pp. 215–
238, 2016, doi: 10.18608/jla.2016.33.11.
[78] J. A. Howell, L. D. Roberts, K. Seaman, and D. C. Gibson, “Are We on Our Way to Becoming a
‘Helicopter University’? Academics’ Views on Learning Analytics,” Technology Knowl Learn, vol.
23, no. 1, pp. 1–20, 2018, doi: 10.1007/s10758-017-9329-9.
[79] V. Venkatesh, M. G. Morris, G. B. Davis, and F. D. Davis, “User acceptance of information
technology: Toward a unified view,” Mis Quart, vol. 27, no. 3, pp. 425–478, 2003, doi:
10.2307/30036540.
[80] E. M. Rogers, Diffusion of innovations, 3:Ed ed. New York: Free Press; London: Collier
Macmillan, 1983.
[81] C. H. Chung, C. Shen, and Y. Z. Qiu, “Students’ acceptance of gamification in higher education,”
Int J Game-based Learn Ijgbl, vol. 9, no. 2, pp. 1–19, 2019, doi: 10.4018/ijgbl.2019040101.
[82] K. Ng, I. K.-W. Lai, K.-K. Ng, and Q.-X. Lyu, “Online Gamified Learning Plat-forms (OGLPs)
for Experiential Learning,” in Technology in Education: Pedagogi-cal Innovations, 4th International
Conference, ICTE 2019, vol. 4, Springer, 2019, pp. 69–78. doi: 10.1007/978-981-13-9895-7_7.
[83] A. Panagiotarou, Y. C. Stamatiou, C. Pierrakeas, and A. Kameas, “Gamification Acceptance for
Learners with Different E-Skills,” Int J Learn Teach Educ Res, vol. 19, no. 2, pp. 263–278, 2020, doi:
10.26803/ijlter.19.2.16.
[84] J. Martí-Parreño, D. Segúi-Mas, and E. Segu\’i-Mas, “Teachers’ Attitude to-wards and Actual Use
of Gamification,” Procedia - Soc Behav Sci, vol. 228, no. June, pp. 682–688, 2016, doi:
10.1016/j.sbspro.2016.07.104.
[85] D. Alabbasi, “Exploring Teachers Perspectives towards Using Gamification Techniques in Online
Learning,” Turkish Online Journal of Educational Technology, 2018.
[86] P. Rodrigues, M. Souza, and E. Figueiredo, “Games and gamification in soft-ware engineering
education: A survey with educators,” in Proceedings - Frontiers in Education Conference, FIE, 2018,
vol. 2018-Octob, pp. 1–9. doi: 10.1109/fie.2018.8658524.
[87] I. Jedel and A. Palmquist, “Perception and Adoption of a Gamified Blended-Learning
Implementation in Upper Secondary Education,” 2021. [Online]. Availa-ble: http://ceur-ws.org/Vol-
2883/
[88] A. Sánchez-Mena and J. Martí-Parreño, “Drivers and Barriers to Adopting Gamification: Teachers’
Perspectives,” The Electronic Journal of e-Learning, vol. 15, p. 434, 2017.
[89] C. Cruaud, “Designing with Teachers: Contrasting Teachers’ Experiences of the Implementation
of a Gamified Application for Foreign Language Learners,” in Games and Education: Designs in and
for Learning, Brill Sense, 2018, pp. 161–178.
Industry Reports
1. Dataspelsbranschen (2018) Spelutvecklarindex. Stockholm: ANGI.
2. HolonIQ (2020) HolonIQ Nordic-Baltic EdTech 50: HolonIQ’s annual list of the most innovative
EdTech startups across the Nordic-Baltic region.